From sallyfloyd at mac.com Tue Sep 4 15:38:28 2007 From: sallyfloyd at mac.com (Sally Floyd) Date: Tue, 4 Sep 2007 15:38:28 -0700 Subject: [Tmrg] [IRSG] draft-irtf-tmrg-metrics-10 IRSG poll In-Reply-To: <20070904073634.GA10956@elstar.local> References: <20070904073634.GA10956@elstar.local> Message-ID: Juergen - > My vote is "Ready to publish" with the following comments: > > a) I think the acronym for the Transport Modeling Research Group is > TMRG and not TRMG (shows up multiple times in the ID). Oops. Thanks, fixed. > b) Section 2.3.1 talks about fairness metrics and introduces Jain's > fairness without saying what x_i actually stands for. Further down > in the discussion of the product measure, we read: > > [...] For our purposes, let x_i be the > throughput for the i-th connection. (In other contexts x_i is > taken > as the power of the i-th connection, and the product measure is > referred to as network power.) > > This text leaves it open whether this definition of x_i only > applies to the discussion of the product measure or also to other > places in the document (like Jain's fairness). I think this should > be clarified. Done. (I moved the definition of x_i so that it becomes before Jain's fairness metric.) > In the discussion of epsilon-fairness, we then read: > > where x_i is the resource allocation to the i-th flow or user. > > I am wondering how resource allocation is actually defined / > measured. Is it the fraction of the bandwidth allocated to the > flow? In the paper where epsilon-fairness is defined, they refer to x_i as the sending rate. I have clarified. > c) On page 11: s/that TCP/than TCP/ Thanks, fixed.c Many thanks for the feedback. The revised version of the draft is at: http://www.icir.org/floyd/papers/draft-irtf-tmrg-metrics-11a.txt - Sally http://www.icir.org/floyd/ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/tmrg-interest/attachments/20070904/891b2378/attachment.html From iyengar at mail.eecis.udel.edu Wed Sep 5 08:46:40 2007 From: iyengar at mail.eecis.udel.edu (Janardhan Iyengar) Date: Wed, 05 Sep 2007 11:46:40 -0400 Subject: [Tmrg] TCP evaluation suite round-table In-Reply-To: References: Message-ID: <46DECF60.7020500@mail.eecis.udel.edu> Hi Lachlan/all, > Can we agree on using > > 10Mbit/s > 100Mbit/s > 622Mbit/s > 1000Mbit/s > 2488Mbit/s > 10Gbit/s > > in all simulations/experiments, unless there is a reason to deviate? I second Doug's point about the endpoint becoming the bottleneck in experiments. We recently did some work trying to saturate 2 GigE links using 3.2 GHz Pentium-4 processors (hyperthreading OFF) with jumbograms, and we recognized two bottlenecks that were very close: 1/ approaching CPU capacity at ends 2/ motherboard backplane capacity (there was also something about the PCI/PCI-express bus limits that I cannot quite remember...) The backplane limit is not an unsurmountable problem, but I wanted to point out that testing with bandwidths beyond 1000Mbit/s may not be feasible in some cases. regards, - jana -- Janardhan R. Iyengar Visiting Assistant Professor Connecticut College http://cs.conncoll.edu/iyengar/ From iyengar at mail.eecis.udel.edu Wed Sep 5 08:55:49 2007 From: iyengar at mail.eecis.udel.edu (Janardhan Iyengar) Date: Wed, 05 Sep 2007 11:55:49 -0400 Subject: [Tmrg] TCP evaluation suite round-table In-Reply-To: References: Message-ID: <46DED185.4040408@mail.eecis.udel.edu> Hi all, Suggested addition to this list: - set of Path MTUs of interest - 1500 bytes, 9000 bytes, (maybe 576 bytes? others?) thanks, - jana Lachlan Andrew wrote: > Greetings all, > > On 8-9 November, a few of us will be getting together at Caltech for a > "round table" to try to agree on some basic parameters and metrics for > TCP evaluation. > > We won't try to answer things like "what fairness metric is best", but > we can agree on some basic parameters. The situation we're trying to > avoid is: > > Group A finds that at 500Mbps, flow 1 reaches 10% of its final > throughput after 30s > Group B finds that at 622Mbps, flow 1 reaches 20% of its final > throughput after 20s > > I'll try to have live video-conferencing via VRVS > so thta those who can't come in > person can still participate. Unfortunately, our videoconferencing > room is small, and so physical attendence will probably be limited to > a dozen or so people. Please let me know if you're interested. > > As basic goals, I'd like to come away from the roundtable with: > - a set of bandwidths that are of interest, say 10, 155, 622, 2500 Mbps > - a set of buffer sizes that are of interest, like BDP or 16384 packets > - a set of distributions of RTT that are of interest > - an agreed notion of "convergence time" > -- e.g., "the average over period x is within y% of the > final average > - an agreed notion of "time to converge to fairness" > -- e.g., "the ratio of averages over period x is within y% > of the final ratio" > -- should this metric depend on the final ratio achieved? > - an agreed notion of "intraflow variability" > -- e.g., what timescales are of interest? > - an agreed set of traffic models for background traffic > > Injong has added to that list: > - a measure of total link utilization > - fluctuation in utilization due to fluctuation in background traffic. > - What is the per-flow fair bandwidth share? > > I think some of those will be "easy", and we can sort them out on the > list before an interactive meeting, to save time for more debatable > ones. It would be good if everyone can throw in some ideas for the > next couple of months so that we can see issues are the hard ones. > > It would be good to have a common set of scenarios which could be > tested by simulation, emulation and real networks. Obviously, the > simulation is the most flexible, and so it may have a larger set of > tests, but we can at least simulate the emulated cases. > > Ulterior motive: I'd like people also to simulate/emulate the > scenarios that can also be tested on WAN-in-Lab :) > > If this works, we can get more ambitious in a second roundtable elsewhere. > > Cheers, > Lachlan > > -- > Lachlan Andrew Dept of Computer Science, Caltech > 1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA > Phone: +1 (626) 395-8820 Fax: +1 (626) 568-3603 > _______________________________________________ > Tmrg-interest mailing list > Tmrg-interest at ICSI.Berkeley.EDU > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/tmrg-interest -- Janardhan R. Iyengar Visiting Assistant Professor Connecticut College http://cs.conncoll.edu/iyengar/ From fred at cisco.com Wed Sep 5 10:42:36 2007 From: fred at cisco.com (Fred Baker) Date: Wed, 5 Sep 2007 10:42:36 -0700 Subject: [Tmrg] TCP evaluation suite round-table In-Reply-To: <46DECF60.7020500@mail.eecis.udel.edu> References: <46DECF60.7020500@mail.eecis.udel.edu> Message-ID: I think you're really missing the boat if you don't test at sub- megabit speeds as well. I could tell stories, like the one in Malawi (deepest darkest Africa) in which a service provider that sold radio links that they reduced to 64 KBPS by throwing away any traffic that arrived faster than that. At least look at 2 MBPS and 256 KBPS. Your TCP variant should run at the low speeds as well as the high ones. On Sep 5, 2007, at 8:46 AM, Janardhan Iyengar wrote: > Hi Lachlan/all, > >> Can we agree on using >> >> 10Mbit/s >> 100Mbit/s >> 622Mbit/s >> 1000Mbit/s >> 2488Mbit/s >> 10Gbit/s >> >> in all simulations/experiments, unless there is a reason to deviate? > > I second Doug's point about the endpoint becoming the bottleneck in > experiments. We recently did some work trying to saturate 2 GigE > links using 3.2 GHz Pentium-4 processors (hyperthreading OFF) with > jumbograms, and we recognized two bottlenecks that were very close: > 1/ approaching CPU capacity at ends > 2/ motherboard backplane capacity > > (there was also something about the PCI/PCI-express bus limits that > I cannot quite remember...) > > The backplane limit is not an unsurmountable problem, but I wanted > to point out that testing with bandwidths beyond 1000Mbit/s may not > be feasible in some cases. > > regards, > - jana > > -- > Janardhan R. Iyengar > Visiting Assistant Professor > Connecticut College > http://cs.conncoll.edu/iyengar/ > _______________________________________________ > Tmrg-interest mailing list > Tmrg-interest at ICSI.Berkeley.EDU > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/tmrg-interest From lachlan.andrew at gmail.com Wed Sep 5 10:56:57 2007 From: lachlan.andrew at gmail.com (Lachlan Andrew) Date: Wed, 5 Sep 2007 10:56:57 -0700 Subject: [Tmrg] TCP evaluation suite round-table In-Reply-To: References: <46DECF60.7020500@mail.eecis.udel.edu> Message-ID: Greetings Fred, Yes, Sally had already suggested testing down to 56kbit/s (Thanks Sally!). Most proposals follow HS-TCP and revert to Reno in these cases, but it would certainly be good to check that the implementations work properly. Cheers, Lachlan On 05/09/07, Fred Baker wrote: > I think you're really missing the boat if you don't test at sub- > megabit speeds as well. I could tell stories, like the one in Malawi > (deepest darkest Africa) in which a service provider that sold radio > links that they reduced to 64 KBPS by throwing away any traffic that > arrived faster than that. At least look at 2 MBPS and 256 KBPS. Your > TCP variant should run at the low speeds as well as the high ones. -- Lachlan Andrew Dept of Computer Science, Caltech 1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA Phone: +1 (626) 395-8820 Fax: +1 (626) 568-3603 From lachlan.andrew at gmail.com Wed Sep 5 11:03:32 2007 From: lachlan.andrew at gmail.com (Lachlan Andrew) Date: Wed, 5 Sep 2007 11:03:32 -0700 Subject: [Tmrg] TCP evaluation suite round-table In-Reply-To: References: Message-ID: Greetings Sally, On 29/08/07, Sally Floyd wrote: > Lachlan wrote > > > I would advocate using the > > sum of the *receive* rates of the flows using each link. > > I think that is a fine idea. > > But it would be easy for each simulation or experiment to also > report the total bps over the link, in each direction. E.g., to > compare with the aggregate receive rates. Or to allow "local" > metrics about throughput vs. delay vs. drop rates for understanding > the queue management. Or some such. True, that would be useful. I certainly wouldn't want people to think they should only do a subset of what we agree on. On the other hand, I would want them to feel obliged to do a superset either... Perhaps the goal should be to find a list of "types" of quantities to measure (like link utilization), and say "*if* you're going to measure quantity X, please include metric Y along with any others that you choose" and "*if* you're interested in bandwidths in this range, please include bandwidths from the following list". Do you have any recommendations? I'll check the wording in your current draft... Cheers, Lachlan -- Lachlan Andrew Dept of Computer Science, Caltech 1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA Phone: +1 (626) 395-8820 Fax: +1 (626) 568-3603 From lachlan.andrew at gmail.com Wed Sep 5 11:32:33 2007 From: lachlan.andrew at gmail.com (Lachlan Andrew) Date: Wed, 5 Sep 2007 11:32:33 -0700 Subject: [Tmrg] TCP evaluation suite round-table In-Reply-To: <46DECF60.7020500@mail.eecis.udel.edu> References: <46DECF60.7020500@mail.eecis.udel.edu> Message-ID: Greetings Jana, On 05/09/07, Janardhan Iyengar wrote: > > I second Doug's point about the endpoint becoming the bottleneck in experiments. We recently did some work trying to saturate 2 GigE links using 3.2 GHz Pentium-4 processors (hyperthreading OFF) with jumbograms, and we recognized two bottlenecks that were very close: > 1/ approaching CPU capacity at ends > 2/ motherboard backplane capacity Yes, there certainly problems getting above 1 Gbps. (More below for those who are intersted in Linux.) > (there was also something about the PCI/PCI-express bus limits that I cannot quite remember...) PCI-express should have no problem. PCI-X can handle up to 7Gbit/s if it is working properly, although I have some buggy cards which limit it to 5Gbit/s. I'm not sure about regular PCI. > testing with bandwidths beyond 1000Mbit/s may not be feasible in some cases. True. However we can specify a wider range of tests than can currently be performed, so that (a) simulations can be consistent (b) when people *do* get fast enough systems, they can be consistent with those simulations. The main question I was have is not about hardware testing at >1Gbps, but whether we can make the more modest move of replacing the non-standard 400Mbit/s (used for Dummynet) by 622Mbit/s so that it can be compared with OC12 hardware. For people interested in Linux: Prompted by Doug's and Lars's comments, I've also just been doing some experiments, and had a case where it took about 4 minutes to do a __release_sock because of SACK processing (mainly tcp_sacktag_write_queue). I didn't check the CPU at the time, unfortunately, but I assume it was high.... Doug suggested looking at /proc/net/softnet_stat to look for packet loss at the driver level, but I didn't observe any -- perhaps ACK clocking was working as it should! I'm currently looking to see if some of this can be helped by dropping SACKs when the backlog is too high. Would that just drop fast-path SACKs? Cheers, Lachlan -- Lachlan Andrew Dept of Computer Science, Caltech 1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA Phone: +1 (626) 395-8820 Fax: +1 (626) 568-3603 From weddy at grc.nasa.gov Wed Sep 5 11:46:07 2007 From: weddy at grc.nasa.gov (Wesley Eddy) Date: Wed, 5 Sep 2007 14:46:07 -0400 Subject: [Tmrg] TCP evaluation suite round-table In-Reply-To: References: Message-ID: <20070905184607.GD28137@grc.nasa.gov> On Wed, Sep 05, 2007 at 10:42:36AM -0700, Fred Baker wrote: > I think you're really missing the boat if you don't test at sub- > megabit speeds as well. I could tell stories, like the one in Malawi > (deepest darkest Africa) in which a service provider that sold radio > links that they reduced to 64 KBPS by throwing away any traffic that > arrived faster than that. At least look at 2 MBPS and 256 KBPS. Your > TCP variant should run at the low speeds as well as the high ones. > Many (most? all?) of the proposals drop back into 2581 behavior if the cwnd is "small", so I (sort of) disagree and think it's alright to focus on the faster rates. I hedge that with "sort of", because I'm assuming that all designs fall back into the well-understood legacy behavior in low bandwidth-delay product scenarios or at least could easily be made to do so, and I don't know if that's a faulty assumption. -- Wesley M. Eddy Verizon Federal Network Systems From lachlan.andrew at gmail.com Wed Sep 5 11:58:40 2007 From: lachlan.andrew at gmail.com (Lachlan Andrew) Date: Wed, 5 Sep 2007 11:58:40 -0700 Subject: [Tmrg] TCP evaluation suite round-table In-Reply-To: <20070905184607.GD28137@grc.nasa.gov> References: <20070905184607.GD28137@grc.nasa.gov> Message-ID: Greetings Wes, On 05/09/07, Wesley Eddy wrote: > > Many (most? all?) of the proposals drop back into 2581 behavior if the > cwnd is "small", so I (sort of) disagree and think it's alright to focus > on the faster rates. I hedge that with "sort of", because I'm assuming > that all designs fall back into the well-understood legacy behavior in > low bandwidth-delay product scenarios or at least could easily be > made to do so, and I don't know if that's a faulty assumption. Good point; "high speed" protocols do that. However, if we want this test suite to apply to testing things like LT-TCP or LP-TCP, then they won't necessarily revert to standard behaviour. I say there's no harm in extending the range of rates in the "preferred list". Nothing forces people to use *all* rates in the list. Cheers, Lachlan -- Lachlan Andrew Dept of Computer Science, Caltech 1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA Phone: +1 (626) 395-8820 Fax: +1 (626) 568-3603 From iyengar at mail.eecis.udel.edu Wed Sep 5 12:08:37 2007 From: iyengar at mail.eecis.udel.edu (Janardhan Iyengar) Date: Wed, 05 Sep 2007 15:08:37 -0400 Subject: [Tmrg] TCP evaluation suite round-table In-Reply-To: <20070905184607.GD28137@grc.nasa.gov> References: <20070905184607.GD28137@grc.nasa.gov> Message-ID: <46DEFEB5.8070808@mail.eecis.udel.edu> Hey Wes, > Many (most? all?) of the proposals drop back into 2581 behavior if the > cwnd is "small", so I (sort of) disagree and think it's alright to focus The point, I think, is to ensure that *all* proposals, *not most*, work at low bandwidth rates too. For most proposals, it might just be a question of making sure that when optimizing for other conditions, nothing broke with the oft-neglected low bandwidth case. But it is a test that needs to be run. regards, - jana -- Janardhan R. Iyengar Visiting Assistant Professor Connecticut College http://cs.conncoll.edu/iyengar/ From fred at cisco.com Wed Sep 5 12:13:23 2007 From: fred at cisco.com (Fred Baker) Date: Wed, 5 Sep 2007 12:13:23 -0700 Subject: [Tmrg] TCP evaluation suite round-table In-Reply-To: <20070905184607.GD28137@grc.nasa.gov> References: <20070905184607.GD28137@grc.nasa.gov> Message-ID: it's supposed to do that, so we won't test for that. I'll pass that thought along to Cisco dev-test. They'll appreciate it. It will reduce their work dramatically. On Sep 5, 2007, at 11:46 AM, Wesley Eddy wrote: > On Wed, Sep 05, 2007 at 10:42:36AM -0700, Fred Baker wrote: >> I think you're really missing the boat if you don't test at sub- >> megabit speeds as well. I could tell stories, like the one in Malawi >> (deepest darkest Africa) in which a service provider that sold radio >> links that they reduced to 64 KBPS by throwing away any traffic that >> arrived faster than that. At least look at 2 MBPS and 256 KBPS. Your >> TCP variant should run at the low speeds as well as the high ones. >> > > > Many (most? all?) of the proposals drop back into 2581 behavior if the > cwnd is "small", so I (sort of) disagree and think it's alright to > focus > on the faster rates. I hedge that with "sort of", because I'm > assuming > that all designs fall back into the well-understood legacy behavior in > low bandwidth-delay product scenarios or at least could easily be > made to do so, and I don't know if that's a faulty assumption. > > -- > Wesley M. Eddy > Verizon Federal Network Systems From weddy at grc.nasa.gov Wed Sep 5 12:40:00 2007 From: weddy at grc.nasa.gov (Wesley Eddy) Date: Wed, 5 Sep 2007 15:40:00 -0400 Subject: [Tmrg] TCP evaluation suite round-table In-Reply-To: References: <20070905184607.GD28137@grc.nasa.gov> Message-ID: <20070905194000.GE28137@grc.nasa.gov> On Wed, Sep 05, 2007 at 12:13:23PM -0700, Fred Baker wrote: > it's supposed to do that, so we won't test for that. > > I'll pass that thought along to Cisco dev-test. They'll appreciate > it. It will reduce their work dramatically. > Understood :), though I'd thought his effort was for evaluating the merits of various proposals on common agreed-upon configurations, not for debugging the implementations which should hopefully be done well before the trials used for comparisons. Debugging requires checking a lot of scheme-specific cases, loss patterns, and edge-cases ... if the desire is to make a debugging checklist, this doesn't even scratch the surface. Lachlan's point that LT-TCP and others might behave differently at low rates is a better reason, and I buy his argument. -- Wesley M. Eddy Verizon Federal Network Systems From sallyfloyd at mac.com Thu Sep 6 11:49:09 2007 From: sallyfloyd at mac.com (Sally Floyd) Date: Thu, 6 Sep 2007 11:49:09 -0700 Subject: [Tmrg] TCP evaluation suite round-table In-Reply-To: <20070905184607.GD28137@grc.nasa.gov> References: <20070905184607.GD28137@grc.nasa.gov> Message-ID: <2277ed41b75d22c844c751fdb716263f@mac.com> > Many (most? all?) of the proposals drop back into 2581 behavior if the > cwnd is "small", so I (sort of) disagree and think it's alright to > focus > on the faster rates. I assume that the test scenarios will also be used for proposed changes such as Quick-Start and other proposals for faster start-up, and those should definitely be tested on a very wide range of path bandwidths. (This is largely orthogonal to whether the congestion control behavior drops back to 2581.) - Sally http://www.icir.org/floyd/ From garmitage at swin.edu.au Thu Sep 6 23:42:06 2007 From: garmitage at swin.edu.au (grenville armitage) Date: Fri, 07 Sep 2007 16:42:06 +1000 Subject: [Tmrg] Logging active TCP details in FreeBSD 5, 6 and 7 Message-ID: <46E0F2BE.8090405@swin.edu.au> All, On the off chance this is of general interest, I'd like to let people know of a FreeBSD kernel module we've developed for logging various TCP state variables in a running kernel while sessions are active. Called SIFTR ("sifter"), we built this for our own research into precisely how a FreeBSD TCP stack behaves when faced with real and artificial (e.g. dummynet) paths. Figured it might also be of interest to others. See http://caia.swin.edu.au/urp/newtcp/tools.html (under SIFTR) for a readme, changelog and tarball. The authors would love to get feedback from anyone trying it out. (SIFTR has been developed and tested mostly under FreeBSD 6.2, but we believe it'll be stable under 5.x and 7.x too.) cheers, gja From lachlan.andrew at gmail.com Fri Sep 7 08:22:23 2007 From: lachlan.andrew at gmail.com (Lachlan Andrew) Date: Fri, 7 Sep 2007 08:22:23 -0700 Subject: [Tmrg] Logging active TCP details in FreeBSD 5, 6 and 7 In-Reply-To: <46E0F2BE.8090405@swin.edu.au> References: <46E0F2BE.8090405@swin.edu.au> Message-ID: Greetings Grenville, Would it be possible to give it an interface like Web100? We're currently building our benchmarking suite to use (a slightly optimized) Web100 to monitor system internals, and it would be great to be able to use SIFTR without too much modification of our suite. Cheers, Lachlan On 06/09/2007, grenville armitage wrote: > All, > > On the off chance this is of general interest, I'd like to let > people know of a FreeBSD kernel module we've developed for > logging various TCP state variables in a running kernel > while sessions are active. > > Called SIFTR ("sifter"), we built this for our own research into > precisely how a FreeBSD TCP stack behaves when faced with real > and artificial (e.g. dummynet) paths. Figured it might also be > of interest to others. > > See http://caia.swin.edu.au/urp/newtcp/tools.html (under SIFTR) > for a readme, changelog and tarball. The authors would love to get > feedback from anyone trying it out. (SIFTR has been developed and > tested mostly under FreeBSD 6.2, but we believe it'll be stable > under 5.x and 7.x too.) > > cheers, > gja -- Lachlan Andrew Dept of Computer Science, Caltech 1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA Phone: +1 (626) 395-8820 Fax: +1 (626) 568-3603 From ccardena at itesm.mx Thu Sep 13 09:41:22 2007 From: ccardena at itesm.mx (=?ISO-8859-15?Q?C=E9sar=20C=E1rdenas?=) Date: Thu, 13 Sep 2007 18:41:22 +0200 Subject: [Tmrg] delay and gooput fairness Message-ID: <46D43F2D000175C4@mailserver1.itesm.mx> Dear all, It there any measure for "delay fairness" or "goodput fairness" of a TCP flow? If yes, I would appreciate if you point me to a good reference, I apologize if these both lists are not adequate for this question. Please accept my best regards, C?sar C?sar C?rdenas-P?rez (ccardena at itesm.mx) Monterrey Tech, Quer?taro Campus, M?xico http://www.qro.itesm.mx Personal Phone: +(33) 633306689 Office Phone: +(33) 145817146 Office Fax: +(33) 145813119 Skype pseudo: cesarcardenas7 All phones and fax from abroad France The content of this data transmission is not considered as an offer, proposal, understanding, or agreement unless it is confirmed in a document signed by a legal representative of ITESM. The content of this data transmission is confidential and it is intended to be delivered only to the addresses, therefore, it shall not be distributed and/or disclosed through any mean without the original sender's previous authorization. If you are not the addressee you are forbidden to use it, either totally or partially, for any purpose. From ccardena at itesm.mx Thu Sep 13 11:02:19 2007 From: ccardena at itesm.mx (=?ISO-8859-15?Q?C=E9sar=20C=E1rdenas?=) Date: Thu, 13 Sep 2007 20:02:19 +0200 Subject: [Tmrg] delay and gooput fairness In-Reply-To: <46D43F2D000175C4@mailserver1.itesm.mx> Message-ID: <46D43F2D0001791C@mailserver1.itesm.mx> Dear all, I realize I made a wrong question: Is there any measure of "delay fairness" or "goodput fairness" for a group of TCP flows served by a FQ algorithm? If yes, do you know a procedure to estimate them when you have 30 replications? Or do you have some suggestions? Many thanks in advance, C?sar >-- Mensaje Original -- >Date: Thu, 13 Sep 2007 18:41:22 +0200 >From: C?sar C?rdenas >Subject: delay and gooput fairness >Reply-To: ccardena at itesm.mx >To: tmrg-interest at icsi.berkeley.edu >Cc: iccrg at cs.ucl.ac.uk > > >Dear all, > >It there any measure for "delay fairness" or "goodput fairness" of a TCP >flow? > >If yes, I would appreciate if you point me to a good reference, > >I apologize if these both lists are not adequate for this question. > >Please accept my best regards, >C?sar > > >C?sar C?rdenas-P?rez (ccardena at itesm.mx) >Monterrey Tech, Quer?taro Campus, M?xico >http://www.qro.itesm.mx >Personal Phone: +(33) 633306689 >Office Phone: +(33) 145817146 >Office Fax: +(33) 145813119 >Skype pseudo: cesarcardenas7 >All phones and fax from abroad France > >The content of this data transmission is not considered as an offer, proposal, >understanding, or agreement unless it is confirmed in a document signed by >a legal representative of ITESM. The content of this data transmission is >confidential and it is intended to be delivered only to the addresses, therefore, >it shall not be distributed and/or disclosed through any mean without the >original sender's previous authorization. If you are not the addressee you >are forbidden to use it, either totally or partially, for any purpose. > > C?sar C?rdenas-P?rez (ccardena at itesm.mx) Monterrey Tech, Quer?taro Campus, M?xico http://www.qro.itesm.mx Personal Phone: +(33) 633306689 Office Phone: +(33) 145817146 Office Fax: +(33) 145813119 Skype pseudo: cesarcardenas7 All phones and fax from abroad France The content of this data transmission is not considered as an offer, proposal, understanding, or agreement unless it is confirmed in a document signed by a legal representative of ITESM. The content of this data transmission is confidential and it is intended to be delivered only to the addresses, therefore, it shall not be distributed and/or disclosed through any mean without the original sender's previous authorization. If you are not the addressee you are forbidden to use it, either totally or partially, for any purpose. From lachlan.andrew at gmail.com Thu Sep 13 14:52:42 2007 From: lachlan.andrew at gmail.com (Lachlan Andrew) Date: Thu, 13 Sep 2007 14:52:42 -0700 Subject: [Tmrg] [Iccrg] RE: delay and gooput fairness In-Reply-To: <46D43F2D0001791C@mailserver1.itesm.mx> References: <46D43F2D000175C4@mailserver1.itesm.mx> <46D43F2D0001791C@mailserver1.itesm.mx> Message-ID: Greetigns C?sar, For delay, you could use Jain's index. (I think he proposed it in the 1989 paper "Analysis of the increase and decrease algorithms for congestion avoidance in computer networks". Could someone correct me?) Jain's index is fairly poor for goodput, but you can use Jain's index on the reciprocal of the goodput. An alternative is to use Kelly's utility maximization framework, and measure the drop in utility due to the inequality of rates. Cheers, Lachlan On 13/09/2007, C?sar C?rdenas wrote: > Dear all, > I realize I made a wrong question: > > Is there any measure of "delay fairness" or "goodput fairness" for a group > of TCP flows served by a FQ algorithm? > If yes, do you know a procedure to estimate them when you have 30 replications? > Or do you have some suggestions? > > Many thanks in advance, > C?sar > > >-- Mensaje Original -- > >Date: Thu, 13 Sep 2007 18:41:22 +0200 > >From: C?sar C?rdenas > >Subject: delay and gooput fairness > >Reply-To: ccardena at itesm.mx > >To: tmrg-interest at icsi.berkeley.edu > >Cc: iccrg at cs.ucl.ac.uk > > > > > >Dear all, > > > >It there any measure for "delay fairness" or "goodput fairness" of a TCP > >flow? > > > >If yes, I would appreciate if you point me to a good reference, > > > >I apologize if these both lists are not adequate for this question. > > > >Please accept my best regards, > >C?sar > > > > > >C?sar C?rdenas-P?rez (ccardena at itesm.mx) > >Monterrey Tech, Quer?taro Campus, M?xico > >http://www.qro.itesm.mx > >Personal Phone: +(33) 633306689 > >Office Phone: +(33) 145817146 > >Office Fax: +(33) 145813119 > >Skype pseudo: cesarcardenas7 > >All phones and fax from abroad France > > > >The content of this data transmission is not considered as an offer, proposal, > >understanding, or agreement unless it is confirmed in a document signed > by > >a legal representative of ITESM. The content of this data transmission is > >confidential and it is intended to be delivered only to the addresses, therefore, > >it shall not be distributed and/or disclosed through any mean without the > >original sender's previous authorization. If you are not the addressee you > >are forbidden to use it, either totally or partially, for any purpose. > > > > > > C?sar C?rdenas-P?rez (ccardena at itesm.mx) > Monterrey Tech, Quer?taro Campus, M?xico > http://www.qro.itesm.mx > Personal Phone: +(33) 633306689 > Office Phone: +(33) 145817146 > Office Fax: +(33) 145813119 > Skype pseudo: cesarcardenas7 > All phones and fax from abroad France > > The content of this data transmission is not considered as an offer, proposal, > understanding, or agreement unless it is confirmed in a document signed by > a legal representative of ITESM. The content of this data transmission is > confidential and it is intended to be delivered only to the addresses, therefore, > it shall not be distributed and/or disclosed through any mean without the > original sender's previous authorization. If you are not the addressee you > are forbidden to use it, either totally or partially, for any purpose. > > > _______________________________________________ > Iccrg mailing list > Iccrg at cs.ucl.ac.uk > http://oakham.cs.ucl.ac.uk/mailman/listinfo/iccrg > -- Lachlan Andrew Dept of Computer Science, Caltech 1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA Phone: +1 (626) 395-8820 Fax: +1 (626) 568-3603 From sallyfloyd at mac.com Fri Sep 14 07:11:03 2007 From: sallyfloyd at mac.com (Sally Floyd) Date: Fri, 14 Sep 2007 07:11:03 -0700 Subject: [Tmrg] [Iccrg] RE: delay and gooput fairness In-Reply-To: <46D43F2D0001791C@mailserver1.itesm.mx> References: <46D43F2D0001791C@mailserver1.itesm.mx> Message-ID: <753b4cccb2f4426a775b09527254b30e@mac.com> Cesar - > Is there any measure of "delay fairness" or "goodput fairness" for a > group > of TCP flows served by a FQ algorithm? You could look at the TMRG document "Metrics for the Evaluation of Congestion Control Mechanisms", available from the TMRG web page at "http://www.icir.org/tmrg/", for pointers to some of the fairness metrics that have been used in the past. - Sally http://www.icir.org/floyd/ From weddy at grc.nasa.gov Mon Sep 17 07:07:34 2007 From: weddy at grc.nasa.gov (Wesley Eddy) Date: Mon, 17 Sep 2007 10:07:34 -0400 Subject: [Tmrg] [Iccrg] RE: delay and gooput fairness In-Reply-To: References: <46D43F2D0001791C@mailserver1.itesm.mx> Message-ID: <20070917140734.GC6573@grc.nasa.gov> On Thu, Sep 13, 2007 at 02:52:42PM -0700, Lachlan Andrew wrote: > Greetigns C?sar, > > For delay, you could use Jain's index. (I think he proposed it in the > 1989 paper "Analysis of the increase and decrease algorithms for > congestion avoidance in computer networks". Could someone correct > me?) > It's very nicely explained in: R. Jain, D. Chiu, and W. Hawe, "A Quantitative Measure Of Fairness And Discrimination For Resource Allocation In Shared Computer Systems", DEC Research Report TR-301, September 1984. online at: http://www.cse.wustl.edu/~jain/papers/fairness.htm -- Wesley M. Eddy Verizon Federal Network Systems From sallyfloyd at mac.com Mon Sep 17 16:00:35 2007 From: sallyfloyd at mac.com (Sally Floyd) Date: Mon, 17 Sep 2007 16:00:35 -0700 Subject: [Tmrg] TCP evaluation suite round-table In-Reply-To: References: Message-ID: Lachlan - On Sep 5, 2007, at 11:03 AM, Lachlan Andrew wrote: > Greetings Sally, > On 29/08/07, Sally Floyd wrote: >> Lachlan wrote >>> I would advocate using the >>> sum of the *receive* rates of the flows using each link. >> >> I think that is a fine idea. >> >> But it would be easy for each simulation or experiment to also >> report the total bps over the link, in each direction. E.g., to >> compare with the aggregate receive rates. Or to allow "local" >> metrics about throughput vs. delay vs. drop rates for understanding >> the queue management. Or some such. > > True, that would be useful. > > I certainly wouldn't want people to think they should only do a > subset of what we agree on. On the other hand, I would want them to > feel obliged to do a superset either... > > Perhaps the goal should be to find a list of "types" of quantities to > measure (like link utilization), and say "*if* you're going to measure > quantity X, please include metric Y along with any others that you > choose" and "*if* you're interested in bandwidths in this range, > please include bandwidths from the following list". Do you have any > recommendations? I'll check the wording in your current draft... I would assume that each scenario in the evaluation suite would have a few key outputs, depending on the metrics being investigated in that scenario. E.g., the main scenarios might look at fairness, and at the tradeoffs between aggregate throughput, delay, and drop rates, and other metrics as well. For example, for looking at the tradeoffs between aggregate throughput, delay, and drop rates, I assume the scenario would include a range of traffic intensities and settings for the queue size (or target average queue size). It *might* be useful to show both application-based metrics (sum of aggregate receive rates) and router-based metrics (for the congested link in question). But for each scenario in the evaluation suite, there might be additional useful information that is easily available, and that it would be useful to have output, simply to help the user understand what is going on if the user is so inclined. That is, "extra" information that is provided, but that is not part of the key metrics being evaluated in that scenario. That the user can look at or not, as they are so inclined... - Sally http://www.icir.org/floyd/ From lachlan.andrew at gmail.com Fri Sep 21 13:47:19 2007 From: lachlan.andrew at gmail.com (Lachlan Andrew) Date: Fri, 21 Sep 2007 13:47:19 -0700 Subject: [Tmrg] TCP evaluation suite round-table In-Reply-To: References: Message-ID: Greetings Sally, On 17/09/2007, Sally Floyd wrote: > > I would assume that each scenario in the evaluation suite would > have a few key outputs. > > But for each scenario in the evaluation suite, there might be > additional useful information that is easily available. Yes, it will be good to specify the key outputs. For a particular implementation of the tests (say an NS2 simulation), other information may be easily available. At the round-table, I was hoping to start on specifying common tests that could be done at different "levels of abstraction": analysis, simulation, emulation, and also real networks. Measuring the throughput of a router port is typically only possible using SNMP statistics which are updated infrequently. For that reason, I'd vote against it as a "key output", although of course an NS2 tool would probably make it available. I'd be happy to be outvoted... Cheers, Lachlan -- Lachlan Andrew Dept of Computer Science, Caltech 1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA Phone: +1 (626) 395-8820 Fax: +1 (626) 568-3603 From lachlan.andrew at gmail.com Mon Sep 24 14:17:47 2007 From: lachlan.andrew at gmail.com (Lachlan Andrew) Date: Mon, 24 Sep 2007 14:17:47 -0700 Subject: [Tmrg] Comments on in draft-irtf-tmrg-metrics-10.txt Message-ID: Greetings Sally, Here are some further comments on draft-irtf-tmrg-metrics-10.txt. - Section 2.1.1 Throughput introduces the difference between throughput and goodput, and then says that maximizing throughput is desirable. This makes it sound as if throughput is more important than goodput, which I don't think was the intention. Perhaps replace "...because they can't be re-assembled into complete packets, and the like." by "...because they can't be re-assembled into complete packets, and the like. Except where clearly stated, this document refers to both throughput and goodput generically as 'throughput'." or rephrase the rest of the section/document in terms of maximizing goodput. - Section 2.4 Robustness for Challenging Environments defines goodput as a *fraction* of the sent data which is received, which seems to contradict the definition in 2.1.1 which is the *total amount* of data which is received. I think that the definition from 2.1.1 is more standard, and would recommend rephrasing the paragraph as: "Goodput: For wireless networks, goodput can be a useful metric, where goodput can be defined as the total amount of useful data delivered to users. A high ratio of goodput to sent data indicates an efficient use of the radio spectrum and lower interference with other users." Another couple of comments: - Section 2.1.2 Delay It might be worth mentioning that "delay" can also include the time spent queued at the sender due to window flow control, as well as at network queues and retransmissions. This is often the dominant source of socket-layer delay. (This relates to Mark Allman's recent comments that spurious timeouts have a cost in terms of window reduction which must be weighed against the reduced waiting time of short RTO.) - Section 2.1.2 Delay When discussing "router-based" vs "flow-based" delay, it might be good to mention that "router-based" delay affects competing traffic, while "flow-based" delay does not. Thus, router-based delay induced by bulk data transfer applications is important, even if they aren't interested in per-packet transfer times. Perhaps the section could be rephrased as: Like throughput, delay can be measured as a router-based metric of queueing delay over time, or as a flow-based metric in terms of per- packet transfer times. For any rate controlled transfer, the per-packet transfer time will include time between when the application generates the packet and when the protocol allows it to be first sent. For reliable transfer, the per-packet transfer time seen by the application includes the possible delay of retransmitting a lost packet. Users of bulk data transfer applications might care about per-packet transfer times only in so far as they affect the per-connection transfer time. On the other end of the spectrum, for users of streaming media, per-packet delay can be a significant concern. Note that in some cases the average delay might not capture the metric of interest to the users; for example, some users might care about the worst-case delay, or about the tail of the delay distribution. Note that queueing delay is experienced by all flows sharing a link. Thus, bulk data transfer applications should still seek to achieve low queueing delay for the benefit of cross traffic, even if not for their own benefit. - (Nit picking) [F98] unnecessarily duplicates [KMT98]. The reference to [F98] in 2.3.2 should also be replaced by [KMT98]. Cheers, Lachlan -- Lachlan Andrew Dept of Computer Science, Caltech 1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA Phone: +1 (626) 395-8820 Fax: +1 (626) 568-3603 From lachlan.andrew at gmail.com Tue Sep 25 12:36:42 2007 From: lachlan.andrew at gmail.com (Lachlan Andrew) Date: Tue, 25 Sep 2007 12:36:42 -0700 Subject: [Tmrg] TCP evaluation suite round-table In-Reply-To: <46DED185.4040408@mail.eecis.udel.edu> References: <46DED185.4040408@mail.eecis.udel.edu> Message-ID: Greetings all, To keep this moving, I've typed up a list of what we've discussed so far, at . Details of the venue etc are also linked to from that page. I'd like to start to finalize numbers. Could those who already expressed an interest in coming on 8-9 November please confirm that they are still coming? We still have half a dozen places at the table, so anyone else interested is cordially invited to let me know. As Wes pointed out, my current aim is not to make a comprehensive test suite, but to have a very core list of tests so that tests by different groups and using different technologies (simulation, dummynet etc) can be compared. For that reason, I'd suggest having a single MTU in the list. If we have jumbo frames, I'd prefer 4470 (the SONET MTU), which can be carried on a wider range of equipment. What do others think? Similarly, Grenville suggested off-list that we consider asymmetric bandwidths. It would be great to test that case, but I'd be inclined not to include it in the "core scenarios". Opinions? Cheers, Lachlan On 05/09/2007, Janardhan Iyengar wrote: > > Suggested addition to this list: > - set of Path MTUs of interest - 1500 bytes, 9000 bytes, (maybe 576 bytes? others?) -- Lachlan Andrew Dept of Computer Science, Caltech 1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA Phone: +1 (626) 395-8820 Fax: +1 (626) 568-3603 From lachlan.andrew at gmail.com Tue Sep 25 13:07:25 2007 From: lachlan.andrew at gmail.com (Lachlan Andrew) Date: Tue, 25 Sep 2007 13:07:25 -0700 Subject: [Tmrg] Round table: Buffer sizes Message-ID: Greetings all, Another question on the list is buffer sizes. I'd obviously like to standardize on the buffer sizes that WAN-in-Lab supports "natively", namely 128 packets at 1Gbps and 16384 packet at 2.5Gbps, but they're fairly ad-hoc choices. WAN-in-Lab can alternatively be cajoled into using buffer sizes of any power of 2 from 128 to 8192 packets. An obvious buffer size to set is some multiple of the BDP, but that is not well defined if flows have different RTTs. Setting the buffer too large will mask the effects of RTT unfairness; for example, setting the buffer to be the size of the maximum BDP would mean that all RTTs are within a factor of 2, even if the actual path delays differ by a factor of 10. Also, Cisco buffer sizes seem to be specified in numbers of packets not bytes, and I believe Dummynet has an option to do that too. This makes a "BDP-sized " buffer hard to define with bidirectional traffic, since the number of packets depends on the fraction of ACKs vs full-sized packets. Given all that, can someone suggest suitable buffer sizes for this core set of tests? Cheers, Lachlan -- Lachlan Andrew Dept of Computer Science, Caltech 1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA Phone: +1 (626) 395-8820 Fax: +1 (626) 568-3603 From jheffner at psc.edu Tue Sep 25 14:53:45 2007 From: jheffner at psc.edu (John Heffner) Date: Tue, 25 Sep 2007 17:53:45 -0400 Subject: [Tmrg] Round table: Buffer sizes In-Reply-To: References: Message-ID: <46F98369.8030605@psc.edu> Lachlan Andrew wrote: > Greetings all, > > Another question on the list is buffer sizes. > > I'd obviously like to standardize on the buffer sizes that WAN-in-Lab > supports "natively", namely 128 packets at 1Gbps and 16384 packet at > 2.5Gbps, but they're fairly ad-hoc choices. WAN-in-Lab can > alternatively be cajoled into using buffer sizes of any power of 2 > from 128 to 8192 packets. > > An obvious buffer size to set is some multiple of the BDP, but that is > not well defined if flows have different RTTs. Setting the buffer too > large will mask the effects of RTT unfairness; for example, setting > the buffer to be the size of the maximum BDP would mean that all RTTs > are within a factor of 2, even if the actual path delays differ by a > factor of 10. > > Also, Cisco buffer sizes seem to be specified in numbers of packets > not bytes, and I believe Dummynet has an option to do that too. This > makes a "BDP-sized " buffer hard to define with bidirectional traffic, > since the number of packets depends on the fraction of ACKs vs > full-sized packets. > > Given all that, can someone suggest suitable buffer sizes for this > core set of tests? Given the relatively wide range of buffer sizes in hardware out there at any given speed, and the significant effect buffer size can have on congestion control, it seems like this should be an extra dimension rather than fixed per link type. Maybe select a couple different sizes based on a set of drain times? Say, {1 ms, 10 ms, 100 ms, 1 sec}? Convert this to bytes or packets by dividing by the link byte or MTU packet rate. -John From sallyfloyd at mac.com Tue Sep 25 15:12:09 2007 From: sallyfloyd at mac.com (Sally Floyd) Date: Tue, 25 Sep 2007 15:12:09 -0700 Subject: [Tmrg] Round table: Buffer sizes In-Reply-To: <46F98369.8030605@psc.edu> References: <46F98369.8030605@psc.edu> Message-ID: <52377da556dd59343ec870df1710d63a@mac.com> > Given the relatively wide range of buffer sizes in hardware out there > at > any given speed, and the significant effect buffer size can have on > congestion control, it seems like this should be an extra dimension > rather than fixed per link type. Yep, that makes sense to me. E.g., one test scenario could keep all other parameters fixed, vary the buffer size (or the average queue size for the AQM mechanism) at the congested link, and explore performance as a function of buffer size. This does not imply a *goal* that all congestion control mechanisms perform equally well for all buffer sizes; just that if a particular congestion control mechanism performs extremely poorly with small buffers, or gets much more or much less of its "share" of the bandwidth with very large buffers, it would be good for a set of "current best practice" simulation scenarios to detect this. As a point of information. - Sally http://www.icir.org/floyd/ From lars.eggert at nokia.com Wed Sep 26 00:31:54 2007 From: lars.eggert at nokia.com (Lars Eggert) Date: Wed, 26 Sep 2007 10:31:54 +0300 Subject: [Tmrg] TCP evaluation suite round-table In-Reply-To: References: <46DED185.4040408@mail.eecis.udel.edu> Message-ID: <20A4D241-511A-4F5C-BA72-2E1B96F25689@nokia.com> Hi, it might make sense to add a one or two test cases that include a GSM or UMTS access link. Both have the interesting characteristic that the link delay is pretty high (~200ms for UMTS, > 1 sec for GSM) and worse, the delay can jump around quite a bit (i.e,, double or more) on very short timescales. This might prove challenging for TCPs that use path delay for congestion estimation. That said, I can't point you at a good model for such links. But maybe someone else can? Lars -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2446 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/tmrg-interest/attachments/20070926/373ae8d5/attachment.bin From sallyfloyd at mac.com Thu Sep 27 12:14:21 2007 From: sallyfloyd at mac.com (Sally Floyd) Date: Thu, 27 Sep 2007 12:14:21 -0700 Subject: [Tmrg] Comments on in draft-irtf-tmrg-metrics-10.txt In-Reply-To: References: Message-ID: <075efda316804da2ff757dc866dbfd07@mac.com> Lachlan - > Here are some further comments on draft-irtf-tmrg-metrics-10.txt. Many thanks, I made all of the changes that you suggested. (This feedback was just in time, as the document has just finished IRTF review, and is now being forwarded to the next stage in the process...) - Sally http://www.icir.org/floyd/ From lachlan.andrew at gmail.com Sat Sep 29 17:51:33 2007 From: lachlan.andrew at gmail.com (Lachlan Andrew) Date: Sat, 29 Sep 2007 17:51:33 -0700 Subject: [Tmrg] Round table: Buffer sizes In-Reply-To: <46F98369.8030605@psc.edu> References: <46F98369.8030605@psc.edu> Message-ID: Greetings John, On 25/09/2007, John Heffner wrote: > Lachlan Andrew wrote: > > > > suitable buffer sizes for this core set of tests? > > > Given the relatively wide range of buffer sizes in hardware out there at > any given speed, and the significant effect buffer size can have on > congestion control, it seems like this should be an extra dimension > rather than fixed per link type. Absolutely. As Sally suggests, it would be good to have some tests specifically along this dimension. That leaves open what value(s?) to use for this parameter for tests along other dimensions. > Maybe select a couple different sizes based on a set of drain times? > Say, {1 ms, 10 ms, 100 ms, 1 sec}? Convert this to bytes or packets by > dividing by the link byte or MTU packet rate. Sounds good. As I pointed out, converting delay to packets isn't as simple as dividing by MTU packet rate, because we have reverse ACKs which aren't full sized. If we have an equal number of forward and backward flows every second ACK is delayed, we need a buffer of roughly (3/2)(drain_time/MTU_rate) packets. Without delayed ACKs, it becomes rougly 2(drain_time/MTU_rate). In my limited experience, buffer sizes are typically powers of 2. I propose that all "core" tests use buffers with a limit (in packets) equal to a power of 2. Do others agree? Cheers, Lachlan -- Lachlan Andrew Dept of Computer Science, Caltech 1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA Phone: +1 (626) 395-8820 Fax: +1 (626) 568-3603