From lachlan.andrew at gmail.com Sun Dec 2 23:49:41 2007 From: lachlan.andrew at gmail.com (Lachlan Andrew) Date: Sun, 2 Dec 2007 23:49:41 -0800 Subject: [Tmrg] Round-table PFLDnet submission In-Reply-To: References: <88d780b40711202301x2b88638dr825ec577a91d9f31@mail.gmail.com> <6629ccb361ac01b8abf0562fc8a996d8@mac.com> Message-ID: Greetings all, Does silence mean people are happy with my new proposal to measure load in terms of simultaneous sessions in a processor sharing M/G/1 queue? We're aiming to have this settled within a week, so now would be a good time to comment on this or any other issues with the document (see attached .dvi). Also, I'd ask all authors to commit regularly to CVS so that we can all see the latest. Currently it looks like the RTT section is entirely empty. Sally, do you mind if I cut-and-paste the discussion of RTTs from your section into that section? Again, I'll take silence as permission :) (We can always back it out of CVS.) Cheers, Lachlan On 28/11/2007, Lachlan Andrew wrote: > Greetings Sally and everyone, > > In the description of delay/throughput tradeoff, it talks about > "moderate congestion" as 1-2% packet loss with NewReno. Unless I'm > mistaken, that says "windows should be about 1/sqrt(0.01)=10 packets" > (to within a small factor). I'd prefer not to quantify the load that > way. Consider some scenarios: > > 56kbit/s: 10 packets of 12000 bits > 200ms. That means that for 56k > tests with inter-city RTTs (50ms), a moderate level of load would be > *half* of one flow. > > 100Mbit/s bottleneck, 100ms path. "Moderate" congestion would be when > 2000 flows each gets about 50kbit/s. To me, that is very heavy load. > Indeed, however large the bottleneck bandwidth is, "moderate" > congestion would be when 100ms paths give 50kbit/s per user. > > > I'd much prefer to specify the load in terms of the offered load as a > fraction of bandwidth. > > I propose an alternative: The "load" is the average number of flows > if the traffic was served by an M/G/1 queue with an ideal > processor-sharing service discipline. > > My reasons are: > 1. This scales properly as capacity increases, and is correctly > independent of RTT > > 2. A processor-sharing M/G/1 queue is a model of roughly what we're > aiming for with a single bottleneck (equal instantaneous rates). > > 3. For loads like 10%, this simply corresponds to 10% of the bandwidth. > > 4. It reflects that, even at extreme overload, we want to consider a > system whose average number of flows doesn't increase with time. > Otherwise, the results would be very sensitive to duration, and we > agreed that we should try to design tests which are not sensitive > to the parameters. > > Thoughts? -- Lachlan Andrew Dept of Computer Science, Caltech 1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA Ph: +1 (626) 395-8820 Fax: +1 (626) 568-3603 http://netlab.caltech.edu/~lachlan -------------- next part -------------- A non-text attachment was scrubbed... Name: pfldnet2008.dvi Type: application/x-dvi Size: 30832 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/tmrg-interest/attachments/20071202/64a00fe6/attachment-0001.dvi From sangtae.ha at gmail.com Mon Dec 3 10:52:17 2007 From: sangtae.ha at gmail.com (SANGTAE HA) Date: Mon, 3 Dec 2007 13:52:17 -0500 Subject: [Tmrg] Traffic Generators (Harpoon and Tmix) Message-ID: <649aecc70712031052k3d82e71ao4b9198374b37c99e@mail.gmail.com> Hi all, We have two compelling traffic generators, Tmix[1] and Harpoon[2], one of them will be used as a common traffic generator for TCP testing. Before deciding which traffic geneator we would go, I list up simple comparisons between them. Feel free to update the table. ---------------------------------------------------------------- Tmix Harpoon ---------------------------------------------------------------- TCP/UDP application-level application-level TCP TCP/UDP ---------------------------------------------------------------- Model *(a,b,t) model inter-arrival time and file size distributions ---------------------------------------------------------------- Trace tcpdump flow-tool (from routers) *manual *manual ---------------------------------------------------------------- Supported Linux Linux FreeBSD (FreeBSD) NS2 ---------------------------------------------------------------- *(a,b,t) = (request size, response size, user think time) * "manual" means it supports user-generated vectors or distribution tables Briefly, Tmix supports more platforms (NS2) while Harpoon includes an additional UDP generation. After reading the Tmix paper, it looks *(a,b,t) model can represent user-interactions better than the model based on inter-arrival and file size distributions. Welcome your comments. Sangtae [1] M. Weigle, P. Adurthi, F. Hernandez-Campos, K. Jeffay and F. D. Smith, Tmix: A Tool for Generating Realistic TCP Application Workloads in ns-2, CCR, July 2006 [2] J. Sommers and P. Barford, Self-Configuring Network Traffic Generation, IMC 2004. From lachlan.andrew at gmail.com Mon Dec 3 11:28:23 2007 From: lachlan.andrew at gmail.com (Lachlan Andrew) Date: Mon, 3 Dec 2007 11:28:23 -0800 Subject: [Tmrg] Traffic Generators (Harpoon and Tmix) In-Reply-To: <649aecc70712031052k3d82e71ao4b9198374b37c99e@mail.gmail.com> References: <649aecc70712031052k3d82e71ao4b9198374b37c99e@mail.gmail.com> Message-ID: Greetings Sangtae, On 03/12/2007, SANGTAE HA wrote: > We have two compelling traffic generators, Tmix[1] and Harpoon[2], one > of them will be used as a common traffic generator for TCP testing. > Before deciding which traffic geneator we would go, I list up simple > comparisons between them. Feel free to update the table. > > ---------------------------------------------------------------- > Tmix Harpoon > ---------------------------------------------------------------- > TCP/UDP application-level application-level > TCP TCP/UDP > ---------------------------------------------------------------- > Model *(a,b,t) model inter-arrival time and > file size distributions > ---------------------------------------------------------------- > Trace tcpdump flow-tool (from routers) > *manual *manual > ---------------------------------------------------------------- > Supported Linux Linux > FreeBSD (FreeBSD) > NS2 > ---------------------------------------------------------------- > > *(a,b,t) = (request size, response size, user think time) > * "manual" means it supports user-generated vectors or distribution tables > > Briefly, Tmix supports more platforms (NS2) while Harpoon includes an > additional UDP generation. > After reading the Tmix paper, it looks *(a,b,t) model can represent > user-interactions better than the model based on inter-arrival and > file size distributions. Thanks for checking this out. I notice that Tmix aims to model non-greedy TCP connections. The "think times" are not times between user connections, but pauses within a connection. Will that make it harder for us to collect statistics? If we're measuring things like "file completion time", it is much harder to define what a "file" is if it is just part of a long-running non-greedy TCP connection. Tmix is clearly a more general model, but I personally prefer the simplicity of considering TCP sources to be greedy. It simplifies distinguishing between the effect of slow-start vs normal operation. Cheers, Lachlan -- Lachlan Andrew Dept of Computer Science, Caltech 1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA Ph: +1 (626) 395-8820 Fax: +1 (626) 568-3603 http://netlab.caltech.edu/~lachlan From sallyfloyd at mac.com Tue Dec 4 00:10:20 2007 From: sallyfloyd at mac.com (Sally Floyd) Date: Tue, 4 Dec 2007 00:10:20 -0800 Subject: [Tmrg] Round-table PFLDnet submission In-Reply-To: References: <88d780b40711202301x2b88638dr825ec577a91d9f31@mail.gmail.com> <6629ccb361ac01b8abf0562fc8a996d8@mac.com> Message-ID: <907c0ed0462c4a2f923764e8eb9cf32c@mac.com> Lachlan - > Does silence mean people are happy with my new proposal to measure > load in terms of simultaneous sessions in a processor sharing M/G/1 > queue? Sorry, I am at IETF in Vancouver this week, and swamped with a number of things, but I will try to get to the PFLDnet email in the next few days... ... > Currently it looks like the RTT section is entirely empty. Sally, do > you mind if I cut-and-paste the discussion of RTTs from your section > into that section? Again, I'll take silence as permission :) (We can > always back it out of CVS.) Any cutting and pasting is fine by me. - Sally http://www.icir.org/floyd/ From cesar at cs.ucla.edu Tue Dec 4 00:27:53 2007 From: cesar at cs.ucla.edu (Cesar Marcondes) Date: Tue, 4 Dec 2007 00:27:53 -0800 Subject: [Tmrg] Fwd: Round-table PFLDnet submission In-Reply-To: <88d780b40712040024i31a3590dh45217a3c19c34a2@mail.gmail.com> References: <88d780b40711202301x2b88638dr825ec577a91d9f31@mail.gmail.com> <6629ccb361ac01b8abf0562fc8a996d8@mac.com> <88d780b40712040024i31a3590dh45217a3c19c34a2@mail.gmail.com> Message-ID: <88d780b40712040027h7ba73735nad38950cb3f14b6c@mail.gmail.com> Dear Lachlan, On Dec 2, 2007 11:49 PM, Lachlan Andrew wrote: > Greetings all, > > Does silence mean people are happy with my new proposal to measure > load in terms of simultaneous sessions in a processor sharing M/G/1 > queue? Sorry, I'm a bit busy these days. I will try to comment on your load proposal a bit. 1) "The load is varied by scaling the interarrival times by a constant. We invite other researchers to test the assumption that the file-size distribution is independent of the load". It sounds reasonable to me, files exist in hard-drives, imho they shouldn't be dependent on the load unless the dynamic content size is dependent on the load. 2) Is there a straightforward algorithm to obtain the mean queue size of M/G/1 using shortest-remaining-processing-time-first? my concern here is more practical, is it necessary to code in ns-2 a M/G/1 queue size solver for the test suite? How about deriving loads based on standard reno? we set a fixed N independent non-infinite TCP streams and measure the load based on standard Reno. And this search of N process would find when the load is ~10% and so on. > We're aiming to have this settled within a week, so now would be a > good time to comment on this or any other issues with the document > (see attached .dvi). > > Also, I'd ask all authors to commit regularly to CVS so that we can > all see the latest. ok. > Currently it looks like the RTT section is entirely empty. Sally, do > you mind if I cut-and-paste the discussion of RTTs from your section > into that section? Again, I'll take silence as permission :) (We can > always back it out of CVS.) I run the scripts from http://www.icir.org/models/sims.html. In the case of figure 4, the site says ... ## Figure 4, web traffic and long-lived flows, with a range of RTTs: ./ns sims.tcl -flows 18 -web 400 -rtts 1 -title two > two.data csh sims.cmd; cp reda.eps sims2.eps; gv sims2.eps & the log with the access links RTTs w/ 18 access links is attached. > > Cheers, > Lachlan > > > On 28/11/2007, Lachlan Andrew wrote: > > Greetings Sally and everyone, > > > > In the description of delay/throughput tradeoff, it talks about > > "moderate congestion" as 1-2% packet loss with NewReno. Unless I'm > > mistaken, that says "windows should be about 1/sqrt(0.01)=10 packets" > > (to within a small factor). I'd prefer not to quantify the load that > > way. Consider some scenarios: > > > > 56kbit/s: 10 packets of 12000 bits > 200ms. That means that for 56k > > tests with inter-city RTTs (50ms), a moderate level of load would be > > *half* of one flow. > > > > 100Mbit/s bottleneck, 100ms path. "Moderate" congestion would be when > > 2000 flows each gets about 50kbit/s. To me, that is very heavy load. > > Indeed, however large the bottleneck bandwidth is, "moderate" > > congestion would be when 100ms paths give 50kbit/s per user. > > > > > > I'd much prefer to specify the load in terms of the offered load as a > > fraction of bandwidth. > > > > I propose an alternative: The "load" is the average number of flows > > if the traffic was served by an M/G/1 queue with an ideal > > processor-sharing service discipline. > > > > My reasons are: > > 1. This scales properly as capacity increases, and is correctly > > independent of RTT > > > > 2. A processor-sharing M/G/1 queue is a model of roughly what we're > > aiming for with a single bottleneck (equal instantaneous rates). > > > > 3. For loads like 10%, this simply corresponds to 10% of the bandwidth. > > > > 4. It reflects that, even at extreme overload, we want to consider a > > system whose average number of flows doesn't increase with time. > > Otherwise, the results would be very sensitive to duration, and we > > agreed that we should try to design tests which are not sensitive > > to the parameters. > > > > Thoughts? > > > -- > > Lachlan Andrew Dept of Computer Science, Caltech > 1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA > Ph: +1 (626) 395-8820 Fax: +1 (626) 568-3603 > http://netlab.caltech.edu/~lachlan > > _______________________________________________ > Tmrg-interest mailing list > Tmrg-interest at ICSI.Berkeley.EDU > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/tmrg-interest > > -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: rtts_bettermodels_fig4.txt Url: http://mailman.ICSI.Berkeley.EDU/pipermail/tmrg-interest/attachments/20071204/6fb20be4/attachment.txt From cesar at cs.ucla.edu Tue Dec 4 00:41:46 2007 From: cesar at cs.ucla.edu (Cesar Marcondes) Date: Tue, 4 Dec 2007 00:41:46 -0800 Subject: [Tmrg] Traffic Generators (Harpoon and Tmix) In-Reply-To: References: <649aecc70712031052k3d82e71ao4b9198374b37c99e@mail.gmail.com> Message-ID: <88d780b40712040041n141be25dgec24a2581f00403f@mail.gmail.com> Dear Sangtae, I agree w/ Lachlan. I think this extra complexity of application behavior forcing the pace of TCP transmissions (by waiting read/write times), just add difficulty to separate the TCP congestion control behavior alone, for the specific goals of the TCP Test Suite. Just my 2 cents, Cesar On Dec 3, 2007 11:28 AM, Lachlan Andrew wrote: > Greetings Sangtae, > > On 03/12/2007, SANGTAE HA wrote: > > We have two compelling traffic generators, Tmix[1] and Harpoon[2], one > > of them will be used as a common traffic generator for TCP testing. > > Before deciding which traffic geneator we would go, I list up simple > > comparisons between them. Feel free to update the table. > > > > ---------------------------------------------------------------- > > Tmix Harpoon > > ---------------------------------------------------------------- > > TCP/UDP application-level application-level > > TCP TCP/UDP > > ---------------------------------------------------------------- > > Model *(a,b,t) model inter-arrival time and > > file size distributions > > ---------------------------------------------------------------- > > Trace tcpdump flow-tool (from routers) > > *manual *manual > > ---------------------------------------------------------------- > > Supported Linux Linux > > FreeBSD (FreeBSD) > > NS2 > > ---------------------------------------------------------------- > > > > *(a,b,t) = (request size, response size, user think time) > > * "manual" means it supports user-generated vectors or distribution tables > > > > Briefly, Tmix supports more platforms (NS2) while Harpoon includes an > > additional UDP generation. > > After reading the Tmix paper, it looks *(a,b,t) model can represent > > user-interactions better than the model based on inter-arrival and > > file size distributions. > > Thanks for checking this out. > > I notice that Tmix aims to model non-greedy TCP connections. The > "think times" are not times between user connections, but pauses > within a connection. Will that make it harder for us to collect > statistics? If we're measuring things like "file completion time", it > is much harder to define what a "file" is if it is just part of a > long-running non-greedy TCP connection. > > Tmix is clearly a more general model, but I personally prefer the > simplicity of considering TCP sources to be greedy. It simplifies > distinguishing between the effect of slow-start vs normal operation. > > Cheers, > Lachlan > > -- > Lachlan Andrew Dept of Computer Science, Caltech > 1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA > Ph: +1 (626) 395-8820 Fax: +1 (626) 568-3603 > http://netlab.caltech.edu/~lachlan > > _______________________________________________ > Tmrg-interest mailing list > Tmrg-interest at ICSI.Berkeley.EDU > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/tmrg-interest > From mweigle at cs.odu.edu Tue Dec 4 04:02:45 2007 From: mweigle at cs.odu.edu (Michele Weigle) Date: Tue, 4 Dec 2007 07:02:45 -0500 Subject: [Tmrg] Traffic Generators (Harpoon and Tmix) Message-ID: There is work currently ongoing to implement the three simulators Tmix, Harpoon, and Swing (from Amin Vahdat's group at UCSD, http://www.cs.ucsd.edu/~kvishwanath/Swing/) in ns-2, GTNetS, and the upcoming ns-3. So far, we have Tmix implemented in ns-2 and GTNetS. We're currently in the process of validating the GTNetS implementation and preparing both versions for release. In addition to this simulation-based work, collaborators at UNC will be building a framework for Linux and FreeBSD in which any of these three simulators could be used in testbed experiments. See http://nsf.gov/awardsearch/showAward.do?AwardNumber=0709081 Also, I believe that the Tmix model can be extended to support UDP traffic, but I'm not sure if that's been implemented yet. Regarding the pauses that are part of the Tmix model, these pauses are used to represent the time between complete application data units (ADUs), which are essentially files. If you were modeling HTTP connections, for example, the 'a's would be requests and 'b's would be responses. You are right that Tmix can model persistent HTTP connections, where there are pauses in a single connection. If you wanted to have a set of long-lived greedy TCP flows, you could construct connection vectors to give you such behavior. -Michele On Dec 3, 2007, at 1:52 PM, SANGTAE HA wrote: Hi all, We have two compelling traffic generators, Tmix[1] and Harpoon[2], one of them will be used as a common traffic generator for TCP testing. Before deciding which traffic geneator we would go, I list up simple comparisons between them. Feel free to update the table. ---------------------------------------------------------------- Tmix Harpoon ---------------------------------------------------------------- TCP/UDP application-level application-level TCP TCP/UDP ---------------------------------------------------------------- Model *(a,b,t) model inter-arrival time and file size distributions ---------------------------------------------------------------- Trace tcpdump flow-tool (from routers) *manual *manual ---------------------------------------------------------------- Supported Linux Linux FreeBSD (FreeBSD) NS2 ---------------------------------------------------------------- *(a,b,t) = (request size, response size, user think time) * "manual" means it supports user-generated vectors or distribution tables Briefly, Tmix supports more platforms (NS2) while Harpoon includes an additional UDP generation. After reading the Tmix paper, it looks *(a,b,t) model can represent user-interactions better than the model based on inter-arrival and file size distributions. Welcome your comments. Sangtae [1] M. Weigle, P. Adurthi, F. Hernandez-Campos, K. Jeffay and F. D. Smith, Tmix: A Tool for Generating Realistic TCP Application Workloads in ns-2, CCR, July 2006 [2] J. Sommers and P. Barford, Self-Configuring Network Traffic Generation, IMC 2004. _______________________________________________ Tmrg-interest mailing list Tmrg-interest at ICSI.Berkeley.EDU http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/tmrg-interest -- Michele Weigle Assistant Professor Department of Computer Science Old Dominion University Norfolk, VA 23539 mweigle at cs.odu.edu http://www.cs.odu.edu/~mweigle (757) 683-6001 ext. 5050 From ldunn at cisco.com Tue Dec 4 13:54:19 2007 From: ldunn at cisco.com (Lawrence D. Dunn) Date: Tue, 4 Dec 2007 15:54:19 -0600 Subject: [Tmrg] Round-table PFLDnet submission In-Reply-To: References: <88d780b40711202301x2b88638dr825ec577a91d9f31@mail.gmail.com> <6629ccb361ac01b8abf0562fc8a996d8@mac.com> Message-ID: Lachlan, SIlence from me just meant that I haven't had time to ponder it sufficiently. Since I trust you quite a bit, if "enough" other weigh in, and rough consensus is declared before I have a chance to think on it, I won't gripe. ;-) Larry -- At 11:49 PM -0800 12/2/07, Lachlan Andrew wrote: >Greetings all, > >Does silence mean people are happy with my new proposal to measure >load in terms of simultaneous sessions in a processor sharing M/G/1 >queue? > >We're aiming to have this settled within a week, so now would be a >good time to comment on this or any other issues with the document >(see attached .dvi). > >Also, I'd ask all authors to commit regularly to CVS so that we can >all see the latest. > >Currently it looks like the RTT section is entirely empty. Sally, do >you mind if I cut-and-paste the discussion of RTTs from your section >into that section? Again, I'll take silence as permission :) (We can >always back it out of CVS.) > >Cheers, >Lachlan > >On 28/11/2007, Lachlan Andrew wrote: >> Greetings Sally and everyone, >> >> In the description of delay/throughput tradeoff, it talks about >> "moderate congestion" as 1-2% packet loss with NewReno. Unless I'm >> mistaken, that says "windows should be about 1/sqrt(0.01)=10 packets" >> (to within a small factor). I'd prefer not to quantify the load that >> way. Consider some scenarios: >> >> 56kbit/s: 10 packets of 12000 bits > 200ms. That means that for 56k >> tests with inter-city RTTs (50ms), a moderate level of load would be >> *half* of one flow. >> >> 100Mbit/s bottleneck, 100ms path. "Moderate" congestion would be when >> 2000 flows each gets about 50kbit/s. To me, that is very heavy load. >> Indeed, however large the bottleneck bandwidth is, "moderate" >> congestion would be when 100ms paths give 50kbit/s per user. >> >> >> I'd much prefer to specify the load in terms of the offered load as a >> fraction of bandwidth. >> >> I propose an alternative: The "load" is the average number of flows >> if the traffic was served by an M/G/1 queue with an ideal >> processor-sharing service discipline. >> >> My reasons are: >> 1. This scales properly as capacity increases, and is correctly >> independent of RTT >> >> 2. A processor-sharing M/G/1 queue is a model of roughly what we're >> aiming for with a single bottleneck (equal instantaneous rates). >> >> 3. For loads like 10%, this simply corresponds to 10% of the bandwidth. >> >> 4. It reflects that, even at extreme overload, we want to consider a >> system whose average number of flows doesn't increase with time. >> Otherwise, the results would be very sensitive to duration, and we >> agreed that we should try to design tests which are not sensitive >> to the parameters. >> >> Thoughts? > > >-- >Lachlan Andrew Dept of Computer Science, Caltech >1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA >Ph: +1 (626) 395-8820 Fax: +1 (626) 568-3603 >http://netlab.caltech.edu/~lachlan > >Content-Type: application/x-dvi; name=pfldnet2008.dvi >X-Attachment-Id: f_f9qpb2lb >Content-Disposition: attachment; filename=pfldnet2008.dvi > >Attachment converted: PB17.1.65GB:pfldnet2008.dvi ( / ) (002431F0) >_______________________________________________ >Tmrg-interest mailing list >Tmrg-interest at ICSI.Berkeley.EDU >http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/tmrg-interest From sallyfloyd at mac.com Tue Dec 4 14:41:59 2007 From: sallyfloyd at mac.com (Sally Floyd) Date: Tue, 4 Dec 2007 14:41:59 -0800 Subject: [Tmrg] Traffic Generators (Harpoon and Tmix) In-Reply-To: References: Message-ID: <6b38e9bc749fadc81e198ff39ebfadb5@mac.com> From Michele: > Regarding the pauses that are part of the Tmix model, these pauses are > used to represent the time between complete application data units > (ADUs), which are essentially files. If you were modeling HTTP > connections, for example, the 'a's would be requests and 'b's would be > responses. You are right that Tmix can model persistent HTTP > connections, where there are pauses in a single connection. If you > wanted to have a set of long-lived greedy TCP flows, you could > construct connection vectors to give you such behavior. But if one wanted to use a realistic traffic model in one's scenarios, that would have to include non-greedy TCP traffic (e.g., HTTP 1.1 traffic, telnet traffic or other user-generated data, etc.) Non-greedy TCP traffic is not unusual in the real world, and can be a significant stressor on congestion control mechanisms that it would be a pity to ignore - e.g., TCP flows that *might* ramp up to a high sending rate, have a data-limited or idle period, and then continue with a lot of data to send again. When we are measuring file completion times, we can use greedy TCP connections to measure them, in a mix with other traffic. The heavy-tailed distribution of user wait times within TCP connections has been under discussion since 1994. E.g., "Wide-Area Traffic: The Failure of Poisson Modeling", Paxson, V. and Floyd, S., SIGCOMM 1994 (or the 1995 IEEE/ACM Transactions on Networking version). And has been included in traffic models in ns-2 for many years (e.g., in Polly Huang's traffic generator listed in "http://www.icir.org/models/trafficgenerators.html". - Sally http://www.icir.org/floyd/ From dovrolis at cc.gatech.edu Tue Dec 4 15:23:21 2007 From: dovrolis at cc.gatech.edu (Constantine Dovrolis) Date: Tue, 04 Dec 2007 18:23:21 -0500 Subject: [Tmrg] Traffic Generators (Harpoon and Tmix) In-Reply-To: <6b38e9bc749fadc81e198ff39ebfadb5@mac.com> References: <6b38e9bc749fadc81e198ff39ebfadb5@mac.com> Message-ID: <4755E169.1030800@cc.gatech.edu> folks, my apologies for jumping into the discussion, but 1. I want to loudly agree with Sally that we should be considering non-greedy TCP flows with heavy-tailed size distribution, and 2. we should be asking whether these non-greedy TCP flows are generated by an open-loop flow arrival process or by a closed-loop process that takes user thinking times (and perhaps limited patience) into account. A couple of related papers: http://www.cc.gatech.edu/fac/Constantinos.Dovrolis/Papers/ravi-openclosed.pdf http://www.cc.gatech.edu/fac/Constantinos.Dovrolis/Papers/pam07-ravi.pdf Constantine -------------------------------------------------------------- Constantine Dovrolis | 3346 KACB | 404-385-4205 Associate Professor | Networking and Telecommunications Group College of Computing | Georgia Institute of Technology dovrolis at cc.gatech.edu http://www.cc.gatech.edu/~dovrolis/ Sally Floyd wrote: > From Michele: >> Regarding the pauses that are part of the Tmix model, these pauses are >> used to represent the time between complete application data units >> (ADUs), which are essentially files. If you were modeling HTTP >> connections, for example, the 'a's would be requests and 'b's would be >> responses. You are right that Tmix can model persistent HTTP >> connections, where there are pauses in a single connection. If you >> wanted to have a set of long-lived greedy TCP flows, you could >> construct connection vectors to give you such behavior. > > But if one wanted to use a realistic traffic model in one's scenarios, > that would have to include non-greedy TCP traffic (e.g., HTTP 1.1 > traffic, telnet traffic or other user-generated data, etc.) Non-greedy > TCP traffic is not unusual in the real world, and can be a significant > stressor on congestion control mechanisms that it would be a pity > to ignore - e.g., TCP flows that *might* ramp up to a high sending rate, > have a data-limited or idle period, and then continue with a lot of > data to send again. > > When we are measuring file completion times, we can use > greedy TCP connections to measure them, in a mix with other traffic. > > The heavy-tailed distribution of user wait times within TCP connections > has been under discussion since 1994. E.g., "Wide-Area Traffic: The > Failure of Poisson Modeling", Paxson, V. and Floyd, S., SIGCOMM 1994 > (or the 1995 IEEE/ACM Transactions on Networking version). And has > been included in traffic models in ns-2 for many years (e.g., in Polly > Huang's traffic generator listed in > "http://www.icir.org/models/trafficgenerators.html". > > - Sally > http://www.icir.org/floyd/ > > _______________________________________________ > Tmrg-interest mailing list > Tmrg-interest at ICSI.Berkeley.EDU > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/tmrg-interest From lachlan.andrew at gmail.com Thu Dec 6 20:23:15 2007 From: lachlan.andrew at gmail.com (Lachlan Andrew) Date: Thu, 6 Dec 2007 20:23:15 -0800 Subject: [Tmrg] Traffic Generators (Harpoon and Tmix) In-Reply-To: <4755E169.1030800@cc.gatech.edu> References: <6b38e9bc749fadc81e198ff39ebfadb5@mac.com> <4755E169.1030800@cc.gatech.edu> Message-ID: On 04/12/2007, Constantine Dovrolis wrote: > folks, my apologies for jumping into the discussion Not at all. Thanks for your input! > 1. I want to loudly agree with Sally that we should be > considering non-greedy TCP flows with heavy-tailed size > distribution, and > 2. we should be asking whether these non-greedy TCP flows > are generated by an open-loop flow arrival process or by a > closed-loop process that takes user thinking times (and > perhaps limited patience) into account. Just to clarify, in point 2, are you suggesting that there are idle/think times both within and between flows? I agree entirely that these are all important effects, which should be included in "version 2" of the test suite. I have several reason for supporting the simpler models for the initial "version 1". I'd be interested in your thoughts on each. My strongest concerns are points 2(ii) and 4. 1. We agreed at the meeting that the load would be "open loop". That allows us to specify the offered load in a protocol-independent way. If the traffic is entirely closed-loop then the load depends on the protocols, making comparisons difficult. (Being open-loop does not preclude modelling the think-time between arrivals within a session.) 2. We need to ask what cost/benefit we get from the more complex models. (i) For some of our tests, this traffic is "cross traffic" which we're not measuring. In these tests, the results of Hohn, Veitch, and Abry (e.g., "The impact of the flow arrival process in Internet traffic") suggest that structure in the flow arrival process doesn't greatly affect the packet level traffic. (ii) For cases where we're going to measure the performance of non-greedy flows, we need to define metrics for their performance which reflect the non-greediness. I don't think such measures are obvious. We can't use connection completion times, average rates, ... 3. These tests are not intended to be exhaustive. As I said before the meeting, I'd rather the meeting result in one or two clearly-defined tests than a complete first draft of a test suite where none of the tests is specified well enough to allow comparisons. 4. I'm afraid of models with too many parameters which have to be estimated. I was under the impression that many studies have found distributions of *connection* sizes, but many fewer (if any) have studied the sizes of "bursts" within a connection. Will it matter if we get the sizes wrong? Another point related to parameter estimation is that I'm worried by the approach we agreed on of assuming that the file-size distribution is independent of the load, so that the load is simply proportional to the session arrival rate. It seems likely to me that higher load occurs when there is a brief influx of longer connections (say some BitTorrent users start up), rather than a brief rise in the session arrival rate. Could this have as big an impact as the choice of whether new "bursts" start their own connections or not? Cheers, Lachlan -- Lachlan Andrew Dept of Computer Science, Caltech 1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA Ph: +1 (626) 395-8820 Fax: +1 (626) 568-3603 http://netlab.caltech.edu/~lachlan From sallyfloyd at mac.com Sun Dec 9 18:21:39 2007 From: sallyfloyd at mac.com (Sally Floyd) Date: Sun, 9 Dec 2007 18:21:39 -0800 Subject: [Tmrg] Traffic Generators (Harpoon and Tmix) In-Reply-To: References: <6b38e9bc749fadc81e198ff39ebfadb5@mac.com> <4755E169.1030800@cc.gatech.edu> Message-ID: <25cf62fb7743f744e0d7d20a7524ee09@mac.com> On Dec 6, 2007, at 8:23 PM, Lachlan Andrew wrote: > On 04/12/2007, Constantine Dovrolis wrote: >> folks, my apologies for jumping into the discussion > > Not at all. Thanks for your input! > >> 1. I want to loudly agree with Sally that we should be >> considering non-greedy TCP flows with heavy-tailed size >> distribution, and >> 2. we should be asking whether these non-greedy TCP flows >> are generated by an open-loop flow arrival process or by a >> closed-loop process that takes user thinking times (and >> perhaps limited patience) into account. > > Just to clarify, in point 2, are you suggesting that there are > idle/think times both within and between flows? > > I agree entirely that these are all important effects, which should be > included in "version 2" of the test suite. I have several reason for > supporting the simpler models for the initial "version 1". I'd be > interested in your thoughts on each. My strongest concerns are points > 2(ii) and 4. > > 1. We agreed at the meeting that the load would be "open loop". That > allows us to specify the offered load in a protocol-independent way. > If the traffic is entirely closed-loop then the load depends on the > protocols, making comparisons difficult. (Being open-loop does not > preclude modelling the think-time between arrivals within a session.) Closed-loop models are just as protocol-independent as open-loop models, I would say. The overall transfer time depends on the protocol used in either case. > 2. We need to ask what cost/benefit we get from the more complex > models. > (i) For some of our tests, this traffic is "cross traffic" which we're > not measuring. In these tests, the results of Hohn, Veitch, and Abry > (e.g., "The impact of the flow arrival process in Internet traffic") > suggest that structure in the flow arrival process doesn't greatly > affect the packet level traffic. > (ii) For cases where we're going to measure the performance of > non-greedy flows, we need to define metrics for their performance > which reflect the non-greediness. I don't think such measures are > obvious. We can't use connection completion times, average rates, ... Even if I was only looking at metrics about the behavior of long-lived flows, I would prefer for the "background traffic" to have user think times within TCP connections. This is more realistic, and increases the burstiness of the aggregate traffic in a way that affects all of the competing traffic. > 3. These tests are not intended to be exhaustive. As I said before > the meeting, I'd rather the meeting result in one or two > clearly-defined tests than a complete first draft of a test suite > where none of the tests is specified well enough to allow comparisons. I think we can do a complete first draft of a test suite. But I agree that these tests are definitely not intended to be exhaustive. > 4. I'm afraid of models with too many parameters which have to be > estimated. I was under the impression that many studies have found > distributions of *connection* sizes, but many fewer (if any) have > studied the sizes of "bursts" within a connection. Will it matter if > we get the sizes wrong? We will get it even more wrong if we don't include user think times within connections. One of the good areas for future work is for researchers to say "by the way, these results are quite sensitive to parameter X", or "these results are not at all sensitive to parameter Y". It is unavoidable, I think, that we will have to learn these things as we go along. > Another point related to parameter estimation is that I'm worried by > the approach we agreed on of assuming that the file-size distribution > is independent of the load, so that the load is simply proportional to > the session arrival rate. It seems likely to me that higher load > occurs when there is a brief influx of longer connections (say some > BitTorrent users start up), rather than a brief rise in the session > arrival rate. Could this have as big an impact as the choice of > whether new "bursts" start their own connections or not? I agree that this is a key concern. There are two ways to go: (1) models where the total load requested in a user session is independent of the level of congestion: and (2) models where the total load requested in a user session is explicitly dependent on the level of congestion. I assume that the world is like (2). As far as I know, more traffic generators are based on model (1). We could make an arbitrary attempt at model (2), or we could use model (1) and explicitly ask researchers to give us model (2) for traffic generation for the future. Either one sounds ok to me. - Sally http://www.icir.org/floyd/ From lachlan.andrew at gmail.com Sun Dec 9 19:07:27 2007 From: lachlan.andrew at gmail.com (Lachlan Andrew) Date: Sun, 9 Dec 2007 19:07:27 -0800 Subject: [Tmrg] Traffic Generators (Harpoon and Tmix) In-Reply-To: <25cf62fb7743f744e0d7d20a7524ee09@mac.com> References: <6b38e9bc749fadc81e198ff39ebfadb5@mac.com> <4755E169.1030800@cc.gatech.edu> <25cf62fb7743f744e0d7d20a7524ee09@mac.com> Message-ID: Greetings Sally and all, On 09/12/2007, Sally Floyd wrote: > On Dec 6, 2007, at 8:23 PM, Lachlan Andrew wrote: > > 1. We agreed at the meeting that the load would be "open loop". That > > allows us to specify the offered load in a protocol-independent way. > > If the traffic is entirely closed-loop then the load depends on the > > protocols, making comparisons difficult. (Being open-loop does not > > preclude modelling the think-time between arrivals within a session.) > > Closed-loop models are just as protocol-independent as open-loop models, > I would say. > The overall transfer time depends on the protocol used in either case. The transfer times depends on the protocol in both cases. However, the *total* amount of cross traffic depends on the protocol in one case but not in the other. With an open-loop model, it is meaningful to talk about "10% cross traffic", because we specify how much data arrives in what long period of time. With a purely closed-loop model, inefficient algorithms will receive less traffic, because flows arrive less often. There is AFAIK no way to specify that a closed-loop model gives x% cross traffic. > Even if I was only looking at metrics about the behavior of long-lived > flows, I would prefer for the "background traffic" to have user think times > within TCP connections. This is more realistic, and increases the burstiness > of the aggregate traffic in a way that affects all of the competing > traffic. There is certainly a good case for making the background traffic as realistic as possible, all else being equal. If Tmix does all the hard work and comes complete with representative traces, I'd be happy for us to specify that cross traffic be non-greedy. I still don't know what metrics would be meaningful if we're measuring non-greedy traffic. This will affect what we do for the cases where all traffic comes from the traffic generators (such as the throughput vs delay scenarios). > One of the good areas for future work is for researchers to say > "by the way, these results are quite sensitive to parameter X", or > "these results are not at all sensitive to parameter Y". It is > unavoidable, > I think, that we will have to learn these things as we go along. Agreed. > There are two ways to go: > (1) models where the total load requested in a user session is > independent of the level of congestion: and > (2) models where the total load requested in a user session is > explicitly dependent on the level of congestion. > > I assume that the world is like (2). As far as I know, more traffic > generators are based on model (1). We could make an arbitrary > attempt at model (2), or we could use model (1) and explicitly ask > researchers to give us model (2) for traffic generation for the future. > Either one sounds ok to me. I'd vote for using (1) and explicitly suggesting that it be modified. It would be a shame to "standardize" an arbitrary model. Cheers, Lachlan -- Lachlan Andrew Dept of Computer Science, Caltech 1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA Ph: +1 (626) 395-8820 Fax: +1 (626) 568-3603 http://netlab.caltech.edu/~lachlan From lachlan.andrew at gmail.com Sun Dec 9 19:38:42 2007 From: lachlan.andrew at gmail.com (Lachlan Andrew) Date: Sun, 9 Dec 2007 19:38:42 -0800 Subject: [Tmrg] Round-table PFLDnet submission In-Reply-To: <753cb8cc282bced6d325993749250d3b@mac.com> References: <20071209215603.v1gsa5smh4e8soc4@tadorne.ens-lyon.fr> <753cb8cc282bced6d325993749250d3b@mac.com> Message-ID: Greetings, On 09/12/2007, Sally Floyd wrote: > > Greetings Romaric, > >> Quoting Lachlan Andrew : > > > > I've given > > reasons that I think this test should be different and *not* have the > > pseudo-random background traffic: > > 1) It adds statistical errors (different experiments will have > > different numbers of flows at the instant of interest) > > 2) It does not really increasing the realism in this case (flows are > > unlikely to arrive within the RTT or so during which the window is > > being reduced, so any flows present at the time are essentially > > "long-lived"). > > > > If anyone can show how it does significantly add realism or how to > > avoid the statistical errors, then let's keep the background traffic. > > Otherwise, I strongly prefer to remove it. > > I think it is important to keep some amount of background traffic and > reverse-path traffic in the scenarios about transients. Just for a > start, the background traffic strongly affects the degree of > synchronization in the loss events (do all flows have a loss at the same > time, or not), and this can strongly affect any of the metrics that are > being measured about the response to the transient events. Synchronization is certainly relevant if there is a significant amount of background traffic. Remember that if we have 10% of traffic being background traffic, then about 80% of the time the only flow in the system is the long-lived flow of interest. Consider a step increase in UDP traffic: a) If the transient occurs during the normal 80%, then we didn't need the cross traffic -- the flow of interest is perfectly synchronized with itself, and will definitely suffer a loss within the first RTT (and most likely each RTT until it drops its rate enough) when the UDP starts. b) If it occurs during the 20%, then we're measuring a statistical outlier, not the normal behaviour. That means that synchronization rate is not a strong reason to introduce cross traffic in this case. You mentioned that synchronization was "for a start". Could you list other reasons? The transient when the UDP is decrease will last many RTTs, and there *is* certainly a case for including cross traffic there. We could test how long it takes for the entire ensemble of flows to reach 80% of full utilization. However, there is also the risk that we might end up measuring the time until a new flow (i.e. slow start) arrives rather than the time until normal congestion avoidance fills the pipe. That would become very sensitive to our traffic model. (Halving the size of files and doubling the rate of arrivals would make a big difference.) Does anyone have any suggestion how to avoid that? Cheers, Lachlan -- Lachlan Andrew Dept of Computer Science, Caltech 1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA Ph: +1 (626) 395-8820 Fax: +1 (626) 568-3603 http://netlab.caltech.edu/~lachlan From lachlan.andrew at gmail.com Mon Dec 10 15:48:32 2007 From: lachlan.andrew at gmail.com (Lachlan Andrew) Date: Mon, 10 Dec 2007 15:48:32 -0800 Subject: [Tmrg] Round-table PFLDnet submission - transient In-Reply-To: <475D043F.6020603@ens-lyon.fr> References: <20071209215603.v1gsa5smh4e8soc4@tadorne.ens-lyon.fr> <475D043F.6020603@ens-lyon.fr> Message-ID: Greetings Romaric, On 10/12/2007, Romaric Guillier wrote: > > However, the max decrease in one RTT isn't very informative. Should > > it be big or small? A flow which randomly sends 0 on odd RTTs and > > 10Gbps on even RTTs will have a very big "maximum decrease", but > > doesn't behave at all well. > > Yep, I agree with you on that, but if you consider the "time to reduce > to 33%" metric, you will get a result like one or less than one RTT, but > it won't tell you either that your protocol is unstable. True. I argued against the "time to reduce to x%" metric at the meeting, but haven't come up with anything better. The reason for making it 33% not 50% as originally proposed was an attempt to look beyond the first "reduction by 50%" that many algorithms have. To me, the "back-off" measure is about how much room is available for the newly-arriving flow, rather than the impact on the existing flow. That is one reason I don't like the "cost" in the original draft -- it only considers the impact on the long-lived flow, not the impact of that flow's response on other flows. Perhaps it would be better to measure the impact on other flows directly. How about the measure number of packets dropped by the UDP sources in the first x seconds If the flow backs off nicely, the number of dropped packets will be small. Similarly, virtual queue algorithms will be rewarded for not causing *any* drops when a small UDP flow starts. > If we want to measure the stability of a protocol when it is facing transient events, No, I wasn't specifically wanting to measure stability. I just think that the sustained back-off is more important than the peak backoff. Maximum is a very fragile statistic. Cheers, Lachlan -- Lachlan Andrew Dept of Computer Science, Caltech 1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA Ph: +1 (626) 395-8820 Fax: +1 (626) 568-3603 http://netlab.caltech.edu/~lachlan From romaric.guillier at ens-lyon.fr Tue Dec 11 00:54:34 2007 From: romaric.guillier at ens-lyon.fr (Romaric Guillier) Date: Tue, 11 Dec 2007 09:54:34 +0100 Subject: [Tmrg] Round-table PFLDnet submission - transient In-Reply-To: References: <20071209215603.v1gsa5smh4e8soc4@tadorne.ens-lyon.fr> <475D043F.6020603@ens-lyon.fr> Message-ID: <475E504A.1020400@ens-lyon.fr> hi Lachlan Andrew wrote: > Greetings Romaric, > > On 10/12/2007, Romaric Guillier wrote: >>> However, the max decrease in one RTT isn't very informative. Should >>> it be big or small? A flow which randomly sends 0 on odd RTTs and >>> 10Gbps on even RTTs will have a very big "maximum decrease", but >>> doesn't behave at all well. >> Yep, I agree with you on that, but if you consider the "time to reduce >> to 33%" metric, you will get a result like one or less than one RTT, but >> it won't tell you either that your protocol is unstable. > > True. I argued against the "time to reduce to x%" metric at the > meeting, but haven't come up with anything better. The reason for > making it 33% not 50% as originally proposed was an attempt to look > beyond the first "reduction by 50%" that many algorithms have. and hope that we won't see new algorithms that will propose a reduction to 33% :) > To me, the "back-off" measure is about how much room is available for > the newly-arriving flow, rather than the impact on the existing flow. > That is one reason I don't like the "cost" in the original draft -- it > only considers the impact on the long-lived flow, not the impact of > that flow's response on other flows. Sorry, distortion of the reality due to the usual topic I'm studying. Of course, you are right, we shouldn't be favouring one part or the other of the problem. But as a transient event could be caused by anything like a routing change causing a sharp decrease of the available bandwidth, sudden congestion or signal power dropping for wireless connexions, I thought it was more interesting to focus on this one flow rather than on the load we generate to simulate the transient event. > Perhaps it would be better to > measure the impact on other flows directly. How about the measure > > number of packets dropped by the UDP sources in the first x seconds > > If the flow backs off nicely, the number of dropped packets will be > small. Similarly, virtual queue algorithms will be rewarded for not > causing *any* drops when a small UDP flow starts. That sounds nice. If the number is too big, the flow is too aggressive and might be dangerous. If the number is too small, well you probably won't get good performance but at least the protocol won't break the internet cheers Romaric From lastewart at swin.edu.au Fri Dec 28 18:22:52 2007 From: lastewart at swin.edu.au (Lawrence Stewart) Date: Sat, 29 Dec 2007 13:22:52 +1100 Subject: [Tmrg] Modular/Pluggable TCP Congestion Control for FreeBSD Message-ID: <4775AF7C.8020407@swin.edu.au> Hi all, We've been involved in a research project to implement and test an emerging TCP congestion control algorithm under FreeBSD. As a part of this, we've put together a patch for FreeBSD 7.0-BETA4 that modularises the congestion control code in the TCP stack. It allows for new congestion control algorithms to be developed as loadable kernel modules. This improves FreeBSD's usefulness as a TCP research platform and makes it easier to customise the stack for specific scenarios like high bandwidth, long delay paths. There is an accompanying technical report "Light-Weight Modular TCP Congestion Control for FreeBSD 7" [1] that covers the design, features, kernel interface and usage of the framework. Also on our website is a beta release of a module that implements the H-TCP[2] congestion control algorithm proposed by the Hamilton Institute. We believe that modular congestion control is a worthwhile addition to FreeBSD. We've performed significant internal testing and there are currently no known issues or regressions with the implementation compared to a 'vanilla' FreeBSD 7.0-BETA4 kernel. We would welcome further review and testing from the wider community in the hope of getting this patch folded into FreeBSD 8-CURRENT. SIFTR [3], our tool for monitoring FreeBSD kernel TCP connection state, has also received a minor update to v1.1.5, with the addition of 6 new, useful variables. All code and documentation is available on our website[3]. Cheers, Jim and Lawrence http://caia.swin.edu.au [1] http://caia.swin.edu.au/reports/071218A/CAIA-TR-071218A.pdf [2] http://www.hamilton.ie/net/htcp3.pdf [3] http://caia.swin.edu.au/urp/newtcp/tools.html From lachlan.andrew at gmail.com Sat Dec 29 13:37:44 2007 From: lachlan.andrew at gmail.com (Lachlan Andrew) Date: Sat, 29 Dec 2007 13:37:44 -0800 Subject: [Tmrg] Metrics for releasing bandwidth Message-ID: Greetings TMRGers, In a transient when new flows start (particularly non-rate-controlled "UDP" sources), how can we measure how quickly a TCP algorithm backs off? One suggestion was measuring how long it takes until it halves its window. Issues with that metric are: - if the UDP flow is less than 50% of the bandwidth, the flow can respond ideally and instantly without ever halving its window. - it ignores how quickly the TCP flow increases its window again after the reduction - it focuses on the experience of the TCP flow, not the impact on the UDP flow - different TCP flows may take different amounts of time to experience their first loss, especially if the new load arrives gradually as in a flash crowd. As an alternative, I propose measuring the number of packets dropped by the UDP flow(s) from when it starts to 10(?) seconds after it reaches its peak rate. This - is meaningful for any rate of UDP cross traffic - captures the entire duration of the transient, not just the start - applies equally well if there are many TCP flows, or the UDP rate increases gradually. Unfortunately, this has a "magic number" of 10s. Can anyone see other flaws, or better metrics? Cheers, Lachlan -- Lachlan Andrew Dept of Computer Science, Caltech 1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA Ph: +1 (626) 395-8820 Fax: +1 (626) 568-3603 http://netlab.caltech.edu/~lachlan