[Tmrg] TCP evaluation suite round-table

[Lachlan Andrew]

Greetings Jana,

On 05/09/07, Janardhan Iyengar <iyengar at mail.eecis.udel.edu> wrote:
>
> I second Doug's point about the endpoint becoming the bottleneck in experiments. We recently did some work trying to saturate 2 GigE links using 3.2 GHz Pentium-4 processors (hyperthreading OFF) with jumbograms, and we recognized two bottlenecks that were very close:
> 1/ approaching CPU capacity at ends
> 2/ motherboard backplane capacity

Yes, there certainly problems getting above 1 Gbps.  (More below for
those who are intersted in Linux.)

> (there was also something about the PCI/PCI-express bus limits that I cannot quite remember...)

PCI-express should have no problem.  PCI-X can handle up to 7Gbit/s if
it is working properly, although I have some buggy cards which limit
it to 5Gbit/s.  I'm not sure about regular PCI.

> testing with bandwidths beyond 1000Mbit/s may not be feasible in some cases.

True.  However we can specify a wider range of tests than can
currently be performed, so that
(a) simulations can be consistent
(b) when people *do* get fast enough systems, they can be consistent
with those simulations.

The main question I was have is not about hardware testing at >1Gbps,
but whether we can make the more modest move of replacing the
non-standard 400Mbit/s (used for Dummynet) by  622Mbit/s  so that it
can be compared with OC12 hardware.


For people interested in Linux:

Prompted by Doug's and Lars's comments, I've also just been doing some
experiments, and had a case where it took about 4 minutes to do a
__release_sock  because of SACK processing (mainly
tcp_sacktag_write_queue).  I didn't check the CPU at the time,
unfortunately, but I assume it was high....  Doug suggested looking at
 /proc/net/softnet_stat  to look for packet loss at the driver level,
but I didn't observe any -- perhaps ACK clocking was working as it
should!

I'm currently looking to see if some of this can be helped by dropping
SACKs when the backlog is too high.  Would that just drop fast-path
SACKs?

Cheers,
Lachlan

-- 
Lachlan Andrew  Dept of Computer Science, Caltech
1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA
Phone: +1 (626) 395-8820    Fax: +1 (626) 568-3603