draft-ietf-ippm-tcp-throughput-tm-08.txt   draft-ietf-ippm-tcp-throughput-tm-09.txt 
Network Working Group B. Constantine Network Working Group B. Constantine
Internet-Draft JDSU Internet-Draft JDSU
Intended status: Informational G. Forget Intended status: Informational G. Forget
Expires: May 14, 2011 Bell Canada (Ext. Consultant) Expires: June 7, 2011 Bell Canada (Ext. Consultant)
Rudiger Geib Rudiger Geib
Deutsche Telekom Deutsche Telekom
Reinhard Schrage Reinhard Schrage
Schrage Consulting Schrage Consulting
November 14, 2010 December 7, 2010
Framework for TCP Throughput Testing Framework for TCP Throughput Testing
draft-ietf-ippm-tcp-throughput-tm-08.txt draft-ietf-ippm-tcp-throughput-tm-09.txt
Abstract Abstract
This framework describes a methodology for measuring end-to-end TCP This framework describes a methodology for measuring end-to-end TCP
throughput performance in a managed IP network. The intention is to throughput performance in a managed IP network. The intention is to
provide a practical methodology to validate TCP layer performance. provide a practical methodology to validate TCP layer performance.
The goal is to provide a better indication of the user experience. The goal is to provide a better indication of the user experience.
In this framework, various TCP and IP parameters are identified and In this framework, various TCP and IP parameters are identified and
should be tested as part of a managed IP network verification. should be tested as part of a managed IP network.
Requirements Language Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119]. document are to be interpreted as described in RFC 2119 [RFC2119].
Status of this Memo Status of this Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
skipping to change at page 1, line 46 skipping to change at page 1, line 46
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on May 14, 2011. This Internet-Draft will expire on June 7, 2011.
Copyright Notice Copyright Notice
Copyright (c) 2010 IETF Trust and the persons identified as the Copyright (c) 2010 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1 Test Set-up and Terminology . . . . . . . . . . . . . . . 4 1.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . 4
1.2 Test Set-up . . . . . . . . . . . . . . . . . . . . . . . 4
2. Scope and Goals of this methodology. . . . . . . . . . . . . . 5 2. Scope and Goals of this methodology. . . . . . . . . . . . . . 5
2.1 TCP Equilibrium. . . . . . . . . . . . . . . . . . . . . . 6 2.1 TCP Equilibrium. . . . . . . . . . . . . . . . . . . . . . 6
3. TCP Throughput Testing Methodology . . . . . . . . . . . . . . 7 3. TCP Throughput Testing Methodology . . . . . . . . . . . . . . 7
3.1 Determine Network Path MTU . . . . . . . . . . . . . . . . 9 3.1 Determine Network Path MTU . . . . . . . . . . . . . . . . 9
3.2. Baseline Round Trip Time and Bandwidth . . . . . . . . . . 10 3.2. Baseline Round Trip Time and Bandwidth . . . . . . . . . . 10
3.2.1 Techniques to Measure Round Trip Time . . . . . . . . 10 3.2.1 Techniques to Measure Round Trip Time . . . . . . . . 10
3.2.2 Techniques to Measure end-to-end Bandwidth. . . . . . 11 3.2.2 Techniques to Measure end-to-end Bandwidth. . . . . . 11
3.3. TCP Throughput Tests . . . . . . . . . . . . . . . . . . . 12 3.3. TCP Throughput Tests . . . . . . . . . . . . . . . . . . . 12
3.3.1 Calculate Ideal TCP Receive Window Size. . . . . . . . 12 3.3.1 Calculate Ideal TCP Receive Window Size. . . . . . . . 12
3.3.2 Metrics for TCP Throughput Tests . . . . . . . . . . . 15 3.3.2 Metrics for TCP Throughput Tests . . . . . . . . . . . 15
skipping to change at page 3, line 30 skipping to change at page 3, line 30
Note that the primary focus of this methodology is managed business Note that the primary focus of this methodology is managed business
class IP networks; i.e. those Ethernet terminated services for which class IP networks; i.e. those Ethernet terminated services for which
businesses are provided an SLA from the network provider. End-users businesses are provided an SLA from the network provider. End-users
with "best effort" access between locations can use this methodology, with "best effort" access between locations can use this methodology,
but this framework and its metrics are intended to be used in a but this framework and its metrics are intended to be used in a
predictable managed IP service environment. predictable managed IP service environment.
So the intent behind this document is to define a methodology for So the intent behind this document is to define a methodology for
testing sustained TCP layer performance. In this document, the testing sustained TCP layer performance. In this document, the
maximum achievable TCP throughput is that amount of data per unit maximum achievable TCP Throughput is that amount of data per unit
time that TCP transports when trying to reach Equilibrium, i.e. time that TCP transports when trying to reach Equilibrium, i.e.
after the initial slow start and congestion avoidance phases. We after the initial slow start and congestion avoidance phases.
refer to this as the maximum achievable TCP Throughput for the TCP
connection(s).
TCP uses a congestion window, (TCP CWND), to determine how many TCP uses a congestion window, (TCP CWND), to determine how many
packets it can send at one time. A larger TCP CWND permits a higher packets it can send at one time. The network path bandwidth delay
throughput. TCP "slow start" and "congestion avoidance" algorithms product (BDP) determines the ideal TCP CWND. With the help of slow
together determine the TCP CWND size. The Maximum TCP CWND size is start and congestion avoidance mechanisms, TCP probes the network
also tributary to the buffer space allocated by the kernel for each path. So up to the bandwidth limit, a larger TCP CWND permits a
socket. For each socket, there is a default buffer size that can be higher throughput. And up to local host limits, TCP "Slow Start" and
changed by the program using a system library called just before "Congestion Avoidance" algorithms together will determine the TCP
opening the socket. There is also a kernel enforced maximum buffer CWND size. The Maximum TCP CWND size is also tributary to the buffer
size. This buffer size can be adjusted at both ends of the socket space allocated by the kernel for each socket. For each socket, there
(send and receive). In order to obtain the maximum throughput, it is a default buffer size that can be changed by the program using a
is critical to use optimal TCP Send and Receive Socket Buffer sizes system library called just before opening the socket. There is also
as well as the optimal TCP Receive Window size. a kernel enforced maximum buffer size. This buffer size can be
adjusted at both ends of the socket (send and receive). In order
to obtain the maximum throughput, it is critical to use optimal TCP
Send and Receive Socket Buffer sizes.
There are many variables to consider when conducting a TCP throughput There are many variables to consider when conducting a TCP throughput
test and this methodology focuses on the most common: test, but this methodology focuses on:
- Path MTU and Maximum Segment Size (MSS)
- RTT and Bottleneck BW - RTT and Bottleneck BW
- Ideal TCP Receive Window (including Ideal Receive Socket Buffer) - Ideal TCP Receive Window (Ideal Receive Socket Buffer)
- Ideal Send Socket Buffer - Ideal Send Socket Buffer
- TCP Congestion Window (TCP CWND) - TCP Congestion Window (TCP CWND)
- Path MTU and Maximum Segment Size (MSS)
- Single Connection and Multiple Connections testing - Single Connection and Multiple Connections testing
This methodology proposes TCP testing that should be performed in This methodology proposes TCP testing that should be performed in
addition to traditional Layer 2/3 type tests. Layer 2/3 tests are addition to traditional Layer 2/3 type tests. Layer 2/3 tests are
required to verify the integrity of the network before conducting TCP required to verify the integrity of the network before conducting TCP
test. Examples include iperf (UDP mode) or manual packet layer test tests. Examples include iperf (UDP mode) or manual packet layer test
techniques where packet throughput, loss, and delay measurements are techniques where packet throughput, loss, and delay measurements are
conducted. When available, standardized testing similar to RFC 2544 conducted. When available, standardized testing similar to RFC 2544
[RFC2544] but adapted for use in operational networks may be used. [RFC2544] but adapted for use in operational networks may be used.
Note: RFC 2544 was never meant to be used outside a lab environment. Note: RFC 2544 was never meant to be used outside a lab environment.
1.1 Test Set-up and Terminology The following 2 sections provide a general overview of the test
methodology.
This section provides a general overview of the test configuration 1.1 Terminology
for this methodology. The test is intended to be conducted on an
end-to-end operational and managed IP network. A multitude of
network architectures and topologies can be tested. The following
set-up diagram is very general and it only illustrates the
segmentation within end user and network provider domains.
Common terminologies used in the test methodology are: Common terminologies used in the test methodology are:
- TCP Throughput Test Device (TCP TTD), refers to compliant TCP
host that generates traffic and measures metrics as defined in
this methodology. i.e. a dedicated communications test instrument.
- Customer Provided Equipment (CPE), refers to customer owned
equipment (routers, switches, computers, etc.)
- Customer Edge (CE), refers to provider owned demarcation device.
- Provider Edge (PE), refers to provider's distribution equipment.
- Bottleneck Bandwidth (BB), lowest bandwidth along the complete - Bottleneck Bandwidth (BB), lowest bandwidth along the complete
path. Bottleneck Bandwidth and Bandwidth are used synonymously path. Bottleneck Bandwidth and Bandwidth are used synonymously
in this document. Most of the time the Bottleneck Bandwidth is in this document. Most of the time the Bottleneck Bandwidth is
in the access portion of the wide area network (CE - PE) in the access portion of the wide area network (CE - PE)
- Customer Provided Equipment (CPE), refers to customer owned - Provider (P), refers to provider core network equipment.
equipment (routers, switches, computers, etc.)
- Customer Edge (CE), refers to provider owned demarcation device.
- End-user: The business enterprise customer. For the purposes of
conducting TCP throughput tests, this may be the IT department.
- Network Under Test (NUT), refers to the tested IP network path. - Network Under Test (NUT), refers to the tested IP network path.
- Provider Edge (PE), refers to provider's distribution equipment.
- P (Provider), refers to provider core network equipment.
- Round-Trip Time (RTT), refers to Layer 4 back and forth delay. - Round-Trip Time (RTT), refers to Layer 4 back and forth delay.
- Round-Trip Delay (RTD), refers to Layer 1 back and forth delay.
- TCP Throughput Test Device (TCP TTD), refers to compliant TCP
host that generates traffic and measures metrics as defined in
this methodology. i.e. a dedicated communications test instrument.
+----+ +----+ +----+ +----+ +---+ +---+ +----+ +----+ +----+ +----+ +----+ +----+ +----+ +----+ +---+ +---+ +----+ +----+ +----+ +----+
| TCP|-| CPE|-| CE |--| PE |-| P |--| P |-| PE |--| CE |-| CPE|-| TCP| | TCP|-| CPE|-| CE |--| PE |-| P |--| P |-| PE |--| CE |-| CPE|-| TCP|
| TTD| | | | |BB| | | | | | | |BB| | | | | TTD| | TTD| | | | |BB| | | | | | | |BB| | | | | TTD|
+----+ +----+ +----+ +----+ +---+ +---+ +----+ +----+ +----+ +----+ +----+ +----+ +----+ +----+ +---+ +---+ +----+ +----+ +----+ +----+
<------------------------ NUT ------------------------> <------------------------ NUT ------------------------>
R >-----------------------------------------------------------| R >-----------------------------------------------------------|
T | T |
T <-----------------------------------------------------------| T <-----------------------------------------------------------|
Note that the NUT may consist of a variety of devices including but Note that the NUT may consist of a variety of devices including but
not limited to, load balancers, proxy servers or WAN acceleration not limited to, load balancers, proxy servers or WAN acceleration
devices. The detailed topology of the NUT should be well understood devices. The detailed topology of the NUT should be well understood
when conducting the TCP throughput tests, although this methodology when conducting the TCP throughput tests, although this methodology
makes no attempt to characterize specific network architectures. makes no attempt to characterize specific network architectures.
1.2 Test Set-up
This methodology is intended for operational and managed IP networks.
A multitude of network architectures and topologies can be tested.
The above set-up diagram is very general and it only illustrates the
segmentation within end-user and network provider domains.
2. Scope and Goals of this Methodology 2. Scope and Goals of this Methodology
Before defining the goals, it is important to clearly define the Before defining the goals, it is important to clearly define the
areas that are out-of-scope. areas that are out-of-scope.
- This methodology is not intended to predict the TCP throughput - This methodology is not intended to predict the TCP throughput
during the transient stages of a TCP connection, such as the initial during the transient stages of a TCP connection, such as the initial
slow start. slow start.
- This methodology is not intended to definitively benchmark TCP - This methodology is not intended to definitively benchmark TCP
skipping to change at page 6, line 5 skipping to change at page 5, line 57
See section 3.3.3. See section 3.3.3.
- Provide specific test conditions like link speed, RTT, TCP Receive - Provide specific test conditions like link speed, RTT, TCP Receive
Window size, Socket Buffer size and maximum achievable TCP throughput Window size, Socket Buffer size and maximum achievable TCP throughput
when trying to reach TCP Equilibrium. For guideline purposes, when trying to reach TCP Equilibrium. For guideline purposes,
provide examples of test conditions and their maximum achievable provide examples of test conditions and their maximum achievable
TCP throughput. Section 2.1 provides specific details concerning the TCP throughput. Section 2.1 provides specific details concerning the
definition of TCP Equilibrium within this methodology while section 3 definition of TCP Equilibrium within this methodology while section 3
provides specific test conditions with examples. provides specific test conditions with examples.
Note that some TCP/IP stack implementations are using Receive Window
Auto-Tuning and cannot be adjusted until this feature is disabled.
- Define three (3) basic metrics to compare the performance of TCP - Define three (3) basic metrics to compare the performance of TCP
connections under various network conditions. See section 3.3.2. connections under various network conditions. See section 3.3.2.
- In test situations where the recommended procedure does not yield - In test situations where the recommended procedure does not yield
the maximum achievable TCP throughput results, this methodology the maximum achievable TCP throughput results, this methodology
provides some possible areas within the end host or network that provides some possible areas within the end host or the network that
should be considered for investigation. Although again, this should be considered for investigation. Although again, this
methodology is not intended to provide a detailed diagnosis on these methodology is not intended to provide a detailed diagnosis on these
issues. See section 3.3.5. issues. See section 3.3.5.
2.1 TCP Equilibrium 2.1 TCP Equilibrium
TCP connections have three (3) fundamental congestion window phases TCP connections have three (3) fundamental congestion window phases :
as documented in [RFC5681].
These 3 phases are:
1 - The Slow Start phase, which occurs at the beginning of a TCP 1 - The Slow Start phase, which occurs at the beginning of a TCP
transmission or after a retransmission time out. transmission or after a retransmission time out.
2 - The Congestion Avoidance phase, during which TCP ramps up to 2 - The Congestion Avoidance phase, during which TCP ramps up to
establish the maximum attainable throughput on an end-to-end network establish the maximum attainable throughput on an end-to-end network
path. Retransmissions are a natural by-product of the TCP congestion path. Retransmissions are a natural by-product of the TCP congestion
avoidance algorithm as it seeks to achieve maximum throughput. avoidance algorithm as it seeks to achieve maximum throughput.
3 - The Retransmission Time-out phase, which could include Fast 3 - The Loss Recovery phase, which could include Fast Retransmit
Retransmit (Tahoe) or Fast Recovery (Reno & New Reno). When multiple (Tahoe) or Fast Recovery (Reno & New Reno). When packet loss occurs,
packet lost occurs, Congestion Avoidance phase transitions to Fast Congestion Avoidance phase transitions either to Fast Retransmission
Retransmission or Fast Recovery depending upon TCP implementations. or Fast Recovery depending upon TCP implementations. If a Time-Out
If a Time-Out occurs, TCP transitions back to the Slow Start phase. occurs, TCP transitions back to the Slow Start phase.
The following diagram depicts these 3 phases. The following diagram depicts these 3 phases.
| Trying to reach TCP Equilibrium >>>>>>>>>>>>> /\ | Trying to reach TCP Equilibrium > > > > > > > > >
/\ | High ssthresh TCP CWND 3 /\ |
/\ | Loss Event * halving Retransmission /\ |High ssthresh TCP CWND
/\ | * \ upon loss Time-Out Adjusted /\ |Loss Event * halving 3-Loss Recovery
/\ | * \ /\ _______ ssthresh /\ | * \ upon loss Adjusted
/\ | * \ / \ /M-Loss | * /\ | * \ / \ Time-Out ssthresh
TCP | * 2 \/ \ / Events |1 * /\ | * \ / \ +--------+ *
Through- | * Congestion\ / |Slow * TCP | * \/ \ / Multiple| *
put | 1 * Avoidance \/ |Start * Through- | * 2-Congestion\ / Loss | *
| Slow * Half | * put | * Avoidance \/ Event | *
| Start * TCP CWND * | * Half | *
|___*_______________________Minimum TCP CWND after Time-Out_ | * TCP CWND | * 1-Slow Start
Time >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | * 1-Slow Start Min TCP CWND after T-O
+-----------------------------------------------------------
Time > > > > > > > > > > > > > > >
Note : ssthresh = Slow Start threshold. Note : ssthresh = Slow Start threshold.
Through the above 3 phases, TCP is trying to reach Equilibrium, but A well tuned and managed IP network with appropriate TCP adjustments
since packet loss is currently its only available feedback indicator, in it's IP hosts and applications should perform very close to TCP
TCP will never reach that goal. Although, a well tuned (and managed) Equilibrium and to the BB (Bottleneck Bandwidth).
IP network with well tuned IP hosts and applications should perform
very close to TCP Equilibrium and to the BB (Bottleneck Bandwidth).
This TCP methodology provides guidelines to measure the maximum This TCP methodology provides guidelines to measure the maximum
achievable TCP throughput or maximum TCP sustained rate obtained achievable TCP throughput or maximum TCP sustained rate obtained
after TCP CWND has stabilized to an optimal value. All maximum after TCP CWND has stabilized to an optimal value. All maximum
achievable TCP throughputs specified in section 3 are with respect to achievable TCP throughputs specified in section 3 are with respect to
this condition. this condition.
It is important to clarify the interaction between the sender's Send It is important to clarify the interaction between the sender's Send
Socket Buffer and the receiver's advertised TCP Receive Window. TCP Socket Buffer and the receiver's advertised TCP Receive Window. TCP
test programs such as iperf, ttcp, etc. allow the sender to control test programs such as iperf, ttcp, etc. allow the sender to control
skipping to change at page 7, line 29 skipping to change at page 7, line 29
3. TCP Throughput Testing Methodology 3. TCP Throughput Testing Methodology
As stated earlier in section 1, it is considered best practice to As stated earlier in section 1, it is considered best practice to
verify the integrity of the network by conducting Layer2/3 tests such verify the integrity of the network by conducting Layer2/3 tests such
as [RFC2544] or other methods of network stress tests. Although, it as [RFC2544] or other methods of network stress tests. Although, it
is important to mention here that RFC 2544 was never meant to be used is important to mention here that RFC 2544 was never meant to be used
outside a lab environment. outside a lab environment.
If the network is not performing properly in terms of packet loss, If the network is not performing properly in terms of packet loss,
jitter, etc. then the TCP layer testing will not be meaningful. A jitter, etc. then the TCP layer testing will not be meaningful. A
dysfunctional network will not reach close enough to TCP Equilibrium dysfunctional network will not acheive optimal TCP throughputs in
to provide optimal TCP throughputs with the available bandwidth. regards with the available bandwidth.
TCP Throughput testing may require cooperation between the end user TCP Throughput testing may require cooperation between the end-user
customer and the network provider. In a Layer 2/3 VPN architecture, customer and the network provider. In a Layer 2/3 VPN architecture,
the testing should be conducted either on the CPE or on the CE device the testing should be conducted either on the CPE or on the CE device
and not on the PE (Provider Edge) router. and not on the PE (Provider Edge) router.
The following represents the sequential order of steps for this The following represents the sequential order of steps for this
testing methodology: testing methodology:
1. Identify the Path MTU. Packetization Layer Path MTU Discovery 1. Identify the Path MTU. Packetization Layer Path MTU Discovery
or PLPMTUD, [RFC4821], MUST be conducted to verify the network path or PLPMTUD, [RFC4821], MUST be conducted to verify the network path
MTU. Conducting PLPMTUD establishes the upper limit for the MSS to MTU. Conducting PLPMTUD establishes the upper limit for the MSS to
skipping to change at page 8, line 9 skipping to change at page 8, line 9
3. TCP Connection Throughput Tests. With baseline measurements 3. TCP Connection Throughput Tests. With baseline measurements
of Round Trip Time and bottleneck bandwidth, single and multiple TCP of Round Trip Time and bottleneck bandwidth, single and multiple TCP
connection throughput tests SHOULD be conducted to baseline network connection throughput tests SHOULD be conducted to baseline network
performance expectations. performance expectations.
4. Traffic Management Tests. Various traffic management and queuing 4. Traffic Management Tests. Various traffic management and queuing
techniques can be tested in this step, using multiple TCP techniques can be tested in this step, using multiple TCP
connections. Multiple connections testing should verify that the connections. Multiple connections testing should verify that the
network is configured properly for traffic shaping versus policing, network is configured properly for traffic shaping versus policing,
various queuing implementations and RED. various queuing implementations and Random Early Discards (RED).
Important to note are some of the key characteristics and Important to note are some of the key characteristics and
considerations for the TCP test instrument. The test host may be a considerations for the TCP test instrument. The test host may be a
standard computer or a dedicated communications test instrument. standard computer or a dedicated communications test instrument.
In both cases, they must be capable of emulating both client and In both cases, they must be capable of emulating both client and
server. server.
The following criteria should be considered when selecting whether The following criteria should be considered when selecting whether
the TCP test host can be a standard computer or has to be a dedicated the TCP test host can be a standard computer or has to be a dedicated
communications test instrument: communications test instrument:
- TCP implementation used by the test host, OS version, i.e. Linux OS - TCP implementation used by the test host, OS version, i.e. Linux OS
kernel using TCP Reno, TCP options supported, etc. These will kernel using TCP New Reno, TCP options supported, etc. These will
obviously be more important when using dedicated communications test obviously be more important when using dedicated communications test
instruments where the TCP implementation may be customized or tuned instruments where the TCP implementation may be customized or tuned
to run in higher performance hardware. When a compliant TCP TTD is to run in higher performance hardware. When a compliant TCP TTD is
used, the TCP implementation MUST be identified in the test results. used, the TCP implementation MUST be identified in the test results.
The compliant TCP TTD should be usable for complete end-to-end The compliant TCP TTD should be usable for complete end-to-end
testing through network security elements and should also be usable testing through network security elements and should also be usable
for testing network sections. for testing network sections.
- More important, the TCP test host MUST be capable to generate - More important, the TCP test host MUST be capable to generate
and receive stateful TCP test traffic at the full link speed of the and receive stateful TCP test traffic at the full link speed of the
network under test. Stateful TCP test traffic means that the test network under test. Stateful TCP test traffic means that the test
host MUST fully implement a TCP stack; this is generally a comment host MUST fully implement a TCP/IP stack; this is generally a comment
aimed at dedicated communications test equipments which sometimes aimed at dedicated communications test equipments which sometimes
"blast" packets with TCP headers. As a general rule of thumb, testing "blast" packets with TCP headers. As a general rule of thumb, testing
TCP throughput at rates greater than 100 Mbit/sec MAY require high TCP throughput at rates greater than 100 Mbit/sec MAY require high
performance server hardware or dedicated hardware based test tools. performance server hardware or dedicated hardware based test tools.
- A compliant TCP Throughput Test Device MUST allow adjusting both - A compliant TCP Throughput Test Device MUST allow adjusting both
Send Socket Buffer and TCP Receive Window sizes. The Receive Socket Send and Receive Socket Buffer sizes. The Receive Socket Buffer MUST
Buffer MUST be large enough to accommodate the TCP Receive Window. be large enough to accommodate the TCP Receive Window Size. Note that
some TCP/IP stack implementations are using Receive Window
Auto-Tuning and cannot be adjusted until this feature is disabled.
- Measuring RTT and retransmissions per connection will generally - Measuring RTT and retransmissions per connection will generally
require a dedicated communications test instrument. In the absence of require a dedicated communications test instrument. In the absence of
dedicated hardware based test tools, these measurements may need to dedicated hardware based test tools, these measurements may need to
be conducted with packet capture tools, i.e. conduct TCP throughput be conducted with packet capture tools, i.e. conduct TCP throughput
tests and analyze RTT and retransmission results in packet captures. tests and analyze RTT and retransmission results in packet captures.
Another option may be to use "TCP Extended Statistics MIB" per Another option may be to use "TCP Extended Statistics MIB" per
[RFC4898]. [RFC4898].
- The RFC4821 PLPMTUD test SHOULD be conducted with a dedicated - The RFC4821 PLPMTUD test SHOULD be conducted with a dedicated
skipping to change at page 11, line 21 skipping to change at page 11, line 21
The objective in this section is to list several techniques The objective in this section is to list several techniques
in order of decreasing accuracy. in order of decreasing accuracy.
- Use test equipment on each end of the network, "looping" the - Use test equipment on each end of the network, "looping" the
far-end tester so that a packet stream can be measured back and forth far-end tester so that a packet stream can be measured back and forth
from end-to-end. This RTT measurement may be compatible with delay from end-to-end. This RTT measurement may be compatible with delay
measurement protocols specified in [RFC5357]. measurement protocols specified in [RFC5357].
- Conduct packet captures of TCP test sessions using "iperf" or FTP, - Conduct packet captures of TCP test sessions using "iperf" or FTP,
or other TCP test applications. By running multiple experiments, or other TCP test applications. By running multiple experiments,
packet captures can then be analyzed to estimate RTT based upon the packet captures can then be analyzed to estimate RTT. It is
SYN -> SYN-ACK from the 3 way handshake at the beginning of the TCP important to note that results based upon the SYN -> SYN-ACK at the
sessions. Although, note that Firewalls might slow down 3 way beginning of TCP sessions should be avoided since Firewalls might
handshakes, so it might be useful to compare with measured RTT later slow down 3 way handshakes.
on in the same capture.
- ICMP Pings may also be adequate to provide round trip time - ICMP pings may also be adequate to provide round trip time
estimations. Some limitations with ICMP Ping may include msec estimates, provided that the packet size is factored into the
resolution and whether the network elements are responding to pings estimates (i.e. pings with different packet sizes might be required).
or not. Also, ICMP is often rate-limited and segregated into Some limitations with ICMP Ping may include msec resolution and
different buffer queues, so it is not as reliable and accurate as whether the network elements are responding to pings or not. Also,
in-band measurements. ICMP is often rate-limited and segregated into different buffer
queues and is not as reliable and accurate as in-band measurements.
3.2.2 Techniques to Measure end-to-end Bandwidth 3.2.2 Techniques to Measure end-to-end Bandwidth
There are many well established techniques available to provide There are many well established techniques available to provide
estimated measures of bandwidth over a network. These measurements estimated measures of bandwidth over a network. These measurements
SHOULD be conducted in both directions of the network, especially for SHOULD be conducted in both directions of the network, especially for
access networks, which may be asymmetrical. Measurements SHOULD use access networks, which may be asymmetrical. Measurements SHOULD use
network capacity techniques defined in [RFC5136]. network capacity techniques defined in [RFC5136].
Before any TCP Throughput test can be done, a bandwidth measurement Before any TCP Throughput test can be done, a bandwidth measurement
skipping to change at page 12, line 49 skipping to change at page 12, line 49
Window size in Bytes. For optimal results, the Send Socket Buffer Window size in Bytes. For optimal results, the Send Socket Buffer
size must be adjusted to the same value at the opposite end of the size must be adjusted to the same value at the opposite end of the
network path. network path.
Ideal TCP RWIN = BDP / 8 Ideal TCP RWIN = BDP / 8
An example would be a T3 link with 25 msec RTT. The BDP would equal An example would be a T3 link with 25 msec RTT. The BDP would equal
~1,105,000 bits and the ideal TCP Receive Window would be ~138 ~1,105,000 bits and the ideal TCP Receive Window would be ~138
KBytes. KBytes.
Note that separate calculations are required on asymetrical paths.
An asymetrical path example would be a 90 msec RTT ADSL line with
5Mbps downstream and 640Kbps upstream. The downstream BDP would equal
~450,000 bits while the upstream one would be only ~57,600 bits.
The following table provides some representative network Link Speeds, The following table provides some representative network Link Speeds,
RTT, BDP, and their associated Ideal TCP Receive Window sizes. RTT, BDP, and their associated Ideal TCP Receive Window sizes.
Table 3.3.1: Link Speed, RTT and calculated BDP & TCP Receive Window Table 3.3.1: Link Speed, RTT and calculated BDP & TCP Receive Window
Link Ideal TCP Link Ideal TCP
Speed* RTT BDP Receive Window Speed* RTT BDP Receive Window
(Mbps) (ms) (bits) (KBytes) (Mbps) (ms) (bits) (KBytes)
--------------------------------------------------------------------- ---------------------------------------------------------------------
1.536 20 30,720 3.84 1.536 20 30,720 3.84
1.536 50 76,800 9.60 1.536 50 76,800 9.60
1.536 100 153,600 19.20 1.536 100 153,600 19.20
44.21 10 442,100 55.26 44.210 10 442,100 55.26
44.21 15 663,150 82.89 44.210 15 663,150 82.89
44.21 25 1,105,250 138.16 44.210 25 1,105,250 138.16
100 1 100,000 12.50 100 1 100,000 12.50
100 2 200,000 25.00 100 2 200,000 25.00
100 5 500,000 62.50 100 5 500,000 62.50
1,000 0.1 100,000 12.50 1,000 0.1 100,000 12.50
1,000 0.5 500,000 62.50 1,000 0.5 500,000 62.50
1,000 1 1,000,000 125.00 1,000 1 1,000,000 125.00
10,000 0.05 500,000 62.50 10,000 0.05 500,000 62.50
10,000 0.3 3,000,000 375.00 10,000 0.3 3,000,000 375.00
* Note that link speed is the bottleneck bandwidth for the NUT * Note that link speed is the bottleneck bandwidth for the NUT
The following serial link speeds are used: The following serial link speeds are used:
- T1 = 1.536 Mbits/sec (for a B8ZS line encoding facility) - T1 = 1.536 Mbits/sec (for a B8ZS line encoding facility)
- T3 = 44.21 Mbits/sec (for a C-Bit Framing facility) - T3 = 44.21 Mbits/sec (for a C-Bit Framing facility)
The above table illustrates the ideal TCP Receive Window size. The above table illustrates the ideal TCP Receive Window size.
If a smaller TCP Receive Window is used, then the TCP Throughput If a smaller TCP Receive Window is used, then the TCP Throughput
is not optimal. To calculate the Ideal TCP Throughput, the following is not optimal. To calculate the TCP Throughput, the following
formula is used: TCP Throughput = TCP RWIN X 8 / RTT formula is used: TCP Throughput = TCP RWIN X 8 / RTT
An example could be a 100 Mbps IP path with 5 ms RTT and a TCP An example could be a 100 Mbps IP path with 5 ms RTT and a TCP
Receive Window size of 16KB, then: Receive Window size of 16KB, then:
TCP Throughput = 16 KBytes X 8 bits / 5 ms. TCP Throughput = 16 KBytes X 8 bits / 5 ms.
TCP Throughput = 128,000 bits / 0.005 sec. TCP Throughput = 128,000 bits / 0.005 sec.
TCP Throughput = 25.6 Mbps. TCP Throughput = 25.6 Mbps.
Another example for a T3 using the same calculation formula is Another example for a T3 using the same calculation formula is
skipping to change at page 15, line 5 skipping to change at page 14, line 54
| | | | | | | | | |
10| +-----+10.2M | | | | 10| +-----+10.2M | | | |
| | | | | | | | | | | | | |
5| +-----+5.1M | | | | | | 5| +-----+5.1M | | | | | |
|_____|_____|______|_____|______|_____|_______|_____|_____ |_____|_____|______|_____|______|_____|_______|_____|_____
16 32 64 128* 16 32 64 128*
TCP Receive Window size in KBytes TCP Receive Window size in KBytes
* Note that 128KB requires [RFC1323] TCP Window scaling option. * Note that 128KB requires [RFC1323] TCP Window scaling option.
Note that some TCP/IP stack implementations are using Receive Window
Auto-Tuning and cannot be adjusted until the feature is disabled.
3.3.2 Metrics for TCP Throughput Tests 3.3.2 Metrics for TCP Throughput Tests
This framework focuses on a TCP throughput methodology and also This framework focuses on a TCP throughput methodology and also
provides several basic metrics to compare results of various provides several basic metrics to compare results of various
throughput tests. It is recognized that the complexity and throughput tests. It is recognized that the complexity and
unpredictability of TCP makes it impossible to develop a complete unpredictability of TCP makes it impossible to develop a complete
set of metrics that accounts for the myriad of variables (i.e. RTT set of metrics that accounts for the myriad of variables (i.e. RTT
variation, loss conditions, TCP implementation, etc.). However, variation, loss conditions, TCP implementation, etc.). However,
these basic metrics will facilitate TCP throughput comparisons these basic metrics will facilitate TCP throughput comparisons
under varying network conditions and between network traffic under varying network conditions and between network traffic
skipping to change at page 15, line 44 skipping to change at page 15, line 44
the Send Socket Buffer sizes must be tuned to equal the bandwidth the Send Socket Buffer sizes must be tuned to equal the bandwidth
delay product (BDP) as described in section 3.3.1. delay product (BDP) as described in section 3.3.1.
The following table illustrates the Ideal TCP Transfer time of a The following table illustrates the Ideal TCP Transfer time of a
single TCP connection when its TCP Receive Window and Send Socket single TCP connection when its TCP Receive Window and Send Socket
Buffer sizes are equal to the BDP. Buffer sizes are equal to the BDP.
Table 3.3.2: Link Speed, RTT, BDP, TCP Throughput, and Table 3.3.2: Link Speed, RTT, BDP, TCP Throughput, and
Ideal TCP Transfer time for a 100 MB File Ideal TCP Transfer time for a 100 MB File
Link Maximum Ideal TCP Link Maximum Ideal TCP
Speed BDP Achievable TCP Transfer time Speed BDP Achievable TCP Transfer time
(Mbps) RTT (ms) (KBytes) Throughput(Mbps) (seconds) (Mbps) RTT (ms) (KBytes) Throughput(Mbps) (seconds)
-------------------------------------------------------------------- --------------------------------------------------------------------
1.536 50 9.6 1.4 571 1.536 50 9.6 1.4 571
44.21 25 138.2 42.8 18 44.21 25 138.2 42.8 18
100 2 25.0 94.9 9 100 2 25.0 94.9 9
1,000 1 125.0 949.2 1 1,000 1 125.0 949.2 1
10,000 0.05 62.5 9,492 0.1 10,000 0.05 62.5 9,492 0.1
Transfer times are rounded for simplicity. Transfer times are rounded for simplicity.
For a 100MB file(100 x 8 = 800 Mbits), the Ideal TCP Transfer Time For a 100MB file(100 x 8 = 800 Mbits), the Ideal TCP Transfer Time
is derived as follows: is derived as follows:
800 Mbits 800 Mbits
Ideal TCP Transfer Time = ----------------------------------- Ideal TCP Transfer Time = -----------------------------------
Maximum Achievable TCP Throughput Maximum Achievable TCP Throughput
skipping to change at page 18, line 45 skipping to change at page 18, line 45
the most common is "iperf". With this tool, hosts are installed at the most common is "iperf". With this tool, hosts are installed at
each end of the network path; one acts as client and the other as each end of the network path; one acts as client and the other as
a server. The Send Socket Buffer and the TCP Receive Window sizes a server. The Send Socket Buffer and the TCP Receive Window sizes
of both client and server can be manually set. The achieved of both client and server can be manually set. The achieved
throughput can then be measured, either uni-directionally or throughput can then be measured, either uni-directionally or
bi-directionally. For higher BDP situations in lossy networks bi-directionally. For higher BDP situations in lossy networks
(long fat networks or satellite links, etc.), TCP options such as (long fat networks or satellite links, etc.), TCP options such as
Selective Acknowledgment SHOULD be considered and become part of Selective Acknowledgment SHOULD be considered and become part of
the window size / throughput characterization. the window size / throughput characterization.
Note that some TCP/IP stack implementations are using Receive Window
Auto-Tuning and cannot be adjusted until this feature is disabled.
Host hardware performance must be well understood before conducting Host hardware performance must be well understood before conducting
the tests described in the following sections. A dedicated the tests described in the following sections. A dedicated
communications test instrument will generally be required, especially communications test instrument will generally be required, especially
for line rates of GigE and 10 GigE. A compliant TCP TTD SHOULD for line rates of GigE and 10 GigE. A compliant TCP TTD SHOULD
provide a warning message when the expected test throughput will provide a warning message when the expected test throughput will
exceed 10% of the network bandwidth capacity. If the throughput test exceed 10% of the network bandwidth capacity. If the throughput test
is expected to exceed 10% of the provider bandwidth, then the test is expected to exceed 10% of the provider bandwidth, then the test
should be coordinated with the network provider. This does not should be coordinated with the network provider. This does not
include the customer premise bandwidth, the 10% refers directly to include the customer premise bandwidth, the 10% refers directly to
the provider's bandwidth (Provider Edge to Provider router). the provider's bandwidth (Provider Edge to Provider router).
skipping to change at page 19, line 34 skipping to change at page 19, line 34
the BDP equates to 312.5 KBytes. the BDP equates to 312.5 KBytes.
TCP Number of TCP Connections TCP Number of TCP Connections
Window to fill available bandwidth Window to fill available bandwidth
------------------------------------- -------------------------------------
16KB 20 16KB 20
32KB 10 32KB 10
64KB 5 64KB 5
128KB 3 128KB 3
Note that some TCP/IP stack implementations are using Receive Window
Auto-Tuning and cannot be adjusted until this feature is disabled.
The TCP Transfer Time metric is useful for conducting multiple The TCP Transfer Time metric is useful for conducting multiple
connection tests. Each connection should be configured to transfer connection tests. Each connection should be configured to transfer
payloads of the same size (i.e. 100 MB), and the TCP Transfer time payloads of the same size (i.e. 100 MB), and the TCP Transfer time
should provide a simple metric to verify the actual versus expected should provide a simple metric to verify the actual versus expected
results. results.
Note that the TCP transfer time is the time for all connections to Note that the TCP transfer time is the time for all connections to
complete the transfer of the configured payload size. From the complete the transfer of the configured payload size. From the
previous table, the 64KB window is considered. Each of the 5 previous table, the 64KB window is considered. Each of the 5
TCP connections would be configured to transfer 100MB, and each one TCP connections would be configured to transfer 100MB, and each one
skipping to change at page 21, line 30 skipping to change at page 21, line 30
be referred to as the "bottleneck bandwidth". be referred to as the "bottleneck bandwidth".
The ability to detect proper traffic shaping is more easily diagnosed The ability to detect proper traffic shaping is more easily diagnosed
when conducting a multiple TCP connections test. Proper shaping will when conducting a multiple TCP connections test. Proper shaping will
provide a fair distribution of the available bottleneck bandwidth, provide a fair distribution of the available bottleneck bandwidth,
while traffic policing will not. while traffic policing will not.
The traffic shaping tests are built upon the concepts of multiple The traffic shaping tests are built upon the concepts of multiple
connections testing as defined in section 3.3.3. Calculating the BDP connections testing as defined in section 3.3.3. Calculating the BDP
for the bottleneck bandwidth is first required before selecting the for the bottleneck bandwidth is first required before selecting the
number of connections and Send Buffer and TCP Receive Window sizes number of connections, the Send Socket Buffer and TCP Receive Window
per connection. sizes per connection.
Similar to the example in section 3.3, a typical test scenario might Similar to the example in section 3.3, a typical test scenario might
be: GigE LAN with a 100Mbps bottleneck bandwidth (rate limited be: GigE LAN with a 100Mbps bottleneck bandwidth (rate limited
logical interface), and 5 msec RTT. This would require five (5) TCP logical interface), and 5 msec RTT. This would require five (5) TCP
connections of 64 KB Send Socket Buffer and TCP Receive Window sizes connections of 64 KB Send Socket Buffer and TCP Receive Window sizes
to evenly fill the bottleneck bandwidth (~100 Mbps per connection). to evenly fill the bottleneck bandwidth (~100 Mbps per connection).
The traffic shaping test should be run over a long enough duration to The traffic shaping test should be run over a long enough duration to
properly exercise network buffers (greater than 30 seconds) and also properly exercise network buffers (greater than 30 seconds) and also
characterize performance during different time periods of the day. characterize performance during different time periods of the day.
skipping to change at page 24, line 16 skipping to change at page 24, line 16
7.1 Normative References 7.1 Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997. Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M. [RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M.
Zekauskas, "A One-way Active Measurement Protocol Zekauskas, "A One-way Active Measurement Protocol
(OWAMP)", RFC 4656, September 2006. (OWAMP)", RFC 4656, September 2006.
[RFC5681] Allman, M., Paxson, V., Stevens W., "TCP Congestion
Control", RFC 5681, September 2009.
[RFC2544] Bradner, S., McQuaid, J., "Benchmarking Methodology for [RFC2544] Bradner, S., McQuaid, J., "Benchmarking Methodology for
Network Interconnect Devices", RFC 2544, June 1999 Network Interconnect Devices", RFC 2544, June 1999
[RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K., Babiarz, [RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K., Babiarz,
J., "A Two-Way Active Measurement Protocol (TWAMP)", J., "A Two-Way Active Measurement Protocol (TWAMP)",
RFC 5357, October 2008 RFC 5357, October 2008
[RFC4821] Mathis, M., Heffner, J., "Packetization Layer Path MTU [RFC4821] Mathis, M., Heffner, J., "Packetization Layer Path MTU
Discovery", RFC 4821, June 2007 Discovery", RFC 4821, June 2007
 End of changes. 47 change blocks. 
121 lines changed or deleted 138 lines changed or added

This html diff was produced by rfcdiff 1.40. The latest version is available from http://tools.ietf.org/tools/rfcdiff/