draft-ietf-ippm-tcp-throughput-tm-04.txt   draft-ietf-ippm-tcp-throughput-tm-05.txt 
Network Working Group B. Constantine Network Working Group B. Constantine
Internet-Draft JDSU Internet-Draft JDSU
Intended status: Informational G. Forget Intended status: Informational G. Forget
Expires: January 9, 2011 Bell Canada (Ext. Consultant) Expires: February 12, 2011 Bell Canada (Ext. Consultant)
L. Jorgenson L. Jorgenson
nooCore nooCore
Reinhard Schrage Reinhard Schrage
Schrage Consulting Schrage Consulting
July 9, 2010 August 12, 2010
TCP Throughput Testing Methodology TCP Throughput Testing Methodology
draft-ietf-ippm-tcp-throughput-tm-04.txt draft-ietf-ippm-tcp-throughput-tm-05.txt
Abstract Abstract
This memo describes a methodology for measuring sustained TCP This memo describes a methodology for measuring sustained TCP
throughput performance in an end-to-end managed network environment. throughput performance in an end-to-end managed network environment.
This memo is intended to provide a practical approach to help users This memo is intended to provide a practical approach to help users
validate the TCP layer performance of a managed network, which should validate the TCP layer performance of a managed network, which should
provide a better indication of end-user application level experience. provide a better indication of end-user application level experience.
In the methodology, various TCP and network parameters are identified In the methodology, various TCP and network parameters are identified
that should be tested as part of the network verification at the TCP that should be tested as part of the network verification at the TCP
layer. layer.
Status of this Memo Status of this Memo
This Internet-Draft is submitted to IETF in full conformance with the This Internet-Draft is submitted to IETF in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet- other groups may also distribute working documents as Internet-
Drafts. Creation date July 9, 2010. Drafts. Creation date August 12, 2010.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt. http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html. http://www.ietf.org/shadow.html.
This Internet-Draft will expire on January 9, 2011. This Internet-Draft will expire on February 12, 2011.
Copyright Notice Copyright Notice
Copyright (c) 2010 IETF Trust and the persons identified as the Copyright (c) 2010 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 26 skipping to change at page 2, line 26
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the BSD License. described in the BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Goals of this Methodology. . . . . . . . . . . . . . . . . . . 4 2. Goals of this Methodology. . . . . . . . . . . . . . . . . . . 4
2.1 TCP Equilibrium State Throughput . . . . . . . . . . . . . 5 2.1 TCP Equilibrium State Throughput . . . . . . . . . . . . . 5
2.2 Metrics for TCP Throughput Tests . . . . . . . . . . . . . 6 2.2 Metrics for TCP Throughput Tests . . . . . . . . . . . . . 6
3. TCP Throughput Testing Methodology . . . . . . . . . . . . . . 6 3. TCP Throughput Testing Methodology . . . . . . . . . . . . . . 7
3.1 Determine Network Path MTU . . . . . . . . . . . . . . . . 8 3.1 Determine Network Path MTU . . . . . . . . . . . . . . . . 8
3.2. Baseline Round-trip Delay and Bandwidth. . . . . . . . . . 9 3.2. Baseline Round-trip Delay and Bandwidth. . . . . . . . . . 10
3.2.1 Techniques to Measure Round Trip Time . . . . . . . . 9 3.2.1 Techniques to Measure Round Trip Time . . . . . . . . 10
3.2.2 Techniques to Measure End-end Bandwidth . . . . . . . 10 3.2.2 Techniques to Measure End-end Bandwidth . . . . . . . 11
3.3. TCP Throughput Tests . . . . . . . . . . . . . . . . . . . 10 3.3. TCP Throughput Tests . . . . . . . . . . . . . . . . . . . 11
3.3.1 Calculate Optimum TCP Window Size. . . . . . . . . . . 11 3.3.1 Calculate Optimum TCP Window Size. . . . . . . . . . . 12
3.3.2 Conducting the TCP Throughput Tests. . . . . . . . . . 14 3.3.2 Conducting the TCP Throughput Tests. . . . . . . . . . 14
3.3.3 Single vs. Multiple TCP Connection Testing . . . . . . 14 3.3.3 Single vs. Multiple TCP Connection Testing . . . . . . 15
3.3.4 Interpretation of the TCP Throughput Results . . . . . 15 3.3.4 Interpretation of the TCP Throughput Results . . . . . 16
3.4. Traffic Management Tests . . . . . . . . . . . . . . . . . 15 3.4. Traffic Management Tests . . . . . . . . . . . . . . . . . 16
3.4.1 Traffic Shaping Tests. . . . . . . . . . . . . . . . . 16 3.4.1 Traffic Shaping Tests. . . . . . . . . . . . . . . . . 16
3.4.1.1 Interpretation of Traffic Shaping Test Results. . . 17 3.4.1.1 Interpretation of Traffic Shaping Test Results. . . 17
3.4.2 RED Tests. . . . . . . . . . . . . . . . . . . . . . . 17 3.4.2 RED Tests. . . . . . . . . . . . . . . . . . . . . . . 18
3.4.2.1 Interpretation of RED Results . . . . . . . . . . . 18 3.4.2.1 Interpretation of RED Results . . . . . . . . . . . 18
4. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 18 4. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 19
5. References . . . . . . . . . . . . . . . . . . . . . . . . . . 19 5. References . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 20 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 20
1. Introduction 1. Introduction
Even though RFC2544 was meant to benchmark network equipment and Testing an operational network prior to customer activation is referred
used by network equipment manufacturers (NEMs), network providers to as "turn-up" testing and the SLA is generally Layer 2/3 packet
have used it to benchmark operational networks in order to throughput, delay, loss and jitter.
verify SLAs (Service Level Agreements) before turning on a service
to their business customers. Testing an operational network prior to
customer activation is referred to as "turn-up" testing and the SLA
is generally Layer 2/3 packet throughput, delay, loss and
jitter.
Network providers are coming to the realization that Layer 2/3 testing Network providers are coming to the realization that Layer 2/3 testing
and TCP layer testing are required to more adequately ensure end-user and TCP layer testing are required to more adequately ensure end-user
satisfaction. Therefore, the network provider community desires to satisfaction. Therefore, the network provider community desires to
measure network throughput performance at the TCP layer. Measuring measure network throughput performance at the TCP layer. Measuring
TCP throughput provides a meaningful measure with respect to the end TCP throughput provides a meaningful measure with respect to the end
user's application SLA (and ultimately reach some level of TCP user's application SLA (and ultimately reach some level of TCP
testing interoperability which does not exist today). testing interoperability which does not exist today).
Additionally, end-users (business enterprises) seek to conduct Additionally, end-users (business enterprises) seek to conduct
repeatable TCP throughput tests between enterprise locations. Since repeatable TCP throughput tests between enterprise locations. Since
these enterprises rely on the networks of the providers, a common test these enterprises rely on the networks of the providers, a common test
methodology (and metrics) would be equally beneficial to both parties. methodology (and metrics) would be equally beneficial to both parties.
So the intent behind this draft TCP throughput work is to define So the intent behind this TCP throughput draft is to define
a methodology for testing sustained TCP layer performance. In this a methodology for testing sustained TCP layer performance. In this
document, sustained TCP throughput is that amount of data per unit document, sustained TCP throughput is that amount of data per unit
time that TCP transports during equilibrium (steady state), i.e. time that TCP transports during equilibrium (steady state), i.e.
after the initial slow start phase. We refer to this state as TCP after the initial slow start phase. We refer to this state as TCP
Equilibrium, and that the equalibrium throughput is the maximum Equilibrium, and that the equalibrium throughput is the maximum
achievable for the TCP connection(s). achievable for the TCP connection(s).
One other important note; the precursor to conducting the TCP tests There are many variables to consider when conducting a TCP throughput
test methodlogy is to perform "network stress tests" such as RFC2544 test and this methodology focuses on some of the most common
Layer 2/3 tests or other conventional tests. Examples include parameters that should be considered such as:
OWAMP or manual packet layer test techniques where packet throughput,
loss, and delay measurements are conducted. It is highly recommended - Path MTU and Maximum Segment Size (MSS)
to run traditional Layer 2/3 type test to verify the integrity of the - RTT and Bottleneck BW
network before conducting TCP tests. - Ideal TCP Window (Bandwidth Delay Product)
- Single Connection and Multiple Connection testing
One other important note, it is highly recommended that traditional
Layer 2/3 type tests are conducted to verify the integrity of the
network before conducting TCP tests. Examples include RFC2544, iperf
(UDP mode), or manual packet layer test techniques where packet
throughput, loss, and delay measurements are conducted.
2. Goals of this Methodology 2. Goals of this Methodology
Before defining the goals of this methodology, it is important to Before defining the goals of this methodology, it is important to
clearly define the areas that are not intended to be measured or clearly define the areas that are not intended to be measured or
analyzed by such a methodology. analyzed by such a methodology.
- The methodology is not intended to predict TCP throughput - The methodology is not intended to predict TCP throughput
behavior during the transient stages of a TCP connection, such behavior during the transient stages of a TCP connection, such
as initial slow start. as initial slow start.
skipping to change at page 6, line 27 skipping to change at page 6, line 27
retransmitted and is defined as: retransmitted and is defined as:
Transmitted Bytes - Retransmitted Bytes Transmitted Bytes - Retransmitted Bytes
--------------------------------------- x 100 --------------------------------------- x 100
Transmitted Bytes Transmitted Bytes
This metric provides a comparative measure between various QoS This metric provides a comparative measure between various QoS
mechanisms such as traffic management, congestion avoidance, and also mechanisms such as traffic management, congestion avoidance, and also
various TCP implementations (i.e. Reno, Vegas, etc.). various TCP implementations (i.e. Reno, Vegas, etc.).
As an example, if 1000 TCP segments were sent and 20 had to be As an example, if 100,000 bytes were sent and 2,000 had to be
retransmitted, the TCP Efficiency would be calculated as: retransmitted, the TCP Efficiency would be calculated as:
1000 - 20 100,000 - 2,000
--------- x 100 = 98% ---------------- x 100 = 98%
1000 100,000
Note that the retranmitted bytes may have occurred more than once,
and these multiple retransmissions are added to the bytes retransmitted
count.
The second metric is the TCP Transfer Time, which is simply the time The second metric is the TCP Transfer Time, which is simply the time
it takes to transfer a block of data across simultaneous TCP it takes to transfer a block of data across simultaneous TCP
connections. The concept is useful when benchmarking traffic connections. The concept is useful when benchmarking traffic
management techniques, where multiple connections are generally management techniques, where multiple connections are generally
required. An example would be the bulk transfer of 10 MB upon 8 required.
separate TCP connections (each connection uploading 10 MB). Each
connection may achieve different throughputs during a test and the
overall throughput rate is not always easy to determine (especially as
the number of connections increases). But by defining the TCP Transfer
Time as the total transfer time of 10MB over all 8 connections, the
single transfer time metric is a useful means to compare various
traffic management techniques (i.e. FiFO, WFQ queuing, WRED, etc.).
3. TCP Throughput Testing Methodology The TCP Transfer time can also be used to provide a normalized ratio of
the actual TCP Transfer Time versus ideal Transfer Time. This ratio
is called the TCP Transfer Index and is defined as:
This section summarizes the specific test methodology to achieve the Actual TCP Transfer Time
goals listed in Section 2. -------------------------
Ideal TCP Transfer Time
An example would be the bulk transfer of 100 MB upon 5 simultaneous TCP
connections over a 500 Mbit/s Ethernet service (each connection
uploading 100 MB). Each connection may achieve different throughputs
during a test and the overall throughput rate is not always easy to
determine (especially as the number of connections increases).
The ideal TCP Transfer Time would be ~8 seconds, but in this example,
the actual TCP Transfer Time was 12 seconds. The TCP Transfer Index
would be 12/8 = 1.5, which indicates that the transfer across all
connections took 1.5 times longer than the ideal.
Note that both the TCP Efficiency and TCP Transfer Time metrics must be
measured during each throughput test. The correlation of TCP Transfer
Time with TCP Efficiency can help to diagnose whether the TCP Transfer
Time was negatively impacted by retransmissions (poor TCP Efficiency).
3. TCP Throughput Testing Methodology
As stated in Section 1, it is considered best practice to verify As stated in Section 1, it is considered best practice to verify
the integrity of the network by conducting Layer2/3 stress tests the integrity of the network by conducting Layer2/3 stress tests
such as RFC2544 (or other methods of network stress tests). If the such as RFC2544 (or other methods of network stress tests). If the
network is not performing properly in terms of packet loss, jitter, network is not performing properly in terms of packet loss, jitter,
etc. then the TCP layer testing will not be meaningful since the etc. then the TCP layer testing will not be meaningful since the
equalibrium throughput would be very difficult to achieve (in a equalibrium throughput would be very difficult to achieve (in a
"dysfunctional" network). "dysfunctional" network).
The following represents the sequential order of steps to conduct the The following represents the sequential order of steps to conduct the
skipping to change at page 8, line 4 skipping to change at page 8, line 20
Whether the TCP test host is a standard computer or dedicated test Whether the TCP test host is a standard computer or dedicated test
instrument, the following areas should be considered when selecting instrument, the following areas should be considered when selecting
a test host: a test host:
- TCP implementation used by the test host OS, i.e. Linux OS kernel - TCP implementation used by the test host OS, i.e. Linux OS kernel
using TCP Reno, TCP options supported, etc. This will obviously be using TCP Reno, TCP options supported, etc. This will obviously be
more important when using custom test equipment where the TCP more important when using custom test equipment where the TCP
implementation may be customized or tuned to run in higher implementation may be customized or tuned to run in higher
performance hardware performance hardware
- Most importantly, the TCP test host must be capable of generating - Most importantly, the TCP test host must be capable of generating
and receiving stateful TCP test traffic at the full link speed of the and receiving stateful TCP test traffic at the full link speed of the
network under test. As a general rule of thumb, testing TCP throughput network under test. As a general rule of thumb, testing TCP throughput
at rates greater than 100 Mbit/sec generally requires high at rates greater than 100 Mbit/sec generally requires high
performance server hardware or dedicated hardware based test tools. performance server hardware or dedicated hardware based test tools.
- To measure RTT and TCP Efficiency per connection, this will generally
require dedicated hardware based test tools. In the absence of
dedciated hardware based test tools, these measurements may need to be
conducted with packet capture tools (conduct TCP throughput tests and
analyze RTT and retransmission results with packet captures).
3.1. Determine Network Path MTU 3.1. Determine Network Path MTU
TCP implementations should use Path MTU Discovery techniques (PMTUD). TCP implementations should use Path MTU Discovery techniques (PMTUD).
PMTUD relies on ICMP 'need to frag' messages to learn the path MTU. PMTUD relies on ICMP 'need to frag' messages to learn the path MTU.
When a device has a packet to send which has the Don't Fragment (DF) When a device has a packet to send which has the Don't Fragment (DF)
bit in the IP header set and the packet is larger than the Maximum bit in the IP header set and the packet is larger than the Maximum
Transmission Unit (MTU) of the next hop link, the packet is dropped Transmission Unit (MTU) of the next hop link, the packet is dropped
and the device sends an ICMP 'need to frag' message back to the host and the device sends an ICMP 'need to frag' message back to the host
that originated the packet. The ICMP 'need to frag' message includes that originated the packet. The ICMP 'need to frag' message includes
the next hop MTU which PMTUD uses to tune the TCP Maximum Segment the next hop MTU which PMTUD uses to tune the TCP Maximum Segment
skipping to change at page 9, line 35 skipping to change at page 10, line 11
half between the unsuccessful search_high and successful search_low half between the unsuccessful search_high and successful search_low
value, and increase by increments of 1/2 when seeking the upper value, and increase by increments of 1/2 when seeking the upper
limit. limit.
3.2. Baseline Round-trip Delay and Bandwidth 3.2. Baseline Round-trip Delay and Bandwidth
Before stateful TCP testing can begin, it is important to baseline Before stateful TCP testing can begin, it is important to baseline
the round trip delay and bandwidth of the network to be tested. the round trip delay and bandwidth of the network to be tested.
These measurements provide estimates of the ideal TCP window size, These measurements provide estimates of the ideal TCP window size,
which will be used in subsequent test steps. These latency and which will be used in subsequent test steps. These latency and
bandwidth tests should be run over a long enough period of time to bandwidth tests should be run during the time of day for which
characterize the performance of the network over the course of a the TCP throughput tests will occur.
meaningful time period.
One example would be to take samples during various times of the work
day. The goal would be to determine a representative minimum, average,
and maximum RTD and bandwidth for the network under test. Topology
changes are to be avoided during this time of initial convergence
(e.g. in crossing BGP4 boundaries).
In some cases, baselining bandwidth may not be required, since a The baseline RTT is used to predict the bandwidth delay product and
network provider's end-to-end topology may be well enough defined. the TCP Transfer Time for the subsequent throughput tests. Since this
methodology requires that RTT be measured during the entire throughput
test, the extent by which the RTT varied during the throughput test can
be quantified.
3.2.1 Techniques to Measure Round Trip Time 3.2.1 Techniques to Measure Round Trip Time
Following the definitions used in the references of the appendix; Following the definitions used in the references of the appendix;
Round Trip Time (RTT) is the time elapsed between the clocking in of Round Trip Time (RTT) is the time elapsed between the clocking in of
the first bit of a payload packet to the receipt of the last bit of the the first bit of a payload packet to the receipt of the last bit of the
corresponding acknowledgement. Round Trip Delay (RTD) is used corresponding acknowledgement. Round Trip Delay (RTD) is used
synonymously to twice the Link Latency. synonymously to twice the Link Latency.
In any method used to baseline round trip delay between network In any method used to baseline round trip delay between network
skipping to change at page 11, line 27 skipping to change at page 11, line 47
rate achieved by a TCP connection within the congestion avoidance rate achieved by a TCP connection within the congestion avoidance
phase on a end-end network path. This section and others will define phase on a end-end network path. This section and others will define
the method to conduct these sustained throughput tests and guidelines the method to conduct these sustained throughput tests and guidelines
of the predicted results. of the predicted results.
With baseline measurements of round trip time and bandwidth With baseline measurements of round trip time and bandwidth
from section 3.2, a series of single and multiple TCP connection from section 3.2, a series of single and multiple TCP connection
throughput tests can be conducted to baseline network performance throughput tests can be conducted to baseline network performance
against expectations. against expectations.
It is recommended to run the tests in each direction independently
first, then run both directions simultaneously. In each case, the TCP
Efficiency and TCP Transfer Time metrics must be measured in each
direction.
3.3.1 Calculate Optimum TCP Window Size 3.3.1 Calculate Optimum TCP Window Size
The optimum TCP window size can be calculated from the bandwidth delay The optimum TCP window size can be calculated from the bandwidth delay
product (BDP), which is: product (BDP), which is:
BDP (bits) = RTT (sec) x Bandwidth (bps) BDP (bits) = RTT (sec) x Bandwidth (bps)
By dividing the BDP by 8, the "ideal" TCP window size is calculated. By dividing the BDP by 8, the "ideal" TCP window size is calculated.
An example would be a T3 link with 25 msec RTT. The BDP would equal An example would be a T3 link with 25 msec RTT. The BDP would equal
~1,105,000 bits and the ideal TCP window would equal ~138,000 bytes. ~1,105,000 bits and the ideal TCP window would equal ~138,000 bytes.
skipping to change at page 18, line 47 skipping to change at page 19, line 28
(generally 20-40%) and the TCP Transfer time would be proportionally (generally 20-40%) and the TCP Transfer time would be proportionally
longer then the ideal transfer time. longer then the ideal transfer time.
Additionally, the TCP Transfer Efficiency metric is useful, since Additionally, the TCP Transfer Efficiency metric is useful, since
non-RED implementations will exhibit a lower TCP Tranfer Efficiency non-RED implementations will exhibit a lower TCP Tranfer Efficiency
than RED implementations. than RED implementations.
4. Acknowledgements 4. Acknowledgements
The author would like to thank Gilles Forget, Loki Jorgenson, The author would like to thank Gilles Forget, Loki Jorgenson,
and Reinhard Schrage for technical review and contributions to this and Reinhard Schrage for technical review and original contributions
draft-03 memo. to this draft-03.
Also thanks to Matt Mathis and Matt Zekauskas for many good comments Also thanks to Matt Mathis and Matt Zekauskas for many good comments
through email exchange and for pointing us to great sources of through email exchange and for pointing us to great sources of
information pertaining to past works in the TCP capacity area. information pertaining to past works in the TCP capacity area.
5. References 5. References
[RFC2581] Allman, M., Paxson, V., Stevens W., "TCP Congestion [RFC2581] Allman, M., Paxson, V., Stevens W., "TCP Congestion
Control", RFC 2581, June 1999. Control", RFC 2581, June 1999.
 End of changes. 24 change blocks. 
60 lines changed or deleted 88 lines changed or added

This html diff was produced by rfcdiff 1.38. The latest version is available from http://tools.ietf.org/tools/rfcdiff/