draft-ietf-ippm-testplan-rfc2680-05.txt   rfc7290.txt 
Network Working Group L. Ciavattone Internet Engineering Task Force (IETF) L. Ciavattone
Internet-Draft AT&T Labs Request for Comments: 7290 AT&T Labs
Intended status: Informational R. Geib Category: Informational R. Geib
Expires: October 5, 2014 Deutsche Telekom ISSN: 2070-1721 Deutsche Telekom
A. Morton A. Morton
AT&T Labs AT&T Labs
M. Wieser M. Wieser
Technical University Darmstadt Technical University Darmstadt
April 3, 2014 July 2014
Test Plan and Results for Advancing RFC 2680 on the Standards Track Test Plan and Results for Advancing RFC 2680 on the Standards Track
draft-ietf-ippm-testplan-rfc2680-05
Abstract Abstract
This memo proposes to advance a performance metric RFC along the This memo provides the supporting test plan and results to advance
standards track, specifically RFC 2680 on One-way Loss Metrics. RFC 2680, a performance metric RFC defining one-way packet loss
Observing that the metric definitions themselves should be the metrics, along the Standards Track. Observing that the metric
primary focus rather than the implementations of metrics, this memo definitions themselves should be the primary focus rather than the
describes the test procedures to evaluate specific metric requirement implementations of metrics, this memo describes the test procedures
clauses to determine if the requirement has been interpreted and to evaluate specific metric requirement clauses to determine if the
implemented as intended. Two completely independent implementations requirement has been interpreted and implemented as intended. Two
have been tested against the key specifications of RFC 2680. completely independent implementations have been tested against the
key specifications of RFC 2680.
Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This document is not an Internet Standards Track specification; it is
provisions of BCP 78 and BCP 79. published for informational purposes.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months This document is a product of the Internet Engineering Task Force
and may be updated, replaced, or obsoleted by other documents at any (IETF). It represents the consensus of the IETF community. It has
time. It is inappropriate to use Internet-Drafts as reference received public review and has been approved for publication by the
material or to cite them other than as "work in progress." Internet Engineering Steering Group (IESG). Not all documents
approved by the IESG are a candidate for any level of Internet
Standard; see Section 2 of RFC 5741.
This Internet-Draft will expire on October 5, 2014. Information about the current status of this document, any errata,
and how to provide feedback on it may be obtained at
http://www.rfc-editor.org/info/rfc7290.
Copyright Notice Copyright Notice
Copyright (c) 2014 IETF Trust and the persons identified as the Copyright (c) 2014 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
This document may contain material from IETF Documents or IETF This document may contain material from IETF Documents or IETF
Contributions published or made publicly available before November Contributions published or made publicly available before November
skipping to change at page 2, line 34 skipping to change at page 3, line 7
modifications of such material outside the IETF Standards Process. modifications of such material outside the IETF Standards Process.
Without obtaining an adequate license from the person(s) controlling Without obtaining an adequate license from the person(s) controlling
the copyright in such materials, this document may not be modified the copyright in such materials, this document may not be modified
outside the IETF Standards Process, and derivative works of it may outside the IETF Standards Process, and derivative works of it may
not be created outside the IETF Standards Process, except to format not be created outside the IETF Standards Process, except to format
it for publication as an RFC or to translate it into languages other it for publication as an RFC or to translate it into languages other
than English. than English.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction ....................................................3
1.1. RFC 2680 Coverage . . . . . . . . . . . . . . . . . . . . 4 1.1. Requirements Language ......................................4
2. A Definition-centric metric advancement process . . . . . . . 4 1.2. RFC 2680 Coverage ..........................................5
3. Test configuration . . . . . . . . . . . . . . . . . . . . . 4 2. A Definition-Centric Metric Advancement Process .................5
4. Error Calibration, RFC 2680 . . . . . . . . . . . . . . . . . 9 3. Test Configuration ..............................................5
4.1. Clock Synchronization Calibration . . . . . . . . . . . . 9 4. Error Calibration and RFC 2680 ..................................9
4.2. Packet Loss Determination Error . . . . . . . . . . . . . 10 4.1. Clock Synchronization Calibration ..........................9
5. Pre-determined Limits on Equivalence . . . . . . . . . . . . 10 4.2. Packet Loss Determination Error ...........................10
6. Tests to evaluate RFC 2680 Specifications . . . . . . . . . . 11 5. Predetermined Limits on Equivalence ............................10
6.1. One-way Loss, ADK Sample Comparison . . . . . . . . . . . 11 6. Tests to Evaluate RFC 2680 Specifications ......................11
6.1.1. 340B/Periodic Cross-imp. results . . . . . . . . . . 12 6.1. One-Way Loss: ADK Sample Comparison .......................11
6.1.2. 64B/Periodic Cross-imp. results . . . . . . . . . . . 13 6.1.1. 340B/Periodic Cross-Implementation Results .........12
6.1.3. 64B/Poisson Cross-imp. results . . . . . . . . . . . 14 6.1.2. 64B/Periodic Cross-Implementation Results ..........14
6.1.4. Conclusions on the ADK Results for One-way Packet 6.1.3. 64B/Poisson Cross-Implementation Results ...........15
Loss . . . . . . . . . . . . . . . . . . . . . . . . 15 6.1.4. Conclusions on the ADK Results for One-Way
6.2. One-way Loss, Delay threshold . . . . . . . . . . . . . . 15 Packet Loss ........................................16
6.2.1. NetProbe results for Loss Threshold . . . . . . . . . 16 6.2. One-Way Loss: Delay Threshold .............................16
6.2.2. Perfas Results for Loss Threshold . . . . . . . . . . 17 6.2.1. NetProbe Results for Loss Threshold ................17
6.2.3. Conclusions for Loss Threshold . . . . . . . . . . . 17 6.2.2. Perfas+ Results for Loss Threshold .................17
6.2.3. Conclusions for Loss Threshold .....................17
6.3. One-way Loss with Out-of-Order Arrival . . . . . . . . . 17 6.3. One-Way Loss with Out-of-Order Arrival ....................17
6.4. Poisson Sending Process Evaluation . . . . . . . . . . . 18 6.4. Poisson Sending Process Evaluation ........................19
6.4.1. NetProbe Results . . . . . . . . . . . . . . . . . . 19 6.4.1. NetProbe Results ...................................19
6.4.2. Perfas+ Results . . . . . . . . . . . . . . . . . . . 20 6.4.2. Perfas+ Results ....................................20
6.4.3. Conclusions for Goodness-of-Fit . . . . . . . . . . . 22 6.4.3. Conclusions for Goodness-of-Fit ....................22
6.5. Implementation of Statistics for One-way Loss . . . . . . 22 6.5. Implementation of Statistics for One-Way Loss .............23
7. Conclusions for RFC 2680bis . . . . . . . . . . . . . . . . . 23 7. Conclusions for a Revision of RFC 2680 .........................23
8. Security Considerations . . . . . . . . . . . . . . . . . . . 23 8. Security Considerations ........................................24
9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 23 9. Acknowledgements ...............................................24
10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 24 10. Appendix - Network Configuration and Sample Commands ..........25
11. Appendix - Network Configuration and sample commands . . . . 24 11. References ....................................................28
12. References . . . . . . . . . . . . . . . . . . . . . . . . . 27 11.1. Normative References .....................................28
12.1. Normative References . . . . . . . . . . . . . . . . . . 27 11.2. Informative References ...................................29
12.2. Informative References . . . . . . . . . . . . . . . . . 28
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 29
1. Introduction 1. Introduction
The IETF (specifically the IP Performance Metrics working group, or The IETF IP Performance Metrics (IPPM) working group has considered
IPPM) has considered how to advance their metrics along the standards how to advance their metrics along the Standards Track since 2001.
track since 2001.
The renewed work effort sought to investigate ways in which the The renewed work effort sought to investigate ways in which the
measurement variability could be reduced and thereby simplify the measurement variability could be reduced in order to thereby simplify
problem of comparison for equivalence. As a result, there is the problem of comparison for equivalence. As a result, there is
consensus (captured in [RFC6576]) that equivalent results from consensus (captured in [RFC6576]) that equivalent results from
independent implementations of metric specifications are sufficient independent implementations of metric specifications are sufficient
evidence that the specifications themselves are clear and evidence that the specifications themselves are clear and
unambiguous; it is the parallel concept of protocol interoperability unambiguous; it is the parallel concept of protocol interoperability
for metric specifications. The advancement process either produces for metric specifications. The advancement process either (1)
confidence that the metric definitions and supporting material are produces confidence that the metric definitions and supporting
clearly worded and unambiguous, OR, identifies ways in which the material are clearly worded and unambiguous or (2) identifies ways in
metric definitions should be revised to achieve clarity. It is a which the metric definitions should be revised to achieve clarity.
non-goal to compare the specific implementations themselves. It is a non-goal to compare the specific implementations themselves.
The process also permits identification of options described in the The process also permits identification of options described in the
metric RFC that were not implemented, so that they can be removed metric RFC that were not implemented, so that they can be removed
from the advancing specification (this is an aspect more typical of from the advancing specification (this is an aspect more typical of
protocol advancement along the standards track). protocol advancement along the Standards Track).
This memo's purpose is to implement the current approach for This memo's purpose is to implement the current approach for
[RFC2680] and document the results. [RFC2680] and document the results.
In particular, this memo documents consensus on the extent of In particular, this memo documents consensus on the extent of
tolerable errors when assessing equivalence in the results. In tolerable errors when assessing equivalence in the results. In
discussions, the IPPM working group agreed that test plan and discussions, the IPPM working group agreed that the test plan
procedures should include the threshold for determining equivalence, and procedures should include the threshold for determining
and this information should be available in advance of cross- equivalence, and this information should be available in advance of
implementation comparisons. This memo includes procedures for same- cross-implementation comparisons. This memo includes procedures for
implementation comparisons to help set the equivalence threshold. same-implementation comparisons to help set the equivalence
threshold.
Another aspect of the metric RFC advancement process is the Another aspect of the metric RFC advancement process is the
requirement to document the work and results. The procedures of requirement to document the work and results. The procedures of
[RFC2026] are expanded in[RFC5657], including sample implementation [RFC2026] are expanded in [RFC5657], including sample implementation
and interoperability reports. This memo follows the template in and interoperability reports. This memo follows the template in
[RFC6808] for the report that accompanies the protocol action request [RFC6808] for the report that accompanies the protocol action request
submitted to the Area Director, including description of the test submitted to the Area Director, including a description of the test
set-up, procedures, results for each implementation, and conclusions. setup, procedures, results for each implementation, and conclusions.
The conclusion reached is that [RFC2680] should be advanced on the The conclusion reached is that [RFC2680], with modifications, should
Standards Track with modifications. The revised text of RFC 2680bis be advanced on the Standards Track. The revised text of RFC 2680
is ready for review [I-D.morton-ippm-2680-bis], but awaits work-in [LOSS-METRIC] is ready for review but awaits work in progress to
progress to update the IPPM Framework [RFC2330]. Therefore, this update the IPPM Framework [RFC2330]. Therefore, this memo documents
memo documents the information to support [RFC2680] advancement, and the information to support the advancement of [RFC2680], and the
the approval of RFC2680bis is left for future action. approval of a revision of RFC 2680 is left for future action.
1.1. RFC 2680 Coverage 1.1. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
Some of these key words were used in [RFC2680], but there are no
requirements specified in this memo.
1.2. RFC 2680 Coverage
This plan is intended to cover all critical requirements and sections This plan is intended to cover all critical requirements and sections
of [RFC2680]. of [RFC2680].
Note that there are only five instances of the requirement term Note that there are only five relevant instances of the requirement
"MUST" in [RFC2680] outside of the boilerplate and [RFC2119] term "MUST" in [RFC2680], outside of the boilerplate and [RFC2119]
reference. reference; the instance of "MUST" in the Security Considerations
section of [RFC2680] is not a basis for implementation equivalence
comparisons.
Material may be added as it is "discovered" (apparently, not all Statements in RFC 2680 that have the character of requirements may be
requirements use requirements language). included if the community reaches consensus that the wording implies
a requirement. At least one instance of an implied requirement has
been found in Section 3.6 of [RFC2680].
2. A Definition-centric metric advancement process 2. A Definition-Centric Metric Advancement Process
The process described in Section 3.5 of [RFC6576] takes as a first The process described in Section 3.5 of [RFC6576] takes as a first
principle that the metric definitions, embodied in the text of the principle that the metric definitions, embodied in the text of the
RFCs, are the objects that require evaluation and possible revision RFCs, are the objects that require evaluation and possible revision
in order to advance to the next step on the standards track. This in order to advance to the next step on the Standards Track. This
memo follows that process. memo follows that process.
3. Test configuration 3. Test Configuration
One metric implementation used was NetProbe version 5.8.5 (an earlier One metric implementation used was NetProbe version 5.8.5 (an earlier
version is used in the WIPM system and deployed world-wide [WIPM]). version is used in the WIPM system and deployed worldwide [WIPM]).
NetProbe uses UDP packets of variable size, and can produce test NetProbe uses UDP packets of variable size and can produce test
streams with Periodic [RFC3432] or Poisson [RFC2330] sample streams with Periodic [RFC3432] or Poisson [RFC2330] sample
distributions. distributions.
The other metric implementation used was Perfas+ version 3.1, The other metric implementation used was Perfas+ version 3.1,
developed by Deutsche Telekom [Perfas]. Perfas+ uses UDP unicast developed by Deutsche Telekom [Perfas]. Perfas+ uses UDP unicast
packets of variable size (but also supports TCP and multicast). Test packets of variable size (but also supports TCP and multicast). Test
streams with periodic, Poisson, or uniform sample distributions may streams with Periodic, Poisson, or uniform sample distributions may
be used. be used.
Figure 1 shows a view of the test path as each Implementation's test Figure 1 shows a view of the test path as each implementation's test
flows pass through the Internet and the L2TPv3 tunnel IDs (1 and 2), flows pass through the Internet and the Layer 2 Tunneling Protocol
based on Figure 1 of [RFC6576]. version 3 (L2TPv3) [RFC3931] tunnel IDs (1 and 2), based on Figure 1
of [RFC6576].
+------------+ +------------+ +------------+ +------------+
| Imp 1 | ,---. | Imp 2 | | Imp 1 | ,---. | Imp 2 |
+------------+ / \ +-------+ +------------+ +------------+ / \ +-------+ +------------+
| V100 ^ V200 / \ | Tunnel| | V300 ^ V400 | V100 ^ V200 / \ | Tunnel| | V300 ^ V400
| | ( ) | Head | | | | | ( ) | Head | | |
+--------+ +------+ | |__| Router| +----------+ +--------+ +------+ | |__| Router| +----------+
|Ethernet| |Tunnel| |Internet | +---B---+ |Ethernet | |Ethernet| |Tunnel| |Internet | +---B---+ |Ethernet |
|Switch |--|Head |-| | | |Switch | |Switch |--|Head |-| | | |Switch |
+-+--+---+ |Router| | | +---+---+--+--+--+----+ +-+--+---+ |Router| | | +---+---+--+--+--+----+
|__| +--A---+ ( ) |Network| |__| |__| +--A---+ ( ) |Network| |__|
\ / |Emulat.| \ / |Emulat.|
U-turn \ / |"netem"| U-turn U-turn \ / |"netem"| U-turn
V300 to V400 `-+-' +-------+ V100 to V200 V300 to V400 `-+-' +-------+ V100 to V200
Implementations ,---. +--------+ Implementations ,---. +--------+
+~~~~~~~~~~~/ \~~~~~~| Remote | +~~~~~~~~~~~/ \~~~~~~| Remote |
+------->-----F2->-| / \ |->---. | +------->-----F2->-| / \ |->---. |
| +---------+ | Tunnel ( ) | | | | +---------+ | Tunnel ( ) | | |
| | transmit|-F1->-| ID 1 | | |->. | | | | transmit|-F1->-| ID 1 | | |->. | |
| | Imp 1 | +~~~~~~~~~| |~~~~| | | | | | Imp 1 | +~~~~~~~~~| |~~~~| | | |
| | receive |-<--+ | | | F1 F2 | | | receive |-<--+ | | | F1 F2 |
| +---------+ | |Internet | | | | | | +---------+ | |Internet | | | | |
*-------<-----+ F1 | | | | | | *-------<-----+ F1 | | | | | |
+---------+ | | +~~~~~~~~~| |~~~~| | | | +---------+ | | +~~~~~~~~~| |~~~~| | | |
| transmit|-* *-| | | |<-* | | | transmit|-* *-| | | |<-* | |
| Imp 2 | | Tunnel ( ) | | | | Imp 2 | | Tunnel ( ) | | |
| receive |-<-F2-| ID 2 \ / |<----* | | receive |-<-F2-| ID 2 \ / |<----* |
+---------+ +~~~~~~~~~~~\ /~~~~~~| Switch | +---------+ +~~~~~~~~~~~\ /~~~~~~| Switch |
`-+-' +--------+ `-+-' +--------+
Illustrations of a test setup with a bi-directional tunnel. The Illustrations of a test setup with a bidirectional tunnel.
upper diagram emphasizes the VLAN connectivity and geographical The upper diagram emphasizes the VLAN connectivity and
location (where "Imp #" is the sender and receiver of implementation geographical location (where "Imp #" is the sender and
1 or 2, either Perfas+ and NetProbe in this test). The lower diagram receiver of implementation 1 or 2 -- either Perfas+ or
shows example flows traveling between two measurement NetProbe in this test). The lower diagram shows example
implementations. For simplicity only two flows are shown, and netem flows traveling between two measurement implementations.
is omitted (it would appear before or after the Internet, depending For simplicity, only two flows are shown, and the netem
on the flow). emulator is omitted (it would appear before or after the
Internet, depending on the flow).
Figure 1 Figure 1
The testing employs the Layer 2 Tunnel Protocol, version 3 (L2TPv3) The testing employs the L2TPv3 [RFC3931] tunnel between test sites on
[RFC3931] tunnel between test sites on the Internet. The tunnel IP the Internet. The tunnel IP and L2TPv3 headers are intended to
and L2TPv3 headers are intended to conceal the test equipment conceal the test equipment addresses and ports from hash functions
addresses and ports from hash functions that would tend to spread that would tend to spread different test streams across parallel
different test streams across parallel network resources, with likely network resources, with likely variation in performance as a result.
variation in performance as a result.
At each end of the tunnel, one pair of VLANs encapsulated in the At each end of the tunnel, one pair of VLANs encapsulated in the
tunnel are looped-back so that test traffic is returned to each test tunnel are looped back so that test traffic is returned to each test
site. Thus, test streams traverse the L2TP tunnel twice, but appear site. Thus, test streams traverse the L2TP tunnel twice but appear
to be one-way tests from the test equipment point of view. to be one-way tests from the point of view of the test equipment.
The network emulator is a host running Fedora 14 Linux [Fedora] with The network emulator is a host running Fedora 14 Linux [FEDORA], with
IP forwarding enabled and the "netem" Network emulator as part of the IP forwarding enabled and the "netem" Network emulator as part of the
Fedora Kernel 2.6.35.11 [netem] loaded and operating. The standard Fedora Kernel 2.6.35.11 [NETEM] loaded and operating. The standard
kernel is "tickless" replacing the previous periodic timer (250HZ, kernel is "tickless", replacing the previous periodic timer (250 Hz,
with 4ms uncertainty) interrupts with on-demand interrupts. with 4 ms uncertainty) interrupts with on-demand interrupts.
Connectivity across the netem/Fedora host was accomplished by Connectivity across the netem/Fedora host was accomplished by
bridging Ethernet VLAN interfaces together with "brctl" commands bridging Ethernet VLAN interfaces together with "brctl" commands
(e.g., eth1.100 <-> eth2.100). The netem emulator was activated on (e.g., eth1.100 <-> eth2.100). The netem emulator was activated on
one interface (eth1) and only operates on test streams traveling in one interface (eth1) and only operated on test streams traveling in
one direction. In some tests, independent netem instances operated one direction. In some tests, independent netem instances operated
separately on each VLAN. See the Appendix for more details. separately on each VLAN. See the Appendix for more details.
The links between the netem emulator host and router and switch were The links between the netem emulator host, the router, and the switch
found to be 100baseTx-HD (100Mbps half duplex) as reported by "mii- were found to be 100BaseTX-HD (100 Mbps half duplex), as reported by
tool" [mii-tool], when testing was complete. Use of half duplex was "mii-tool" [MII-TOOL] when testing was complete. The use of half
not intended, but probably added a small amount of delay variation duplex was not intended but probably added a small amount of delay
that could have been avoided in full duplex mode. variation that could have been avoided in full-duplex mode.
Each individual test was run with common packet rates (1 pps, 10pps) Each individual test was run with common packet rates (1 pps, 10 pps)
Poisson/Periodic distributions, and IP packet sizes of 64, 340, and Poisson/Periodic distributions, and IP packet sizes of 64, 340, and
500 Bytes. 500 bytes.
For these tests, a stream of at least 300 packets was sent from For these tests, a stream of at least 300 packets was sent from
source to destination in each implementation. Periodic streams (as source to destination in each implementation. Periodic streams (as
per [RFC3432]) with 1 second spacing were used, except as noted. per [RFC3432]) with 1-second spacing were used, except as noted.
As required in Section 2.8.1 of [RFC2680], packet Type-P must be As required in Section 2.8.1 of [RFC2680], packet Type-P must be
reported. The packet Type-P for this test was IP-UDP with Best reported. The packet Type-P for this test was IP-UDP with Best
Effort DSCP. These headers were encapsulated according to the L2TPv3 Effort Differentiated Services Code Point (DSCP). These headers were
specifications [RFC3931], and thus may not influence the treatment encapsulated according to the L2TPv3 specification [RFC3931] and were
received as the packets traversed the Internet. unlikely to influence the treatment received as the packets traversed
the Internet.
With the L2TPv3 tunnel in use, the metric name for the testing With the L2TPv3 tunnel in use, the metric name for the testing
configured here (with respect to the IP header exposed to Internet configured here (with respect to the IP header exposed to Internet
processing) is: processing) is:
Type-IP-protocol-115-One-way-Packet-Loss-<StreamType>-Stream Type-IP-protocol-115-One-way-Packet-Loss-<StreamType>-Stream
With (Section 3.2. [RFC2680]) metric parameters: With (Section 3.2 of [RFC2680]) metric parameters:
+ Src, the IP address of a host (12.3.167.16 or 193.159.144.8) + Src, the IP address of a host (12.3.167.16 or 193.159.144.8)
+ Dst, the IP address of a host (193.159.144.8 or 12.3.167.16) + Dst, the IP address of a host (193.159.144.8 or 12.3.167.16)
+ T0, a time + T0, a time
+ Tf, a time + Tf, a time
+ lambda, a rate in reciprocal seconds + lambda, a rate in reciprocal seconds
+ Thresh, a maximum waiting time in seconds (see Section 2.8.2 of + Thresh, a maximum waiting time in seconds (see Section 2.8.2 of
[RFC2680]) and (Section 3.8. [RFC2680]) [RFC2680])
Metric Units: A sequence of pairs; the elements of each pair are: Metric Units: A sequence of pairs; the elements of each pair are:
+ T, a time, and + T, a time, and
+ L, either a zero or a one + L, either a zero or a one
The values of T in the sequence are monotonically increasing. Note The values of T in the sequence are monotonically increasing.
that T would be a valid parameter of *singleton* Type-P-One-way- Note that T would be a valid parameter of *singleton*
Packet-Loss, and that L would be a valid value of Type-P-One-way- Type-P-One-way-Packet-Loss and that L would be a valid value of
Packet Loss (see Section 2 of [RFC2680]). Type-P-One-way-Packet-Loss (see Section 3.3 of [RFC2680]).
Also, Section 2.8.4 of [RFC2680] recommends that the path SHOULD be Also, Section 2.8.4 of [RFC2680] recommends that the path SHOULD be
reported. In this test set-up, most of the path details will be reported. In this test setup, most of the path details will be
concealed from the implementations by the L2TPv3 tunnels, thus a more concealed from the implementations by the L2TPv3 tunnels; thus, a
informative path trace route can be conducted by the routers at each more informative path traceroute can be conducted by the routers at
location. each location.
When NetProbe is used in production, a traceroute is conducted in When NetProbe is used in production, a traceroute is conducted in
parallel at the outset of measurements. parallel at the outset of measurements.
Perfas+ does not support traceroute. Perfas+ does not support traceroute.
IPLGW#traceroute 193.159.144.8 IPLGW#traceroute 193.159.144.8
Type escape sequence to abort. Type escape sequence to abort.
Tracing the route to 193.159.144.8 Tracing the route to 193.159.144.8
skipping to change at page 9, line 34 skipping to change at page 9, line 34
8 217.89.29.55 [AS 3320] 88 msec 88 msec 88 msec 8 217.89.29.55 [AS 3320] 88 msec 88 msec 88 msec
9 * * * 9 * * *
NetProbe Traceroute NetProbe Traceroute
It was only possible to conduct the traceroute for the measured path It was only possible to conduct the traceroute for the measured path
on one of the tunnel-head routers (the normal trace facilities of the on one of the tunnel-head routers (the normal trace facilities of the
measurement systems are confounded by the L2TPv3 tunnel measurement systems are confounded by the L2TPv3 tunnel
encapsulation). encapsulation).
4. Error Calibration, RFC 2680 4. Error Calibration and RFC 2680
An implementation is required to report calibration results on clock An implementation is required to report calibration results on clock
synchronization in Section 2.8.3 of [RFC2680] (also required in synchronization per Section 2.8.3 of [RFC2680] (also required in
Section 3.7 of [RFC2680] for sample metrics). Section 3.7 of [RFC2680] for sample metrics).
Also, it is recommended to report the probability that a packet Also, it is recommended to report the probability that a packet
successfully arriving at the destination network interface is successfully arriving at the destination network interface is
incorrectly designated as lost due to resource exhaustion in incorrectly designated as lost due to resource exhaustion in
Section 2.8.3 of [RFC2680]. Section 2.8.3 of [RFC2680].
4.1. Clock Synchronization Calibration 4.1. Clock Synchronization Calibration
For NetProbe and Perfas+ clock synchronization test results, refer to For NetProbe and Perfas+ clock synchronization test results, refer to
Section 4 of [RFC6808]. Section 4 of [RFC6808].
4.2. Packet Loss Determination Error 4.2. Packet Loss Determination Error
Since both measurement implementations have resource limitations, it Since both measurement implementations have resource limitations, it
is theoretically possible that these limits could be exceeded and a is theoretically possible that these limits could be exceeded and a
packet that arrived at the destination successfully might be packet that arrived at the destination successfully might be
discarded in error. discarded in error.
In previous test efforts [I-D.morton-ippm-advance-metrics], NetProbe In previous test efforts [ADV-METRICS], NetProbe produced six
produced 6 multicast streams with an aggregate bit rate over 53 Mbit/ multicast streams with an aggregate bit rate over 53 Mbit/s, in order
s, in order to characterize the 1-way capacity of a NISTNet-based to characterize the one-way capacity of an emulator based on NIST
emulator. Neither the emulator nor the pair of NetProbe Net. Neither the emulator nor the pair of NetProbe implementations
implementations used in this testing dropped any packets in these used in this testing dropped any packets in these streams.
streams.
The maximum load used here between any 2 NetProbe implementations was The maximum load used here between any two NetProbe implementations
11.5 Mbit/s divided equally among 3 unicast test streams. We was 11.5 Mbit/s divided equally among three unicast test streams. We
concluded that steady resource usage does not contribute error concluded that steady resource usage does not contribute error
(additional loss) to the measurements. (additional loss) to the measurements.
5. Pre-determined Limits on Equivalence 5. Predetermined Limits on Equivalence
In this section, we provide the numerical limits on comparisons In this section, we provide the numerical limits on comparisons
between implementations in order to declare that the results are between implementations in order to declare that the results are
equivalent and therefore, the tested specification is clear. equivalent and that the tested specification is therefore clear.
A key point is that the allowable errors, corrections, and confidence A key point is that the allowable errors, corrections, and confidence
levels only need to be sufficient to detect misinterpretation of the levels only need to be sufficient to detect any misinterpretation of
tested specification resulting in diverging implementations. the tested specification that would indicate diverging
implementations.
Also, the allowable error must be sufficient to compensate for Also, the allowable error must be sufficient to compensate for
measured path differences. It was simply not possible to measure measured path differences. It was simply not possible to measure
fully identical paths in the VLAN-loopback test configuration used, fully identical paths in the VLAN-loopback test configuration used,
and this practical compromise must be taken into account. and this practical compromise must be taken into account.
For Anderson-Darling K-sample (ADK) [ADK] comparisons, the required For Anderson-Darling K-sample (ADK) [ADK] comparisons, the required
confidence factor for the cross-implementation comparisons SHALL be confidence factor for the cross-implementation comparisons SHALL be
the smallest of: the smallest of:
o 0.95 confidence factor at 1 packet resolution, or o 0.95 confidence factor at 1-packet resolution, or
o the smallest confidence factor (in combination with resolution) of o the smallest confidence factor (in combination with resolution) of
the two same-implementation comparisons for the same test the two same-implementation comparisons for the same test
conditions (if the number of streams is sufficient to allow such conditions (if the number of streams is sufficient to allow such
comparisons). comparisons).
For Anderson-Darling Goodness-of-Fit (ADGoF) [Radgof] comparisons, For Anderson-Darling Goodness-of-Fit (ADGoF) [RADGOF] comparisons,
the required level of significance for the same-implementation the required level of significance for the same-implementation
Goodness-of-Fit (GoF) SHALL be 0.05 or 5%, as specified in Goodness-of-Fit (GoF) SHALL be 0.05 or 5%, as specified in
Section 11.4 of [RFC2330]. This is equivalent to a 95% confidence Section 11.4 of [RFC2330]. This is equivalent to a 95% confidence
factor. factor.
6. Tests to evaluate RFC 2680 Specifications 6. Tests to Evaluate RFC 2680 Specifications
This section describes some results from production network (cross- This section describes some results from production network (cross-
Internet) tests with measurement devices implementing IPPM metrics Internet) tests with measurement devices implementing IPPM metrics
and a network emulator to create relevant conditions, to determine and a network emulator to create relevant conditions, to determine
whether the metric definitions were interpreted consistently by whether the metric definitions were interpreted consistently by
implementors. implementors.
The procedures are similar contained in Appendix A.1 of [RFC6576] for The procedures are similar to those contained in Appendix A.1 of
One-way Delay. [RFC6576] for one-way delay.
6.1. One-way Loss, ADK Sample Comparison 6.1. One-Way Loss: ADK Sample Comparison
This test determines if implementations produce results that appear This test determines if implementations produce results that appear
to come from a common packet loss distribution, as an overall to come from a common packet loss distribution, as an overall
evaluation of Section 3 of [RFC2680], "A Definition for Samples of evaluation of Section 3 of [RFC2680] ("A Definition for Samples of
One-way Packet Loss". Same-implementation comparison results help to One-way Packet Loss"). Same-implementation comparison results help
set the threshold of equivalence that will be applied to cross- to set the threshold of equivalence that will be applied to cross-
implementation comparisons. implementation comparisons.
This test is intended to evaluate measurements in sections 2, 3, and This test is intended to evaluate measurements in Sections 2, 3, and
4 of [RFC2680]. 4 of [RFC2680].
By testing the extent to which the counts of one-way packet loss By testing the extent to which the counts of one-way packet loss on
counts on different test streams of two [RFC2680] implementations different test streams of two [RFC2680] implementations appear to be
appear to be from the same loss process, we reduce comparison steps from the same loss process, we reduce comparison steps because
because comparing the resulting summary statistics (as defined in comparing the resulting summary statistics (as defined in Section 4
Section 4 of [RFC2680]) would require a redundant set of equivalence of [RFC2680]) would require a redundant set of equivalence
evaluations. We can easily check whether the single statistic in evaluations. We can easily check whether the single statistic in
Section 4 of [RFC2680] was implemented, and report on that fact. Section 4 of [RFC2680] was implemented and report on that fact.
1. Configure an L2TPv3 path between test sites, and each pair of 1. Configure an L2TPv3 path between test sites, and each pair of
measurement devices to operate tests in their designated pair of measurement devices to operate tests in their designated pair of
VLANs. VLANs.
2. Measure a sample of one-way packet loss singletons with 2 or more 2. Measure a sample of one-way packet loss singletons with two or
implementations, using identical options and network emulator more implementations, using identical options and network
settings (if used). emulator settings (if used).
3. Measure a sample of one-way packet loss singletons with *four or 3. Measure a sample of one-way packet loss singletons with *four or
more* instances of the *same* implementations, using identical more* instances of the *same* implementations, using identical
options, noting that connectivity differences SHOULD be the same options, noting that connectivity differences SHOULD be the same
as for cross implementation testing. as for cross-implementation testing.
4. If less than ten test streams are available, skip to step 7. 4. If less than ten test streams are available, skip to step 7.
5. Apply the ADK comparison procedures (see Appendix C of [RFC6576]) 5. Apply the ADK comparison procedures (see Appendix B of
and determine the resolution and confidence factor for [RFC6576]), and determine the resolution and confidence factor
distribution equivalence of each same-implementation comparison for distribution equivalence of each same-implementation
and each cross-implementation comparison. comparison and each cross-implementation comparison.
6. Take the coarsest resolution and confidence factor for 6. Take the coarsest resolution and confidence factor for
distribution equivalence from the same-implementation pairs, or distribution equivalence from the same-implementation pairs, or
the limit defined in Section 5 above, as a limit on the the limit defined in Section 5 above, as a limit on the
equivalence threshold for these experimental conditions. equivalence threshold for these experimental conditions.
7. Compare the cross-implementation ADK performance with the 7. Compare the cross-implementation ADK performance with the
equivalence threshold determined in step 5 to determine if equivalence threshold determined in step 5 to determine if
equivalence can be declared. equivalence can be declared.
The metric parameters varied for each loss test, and they are listed The metric parameters varied for each loss test, and they are listed
first in each sub-section below. first in each sub-section below.
The cross-implementation comparison uses a simple ADK analysis The cross-implementation comparison uses a simple ADK analysis
[Rtool] [Radk], where all NetProbe loss counts are compared with all [RTOOL] [RADK], where all NetProbe loss counts are compared with all
Perfas+ loss results. Perfas+ loss results.
In the result analysis of this section: In the results analysis of this section:
o All comparisons used 1 packet resolution. o All comparisons used 1-packet resolution.
o No Correction Factors were applied. o No correction factors were applied.
o The 0.95 confidence factor (1.960 for cross-implementation o The 0.95 confidence factor (and ADK criterion for t.obs < 1.960
comparison) was used. for cross-implementation comparison) was used.
6.1.1. 340B/Periodic Cross-imp. results 6.1.1. 340B/Periodic Cross-Implementation Results
Tests described in this section used: Tests described in this section used:
o IP header + payload = 340 octets o IP header + payload = 340 octets
o Periodic sampling at 1 packet per second o Periodic sampling at 1 packet per second
o Test duration = 1200 seconds (during April 7, 2011, EDT) o Test duration = 1200 seconds (during April 7, 2011, EDT)
The netem emulator was set for 100 ms constant delay, with a 10% loss
The netem emulator was set for 100ms constant delay, with 10% loss
ratio. In this experiment, the netem emulator was configured to ratio. In this experiment, the netem emulator was configured to
operate independently on each VLAN and thus the emulator itself is a operate independently on each VLAN; thus, the emulator itself is a
potential source of error when comparing streams that traverse the potential source of error when comparing streams that traverse the
test path in different directions. test path in different directions.
======================================= =======================================
A07bps_loss <- c(114, 175, 138, 142, 181, 105) (NetProbe) A07bps_loss <- c(114, 175, 138, 142, 181, 105) (NetProbe)
A07per_loss <- c(115, 128, 136, 127, 139, 138) (Perfas+) A07per_loss <- c(115, 128, 136, 127, 139, 138) (Perfas+)
> A07bps_loss <- c(114, 175, 138, 142, 181, 105) > A07bps_loss <- c(114, 175, 138, 142, 181, 105)
> A07per_loss <- c(115, 128, 136, 127, 139, 138) > A07per_loss <- c(115, 128, 136, 127, 139, 138)
skipping to change at page 13, line 32 skipping to change at page 13, line 37
Mean of Anderson Darling Criterion: 1 Mean of Anderson Darling Criterion: 1
Standard deviation of Anderson Darling Criterion: 0.6569 Standard deviation of Anderson Darling Criterion: 0.6569
T = (Anderson Darling Criterion - mean)/sigma T = (Anderson Darling Criterion - mean)/sigma
Null Hypothesis: All samples come from a common population. Null Hypothesis: All samples come from a common population.
t.obs P-value extrapolation t.obs P-value extrapolation
not adj. for ties 0.52043 0.20604 0 not adj. for ties 0.52043 0.20604 0
adj. for ties 0.62679 0.18607 0 adj. for ties 0.62679 0.18607 0
>
======================================= =======================================
The cross-implementation comparisons pass the ADK criterion. The cross-implementation comparisons pass the ADK criterion
(t.obs < 1.960).
6.1.2. 64B/Periodic Cross-imp. results 6.1.2. 64B/Periodic Cross-Implementation Results
Tests described in this section used: Tests described in this section used:
o IP header + payload = 64 octets o IP header + payload = 64 octets
o Periodic sampling at 1 packet per second o Periodic sampling at 1 packet per second
o Test duration = 300 seconds (during March 24, 2011, EDT) o Test duration = 300 seconds (during March 24, 2011, EDT)
The netem emulator was set for 0ms constant delay, with 10% loss The netem emulator was set for 0 ms constant delay, with a 10% loss
ratio. ratio.
======================================= =======================================
> M24per_loss <- c(42,34,35,35) (Perfas+) > M24per_loss <- c(42,34,35,35) (Perfas+)
> M24apd_23BC_loss <- c(27,39,29,24) (NetProbe) > M24apd_23BC_loss <- c(27,39,29,24) (NetProbe)
> M24apd_loss23BC_ADK <- adk.test(M24apd_23BC_loss,M24per_loss) > M24apd_loss23BC_ADK <- adk.test(M24apd_23BC_loss,M24per_loss)
> M24apd_loss23BC_ADK > M24apd_loss23BC_ADK
Anderson-Darling k-sample test. Anderson-Darling k-sample test.
skipping to change at page 14, line 30 skipping to change at page 14, line 43
T = (Anderson Darling Criterion - mean)/sigma T = (Anderson Darling Criterion - mean)/sigma
Null Hypothesis: All samples come from a common population. Null Hypothesis: All samples come from a common population.
t.obs P-value extrapolation t.obs P-value extrapolation
not adj. for ties 0.76921 0.16200 0 not adj. for ties 0.76921 0.16200 0
adj. for ties 0.90935 0.14113 0 adj. for ties 0.90935 0.14113 0
Warning: At least one sample size is less than 5. Warning: At least one sample size is less than 5.
p-values may not be very accurate. p-values may not be very accurate.
>
======================================= =======================================
The cross-implementation comparisons pass the ADK criterion. The cross-implementation comparisons pass the ADK criterion.
6.1.3. 64B/Poisson Cross-imp. results 6.1.3. 64B/Poisson Cross-Implementation Results
Tests described in this section used: Tests described in this section used:
o IP header + payload = 64 octets o IP header + payload = 64 octets
o Poisson sampling at lambda = 1 packet per second o Poisson sampling at lambda = 1 packet per second
o Test duration = 20 minutes (during April 27, 2011, EDT) o Test duration = 1200 seconds (during April 27, 2011, EDT)
The netem configuration was 0ms delay and 10% loss, but there were The netem configuration was 0 ms delay and 10% loss, but there were
two passes through an emulator for each stream, and loss emulation two passes through an emulator for each stream, and loss emulation
was present for 18 minutes of the 20 minute test. was present for 18 minutes of the 20-minute (1200-second) test.
======================================= =======================================
A27aps_loss <- c(91,110,113,102,111,109,112,113) (NetProbe) A27aps_loss <- c(91,110,113,102,111,109,112,113) (NetProbe)
A27per_loss <- c(95,123,126,114) (Perfas+) A27per_loss <- c(95,123,126,114) (Perfas+)
A27cross_loss_ADK <- adk.test(A27aps_loss, A27per_loss) A27cross_loss_ADK <- adk.test(A27aps_loss, A27per_loss)
> A27cross_loss_ADK > A27cross_loss_ADK
Anderson-Darling k-sample test. Anderson-Darling k-sample test.
skipping to change at page 15, line 32 skipping to change at page 15, line 46
T = (Anderson Darling Criterion - mean)/sigma T = (Anderson Darling Criterion - mean)/sigma
Null Hypothesis: All samples come from a common population. Null Hypothesis: All samples come from a common population.
t.obs P-value extrapolation t.obs P-value extrapolation
not adj. for ties 2.15099 0.04145 0 not adj. for ties 2.15099 0.04145 0
adj. for ties 1.93129 0.05125 0 adj. for ties 1.93129 0.05125 0
Warning: At least one sample size is less than 5. Warning: At least one sample size is less than 5.
p-values may not be very accurate. p-values may not be very accurate.
> >
======================================= =======================================
The cross-implementation comparisons barely pass the ADK criterion at The cross-implementation comparisons barely pass the ADK criterion at
95% = 1.960 when adjusting for ties. 95% = 1.960 when adjusting for ties.
6.1.4. Conclusions on the ADK Results for One-way Packet Loss 6.1.4. Conclusions on the ADK Results for One-Way Packet Loss
We conclude that the two implementations are capable of producing We conclude that the two implementations are capable of producing
equivalent one-way packet loss measurements based on their equivalent one-way packet loss measurements based on their
interpretation of [RFC2680]. interpretation of [RFC2680].
6.2. One-way Loss, Delay threshold 6.2. One-Way Loss: Delay Threshold
This test determines if implementations use the same configured This test determines if implementations use the same configured
maximum waiting time delay from one measurement to another under maximum waiting time delay from one measurement to another under
different delay conditions, and correctly declare packets arriving in different delay conditions and correctly declare packets arriving in
excess of the waiting time threshold as lost. excess of the waiting time threshold as lost.
See Section 2.8.2 of [RFC2680]. See Section 2.8.2 of [RFC2680].
1. Configure an L2TPv3 path between test sites, and each pair of 1. Configure an L2TPv3 path between test sites, and each pair of
measurement devices to operate tests in their designated pair of measurement devices to operate tests in their designated pair of
VLANs. VLANs.
2. Configure the network emulator to add 1sec one-way constant delay 2. Configure the network emulator to add 1 second of one-way
in one direction of transmission. constant delay in one direction of transmission.
3. Measure (average) one-way delay with 2 or more implementations, 3. Measure (average) one-way delay with two or more implementations,
using identical waiting time thresholds (Thresh) for loss set at using identical waiting time thresholds (Thresh) for loss set at
3 seconds. 3 seconds.
4. Configure the network emulator to add 3 sec one-way constant 4. Configure the network emulator to add 3 seconds of one-way
delay in one direction of transmission equivalent to 2 seconds of constant delay in one direction of transmission equivalent to
additional one-way delay (or change the path delay while test is 2 seconds of additional one-way delay (or change the path delay
in progress, when there are sufficient packets at the first delay while the test is in progress, when there are sufficient packets
setting). at the first delay setting).
5. Repeat/continue measurements. 5. Repeat/continue measurements.
6. Observe that the increase measured in step 5 caused all packets 6. Observe that the increase measured in step 5 caused all packets
with 2 sec additional delay to be declared lost, and that all with 2 seconds of additional delay to be declared lost and that
packets that arrive successfully in step 3 are assigned a valid all packets that arrive successfully in step 3 are assigned a
one-way delay. valid one-way delay.
The common parameters used for tests in this section are: The common parameters used for tests in this section are:
o IP header + payload = 64 octets o IP header + payload = 64 octets
o Poisson sampling at lambda = 1 packet per second o Poisson sampling at lambda = 1 packet per second
o Test duration = 900 seconds total (March 21, 2011 EDT) o Test duration = 900 seconds total (March 21, 2011 EDT)
The netem emulator settings added constant delays as specified in the The netem emulator settings added constant delays as specified in the
procedure above. procedure above.
6.2.1. NetProbe results for Loss Threshold 6.2.1. NetProbe Results for Loss Threshold
In NetProbe, the Loss Threshold was implemented uniformly over all In NetProbe, the loss threshold was implemented uniformly over all
packets as a post-processing routine. With the Loss Threshold set at packets as a post-processing routine. With the loss threshold set at
3 seconds, all packets with one-way delay >3 seconds were marked 3 seconds, all packets with one-way delay >3 seconds were marked
"Lost" and included in the Lost Packet list with their transmission "Lost" and included in the Lost Packet list with their transmission
time (as required in Section 3.3 of [RFC2680]). This resulted in 342 time (as required in Section 3.3 of [RFC2680]). This resulted in
packets designated as lost in one of the test streams (with average 342 packets designated as lost in one of the test streams (with
delay = 3.091 sec). average delay = 3.091 sec).
6.2.2. Perfas Results for Loss Threshold 6.2.2. Perfas+ Results for Loss Threshold
Perfas+ uses a fixed Loss Threshold which was not adjustable during Perfas+ uses a fixed loss threshold, which was not adjustable during
this study. The Loss Threshold is approximately one minute, and this study. The loss threshold is approximately one minute, and
emulation of a delay of this size was not attempted. However, it is emulation of a delay of this size was not attempted. However, it is
possible to implement any delay threshold desired with a post- possible to implement any delay threshold desired with a
processing routine and subsequent analysis. Using this method, 195 post-processing routine and subsequent analysis. Using this method,
packets would be declared lost (with average delay = 3.091 sec). 195 packets would be declared lost (with average delay = 3.091 sec).
6.2.3. Conclusions for Loss Threshold 6.2.3. Conclusions for Loss Threshold
Both implementations assume that any constant delay value desired can Both implementations assume that any constant delay value desired can
be used as the Loss Threshold, since all delays are stored as a pair be used as the loss threshold, since all delays are stored as a pair
<Time, Delay> as required in [RFC2680]. This is a simple way to <Time, Delay> as required in [RFC2680]. This is a simple way to
enforce the constant loss threshold envisioned in [RFC2680] (see enforce the constant loss threshold envisioned in [RFC2680] (see
specific section reference above). We take the position that the Section 2.8.2 of [RFC2680]). We take the position that the
assumption of post-processing is compliant, and that the text of the assumption of post-processing is compliant and that the text of the
RFC should be revised slightly to include this point. revision of RFC 2680 should be revised slightly to include this
point.
6.3. One-way Loss with Out-of-Order Arrival 6.3. One-Way Loss with Out-of-Order Arrival
Section 3.6 of [RFC2680] indicates that implementations need to Section 3.6 of [RFC2680] indicates, with a lowercase "must" in the
ensure that reordered packets are handled correctly using an text, that implementations need to ensure that reordered packets are
uncapitalized "must". In essence, this is an implied requirement handled correctly. In essence, this is an implied requirement
because the correct packet must be identified as lost if it fails to because the correct packet must be identified as lost if it fails to
arrive before its delay threshold under all circumstances, and arrive before its delay threshold under all circumstances, and
reordering is always a possibility on IP network paths. See reordering is always a possibility on IP network paths. See
[RFC4737] for the definition of reordering used in IETF standard- [RFC4737] for the definition of reordering used in IETF
compliant measurements. standard-compliant measurements.
Using the procedure of section 6.1, the netem emulator was set to The netem emulator can produce packet reordering because each
introduce 10% loss, significant delay (2000 ms) and delay variation packet's delay is drawn from an independent distribution. Here,
(1000 ms), which was sufficient to produce packet reordering because significant delay (2000 ms) and delay variation (1000 ms) were
each packet's emulated delay is independent from others. sufficient to produce packet reordering. Using the procedure
described in Section 6.1, the netem emulator was set to introduce 10%
loss while reordering was present.
The tests described in this section used: The tests described in this section used:
o IP header + payload = 64 octets o IP header + payload = 64 octets
o Periodic sampling = 1 packet per second o Periodic sampling = 1 packet per second
o Test duration = 600 seconds (during May 2, 2011, EDT) o Test duration = 600 seconds (during May 2, 2011, EDT)
======================================= =======================================
> Y02aps_loss <- c(53,45,67,55) (NetProbe) > Y02aps_loss <- c(53,45,67,55) (NetProbe)
> Y02per_loss <- c(59,62,67,69) (Perfas+) > Y02per_loss <- c(59,62,67,69) (Perfas+)
> Y02cross_loss_ADK <- adk.test(Y02aps_loss, Y02per_loss) > Y02cross_loss_ADK <- adk.test(Y02aps_loss, Y02per_loss)
> Y02cross_loss_ADK > Y02cross_loss_ADK
Anderson-Darling k-sample test. Anderson-Darling k-sample test.
Number of samples: 2 Number of samples: 2
Sample sizes: 4 4 Sample sizes: 4 4
skipping to change at page 18, line 29 skipping to change at page 18, line 41
T = (Anderson Darling Criterion - mean)/sigma T = (Anderson Darling Criterion - mean)/sigma
Null Hypothesis: All samples come from a common population. Null Hypothesis: All samples come from a common population.
t.obs P-value extrapolation t.obs P-value extrapolation
not adj. for ties 1.11282 0.11531 0 not adj. for ties 1.11282 0.11531 0
adj. for ties 1.19571 0.10616 0 adj. for ties 1.19571 0.10616 0
Warning: At least one sample size is less than 5. Warning: At least one sample size is less than 5.
p-values may not be very accurate. p-values may not be very accurate.
> >
======================================= =======================================
The test results indicate that extensive reordering was present. The test results indicate that extensive reordering was present.
Both implementations capture the extensive delay variation between Both implementations capture the extensive delay variation between
adjacent packets. In NetProbe, packet arrival order is preserved in adjacent packets. In NetProbe, packet arrival order is preserved in
the raw measurement files, so an examination of arrival packet the raw measurement files, so an examination of arrival packet
sequence numbers also indicates reordering. sequence numbers also reveals reordering.
Despite extensive continuous packet reordering present in the Despite extensive continuous packet reordering present in the
transmission path, the distributions of loss counts from the two transmission path, the distributions of loss counts from the two
implementations pass the ADK criterion at 95% = 1.960. implementations pass the ADK criterion at 95% = 1.960.
6.4. Poisson Sending Process Evaluation 6.4. Poisson Sending Process Evaluation
Section 3.7 of [RFC2680] indicates that implementations need to Section 3.7 of [RFC2680] indicates that implementations need to
ensure that their sending process is reasonably close to a classic ensure that their sending process is reasonably close to a classic
Poisson distribution when used. Much more detail on sample Poisson distribution when used. Much more detail on sample
distribution generation and Goodness-of-Fit testing is specified in distribution generation and Goodness-of-Fit testing is specified in
Section 11.4 of [RFC2330] and the Appendix of [RFC2330]. Section 11.4 of [RFC2330] and the Appendix of [RFC2330].
In this section, each implementation's Poisson distribution is In this section, each implementation's Poisson distribution is
compared with an idealistic version of the distribution available in compared with an idealistic version of the distribution available in
the base functionality of the R-tool for Statistical Analysis[Rtool], the base functionality of the R-tool for Statistical Analysis [RTOOL]
and performed using the Anderson-Darling Goodness-of-Fit test package and performed using the Anderson-Darling Goodness-of-Fit test package
(ADGofTest) [Radgof]. The Goodness-of-Fit criterion derived from (ADGofTest) [RADGOF]. The Goodness-of-Fit criterion derived from
[RFC2330] requires a test statistic value AD <= 2.492 for 5% [RFC2330] requires a test statistic value AD <= 2.492 for 5%
significance. The Appendix of [RFC2330] also notes that there may be significance. The Appendix of [RFC2330] also notes that there may be
difficulty satisfying the ADGofTest when the sample includes many difficulty satisfying the ADGofTest when the sample includes many
packets (when 8192 were used, the test always failed, but smaller packets (when 8192 were used, the test always failed, but smaller
sets of the stream passed). sets of the stream passed).
Both implementations were configured to produce Poisson distributions Both implementations were configured to produce Poisson distributions
with lambda = 1 packet per second, and assign received packet with lambda = 1 packet per second and to assign received packet
timestamps in the measurement application (above UDP layer, see the timestamps in the measurement application (above the UDP layer; see
calibration results in Section 4 of [RFC6808] for assessment of the calibration results in Section 4 of [RFC6808] for error
error). assessment).
6.4.1. NetProbe Results 6.4.1. NetProbe Results
Section 11.4 of [RFC2330] suggests three possible measurement points Section 11.4 of [RFC2330] suggests three possible measurement points
to evaluate the Poisson distribution. The NetProbe analysis uses to evaluate the Poisson distribution. The NetProbe analysis uses
"user-level timestamps made just before or after the system call for "user-level timestamps made just before or after the system call for
transmitting the packet". transmitting the packet".
The statistical summary for two NetProbe streams is below: The statistical summary for two NetProbe streams is below:
======================================= =======================================
> summary(a27ms$s1[2:1152]) > summary(a27ms$s1[2:1152])
Min. 1st Qu. Median Mean 3rd Qu. Max. Min. 1st Qu. Median Mean 3rd Qu. Max.
0.0100 0.2900 0.6600 0.9846 1.3800 8.6390 0.0100 0.2900 0.6600 0.9846 1.3800 8.6390
> summary(a27ms$s2[2:1152]) > summary(a27ms$s2[2:1152])
Min. 1st Qu. Median Mean 3rd Qu. Max. Min. 1st Qu. Median Mean 3rd Qu. Max.
0.010 0.280 0.670 0.979 1.365 8.829 0.010 0.280 0.670 0.979 1.365 8.829
======================================= =======================================
We see that both of the means are near the specified lambda = 1.
We see that both the Means are near the specified lambda = 1. The results of ADGoF tests for these two streams are shown below:
The results of ADGoF tests for these two streams is shown below:
======================================= =======================================
> ad.test( a27ms$s1[2:101], pexp, 1) > ad.test( a27ms$s1[2:101], pexp, 1)
Anderson-Darling GoF Test Anderson-Darling GoF Test
data: a27ms$s1[2:101] and pexp data: a27ms$s1[2:101] and pexp
AD = 0.8908, p-value = 0.4197 AD = 0.8908, p-value = 0.4197
alternative hypothesis: NA alternative hypothesis: NA
skipping to change at page 20, line 41 skipping to change at page 20, line 44
> ad.test( a27ms$s2[2:1001], pexp, 1) > ad.test( a27ms$s2[2:1001], pexp, 1)
Anderson-Darling GoF Test Anderson-Darling GoF Test
data: a27ms$s2[2:1001] and pexp data: a27ms$s2[2:1001] and pexp
AD = 0.6913, p-value = 0.5661 AD = 0.6913, p-value = 0.5661
alternative hypothesis: NA alternative hypothesis: NA
======================================= =======================================
We see that both 100 and 1000 packet sets from two different streams We see that both sets of 100 packets and 1000 packets from two
(s1 and s2) all passed the AD <= 2.492 criterion. different streams (s1 and s2) all passed the AD <= 2.492 criterion.
6.4.2. Perfas+ Results 6.4.2. Perfas+ Results
Section 11.4 of [RFC2330] suggests three possible measurement points Section 11.4 of [RFC2330] suggests three possible measurement points
to evaluate the Poisson distribution. The Perfas+ analysis uses to evaluate the Poisson distribution. The Perfas+ analysis uses
"wire times for the packets as recorded using a packet filter". "wire times for the packets as recorded using a packet filter".
However, due to limited access at the Perfas+ side of the test setup, However, due to limited access at the Perfas+ side of the test setup,
the captures were made after the Perfas+ streams traversed the the captures were made after the Perfas+ streams traversed the
production network, adding a small amount of unwanted delay variation production network, adding a small amount of unwanted delay variation
to the wire times (and possibly error due to packet loss). to the wire times (and possibly error due to packet loss).
The statistical summary for two Perfas+ streams is below: The statistical summary for two Perfas+ streams is below:
======================================= =======================================
> summary(a27pe$p1) > summary(a27pe$p1)
skipping to change at page 21, line 18 skipping to change at page 21, line 23
> summary(a27pe$p1) > summary(a27pe$p1)
Min. 1st Qu. Median Mean 3rd Qu. Max. Min. 1st Qu. Median Mean 3rd Qu. Max.
0.004 0.347 0.788 1.054 1.548 4.231 0.004 0.347 0.788 1.054 1.548 4.231
> summary(a27pe$p2) > summary(a27pe$p2)
Min. 1st Qu. Median Mean 3rd Qu. Max. Min. 1st Qu. Median Mean 3rd Qu. Max.
0.0010 0.2710 0.7080 0.9696 1.3740 7.1160 0.0010 0.2710 0.7080 0.9696 1.3740 7.1160
======================================= =======================================
We see that both the means are near the specified lambda = 1. We see that both of the means are near the specified lambda = 1.
The results of ADGoF tests for these two streams is shown below: The results of ADGoF tests for these two streams are shown below:
======================================= =======================================
> ad.test(a27pe$p1, pexp, 1 ) > ad.test(a27pe$p1, pexp, 1 )
Anderson-Darling GoF Test Anderson-Darling GoF Test
data: a27pe$p1 and pexp data: a27pe$p1 and pexp
AD = 1.1364, p-value = 0.2930 AD = 1.1364, p-value = 0.2930
alternative hypothesis: NA alternative hypothesis: NA
skipping to change at page 22, line 27 skipping to change at page 22, line 32
Anderson-Darling GoF Test Anderson-Darling GoF Test
data: a27pe$p2[101:193] and pexp data: a27pe$p2[101:193] and pexp
AD = 0.3381, p-value = 0.9068 AD = 0.3381, p-value = 0.9068
alternative hypothesis: NA alternative hypothesis: NA
> >
======================================= =======================================
We see that both 193, 100, and 93 packet sets from two different We see that sets of 193, 100, and 93 packets from two different
streams (p1 and p2) all passed the AD <= 2.492 criterion. streams (p1 and p2) all passed the AD <= 2.492 criterion.
6.4.3. Conclusions for Goodness-of-Fit 6.4.3. Conclusions for Goodness-of-Fit
Both NetProbe and Perfas+ implementations produce adequate Poisson Both NetProbe and Perfas+ implementations produce adequate Poisson
distributions according to the Anderson-Darling Goodness-of-Fit at distributions according to the Anderson-Darling Goodness-of-Fit at
the 5% significance (1-alpha = 0.05, or 95% confidence level). the 5% significance (1-alpha = 0.05, or 95% confidence level).
6.5. Implementation of Statistics for One-way Loss 6.5. Implementation of Statistics for One-Way Loss
We check which statistics were implemented, and report on those We check to see which statistics were implemented and report on those
facts, noting that Section 4 of [RFC2680] does not specify the facts, noting that Section 4 of [RFC2680] does not specify the
calculations exactly, and gives only some illustrative examples. calculations exactly and only gives some illustrative examples.
NetProbe Perfas NetProbe Perfas+
4.1. Type-P-One-way-Packet-Loss-Average yes yes Type-P-One-way-Packet-Loss-Average yes yes
(this is more commonly referred to as loss ratio) (this is more commonly referred
to as "loss ratio")
Implementation of RFC 2680 Section 4 Statistics
Implementation of Section 4 Statistics
We note that implementations refer to this metric as a loss ratio, We note that implementations refer to this metric as a loss ratio,
and this is an area for likely revision of the text to make it more and this is an area for likely revision of the text to make it more
consistent with wide-spread usage. consistent with widespread usage.
7. Conclusions for RFC 2680bis 7. Conclusions for a Revision of RFC 2680
This memo concludes that [RFC2680] should be advanced on the This memo concludes that [RFC2680] should be advanced on the
standards track, and recommends the following edits to improve the Standards Track and recommends the following edits to improve the
text (which are not deemed significant enough to affect maturity). text (which are not deemed significant enough to affect maturity).
o Revise Type-P-One-way-Packet-Loss-Ave to Type-P-One-way-Delay- o Revise Type-P-One-way-Packet-Loss-Ave to
Packet-Loss-Ratio . Type-P-One-way-Delay-Packet-Loss-Ratio.
o Regarding implementation of the loss delay threshold (section o Regarding implementation of the loss delay threshold
6.2), the assumption of post-processing is compliant, and the text (Section 6.2), the assumption of post-processing is compliant, and
of RFC 2680bis should be revised slightly to include this point. the text of the revision of RFC 2680 should be revised slightly to
include this point.
o The IETF has reached consensus on guidance for reporting metrics o The IETF has reached consensus on guidance for reporting metrics
in [RFC6703], and this memo should be referenced in RFC2680bis to [RFC6703], and this memo should be referenced in a revision of
incorporate recent experience where appropriate. RFC 2680 to incorporate recent experience where appropriate.
We note that there are at least two Errata on [RFC2680] and these We note that there are at least two errata for [RFC2680], and it
should be processed as part of the editing process. appears that these minor revisions should be incorporated in a
revision of RFC 2680.
We recognize the existence of BCP 170 [RFC6390] providing guidelines The authors that revise [RFC2680] should review all errata filed at
for development of drafts describing new performance metrics. the time the document is being written. They should not rely upon
However, the advancement of [RFC2680] represents fine-tuning of long- this document to indicate all relevant errata updates.
standing specifications based on experience that helped to formulate
BCP 170, and material that satisfies some of the requirements of We recognize the existence of BCP 170 [RFC6390], which provides
[RFC6390] can be found in other RFCs, such as the IPPM Framework guidelines for development of documents describing new performance
[RFC2330]. Thus, no specific changes to address BCP 170 guidelines metrics. However, the advancement of [RFC2680] represents fine-
are recommended for RFC 2680bis. tuning of long-standing specifications based on experience that
helped to formulate BCP 170, and material that satisfies some of the
requirements of [RFC6390] can be found in other RFCs, such as the
IPPM Framework [RFC2330]. Thus, no specific changes to address
BCP 170 guidelines are recommended for a revision of RFC 2680.
8. Security Considerations 8. Security Considerations
The security considerations that apply to any active measurement of The security considerations that apply to any active measurement of
live networks are relevant here as well. See [RFC4656] and live networks are relevant here as well. See [RFC4656] and
[RFC5357]. [RFC5357].
9. IANA Considerations 9. Acknowledgements
This memo makes no requests of IANA, and the authors hope that IANA
personnel will be able to use their valuable time in other worthwhile
pursuits.
10. Acknowledgements
The authors thank Lars Eggert for his continued encouragement to The authors thank Lars Eggert for his continued encouragement to
advance the IPPM metrics during his tenure as AD Advisor. advance the IPPM metrics during his tenure as AD Advisor.
Nicole Kowalski supplied the needed CPE router for the NetProbe side Nicole Kowalski supplied the needed Customer Premises Equipment (CPE)
of the test set-up, and graciously managed her testing in spite of router for the NetProbe side of the test setup and graciously managed
issues caused by dual-use of the router. Thanks Nicole! her testing in spite of issues caused by dual-use of the router.
Thanks, Nicole!
The "NetProbe Team" also acknowledges many useful discussions on The "NetProbe Team" also acknowledges many useful discussions on
statistical interpretation with Ganga Maguluri. statistical interpretation with Ganga Maguluri.
Constructive comments and helpful reviews where also provided by Bill Constructive comments and helpful reviews were also provided by Bill
Cerveny, Joachim Fabini, and Ann Cerveny. Cerveny, Joachim Fabini, and Ann Cerveny.
11. Appendix - Network Configuration and sample commands 10. Appendix - Network Configuration and Sample Commands
This Appendix provides some background information on the host This Appendix provides some background information on the host
configuration and sample tc commands for the "netem" network configuration and sample tc commands for the "netem" network
emulator, as described in Section 3 and Figure 1 in the body of this emulator, as described in Section 3 and Figure 1 of this memo. These
memo. These details are also applicable to the test plan in details are also applicable to the test plan in [RFC6808].
[RFC6808].
The host interface and configuration is shown below: The host interface and configuration are shown below. Due to the
limit of 72 characters per line, line breaks were added to the "tc"
commands in the output below.
[system@dell4-4 ~]$ su [system@dell4-4 ~]$ su
Password: Password:
[root@dell4-4 system]# service iptables save [root@dell4-4 system]# service iptables save
iptables: Saving firewall rules to /etc/sysconfig/iptables:[ OK ] iptables: Saving firewall rules to /etc/sysconfig/iptables:[ OK ]
[root@dell4-4 system]# service iptables stop [root@dell4-4 system]# service iptables stop
iptables: Flushing firewall rules: [ OK ] iptables: Flushing firewall rules: [ OK ]
iptables: Setting chains to policy ACCEPT: nat filter [ OK ] iptables: Setting chains to policy ACCEPT: nat filter [ OK ]
iptables: Unloading modules: [ OK ] iptables: Unloading modules: [ OK ]
[root@dell4-4 system]# brctl show [root@dell4-4 system]# brctl show
bridge name bridge id STP enabled interfaces bridge name bridge id STP enabled interfaces
virbr0 8000.000000000000 yes virbr0 8000.000000000000 yes
[root@dell4-4 system]# ifconfig eth1.300 0.0.0.0 promisc up [root@dell4-4 system]# ifconfig eth1.300 0.0.0.0 promisc up
[root@dell4-4 system]# ifconfig eth1.400 0.0.0.0 promisc up [root@dell4-4 system]# ifconfig eth1.400 0.0.0.0 promisc up
[root@dell4-4 system]# ifconfig eth2.400 0.0.0.0 promisc up [root@dell4-4 system]# ifconfig eth2.400 0.0.0.0 promisc up
[root@dell4-4 system]# ifconfig eth2.300 0.0.0.0 promisc up [root@dell4-4 system]# ifconfig eth2.300 0.0.0.0 promisc up
[root@dell4-4 system]# brctl addbr br300 [root@dell4-4 system]# brctl addbr br300
[root@dell4-4 system]# brctl addif br300 eth1.300 [root@dell4-4 system]# brctl addif br300 eth1.300
[root@dell4-4 system]# brctl addif br300 eth2.300 [root@dell4-4 system]# brctl addif br300 eth2.300
[root@dell4-4 system]# ifconfig br300 up [root@dell4-4 system]# ifconfig br300 up
[root@dell4-4 system]# brctl addbr br400 [root@dell4-4 system]# brctl addbr br400
[root@dell4-4 system]# brctl addif br400 eth1.400 [root@dell4-4 system]# brctl addif br400 eth1.400
[root@dell4-4 system]# brctl addif br400 eth2.400 [root@dell4-4 system]# brctl addif br400 eth2.400
[root@dell4-4 system]# ifconfig br400 up [root@dell4-4 system]# ifconfig br400 up
[root@dell4-4 system]# brctl show
bridge name bridge id STP enabled interfaces
br300 8000.0002b3109b8a no eth1.300
eth2.300
br400 8000.0002b3109b8a no eth1.400
eth2.400
virbr0 8000.000000000000 yes
[root@dell4-4 system]# brctl show [root@dell4-4 system]# brctl showmacs br300
bridge name bridge id STP enabled interfaces port no mac addr is local? ageing timer
br300 8000.0002b3109b8a no eth1.300 2 00:02:b3:10:9b:8a yes 0.00
eth2.300 1 00:02:b3:10:9b:99 yes 0.00
br400 8000.0002b3109b8a no eth1.400 1 00:02:b3:c4:c9:7a no 0.52
eth2.400 2 00:02:b3:cf:02:c6 no 0.52
virbr0 8000.000000000000 yes 2 00:0b:5f:54:de:81 no 0.01
[root@dell4-4 system]# brctl showmacs br400
port no mac addr is local? ageing timer
2 00:02:b3:10:9b:8a yes 0.00
1 00:02:b3:10:9b:99 yes 0.00
2 00:02:b3:c4:c9:7a no 0.60
1 00:02:b3:cf:02:c6 no 0.42
2 00:0b:5f:54:de:81 no 0.33
[root@dell4-4 system]# tc qdisc add dev eth1.300 root netem
delay 100ms
[root@dell4-4 system]# brctl showmacs br300 [root@dell4-4 system]# ifconfig eth1.200 0.0.0.0 promisc up
port no mac addr is local? ageing timer [root@dell4-4 system]# vconfig add eth1 100
2 00:02:b3:10:9b:8a yes 0.00 Added VLAN with VID == 100 to IF -:eth1:-
1 00:02:b3:10:9b:99 yes 0.00
1 00:02:b3:c4:c9:7a no 0.52
2 00:02:b3:cf:02:c6 no 0.52
2 00:0b:5f:54:de:81 no 0.01
[root@dell4-4 system]# brctl showmacs br400
port no mac addr is local? ageing timer
2 00:02:b3:10:9b:8a yes 0.00
1 00:02:b3:10:9b:99 yes 0.00
2 00:02:b3:c4:c9:7a no 0.60
1 00:02:b3:cf:02:c6 no 0.42
2 00:0b:5f:54:de:81 no 0.33
[root@dell4-4 system]# tc qdisc add dev eth1.300 root netem delay 100ms
[root@dell4-4 system]# ifconfig eth1.200 0.0.0.0 promisc up [root@dell4-4 system]# ifconfig eth1.100 0.0.0.0 promisc up
[root@dell4-4 system]# vconfig add eth1 100
Added VLAN with VID == 100 to IF -:eth1:-
[root@dell4-4 system]# ifconfig eth1.100 0.0.0.0 promisc up [root@dell4-4 system]# vconfig add eth2 100
Added VLAN with VID == 100 to IF -:eth2:-
[root@dell4-4 system]# vconfig add eth2 100 [root@dell4-4 system]# ifconfig eth2.100 0.0.0.0 promisc up
Added VLAN with VID == 100 to IF -:eth2:- [root@dell4-4 system]# ifconfig eth2.200 0.0.0.0 promisc up
[root@dell4-4 system]# brctl addbr br100
[root@dell4-4 system]# brctl addif br100 eth1.100
[root@dell4-4 system]# brctl addif br100 eth2.100
[root@dell4-4 system]# ifconfig br100 up
[root@dell4-4 system]# brctl addbr br200
[root@dell4-4 system]# brctl addif br200 eth1.200
[root@dell4-4 system]# brctl addif br200 eth2.200
[root@dell4-4 system]# ifconfig br200 up
[root@dell4-4 system]# brctl show
bridge name bridge id STP enabled interfaces
br100 8000.0002b3109b8a no eth1.100
eth2.100
br200 8000.0002b3109b8a no eth1.200
eth2.200
br300 8000.0002b3109b8a no eth1.300
eth2.300
br400 8000.0002b3109b8a no eth1.400
eth2.400
virbr0 8000.000000000000 yes
[root@dell4-4 system]# ifconfig eth2.100 0.0.0.0 promisc up [root@dell4-4 system]# brctl showmacs br100
[root@dell4-4 system]# ifconfig eth2.200 0.0.0.0 promisc up port no mac addr is local? ageing timer
[root@dell4-4 system]# brctl addbr br100 2 00:02:b3:10:9b:8a yes 0.00
[root@dell4-4 system]# brctl addif br100 eth1.100 1 00:02:b3:10:9b:99 yes 0.00
[root@dell4-4 system]# brctl addif br100 eth2.100 1 00:0a:e4:83:89:07 no 0.19
[root@dell4-4 system]# ifconfig br100 up 2 00:0b:5f:54:de:81 no 0.91
[root@dell4-4 system]# brctl addbr br200 2 00:e0:ed:0f:72:86 no 1.28
[root@dell4-4 system]# brctl addif br200 eth1.200 [root@dell4-4 system]# brctl showmacs br200
[root@dell4-4 system]# brctl addif br200 eth2.200 port no mac addr is local? ageing timer
[root@dell4-4 system]# ifconfig br200 up 2 00:02:b3:10:9b:8a yes 0.00
[root@dell4-4 system]# brctl show 1 00:02:b3:10:9b:99 yes 0.00
bridge name bridge id STP enabled interfaces 2 00:0a:e4:83:89:07 no 1.14
br100 8000.0002b3109b8a no eth1.100 2 00:0b:5f:54:de:81 no 1.87
eth2.100 1 00:e0:ed:0f:72:86 no 0.24
br200 8000.0002b3109b8a no eth1.200 [root@dell4-4 system]# tc qdisc add dev eth1.100 root netem
eth2.200 delay 100ms
br300 8000.0002b3109b8a no eth1.300 [root@dell4-4 system]#
eth2.300
br400 8000.0002b3109b8a no eth1.400
eth2.400
virbr0 8000.000000000000 yes
[root@dell4-4 system]# brctl showmacs br100
port no mac addr is local? ageing timer
2 00:02:b3:10:9b:8a yes 0.00
1 00:02:b3:10:9b:99 yes 0.00
1 00:0a:e4:83:89:07 no 0.19
2 00:0b:5f:54:de:81 no 0.91
2 00:e0:ed:0f:72:86 no 1.28
[root@dell4-4 system]# brctl showmacs br200
port no mac addr is local? ageing timer
2 00:02:b3:10:9b:8a yes 0.00
1 00:02:b3:10:9b:99 yes 0.00
2 00:0a:e4:83:89:07 no 1.14
2 00:0b:5f:54:de:81 no 1.87
1 00:e0:ed:0f:72:86 no 0.24
[root@dell4-4 system]# tc qdisc add dev eth1.100 root netem delay 100ms
[root@dell4-4 system]#
====================================================================== =====================================================================
Some sample tc command lines controlling netem and its impairments Some sample tc command lines controlling netem and its impairments
are given below. are given below.
tc qdisc add dev eth1.100 root netem loss 0% tc qdisc add dev eth1.100 root netem loss 0%
tc qdisc add dev eth1.200 root netem loss 0% tc qdisc add dev eth1.200 root netem loss 0%
tc qdisc add dev eth1.300 root netem loss 0% tc qdisc add dev eth1.300 root netem loss 0%
tc qdisc add dev eth1.400 root netem loss 0% tc qdisc add dev eth1.400 root netem loss 0%
Add delay and delay variation: Add delay and delay variation:
tc qdisc change dev eth1.100 root netem delay 100ms 50ms tc qdisc change dev eth1.100 root netem delay 100ms 50ms
tc qdisc change dev eth1.200 root netem delay 100ms 50ms tc qdisc change dev eth1.200 root netem delay 100ms 50ms
tc qdisc change dev eth1.300 root netem delay 100ms 50ms tc qdisc change dev eth1.300 root netem delay 100ms 50ms
tc qdisc change dev eth1.400 root netem delay 100ms 50ms tc qdisc change dev eth1.400 root netem delay 100ms 50ms
Add delay, delay variation, and loss: Add delay, delay variation, and loss:
tc qdisc change dev eth1 root netem delay 2000ms 1000ms loss 10% tc qdisc change dev eth1 root netem delay 2000ms 1000ms loss 10%
===================================================================== =====================================================================
12. References 11. References
12.1. Normative References 11.1. Normative References
[RFC2026] Bradner, S., "The Internet Standards Process -- Revision [RFC2026] Bradner, S., "The Internet Standards Process --
3", BCP 9, RFC 2026, October 1996. Revision 3", BCP 9, RFC 2026, October 1996.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997. Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis,
"Framework for IP Performance Metrics", RFC 2330, May "Framework for IP Performance Metrics", RFC 2330,
1998. May 1998.
[RFC2680] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way [RFC2680] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way
Packet Loss Metric for IPPM", RFC 2680, September 1999. Packet Loss Metric for IPPM", RFC 2680, September 1999.
[RFC3432] Raisanen, V., Grotefeld, G., and A. Morton, "Network [RFC3432] Raisanen, V., Grotefeld, G., and A. Morton, "Network
performance measurement with periodic streams", RFC 3432, performance measurement with periodic streams", RFC 3432,
November 2002. November 2002.
[RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M. [RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M.
Zekauskas, "A One-way Active Measurement Protocol Zekauskas, "A One-way Active Measurement Protocol
skipping to change at page 28, line 9 skipping to change at page 29, line 13
BCP 176, RFC 6576, March 2012. BCP 176, RFC 6576, March 2012.
[RFC6703] Morton, A., Ramachandran, G., and G. Maguluri, "Reporting [RFC6703] Morton, A., Ramachandran, G., and G. Maguluri, "Reporting
IP Network Performance Metrics: Different Points of View", IP Network Performance Metrics: Different Points of View",
RFC 6703, August 2012. RFC 6703, August 2012.
[RFC6808] Ciavattone, L., Geib, R., Morton, A., and M. Wieser, "Test [RFC6808] Ciavattone, L., Geib, R., Morton, A., and M. Wieser, "Test
Plan and Results Supporting Advancement of RFC 2679 on the Plan and Results Supporting Advancement of RFC 2679 on the
Standards Track", RFC 6808, December 2012. Standards Track", RFC 6808, December 2012.
12.2. Informative References 11.2. Informative References
[ADK] Scholz, F. and M. Stephens, "K-sample Anderson-Darling [ADK] Scholz, F. and M. Stephens, "K-Sample Anderson-Darling
Tests of Fit, for Continuous and Discrete cases", Tests of Fit, for Continuous and Discrete Cases",
University of Washington, Technical Report No. 81, May University of Washington, Technical Report No. 81,
1986. May 1986.
[Fedora] "http://fedoraproject.org/", . [ADV-METRICS]
Morton, A., "Lab Test Results for Advancing Metrics on the
Standards Track", Work in Progress, October 2010.
[I-D.morton-ippm-2680-bis] [FEDORA] "Fedora", <http://fedoraproject.org/>.
Almes, G., Zekauskas, M., and A. Morton, "A One-Way Loss
Metric for IPPM", draft-morton-ippm-2680-bis-02 (work in
progress), February 2014.
[I-D.morton-ippm-advance-metrics] [LOSS-METRIC]
Morton, A., "Lab Test Results for Advancing Metrics on the Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton,
Standards Track", draft-morton-ippm-advance-metrics-02 Ed., "A One-Way Loss Metric for IPPM", Work in Progress,
(work in progress), October 2010. July 2014.
[Perfas] Heidemann, C., "Qualitaet in IP-Netzen Messverfahren", [MII-TOOL]
published by ITG Fachgruppe, 2nd meeting 5.2.3 (NGN) Hinds, D., Becker, D., and B. Eckenfels, "Linux System
http://www.itg523.de/oeffentlich/01nov/ Administrator's Manual", February 2013,
Heidemann_QOS_Messverfahren.pdf , November 2001. <http://man7.org/linux/man-pages/man8/mii-tool.8.html>.
[RFC3931] Lau, J., Townsley, M., and I. Goyret, "Layer Two Tunneling [NETEM] Linux Foundation, "netem",
Protocol - Version 3 (L2TPv3)", RFC 3931, March 2005. <http://www.linuxfoundation.org/collaborate/workgroups/
networking/netem>.
[Radgof] Bellosta, C., "ADGofTest: Anderson-Darling Goodness-of-Fit [Perfas] Heidemann, C., "Qualitaet in IP-Netzen Messverfahren",
Test. R package version 0.3.", http://cran.r-project.org/ published by ITG Fachgruppe, 2nd meeting 5.2.3,
web/packages/ADGofTest/index.html, December 2011. November 2001, <www.itg523.de/oeffentlich/01nov/
Heidemann_QOS_Messverfahren.pdf>.
[Radk] Scholz, F., "adk: Anderson-Darling K-Sample Test and [RADGOF] Bellosta, C., "ADGofTest: Anderson-Darling Goodness-of-Fit
Combinations of Such Tests. R package version 1.0.", , Test. R package version 0.3.", R-Package Version 0.3,
2008. December 2011, <http://cran.r-project.org/web/packages/
ADGofTest/index.html>.
[Rtool] R Development Core Team, , "R: A language and environment [RADK] Scholz, F., "ADK: Anderson-Darling K-Sample Test and
for statistical computing. R Foundation for Statistical Combinations of Such Tests. R package version 1.0.", 2008.
Computing, Vienna, Austria. ISBN 3-900051-07-0, URL
http://www.R-project.org/", , 2011.
[WIPM] "AT&T Global IP Network", [RFC3931] Lau, J., Townsley, M., and I. Goyret, "Layer Two Tunneling
http://ipnetwork.bgtmo.ip.att.net/pws/index.html, 2012. Protocol - Version 3 (L2TPv3)", RFC 3931, March 2005.
[mii-tool] [RTOOL] R Development Core Team, "R: A Language and Environment
"http://man7.org/linux/man-pages/man8/mii-tool.8.html", . for Statistical Computing", ISBN 3-900051-07-0, 2014,
<http://www.R-project.org/>.
[netem] "http://www.linuxfoundation.org/collaborate/workgroups/ [WIPM] AT&T, "AT&T Global IP Network", 2014,
networking/netem", . <http://ipnetwork.bgtmo.ip.att.net/pws/index.html>.
Authors' Addresses Authors' Addresses
Len Ciavattone Len Ciavattone
AT&T Labs AT&T Labs
200 Laurel Avenue South 200 Laurel Avenue South
Middletown, NJ 07748 Middletown, NJ 07748
USA USA
Phone: +1 732 420 1239 Phone: +1 732 420 1239
Email: lencia@att.com EMail: lencia@att.com
Ruediger Geib Ruediger Geib
Deutsche Telekom Deutsche Telekom
Heinrich Hertz Str. 3-7 Heinrich Hertz Str. 3-7
Darmstadt 64295 Darmstadt 64295
Germany Germany
Phone: +49 6151 58 12747 Phone: +49 6151 58 12747
Email: Ruediger.Geib@telekom.de EMail: Ruediger.Geib@telekom.de
Al Morton Al Morton
AT&T Labs AT&T Labs
200 Laurel Avenue South 200 Laurel Avenue South
Middletown, NJ 07748 Middletown, NJ 07748
USA USA
Phone: +1 732 420 1571 Phone: +1 732 420 1571
Fax: +1 732 368 1192 Fax: +1 732 368 1192
Email: acmorton@att.com EMail: acmorton@att.com
URI: http://home.comcast.net/~acmacm/ URI: http://home.comcast.net/~acmacm/
Matthias Wieser Matthias Wieser
Technical University Darmstadt Technical University Darmstadt
Darmstadt Darmstadt
Germany Germany
Email: matthias_michael.wieser@stud.tu-darmstadt.de EMail: matthias_michael.wieser@stud.tu-darmstadt.de
 End of changes. 162 change blocks. 
469 lines changed or deleted 487 lines changed or added

This html diff was produced by rfcdiff 1.41. The latest version is available from http://tools.ietf.org/tools/rfcdiff/