draft-ietf-ippm-route-09.txt   draft-ietf-ippm-route-10.txt 
Network Working Group J. Alvarez-Hamelin Network Working Group J. Alvarez-Hamelin
Internet-Draft Universidad de Buenos Aires Internet-Draft Universidad de Buenos Aires
Updates: 2330 (if approved) A. Morton Updates: 2330 (if approved) A. Morton
Intended status: Standards Track AT&T Labs Intended status: Standards Track AT&T Labs
Expires: January 10, 2021 J. Fabini Expires: February 14, 2021 J. Fabini
TU Wien TU Wien
C. Pignataro C. Pignataro
Cisco Systems, Inc. Cisco Systems, Inc.
R. Geib R. Geib
Deutsche Telekom Deutsche Telekom
July 9, 2020 August 13, 2020
Advanced Unidirectional Route Assessment (AURA) Advanced Unidirectional Route Assessment (AURA)
draft-ietf-ippm-route-09 draft-ietf-ippm-route-10
Abstract Abstract
This memo introduces an advanced unidirectional route assessment This memo introduces an advanced unidirectional route assessment
(AURA) metric and associated measurement methodology, based on the IP (AURA) metric and associated measurement methodology, based on the IP
Performance Metrics (IPPM) Framework RFC 2330. This memo updates RFC Performance Metrics (IPPM) Framework RFC 2330. This memo updates RFC
2330 in the areas of path-related terminology and path description, 2330 in the areas of path-related terminology and path description,
primarily to include the possibility of parallel subpaths between a primarily to include the possibility of parallel subpaths between a
given Source and Destination pair, owing to the presence of multi- given Source and Destination pair, owing to the presence of multi-
path technologies. path technologies.
Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP
14[RFC2119] [RFC8174] when, and only when, they appear in all
capitals, as shown here.
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on January 10, 2021.
This Internet-Draft will expire on February 14, 2021.
Copyright Notice Copyright Notice
Copyright (c) 2020 IETF Trust and the persons identified as the Copyright (c) 2020 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1. Issues with Earlier Work to define Route . . . . . . . . 3 1.1. Issues with Earlier Work to Define a Route Metric . . . . 3
1.2. Requirements Language . . . . . . . . . . . . . . . . . . 4
2. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3. Route Metric Terms and Definitions . . . . . . . . . . . . . 5 3. Route Metric Specifications . . . . . . . . . . . . . . . . . 5
3.1. Formal Name . . . . . . . . . . . . . . . . . . . . . . . 6 3.1. Terms and Definitions . . . . . . . . . . . . . . . . . . 5
3.2. Parameters . . . . . . . . . . . . . . . . . . . . . . . 6 3.2. Formal Name . . . . . . . . . . . . . . . . . . . . . . . 6
3.3. Metric Definitions . . . . . . . . . . . . . . . . . . . 7 3.3. Parameters . . . . . . . . . . . . . . . . . . . . . . . 6
3.4. Related Round-Trip Delay and Loss Definitions . . . . . . 9 3.4. Metric Definitions . . . . . . . . . . . . . . . . . . . 7
3.5. Discussion . . . . . . . . . . . . . . . . . . . . . . . 9 3.5. Related Round-Trip Delay and Loss Definitions . . . . . . 9
3.6. Reporting the Metric . . . . . . . . . . . . . . . . . . 10 3.6. Discussion . . . . . . . . . . . . . . . . . . . . . . . 10
4. Route Assessment Methodologies . . . . . . . . . . . . . . . 10 3.7. Reporting the Metric . . . . . . . . . . . . . . . . . . 10
4. Route Assessment Methodologies . . . . . . . . . . . . . . . 11
4.1. Active Methodologies . . . . . . . . . . . . . . . . . . 11 4.1. Active Methodologies . . . . . . . . . . . . . . . . . . 11
4.1.1. Temporal Composition for Route Metrics . . . . . . . 13 4.1.1. Temporal Composition for Route Metrics . . . . . . . 13
4.1.2. Routing Class Identification . . . . . . . . . . . . 14 4.1.2. Routing Class Identification . . . . . . . . . . . . 15
4.1.3. Intermediate Observation Point Route Measurement . . 15 4.1.3. Intermediate Observation Point Route Measurement . . 16
4.2. Hybrid Methodologies . . . . . . . . . . . . . . . . . . 15 4.2. Hybrid Methodologies . . . . . . . . . . . . . . . . . . 16
4.3. Combining Different Methods . . . . . . . . . . . . . . . 16 4.3. Combining Different Methods . . . . . . . . . . . . . . . 17
5. Background on Round-Trip Delay Measurement Goals . . . . . . 17 5. Background on Round-Trip Delay Measurement Goals . . . . . . 17
6. RTD Measurements Statistics . . . . . . . . . . . . . . . . . 18 6. RTD Measurements Statistics . . . . . . . . . . . . . . . . . 18
7. Security Considerations . . . . . . . . . . . . . . . . . . . 20 7. Security Considerations . . . . . . . . . . . . . . . . . . . 20
8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21
9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 20 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 21
10. Appendix I MPLS Methods for Route Assessment . . . . . . . . 20 10. Appendix I MPLS Methods for Route Assessment . . . . . . . . 21
11. References . . . . . . . . . . . . . . . . . . . . . . . . . 21 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 22
11.1. Normative References . . . . . . . . . . . . . . . . . . 21 11.1. Normative References . . . . . . . . . . . . . . . . . . 22
11.2. Informative References . . . . . . . . . . . . . . . . . 24 11.2. Informative References . . . . . . . . . . . . . . . . . 24
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 25 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 26
1. Introduction 1. Introduction
The IETF IP Performance Metrics (IPPM) working group first created a The IETF IP Performance Metrics (IPPM) working group first created a
framework for metric development in [RFC2330]. This framework has framework for metric development in [RFC2330]. This framework has
stood the test of time and enabled development of many fundamental stood the test of time and enabled development of many fundamental
metrics. It has been updated in the area of metric composition metrics. It has been updated in the area of metric composition
[RFC5835], and in several areas related to active stream measurement [RFC5835], and in several areas related to active stream measurement
of modern networks with reactive properties [RFC7312]. of modern networks with reactive properties [RFC7312].
The [RFC2330] framework motivated the development of "performance and The [RFC2330] framework motivated the development of "performance and
reliability metrics for paths through the Internet," and Section 5 of reliability metrics for paths through the Internet," and Section 5 of
[RFC2330] defines terms that support description of a path under [RFC2330] defines terms that support description of a path under
test. However, metrics for assessment of path components and related test. However, metrics for assessment of paths and related
performance aspects had not been attempted in IPPM when the [RFC2330] performance aspects had not been attempted in IPPM when the [RFC2330]
framework was written. framework was written.
This memo takes up the route measurement challenge and specifies a This memo takes up the route measurement challenge and specifies a
new route metric, two practical frameworks for methods of measurement new route metric, two practical frameworks for methods of measurement
(using either active or hybrid active-passive methods [RFC7799]), and (using either active or hybrid active-passive methods [RFC7799]), and
Round-Trip Delay and link information discovery using the results of Round-Trip Delay and link information discovery using the results of
measurements. All route measurements are limited by the willingness measurements. All route measurements are limited by the willingness
of hosts along the path to be discovered, to cooperate with the of hosts along the path to be discovered, to cooperate with the
methods used, or to recognize that the measurement operation is methods used, or to recognize that the measurement operation is
taking place (such as when tunnels are present). taking place (such as when tunnels are present).
1.1. Issues with Earlier Work to define Route 1.1. Issues with Earlier Work to Define a Route Metric
Section 7 of [RFC2330] presented a simple example of a "route" metric Section 7 of [RFC2330] presented a simple example of a "route" metric
along with several other examples. The example is reproduced below along with several other examples. The example is reproduced below
(where the reference is to Section 5 of [RFC2330]): (where the reference is to Section 5 of [RFC2330]):
"route: The path, as defined in Section 5, from A to B at a given "route: The path, as defined in Section 5, from A to B at a given
time." time."
This example provides a starting point to develop a more complete This example provides a starting point to develop a more complete
definition of route. Areas needing clarification include: definition of route. Areas needing clarification include:
Time: In practice, the route will be assessed over a time interval, Time: In practice, the route will be assessed over a time interval,
because active path detection methods like [PT] rely on TTL limits because active path detection methods like Paris Traceroute [PT]
for their operation and cannot accomplish discovery of all hosts rely on hop limits for their operation and cannot accomplish
using a single packet. discovery of all hosts using a single packet.
Type-P: The legacy route definition lacks the option to cater for Type-P: The legacy route definition lacks the option to cater for
packet-dependent routing. In this memo, we assess the route for a packet-dependent routing. In this memo, we assess the route for a
specific packet of Type-P, and reflect this in the metric specific packet of Type-P, and reflect this in the metric
definition. The methods of measurement determine the specific definition. The methods of measurement determine the specific
Type-P used. Type-P used.
Parallel Paths: Parallel paths are a reality of the Internet and a Parallel Paths: Parallel paths are a reality of the Internet and a
strength of advanced route assessment methods, so the metric must strength of advanced route assessment methods, so the metric must
acknowledge this possibility. Use of Equal Cost Multi-Path (ECMP) acknowledge this possibility. Use of Equal Cost Multi-Path (ECMP)
and Unequal Cost Multi-Path (UCMP) technologies are common sources and Unequal Cost Multi-Path (UCMP) technologies are common sources
of parallel subpaths. of parallel subpaths.
Cloud Subpath: May contain hosts that do not decrement TTL or Hop Cloud Subpath: May contain hosts that do not decrement hop limit,
Limit, but may have two or more exchange links connecting but may have two or more exchange links connecting "discoverable"
"discoverable" hosts or routers. Parallel subpaths contained hosts or routers. Parallel subpaths contained within clouds
within clouds cannot be discovered. The assessment methods only cannot be discovered. The assessment methods only discover hosts
discover hosts or routers on the path that decrement TTL or Hop or routers on the path that decrement hop limit, or cooperate with
Count, or cooperate with interrogation protocols. The presence of interrogation protocols. The presence of tunnels and nested
tunnels and nested tunnels further complicate assessment by hiding tunnels further complicate assessment by hiding hops.
hops.
Hop: Although the [RFC2330] definition of a hop was a link-host Hop: Although the [RFC2330] definition of a hop was a link-host
pair, only hosts that are discoverable or have the capability to pair, only hosts that are discoverable or have the capability to
cooperate with interrogation protocols where link information may cooperate with interrogation protocols where link information may
be exposed. be exposed.
The refined definition of Route metrics begins in the sections that The refined definition of Route metrics begins in the sections that
follow. follow.
1.2. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP
14[RFC2119] [RFC8174] when, and only when, they appear in all
capitals, as shown here.
2. Scope 2. Scope
The purpose of this memo is to add new route metrics and methods of The purpose of this memo is to add new route metrics and methods of
measurement to the existing set of IPPM metrics. measurement to the existing set of IPPM metrics.
The scope is to define route metrics that can identify the path taken The scope is to define route metrics that can identify the path taken
by a packet or a flow traversing the Internet between two hosts. by a packet or a flow traversing the Internet between two hosts.
Although primarily intended for hosts communicating on the Internet Although primarily intended for hosts communicating on the Internet,
with IP, the definitions and metrics are constructed to be applicable the definitions and metrics are constructed to be applicable to other
to other network domains, if desired. The methods of measurement to network domains, if desired. The methods of measurement to assess
assess the path may not be able to discover all hosts comprising the the path may not be able to discover all hosts comprising the path,
path, but such omissions are often deterministic and explainable but such omissions are often deterministic and explainable sources of
sources of error. error.
Also, to specify a framework for active methods of measurement which This memo also specifies a framework for active methods of
use the techniques described in [PT] at a minimum, and a framework measurement which uses the techniques described in [PT], as well as a
for hybrid active-passive methods of measurement, such as the Hybrid framework for hybrid active-passive methods of measurement such as
Type I method [RFC7799] described in the Hybrid Type I method [RFC7799] described in
[I-D.ietf-ippm-ioam-data](intended only for single administrative [I-D.ietf-ippm-ioam-data]. Methods using [I-D.ietf-ippm-ioam-data]
domains), which do not rely on ICMP and provide a protocol for are intended only for single administrative domains that provide a
explicit interrogation of nodes on a path. Combinations of active protocol for explicit interrogation of nodes on a path. Combinations
methods and hybrid active-passive methods are also in-scope. of active methods and hybrid active-passive methods are also in-
scope.
Further, this memo provides additional analysis of the round-trip Further, this memo provides additional analysis of the round-trip
delay measurements made possible by the methods, in an effort to delay measurements made possible by the methods, in an effort to
discover more details about the path, such as the link technology in discover more details about the path, such as the link technology in
use. use.
This memo updates Section 5 of [RFC2330] in the areas of path-related This memo updates Section 5 of [RFC2330] in the areas of path-related
terminology and path description, primarily to include the terminology and path description, primarily to include the
possibility of parallel subpaths between a given Source and possibility of parallel subpaths between a given Source and
Destination address pair (possibly resulting from Equal Cost Multi- Destination address pair (possibly resulting from Equal Cost Multi-
skipping to change at page 5, line 24 skipping to change at page 5, line 27
to assess the reverse path from any host on the path to the host to assess the reverse path from any host on the path to the host
attempting the path measurement. The reverse path contribution to attempting the path measurement. The reverse path contribution to
delay will be that experienced by ICMP packets (in active methods), delay will be that experienced by ICMP packets (in active methods),
and may be different from delays experienced by UDP or TCP packets. and may be different from delays experienced by UDP or TCP packets.
Also, the round trip delay will include an unknown contribution of Also, the round trip delay will include an unknown contribution of
processing time at the host that generates the ICMP response. processing time at the host that generates the ICMP response.
Therefore, the ICMP-based active methods are not supposed to yield Therefore, the ICMP-based active methods are not supposed to yield
accurate, reproducible estimations of the Round-Trip Delay that UDP accurate, reproducible estimations of the Round-Trip Delay that UDP
or TCP packets will experience. or TCP packets will experience.
3. Route Metric Terms and Definitions 3. Route Metric Specifications
This section sets requirements for the following components to This section sets requirements for the components of the Route
support the Route Metric: Metric.
3.1. Terms and Definitions
Host A Host as defined in [RFC2330] (a computer capable of IP Host A Host as defined in [RFC2330] (a computer capable of IP
communication, includes routers), a.k.a. RFC 2330 Host. communication, includes routers), a.k.a. RFC 2330 Host.
Node A Node is any network function on the path capable of IP-layer Node A Node is any network function on the path capable of IP-layer
Communication, includes RFC 2330 Hosts. Communication, includes RFC 2330 Hosts.
Node Identity The unique address for Nodes communicating within the Node Identity The unique address for Nodes communicating within the
network domain. For Nodes communicating on the Internet with IP, network domain. For Nodes communicating on the Internet with IP,
it is the globally routable IP address(es) which the Node uses it is the globally routable IP address which the Node uses when
when communicating with other Nodes under normal or error communicating with other Nodes under normal or error conditions.
conditions. The Node Identity revealed (and its connection to a The Node Identity revealed (and its connection to a Node Name
Node Name through reverse DNS) determines whether interfaces to through reverse DNS) determines whether interfaces to parallel
parallel links can be associated with a single Node, or appear to links can be associated with a single Node, or appear to identify
identify unique Nodes. unique Nodes.
Discoverable Node Nodes that convey their Node Identity according to Discoverable Node Nodes that convey their Node Identity according to
the requirements of their network domain, such as when error the requirements of their network domain, such as when error
conditions are detected by that Node. For Nodes communicating conditions are detected by that Node. For Nodes communicating
with IP packets, compliance with Section 3.2.2.4 of [RFC1122] when with IP packets, compliance with Section 3.2.2.4 of [RFC1122] when
discarding a packet due to TTL or Hop Limit Exceeded condition, discarding a packet due to TTL or Hop Limit Exceeded condition,
MUST result in sending the corresponding Time Exceeded message MUST result in sending the corresponding Time Exceeded message
(containing a form of Node identity) to the source. This (containing a form of Node identity) to the source. This
requirement is also consistent with section 5.3.1 of [RFC1812] for requirement is also consistent with section 5.3.1 of [RFC1812] for
routers. routers.
Cooperating Node Nodes that MUST respond to direct queries for their Cooperating Node Nodes that respond to direct queries for their Node
Node identity as part of a previously agreed and established identity as part of a previously agreed and established
interrogation protocol. Nodes SHOULD also provide information interrogation protocol. Nodes SHOULD also provide information
such as arrival/departure interface identification, arrival such as arrival/departure interface identification, arrival
timestamp, and any relevant information about the Node or specific timestamp, and any relevant information about the Node or specific
link which delivered the query to the Node. link which delivered the query to the Node.
Hop A Hop MUST contain a Node Identity, and MAY contain arrival and/ Hop specification A Hop specification MUST contain a Node Identity,
or departure interface identification, round trip delay, and an and MAY contain arrival and/or departure interface identification,
arrival timestamp. round trip delay, and an arrival timestamp.
Routing Class A route that treats equally a class C of different Routing Class A route that treats equally a class of different types
types of packets (unrelated to address classes of the past). of packets, designated "C" (unrelated to address classes of the
Knowledge of such a class allows any one of the types of packets past) [RFC2330] [RFC8468]. Knowledge of such a class allows any
within that class to be used for subsequent measurement of the one of the types of packets within that class to be used for
route. subsequent measurement of the route. The designator "class C" is
used for historical reasons, see [RFC2330].
3.1. Formal Name 3.2. Formal Name
Type-P-Route-Ensemble-Method-Variant, abbreviated as Route Ensemble. The formal name of the metric is:
Type-P-Route-Ensemble-Method-Variant
abbreviated as Route Ensemble.
Note that Type-P depends heavily on the chosen method and variant. Note that Type-P depends heavily on the chosen method and variant.
3.2. Parameters 3.3. Parameters
This section lists the REQUIRED input factors to specify a Route This section lists the REQUIRED input factors to define and measure a
metric. Route metric, as specified in this memo.
o Src, the address of a Node (such as the globally routable IP o Src, the address of a Node (such as the globally routable IP
address). address).
o Dst, the address of a Node (such as the globally routable IP o Dst, the address of a Node (such as the globally routable IP
address). address).
o i, the limit on the number of Hops a specific packet may visit as o i, the limit on the number of Hops a specific packet may visit as
it traverses from the Node at Src to the Node at Dst (such as the it traverses from the Node at Src to the Node at Dst (such as the
TTL or Hop Limit). TTL or Hop Limit).
skipping to change at page 7, line 32 skipping to change at page 7, line 39
o flow, the stream of packets with the same n-tuple of designated o flow, the stream of packets with the same n-tuple of designated
header fields that (when held constant) result in identical header fields that (when held constant) result in identical
treatment in a multi-path decision (such as the decision taken in treatment in a multi-path decision (such as the decision taken in
load balancing). Note: The IPv6 flow label MAY be included in the load balancing). Note: The IPv6 flow label MAY be included in the
flow definition if the MP(Src) is a Tunnel End Point (TEP) flow definition if the MP(Src) is a Tunnel End Point (TEP)
complying with [RFC6438] guidelines. complying with [RFC6438] guidelines.
o Type-P, the complete description of the packets for which this o Type-P, the complete description of the packets for which this
assessment applies (including the flow-defining fields). assessment applies (including the flow-defining fields).
3.3. Metric Definitions 3.4. Metric Definitions
This section defines the REQUIRED measurement components of the Route This section defines the REQUIRED measurement components of the Route
metrics (unless otherwise indicated): metrics (unless otherwise indicated):
M, the total number of packets sent between T0 and Tf. M, the total number of packets sent between T0 and Tf.
N, the smallest value of i needed for a packet to be received at Dst N, the smallest value of i needed for a packet to be received at Dst
(sent between T0 and Tf). (sent between T0 and Tf).
Nmax, the largest value of i needed for a packet to be received at Nmax, the largest value of i needed for a packet to be received at
Dst (sent between T0 and Tf). Nmax may be equal to N. Dst (sent between T0 and Tf). Nmax may be equal to N.
Next define a *singleton* definition for a Hop on the path, with Next define a *singleton* definition for a Node on the path, with
sufficient indexes to identify all Hops identified in a measurement sufficient indexes to identify all Nodes identified in a measurement
interval. interval (where *singleton* is part of the IPPM Framework [RFC2330]).
A Hop, designated h(i,j), the IP address and/or identity of A Hop Specification, designated h(i,j), the IP address and/or
Discoverable Nodes (or Cooperating Nodes) that are i hops away from identity of Discoverable Nodes (or Cooperating Nodes) that are i hops
the Node with address = Src and part of Route j during the away from the Node with address = Src and part of Route j during the
measurement interval, T0 to Tf. As defined here, a Hop singleton measurement interval, T0 to Tf. As defined here, a Hop singleton
measurement MUST contain a Node Identity, hid(i,j), and MAY contain measurement MUST contain a Node Identity, hid(i,j), and MAY contain
one or more of the following attributes: one or more of the following attributes:
o a(i,j) Arrival Interface ID (e.g., when [RFC5837] is supported) o a(i,j) Arrival Interface ID (e.g., when [RFC5837] is supported)
o d(i,j) Departure Interface ID (e.g., when [RFC5837] is supported) o d(i,j) Departure Interface ID (e.g., when [RFC5837] is supported)
o t(i,j) Arrival Timestamp (where t(i,j) is ideally supplied by the o t(i,j) Arrival Timestamp, where t(i,j) is ideally supplied by the
Hop, or approximated from the sending time of the packet that Hop. (Note that t(i,j) might be approximated from the sending time
revealed the Hop) of the packet that revealed the Hop, e.g., when the round trip
response time is available and divided by 2.)
o Measurements of Round-Trip Delay (for each packet that reveals the o Measurements of Round-Trip Delay (for each packet that reveals the
same Node Identity and flow attributes, then this attribute is same Node Identity and flow attributes, then this attribute is
computed, see next section) computed, see next section)
Node Identities and related information can be ordered by their Node Identities and related information can be ordered by their
distance from the Node with address Src in Hops h(i,j). Based on distance from the Node with address Src in Hops h(i,j). Based on
this, two forms of Routes are distinguished: this, two forms of Routes are distinguished:
A Route Ensemble is defined as the combination of all routes A Route Ensemble is defined as the combination of all routes
skipping to change at page 9, line 29 skipping to change at page 9, line 38
Note that some h(i,j) may be empty (null) in the case that systems do Note that some h(i,j) may be empty (null) in the case that systems do
not reply (not discoverable, or not cooperating). not reply (not discoverable, or not cooperating).
h(i-1,j) and h(i,j) are the Hops on the same Member Route one hop h(i-1,j) and h(i,j) are the Hops on the same Member Route one hop
away from each other. away from each other.
Hop h(i,j) may be identical with h(k,l) for i!=k and j!=l ; which Hop h(i,j) may be identical with h(k,l) for i!=k and j!=l ; which
means there may be portions shared among different Member Routes means there may be portions shared among different Member Routes
(parts of Member Routes may overlap). (parts of Member Routes may overlap).
3.4. Related Round-Trip Delay and Loss Definitions 3.5. Related Round-Trip Delay and Loss Definitions
RTD(i,j,T) is defined as a singleton of the [RFC2681] Round-Trip RTD(i,j,T) is defined as a singleton of the [RFC2681] Round-Trip
Delay between the Node with address = Src and the Node at Hop h(i,j) Delay between the Node with address = Src and the Node at Hop h(i,j)
at time T. at time T.
RTL(i,j,T) is defined as a singleton of the [RFC6673] Round-trip Loss RTL(i,j,T) is defined as a singleton of the [RFC6673] Round-trip Loss
between the Node with address = Src and the Node at Hop h(i,j) at between the Node with address = Src and the Node at Hop h(i,j) at
time T. time T.
3.5. Discussion 3.6. Discussion
Depending on the way that Node Identity is revealed, it may be Depending on the way that Node Identity is revealed, it may be
difficult to determine parallel subpaths between the same pair of difficult to determine parallel subpaths between the same pair of
Nodes (i.e. multiple parallel links). It is easier to detect Nodes (i.e. multiple parallel links). It is easier to detect
parallel subpaths involving different Nodes. parallel subpaths involving different Nodes.
o If a pair of discovered Nodes identify two different addresses, o If a pair of discovered Nodes identify two different addresses (IP
then they will appear to be different Nodes. or not), then they will appear to be different Nodes. See item
below.
o If a pair of discovered Nodes identify two different IP addresses, o If a pair of discovered Nodes identify two different IP addresses,
and the IP addresses resolve to the same Node name (in the DNS), and the IP addresses resolve to the same Node name (in the DNS),
then they will appear to be the same Nodes. then they will appear to be the same Nodes.
o If a discovered Node always replies using the same network o If a discovered Node always replies using the same network
address, regardless of the interface a packet arrives on, then address, regardless of the interface a packet arrives on, then
multiple parallel links cannot be detected in that network domain. multiple parallel links cannot be detected in that network domain.
This condition may apply to traceroute-style methods, but may not This condition may apply to traceroute-style methods, but may not
apply to other hybrid methods based on In-situ Operations, apply to other hybrid methods based on In-situ Operations,
Administration, and Maintenance (IOAM). Administration, and Maintenance (IOAM). For example, if the
[RFC5837] ICMP extension mechanism is implemented, then parallel
links can be detected with the discovery traceroute-style methods.
o If parallel links between routers are aggregated below the IP o If parallel links between routers are aggregated below the IP
layer, then from Node point of view, all these links share the layer, then from Node point of view, all these links share the
same pair of IP addresses. The existence of these parallel links same pair of IP addresses. The existence of these parallel links
can't be detected at IP layer. This applies to other network can't be detected at the IP layer. This applies to other network
domains with layers below them, as well. This condition may apply domains with layers below them, as well. This condition may apply
to traceroute-style methods, but may not apply to other hybrid to traceroute-style methods, but may not apply to other hybrid
methods based on IOAM. methods based on IOAM.
When a route assessment employs IP packets (for example), the reality When a route assessment employs IP packets (for example), the reality
of flow assignment to parallel subpaths involves layers above IP. of flow assignment to parallel subpaths involves layers above IP.
Thus, the measured Route Ensemble is applicable to IP and higher Thus, the measured Route Ensemble is applicable to IP and higher
layers (as described in the methodology's packet of Type-P and flow layers (as described in the methodology's packet of Type-P and flow
parameters). parameters).
3.6. Reporting the Metric 3.7. Reporting the Metric
An Information Model and an XML Data Model for Storing Traceroute An Information Model and an XML Data Model for Storing Traceroute
Measurements is available in [RFC5388]. The measured information at Measurements is available in [RFC5388]. The measured information at
each hop includes four pieces of information: a one-dimensional hop each hop includes four pieces of information: a one-dimensional hop
index, Node symbolic address, Node IP address, and RTD for each index, Node symbolic address, Node IP address, and RTD for each
response. response.
The description of Hop information that may be collected according to The description of Hop information that may be collected according to
this memo covers more dimensions, as defined in Section 3.3 above. this memo covers more dimensions, as defined in Section 3.4 above.
For example, the Hop index is two-dimensional to capture the For example, the Hop index is two-dimensional to capture the
complexity of a Route Ensemble, and it contains corresponding Node complexity of a Route Ensemble, and it contains corresponding Node
identities at a minimum. The models need to be expanded to include identities at a minimum. The models need to be expanded to include
these features, as well as Arrival Interface ID, Departure Interface these features, as well as Arrival Interface ID, Departure Interface
ID, and Arrival Timestamp, when available. The original sending ID, and Arrival Timestamp, when available. The original sending
Timestamp from the Src Node anchors a particular measurement in time. Timestamp from the Src Node anchors a particular measurement in time.
4. Route Assessment Methodologies 4. Route Assessment Methodologies
There are two classes of methods described in this section, active There are two classes of methods described in this section, active
skipping to change at page 11, line 13 skipping to change at page 11, line 29
follow. follow.
4.1. Active Methodologies 4.1. Active Methodologies
This section describes the method employed by current open source This section describes the method employed by current open source
tools, thereby providing a practical framework for further advanced tools, thereby providing a practical framework for further advanced
techniques to be included as method variants. This method is techniques to be included as method variants. This method is
applicable for use across multiple administrative domains. applicable for use across multiple administrative domains.
Internet routing is complex because it depends on the policies of Internet routing is complex because it depends on the policies of
thousands of Autonomous Systems (AS). While most of the routers thousands of Autonomous Systems (AS). Most routers perform load
perform load balancing on flows using Equal Cost Multiple Path balancing on flows using a form of Equal Cost Multiple Path (ECMP).
(ECMP), a few still divide the workload through packet-based [RFC2991] describes a number of flow-based or hashed approaches
techniques. The former scenario is defined according to [RFC2991], (e.g., Modulo-N Hash, Hash-Threshold, Highest Random Weight (HRW)),
while the latter generates a round-robin scheme to deliver every new and makes some good suggestions. Flow-based ECMP avoids increased
outgoing packet. ECMP uses a hashing function to ensure that every packet delay variation and possibly overwhelming levels of packet
packet of a flow is delivered by the same path, and this avoids reordering in flows.
increasing the packet delay variation and possibly producing
overwhelming packet reordering in TCP flows. A few routers still divide the workload through packet-based
techniques, such as a round-robin scheme to distribute every new
outgoing packet to multiple links, as explained in [RFC2991]. The
methods described in this section assume flow-based ECMP.
Taking into account that Internet protocol was designed under the Taking into account that Internet protocol was designed under the
"end-to-end" principle, the IP payload and its header do not provide "end-to-end" principle, the IP payload and its header do not provide
any information about the routes or path necessary to reach some any information about the routes or path necessary to reach some
destination. For this reason, the popular tool traceroute was destination. For this reason, the popular tool traceroute was
developed to gather the IP addresses of each hop along a path using developed to gather the IP addresses of each hop along a path using
the ICMP protocol [RFC0792]. Traceroute also measures RTD from each the ICMP protocol [RFC0792]. Traceroute also measures RTD from each
hop. However, the growing complexity of the Internet makes it more hop. However, the growing complexity of the Internet makes it more
challenging to develop an accurate traceroute implementation. For challenging to develop an accurate traceroute implementation. For
instance, the early traceroute tools would be inaccurate in the instance, the early traceroute tools would be inaccurate in the
current network, mainly because they were not designed to retain a current network, mainly because they were not designed to retain a
flow state. However, evolved traceroute tools, such as Paris- flow state. However, evolved traceroute tools, such as Paris-
traceroute [PT] [MLB] and Scamper [SCAMPER], expect to encounter ECMP traceroute [PT] [MLB] and Scamper [SCAMPER], expect to encounter ECMP
and achieve more accurate results when they do, where Scamper ensures and achieve more accurate results when they do, where Scamper ensures
traceroute packets will follow the same path in 98% of traceroute packets will follow the same path in 98% of
cases[SCAMPER]. cases[SCAMPER].
Today's traceroute tools send Type-P of packets, either ICMP, UDP, or Today's traceroute tools send Type-P of packets, either ICMP, UDP, or
TCP. UDP and TCP are used when a particular characteristic needs to TCP. UDP and TCP are used when a particular characteristic needs to
be verified, such as filtering or traffic shaping on specific ports be verified, such as filtering or traffic shaping on specific ports
(i.e., services). [SCAMPER] supports IPv6 traceroute measurements, (i.e., services). UDP and TCP traceroute are also used when ICMP
keeping the FlowLabel constant in all packets. responses are not received. [SCAMPER] supports IPv6 traceroute
measurements, keeping the FlowLabel constant in all packets.
Paris-traceroute allows its users to measure RTD in every hop of the Paris-traceroute allows its users to measure the RTD to every Node of
path for a particular flow. Furthermore, either Paris-traceroute or the path for a particular flow. Furthermore, either Paris-traceroute
Scamper is capable of unveiling the many available paths between a or Scamper is capable of unveiling the many available paths between a
source and destination (which are visible to this method). This task source and destination (which are visible to active methods). This
is accomplished by repeating complete traceroute measurements with task is accomplished by repeating complete traceroute measurements
different flow parameters for each measurement; Paris-traceroute with different flow parameters for each measurement; Paris-traceroute
provides "exhaustive" mode while scamper provides "tracelb" (stands provides "exhaustive" mode while scamper provides "tracelb" (stands
for traceroute load balance). The Framework for IP Performance for traceroute load balance). The Framework for IP Performance
Metrics (IPPM) ([RFC2330] updated by[RFC7312]) has the flexibility to Metrics (IPPM) ([RFC2330] updated by[RFC7312]) has the flexibility to
require that the Round-Trip Delay measurement [RFC2681] uses packets require that the Round-Trip Delay measurement [RFC2681] uses packets
with the constraints to assure that all packets in a single with the constraints to assure that all packets in a single
measurement appear as the same flow. This flexibility covers ICMP, measurement appear as the same flow. This flexibility covers ICMP,
UDP, and TCP. The accompanying methodology of [RFC2681] needs to be UDP, and TCP. The accompanying methodology of [RFC2681] needs to be
expanded to report the sequential hop identifiers along with RTD expanded to report the sequential hop identifiers along with RTD
measurements, but no new metric definition is needed. measurements, but no new metric definition is needed.
The advanced route assessment methods used in Paris-traceroute [PT] The advanced route assessment methods used in Paris-traceroute [PT]
keep the critical fields constant for every packet to maintain the keep the critical fields constant for every packet to maintain the
appearance of the same flow. In IPv6, it is sufficient to be routed appearance of the same flow. When considering IPv6 headers, it is
identically if the IP source and destination addresses and the necessary to ensure that the IP source and destination addresses and
FlowLabel are constant, see [RFC6437]. In IPv4, certain fields of the FlowLabel are constant (but note that many routers ignore the
the IP header and the first four bytes of the IP payload should FlowLabel field at this time), see [RFC6437]. Use of IPv6 Extension
remain constant in a flow. In the IPv4 header, the IP source and Headers may add critical fields, and SHOULD be avoided. In IPv4,
destination addresses, protocol number, and Diffserv fields identify certain fields of the IP header and the first four bytes of the IP
flows. The first four payload bytes include the UDP and TCP ports, payload should remain constant in a flow. In the IPv4 header, the IP
and the ICMP type, code, and checksum fields. source and destination addresses, protocol number, and Diffserv
fields identify flows. The first four payload bytes include the UDP
and TCP ports, and the ICMP type, code, and checksum fields.
Maintaining a constant ICMP checksum in IPv4 is most challenging, as Maintaining a constant ICMP checksum in IPv4 is most challenging, as
the ICMP sequence number or identifier fields will usually change for the ICMP sequence number or identifier fields will usually change for
different probes of the same path. Probes should use arbitrary bytes different probes of the same path. Probes should use arbitrary bytes
in the ICMP data field to offset changes to sequence number and in the ICMP data field to offset changes to sequence number and
identifier, thus keeping the checksum constant. identifier, thus keeping the checksum constant.
Finally, it is also essential to route the resulting ICMP Time Finally, it is also essential to route the resulting ICMP Time
Exceeded messages along a consistent path. In IPv6, the fields above Exceeded messages along a consistent path. In IPv6, the fields above
are sufficient. In IPv4, the ICMP Time Exceeded message will contain are sufficient. In IPv4, the ICMP Time Exceeded message will contain
skipping to change at page 12, line 48 skipping to change at page 13, line 24
should be[PT]: should be[PT]:
o TCP case: For IPv4, the fields Src, Dst, port-Src, port_Dst, o TCP case: For IPv4, the fields Src, Dst, port-Src, port_Dst,
sequence number, and Diffserv Field SHOULD be the same. For IPv6, sequence number, and Diffserv Field SHOULD be the same. For IPv6,
the field FlowLabel, Src and Dst SHOULD be the same. the field FlowLabel, Src and Dst SHOULD be the same.
o UDP case: For IPv4, the fields Src, Dst, port-Src, port-Dst, o UDP case: For IPv4, the fields Src, Dst, port-Src, port-Dst,
Diffserv should be the same, and the UDP-checksum SHOULD change to Diffserv should be the same, and the UDP-checksum SHOULD change to
keep the IP checksum of the ICMP time exceeded reply constant. keep the IP checksum of the ICMP time exceeded reply constant.
Then, the data length should be fixed, and the data field is used Then, the data length should be fixed, and the data field is used
to fixing it (consider that ICMP checksum uses its data field, to make it so (consider that ICMP checksum uses its data field,
which contains the original IP header plus 8 bytes of UDP, where which contains the original IP header plus 8 bytes of UDP, where
TTL, IP identification, IP checksum, and UDP checksum changes). TTL, IP identification, IP checksum, and UDP checksum changes).
For IPv6, the field FlowLabel, and Source and Destination For IPv6, the field FlowLabel, and Source and Destination
addresses SHOULD be the same. addresses SHOULD be the same.
o ICMP case: For IPv4, the Data field SHOULD compensate variations o ICMP case: For IPv4, the Data field SHOULD compensate variations
on TTL or Hop Limit, IP identification, and IP checksum for every on TTL or Hop Limit, IP identification, and IP checksum for every
packet. There is no need to consider ICMPv6 because only packet. There is no need to consider ICMPv6 because only
FlowLabel of IPv6 and Source and Destination addresses are used, FlowLabel of IPv6 and Source and Destination addresses are used,
and all of them SHOULD be constant. and all of them SHOULD be constant.
Then, the way to identify different hops and attempts of the same Then, the way to identify different hops and attempts of the same
skipping to change at page 13, line 15 skipping to change at page 13, line 37
For IPv6, the field FlowLabel, and Source and Destination For IPv6, the field FlowLabel, and Source and Destination
addresses SHOULD be the same. addresses SHOULD be the same.
o ICMP case: For IPv4, the Data field SHOULD compensate variations o ICMP case: For IPv4, the Data field SHOULD compensate variations
on TTL or Hop Limit, IP identification, and IP checksum for every on TTL or Hop Limit, IP identification, and IP checksum for every
packet. There is no need to consider ICMPv6 because only packet. There is no need to consider ICMPv6 because only
FlowLabel of IPv6 and Source and Destination addresses are used, FlowLabel of IPv6 and Source and Destination addresses are used,
and all of them SHOULD be constant. and all of them SHOULD be constant.
Then, the way to identify different hops and attempts of the same Then, the way to identify different hops and attempts of the same
flow is: IPv4 flow is:
o TCP case: The IP identification field. o TCP case: The IP identification field.
o UDP case: The IP identification field. o UDP case: The IP identification field.
o ICMP case: The IP identification field, and ICMP Sequence number. o ICMP case: The IP identification field, and ICMP Sequence number.
4.1.1. Temporal Composition for Route Metrics 4.1.1. Temporal Composition for Route Metrics
The Active Route Assessment Methods described above have the ability The Active Route Assessment Methods described above have the ability
skipping to change at page 14, line 16 skipping to change at page 14, line 39
where ECMP load balancing is observed (because increasing or where ECMP load balancing is observed (because increasing or
decreasing the pool of links changes the hash calculations). decreasing the pool of links changes the hash calculations).
Optionally, measurement systems may take advantage of the inferences Optionally, measurement systems may take advantage of the inferences
above when seeking to reduce measurement iterations, after exhaustive above when seeking to reduce measurement iterations, after exhaustive
measurements indicate that the time-stable properties are present. measurements indicate that the time-stable properties are present.
Repetitive Active Route measurement systems: Repetitive Active Route measurement systems:
1. SHOULD occasionally check path portions which have exhibited 1. SHOULD occasionally check path portions which have exhibited
stable results over time, particularly ingress and egress stable results over time, particularly ingress and egress
portions of the path. portions of the path (e.g., daily checks if measuring many times
during a day).
2. SHOULD continue testing portions of the path that have previously 2. SHOULD continue testing portions of the path that have previously
exhibited ECMP load balancing. exhibited ECMP load balancing.
3. SHALL trigger re-assessment of the complete path and Route 3. SHALL trigger re-assessment of the complete path and Route
Ensemble, if any change in hops is observed for a specific (and Ensemble, if any change in hops is observed for a specific (and
previously tested) flow. previously tested) flow.
4.1.2. Routing Class Identification 4.1.2. Routing Class Identification
skipping to change at page 15, line 25 skipping to change at page 16, line 10
o ability to direct the emphasis of current measurements on ECMP o ability to direct the emphasis of current measurements on ECMP
portions of the path, based on recent past measurement results portions of the path, based on recent past measurement results
(the Routing Class of some portions of the path is essentially (the Routing Class of some portions of the path is essentially
"all packets"). "all packets").
4.1.3. Intermediate Observation Point Route Measurement 4.1.3. Intermediate Observation Point Route Measurement
There are many examples where passive monitoring of a flow at an There are many examples where passive monitoring of a flow at an
Observation Point within the network can detect unexpected Round Trip Observation Point within the network can detect unexpected Round Trip
Delay or Delay Variation. But how can the cause of the anomalous Delay or Delay Variation. But how can the cause of the anomalous
delay be investigated further --from the Observation Point -- delay be investigated further *from the Observation Point* possibly
possibly located at an intermediate point on the path? located at an intermediate point on the path?
In this case, knowledge that the flow of interest belongs to a In this case, knowledge that the flow of interest belongs to a
specific Routing Class C will enable measurement of the route where specific Routing Class C will enable measurement of the route where
anomalous delay has been observed. Specifically, Round-Trip Delay anomalous delay has been observed. Specifically, Round-Trip Delay
assessment to each Hop on the path between the Observation Point and assessment to each Hop on the path between the Observation Point and
the Destination for the flow of interest may discover high or the Destination for the flow of interest may discover high or
variable delay on a specific link and Hop combination. variable delay on a specific link and Hop combination.
The determination of a Routing Class C which includes the flow of The determination of a Routing Class C which includes the flow of
interest is as described in the section above, aided by computation interest is as described in the section above, aided by computation
skipping to change at page 16, line 10 skipping to change at page 16, line 42
specific abilities: specific abilities:
o Store the identity of nodes that a packet has visited in header o Store the identity of nodes that a packet has visited in header
data fields, in the order the packet visited the nodes. data fields, in the order the packet visited the nodes.
o Support of a "Loopback" capability, where a copy of the packet is o Support of a "Loopback" capability, where a copy of the packet is
returned to the encapsulating node, and the packet is processed returned to the encapsulating node, and the packet is processed
like any other IOAM packet on the return transfer. like any other IOAM packet on the return transfer.
In addition to node identity, nodes may also identify the ingress and In addition to node identity, nodes may also identify the ingress and
egress interfaces utilized by the tracing packet, the time of day egress interfaces utilized by the tracing packet, the absolute time
when the packet was processed, and other generic data (as described when the packet was processed, and other generic data (as described
in section 4 of [I-D.ietf-ippm-ioam-data]). Interface identification in section 4 of [I-D.ietf-ippm-ioam-data]). Interface identification
isn't necessarily limited to IP, i.e. different links in a bundle isn't necessarily limited to IP, i.e. different links in a bundle
(LACP) could be identified. Equally well, links without explicit IP (LACP) could be identified. Equally well, links without explicit IP
addresses can be identified (like with unnumbered interfaces in an addresses can be identified (like with unnumbered interfaces in an
IGP deployment). IGP deployment).
Note that the Type-P packet specification for this method will likely Note that the Type-P packet specification for this method will likely
be a partial specification, because most of the packet fields are be a partial specification, because most of the packet fields are
determined by the user traffic. The packet (encapsulation) header(s) determined by the user traffic. The packet (encapsulation) header(s)
added by the Hybrid method can certainly be specified in Type-P, in added by the Hybrid method can certainly be specified in Type-P, in
unpopulated form. unpopulated form.
skipping to change at page 16, line 40 skipping to change at page 17, line 23
Discoverable Nodes, then a more complete view of Hops on the path is Discoverable Nodes, then a more complete view of Hops on the path is
possible when a hybrid method (or interrogation protocol) is applied possible when a hybrid method (or interrogation protocol) is applied
and the results are combined with the active method results collected and the results are combined with the active method results collected
across all other domains. across all other domains.
In order to combine the results of active and hybrid/interrogation In order to combine the results of active and hybrid/interrogation
methods, the network Nodes that are part of a domain supporting an methods, the network Nodes that are part of a domain supporting an
interrogation protocol have the following attributes: interrogation protocol have the following attributes:
1. Nodes at the ingress to the domain SHOULD be both Discoverable 1. Nodes at the ingress to the domain SHOULD be both Discoverable
and Cooperating, and SHOULD reveal the same Node Identity in and Cooperating.
response to both active and hybrid methods.
2. Any Nodes within the domain that are both Discoverable and 2. Any Nodes within the domain that are both Discoverable and
Cooperating SHOULD reveal the same Node Identity in response to Cooperating SHOULD reveal the same Node Identity in response to
both active and hybrid methods. both active and hybrid methods.
3. Nodes at the egress to the domain SHOULD be both Discoverable and 3. Nodes at the egress to the domain SHOULD be both Discoverable and
Cooperating, and SHOULD reveal the same Node Identity in response Cooperating, and SHOULD reveal the same Node Identity in response
to both active and hybrid methods. to both active and hybrid methods.
When Nodes follow these requirements, it becomes a simple matter to When Nodes follow these requirements, it becomes a simple matter to
skipping to change at page 17, line 37 skipping to change at page 18, line 17
4. Inter-domain paths 4. Inter-domain paths
This categorization is widely accepted in the literature and among This categorization is widely accepted in the literature and among
operators alike, and it can be trusted with empirical data and operators alike, and it can be trusted with empirical data and
several sources as ground of truth (e.g., [RTTSub] ) but it is an several sources as ground of truth (e.g., [RTTSub] ) but it is an
inference measurement nonetheless [bdrmap][IDCong]. inference measurement nonetheless [bdrmap][IDCong].
The first two categories correspond to the physical distance The first two categories correspond to the physical distance
dependency on Round-Trip Delay (RTD), the next one binds RTD with dependency on Round-Trip Delay (RTD), the next one binds RTD with
queueing delay on routers, and the last one helps to identify queuing delay on routers, and the last one helps to identify
different ASes using traceroutes. Due to the significant different ASes using traceroutes. Due to the significant
contribution of propagation delay in long-distance hops, RTD will be contribution of propagation delay in long-distance hops, RTD will be
on the order of 100ms on transatlantic hops, depending on the on the order of 100ms on transatlantic hops, depending on the
geolocation of the vantage points. Moreover, RTD is typically higher geolocation of the vantage points. Moreover, RTD is typically higher
than 480ms when two hops are connected using geostationary satellite than 480ms when two hops are connected using geostationary satellite
technology (i.e., their orbit is at 36000km). Detecting congestion technology (i.e., their orbit is at 36000km). Detecting congestion
with latency implies deeper mathematical understanding since network with latency implies deeper mathematical understanding since network
traffic load is not stationary. Nonetheless, as the first approach, traffic load is not stationary. Nonetheless, as the first approach,
a link seems to be congested if, after sending several traceroute a link seems to be congested if observing different/varying
probes, it is possible to detect congestion observing different statistical results after sending several traceroute probes (e.g.,
statistics parameters (e.g., see [IDCong]). Finally, to recognize see [IDCong]). Finally, to recognize distinctive ASes in the same
distinctive ASes in the same traceroute path is challenging, because traceroute path is challenging, because more data is needed, like AS
more data is needed, like AS relationships and RIR delegations among relationships and RIR delegations among other (for more detail,
other (for more detail, please consult [bdrmap]). please consult [bdrmap]).
6. RTD Measurements Statistics 6. RTD Measurements Statistics
Several articles have shown that network traffic presents a self- Several articles have shown that network traffic presents a self-
similar nature [SSNT] [MLRM] which is accountable for filling the similar nature [SSNT] [MLRM] which is accountable for filling the
queues of the routers. Moreover, router queues are designed to queues of the routers. Moreover, router queues are designed to
handle traffic bursts, which is one of the most remarkable features handle traffic bursts, which is one of the most remarkable features
of self-similarity. Naturally, while queue length increases, the of self-similarity. Naturally, while queue length increases, the
delay to traverse the queue increases as well and leads to an delay to traverse the queue increases as well and leads to an
increase on RTD. Due to traffic bursts generating short-term increase on RTD. Due to traffic bursts generating short-term
skipping to change at page 18, line 42 skipping to change at page 19, line 22
This procedure allows us to update the quartiles value whenever a new This procedure allows us to update the quartiles value whenever a new
measurement arrives, which is radically different from classic measurement arrives, which is radically different from classic
methods of computing quartiles because they need to use the whole methods of computing quartiles because they need to use the whole
dataset to compute the values. This way of calculus provides savings dataset to compute the values. This way of calculus provides savings
in memory and computing time. in memory and computing time.
To sum up, the proposed measurement procedure consists of performing To sum up, the proposed measurement procedure consists of performing
traceroutes several times to obtain samples of the RTD in every hop traceroutes several times to obtain samples of the RTD in every hop
from a path, during a time window (W), and compute the quartiles for from a path, during a time window (W), and compute the quartiles for
every hop. This procedure could be done for a single Member Route every hop. This procedure could be done for a single Member Route
flow, with parameter E set as False, or for every detected Route flow, a non-exhaustive search with parameter E (defined below) set as
Ensemble flow (E=True). False, or for every detected Route Ensemble flow (E=True).
The identification of a specific Hop in traceroute is based on the IP The identification of a specific Hop in traceroute is based on the IP
origin address of the returned ICMP Time Exceeded packet, and on the origin address of the returned ICMP Time Exceeded packet, and on the
distance identified by the value set in the TTL field inserted by distance identified by the value set in the TTL (or Hop Limit) field
traceroute. As this specific Hop can be reached by different paths, inserted by traceroute. As this specific Hop can be reached by
also the IP source and destination addresses of the traceroute packet different paths, also the IP source and destination addresses of the
need to be recorded. Finally, different return paths are traceroute packet need to be recorded. Finally, different return
distinguished by evaluating the ICMP Time Exceeded TTL (of the reply paths are distinguished by evaluating the ICMP Time Exceeded TTL (or
message): if this TTL is constant for different paths containing the Hop Limit) of the reply message: if this TTL (or Hop Limit) is
same Hop, the return paths have the same distance. Moreover, this constant for different paths containing the same Hop, the return
distance can be estimated considering that the TTL value is normally paths have the same distance. Moreover, this distance can be
estimated considering that the TTL (or Hop Limit) value is normally
initialized with values 64, 128, or 255. The 5-tuple (origin IP, initialized with values 64, 128, or 255. The 5-tuple (origin IP,
destination IP, reply IP, distance, response TTL) univocally destination IP, reply IP, distance, response TTL or Hop Limit)
identifies every measurement. unequivocally identifies every measurement.
This algorithm below runs in the origin of the traceroute. It This algorithm below runs in the origin of the traceroute. It
returns the Qs quartiles for every Hop and Alt (alternative paths returns the Qs quartiles for every Hop and Alt (alternative paths
because of balancing). Notice that the "Alt" parameter condenses the because of balancing). Notice that the "Alt" parameter condenses the
parameters of the 5-tuple (origin IP, destination IP, reply IP, parameters of the 5-tuple (origin IP, destination IP, reply IP,
distance, response TTL), i.e., one for each possible combination. distance, response TTL), i.e., one for each possible combination.
================================================================ ================================================================
1 input: W (window time of the measurement) 0 input: W (window time of the measurement)
2 i_t (time between two measurements) 1 i_t (time between two measurements, set the i_t time
2 long enough to avoid incomplete results)
3 E (True: exhaustive, False: a single path) 3 E (True: exhaustive, False: a single path)
4 Dst (destination IP address) 4 Dst (destination IP address)
5 output: Qs (quartiles for every Hop and Alt) 5 output: Qs (quartiles for every Hop and Alt)
---------------------------------------------------------------- ----------------------------------------------------------------
6 T := start_timer(W) 6 T := start_timer(W)
7 while T is not finished do: 7 while T is not finished do:
8 | start_timer(i_t) 8 | start_timer(i_t)
9 | RTD(Hop,Alt) = advanced-traceroute(Dst,E) 9 | RTD(Hop,Alt) = advanced-traceroute(Dst,E)
10 | for each Hop and Alt in RTD do: 10 | for each Hop and Alt in RTD do:
11 | | Qs[Dst,Hop,Alt] := ComputeQs(RTD(Hop,Alt)) 11 | | Qs[Dst,Hop,Alt] := ComputeQs(RTD(Hop,Alt))
skipping to change at page 20, line 16 skipping to change at page 21, line 5
The security considerations that apply to any active measurement of The security considerations that apply to any active measurement of
live paths are relevant here as well. See [RFC4656] and [RFC5357]. live paths are relevant here as well. See [RFC4656] and [RFC5357].
The active measurement process of "changing several fields to keep The active measurement process of "changing several fields to keep
the checksum of different packets identical" does not require special the checksum of different packets identical" does not require special
security considerations because it is part of synthetic traffic security considerations because it is part of synthetic traffic
generation, and is designed to have minimal to zero impact on network generation, and is designed to have minimal to zero impact on network
processing (to process the packets for ECMP). processing (to process the packets for ECMP).
Some of the protocols used (e.g., ICMP) do not provide cryptographic
protection for the requested/returned data, and there are risks of
processing untrusted data in general, but these are limitations of
the existing protocols where we are applying new methods.
For applicable Hybrid methods, the security considerations For applicable Hybrid methods, the security considerations
in[I-D.ietf-ippm-ioam-data] apply. in[I-D.ietf-ippm-ioam-data] apply.
When considering privacy of those involved in measurement or those When considering privacy of those involved in measurement or those
whose traffic is measured, the sensitive information available to whose traffic is measured, the sensitive information available to
potential observers is greatly reduced when using active techniques potential observers is greatly reduced when using active techniques
which are within this scope of work. Passive observations of user which are within this scope of work. Passive observations of user
traffic for measurement purposes raise many privacy issues. We refer traffic for measurement purposes raise many privacy issues. We refer
the reader to the privacy considerations described in the Large Scale the reader to the privacy considerations described in the Large Scale
Measurement of Broadband Performance (LMAP) Framework [RFC7594], Measurement of Broadband Performance (LMAP) Framework [RFC7594],
which covers active and passive techniques. which covers active and passive techniques.
8. IANA Considerations 8. IANA Considerations
This memo makes no requests of IANA. We thank the good folks at IANA This memo makes no requests of IANA. We thank the good folks at IANA
for having checked this section anyway. for having checked this section anyway.
9. Acknowledgements 9. Acknowledgements
The original 3 authors acknowledge Ruediger Geib, for his penetrating The original 3 authors (Ignacio, Al, Joachim) acknowledge Ruediger
comments on the initial draft, and his initial text for the Geib, for his penetrating comments on the initial draft, and his
Appendix on MPLS. Carlos Pignataro challenged the authors to initial text for the Appendix on MPLS. Carlos Pignataro challenged
consider a wider scope, and applied his substantial expertise with the authors to consider a wider scope, and applied his substantial
many technologies and their measurement features in his extensive expertise with many technologies and their measurement features in
comments. Frank Brockners also shared useful comments, so did Footer his extensive comments. Frank Brockners also shared useful comments,
Foote. We thank them all! so did Footer Foote. We thank them all!
10. Appendix I MPLS Methods for Route Assessment 10. Appendix I MPLS Methods for Route Assessment
A Node assessing an MPLS path must be part of the MPLS domain where A Node assessing an MPLS path must be part of the MPLS domain where
the path is implemented. When this condition is met, RFC 8029 the path is implemented. When this condition is met, RFC 8029
provides a powerful set of mechanisms to detect "correct operation of provides a powerful set of mechanisms to detect "correct operation of
the data plane, as well as a mechanism to verify the data plane the data plane, as well as a mechanism to verify the data plane
against the control plane" [RFC8029]. against the control plane" [RFC8029].
MPLS routing is based on the presence of a Forwarding Equivalence MPLS routing is based on the presence of a Forwarding Equivalence
skipping to change at page 21, line 39 skipping to change at page 22, line 33
to detect arbitrary MPLS paths between any routers within a single to detect arbitrary MPLS paths between any routers within a single
MPLS domain. The combination of MPLS forwarding, Segment Routing and MPLS domain. The combination of MPLS forwarding, Segment Routing and
MPLS traceroute offers a simple architecture and a powerful mechanism MPLS traceroute offers a simple architecture and a powerful mechanism
to detect and validate (segment routed) MPLS paths. to detect and validate (segment routed) MPLS paths.
11. References 11. References
11.1. Normative References 11.1. Normative References
[I-D.ietf-ippm-ioam-data] [I-D.ietf-ippm-ioam-data]
Brockners, F., Bhandari, S., Pignataro, C., Gredler, H., Brockners, F., Bhandari, S., and T. Mizrahi, "Data Fields
Leddy, J., Youell, S., Mizrahi, T., Mozes, D., Lapukhov, for In-situ OAM", draft-ietf-ippm-ioam-data-10 (work in
P., remy@barefootnetworks.com, r., daniel.bernier@bell.ca, progress), July 2020.
d., and J. Lemon, "Data Fields for In-situ OAM", draft-
ietf-ippm-ioam-data-09 (work in progress), March 2020.
[RFC0792] Postel, J., "Internet Control Message Protocol", STD 5, [RFC0792] Postel, J., "Internet Control Message Protocol", STD 5,
RFC 792, DOI 10.17487/RFC0792, September 1981, RFC 792, DOI 10.17487/RFC0792, September 1981,
<https://www.rfc-editor.org/info/rfc792>. <https://www.rfc-editor.org/info/rfc792>.
[RFC1122] Braden, R., Ed., "Requirements for Internet Hosts - [RFC1122] Braden, R., Ed., "Requirements for Internet Hosts -
Communication Layers", STD 3, RFC 1122, Communication Layers", STD 3, RFC 1122,
DOI 10.17487/RFC1122, October 1989, DOI 10.17487/RFC1122, October 1989,
<https://www.rfc-editor.org/info/rfc1122>. <https://www.rfc-editor.org/info/rfc1122>.
skipping to change at page 22, line 28 skipping to change at page 23, line 19
[RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis,
"Framework for IP Performance Metrics", RFC 2330, "Framework for IP Performance Metrics", RFC 2330,
DOI 10.17487/RFC2330, May 1998, DOI 10.17487/RFC2330, May 1998,
<https://www.rfc-editor.org/info/rfc2330>. <https://www.rfc-editor.org/info/rfc2330>.
[RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip [RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip
Delay Metric for IPPM", RFC 2681, DOI 10.17487/RFC2681, Delay Metric for IPPM", RFC 2681, DOI 10.17487/RFC2681,
September 1999, <https://www.rfc-editor.org/info/rfc2681>. September 1999, <https://www.rfc-editor.org/info/rfc2681>.
[RFC2991] Thaler, D. and C. Hopps, "Multipath Issues in Unicast and
Multicast Next-Hop Selection", RFC 2991,
DOI 10.17487/RFC2991, November 2000,
<https://www.rfc-editor.org/info/rfc2991>.
[RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M. [RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M.
Zekauskas, "A One-way Active Measurement Protocol Zekauskas, "A One-way Active Measurement Protocol
(OWAMP)", RFC 4656, DOI 10.17487/RFC4656, September 2006, (OWAMP)", RFC 4656, DOI 10.17487/RFC4656, September 2006,
<https://www.rfc-editor.org/info/rfc4656>. <https://www.rfc-editor.org/info/rfc4656>.
[RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K., and J.
Babiarz, "A Two-Way Active Measurement Protocol (TWAMP)",
RFC 5357, DOI 10.17487/RFC5357, October 2008,
<https://www.rfc-editor.org/info/rfc5357>.
[RFC5388] Niccolini, S., Tartarelli, S., Quittek, J., Dietz, T., and [RFC5388] Niccolini, S., Tartarelli, S., Quittek, J., Dietz, T., and
M. Swany, "Information Model and XML Data Model for M. Swany, "Information Model and XML Data Model for
Traceroute Measurements", RFC 5388, DOI 10.17487/RFC5388, Traceroute Measurements", RFC 5388, DOI 10.17487/RFC5388,
December 2008, <https://www.rfc-editor.org/info/rfc5388>. December 2008, <https://www.rfc-editor.org/info/rfc5388>.
[RFC5835] Morton, A., Ed. and S. Van den Berghe, Ed., "Framework for
Metric Composition", RFC 5835, DOI 10.17487/RFC5835, April
2010, <https://www.rfc-editor.org/info/rfc5835>.
[RFC5837] Atlas, A., Ed., Bonica, R., Ed., Pignataro, C., Ed., Shen,
N., and JR. Rivers, "Extending ICMP for Interface and
Next-Hop Identification", RFC 5837, DOI 10.17487/RFC5837,
April 2010, <https://www.rfc-editor.org/info/rfc5837>.
[RFC6437] Amante, S., Carpenter, B., Jiang, S., and J. Rajahalme,
"IPv6 Flow Label Specification", RFC 6437,
DOI 10.17487/RFC6437, November 2011,
<https://www.rfc-editor.org/info/rfc6437>.
[RFC6438] Carpenter, B. and S. Amante, "Using the IPv6 Flow Label [RFC6438] Carpenter, B. and S. Amante, "Using the IPv6 Flow Label
for Equal Cost Multipath Routing and Link Aggregation in for Equal Cost Multipath Routing and Link Aggregation in
Tunnels", RFC 6438, DOI 10.17487/RFC6438, November 2011, Tunnels", RFC 6438, DOI 10.17487/RFC6438, November 2011,
<https://www.rfc-editor.org/info/rfc6438>. <https://www.rfc-editor.org/info/rfc6438>.
[RFC6673] Morton, A., "Round-Trip Packet Loss Metrics", RFC 6673, [RFC6673] Morton, A., "Round-Trip Packet Loss Metrics", RFC 6673,
DOI 10.17487/RFC6673, August 2012, DOI 10.17487/RFC6673, August 2012,
<https://www.rfc-editor.org/info/rfc6673>. <https://www.rfc-editor.org/info/rfc6673>.
[RFC7312] Fabini, J. and A. Morton, "Advanced Stream and Sampling
Framework for IP Performance Metrics (IPPM)", RFC 7312,
DOI 10.17487/RFC7312, August 2014,
<https://www.rfc-editor.org/info/rfc7312>.
[RFC7799] Morton, A., "Active and Passive Metrics and Methods (with [RFC7799] Morton, A., "Active and Passive Metrics and Methods (with
Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799, Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799,
May 2016, <https://www.rfc-editor.org/info/rfc7799>. May 2016, <https://www.rfc-editor.org/info/rfc7799>.
[RFC8029] Kompella, K., Swallow, G., Pignataro, C., Ed., Kumar, N., [RFC8029] Kompella, K., Swallow, G., Pignataro, C., Ed., Kumar, N.,
Aldrin, S., and M. Chen, "Detecting Multiprotocol Label Aldrin, S., and M. Chen, "Detecting Multiprotocol Label
Switched (MPLS) Data-Plane Failures", RFC 8029, Switched (MPLS) Data-Plane Failures", RFC 8029,
DOI 10.17487/RFC8029, March 2017, DOI 10.17487/RFC8029, March 2017,
<https://www.rfc-editor.org/info/rfc8029>. <https://www.rfc-editor.org/info/rfc8029>.
skipping to change at page 24, line 45 skipping to change at page 25, line 5
calculation of quartiles and histograms without storing calculation of quartiles and histograms without storing
observations", Communications of the ACM 28.10 (1985): observations", Communications of the ACM 28.10 (1985):
1076-1085, 2015. 1076-1085, 2015.
[PT] Augustin, B., Cuvellier, X., Orgogozo, B., Viger, F., [PT] Augustin, B., Cuvellier, X., Orgogozo, B., Viger, F.,
Friedman, T., Latapy, M., Magnien, C., and R. Teixeira, Friedman, T., Latapy, M., Magnien, C., and R. Teixeira,
"Avoiding traceroute anomalies with Paris traceroute", "Avoiding traceroute anomalies with Paris traceroute",
Proceedings of the 6th ACM SIGCOMM conference on Internet Proceedings of the 6th ACM SIGCOMM conference on Internet
measurement, pp. 153-158. ACM, 2006., 2006. measurement, pp. 153-158. ACM, 2006., 2006.
[RFC2991] Thaler, D. and C. Hopps, "Multipath Issues in Unicast and
Multicast Next-Hop Selection", RFC 2991,
DOI 10.17487/RFC2991, November 2000,
<https://www.rfc-editor.org/info/rfc2991>.
[RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K., and J.
Babiarz, "A Two-Way Active Measurement Protocol (TWAMP)",
RFC 5357, DOI 10.17487/RFC5357, October 2008,
<https://www.rfc-editor.org/info/rfc5357>.
[RFC5835] Morton, A., Ed. and S. Van den Berghe, Ed., "Framework for
Metric Composition", RFC 5835, DOI 10.17487/RFC5835, April
2010, <https://www.rfc-editor.org/info/rfc5835>.
[RFC5837] Atlas, A., Ed., Bonica, R., Ed., Pignataro, C., Ed., Shen,
N., and JR. Rivers, "Extending ICMP for Interface and
Next-Hop Identification", RFC 5837, DOI 10.17487/RFC5837,
April 2010, <https://www.rfc-editor.org/info/rfc5837>.
[RFC6437] Amante, S., Carpenter, B., Jiang, S., and J. Rajahalme,
"IPv6 Flow Label Specification", RFC 6437,
DOI 10.17487/RFC6437, November 2011,
<https://www.rfc-editor.org/info/rfc6437>.
[RFC7312] Fabini, J. and A. Morton, "Advanced Stream and Sampling
Framework for IP Performance Metrics (IPPM)", RFC 7312,
DOI 10.17487/RFC7312, August 2014,
<https://www.rfc-editor.org/info/rfc7312>.
[RFC7325] Villamizar, C., Ed., Kompella, K., Amante, S., Malis, A., [RFC7325] Villamizar, C., Ed., Kompella, K., Amante, S., Malis, A.,
and C. Pignataro, "MPLS Forwarding Compliance and and C. Pignataro, "MPLS Forwarding Compliance and
Performance Requirements", RFC 7325, DOI 10.17487/RFC7325, Performance Requirements", RFC 7325, DOI 10.17487/RFC7325,
August 2014, <https://www.rfc-editor.org/info/rfc7325>. August 2014, <https://www.rfc-editor.org/info/rfc7325>.
[RFC7594] Eardley, P., Morton, A., Bagnulo, M., Burbridge, T., [RFC7594] Eardley, P., Morton, A., Bagnulo, M., Burbridge, T.,
Aitken, P., and A. Akhter, "A Framework for Large-Scale Aitken, P., and A. Akhter, "A Framework for Large-Scale
Measurement of Broadband Performance (LMAP)", RFC 7594, Measurement of Broadband Performance (LMAP)", RFC 7594,
DOI 10.17487/RFC7594, September 2015, DOI 10.17487/RFC7594, September 2015,
<https://www.rfc-editor.org/info/rfc7594>. <https://www.rfc-editor.org/info/rfc7594>.
 End of changes. 65 change blocks. 
198 lines changed or deleted 224 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/