draft-ietf-ippm-route-07.txt   draft-ietf-ippm-route-08.txt 
Network Working Group J. Alvarez-Hamelin Network Working Group J. Alvarez-Hamelin
Internet-Draft Universidad de Buenos Aires Internet-Draft Universidad de Buenos Aires
Updates: 2330 (if approved) A. Morton Updates: 2330 (if approved) A. Morton
Intended status: Standards Track AT&T Labs Intended status: Standards Track AT&T Labs
Expires: June 13, 2020 J. Fabini Expires: December 20, 2020 J. Fabini
TU Wien TU Wien
C. Pignataro C. Pignataro
Cisco Systems, Inc. Cisco Systems, Inc.
R. Geib R. Geib
Deutsche Telekom Deutsche Telekom
December 11, 2019 June 18, 2020
Advanced Unidirectional Route Assessment (AURA) Advanced Unidirectional Route Assessment (AURA)
draft-ietf-ippm-route-07 draft-ietf-ippm-route-08
Abstract Abstract
This memo introduces an advanced unidirectional route assessment This memo introduces an advanced unidirectional route assessment
(AURA) metric and associated measurement methodology, based on the IP (AURA) metric and associated measurement methodology, based on the IP
Performance Metrics (IPPM) Framework RFC 2330. This memo updates RFC Performance Metrics (IPPM) Framework RFC 2330. This memo updates RFC
2330 in the areas of path-related terminology and path description, 2330 in the areas of path-related terminology and path description,
primarily to include the possibility of parallel subpaths between a primarily to include the possibility of parallel subpaths between a
given Source and Destination pair, owing to the presence of multi- given Source and Destination pair, owing to the presence of multi-
path technologies. path technologies.
skipping to change at page 2, line 4 skipping to change at page 2, line 4
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on June 13, 2020. This Internet-Draft will expire on December 20, 2020.
Copyright Notice Copyright Notice
Copyright (c) 2019 IETF Trust and the persons identified as the Copyright (c) 2020 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
skipping to change at page 2, line 34 skipping to change at page 2, line 34
1.1. Issues with Earlier Work to define Route . . . . . . . . 3 1.1. Issues with Earlier Work to define Route . . . . . . . . 3
2. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3. Route Metric Terms and Definitions . . . . . . . . . . . . . 5 3. Route Metric Terms and Definitions . . . . . . . . . . . . . 5
3.1. Formal Name . . . . . . . . . . . . . . . . . . . . . . . 6 3.1. Formal Name . . . . . . . . . . . . . . . . . . . . . . . 6
3.2. Parameters . . . . . . . . . . . . . . . . . . . . . . . 6 3.2. Parameters . . . . . . . . . . . . . . . . . . . . . . . 6
3.3. Metric Definitions . . . . . . . . . . . . . . . . . . . 7 3.3. Metric Definitions . . . . . . . . . . . . . . . . . . . 7
3.4. Related Round-Trip Delay and Loss Definitions . . . . . . 9 3.4. Related Round-Trip Delay and Loss Definitions . . . . . . 9
3.5. Discussion . . . . . . . . . . . . . . . . . . . . . . . 9 3.5. Discussion . . . . . . . . . . . . . . . . . . . . . . . 9
3.6. Reporting the Metric . . . . . . . . . . . . . . . . . . 10 3.6. Reporting the Metric . . . . . . . . . . . . . . . . . . 10
4. Route Assessment Methodologies . . . . . . . . . . . . . . . 10 4. Route Assessment Methodologies . . . . . . . . . . . . . . . 10
4.1. Active Methodologies . . . . . . . . . . . . . . . . . . 10 4.1. Active Methodologies . . . . . . . . . . . . . . . . . . 11
4.1.1. Temporal Composition for Route Metrics . . . . . . . 12 4.1.1. Temporal Composition for Route Metrics . . . . . . . 13
4.1.2. Routing Class C Identification . . . . . . . . . . . 13 4.1.2. Routing Class Identification . . . . . . . . . . . . 14
4.1.3. Intermediate Observation Point Route Measurement . . 14 4.1.3. Intermediate Observation Point Route Measurement . . 15
4.2. Hybrid Methodologies . . . . . . . . . . . . . . . . . . 15 4.2. Hybrid Methodologies . . . . . . . . . . . . . . . . . . 15
4.3. Combining Different Methods . . . . . . . . . . . . . . . 15 4.3. Combining Different Methods . . . . . . . . . . . . . . . 16
5. Background on Round-Trip Delay Measurement Goals . . . . . . 16 5. Background on Round-Trip Delay Measurement Goals . . . . . . 17
6. Tools to Measure Delays in the Internet . . . . . . . . . . . 17 6. RTD Measurements Statistics . . . . . . . . . . . . . . . . . 18
7. RTD Measurements Statistics . . . . . . . . . . . . . . . . . 18 7. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 20
8. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 20 8. Security Considerations . . . . . . . . . . . . . . . . . . . 20
9. Security Considerations . . . . . . . . . . . . . . . . . . . 20 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20
10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 20
11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 21 11. Appendix I MPLS Methods for Route Assessment . . . . . . . . 21
12. Appendix I MPLS Methods for Route Assessment . . . . . . . . 21 12. References . . . . . . . . . . . . . . . . . . . . . . . . . 21
13. References . . . . . . . . . . . . . . . . . . . . . . . . . 22 12.1. Normative References . . . . . . . . . . . . . . . . . . 22
13.1. Normative References . . . . . . . . . . . . . . . . . . 22 12.2. Informative References . . . . . . . . . . . . . . . . . 24
13.2. Informative References . . . . . . . . . . . . . . . . . 25 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 25
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 26
1. Introduction 1. Introduction
The IETF IP Performance Metrics (IPPM) working group first created a The IETF IP Performance Metrics (IPPM) working group first created a
framework for metric development in [RFC2330]. This framework has framework for metric development in [RFC2330]. This framework has
stood the test of time and enabled development of many fundamental stood the test of time and enabled development of many fundamental
metrics. It has been updated in the area of metric composition metrics. It has been updated in the area of metric composition
[RFC5835], and in several areas related to active stream measurement [RFC5835], and in several areas related to active stream measurement
of modern networks with reactive properties [RFC7312]. of modern networks with reactive properties [RFC7312].
The [RFC2330] framework motivated the development of "performance and The [RFC2330] framework motivated the development of "performance and
reliability metrics for paths through the Internet," and Section 5 of reliability metrics for paths through the Internet," and Section 5 of
[RFC2330] defines terms that support description of a path under [RFC2330] defines terms that support description of a path under
test. However, metrics for assessment of path components and related test. However, metrics for assessment of path components and related
performance aspects had not been attempted in IPPM when the [RFC2330] performance aspects had not been attempted in IPPM when the [RFC2330]
framework was written. framework was written.
This memo takes-up the route measurement challenge and specifies a This memo takes up the route measurement challenge and specifies a
new route metric, two practical frameworks for methods of measurement new route metric, two practical frameworks for methods of measurement
(using either active or hybrid active-passive methods [RFC7799]), and (using either active or hybrid active-passive methods [RFC7799]), and
Round-Trip Delay and link information discovery using the results of Round-Trip Delay and link information discovery using the results of
measurements. All route measurements are limited by the willingness measurements. All route measurements are limited by the willingness
of hosts along the path to be discovered, to cooperate with the of hosts along the path to be discovered, to cooperate with the
methods used, or to recognize that the measurement operation is methods used, or to recognize that the measurement operation is
taking place (such as when tunnels are present). taking place (such as when tunnels are present).
1.1. Issues with Earlier Work to define Route 1.1. Issues with Earlier Work to define Route
skipping to change at page 4, line 21 skipping to change at page 4, line 21
Cloud Subpath: May contain hosts that do not decrement TTL or Hop Cloud Subpath: May contain hosts that do not decrement TTL or Hop
Limit, but may have two or more exchange links connecting Limit, but may have two or more exchange links connecting
"discoverable" hosts or routers. Parallel subpaths contained "discoverable" hosts or routers. Parallel subpaths contained
within clouds cannot be discovered. The assessment methods only within clouds cannot be discovered. The assessment methods only
discover hosts or routers on the path that decrement TTL or Hop discover hosts or routers on the path that decrement TTL or Hop
Count, or cooperate with interrogation protocols. The presence of Count, or cooperate with interrogation protocols. The presence of
tunnels and nested tunnels further complicate assessment by hiding tunnels and nested tunnels further complicate assessment by hiding
hops. hops.
Hop: Although the [RFC2330] definition of a hop was a link-host Hop: Although the [RFC2330] definition of a hop was a link-host
pair, only hosts are discoverable or have the capability to pair, only hosts that are discoverable or have the capability to
cooperate with interrogation protocols where link information may cooperate with interrogation protocols where link information may
be exposed. be exposed.
The refined definition of Route metrics begins in the sections that The refined definition of Route metrics begins in the sections that
follow. follow.
2. Scope 2. Scope
The purpose of this memo is to add new route metrics and methods of The purpose of this memo is to add new route metrics and methods of
measurement to the existing set of IPPM metrics. measurement to the existing set of IPPM metrics.
skipping to change at page 6, line 18 skipping to change at page 6, line 18
Node identity as part of a previously agreed and established Node identity as part of a previously agreed and established
interrogation protocol. Nodes SHOULD also provide information interrogation protocol. Nodes SHOULD also provide information
such as arrival/departure interface identification, arrival such as arrival/departure interface identification, arrival
timestamp, and any relevant information about the Node or specific timestamp, and any relevant information about the Node or specific
link which delivered the query to the Node. link which delivered the query to the Node.
Hop A Hop MUST contain a Node Identity, and MAY contain arrival and/ Hop A Hop MUST contain a Node Identity, and MAY contain arrival and/
or departure interface identification, round trip delay, and an or departure interface identification, round trip delay, and an
arrival timestamp. arrival timestamp.
Routing Class A route that treats equally a class C of different
types of packets. Knowledge of such a class allows any one of the
types of packets within that class to be used for subsequent
measurement of the route.
3.1. Formal Name 3.1. Formal Name
Type-P-Route-Ensemble-Method-Variant, abbreviated as Route Ensemble. Type-P-Route-Ensemble-Method-Variant, abbreviated as Route Ensemble.
Note that Type-P depends heavily on the chosen method and variant. Note that Type-P depends heavily on the chosen method and variant.
3.2. Parameters 3.2. Parameters
This section lists the REQUIRED input factors to specify a Route This section lists the REQUIRED input factors to specify a Route
metric. metric.
skipping to change at page 7, line 40 skipping to change at page 7, line 43
metrics (unless otherwise indicated): metrics (unless otherwise indicated):
M, the total number of packets sent between T0 and Tf. M, the total number of packets sent between T0 and Tf.
N, the smallest value of i needed for a packet to be received at Dst N, the smallest value of i needed for a packet to be received at Dst
(sent between T0 and Tf). (sent between T0 and Tf).
Nmax, the largest value of i needed for a packet to be received at Nmax, the largest value of i needed for a packet to be received at
Dst (sent between T0 and Tf). Nmax may be equal to N. Dst (sent between T0 and Tf). Nmax may be equal to N.
Next, define a *singleton* definition for a Hop on the path, with Next define a *singleton* definition for a Hop on the path, with
sufficient indexes to identify all Hops identified in a measurement sufficient indexes to identify all Hops identified in a measurement
interval. interval.
A Hop, designated h(i,j), the IP address and/or identity of one of j A Hop, designated h(i,j), the IP address and/or identity of
Discoverable Nodes (or Cooperating Nodes) that are i hops away from Discoverable Nodes (or Cooperating Nodes) that are i hops away from
the Node with address = Src during the measurement interval, T0 to the Node with address = Src and part of Route j during the
Tf. As defined above, a Hop singleton measurement MUST contain a measurement interval, T0 to Tf. As defined here, a Hop singleton
Node Identity, hid(i,j), and MAY contain one or more of the following measurement MUST contain a Node Identity, hid(i,j), and MAY contain
attributes: one or more of the following attributes:
o a(i,j) Arrival Interface ID (e.g., when [RFC5837] is supported) o a(i,j) Arrival Interface ID (e.g., when [RFC5837] is supported)
o d(i,j) Departure Interface ID (e.g., when [RFC5837] is supported) o d(i,j) Departure Interface ID (e.g., when [RFC5837] is supported)
o t(i,j) Arrival Timestamp (where t(i,j) is ideally supplied by the o t(i,j) Arrival Timestamp (where t(i,j) is ideally supplied by the
Hop, or approximated from the sending time of the packet that Hop, or approximated from the sending time of the packet that
revealed the Hop) revealed the Hop)
o Measurements of Round-Trip Delay (for each packet that reveals the o Measurements of Round-Trip Delay (for each packet that reveals the
same Node Identity and flow attributes, but not timestamp of same Node Identity and flow attributes, then this attribute is
course, see next section) computed, see next section)
Node Identities and related information can be ordered by their Node Identities and related information can be ordered by their
distance from the Node with address Src in Hops h(i,j). Based on distance from the Node with address Src in Hops h(i,j). Based on
this, two forms of Routes are distinguished: this, two forms of Routes are distinguished:
A Route Ensemble is defined as the combination of all routes A Route Ensemble is defined as the combination of all routes
traversed by different flows from the Node at Src address to the Node traversed by different flows from the Node at Src address to the Node
at Dst address. A single Route traversed by a single flow at Dst address. A single Route traversed by a single flow
(determined by an unambiguous tuple of addresses Src and Dst, and (determined by an unambiguous tuple of addresses Src and Dst, and
other identical flow criteria) is a member of the Route Ensemble and other identical flow criteria) is a member of the Route Ensemble and
called a Member Route. called a Member Route.
Using h(i,j) and components and parameters, further define: Using h(i,j) and components and parameters, further define:
When considering the set of Hops in the context of a single flow, a When considering the set of Hops in the context of a single flow, a
Member Route j is an ordered list {h(1,j), ... h(Nj, j)} where h(i-1, Member Route j is an ordered list {h(1,j), ... h(Nj, j)} where h(i-1,
j) and h(i, j) are by 1 hop away from each other and Nj satisfying j) and h(i, j) are 1 hop away from each other and Nj satisfying
h(Nj,j)=Dst is the minimum count of Hops needed by the packet on h(Nj,j)=Dst is the minimum count of Hops needed by the packet on
Member Route j to reach Dst. Member Routes must be unique. The Member Route j to reach Dst. Member Routes must be unique. The
uniqueness property requires that any two Member routes j and k that uniqueness property requires that any two Member routes j and k that
are part of the same Route Ensemble differ either in terms of minimum are part of the same Route Ensemble differ either in terms of minimum
hop count Nj and Nk to reach the destination Dst, or, in the case of hop count Nj and Nk to reach the destination Dst, or, in the case of
identical hop count Nj=Nk, they have at least one distinct Hop: identical hop count Nj=Nk, they have at least one distinct Hop:
h(i,j) != h(i, k) for at least one i (i=1..Nj). h(i,j) != h(i,k) for at least one i (i=1..Nj).
All the optional information collected to describe a Member Route, All the optional information collected to describe a Member Route,
such as the arrival interface, departure interface, and Round Trip such as the arrival interface, departure interface, and Round Trip
Delay at each Hop, turns each list item into a rich structure. There Delay at each Hop, turns each list item into a rich structure. There
may be information on the links between Hops, possibly information on may be information on the links between Hops, possibly information on
the routing (arrival interface and departure interface), an estimate the routing (arrival interface and departure interface), an estimate
of distance between Hops based on Round-Trip Delay measurements and of distance between Hops based on Round-Trip Delay measurements and
calculations, and a time stamp indicating when all these additional calculations, and a time stamp indicating when all these additional
details were valid. details were valid.
skipping to change at page 11, line 7 skipping to change at page 11, line 12
(usually within a single domain). Description of these methods (usually within a single domain). Description of these methods
follow. follow.
4.1. Active Methodologies 4.1. Active Methodologies
This section describes the method employed by current open source This section describes the method employed by current open source
tools, thereby providing a practical framework for further advanced tools, thereby providing a practical framework for further advanced
techniques to be included as method variants. This method is techniques to be included as method variants. This method is
applicable for use across multiple administrative domains. applicable for use across multiple administrative domains.
Paris-traceroute [PT] provides some measure of protection from path Internet routing is complex because it depends on the policies of
variation generated by ECMP load balancing, and it ensures traceroute thousands of Autonomous Systems (AS). While most of the routers
packets will follow the same path in 98% of cases according to perform load balancing on flows using Equal Cost Multiple Path
[SCAMPER]. If it is necessary to find every usable path between two (ECMP), a few still divide the workload through packet-based
Nodes, Paris-traceroute provides "exhaustive" mode while scamper techniques. The former scenario is defined according to [RFC2991],
provides "tracelb" (stands for traceroute load balance). while the latter generates a round-robin scheme to deliver every new
outgoing packet. ECMP uses a hashing function to ensure that every
packet of a flow is delivered by the same path, and this avoids
increasing the packet delay variation and possibly producing
overwhelming packet reordering in TCP flows.
The Type-P of packets used could be ICMP (as in the original Taking into account that Internet protocol was designed under the
traceroute), UDP or TCP. The latter are used when a particular "end-to-end" principle, the IP payload and its header do not provide
characteristic needs to be to verified, such as filtering or traffic any information about the routes or path necessary to reach some
shaping on specific ports (i.e., services). [SCAMPER] supports IPv6 destination. For this reason, the popular tool traceroute was
traceroute measurements, keeping the FlowLabel constant in all developed to gather the IP addresses of each hop along a path using
packets. the ICMP protocol [RFC0792]. Traceroute also measures RTD from each
hop. However, the growing complexity of the Internet makes it more
challenging to develop an accurate traceroute implementation. For
instance, the early traceroute tools would be inaccurate in the
current network, mainly because they were not designed to retain a
flow state. However, evolved traceroute tools, such as Paris-
traceroute [PT] [MLB] and Scamper [SCAMPER], expect to encounter ECMP
and achieve more accurate results when they do, where Scamper ensures
traceroute packets will follow the same path in 98% of
cases[SCAMPER].
Today's traceroute tools send Type-P of packets, either ICMP, UDP, or
TCP. UDP and TCP are used when a particular characteristic needs to
be verified, such as filtering or traffic shaping on specific ports
(i.e., services). [SCAMPER] supports IPv6 traceroute measurements,
keeping the FlowLabel constant in all packets.
Paris-traceroute allows its users to measure RTD in every hop of the
path for a particular flow. Furthermore, either Paris-traceroute or
Scamper is capable of unveiling the many available paths between a
source and destination (which are visible to this method). This task
is accomplished by repeating complete traceroute measurements with
different flow parameters for each measurement; Paris-traceroute
provides "exhaustive" mode while scamper provides "tracelb" (stands
for traceroute load balance). The Framework for IP Performance
Metrics (IPPM) ([RFC2330] updated by [RFC7312]) has the flexibility
to require that the Round-Trip Delay measurement [RFC2681] uses
packets with the constraints to assure that all packets in a single
measurement appear as the same flow. This flexibility covers ICMP,
UDP, and TCP. The accompanying methodology of [RFC2681] needs to be
expanded to report the sequential hop identifiers along with RTD
measurements, but no new metric definition is needed.
The advanced route assessment methods used in Paris-traceroute [PT] The advanced route assessment methods used in Paris-traceroute [PT]
keep the critical fields constant for every packet to maintain the keep the critical fields constant for every packet to maintain the
appearance of the same flow. Since route assessment can be conducted appearance of the same flow. In IPv6, it is sufficient to be routed
using TCP, UDP or ICMP packets, this method REQUIRES the Diffserv identically if the IP source and destination addresses and the
field, the protocol number, IP source and destination addresses, and FlowLabel are constant, see [RFC6437]. In IPv4, certain fields of
the port settings for TCP or UDP to be kept constant. For ICMP the IP header and the first four bytes of the IP payload should
probes, the method additionally REQUIRES keeping the type, code, and remain constant in a flow. In the IPv4 header, the IP source and
ICMP checksum constant (the latter occupy the corresponding positions destination addresses, protocol number, and Diffserv fields identify
in the header of an IP packet, e.g., bytes 20 to 23 when the header flows. The first four payload bytes include the UDP and TCP ports,
IP has no options). and the ICMP type, code, and checksum fields.
Maintaining a constant checksum in ICMP is most challenging because Maintaining a constant ICMP checksum in IPv4 is most challenging, as
the ICMP Sequence Number is part of the calculation. The advanced the ICMP sequence number or identifier fields will usually change for
traceroute method requires calculations using the IP Sequence Number different probes of the same path. Probes should use arbitrary bytes
Field and the Identifier Field, yielding a constant ICMP checksum in in the ICMP data field to offset changes to sequence number and
successive packets. For an example of calculations to maintain a identifier, thus keeping the checksum constant.
constant checksum, see Appendix A of [RFC7820], where revision of a
timestamp field is complemented by modifying the 2 octet checksum
complement field (these fields take the roles of the ICMP Sequence
Number and Identifier Fields, respectively).
For TCP and UDP packets, the checksum must also be kept constant. Finally, it is also essential to route the resulting ICMP Time
Therefore, the first four bytes of UDP (or TCP) data field are Exceeded messages along a consistent path. In IPv6, the fields above
modified to compensate for fields that change from packet to packet. are sufficient. In IPv4, the ICMP Time Exceeded message will contain
the IP header and the first eight bytes of the IP payload, which
affects its ICMP checksum. The TCP sequence number, UDP Length, and
UDP checksum will affect this value, and should remain constant.
Finally, the return path is also important to check. Taking into Formally, to maintain the same flow in the measurements to a
account that it is an ICMP time exceeded (during transit) packet, the particular hop, the Type-P-Route-Ensemble-Method-Variant packets
source and destination IP are constant for every reply. Then, we should be[PT]:
should consider the fields in the first 32 bits of the protocol on
the top of IP: the type and code of ICMP packet, and its checksum.
Again, to keep the ICMP checksum constant for the returning packets,
we need to consider the whole ICMP message. It contains the IP
header of the discarded packet plus the first 8 bytes of the IP
payload; that is some of the fields of the TCP header, the UDP header
plus four data bytes, the ICMP header plus four bytes. Therefore,
for the UDP case the data field is used to maintain the ICMP checksum
constant in the returning packet. For the ICMP case, the identifier
and sequence fields of the sent ICMP probe are manipulated to be
constant. The TCP case presents no problem because its first eight
bytes will be the same for every packet probe.
Formally, to maintain the same flow in the measurements to a certain o TCP case: For IPv4, the fields Src, Dst, port-Src, port_Dst,
hop, the Type-P-Route-Ensemble-Method-Variant packets should be[PT]: sequence number, and Diffserv Field SHOULD be the same. For IPv6,
the field FlowLabel, Src and Dst SHOULD be the same.
o TCP case: Fields Src, Dst, port-Src, port_Dst, and Diffserv Field o UDP case: For IPv4, the fields Src, Dst, port-Src, port-Dst,
should be the same. Diffserv should be the same, and the UDP-checksum SHOULD change to
keep the IP checksum of the ICMP time exceeded reply constant.
Then, the data length should be fixed, and the data field is used
to fixing it (consider that ICMP checksum uses its data field,
which contains the original IP header plus 8 bytes of UDP, where
TTL, IP identification, IP checksum, and UDP checksum changes).
o UDP case: Fields Src, Dst, port-Src, port-Dst, and Diffserv Field For IPv6, the field FlowLabel, and Source and Destination
should be the same, the UDP-checksum should change to keep the IP addresses SHOULD be the same.
checksum of the ICMP time exceeded reply constant. Then, the data
length should be fixed, and the data field is used to fixing it
(consider that ICMP checksum uses its data field, which contains
the original IP header plus 8 bytes of UDP, where TTL or Hop
Limit, IP identification, IP checksum, and UDP checksum changes).
o ICMP case: The Data field should compensate variations on TTL or o ICMP case: For IPv4, the Data field SHOULD compensate variations
Hop Limit, IP identification, and IP checksum for every packet. on TTL or Hop Limit, IP identification, and IP checksum for every
packet. There is no need to consider ICMPv6 because only
FlowLabel of IPv6 and Source and Destination addresses are used,
and all of them SHOULD be constant.
Then, the way to identify different hops and attempts of the same Then, the way to identify different hops and attempts of the same
flow is: flow is:
o TCP case: The IP identification field. o TCP case: The IP identification field.
o UDP case: The IP identification field. o UDP case: The IP identification field.
o ICMP case: The IP identification field, and ICMP Sequence number. o ICMP case: The IP identification field, and ICMP Sequence number.
skipping to change at page 13, line 43 skipping to change at page 14, line 25
stable results over time, particularly ingress and egress stable results over time, particularly ingress and egress
portions of the path. portions of the path.
2. SHOULD continue testing portions of the path that have previously 2. SHOULD continue testing portions of the path that have previously
exhibited ECMP load balancing. exhibited ECMP load balancing.
3. SHALL trigger re-assessment of the complete path and Route 3. SHALL trigger re-assessment of the complete path and Route
Ensemble, if any change in hops is observed for a specific (and Ensemble, if any change in hops is observed for a specific (and
previously tested) flow. previously tested) flow.
4.1.2. Routing Class C Identification 4.1.2. Routing Class Identification
There is an opportunity to apply the [RFC2330] notion of equal There is an opportunity to apply the [RFC2330] notion of equal
treatment for a class of packets, "...very useful to know if a given treatment for a class of packets, "...very useful to know if a given
Internet component treats equally a class C of different types of Internet component treats equally a class C of different types of
packets", as it applies to Route measurements. Knowledge of "class packets", as it applies to Route measurements. The notion of class C
C" parameters (unrelated to address classes of the past) on a path was examined further in [RFC8468] as it applied to load-balancing
potentially reduces the number of flows required for a given method flows over parallel paths, which is the case we develop here.
to assess a Route Ensemble over time. Knowledge of class C parameters (unrelated to address classes of the
past) on a path potentially reduces the number of flows required for
a given method to assess a Route Ensemble over time.
First, recognize that each Member Route of a Route Ensemble will have First, recognize that each Member Route of a Route Ensemble will have
a corresponding Routing Class C. Class C can be discovered by a corresponding class C. Class C can be discovered by testing with
testing with multiple flows, all of which traverse the unique set of multiple flows, all of which traverse the unique set of hops that
hops that comprise a specific Member Route. comprise a specific Member Route.
Second, recognize that the different Routing Classes depend primarily Second, recognize that the different classes depend primarily on the
on the hash functions used at each instance of ECMP load balancing on hash functions used at each instance of ECMP load balancing on the
the path. path.
Third, recognize the synergy with Temporal Composition methods Third, recognize the synergy with Temporal Composition methods
(described above) where evaluation intends to discover time-stable (described above), where evaluation intends to discover time-stable
portions of each Member Route so that more emphasis can be placed on portions of each Member Route, so that more emphasis can be placed on
ECMP portions that also determine Class C. ECMP portions that also determine class C.
The methods to assess the various Routing Class C characteristics The methods to assess the various class C characteristics benefit
benefit from the following measurement capabilities: from the following measurement capabilities:
o flows designed to determine which n-tuple header fields are o flows designed to determine which n-tuple header fields are
considered by a given hash function and ECMP hop on the path, and considered by a given hash function and ECMP hop on the path, and
which are not. This operation immediately narrows the search which are not. This operation immediately narrows the search
space, where possible, and partially defines a Routing Class C. space, where possible, and partially defines a class C.
o a priori knowledge of the possible types of hash functions in use o a priori knowledge of the possible types of hash functions in use
also helps to design the flows for testing (major router vendors also helps to design the flows for testing (major router vendors
publish information about these hash functions, examples are here publish information about these hash functions, examples are here
https://www.researchgate.net/ [LOAD_BALANCE].
publication/281571413_COMPARISON_OF_HASH_STRATEGIES_FOR_FLOW-
BASED_LOAD_BALANCING ).
o ability to direct the emphasis of current measurements on ECMP o ability to direct the emphasis of current measurements on ECMP
portions of the path, based on recent past measurement results portions of the path, based on recent past measurement results
(the Routing Class C of some portions of the path is essentially (the Routing Class of some portions of the path is essentially
"all packets"). "all packets").
4.1.3. Intermediate Observation Point Route Measurement 4.1.3. Intermediate Observation Point Route Measurement
There are many examples where passive monitoring of a flow at an There are many examples where passive monitoring of a flow at an
Observation Point within the network can detect unexpected Round Trip Observation Point within the network can detect unexpected Round Trip
Delay or Delay Variation. But how can the cause of the anomalous Delay or Delay Variation. But how can the cause of the anomalous
delay be investigated further --from the Observation Point -- delay be investigated further --from the Observation Point --
possibly located at an intermediate point on the path? possibly located at an intermediate point on the path?
skipping to change at page 15, line 19 skipping to change at page 15, line 49
4.2. Hybrid Methodologies 4.2. Hybrid Methodologies
The Hybrid Type I methods provide an alternative method for Route The Hybrid Type I methods provide an alternative method for Route
Member assessment. As mentioned in the Scope section, Member assessment. As mentioned in the Scope section,
[I-D.ietf-ippm-ioam-data] provides a possible set of data fields that [I-D.ietf-ippm-ioam-data] provides a possible set of data fields that
would support route identification. would support route identification.
In general, nodes in the measured domain would be equipped with In general, nodes in the measured domain would be equipped with
specific abilities: specific abilities:
o Support of the "Loopback" Flag (L-bit), where a copy of the packet o Store the identity of nodes that a packet has visited in header
is returned to the encapsulating node, and the packet is processed data fields, in the order the packet visited the nodes.
o Support of a "Loopback" capability, where a copy of the packet is
returned to the encapsulating node, and the packet is processed
like any other IOAM packet on the return transfer. like any other IOAM packet on the return transfer.
In addition to node identity, nodes may also identify the ingress and In addition to node identity, nodes may also identify the ingress and
egress interfaces utilized by the tracing packet, the time of day egress interfaces utilized by the tracing packet, the time of day
when the packet was processed, and other generic data (as described when the packet was processed, and other generic data (as described
in section 4 of [I-D.ietf-ippm-ioam-data]). Interface identification in section 4 of [I-D.ietf-ippm-ioam-data]). Interface identification
isn't necessarily limited to IP, i.e. different links in a bundle isn't necessarily limited to IP, i.e. different links in a bundle
(LACP) could be identified. Equally well, links without explicit IP (LACP) could be identified. Equally well, links without explicit IP
addresses can be identified (like with unnumbered interfaces in an addresses can be identified (like with unnumbered interfaces in an
IGP deployment). IGP deployment).
Note that Type-P packet specification for this method will likely be Note that the Type-P packet specification for this method will likely
a partial specification, because most of the packet fields are be a partial specification, because most of the packet fields are
determined by the user traffic. The packet header(s) added by the determined by the user traffic. The packet (encapsulation) header(s)
Hybrid method can certainly be specified in Type-P. added by the Hybrid method can certainly be specified in Type-P, in
unpopulated form.
4.3. Combining Different Methods 4.3. Combining Different Methods
In principle, there are advantages if the entity conducting Route In principle, there are advantages if the entity conducting Route
measurements can utilize both forms of advanced methods (active and measurements can utilize both forms of advanced methods (active and
hybrid), and combine the results. For example, if there are Nodes hybrid), and combine the results. For example, if there are Nodes
involved in the path that qualify as Cooperating Nodes, but not as involved in the path that qualify as Cooperating Nodes, but not as
Discoverable Nodes, then a more complete view of Hops on the path is Discoverable Nodes, then a more complete view of Hops on the path is
possible when a hybrid method (or interrogation protocol) is applied possible when a hybrid method (or interrogation protocol) is applied
and the results are combined with the active method results collected and the results are combined with the active method results collected
skipping to change at page 16, line 48 skipping to change at page 17, line 36
3. Congestion 3. Congestion
4. Inter-domain paths 4. Inter-domain paths
This categorization is widely accepted in the literature and among This categorization is widely accepted in the literature and among
operators alike, and it can be trusted with empirical data and operators alike, and it can be trusted with empirical data and
several sources as ground of truth (e.g., [RTTSub] ) but it is an several sources as ground of truth (e.g., [RTTSub] ) but it is an
inference measurement nonetheless [bdrmap][IDCong]. inference measurement nonetheless [bdrmap][IDCong].
The first two categories correspond to the physical distance The first two categories correspond to the physical distance
dependency on Round-Trip Delay (RTD) while the last one binds RTD dependency on Round-Trip Delay (RTD), the next one binds RTD with
with queueing delay on routers. Due to the significant contribution queueing delay on routers, and the last one helps to identify
of propagation delay in long distance hops, RTD will be on the order different ASes using traceroutes. Due to the significant
of 100ms on transatlantic hops, depending on the geolocation of the contribution of propagation delay in long-distance hops, RTD will be
vantage points. Moreover, RTD is typically greater than 480ms when on the order of 100ms on transatlantic hops, depending on the
two hops are connected using geostationary satellite technology geolocation of the vantage points. Moreover, RTD is typically higher
(i.e., their orbit is at 36000km). Detecting congestion with latency than 480ms when two hops are connected using geostationary satellite
implies deeper mathematical understanding since network traffic load technology (i.e., their orbit is at 36000km). Detecting congestion
is not stationary. Nonetheless, as the first approach, a link seems with latency implies deeper mathematical understanding since network
to be congested if after sending several traceroute probes, it is traffic load is not stationary. Nonetheless, as the first approach,
possible to detect congestion observing different statistics a link seems to be congested if, after sending several traceroute
parameters (e.g., see [IDCong]). probes, it is possible to detect congestion observing different
statistics parameters (e.g., see [IDCong]). Finally, to recognize
6. Tools to Measure Delays in the Internet distinctive ASes in the same traceroute path is challenging, because
more data is needed, like AS relationships and RIR delegations among
Internet routing is complex because it depends on the policies of other (for more detail, please consult [bdrmap]).
thousands Autonomous Systems (AS). While most of the routers perform
load balancing on flows using Equal Cost Multiple Path (ECMP), a few
still divide the workload through packet-based techniques. The
former scenario is defined according to [RFC2991] while the latter
generates a round-robin scheme to deliver every new outgoing packet.
ECMP uses a hashing function to ensure that every packet of a flow is
delivered by the same path, and this avoids increasing the packet
delay variation and possibly producing overwhelming packet reordering
in TCP flows.
Taking into account that Internet protocol was designed under the
"end-to-end" principle, the IP payload and its header do not provide
any information about the routes or path necessary to reach some
destination. For this reason, the well-known tool traceroute was
developed to gather the IP addresses of each hop along a path using
the ICMP protocol [RFC0792]. Besides, traceroute adds the measured
RTD from each hop. However, the growing complexity of the Internet
makes it more challenging to develop an accurate traceroute
implementation. For instance, the early traceroute tools would be
inaccurate in the current network, mainly because they were not
designed to retain flow state. However, evolved traceroute tools,
such as Paris-traceroute [PT] [MLB] and Scamper [SCAMPER], expect to
encounter ECMP and achieve more accurate results when they do.
Paris-traceroute-like tools operate in the following way: every
packet should follow the same path because the sensitive fields of
the header are controlled to appear as the same flow. This means
that source and destination IP addresses, source and destination port
numbers are the same in every packet. Additionally, Differentiated
Services Code Point (DSCP), checksum and ICMP code should remain
constant since they may affect the path selection.
Today's traceroute tools can send either UDP, TCP or ICMP packet
probes. Since ICMP header does not include transport layer
information, there are no fields for source and destination port
numbers. For this reason, these tools keep constant ICMP type, code,
and checksum fields to generate a kind of flow. However, the
checksum may vary in every packet, therefore when probes use ICMP
packets, ICMP Identifier and sequence number are manipulated to
maintain constant checksum in every packet. On the other hand, when
UDP probes are generated, the expected variation in the checksum of
each packet is again compensated by manipulating the payload.
Paris-traceroute allows its users to measure RTD in every hop of the
path for a particular flow. Furthermore, either Paris-traceroute or
Scamper is capable of unveiling the many available paths between a
source and destination (which are visible to this method). This task
is accomplished by repeating complete traceroute measurements with
different flow parameters for each measurement. The Framework for IP
Performance Metrics (IPPM) ([RFC2330] updated by [RFC7312]) has the
flexibility to require that the Round-Trip Delay measurement
[RFC2681] uses packets with the constraints to assure that all
packets in a single measurement appear as the same flow. This
flexibility covers ICMP, UDP, and TCP. The accompanying methodology
of [RFC2681] needs to be expanded to report the sequential hop
identifiers along with RTD measurements, but no new metric definition
is needed.
7. RTD Measurements Statistics 6. RTD Measurements Statistics
Several articles have shown that network traffic presents a self- Several articles have shown that network traffic presents a self-
similar nature [SSNT] [MLRM] which is accountable for filling the similar nature [SSNT] [MLRM] which is accountable for filling the
queues of the routers. Moreover, router queues are designed to queues of the routers. Moreover, router queues are designed to
handle traffic bursts, which is one of the most remarkable features handle traffic bursts, which is one of the most remarkable features
of self-similarity. Naturally, while queue length increases, the of self-similarity. Naturally, while queue length increases, the
delay to traverse the queue increases as well and leads to an delay to traverse the queue increases as well and leads to an
increase on RTD. Due to traffic bursts generate short-term overflow increase on RTD. Due to traffic bursts generating short-term
on buffers (spiky patterns), every RTD only depicts the queueing overflow on buffers (spiky patterns), every RTD only depicts the
status on the instant when that packet probe was in transit. For queueing status on the instant when that packet probe was in transit.
this reason, several RTD measurements during a time window could For this reason, several RTD measurements during a time window could
begin to describe the random behavior of latency. Loss must also be begin to describe the random behavior of latency. Loss must also be
accounted for in the methodology. accounted for in the methodology.
To understand the ongoing process, examining the quartiles provides a To understand the ongoing process, examining the quartiles provides a
non-parametric way of analysis. Quartiles are defined by five non-parametric way of analysis. Quartiles are defined by five
values: minimum RTD (m), RTD value of the 25% of the Empirical values: minimum RTD (m), RTD value of the 25% of the Empirical
Cumulative Distribution Function (ECDF) (Q1), the median value (Q2), Cumulative Distribution Function (ECDF) (Q1), the median value (Q2),
the RTD value of the 75% of the ECDF (Q3) and the maximum RTD (M). the RTD value of the 75% of the ECDF (Q3) and the maximum RTD (M).
Congestion can be inferred when RTD measurements are spread apart, Congestion can be inferred when RTD measurements are spread apart,
and consequently, the Inter-Quartile Range (IQR), the distance and consequently, the Inter-Quartile Range (IQR), the distance
skipping to change at page 19, line 11 skipping to change at page 18, line 38
This procedure requires to compute quartile values "on the fly" using This procedure requires to compute quartile values "on the fly" using
the algorithm presented in [P2]. the algorithm presented in [P2].
This procedure allows us to update the quartiles value whenever a new This procedure allows us to update the quartiles value whenever a new
measurement arrives, which is radically different from classic measurement arrives, which is radically different from classic
methods of computing quartiles because they need to use the whole methods of computing quartiles because they need to use the whole
dataset to compute the values. This way of calculus provides savings dataset to compute the values. This way of calculus provides savings
in memory and computing time. in memory and computing time.
To sum up, the proposed measurement procedure consists in performing To sum up, the proposed measurement procedure consists of performing
traceroutes several times to obtain samples of the RTD in every hop traceroutes several times to obtain samples of the RTD in every hop
from a path, during a time window (W) and compute the quantiles for from a path, during a time window (W), and compute the quartiles for
every hop. This could be done for a single Member Route flow or for every hop. This procedure could be done for a single Member Route
every detected Route Ensemble flow. flow, with parameter E set as False, or for every detected Route
Ensemble flow (E=True).
The identification of a specific Hop in traceroute is based on the IP The identification of a specific Hop in traceroute is based on the IP
origin address of the returned ICMP Time Exceeded packet, and on the origin address of the returned ICMP Time Exceeded packet, and on the
distance identified by the value set in the TTL field inserted by distance identified by the value set in the TTL field inserted by
traceroute. As this specific Hop can be reached by different paths, traceroute. As this specific Hop can be reached by different paths,
also the IP source and destination addresses of the traceroute packet also the IP source and destination addresses of the traceroute packet
need to be recorded. Finally, different return paths are need to be recorded. Finally, different return paths are
distinguished by evaluating the ICMP Time Exceeded TTL (of the reply distinguished by evaluating the ICMP Time Exceeded TTL (of the reply
message): if this TTL is constant for different paths containing the message): if this TTL is constant for different paths containing the
same Hop, the return paths have the same distance. Moreover, this same Hop, the return paths have the same distance. Moreover, this
distance can be estimated considering that the TTL value is normally distance can be estimated considering that the TTL value is normally
initialized with values 64, 128, or 255. The 5-tuple (origin IP, initialized with values 64, 128, or 255. The 5-tuple (origin IP,
destination IP, reply IP, distance, response TTL) univocally destination IP, reply IP, distance, response TTL) univocally
identifies every measurement. identifies every measurement.
This algorithm below runs in the origin of the traceroute. It This algorithm below runs in the origin of the traceroute. It
returns the Qs quartiles for every Hop and Alt (alternative paths returns the Qs quartiles for every Hop and Alt (alternative paths
because of balancing). The following parameters are used: W window because of balancing). Notice that the "Alt" parameter condenses the
time of the measurement, i_t time between two measurements, Dst parameters of the 5-tuple (origin IP, destination IP, reply IP,
destination IP address. distance, response TTL), i.e., one for each possible combination.
================================================================ ================================================================
1 input: W (window time of the measurement) 1 input: W (window time of the measurement)
2 i_t (time between two measurements) 2 i_t (time between two measurements)
3 E (True: exhaustive, False: a single path) 3 E (True: exhaustive, False: a single path)
4 Dst (destination IP address) 4 Dst (destination IP address)
5 output: Qs (quartiles for every Hop and Alt) 5 output: Qs (quartiles for every Hop and Alt)
---------------------------------------------------------------- ----------------------------------------------------------------
6 T := start_timer(W) 6 T := start_timer(W)
7 while T is not finished do: 7 while T is not finished do:
skipping to change at page 20, line 28 skipping to change at page 19, line 39
12 | done 12 | done
13 | wait until i_t timer is expired 13 | wait until i_t timer is expired
14 done 14 done
15 return (Qs) 15 return (Qs)
================================================================ ================================================================
During the time W, lines 6 and 7 assure that the measurement loop is During the time W, lines 6 and 7 assure that the measurement loop is
made. Line 8 and 13 set a timer for each cycle of measurements. A made. Line 8 and 13 set a timer for each cycle of measurements. A
cycle comprises the traceroutes packets, considering every possible cycle comprises the traceroutes packets, considering every possible
Hop and the alternatives paths in the Alt variable (ensured in lines Hop and the alternatives paths in the Alt variable (ensured in lines
9-12). In line 9 the advance-traceroute could be either Paris- 9-12). In line 9, the advance-traceroute could be either Paris-
traceroute or Scamper, which will use "exhaustive" mode or "tracelb" traceroute or Scamper, which will use the "exhaustive" mode or
option if E is set True, respectively. The procedure returns a list "tracelb" option if E is set True, respectively. The procedure
of tuples (m,Q1,Q2,Q3,M) for each intermediate hop in the path returns a list of tuples (m,Q1,Q2,Q3,M) for each intermediate hop, or
towards the Dst. Additionally, it could also return path variations "Alt" in as a function of the 5-tuple, in the path towards the Dst.
using "alt" variable.Finally, lines 10 and 12 stores each measurement Finally, lines 10 through 12 stores each measurement into the real-
into the real-time quartiles computation. time quartiles computation.
8. Conclusions Notice there are cases where the even having a unique hop at distance
h from the Src to Dst, the returning path could have several
possibilities, yielding in different total paths. In this situation,
the algorithm will return more "Alt" for this particular hop.
Combining the method proposed in Section 4 and statistics in 7. Conclusions
Section 7, we can measure the performance of paths interconnecting
two endpoints in Internet, and attempt the categorization of link
types and congestion presence based on RTD.
9. Security Considerations This document introduces a method to perform statistical RTD
measurements in a path, according to the actual state of the art
regarding the traffic nature and the flow balance method in ECMP
cases, which can help to tackle different performance situations in
the network. Some of these cases are enumerated in Section 5, while
our method is proposed in Section 4, and the algorithm in Section 6.
The importance of this algorithm is that it deals with the different
topological aspects and the self-similar (i.e., not Poisson-
distributed) nature of the traffic.
8. Security Considerations
The security considerations that apply to any active measurement of The security considerations that apply to any active measurement of
live paths are relevant here as well. See [RFC4656] and [RFC5357]. live paths are relevant here as well. See [RFC4656] and [RFC5357].
The active measurement process of "changing several fields to keep The active measurement process of "changing several fields to keep
the checksum of different packets identical" does not require special the checksum of different packets identical" does not require special
security considerations because it is part of synthetic traffic security considerations because it is part of synthetic traffic
generation, and is designed to have minimal to zero impact on network generation, and is designed to have minimal to zero impact on network
processing (to process the packets for ECMP). processing (to process the packets for ECMP).
skipping to change at page 21, line 17 skipping to change at page 20, line 40
When considering privacy of those involved in measurement or those When considering privacy of those involved in measurement or those
whose traffic is measured, the sensitive information available to whose traffic is measured, the sensitive information available to
potential observers is greatly reduced when using active techniques potential observers is greatly reduced when using active techniques
which are within this scope of work. Passive observations of user which are within this scope of work. Passive observations of user
traffic for measurement purposes raise many privacy issues. We refer traffic for measurement purposes raise many privacy issues. We refer
the reader to the privacy considerations described in the Large Scale the reader to the privacy considerations described in the Large Scale
Measurement of Broadband Performance (LMAP) Framework [RFC7594], Measurement of Broadband Performance (LMAP) Framework [RFC7594],
which covers active and passive techniques. which covers active and passive techniques.
10. IANA Considerations 9. IANA Considerations
This memo makes no requests of IANA. We thank the good folks at IANA This memo makes no requests of IANA. We thank the good folks at IANA
for having checked this section anyway. for having checked this section anyway.
11. Acknowledgements 10. Acknowledgements
The original 3 authors acknowledge Ruediger Geib, for his penetrating The original 3 authors acknowledge Ruediger Geib, for his penetrating
comments on the initial draft, and his initial text for the comments on the initial draft, and his initial text for the
Appendix on MPLS. Carlos Pignataro challenged the authors to Appendix on MPLS. Carlos Pignataro challenged the authors to
consider a wider scope, and applied his substantial expertise with consider a wider scope, and applied his substantial expertise with
many technologies and their measurement features in his extensive many technologies and their measurement features in his extensive
comments. Frank Brockners also shared useful comments, so did Footer comments. Frank Brockners also shared useful comments, so did Footer
Foote. We thank them all! Foote. We thank them all!
12. Appendix I MPLS Methods for Route Assessment 11. Appendix I MPLS Methods for Route Assessment
A Node assessing an MPLS path must be part of the MPLS domain where A Node assessing an MPLS path must be part of the MPLS domain where
the path is implemented. When this condition is met, RFC 8029 the path is implemented. When this condition is met, RFC 8029
provides a powerful set of mechanisms to detect "correct operation of provides a powerful set of mechanisms to detect "correct operation of
the data plane, as well as a mechanism to verify the data plane the data plane, as well as a mechanism to verify the data plane
against the control plane" [RFC8029]. against the control plane" [RFC8029].
MPLS routing is based on the presence of a Forwarding Equivalence MPLS routing is based on the presence of a Forwarding Equivalence
Class (FEC) Stack in all visited Nodes. Selecting one of several Class (FEC) Stack in all visited Nodes. Selecting one of several
Equal Cost Multi Path (ECMP) is however based on information hidden Equal Cost Multi Path (ECMP) is however based on information hidden
skipping to change at page 22, line 22 skipping to change at page 21, line 46
address (if available). This ensures that the next hop will receive address (if available). This ensures that the next hop will receive
a properly coded MPLS Echo request in the next step route of a properly coded MPLS Echo request in the next step route of
assessment. assessment.
[RFC8403] explains how a central Path Monitoring System could be used [RFC8403] explains how a central Path Monitoring System could be used
to detect arbitrary MPLS paths between any routers within a single to detect arbitrary MPLS paths between any routers within a single
MPLS domain. The combination of MPLS forwarding, Segment Routing and MPLS domain. The combination of MPLS forwarding, Segment Routing and
MPLS traceroute offers a simple architecture and a powerful mechanism MPLS traceroute offers a simple architecture and a powerful mechanism
to detect and validate (segment routed) MPLS paths. to detect and validate (segment routed) MPLS paths.
13. References 12. References
12.1. Normative References
13.1. Normative References
[I-D.ietf-ippm-ioam-data] [I-D.ietf-ippm-ioam-data]
Brockners, F., Bhandari, S., Pignataro, C., Gredler, H., Brockners, F., Bhandari, S., Pignataro, C., Gredler, H.,
Leddy, J., Youell, S., Mizrahi, T., Mozes, D., Lapukhov, Leddy, J., Youell, S., Mizrahi, T., Mozes, D., Lapukhov,
P., remy@barefootnetworks.com, r., daniel.bernier@bell.ca, P., remy@barefootnetworks.com, r., daniel.bernier@bell.ca,
d., and J. Lemon, "Data Fields for In-situ OAM", draft- d., and J. Lemon, "Data Fields for In-situ OAM", draft-
ietf-ippm-ioam-data-08 (work in progress), October 2019. ietf-ippm-ioam-data-09 (work in progress), March 2020.
[RFC0792] Postel, J., "Internet Control Message Protocol", STD 5, [RFC0792] Postel, J., "Internet Control Message Protocol", STD 5,
RFC 792, DOI 10.17487/RFC0792, September 1981, RFC 792, DOI 10.17487/RFC0792, September 1981,
<https://www.rfc-editor.org/info/rfc792>. <https://www.rfc-editor.org/info/rfc792>.
[RFC1122] Braden, R., Ed., "Requirements for Internet Hosts - [RFC1122] Braden, R., Ed., "Requirements for Internet Hosts -
Communication Layers", STD 3, RFC 1122, Communication Layers", STD 3, RFC 1122,
DOI 10.17487/RFC1122, October 1989, DOI 10.17487/RFC1122, October 1989,
<https://www.rfc-editor.org/info/rfc1122>. <https://www.rfc-editor.org/info/rfc1122>.
skipping to change at page 23, line 10 skipping to change at page 22, line 36
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997, DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>. <https://www.rfc-editor.org/info/rfc2119>.
[RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis,
"Framework for IP Performance Metrics", RFC 2330, "Framework for IP Performance Metrics", RFC 2330,
DOI 10.17487/RFC2330, May 1998, DOI 10.17487/RFC2330, May 1998,
<https://www.rfc-editor.org/info/rfc2330>. <https://www.rfc-editor.org/info/rfc2330>.
[RFC2460] Deering, S. and R. Hinden, "Internet Protocol, Version 6
(IPv6) Specification", RFC 2460, DOI 10.17487/RFC2460,
December 1998, <https://www.rfc-editor.org/info/rfc2460>.
[RFC2675] Borman, D., Deering, S., and R. Hinden, "IPv6 Jumbograms",
RFC 2675, DOI 10.17487/RFC2675, August 1999,
<https://www.rfc-editor.org/info/rfc2675>.
[RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip [RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip
Delay Metric for IPPM", RFC 2681, DOI 10.17487/RFC2681, Delay Metric for IPPM", RFC 2681, DOI 10.17487/RFC2681,
September 1999, <https://www.rfc-editor.org/info/rfc2681>. September 1999, <https://www.rfc-editor.org/info/rfc2681>.
[RFC2991] Thaler, D. and C. Hopps, "Multipath Issues in Unicast and [RFC2991] Thaler, D. and C. Hopps, "Multipath Issues in Unicast and
Multicast Next-Hop Selection", RFC 2991, Multicast Next-Hop Selection", RFC 2991,
DOI 10.17487/RFC2991, November 2000, DOI 10.17487/RFC2991, November 2000,
<https://www.rfc-editor.org/info/rfc2991>. <https://www.rfc-editor.org/info/rfc2991>.
[RFC4494] Song, JH., Poovendran, R., and J. Lee, "The AES-CMAC-96
Algorithm and Its Use with IPsec", RFC 4494,
DOI 10.17487/RFC4494, June 2006,
<https://www.rfc-editor.org/info/rfc4494>.
[RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M. [RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M.
Zekauskas, "A One-way Active Measurement Protocol Zekauskas, "A One-way Active Measurement Protocol
(OWAMP)", RFC 4656, DOI 10.17487/RFC4656, September 2006, (OWAMP)", RFC 4656, DOI 10.17487/RFC4656, September 2006,
<https://www.rfc-editor.org/info/rfc4656>. <https://www.rfc-editor.org/info/rfc4656>.
[RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K., and J. [RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K., and J.
Babiarz, "A Two-Way Active Measurement Protocol (TWAMP)", Babiarz, "A Two-Way Active Measurement Protocol (TWAMP)",
RFC 5357, DOI 10.17487/RFC5357, October 2008, RFC 5357, DOI 10.17487/RFC5357, October 2008,
<https://www.rfc-editor.org/info/rfc5357>. <https://www.rfc-editor.org/info/rfc5357>.
[RFC5388] Niccolini, S., Tartarelli, S., Quittek, J., Dietz, T., and [RFC5388] Niccolini, S., Tartarelli, S., Quittek, J., Dietz, T., and
M. Swany, "Information Model and XML Data Model for M. Swany, "Information Model and XML Data Model for
Traceroute Measurements", RFC 5388, DOI 10.17487/RFC5388, Traceroute Measurements", RFC 5388, DOI 10.17487/RFC5388,
December 2008, <https://www.rfc-editor.org/info/rfc5388>. December 2008, <https://www.rfc-editor.org/info/rfc5388>.
[RFC5644] Stephan, E., Liang, L., and A. Morton, "IP Performance
Metrics (IPPM): Spatial and Multicast", RFC 5644,
DOI 10.17487/RFC5644, October 2009,
<https://www.rfc-editor.org/info/rfc5644>.
[RFC5835] Morton, A., Ed. and S. Van den Berghe, Ed., "Framework for [RFC5835] Morton, A., Ed. and S. Van den Berghe, Ed., "Framework for
Metric Composition", RFC 5835, DOI 10.17487/RFC5835, April Metric Composition", RFC 5835, DOI 10.17487/RFC5835, April
2010, <https://www.rfc-editor.org/info/rfc5835>. 2010, <https://www.rfc-editor.org/info/rfc5835>.
[RFC5837] Atlas, A., Ed., Bonica, R., Ed., Pignataro, C., Ed., Shen, [RFC5837] Atlas, A., Ed., Bonica, R., Ed., Pignataro, C., Ed., Shen,
N., and JR. Rivers, "Extending ICMP for Interface and N., and JR. Rivers, "Extending ICMP for Interface and
Next-Hop Identification", RFC 5837, DOI 10.17487/RFC5837, Next-Hop Identification", RFC 5837, DOI 10.17487/RFC5837,
April 2010, <https://www.rfc-editor.org/info/rfc5837>. April 2010, <https://www.rfc-editor.org/info/rfc5837>.
[RFC6282] Hui, J., Ed. and P. Thubert, "Compression Format for IPv6
Datagrams over IEEE 802.15.4-Based Networks", RFC 6282,
DOI 10.17487/RFC6282, September 2011,
<https://www.rfc-editor.org/info/rfc6282>.
[RFC6437] Amante, S., Carpenter, B., Jiang, S., and J. Rajahalme, [RFC6437] Amante, S., Carpenter, B., Jiang, S., and J. Rajahalme,
"IPv6 Flow Label Specification", RFC 6437, "IPv6 Flow Label Specification", RFC 6437,
DOI 10.17487/RFC6437, November 2011, DOI 10.17487/RFC6437, November 2011,
<https://www.rfc-editor.org/info/rfc6437>. <https://www.rfc-editor.org/info/rfc6437>.
[RFC6438] Carpenter, B. and S. Amante, "Using the IPv6 Flow Label [RFC6438] Carpenter, B. and S. Amante, "Using the IPv6 Flow Label
for Equal Cost Multipath Routing and Link Aggregation in for Equal Cost Multipath Routing and Link Aggregation in
Tunnels", RFC 6438, DOI 10.17487/RFC6438, November 2011, Tunnels", RFC 6438, DOI 10.17487/RFC6438, November 2011,
<https://www.rfc-editor.org/info/rfc6438>. <https://www.rfc-editor.org/info/rfc6438>.
[RFC6564] Krishnan, S., Woodyatt, J., Kline, E., Hoagland, J., and
M. Bhatia, "A Uniform Format for IPv6 Extension Headers",
RFC 6564, DOI 10.17487/RFC6564, April 2012,
<https://www.rfc-editor.org/info/rfc6564>.
[RFC6673] Morton, A., "Round-Trip Packet Loss Metrics", RFC 6673, [RFC6673] Morton, A., "Round-Trip Packet Loss Metrics", RFC 6673,
DOI 10.17487/RFC6673, August 2012, DOI 10.17487/RFC6673, August 2012,
<https://www.rfc-editor.org/info/rfc6673>. <https://www.rfc-editor.org/info/rfc6673>.
[RFC7045] Carpenter, B. and S. Jiang, "Transmission and Processing
of IPv6 Extension Headers", RFC 7045,
DOI 10.17487/RFC7045, December 2013,
<https://www.rfc-editor.org/info/rfc7045>.
[RFC7312] Fabini, J. and A. Morton, "Advanced Stream and Sampling [RFC7312] Fabini, J. and A. Morton, "Advanced Stream and Sampling
Framework for IP Performance Metrics (IPPM)", RFC 7312, Framework for IP Performance Metrics (IPPM)", RFC 7312,
DOI 10.17487/RFC7312, August 2014, DOI 10.17487/RFC7312, August 2014,
<https://www.rfc-editor.org/info/rfc7312>. <https://www.rfc-editor.org/info/rfc7312>.
[RFC7799] Morton, A., "Active and Passive Metrics and Methods (with [RFC7799] Morton, A., "Active and Passive Metrics and Methods (with
Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799, Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799,
May 2016, <https://www.rfc-editor.org/info/rfc7799>. May 2016, <https://www.rfc-editor.org/info/rfc7799>.
[RFC7820] Mizrahi, T., "UDP Checksum Complement in the One-Way
Active Measurement Protocol (OWAMP) and Two-Way Active
Measurement Protocol (TWAMP)", RFC 7820,
DOI 10.17487/RFC7820, March 2016,
<https://www.rfc-editor.org/info/rfc7820>.
[RFC8029] Kompella, K., Swallow, G., Pignataro, C., Ed., Kumar, N., [RFC8029] Kompella, K., Swallow, G., Pignataro, C., Ed., Kumar, N.,
Aldrin, S., and M. Chen, "Detecting Multiprotocol Label Aldrin, S., and M. Chen, "Detecting Multiprotocol Label
Switched (MPLS) Data-Plane Failures", RFC 8029, Switched (MPLS) Data-Plane Failures", RFC 8029,
DOI 10.17487/RFC8029, March 2017, DOI 10.17487/RFC8029, March 2017,
<https://www.rfc-editor.org/info/rfc8029>. <https://www.rfc-editor.org/info/rfc8029>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>. May 2017, <https://www.rfc-editor.org/info/rfc8174>.
13.2. Informative References [RFC8468] Morton, A., Fabini, J., Elkins, N., Ackermann, M., and V.
Hegde, "IPv4, IPv6, and IPv4-IPv6 Coexistence: Updates for
the IP Performance Metrics (IPPM) Framework", RFC 8468,
DOI 10.17487/RFC8468, November 2018,
<https://www.rfc-editor.org/info/rfc8468>.
12.2. Informative References
[bdrmap] Luckie, M., Dhamdhere, A., Huffaker, B., Clark, D., and [bdrmap] Luckie, M., Dhamdhere, A., Huffaker, B., Clark, D., and
KC. Claffy, "bdrmap: Inference of Borders Between IP KC. Claffy, "bdrmap: Inference of Borders Between IP
Networks", In Proceedings of the 2016 ACM on Internet Networks", In Proceedings of the 2016 ACM on Internet
Measurement Conference, pp. 381-396. ACM, 2016. Measurement Conference, pp. 381-396. ACM, 2016.
[IDCong] Luckie, M., Dhamdhere, A., Clark, D., and B. Huffaker, [IDCong] Luckie, M., Dhamdhere, A., Clark, D., and B. Huffaker,
"Challenges in inferring Internet interdomain "Challenges in inferring Internet interdomain
congestion", In Proceedings of the 2014 Conference on congestion", In Proceedings of the 2014 Conference on
Internet Measurement Conference, pp. 15-22. ACM, 2014. Internet Measurement Conference, pp. 15-22. ACM, 2014.
[LOAD_BALANCE]
Sanguanpong, S., Pittayapitak, W., and K. Kasom Koht-Arsa,
"COMPARISON OF HASH STRATEGIES FOR FLOW-BASED LOAD
BALANCING", International Journal of Electronic Commerce
Studies, Vol.6, No.2, pp.259-268.
http://dx.doi.org/10.7903/ijecs.1346, 2015.
[MLB] Augustin, B., Friedman, T., and R. Teixeira, "Measuring [MLB] Augustin, B., Friedman, T., and R. Teixeira, "Measuring
load-balanced paths in the Internet", Proceedings of the load-balanced paths in the Internet", Proceedings of the
7th ACM SIGCOMM conference on Internet measurement, pp. 7th ACM SIGCOMM conference on Internet measurement, pp.
149-160. ACM, 2007., 2007. 149-160. ACM, 2007., 2007.
[MLRM] Fontugne, R., Mazel, J., and K. Fukuda, "An empirical [MLRM] Fontugne, R., Mazel, J., and K. Fukuda, "An empirical
mixture model for large-scale RTT measurements", 2015 mixture model for large-scale RTT measurements", 2015
IEEE Conference on Computer Communications (INFOCOM), pp. IEEE Conference on Computer Communications (INFOCOM), pp.
2470-2478. IEEE, 2015., 2015. 2470-2478. IEEE, 2015., 2015.
[P2] Jain, R. and I. Chlamtac, "The P 2 algorithm for dynamic [P2] Jain, R. and I. Chlamtac, "The P 2 algorithm for dynamic
calculation of quantiles and histograms without storing calculation of quartiles and histograms without storing
observations", Communications of the ACM 28.10 (1985): observations", Communications of the ACM 28.10 (1985):
1076-1085, 2015. 1076-1085, 2015.
[PT] Augustin, B., Cuvellier, X., Orgogozo, B., Viger, F., [PT] Augustin, B., Cuvellier, X., Orgogozo, B., Viger, F.,
Friedman, T., Latapy, M., Magnien, C., and R. Teixeira, Friedman, T., Latapy, M., Magnien, C., and R. Teixeira,
"Avoiding traceroute anomalies with Paris traceroute", "Avoiding traceroute anomalies with Paris traceroute",
Proceedings of the 6th ACM SIGCOMM conference on Internet Proceedings of the 6th ACM SIGCOMM conference on Internet
measurement, pp. 153-158. ACM, 2006., 2006. measurement, pp. 153-158. ACM, 2006., 2006.
[RFC7325] Villamizar, C., Ed., Kompella, K., Amante, S., Malis, A., [RFC7325] Villamizar, C., Ed., Kompella, K., Amante, S., Malis, A.,
skipping to change at page 26, line 41 skipping to change at page 25, line 47
[SSNT] Park, K. and W. Willinger, "Self-Similar Network Traffic [SSNT] Park, K. and W. Willinger, "Self-Similar Network Traffic
and Performance Evaluation (1st ed.)", John Wiley & Sons, and Performance Evaluation (1st ed.)", John Wiley & Sons,
Inc., New York, NY, USA, 2000. Inc., New York, NY, USA, 2000.
Authors' Addresses Authors' Addresses
J. Ignacio Alvarez-Hamelin J. Ignacio Alvarez-Hamelin
Universidad de Buenos Aires Universidad de Buenos Aires
Av. Paseo Colon 850 Av. Paseo Colon 850
Buenos Aires C1063ACV Buenos Aires C1063ACV
Argentine Argentina
Phone: +54 11 5285-0716 Phone: +54 11 5285-0716
Email: ihameli@cnet.fi.uba.ar Email: ihameli@cnet.fi.uba.ar
URI: http://cnet.fi.uba.ar/ignacio.alvarez-hamelin/ URI: http://cnet.fi.uba.ar/ignacio.alvarez-hamelin/
Al Morton Al Morton
AT&T Labs AT&T Labs
200 Laurel Avenue South 200 Laurel Avenue South
Middletown, NJ 07748 Middletown, NJ 07748
USA USA
 End of changes. 64 change blocks. 
268 lines changed or deleted 230 lines changed or added

This html diff was produced by rfcdiff 1.47. The latest version is available from http://tools.ietf.org/tools/rfcdiff/