draft-ietf-ippm-alt-mark-14.txt   rfc8321.txt 
Network Working Group G. Fioccola, Ed. Internet Engineering Task Force (IETF) G. Fioccola, Ed.
Internet-Draft A. Capello Request for Comments: 8321 A. Capello
Intended status: Experimental M. Cociglio Category: Experimental M. Cociglio
Expires: June 10, 2018 L. Castaldelli ISSN: 2070-1721 L. Castaldelli
Telecom Italia Telecom Italia
M. Chen M. Chen
L. Zheng L. Zheng
Huawei Technologies Huawei Technologies
G. Mirsky G. Mirsky
ZTE ZTE
T. Mizrahi T. Mizrahi
Marvell Marvell
December 7, 2017 January 2018
Alternate Marking method for passive and hybrid performance monitoring Alternate-Marking Method for Passive and Hybrid Performance Monitoring
draft-ietf-ippm-alt-mark-14
Abstract Abstract
This document describes a method to perform packet loss, delay and This document describes a method to perform packet loss, delay, and
jitter measurements on live traffic. This method is based on jitter measurements on live traffic. This method is based on an
Alternate Marking (Coloring) technique. A report is provided in Alternate-Marking (coloring) technique. A report is provided in
order to explain an example and show the method applicability. This order to explain an example and show the method applicability. This
technology can be applied in various situations as detailed in this technology can be applied in various situations, as detailed in this
document and could be considered passive or hybrid depending on the document, and could be considered Passive or Hybrid depending on the
application. application.
Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP
14 [RFC2119] [RFC8174] when, and only when, they appear in all
capitals, as shown here.
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This document is not an Internet Standards Track specification; it is
provisions of BCP 78 and BCP 79. published for examination, experimental implementation, and
evaluation.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months This document defines an Experimental Protocol for the Internet
and may be updated, replaced, or obsoleted by other documents at any community. This document is a product of the Internet Engineering
time. It is inappropriate to use Internet-Drafts as reference Task Force (IETF). It represents the consensus of the IETF
material or to cite them other than as "work in progress." community. It has received public review and has been approved for
publication by the Internet Engineering Steering Group (IESG). Not
all documents approved by the IESG are a candidate for any level of
Internet Standard; see Section 2 of RFC 7841.
This Internet-Draft will expire on June 10, 2018. Information about the current status of this document, any errata,
and how to provide feedback on it may be obtained at
https://www.rfc-editor.org/info/rfc8321.
Copyright Notice Copyright Notice
Copyright (c) 2017 IETF Trust and the persons identified as the Copyright (c) 2018 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Overview of the method . . . . . . . . . . . . . . . . . . . 5 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 5
3. Detailed description of the method . . . . . . . . . . . . . 6 2. Overview of the Method . . . . . . . . . . . . . . . . . . . 5
3.1. Packet loss measurement . . . . . . . . . . . . . . . . . 6 3. Detailed Description of the Method . . . . . . . . . . . . . 6
3.1.1. Coloring the packets . . . . . . . . . . . . . . . . 11 3.1. Packet Loss Measurement . . . . . . . . . . . . . . . . . 6
3.1.2. Counting the packets . . . . . . . . . . . . . . . . 11 3.1.1. Coloring the Packets . . . . . . . . . . . . . . . . 11
3.1.3. Collecting data and calculating packet loss . . . . . 12 3.1.2. Counting the Packets . . . . . . . . . . . . . . . . 12
3.2. Timing aspects . . . . . . . . . . . . . . . . . . . . . 13 3.1.3. Collecting Data and Calculating Packet Loss . . . . . 13
3.3. One-way delay measurement . . . . . . . . . . . . . . . . 14 3.2. Timing Aspects . . . . . . . . . . . . . . . . . . . . . 13
3.3.1. Single marking methodology . . . . . . . . . . . . . 14 3.3. One-Way Delay Measurement . . . . . . . . . . . . . . . . 15
3.3.2. Double marking methodology . . . . . . . . . . . . . 16 3.3.1. Single-Marking Methodology . . . . . . . . . . . . . 15
3.4. Delay variation measurement . . . . . . . . . . . . . . . 17 3.3.2. Double-Marking Methodology . . . . . . . . . . . . . 17
3.4. Delay Variation Measurement . . . . . . . . . . . . . . . 18
4. Considerations . . . . . . . . . . . . . . . . . . . . . . . 18 4. Considerations . . . . . . . . . . . . . . . . . . . . . . . 18
4.1. Synchronization . . . . . . . . . . . . . . . . . . . . . 18 4.1. Synchronization . . . . . . . . . . . . . . . . . . . . . 19
4.2. Data Correlation . . . . . . . . . . . . . . . . . . . . 19 4.2. Data Correlation . . . . . . . . . . . . . . . . . . . . 19
4.3. Packet Re-ordering . . . . . . . . . . . . . . . . . . . 20 4.3. Packet Reordering . . . . . . . . . . . . . . . . . . . . 20
5. Applications, implementation and deployment . . . . . . . . . 20 5. Applications, Implementation, and Deployment . . . . . . . . 21
5.1. Report on the operational experiment . . . . . . . . . . 21 5.1. Report on the Operational Experiment . . . . . . . . . . 22
5.1.1. Metric transparency . . . . . . . . . . . . . . . . . 23 5.1.1. Metric Transparency . . . . . . . . . . . . . . . . . 24
6. Hybrid measurement . . . . . . . . . . . . . . . . . . . . . 24 6. Hybrid Measurement . . . . . . . . . . . . . . . . . . . . . 24
7. Compliance with RFC6390 guidelines . . . . . . . . . . . . . 24 7. Compliance with Guidelines from RFC 6390 . . . . . . . . . . 25
8. Security Considerations . . . . . . . . . . . . . . . . . . . 26 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 27
9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 28 9. Security Considerations . . . . . . . . . . . . . . . . . . . 27
10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 28 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 28
11. References . . . . . . . . . . . . . . . . . . . . . . . . . 28 10.1. Normative References . . . . . . . . . . . . . . . . . . 28
11.1. Normative References . . . . . . . . . . . . . . . . . . 28 10.2. Informative References . . . . . . . . . . . . . . . . . 29
11.2. Informative References . . . . . . . . . . . . . . . . . 29 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . 32
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 31 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 32
1. Introduction 1. Introduction
Nowadays, most Service Providers' networks carry traffic with Nowadays, most Service Providers' networks carry traffic with
contents that are highly sensitive to packet loss [RFC7680], delay contents that are highly sensitive to packet loss [RFC7680], delay
[RFC7679], and jitter [RFC3393]. [RFC7679], and jitter [RFC3393].
In view of this scenario, Service Providers need methodologies and In view of this scenario, Service Providers need methodologies and
tools to monitor and measure network performances with an adequate tools to monitor and measure network performance with an adequate
accuracy, in order to constantly control the quality of experience accuracy, in order to constantly control the quality of experience
perceived by their customers. On the other hand, performance perceived by their customers. On the other hand, performance
monitoring provides useful information for improving network monitoring provides useful information for improving network
management (e.g. isolation of network problems, troubleshooting, management (e.g., isolation of network problems, troubleshooting,
etc.). etc.).
A lot of work related to OAM, that includes also performance A lot of work related to Operations, Administration, and Maintenance
monitoring techniques, has been done by Standards Developing (OAM), which also includes performance monitoring techniques, has
Organizations(SDOs): [RFC7276] provides a good overview of existing been done by Standards Developing Organizations (SDOs): [RFC7276]
OAM mechanisms defined in IETF, ITU-T and IEEE. Considering IETF, a provides a good overview of existing OAM mechanisms defined in the
lot of work has been done on fault detection and connectivity IETF, ITU-T, and IEEE. In the IETF, a lot of work has been done on
verification, while a minor effort has been dedicated so far to fault detection and connectivity verification, while a minor effort
performance monitoring. The IPPM WG has defined standard metrics to has been thus far dedicated to performance monitoring. The IPPM WG
measure network performance; however, the methods developed in this has defined standard metrics to measure network performance; however,
WG mainly refer to focus on active measurement techniques. More the methods developed in this WG mainly refer to focus on Active
recently, the MPLS WG has defined mechanisms for measuring packet measurement techniques. More recently, the MPLS WG has defined
loss, one-way and two-way delay, and delay variation in MPLS mechanisms for measuring packet loss, one-way and two-way delay, and
networks[RFC6374], but their applicability to passive measurements delay variation in MPLS networks [RFC6374], but their applicability
has some limitations, especially for pure connection-less networks. to Passive measurements has some limitations, especially for pure
connection-less networks.
The lack of adequate tools to measure packet loss with the desired The lack of adequate tools to measure packet loss with the desired
accuracy drove an effort to design a new method for the performance accuracy drove an effort to design a new method for the performance
monitoring of live traffic, easy to implement and deploy. The effort monitoring of live traffic, which is easy to implement and deploy.
led to the method described in this document: basically, it is a The effort led to the method described in this document: basically,
passive performance monitoring technique, potentially applicable to it is a Passive performance monitoring technique, potentially
any kind of packet based traffic, including Ethernet, IP, and MPLS, applicable to any kind of packet-based traffic, including Ethernet,
both unicast and multicast. The method addresses primarily packet IP, and MPLS, both unicast and multicast. The method addresses
loss measurement, but it can be easily extended to one-way delay and primarily packet loss measurement, but it can be easily extended to
delay variation measurements as well. one-way or two-way delay and delay variation measurements as well.
The method has been explicitly designed for passive measurements but The method has been explicitly designed for Passive measurements, but
it can also be used with active probes. Passive measurements are it can also be used with Active probes. Passive measurements are
usually more easily understood by customers and provide a much better usually more easily understood by customers and provide much better
accuracy, especially for packet loss measurements. accuracy, especially for packet loss measurements.
RFC 7799 [RFC7799] defines passive and hybrid methods of measurement. RFC 7799 [RFC7799] defines Passive and Hybrid Methods of Measurement.
In particular, Passive Methods of Measurement are based solely on In particular, Passive Methods of Measurement are based solely on
observations of an undisturbed and unmodified packet stream of observations of an undisturbed and unmodified packet stream of
interest; Hybrid Methods are Methods of Measurement that use a interest; Hybrid Methods are Methods of Measurement that use a
combination of Active Methods and Passive Methods. combination of Active Methods and Passive Methods.
Taking into consideration these definitions, Alternate Marking Method Taking into consideration these definitions, the Alternate-Marking
could be considered Hybrid or Passive depending on the case. In case Method could be considered Hybrid or Passive, depending on the case.
the marking method is obtained by changing existing field values of In the case where the marking method is obtained by changing existing
the packets (e.g. DSCP field), the technique is Hybrid. In case the field values of the packets (e.g., the Differentiated Services Code
marking field is dedicated, reserved and is included in the protocol Point (DSCP) field), the technique is Hybrid. In the case where the
specification Alternate Marking technique can be considered as marking field is dedicated, reserved, and included in the protocol
Passive (e.g. RFC6374 Synonymous Flow Label or OAM Marking Bits in specification, the Alternate-Marking technique can be considered as
BIER Header). Passive (e.g., Synonymous Flow Label as described in [SFL-FRAMEWORK]
or OAM Marking Bits as described in [PM-MM-BIER]).
The advantages of the method described in this document are: The advantages of the method described in this document are:
o easy implementation: it can be implemented or by using features o easy implementation: it can be implemented by using features
already available on major routing platforms as described in already available on major routing platforms, as described in
Section 5.1 or by applying an optimized implementation of the Section 5.1, or by applying an optimized implementation of the
method for both legacy and newest technologies; method for both legacy and newest technologies;
o low computational effort: the additional load on processing is o low computational effort: the additional load on processing is
negligible; negligible;
o accurate packet loss measurement: single packet loss granularity o accurate packet loss measurement: single packet loss granularity
is achieved with a passive measurement; is achieved with a Passive measurement;
o potential applicability to any kind of packet/frame -based o potential applicability to any kind of packet-based or frame-based
traffic: Ethernet, IP, MPLS, etc., both unicast and multicast; traffic: Ethernet, IP, MPLS, etc., and both unicast and multicast;
o robustness: the method can tolerate out of order packets and it's o robustness: the method can tolerate out-of-order packets, and it's
not based on "special" packets whose loss could have a negative not based on "special" packets whose loss could have a negative
impact; impact;
o flexibility: all the timestamp formats are allowed, because they o flexibility: all the timestamp formats are allowed, because they
are managed out-of-band. The format (the Network Time Protocol are managed out of band. The format (the Network Time Protocol
(NTP) RFC 5905 [RFC5905] or the IEEE 1588 Precision Time Protocol (NTP) [RFC5905] or the IEEE 1588 Precision Time Protocol (PTP)
(PTP) [IEEE-1588]) depends on the precision you want; [IEEE-1588]) depends on the precision you want; and
o no interoperability issues: the features required to experiment o no interoperability issues: the features required to experiment
and test the method (as described in Section 5.1) are available on and test the method (as described in Section 5.1) are available on
all current routing platforms. Both a centarlized or distributed all current routing platforms. Both a centralized or distributed
solution can be used to harvest data from the routers. solution can be used to harvest data from the routers.
The method doesn't raise any specific need for protocol extension, The method doesn't raise any specific need for protocol extension,
but it could be further improved by means of some extension to but it could be further improved by means of some extension to
existing protocols. Specifically, the use of DiffServ bits for existing protocols. Specifically, the use of Diffserv bits for
coloring the packets could not be a viable solution in some cases: a coloring the packets could not be a viable solution in some cases: a
standard method to color the packets for this specific application standard method to color the packets for this specific application
could be beneficial. could be beneficial.
2. Overview of the method 1.1. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP
14 [RFC2119] [RFC8174] when, and only when, they appear in all
capitals, as shown here.
2. Overview of the Method
In order to perform packet loss measurements on a production traffic In order to perform packet loss measurements on a production traffic
flow, different approaches exist. The most intuitive one consists in flow, different approaches exist. The most intuitive one consists in
numbering the packets, so that each router that receives the flow can numbering the packets so that each router that receives the flow can
immediately detect a packet missing. This approach, though very immediately detect a packet that is missing. This approach, though
simple in theory, is not simple to achieve: it requires the insertion very simple in theory, is not simple to achieve: it requires the
of a sequence number into each packet and the devices must be able to insertion of a sequence number into each packet, and the devices must
extract the number and check it in real time. Such a task can be be able to extract the number and check it in real time. Such a task
difficult to implement on live traffic: if UDP is used as the can be difficult to implement on live traffic: if UDP is used as the
transport protocol, the sequence number is not available; on the transport protocol, the sequence number is not available; on the
other hand, if a higher layer sequence number (e.g. in the RTP other hand, if a higher-layer sequence number (e.g., in the RTP
header) is used, extracting that information from each packet and header) is used, extracting that information from each packet and
process it in real time could overload the device. processing it in real time could overload the device.
An alternate approach is to count the number of packets sent on one An alternate approach is to count the number of packets sent on one
end, the number of packets received on the other end, and to compare end, count the number of packets received on the other end, and
the two values. This operation is much simpler to implement, but compare the two values. This operation is much simpler to implement,
requires that the devices performing the measurement are in sync: in but it requires the devices performing the measurement to be in sync:
order to compare two counters it is required that they refer exactly in order to compare two counters, it is required that they refer
to the same set of packets. Since a flow is continuous and cannot be exactly to the same set of packets. Since a flow is continuous and
stopped when a counter has to be read, it can be difficult to cannot be stopped when a counter has to be read, it can be difficult
determine exactly when to read the counter. A possible solution to to determine exactly when to read the counter. A possible solution
overcome this problem is to virtually split the flow in consecutive to overcome this problem is to virtually split the flow in
blocks by inserting periodically a delimiter so that each counter consecutive blocks by periodically inserting a delimiter so that each
refers exactly to the same block of packets. The delimiter could be counter refers exactly to the same block of packets. The delimiter
for example a special packet inserted artificially into the flow. could be, for example, a special packet inserted artificially into
However, delimiting the flow using specific packets has some the flow. However, delimiting the flow using specific packets has
limitations. First, it requires generating additional packets within some limitations. First, it requires generating additional packets
the flow and requires the equipment to be able to process those within the flow and requires the equipment to be able to process
packets. In addition, the method is vulnerable to out of order those packets. In addition, the method is vulnerable to out-of-order
reception of delimiting packets and, to a lesser extent, to their reception of delimiting packets and, to a lesser extent, to their
loss. loss.
The method proposed in this document follows the second approach, but The method proposed in this document follows the second approach, but
it doesn't use additional packets to virtually split the flow in it doesn't use additional packets to virtually split the flow in
blocks. Instead, it "marks" the packets so that the packets blocks. Instead, it "marks" the packets so that the packets
belonging to the same block will have the same color, whilst belonging to the same block will have the same color, whilst
consecutive blocks will have different colors. Each change of color consecutive blocks will have different colors. Each change of color
represents a sort of auto-synchronization signal that guarantees the represents a sort of auto-synchronization signal that guarantees the
consistency of measurements taken by different devices along the path consistency of measurements taken by different devices along the path
(see also [I-D.cociglio-mboned-multicast-pm] and (see also [IP-MULTICAST-PM] and [OPSAWG-P3M], where this technique
[I-D.tempia-opsawg-p3m], where this technique was introduced). was introduced).
Figure 1 represents a very simple network and shows how the method Figure 1 represents a very simple network and shows how the method
can be used to measure packet loss on different network segments: by can be used to measure packet loss on different network segments: by
enabling the measurement on several interfaces along the path, it is enabling the measurement on several interfaces along the path, it is
possible to perform link monitoring, node monitoring or end-to-end possible to perform link monitoring, node monitoring, or end-to-end
monitoring. The method is flexible enough to measure packet loss on monitoring. The method is flexible enough to measure packet loss on
any segment of the network and can be used to isolate the faulty any segment of the network and can be used to isolate the faulty
element. element.
Traffic flow Traffic Flow
========================================================> ========================================================>
+------+ +------+ +------+ +------+ +------+ +------+ +------+ +------+
---<> R1 <>-----<> R2 <>-----<> R3 <>-----<> R4 <>--- ---<> R1 <>-----<> R2 <>-----<> R3 <>-----<> R4 <>---
+------+ +------+ +------+ +------+ +------+ +------+ +------+ +------+
. . . . . . . . . . . .
. . . . . . . . . . . .
. <------> <-------> . . <------> <-------> .
. Node Packet Loss Link Packet Loss . . Node Packet Loss Link Packet Loss .
. . . .
<---------------------------------------------------> <--------------------------------------------------->
End-to-End Packet loss End-to-End Packet Loss
Figure 1: Available measurements Figure 1: Available Measurements
3. Detailed description of the method 3. Detailed Description of the Method
This section describes in detail how the method operate. A special This section describes, in detail, how the method operates. A
emphasis is given to the measurement of packet loss, that represents special emphasis is given to the measurement of packet loss, which
the core application of the method, but applicability to delay and represents the core application of the method, but applicability to
jitter measurements is also considered. delay and jitter measurements is also considered.
3.1. Packet loss measurement 3.1. Packet Loss Measurement
The basic idea is to virtually split traffic flows into consecutive The basic idea is to virtually split traffic flows into consecutive
blocks: each block represents a measurable entity unambiguously blocks: each block represents a measurable entity unambiguously
recognizable by all network devices along the path. By counting the recognizable by all network devices along the path. By counting the
number of packets in each block and comparing the values measured by number of packets in each block and comparing the values measured by
different network devices along the path, it is possible to measure different network devices along the path, it is possible to measure
packet loss occurred in any single block between any two points. packet loss occurred in any single block between any two points.
As discussed in the previous section, a simple way to create the As discussed in the previous section, a simple way to create the
blocks is to "color" the traffic (two colors are sufficient) so that blocks is to "color" the traffic (two colors are sufficient), so that
packets belonging to different consecutive blocks will have different packets belonging to different consecutive blocks will have different
colors. Whenever the color changes, the previous block terminates colors. Whenever the color changes, the previous block terminates
and the new one begins. Hence, all the packets belonging to the same and the new one begins. Hence, all the packets belonging to the same
block will have the same color and packets of different consecutive block will have the same color and packets of different consecutive
blocks will have different colors. The number of packets in each blocks will have different colors. The number of packets in each
block depends on the criterion used to create the blocks: block depends on the criterion used to create the blocks:
o if the color is switched after a fixed number of packets, then o if the color is switched after a fixed number of packets, then
each block will contain the same number of packets (except for any each block will contain the same number of packets (except for any
losses); losses); and
o if the color is switched according to a fixed timer, then the o if the color is switched according to a fixed timer, then the
number of packets may be different in each block depending on the number of packets may be different in each block depending on the
packet rate. packet rate.
The following figure shows how a flow looks like when it is split in The following figure shows how a flow looks like when it is split in
traffic blocks with colored packets. traffic blocks with colored packets.
A: packet with A coloring A: packet with A coloring
B: packet with B coloring B: packet with B coloring
| | | | | | | | | |
| | Traffic flow | | | | Traffic Flow | |
-------------------------------------------------------------------> ------------------------------------------------------------------->
BBBBBBB AAAAAAAAAAA BBBBBBBBBBB AAAAAAAAAAA BBBBBBBBBBB AAAAAAA BBBBBBB AAAAAAAAAAA BBBBBBBBBBB AAAAAAAAAAA BBBBBBBBBBB AAAAAAA
-------------------------------------------------------------------> ------------------------------------------------------------------->
... | Block 5 | Block 4 | Block 3 | Block 2 | Block 1 ... | Block 5 | Block 4 | Block 3 | Block 2 | Block 1
| | | | | | | | | |
Figure 2: Traffic coloring Figure 2: Traffic Coloring
Figure 3 shows how the method can be used to measure link packet loss Figure 3 shows how the method can be used to measure link packet loss
between two adjacent nodes. between two adjacent nodes.
Referring to the figure, let's assume we want to monitor the packet Referring to the figure, let's assume we want to monitor the packet
loss on the link between two routers: router R1 and router R2. loss on the link between two routers: router R1 and router R2.
According to the method, the traffic is colored alternatively with According to the method, the traffic is colored alternatively with
two different colors, A and B. Whenever the color changes, the two different colors: A and B. Whenever the color changes, the
transition generates a sort of square-wave signal, as depicted in the transition generates a sort of square-wave signal, as depicted in the
following figure. following figure.
Color A ----------+ +-----------+ +---------- Color A ----------+ +-----------+ +----------
| | | | | | | |
Color B +-----------+ +-----------+ Color B +-----------+ +-----------+
Block n ... Block 3 Block 2 Block 1 Block n ... Block 3 Block 2 Block 1
<---------> <---------> <---------> <---------> <---------> <---------> <---------> <---------> <---------> <--------->
Traffic flow Traffic Flow
===========================================================> ===========================================================>
Color ...AAAAAAAAAAA BBBBBBBBBBB AAAAAAAAAAA BBBBBBBBBBB AAAAAAA... Color ...AAAAAAAAAAA BBBBBBBBBBB AAAAAAAAAAA BBBBBBBBBBB AAAAAAA...
===========================================================> ===========================================================>
Figure 3: Computation of link packet loss Figure 3: Computation of Link Packet Loss
Traffic coloring could be done by R1 itself or it is already done Traffic coloring can be done by R1 itself if the traffic is not
before. R1 needs two counters, C(A)R1 and C(B)R1, on its egress already colored. R1 needs two counters, C(A)R1 and C(B)R1, on its
interface: C(A)R1 counts the packets with color A and C(B)R1 counts egress interface: C(A)R1 counts the packets with color A and C(B)R1
those with color B. As long as traffic is colored A, only counter counts those with color B. As long as traffic is colored as A, only
C(A)R1 will be incremented, while C(B)R1 is not incremented; vice counter C(A)R1 will be incremented, while C(B)R1 is not incremented;
versa, when the traffic is colored as B, only C(B)R1 is incremented. conversely, when the traffic is colored as B, only C(B)R1 is
C(A)R1 and C(B)R1 can be used as reference values to determine the incremented. C(A)R1 and C(B)R1 can be used as reference values to
packet loss from R1 to any other measurement point down the path. determine the packet loss from R1 to any other measurement point down
Router R2, similarly, will need two counters on its ingress the path. Router R2, similarly, will need two counters on its
interface, C(A)R2 and C(B)R2, to count the packets received on that ingress interface, C(A)R2 and C(B)R2, to count the packets received
interface and colored with color A and B respectively. When an A on that interface and colored with A and B, respectively. When an A
block ends, it is possible to compare C(A)R1 and C(A)R2 and calculate block ends, it is possible to compare C(A)R1 and C(A)R2 and calculate
the packet loss within the block; similarly, when the successive B the packet loss within the block; similarly, when the successive B
block terminates, it is possible to compare C(B)R1 with C(B)R2, and block terminates, it is possible to compare C(B)R1 with C(B)R2, and
so on for every successive block. so on, for every successive block.
Likewise, by using two counters on R2 egress interface it is possible Likewise, by using two counters on the R2 egress interface, it is
to count the packets sent out of R2 interface and use them as possible to count the packets sent out of the R2 interface and use
reference values to calculate the packet loss from R2 to any them as reference values to calculate the packet loss from R2 to any
measurement point down R2. measurement point down R2.
Using a fixed timer for color switching offers a better control over Using a fixed timer for color switching offers better control over
the method: the (time) length of the blocks can be chosen large the method: the (time) length of the blocks can be chosen large
enough to simplify the collection and the comparison of measures enough to simplify the collection and the comparison of measures
taken by different network devices. It's preferable to read the taken by different network devices. It's preferable to read the
value of the counters not immediately after the color switch: some value of the counters not immediately after the color switch: some
packets could arrive out of order and increment the counter packets could arrive out of order and increment the counter
associated to the previous block (color), so it is worth waiting for associated with the previous block (color), so it is worth waiting
some time. A safe choice is to wait L/2 time units (where L is the for some time. A safe choice is to wait L/2 time units (where L is
duration for each block) after the color switch, to read the still the duration for each block) after the color switch, to read the
counter of the previous color, so the possibility to read a running still counter of the previous color, so the possibility of reading a
counter instead of a still one is minimized. The drawback is that running counter instead of a still one is minimized. The drawback is
the longer the duration of the block, the less frequent the that the longer the duration of the block, the less frequent the
measurement can be taken. measurement can be taken.
The following table shows how the counters can be used to calculate The following table shows how the counters can be used to calculate
the packet loss between R1 and R2. The first column lists the the packet loss between R1 and R2. The first column lists the
sequence of traffic blocks while the other columns contain the sequence of traffic blocks, while the other columns contain the
counters of A-colored packets and B-colored packets for R1 and R2. counters of A-colored packets and B-colored packets for R1 and R2.
In this example, we assume that the values of the counters are reset In this example, we assume that the values of the counters are reset
to zero whenever a block ends and its associated counter has been to zero whenever a block ends and its associated counter has been
read: with this assumption, the table shows only relative values, read: with this assumption, the table shows only relative values,
that is the exact number of packets of each color within each block. which is the exact number of packets of each color within each block.
If the values of the counters were not reset, the table would contain If the values of the counters were not reset, the table would contain
cumulative values, but the relative values could be determined simply cumulative values, but the relative values could be determined simply
by difference from the value of the previous block of the same color. by the difference from the value of the previous block of the same
color.
The color is switched on the basis of a fixed timer (not shown in the The color is switched on the basis of a fixed timer (not shown in the
table), so the number of packets in each block is different. table), so the number of packets in each block is different.
+-------+--------+--------+--------+--------+------+ +-------+--------+--------+--------+--------+------+
| Block | C(A)R1 | C(B)R1 | C(A)R2 | C(B)R2 | Loss | | Block | C(A)R1 | C(B)R1 | C(A)R2 | C(B)R2 | Loss |
+-------+--------+--------+--------+--------+------+ +-------+--------+--------+--------+--------+------+
| 1 | 375 | 0 | 375 | 0 | 0 | | 1 | 375 | 0 | 375 | 0 | 0 |
| | | | | | |
| 2 | 0 | 388 | 0 | 388 | 0 | | 2 | 0 | 388 | 0 | 388 | 0 |
| | | | | | |
| 3 | 382 | 0 | 381 | 0 | 1 | | 3 | 382 | 0 | 381 | 0 | 1 |
| | | | | | |
| 4 | 0 | 377 | 0 | 374 | 3 | | 4 | 0 | 377 | 0 | 374 | 3 |
| | | | | | |
| ... | ... | ... | ... | ... | ... | | ... | ... | ... | ... | ... | ... |
| | | | | | |
| 2n | 0 | 387 | 0 | 387 | 0 | | 2n | 0 | 387 | 0 | 387 | 0 |
| | | | | | |
| 2n+1 | 379 | 0 | 377 | 0 | 2 | | 2n+1 | 379 | 0 | 377 | 0 | 2 |
+-------+--------+--------+--------+--------+------+ +-------+--------+--------+--------+--------+------+
Table 1: Evaluation of counters for packet loss measurements Table 1: Evaluation of Counters for Packet Loss Measurements
During an A block (blocks 1, 3 and 2n+1), all the packets are During an A block (blocks 1, 3, and 2n+1), all the packets are
A-colored, therefore the C(A) counters are incremented to the number A-colored; therefore, the C(A) counters are incremented to the number
seen on the interface, while C(B) counters are zero. Vice versa, seen on the interface, while C(B) counters are zero. Conversely,
during a B block (blocks 2, 4 and 2n), all the packets are B-colored: during a B block (blocks 2, 4, and 2n), all the packets are
C(A) counters are zero, while C(B) counters are incremented. B-colored: C(A) counters are zero, while C(B) counters are
incremented.
When a block ends (because of color switching) the relative counters When a block ends (because of color switching), the relative counters
stop incrementing and it is possible to read them, compare the values stop incrementing; it is possible to read them, compare the values
measured on router R1 and R2 and calculate the packet loss within measured on routers R1 and R2, and calculate the packet loss within
that block. that block.
For example, looking at the table above, during the first block For example, looking at the table above, during the first block
(A-colored), C(A)R1 and C(A)R2 have the same value (375), which (A-colored), C(A)R1 and C(A)R2 have the same value (375), which
corresponds to the exact number of packets of the first block (no corresponds to the exact number of packets of the first block (no
loss). Also during the second block (B-colored) R1 and R2 counters loss). Also, during the second block (B-colored), R1 and R2 counters
have the same value (388), which corresponds to the number of packets have the same value (388), which corresponds to the number of packets
of the second block (no loss). During blocks three and four, R1 and of the second block (no loss). During the third and fourth blocks,
R2 counters are different, meaning that some packets have been lost: R1 and R2 counters are different, meaning that some packets have been
in the example, one single packet (382-381) was lost during block lost: in the example, one single packet (382-381) was lost during
three and three packets (377-374) were lost during block four. block three, and three packets (377-374) were lost during block four.
The method applied to R1 and R2 can be extended to any other router The method applied to R1 and R2 can be extended to any other router
and applied to more complex networks, as far as the measurement is and applied to more complex networks, as far as the measurement is
enabled on the path followed by the traffic flow(s) being observed. enabled on the path followed by the traffic flow(s) being observed.
It's worth mentioning two different strategies that can be used when It's worth mentioning two different strategies that can be used when
implementing the method: implementing the method:
o flow-based: the flow-based strategy is used when only a limited o flow-based: the flow-based strategy is used when only a limited
number of traffic flows need to be monitored. According to this number of traffic flows need to be monitored. According to this
strategy, only a subset of the flows is colored. Counters for strategy, only a subset of the flows is colored. Counters for
packet loss measurements can be instantiated for each single flow, packet loss measurements can be instantiated for each single flow,
or for the set as a whole, depending on the desired granularity. or for the set as a whole, depending on the desired granularity.
A relevant problem with this approach is the necessity to know in A relevant problem with this approach is the necessity to know in
advance the path followed by flows that are subject to advance the path followed by flows that are subject to
measurement. Path rerouting and traffic load-balancing increase measurement. Path rerouting and traffic load-balancing increase
the issue complexity, especially for unicast traffic. The problem the issue complexity, especially for unicast traffic. The problem
is easier to solve for multicast traffic where load balancing is is easier to solve for multicast traffic, where load-balancing is
seldom used and static joins are frequently used to force traffic seldom used and static joins are frequently used to force traffic
forwarding and replication. forwarding and replication.
o link-based: measurements are performed on all the traffic on a o link-based: measurements are performed on all the traffic on a
link by link basis. The link could be a physical link or a link-by-link basis. The link could be a physical link or a
logical link. Counters could be instantiated for the traffic as a logical link. Counters could be instantiated for the traffic as a
whole or for each traffic class (in case it is desired to monitor whole or for each traffic class (in case it is desired to monitor
each class separately), but in the second case a couple of each class separately), but in the second case, a couple of
counters is needed for each class. counters are needed for each class.
As mentioned, the flow-based measurement requires the identification As mentioned, the flow-based measurement requires the identification
of the flow to be monitored and the discovery of the path followed by of the flow to be monitored and the discovery of the path followed by
the selected flow. It is possible to monitor a single flow or the selected flow. It is possible to monitor a single flow or
multiple flows grouped together, but in this case measurement is multiple flows grouped together, but in this case, measurement is
consistent only if all the flows in the group follow the same path. consistent only if all the flows in the group follow the same path.
Moreover if a measurement is performed by grouping many flows, it is Moreover, if a measurement is performed by grouping many flows, it is
not possible to determine exactly which flow was affected by packets not possible to determine exactly which flow was affected by packet
loss. In order to have measures per single flow it is necessary to loss. In order to have measures per single flow, it is necessary to
configure counters for each specific flow. Once the flow(s) to be configure counters for each specific flow. Once the flow(s) to be
monitored have been identified, it is necessary to configure the monitored has been identified, it is necessary to configure the
monitoring on the proper nodes. Configuring the monitoring means monitoring on the proper nodes. Configuring the monitoring means
configuring the rule to intercept the traffic and configuring the configuring the rule to intercept the traffic and configuring the
counters to count the packets. To have just an end-to-end counters to count the packets. To have just an end-to-end
monitoring, it is sufficient to enable the monitoring on the first monitoring, it is sufficient to enable the monitoring on the first-
and the last hop routers of the path: the mechanism is completely and last-hop routers of the path: the mechanism is completely
transparent to intermediate nodes and independent from the path transparent to intermediate nodes and independent from the path
followed by traffic flows. On the contrary, to monitor the flow on a followed by traffic flows. On the contrary, to monitor the flow on a
hop-by-hop basis along its whole path it is necessary to enable the hop-by-hop basis along its whole path, it is necessary to enable the
monitoring on every node from the source to the destination. In case monitoring on every node from the source to the destination. In case
the exact path followed by the flow is not known a priori (i.e. the the exact path followed by the flow is not known a priori (i.e., the
flow has multiple paths to reach the destination) it is necessary to flow has multiple paths to reach the destination), it is necessary to
enable the monitoring system on every path: counters on interfaces enable the monitoring system on every path: counters on interfaces
traversed by the flow will report packet count, counters on other traversed by the flow will report packet count, whereas counters on
interfaces will be null. other interfaces will be null.
3.1.1. Coloring the packets 3.1.1. Coloring the Packets
The coloring operation is fundamental in order to create packet The coloring operation is fundamental in order to create packet
blocks. This implies choosing where to activate the coloring and how blocks. This implies choosing where to activate the coloring and how
to color the packets. to color the packets.
In case of flow-based measurements, the flow to monitor can be In case of flow-based measurements, the flow to monitor can be
defined by a set of selection rules (e.g. headers fields) used to defined by a set of selection rules (e.g., header fields) used to
match a subset of the packets; in this way it is possible to control match a subset of the packets; in this way, it is possible to control
the number of involved nodes, the path followed by the packets and the number of involved nodes, the path followed by the packets, and
the size of the flows. it is possible, in general, to have multiple the size of the flows. It is possible, in general, to have multiple
coloring nodes or a single coloring node that is easier to manage and coloring nodes or a single coloring node that is easier to manage and
doesn't rise any risk of conflict. Coloring in multiple nodes can be doesn't raise any risk of conflict. Coloring in multiple nodes can
done and the requirement is that the coloring must change be done, and the requirement is that the coloring must change
periodically between the nodes according to the timing considerations periodically between the nodes according to the timing considerations
in Section 3.2; so every node, that is designated as a measurement in Section 3.2; so every node that is designated as a measurement
point along the path, should be able to identify unambiguously the point along the path should be able to identify unambiguously the
colored packets. Furthermore [I-D.fioccola-ippm-multipoint-alt-mark] colored packets. Furthermore, [MULTIPOINT-ALT-MM] generalizes the
generalizes the coloring for multipoint to multipoint flow. In coloring for multipoint-to-multipoint flow. In addition, it can be
addition, it can be advantageous to color the flow as close as advantageous to color the flow as close as possible to the source
possible to the source because it allows an end-to-end measure if a because it allows an end-to-end measure if a measurement point is
measurement point is enabled on the last-hop router as well. enabled on the last-hop router as well.
For link-based measurements, all traffic needs to be colored when For link-based measurements, all traffic needs to be colored when
transmitted on the link. If the traffic had already been colored, transmitted on the link. If the traffic had already been colored,
then it has to be re-colored because the color must be consistent on then it has to be re-colored because the color must be consistent on
the link. This means that each hop along the path must (re-)color the link. This means that each hop along the path must (re-)color
the traffic; the color is not required to be consistent along the traffic; the color is not required to be consistent along
different links. different links.
Traffic coloring can be implemented by setting a specific bit in the Traffic coloring can be implemented by setting a specific bit in the
packet header and changing the value of that bit periodically. How packet header and changing the value of that bit periodically. How
to choose the marking field depends on the application and is out of to choose the marking field depends on the application and is out of
scope here. However some applications are reported in Section 5. scope here. However, some applications are reported in Section 5.
3.1.2. Counting the packets 3.1.2. Counting the Packets
For flow-based measurements, assuming that the coloring of the For flow-based measurements, assuming that the coloring of the
packets is performed only by the source nodes, the nodes between packets is performed only by the source nodes, the nodes between
source and destination (included) have to count the colored packets source and destination (included) have to count the colored packets
that they receive and forward: this operation can be enabled on every that they receive and forward: this operation can be enabled on every
router along the path or only on a subset, depending on which network router along the path or only on a subset, depending on which network
segment is being monitored (a single link, a particular metro area, segment is being monitored (a single link, a particular metro area,
the backbone, the whole path). Since the color switches periodically the backbone, or the whole path). Since the color switches
between two values, two counters (one for each value) are needed: one periodically between two values, two counters (one for each value)
counter for packets with color A and one counter for packets with are needed: one counter for packets with color A and one counter for
color B. For each flow (or group of flows) being monitored and for packets with color B. For each flow (or group of flows) being
every interface where the monitoring is active, a couple of counters monitored and for every interface where the monitoring is Active, a
is needed. For example, in order to monitor separately 3 flows on a couple of counters are needed. For example, in order to separately
router with 4 interfaces involved, 24 counters are needed (2 counters monitor three flows on a router with four interfaces involved, 24
for each of the 3 flows on each of the 4 interfaces). Furthermore counters are needed (two counters for each of the three flows on each
[I-D.fioccola-ippm-multipoint-alt-mark] generalizes the counting for of the four interfaces). Furthermore, [MULTIPOINT-ALT-MM]
multipoint to multipoint flow. generalizes the counting for multipoint-to-multipoint flow.
In case of link-based measurements the behaviour is similar except In case of link-based measurements, the behavior is similar except
that coloring and counting operations are performed on a link by link that coloring and counting operations are performed on a link-by-link
basis at each endpoint of the link. basis at each endpoint of the link.
Another important aspect to take into consideration is when to read Another important aspect to take into consideration is when to read
the counters: in order to count the exact number of packets of a the counters: in order to count the exact number of packets of a
block the routers must perform this operation when that block has block, the routers must perform this operation when that block has
ended: in other words, the counter for color A must be read when the ended; in other words, the counter for color A must be read when the
current block has color B, in order to be sure that the value of the current block has color B, in order to be sure that the value of the
counter is stable. This task can be accomplished in two ways. The counter is stable. This task can be accomplished in two ways. The
general approach suggests to read the counters periodically, many general approach suggests reading the counters periodically, many
times during a block duration, and to compare these successive times during a block duration, and comparing these successive
readings: when the counter stops incrementing means that the current readings: when the counter stops incrementing, it means that the
block has ended and its value can be elaborated safely. current block has ended, and its value can be elaborated safely.
Alternatively, if the coloring operation is performed on the basis of Alternatively, if the coloring operation is performed on the basis of
a fixed timer, it is possible to configure the reading of the a fixed timer, it is possible to configure the reading of the
counters according to that timer: for example, reading the counter counters according to that timer: for example, reading the counter
for color A every period in the middle of the subsequent block with for color A every period in the middle of the subsequent block with
color B is a safe choice. A sufficient margin should be considered color B is a safe choice. A sufficient margin should be considered
between the end of a block and the reading of the counter, in order between the end of a block and the reading of the counter, in order
to take into account any out-of-order packets. to take into account any out-of-order packets.
3.1.3. Collecting data and calculating packet loss 3.1.3. Collecting Data and Calculating Packet Loss
The nodes enabled to perform performance monitoring collect the value The nodes enabled to perform performance monitoring collect the value
of the counters, but they are not able to directly use this of the counters, but they are not able to directly use this
information to measure packet loss, because they only have their own information to measure packet loss, because they only have their own
samples. For this reason, an external Network Management System samples. For this reason, an external Network Management System
(NMS) can be used to collect and elaborate data and to perform packet (NMS) can be used to collect and elaborate data and to perform packet
loss calculation. The NMS compares the values of counters from loss calculation. The NMS compares the values of counters from
different nodes and can calculate if some packets were lost (even a different nodes and can calculate if some packets were lost (even a
single packet) and also where packets were lost. single packet) and where those packets were lost.
The value of the counters needs to be transmitted to the NMS as soon The value of the counters needs to be transmitted to the NMS as soon
as it has been read. This can be accomplished by using SNMP or FTP as it has been read. This can be accomplished by using SNMP or FTP
and can be done in Push Mode or Polling Mode. In the first case, and can be done in Push Mode or Polling Mode. In the first case,
each router periodically sends the information to the NMS, in the each router periodically sends the information to the NMS; in the
latter case it is the NMS that periodically polls routers to collect latter case, it is the NMS that periodically polls routers to collect
information. In any case, the NMS has to collect all the relevant information. In any case, the NMS has to collect all the relevant
values from all the routers within one cycle of the timer. values from all the routers within one cycle of the timer.
it would be also possible to use a protocol to exchange values of It would also be possible to use a protocol to exchange values of
counters between the two endpoints in order to let them perform the counters between the two endpoints in order to let them perform the
packet loss calculation for each traffic direction. packet loss calculation for each traffic direction.
A possible approach for the performance measurement architecture is A possible approach for the performance measurement (PM) architecture
explained in [I-D.chen-ippm-coloring-based-ipfpm-framework], while is explained in [COLORING], while [IP-FLOW-REPORT] introduces new
[I-D.chen-ippm-ipfpm-report] introduces new information elements of information elements of IP Flow Information Export (IPFIX) [RFC7011].
IPFIX (RFC 7011 [RFC7011]).
3.2. Timing aspects 3.2. Timing Aspects
This document introduces two color switching method: one is based on This document introduces two color-switching methods: one is based on
fixed number of packet, the other is based on fixed timer. But the a fixed number of packets, and the other is based on a fixed timer.
method based on fixed timer is preferable because is more But the method based on a fixed timer is preferable because it is
deterministic, and will be considered in the rest of the dcoument. more deterministic, and it will be considered in the rest of the
document.
By considering the clock error between network devices R1 and R2, In general, clocks in network devices are not accurate and for this
they must be synchronized to the same clock reference with an reason, there is a clock error between the measurement points R1 and
accuracy of +/- L/2 time units, where L is the time duration of the R2. But, to implement the methodology, they must be synchronized to
block. So each colored packet can be assigned to the right batch by the same clock reference with an accuracy of +/- L/2 time units,
each router. This is because the minimum time distance between two where L is the fixed time duration of the block. So each colored
packets of the same color but belonging to different batches is L packet can be assigned to the right batch by each router. This is
time units. because the minimum time distance between two packets of the same
color but that belong to different batches is L time units.
In practice, there are also out of order at batch boundaries, In practice, in addition to clock errors, the delay between
strictly related to the delay between measurement points. This means measurement points also affects the implementation of the methodology
that, without considering clock error, we wait L/2 after color because each packet can be delayed differently, and this can produce
switching to be sure to take a still counter. out of order at batch boundaries. This means that, without
considering clock error, we wait L/2 after color switching to be sure
to take a still counter.
In summary we need to take into account two contributions: clock In summary, we need to take into account two contributions: clock
error between network devices and the interval we need to wait to error between network devices and the interval we need to wait to
avoid out of order because of network delay. avoid packets being out of order because of network delay.
The following figure explains both issues. The following figure explains both issues.
...BBBBBBBBB | AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA | BBBBBBBBB... ...BBBBBBBBB | AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA | BBBBBBBBB...
|<======================================>| |<======================================>|
| L | | L |
...=========>|<==================><==================>|<==========... ...=========>|<==================><==================>|<==========...
| L/2 L/2 | | L/2 L/2 |
|<===>| |<===>| |<===>| |<===>|
d | | d d | | d
|<==========================>| |<==========================>|
available counting interval available counting interval
Figure 4: Timing aspects Figure 4: Timing Aspects
It is assumed that all network devices are synchronized to a common It is assumed that all network devices are synchronized to a common
reference time with an accuracy of +/- A/2. Thus, the difference reference time with an accuracy of +/- A/2. Thus, the difference
between the clock values of any two network devices is bounded by A. between the clock values of any two network devices is bounded by A.
The guardband d is given by: The guard band d is given by:
d = A + D_max - D_min, d = A + D_max - D_min,
where A is the clock accuracy, D_max is an upper bound on the network where A is the clock accuracy, D_max is an upper bound on the network
delay between the network devices, and D_min is a lower bound on the delay between the network devices, and D_min is a lower bound on the
delay. delay.
The available counting interval is L - 2d that must be > 0. The available counting interval is L - 2d that must be > 0.
The condition that must be satisfied and is a requirement on the The condition that must be satisfied and is a requirement on the
synchronization accuracy is: synchronization accuracy is:
d < L/2. d < L/2.
3.3. One-way delay measurement 3.3. One-Way Delay Measurement
The same principle used to measure packet loss can be applied also to The same principle used to measure packet loss can be applied also to
one-way delay measurement. There are three alternatives, as one-way delay measurement. There are three alternatives, as
described hereinafter. described hereinafter.
3.3.1. Single marking methodology Note that, for all the one-way delay alternatives described in the
next sections, by summing the one-way delays of the two directions of
a path, it is always possible to measure the two-way delay (round-
trip "virtual" delay).
3.3.1. Single-Marking Methodology
The alternation of colors can be used as a time reference to The alternation of colors can be used as a time reference to
calculate the delay. Whenever the color changes (that means that a calculate the delay. Whenever the color changes (which means that a
new block has started) a network device can store the timestamp of new block has started), a network device can store the timestamp of
the first packet of the new block; that timestamp can be compared the first packet of the new block; that timestamp can be compared
with the timestamp of the same packet on a second router to compute with the timestamp of the same packet on a second router to compute
packet delay. Considering Figure 2, R1 stores a timestamp TS(A1)R1 packet delay. When looking at Figure 2, R1 stores the timestamp
when it sends the first packet of block 1 (A-colored), a timestamp TS(A1)R1 when it sends the first packet of block 1 (A-colored), the
TS(B2)R1 when it sends the first packet of block 2 (B-colored) and so timestamp TS(B2)R1 when it sends the first packet of block 2
on for every other block. R2 performs the same operation on the (B-colored), and so on for every other block. R2 performs the same
receiving side, recording TS(A1)R2, TS(B2)R2 and so on. Since the operation on the receiving side, recording TS(A1)R2, TS(B2)R2, and so
timestamps refer to specific packets (the first packet of each block) on. Since the timestamps refer to specific packets (the first packet
we are sure that timestamps compared to compute delay refer to the of each block), we are sure that timestamps compared to compute delay
same packets. By comparing TS(A1)R1 with TS(A1)R2 (and similarly refer to the same packets. By comparing TS(A1)R1 with TS(A1)R2 (and
TS(B2)R1 with TS(B2)R2 and so on) it is possible to measure the delay similarly TS(B2)R1 with TS(B2)R2, and so on), it is possible to
between R1 and R2. In order to have more measurements, it is measure the delay between R1 and R2. In order to have more
possible to take and store more timestamps, referring to other measurements, it is possible to take and store more timestamps,
packets within each block. referring to other packets within each block.
In order to coherently compare timestamps collected on different In order to coherently compare timestamps collected on different
routers, the clocks on the network nodes must be in sync. routers, the clocks on the network nodes must be in sync.
Furthermore, a measurement is valid only if no packet loss occurs and Furthermore, a measurement is valid only if no packet loss occurs and
if packet misordering can be avoided, otherwise the first packet of a if packet misordering can be avoided; otherwise, the first packet of
block on R1 could be different from the first packet of the same a block on R1 could be different from the first packet of the same
block on R2 (f.i. if that packet is lost between R1 and R2 or it block on R2 (for instance, if that packet is lost between R1 and R2
arrives after the next one). or it arrives after the next one).
The following table shows how timestamps can be used to calculate the The following table shows how timestamps can be used to calculate the
delay between R1 and R2. The first column lists the sequence of delay between R1 and R2. The first column lists the sequence of
blocks while other columns contain the timestamp referring to the blocks, while other columns contain the timestamp referring to the
first packet of each block on R1 and R2. The delay is computed as a first packet of each block on R1 and R2. The delay is computed as a
difference between timestamps. For the sake of simplicity, all the difference between timestamps. For the sake of simplicity, all the
values are expressed in milliseconds. values are expressed in milliseconds.
+-------+---------+---------+---------+---------+-------------+ +-------+---------+---------+---------+---------+-------------+
| Block | TS(A)R1 | TS(B)R1 | TS(A)R2 | TS(B)R2 | Delay R1-R2 | | Block | TS(A)R1 | TS(B)R1 | TS(A)R2 | TS(B)R2 | Delay R1-R2 |
+-------+---------+---------+---------+---------+-------------+ +-------+---------+---------+---------+---------+-------------+
| 1 | 12.483 | - | 15.591 | - | 3.108 | | 1 | 12.483 | - | 15.591 | - | 3.108 |
| | | | | | |
| 2 | - | 6.263 | - | 9.288 | 3.025 | | 2 | - | 6.263 | - | 9.288 | 3.025 |
| | | | | | |
| 3 | 27.556 | - | 30.512 | - | 2.956 | | 3 | 27.556 | - | 30.512 | - | 2.956 |
| | | | | | |
| | - | 18.113 | - | 21.269 | 3.156 | | | - | 18.113 | - | 21.269 | 3.156 |
| | | | | | |
| ... | ... | ... | ... | ... | ... | | ... | ... | ... | ... | ... | ... |
| | | | | | |
| 2n | 77.463 | - | 80.501 | - | 3.038 | | 2n | 77.463 | - | 80.501 | - | 3.038 |
| | | | | | |
| 2n+1 | - | 24.333 | - | 27.433 | 3.100 | | 2n+1 | - | 24.333 | - | 27.433 | 3.100 |
+-------+---------+---------+---------+---------+-------------+ +-------+---------+---------+---------+---------+-------------+
Table 2: Evaluation of timestamps for delay measurements Table 2: Evaluation of Timestamps for Delay Measurements
The first row shows timestamps taken on R1 and R2 respectively and The first row shows timestamps taken on R1 and R2, respectively, and
referring to the first packet of block 1 (which is A-colored). Delay refers to the first packet of block 1 (which is A-colored). Delay
can be computed as a difference between the timestamp on R2 and the can be computed as a difference between the timestamp on R2 and the
timestamp on R1. Similarly, the second row shows timestamps (in timestamp on R1. Similarly, the second row shows timestamps (in
milliseconds) taken on R1 and R2 and referring to the first packet of milliseconds) taken on R1 and R2 and refers to the first packet of
block 2 (which is B-colored). Comparing timestamps taken on block 2 (which is B-colored). By comparing timestamps taken on
different nodes in the network and referring to the same packets different nodes in the network and referring to the same packets
(identified using the alternation of colors) it is possible to (identified using the alternation of colors), it is possible to
measure delay on different network segments. measure delay on different network segments.
For the sake of simplicity, in the above example a single measurement For the sake of simplicity, in the above example, a single
is provided within a block, taking into account only the first packet measurement is provided within a block, taking into account only the
of each block. The number of measurements can be easily increased by first packet of each block. The number of measurements can be easily
considering multiple packets in the block: for instance, a timestamp increased by considering multiple packets in the block: for instance,
could be taken every N packets, thus generating multiple delay a timestamp could be taken every N packets, thus generating multiple
measurements. Taking this to the limit, in principle the delay could delay measurements. Taking this to the limit, in principle, the
be measured for each packet, by taking and comparing the delay could be measured for each packet by taking and comparing the
corresponding timestamps (possible but impractical from an corresponding timestamps (possible but impractical from an
implementation point of view). implementation point of view).
3.3.1.1. Mean delay 3.3.1.1. Mean Delay
As mentioned before, the method previously exposed for measuring the As mentioned before, the method previously exposed for measuring the
delay is sensitive to out of order reception of packets. In order to delay is sensitive to out-of-order reception of packets. In order to
overcome this problem, a different approach has been considered: it overcome this problem, a different approach has been considered: it
is based on the concept of mean delay. The mean delay is calculated is based on the concept of mean delay. The mean delay is calculated
by considering the average arrival time of the packets within a by considering the average arrival time of the packets within a
single block. The network device locally stores a timestamp for each single block. The network device locally stores a timestamp for each
packet received within a single block: summing all the timestamps and packet received within a single block: summing all the timestamps and
dividing by the total number of packets received, the average arrival dividing by the total number of packets received, the average arrival
time for that block of packets can be calculated. By subtracting the time for that block of packets can be calculated. By subtracting the
average arrival times of two adjacent devices it is possible to average arrival times of two adjacent devices, it is possible to
calculate the mean delay between those nodes. When computing the calculate the mean delay between those nodes. When computing the
mean delay, measurement error could be augmented by accumulating mean delay, the measurement error could be augmented by accumulating
measurement error of a lot of packets. This method is robust to out the measurement error of a lot of packets. This method is robust to
of order packets and also to packet loss (only a small error is out-of-order packets and also to packet loss (only a small error is
introduced). Moreover, it greatly reduces the number of timestamps introduced). Moreover, it greatly reduces the number of timestamps
(only one per block for each network device) that have to be (only one per block for each network device) that have to be
collected by the management system. On the other hand, it only gives collected by the management system. On the other hand, it only gives
one measure for the duration of the block (f.i. 5 minutes), and it one measure for the duration of the block (for instance, 5 minutes),
doesn't give the minimum, maximum and median delay values (RFC 6703 and it doesn't give the minimum, maximum, and median delay values
[RFC6703]). This limitation could be overcome by reducing the [RFC6703]. This limitation could be overcome by reducing the
duration of the block (f.i. from 5 minutes to a few seconds), that duration of the block (for instance, from 5 minutes to a few
implicates an highly optimized implementation of the method. seconds), which implicates a highly optimized implementation of the
method.
By summing the mean delays of the two directions of a path, it is
also possible to measure the two-way mean delay (round-trip delay).
3.3.2. Double marking methodology 3.3.2. Double-Marking Methodology
The Single marking methodology for one-way delay measurement is The Single-Marking methodology for one-way delay measurement is
sensitive to out of order reception of packets. The first approach sensitive to out-of-order reception of packets. The first approach
to overcome this problem is described before and is based on the to overcome this problem has been described before and is based on
concept of mean delay. But the limitation of mean delay is that it the concept of mean delay. But the limitation of mean delay is that
doesn't give information about the delay values distribution for the it doesn't give information about the delay value's distribution for
duration of the block. Additionally it may be useful to have not the duration of the block. Additionally, it may be useful to have
only the mean delay but also the minimum, maximum and median delay not only the mean delay but also the minimum, maximum, and median
values and, in wider terms, to know more about the statistic delay values and, in wider terms, to know more about the statistic
distribution of delay values. So in order to have more information distribution of delay values. So, in order to have more information
about the delay and to overcome out of order issues, a different about the delay and to overcome out-of-order issues, a different
approach can be introduced: it is based on double marking approach can be introduced; it is based on a Double-Marking
methodology. methodology.
Basically, the idea is to use the first marking to create the Basically, the idea is to use the first marking to create the
alternate flow and, within this colored flow, a second marking to alternate flow and, within this colored flow, a second marking to
select the packets for measuring delay/jitter. The first marking is select the packets for measuring delay/jitter. The first marking is
needed for packet loss and mean delay measurement. The second needed for packet loss and mean delay measurement. The second
marking creates a new set of marked packets that are fully identified marking creates a new set of marked packets that are fully identified
over the network, so that a network device can store the timestamps over the network, so that a network device can store the timestamps
of these packets; these timestamps can be compared with the of these packets; these timestamps can be compared with the
timestamps of the same packets on a second router to compute packet timestamps of the same packets on a second router to compute packet
delay values for each packet. The number of measurements can be delay values for each packet. The number of measurements can be
easily increased by changing the frequency of the second marking. easily increased by changing the frequency of the second marking.
But the frequency of the second marking must be not too high in order But the frequency of the second marking must not be too high in order
to avoid out of order issues. Between packets with the second to avoid out-of-order issues. Between packets with the second
marking there should be a security time gap (e.g. this gap could be, marking, there should be a security time gap (e.g., this gap could
at the minimum, the mean network delay calculated with the previous be, at the minimum, the mean network delay calculated with the
methodology) to avoid out of order issues and also to have a number previous methodology) to avoid out-of-order issues and also to have a
of measurement packets that is rate independent. If a second marking number of measurement packets that are rate independent. If a
packet is lost, the delay measurement for the considered block is second-marking packet is lost, the delay measurement for the
corrupted and should be discarded. considered block is corrupted and should be discarded.
Mean delay is calculated on all the packets of a sample and is a Mean delay is calculated on all the packets of a sample and is a
simple computation to be performed for single marking method. In simple computation to be performed for a Single-Marking Method. In
some cases the mean delay measure is not sufficient to characterize some cases, the mean delay measure is not sufficient to characterize
the sample, and more statistics of delay extent data are needed, e.g. the sample, and more statistics of delay extent data are needed,
percentiles, variance and median delay values. The conventional e.g., percentiles, variance, and median delay values. The
range (maximum-minimum) should be avoided for several reasons, conventional range (maximum-minimum) should be avoided for several
including stability of the maximum delay due to the influence by reasons, including stability of the maximum delay due to the
outliers. RFC 5481 [RFC5481] Section 6.5 highlights how the 99.9th influence by outliers. RFC 5481 [RFC5481], Section 6.5 highlights
percentile of delay and delay variation is more helpful to how the 99.9th percentile of delay and delay variation is more
performance planners. To overcome this drawback the idea is to helpful to performance planners. To overcome this drawback, the idea
couple the mean delay measure for the entire batch with double is to couple the mean delay measure for the entire batch with a
marking method, where a subset of batch packets are selected for Double-Marking Method, where a subset of batch packets is selected
extensive delay calculation by using a second marking. In this way for extensive delay calculation by using a second marking. In this
it is possible to perform a detailed analysis on these double marked way, it is possible to perform a detailed analysis on these double-
packets. Please note that there are classic algorithms for median marked packets. Please note that there are classic algorithms for
and variance calculation, but are out of the scope of this document. median and variance calculation, but they are out of the scope of
The comparison between the mean delay for the entire batch and the this document. The comparison between the mean delay for the entire
mean delay on these double marked packets gives an useful information batch and the mean delay on these double-marked packets gives useful
since it is possible to understand if the double marking measurements information since it is possible to understand if the Double-Marking
are actually representative of the delay trends. measurements are actually representative of the delay trends.
3.4. Delay variation measurement 3.4. Delay Variation Measurement
Similarly to one-way delay measurement (both for single marking and Similar to one-way delay measurement (both for Single Marking and
double marking), the method can also be used to measure the inter- Double Marking), the method can also be used to measure the inter-
arrival jitter. We refer to the definition in RFC 3393 [RFC3393]. arrival jitter. We refer to the definition in RFC 3393 [RFC3393].
The alternation of colors, for single marking method, can be used as The alternation of colors, for a Single-Marking Method, can be used
a time reference to measure delay variations. In case of double as a time reference to measure delay variations. In case of Double
marking, the time reference is given by the second marked packets. Marking, the time reference is given by the second-marked packets.
Considering the example depicted in Figure 2, R1 stores the timestamp
Considering the example depicted in Figure 2, R1 stores a timestamp TS(A)R1 whenever it sends the first packet of a block, and R2 stores
TS(A)R1 whenever it sends the first packet of a block and R2 stores a the timestamp TS(B)R2 whenever it receives the first packet of a
timestamp TS(B)R2 whenever it receives the first packet of a block. block. The inter-arrival jitter can be easily derived from one-way
The inter-arrival jitter can be easily derived from one-way delay delay measurement, by evaluating the delay variation of consecutive
measurement, by evaluating the delay variation of consecutive
samples. samples.
The concept of mean delay can also be applied to delay variation, by The concept of mean delay can also be applied to delay variation, by
evaluating the average variation of the interval between consecutive evaluating the average variation of the interval between consecutive
packets of the flow from R1 to R2. packets of the flow from R1 to R2.
4. Considerations 4. Considerations
This section highlights some considerations about the methodology. This section highlights some considerations about the methodology.
4.1. Synchronization 4.1. Synchronization
The Alternate Marking technique does not require a strong The Alternate-Marking technique does not require a strong
synchronization, especially for packet loss and two-way delay synchronization, especially for packet loss and two-way delay
measurement. Only one-way delay measurement requires network devices measurement. Only one-way delay measurement requires network devices
to have synchronized clocks. to have synchronized clocks.
The color switching is the reference for all the network devices, and Color switching is the reference for all the network devices, and the
the only requirement to be achieved is that all network devices have only requirement to be achieved is that all network devices have to
to recognize the right batch along the path. recognize the right batch along the path.
If the length of the measurement period is L time units, then all If the length of the measurement period is L time units, then all
network devices must be synchronized to the same clock reference with network devices must be synchronized to the same clock reference with
an accuracy of +/- L/2 time units (without considering network an accuracy of +/- L/2 time units (without considering network
delay). This level of accuracy guarantees that all network devices delay). This level of accuracy guarantees that all network devices
consistently match the color bit to the correct block. For example, consistently match the color bit to the correct block. For example,
if the color is toggeled every second (L = 1 second), then clocks if the color is toggled every second (L = 1 second), then clocks must
must be synchronized with an accuracy of +/- 0.5 second to a common be synchronized with an accuracy of +/- 0.5 second to a common time
time reference. reference.
This synchronization requirement can be satisfied even with a This synchronization requirement can be satisfied even with a
relatively inaccurate synchronization method. This is true for relatively inaccurate synchronization method. This is true for
packet loss and two-way delay measurement, instead, for one-way delay packet loss and two-way delay measurement, but not for one-way delay
measurement clock synchronization must be accurate. measurement, where clock synchronization must be accurate.
Therefore, a system that uses only packet loss and two-way delay Therefore, a system that uses only packet loss and two-way delay
measurement does not require synchronization. This is because the measurement does not require synchronization. This is because the
value of the clocks of network devices does not affect the value of the clocks of network devices does not affect the
computation of the two-way delay measurement. computation of the two-way delay measurement.
4.2. Data Correlation 4.2. Data Correlation
Data Correlation is the mechanism to compare counters and timestamps Data correlation is the mechanism to compare counters and timestamps
for packet loss, delay and delay variation calculation. It could be for packet loss, delay, and delay variation calculation. It could be
performed in several ways depending on the alternate marking performed in several ways depending on the Alternate-Marking
application and use case. application and use case. Some possibilities are to:
o A possibility is to use a centralized solution using Network o use a centralized solution using NMS to correlate data; and
Management System (NMS) to correlate data;
o Another possibility is to define a protocol based distributed o define a protocol-based distributed solution by introducing a new
solution, by defining a new protocol or by extending the existing protocol or by extending the existing protocols (e.g., see RFC
protocols (e.g. RFC6374, TWAMP, OWAMP) in order to communicate 6374 [RFC6374] or the Two-Way Active Measurement Protocol (TWAMP)
the counters and timestamps between nodes. as defined in RFC 5357 [RFC5357] or the One-Way Active Measurement
Protocol (OWAMP) as defined in RFC 4656 [RFC4656]) in order to
communicate the counters and timestamps between nodes.
In the following paragraphs an example data correlation mechanism is In the following paragraphs, an example data correlation mechanism is
explained and could be use independently of the adopted solutions. explained and could be used independently of the adopted solutions.
When data is collected on the upstream and downstream node, e.g., When data is collected on the upstream and downstream nodes, e.g.,
packet counts for packet loss measurement or timestamps for packet packet counts for packet loss measurement or timestamps for packet
delay measurement, and periodically reported to or pulled by other delay measurement, and is periodically reported to or pulled by other
nodes or NMS, a certain data correlation mechanism SHOULD be in use nodes or an NMS, a certain data correlation mechanism SHOULD be in
to help the nodes or NMS to tell whether any two or more packet use to help the nodes or NMS tell whether any two or more packet
counts are related to the same block of markers, or any two counts are related to the same block of markers or if any two
timestamps are related to the same marked packet. timestamps are related to the same marked packet.
The alternate marking method described in this document literally The Alternate-Marking Method described in this document literally
split the packets of the measured flow into different measurement splits the packets of the measured flow into different measurement
blocks, in addition a Block Number could be assigned to each of such blocks; in addition, a Block Number (BN) could be assigned to each
measurement block. The BN is generated each time a node reads the such measurement block. The BN is generated each time a node reads
data (packet counts or timestamps), and is associated with each the data (packet counts or timestamps) and is associated with each
packet count and timestamp reported to or pulled by other nodes or packet count and timestamp reported to or pulled by other nodes or
NMS. The value of BN could be calculated as the modulo of the local NMSs. The value of a BN could be calculated as the modulo of the
time (when the data are read) and the interval of the marking time local time (when the data are read) and the interval of the marking
period. time period.
When the nodes or NMS see, for example, same BNs associated with two When the nodes or NMS see, for example, the same BNs associated with
packet counts from an upstream and a downstream node respectively, it two packet counts from an upstream and a downstream node,
considers that these two packet counts corresponding to the same respectively, it considers that these two packet counts correspond to
block, i.e. that these two packet counts belong to the same block of the same block, i.e., these two packet counts belong to the same
markers from the upstream and downstream node. The assumption of block of markers from the upstream and downstream nodes. The
this BN mechanism is that the measurement nodes are time assumption of this BN mechanism is that the measurement nodes are
synchronized. This requires the measurement nodes to have a certain time synchronized. This requires the measurement nodes to have a
time synchronization capability (e.g., the Network Time Protocol certain time synchronization capability (e.g., the Network Time
(NTP) RFC 5905 [RFC5905], or the IEEE 1588 Precision Time Protocol Protocol (NTP) [RFC5905] or the IEEE 1588 Precision Time Protocol
(PTP) [IEEE-1588]). Synchronization aspects are further discussed in (PTP) [IEEE-1588]). Synchronization aspects are further discussed in
Section 4. Section 4.1.
4.3. Packet Re-ordering 4.3. Packet Reordering
Due to ECMP, packet re-ordering is very common in IP network. The Due to ECMP, packet reordering is very common in an IP network. The
accuracy of marking based PM, especially packet loss measurement, may accuracy of a marking-based PM, especially packet loss measurement,
be affected by packet re-ordering. Take a look at the following may be affected by packet reordering. Take a look at the following
example: example:
Block : 1 | 2 | 3 | 4 | 5 |... Block : 1 | 2 | 3 | 4 | 5 |...
--------|---------|---------|---------|---------|---------|--- --------|---------|---------|---------|---------|---------|---
Node R1 : AAAAAAA | BBBBBBB | AAAAAAA | BBBBBBB | AAAAAAA |... Node R1 : AAAAAAA | BBBBBBB | AAAAAAA | BBBBBBB | AAAAAAA |...
Node R2 : AAAAABB | AABBBBA | AAABAAA | BBBBBBA | ABAAABA |... Node R2 : AAAAABB | AABBBBA | AAABAAA | BBBBBBA | ABAAABA |...
Figure 5: Packet Reordering Figure 5: Packet Reordering
In Figure 5 the packet stream for Node R1 isn't being reordered, and In Figure 5, the packet stream for Node R1 isn't being reordered and
can be safely assigned to interval blocks, but the packet stream for can be safely assigned to interval blocks, but the packet stream for
Node R2 is being reordered, so, looking at the packet with the marker Node R2 is being reordered; so, looking at the packet with the marker
of "B" in block 3, there is no safe way to tell whether the packet of "B" in block 3, there is no safe way to tell whether the packet
belongs to block 2 or block 4. belongs to block 2 or block 4.
In general there is the need to assign packets with the marker of "B" In general, there is the need to assign packets with the marker of
or "A" to the right interval blocks. Most of the packet re-ordering "B" or "A" to the right interval blocks. Most of the packet
occur at the edge of adjacent blocks, and they are easy to handle if reordering occurs at the edge of adjacent blocks, and they are easy
the interval of each block is sufficient large. Then, it can assume to handle if the interval of each block is sufficiently large. Then,
that the packets with different marker belong to the block that they it can be assumed that the packets with different markers belong to
are more close to. If the interval is small, it is difficult and the block that they are closer to. If the interval is small, it is
sometime impossible to determine to which block a packet belongs. difficult and sometimes impossible to determine to which block a
packet belongs.
To choose a proper interval is important and how to choose a proper To choose a proper interval is important, and how to choose a proper
interval is out of the scope of this document. But an implementation interval is out of the scope of this document. But an implementation
SHOULD provide a way to configure the interval and allow a certain SHOULD provide a way to configure the interval and allow a certain
degree of packet re-ordering. degree of packet reordering.
5. Applications, implementation and deployment 5. Applications, Implementation, and Deployment
The methodology described in the previous sections can be applied in The methodology described in the previous sections can be applied in
various situations. Basically Alternate Marking technique could be various situations. Basically, the Alternate-Marking technique could
used in many cases for performance measurement. The only requirement be used in many cases for performance measurement. The only
is to select and mark the flow to be monitored; in this way packets requirement is to select and mark the flow to be monitored; in this
are batched by the sender and each batch is alternately marked such way, packets are batched by the sender, and each batch is alternately
that can be easily recognized by the receiver. marked such that it can be easily recognized by the receiver.
Some recent alternate marking method applications are listed below: Some recent Alternate-Marking Method applications are listed below:
o IP flow performance measurement (IPFPM): this application of o IP Flow Performance Measurement (IPFPM): this application of the
marking method is described in marking method is described in [COLORING]. As an example, in this
[I-D.chen-ippm-coloring-based-ipfpm-framework]. As an example, in document, the last reserved bit of the Flag field of the IPv4
this document, the last reserved bit of the Flag field of the IPv4
header is proposed to be used for marking, while a solution for header is proposed to be used for marking, while a solution for
IPv6 could be to leverage the IPv6 extension header for marking. IPv6 could be to leverage the IPv6 extension header for marking.
o OAM Passive Performance Measurement: In o OAM Passive Performance Measurement: In [RFC8296], two OAM bits
[I-D.ietf-bier-mpls-encapsulation] two OAM bits from Bit Index from the Bit Index Explicit Replication (BIER) header are reserved
Explicit Replication (BIER) Header are reserved for the passive for the Passive performance measurement marking method.
performance measurement marking method. [I-D.ietf-bier-pmmm-oam] [PM-MM-BIER] details the measurement for multicast service over
details the measurement for multicast service over BIER domain. the BIER domain. In addition, the Alternate-Marking Method could
In addition, the alternate marking method could also be used in a also be used in a Service Function Chaining (SFC) domain. Lastly,
Service Function Chaining (SFC) domain. Lastly the application of the application of the marking method to Network Virtualization
the marking method to Network Virtualization Overlays (NVO3) over Layer 3 (NVO3) protocols is considered by [NVO3-ENCAPS].
protocols is considered by [I-D.ietf-nvo3-encap].
o RFC6374 Use Case: RFC6374 [RFC6374] uses the LM packet as the o MPLS Performance Measurement: RFC 6374 [RFC6374] uses the Loss
packet accounting demarcation point. Unfortunately this gives Measurement (LM) packet as the packet accounting demarcation
rise to a number of problems that may lead to significant packet point. Unfortunately, this gives rise to a number of problems
accounting errors in certain situations. that may lead to significant packet accounting errors in certain
[I-D.ietf-mpls-flow-ident] discusses the desired capabilities for situations. [MPLS-FLOW] discusses the desired capabilities for
MPLS flow identification in order to perform a better in-band MPLS flow identification in order to perform a better in-band
performance monitoring of user data packets. A method of performance monitoring of user data packets. A method of
accomplishing identification is Synonymous Flow Labels (SFL) accomplishing identification is Synonymous Flow Labels (SFLs)
introduced in [I-D.bryant-mpls-sfl-framework], while introduced in [SFL-FRAMEWORK], while [SYN-FLOW-LABELS] describes
[I-D.ietf-mpls-rfc6374-sfl] describes RFC6374 performance performance measurements in RFC 6374 with SFL.
measurements with SFL.
o active performance measurement: o Active Performance Measurement: [ALT-MM-AMP] describes how to
[I-D.fioccola-ippm-alt-mark-active] describes how to extend the extend the existing Active Measurement Protocol, in order to
existing Active Measurement Protocol, in order to implement implement the Alternate-Marking methodology. [ALT-MM-SLA]
alternate marking methodology. describes an extension to the Cisco SLA Protocol Measurement-Type
[I-D.fioccola-ippm-rfc6812-alt-mark-ext] describes an extension to UDP-Measurement.
the Cisco SLA Protocol Measurement-Type UDP-Measurement.
An example of implementation and deployment is explained in the next An example of implementation and deployment is explained in the next
section, just to clarify how the method can work. section, just to clarify how the method can work.
5.1. Report on the operational experiment 5.1. Report on the Operational Experiment
The method described in this document, also called PNPM (Packet The method described in this document, also called Packet Network
Network Performance Monitoring), has been invented and engineered in Performance Monitoring (PNPM), has been invented and engineered in
Telecom Italia. Telecom Italia.
It is important to highlight that the general description of the It is important to highlight that the general description of the
methodology in this document is a consequence of the operational methodology in this document is a consequence of the operational
experiment. The foundational elements of the technique have been experiment. The fundamental elements of the technique have been
tested and the lessons learnt from the operational experiment tested, and the lessons learned from the operational experiment
inspired the formalization of the Alternate Marking Method as inspired the formalization of the Alternate-Marking Method as
detailed in the previous sections. detailed in the previous sections.
The methodology is experimented in Telecom Italia's network and is The methodology has been used experimentally in Telecom Italia's
applied to multicast IPTV channels or other specific traffic flows network and is applied to multicast IPTV channels or other specific
with high QoS requirements (i.e. Mobile Backhauling traffic realized traffic flows with high QoS requirements (i.e., Mobile Backhauling
with a VPN MPLS). traffic realized with a VPN MPLS).
This technology has been employed by leveraging functions and tools This technology has been employed by leveraging functions and tools
available on IP routers and it's currently being used to monitor available on IP routers, and it's currently being used to monitor
packet loss in some portions of the Telecom Italia's network. The packet loss in some portions of Telecom Italia's network. The
application of the method to delay measurement has also been application of this method for delay measurement has also been
evaluated in Telecom Italia's labs. evaluated in Telecom Italia's labs.
This Section describes how the experiment has been executed, in This section describes how the experiment has been executed,
particular how the features currently available on existing routing particularly, how the features currently available on existing
platforms can be used to apply the method, in order to give an routing platforms can be used to apply the method, in order to give
example of implementation and deployment. an example of implementation and deployment.
The operational test, here described, uses the flow-based strategy, The operational test, described herein, uses the flow-based strategy,
as defined in Section 3. Instead the link-based strategy could be as defined in Section 3. Instead, the link-based strategy could be
applied to physical link or a logical link (e.g. Ethernet VLAN or a applied to a physical link or a logical link (e.g., an Ethernet VLAN
MPLS PW). or an MPLS Pseudowire (PW)).
The implementation of the method leverages the available router The implementation of the method leverages the available router
functions, since the experiment has been done by a Service Provider functions, since the experiment has been done by a Service Provider
(as Telecom Itlaia is) on its own network. So, with current router (as Telecom Italia is) on its own network. So, with current router
implementations, only QoS related fields and features offer the implementations, only QoS-related fields and features offer the
required flexibility to set bits in the packet header. In case a required flexibility to set bits in the packet header. In case a
Service Provider only uses the three most significant bits of the Service Provider only uses the three most-significant bits of the
DSCP field (corresponding to IP Precedence) for QoS classification DSCP field (corresponding to IP Precedence) for QoS classification
and queuing, it is possible to use the two less significant bits of and queuing, it is possible to use the two least-significant bits of
the DSCP field (bit 0 and bit 1) to implement the method without the DSCP field (bit 0 and bit 1) to implement the method without
affecting QoS policies. That is the approach used for the affecting QoS policies. That is the approach used for the
experiment. One of the two bits (bit 0) could be used to identify experiment. One of the two bits (bit 0) could be used to identify
flows subject to traffic monitoring (set to 1 if the flow is under flows subject to traffic monitoring (set to 1 if the flow is under
monitoring, otherwise it is set to 0), while the second (bit 1) can monitoring, otherwise, it is set to 0), while the second (bit 1) can
be used for coloring the traffic (switching between values 0 and 1, be used for coloring the traffic (switching between values 0 and 1,
corresponding to color A and B) and creating the blocks. corresponding to colors A and B) and creating the blocks.
The experiment considers a flow as all the packets sharing the same The experiment considers a flow as all the packets sharing the same
source IP address or the same destination IP address, depending on source IP address or the same destination IP address, depending on
the direction. In practice, once the flow has been defined, coloring the direction. In practice, once the flow has been defined, traffic
the traffic using the DSCP field can be implemented by configuring on coloring using the DSCP field can be implemented by configuring an
the router output interface an access list that intercepts the access-list on the router output interface. The access-list
flow(s) to be monitored and applies to them a policy that sets the intercepts the flow(s) to be monitored and applies a policy to them
DSCP field accordingly. Since traffic coloring has to be switched that sets the DSCP field accordingly. Since traffic coloring has to
between the two values over time, the policy needs to be modified be switched between the two values over time, the policy needs to be
periodically: an automatic script is used to perform this task on the modified periodically. An automatic script is used to perform this
basis of a fixed timer. The automatic script is loaded on board of task on the basis of a fixed timer. The automatic script is loaded
the router and automatizes the basic operations that are needed to on board of the router and automatizes the basic operations that are
realize the methodology. needed to realize the methodology.
After the traffic is colored using the DSCP field, all the routers on After the traffic is colored using the DSCP field, all the routers on
the path can perform the counting. For this purpose an access-list the path can perform the counting. For this purpose, an access-list
that matches specific DSCP values can be used to count the packets of that matches specific DSCP values can be used to count the packets of
the flow(s) being monitored. The same access-list can be installed the flow(s) being monitored. The same access-list can be installed
on all the routers of the path. In addition, network flow on all the routers of the path. In addition, network flow
monitoring, such as provided by IPFIX (RFC 7011 [RFC7011]), can be monitoring, such as provided by IPFIX [RFC7011], can be used to
used to recognize timestamps of first/last packet of a batch in order recognize timestamps of the first/last packet of a batch in order to
to enable one of the alternatives to measure the delay as detailed in enable one of the alternatives to measure the delay as detailed in
Section 3.3. Section 3.3.
In the Telecom Italia's experiment the timer is set to 5 minutes, so In Telecom Italia's experiment, the timer is set to 5 minutes, so the
the sequence of actions of the script is also executed every 5 sequence of actions of the script is also executed every 5 minutes.
minutes. This value has showed to be a good compromise between This value has shown to be a good compromise between measurement
measurement frequency and stability of the measurement (i.e. frequency and stability of the measurement (i.e., the possibility of
possibility to collect all the measures referring to the same block). collecting all the measures referring to the same block).
For this expertiment, both counters and any other data are collected For this experiment, both counters and any other data are collected
by using the automatic script that sends out these to a Network by using the automatic script that sends these out to an NMS. The
Management System (NMS). The NMS is responsible for packet loss NMS is responsible for packet loss calculation, performed by
calculation, performed by comparing the values of counters from the comparing the values of counters from the routers along the flow
routers along the flow(s) path. 5 minutes timer for color switching path(s). A 5-minute timer for color switching is a safe choice for
is a safe choice for reading the counters and is also coherent with reading the counters and is also coherent with the reporting window
the reporting window of the NMS. of the NMS.
Note that the use of the DSCP field for marking implies that the Note that the use of the DSCP field for marking implies that the
method in this case works reliably only within a single management method in this case works reliably only within a single management
and operation domain. and operation domain.
Lastly, the Telecom Italia experiment scales up to 1000 flows Lastly, the Telecom Italia experiment scales up to 1000 flows
monitored together on a single router, while an implementation on monitored together on a single router, while an implementation on
dedicated hardware scales more, but it was tested only in labs for dedicated hardware scales more, but it was tested only in labs for
now. now.
5.1.1. Metric transparency 5.1.1. Metric Transparency
Since a Service Provider application is described here, the method Since a Service Provider application is described here, the method
can be applied to end-to-end services supplied to Customers. So it can be applied to end-to-end services supplied to customers. So it
is important to highlight that the method MUST be transparent outside is important to highlight that the method MUST be transparent outside
the Service Provider domain. the Service Provider domain.
In Telecom Italia's implementation the source node colors the packets In Telecom Italia's implementation, the source node colors the
with a policy that is modified periodically via an automatic script packets with a policy that is modified periodically via an automatic
in order to alternate the DSCP field of the packets. The nodes script in order to alternate the DSCP field of the packets. The
between source and destination (included) have to count with an nodes between source and destination (included) have to use an
access-list the colored packets that they receive and forward. access-list to count the colored packets that they receive and
forward.
Moreover the destination node has an important role: the colored Moreover, the destination node has an important role: the colored
packets are intercepted and a policy restores and sets the DSCP field packets are intercepted and a policy restores and sets the DSCP field
of all the packets to the initial value. In this way the metric is of all the packets to the initial value. In this way, the metric is
transparent because outside the section of the network under transparent because outside the section of the network under
monitoring the traffic flow is unchanged. monitoring, the traffic flow is unchanged.
In such a case, thanks to this restoring technique, network elements In such a case, thanks to this restoring technique, network elements
outside the Alternate Marking monitoring domain (e.g. the two outside the Alternate-Marking monitoring domain (e.g., the two
Provider Edge nodes of the Mobile Backhauling VPN MPLS) are totally Provider Edge nodes of the Mobile Backhauling VPN MPLS) are totally
anaware that packets were marked. So this restoring technique makes unaware that packets were marked. So this restoring technique makes
Alternate Marking completely transparent outside its monitoring Alternate Marking completely transparent outside its monitoring
domain. domain.
6. Hybrid measurement 6. Hybrid Measurement
The method has been explicitly designed for passive measurements but The method has been explicitly designed for Passive measurements, but
it can also be used with active measurements. In order to have both it can also be used with Active measurements. In order to have both
end to end measurements and intermediate measurements (hybrid end-to-end measurements and intermediate measurements (Hybrid
measurements) two end points can exchanges artificial traffic flows measurements), two endpoints can exchange artificial traffic flows
and apply alternate marking over these flows. In the intermediate and apply Alternate Marking over these flows. In the intermediate
points artificial traffic is managed in the same way as real traffic points, artificial traffic is managed in the same way as real traffic
and measured as specified before. So the application of marking and measured as specified before. So the application of the marking
method can simplify also the active measurement, as explained in method can also simplify the Active measurement, as explained in
[I-D.fioccola-ippm-alt-mark-active]. [ALT-MM-AMP].
7. Compliance with RFC6390 guidelines 7. Compliance with Guidelines from RFC 6390
RFC6390 [RFC6390] defines a framework and a process for developing RFC 6390 [RFC6390] defines a framework and a process for developing
Performance Metrics for protocols above and below the IP layer (such Performance Metrics for protocols above and below the IP layer (such
as IP-based applications that operate over reliable or datagram as IP-based applications that operate over reliable or datagram
transport protocols). transport protocols).
This document doesn't aim to propose a new Performance Metric but a This document doesn't aim to propose a new Performance Metric but
new method of measurement for a few Performance Metrics that have rather a new Method of Measurement for a few Performance Metrics that
already been standardized. Nevertheless, it's worth applying have already been standardized. Nevertheless, it's worth applying
[RFC6390] guidelines to the present document, in order to provide a guidelines from [RFC6390] to the present document, in order to
more complete and coherent description of the proposed method. We provide a more complete and coherent description of the proposed
used a subset of the Performance Metric Definition template defined method. We used a combination of the Performance Metric Definition
by [RFC6390]. template defined in Section 5.4 of [RFC6390] and the Dependencies
laid out in Section 5.5 of that document.
o Metric name and description: as already stated, this document o Metric Name / Metric Description: as already stated, this document
doesn't propose any new Performance Metric. On the contrary, it doesn't propose any new Performance Metrics. On the contrary, it
describes a novel method for measuring packet loss [RFC7680]. The describes a novel method for measuring packet loss [RFC7680]. The
same concept, with small differences, can also be used to measure same concept, with small differences, can also be used to measure
delay [RFC7679], and jitter [RFC3393]. The document mainly delay [RFC7679] and jitter [RFC3393]. The document mainly
describes the applicability to packet loss measurement. describes the applicability to packet loss measurement.
o Method of Measurement or Calculation: according to the method o Method of Measurement or Calculation: according to the method
described in the previous sections, the number of packets lost is described in the previous sections, the number of packets lost is
calculated by subtracting the value of the counter on the source calculated by subtracting the value of the counter on the source
node from the value of the counter on the destination node. Both node from the value of the counter on the destination node. Both
counters must refer to the same color. The calculation is counters must refer to the same color. The calculation is
performed when the value of the counters is in a steady state. performed when the value of the counters is in a steady state.
The steady state is an intrinsic characteristic of the marking The steady state is an intrinsic characteristic of the marking
method counters because the alternation of color makes the method counters because the alternation of color makes the
counters associated to each color still one at a time for the counters associated with each color still one at a time for the
duration of a marking period. duration of a marking period.
o Units of Measurement: the method calculates and reports the exact o Units of Measurement: the method calculates and reports the exact
number of packets sent by the source node and not received by the number of packets sent by the source node and not received by the
destination node. destination node.
o Measurement Points: the measurement can be performed between o Measurement Point(s) with Potential Measurement Domain: the
adjacent nodes, on a per-link basis, or along a multi-hop path, measurement can be performed between adjacent nodes, on a per-link
provided that the traffic under measurement follows that path. In basis, or along a multi-hop path, provided that the traffic under
case of a multi-hop path, the measurements can be performed both measurement follows that path. In case of a multi-hop path, the
end-to-end and hop-by-hop. measurements can be performed both end-to-end and hop-by-hop.
o Measurement Timing: the method have a constraint on the frequency o Measurement Timing: the method has a constraint on the frequency
of measurements. This is detailed in Section 3.2, where it is of measurements. This is detailed in Section 3.2, where it is
specified that the marking period and the guardband interval are specified that the marking period and the guard band interval are
strictly related each other to avoid out of order issues. That is strictly related each other to avoid out-of-order issues. That is
because, in order to perform a measure, the counter must be in a because, in order to perform a measurement, the counter must be in
steady state and this happens when the traffic is being colored a steady state, and this happens when the traffic is being colored
with the alternate color. As an example in the experiment of the with the alternate color. As an example, in the experiment of the
method the time interval is set to 5 minutes, while other method, the time interval is set to 5 minutes, while other
optimized implementations can also use a marking period of a few optimized implementations can also use a marking period of a few
seconds. seconds.
o Implementation: the experiment of the method uses two encodings of o Implementation: the experiment of the method uses two encodings of
the DSCP field to color the packets; this enables the use of the DSCP field to color the packets; this enables the use of
policy configurations on the router to color the packets and policy configurations on the router to color the packets and
accordingly configure the counter for each color. The path accordingly configure the counter for each color. The path
followed by traffic being measured should be known in advance in followed by traffic being measured should be known in advance in
order to configure the counters along the path and be able to order to configure the counters along the path and be able to
compare the correct values. compare the correct values.
o Verification: both in the Lab and in the operational network the o Verification: both in the lab and in the operational network, the
methodology has been tested and experimented for packet loss and methodology has been tested and experimented for packet loss and
delay measurements by using traffic generators together with delay measurements by using traffic generators together with
precision test instruments and network emulators. precision test instruments and network emulators.
o Use and Applications: the method can be used to measure packet o Use and Applications: the method can be used to measure packet
loss with high precision on live traffic; moreover, by combining loss with high precision on live traffic; moreover, by combining
end-to-end and per-link measurements, the method is useful to end-to-end and per-link measurements, the method is useful to
pinpoint the single link that is experiencing loss events. pinpoint the single link that is experiencing loss events.
o Reporting Model: the value of the counters has to be sent to a o Reporting Model: the value of the counters has to be sent to a
centralized management system that perform the calculations; such centralized management system that performs the calculations; such
samples must contain a reference to the time interval they refer samples must contain a reference to the time interval they refer
to, so that the management system can perform the correct to, so that the management system can perform the correct
correlation; the samples have to be sent while the corresponding correlation; the samples have to be sent while the corresponding
counter is in a steady state (within a time interval), otherwise counter is in a steady state (within a time interval); otherwise,
the value of the sample should be stored locally. the value of the sample should be stored locally.
o Dependencies: the values of the counters have to be correlated to o Dependencies: the values of the counters have to be correlated to
the time interval they refer to; moreover, as far the experiment the time interval they refer to; moreover, because the experiment
of the method is based on DSCP values, there are significant of the method is based on DSCP values, there are significant
dependencies on the usage of the DSCP field: it must be possible dependencies on the usage of the DSCP field: it must be possible
to rely on unused DSCP values without affecting QoS-related to rely on unused DSCP values without affecting QoS-related
configuration and behavior; moreover, the intermediate nodes must configuration and behavior; moreover, the intermediate nodes must
not change the value of the DSCP field not to alter the not change the value of the DSCP field not to alter the
measurement. measurement.
o Organization of Results: the method of measurement produces o Organization of Results: the Method of Measurement produces
singletons. singletons.
o Parameters: currently, the main parameter of the method is the o Parameters: currently, the main parameter of the method is the
time interval used to alternate the colors and read the counters. time interval used to alternate the colors and read the counters.
8. Security Considerations 8. IANA Considerations
This document has no IANA actions.
9. Security Considerations
This document specifies a method to perform measurements in the This document specifies a method to perform measurements in the
context of a Service Provider's network and has not been developed to context of a Service Provider's network and has not been developed to
conduct Internet measurements, so it does not directly affect conduct Internet measurements, so it does not directly affect
Internet security nor applications which run on the Internet. Internet security nor applications that run on the Internet.
However, implementation of this method must be mindful of security However, implementation of this method must be mindful of security
and privacy concerns. and privacy concerns.
There are two types of security concerns: potential harm caused by There are two types of security concerns: potential harm caused by
the measurements and potential harm to the measurements. the measurements and potential harm to the measurements.
o Harm caused by the measurement: the measurements described in this o Harm caused by the measurement: the measurements described in this
document are passive, so there are no new packets injected into document are Passive, so there are no new packets injected into
the network causing potential harm to the network itself and to the network causing potential harm to the network itself and to
data traffic. Nevertheless, the method implies modifications on data traffic. Nevertheless, the method implies modifications on
the fly to a header or encapsulation of the data packets: this the fly to a header or encapsulation of the data packets: this
must be performed in a way that doesn't alter the quality of must be performed in a way that doesn't alter the quality of
service experienced by packets subject to measurements and that service experienced by packets subject to measurements and that
preserve stability and performance of routers doing the preserves stability and performance of routers doing the
measurements. One of the main security threats in OAM protocols measurements. One of the main security threats in OAM protocols
is network reconnaissance; an attacker can gather information is network reconnaissance; an attacker can gather information
about the network performance by passively eavesdropping to OAM about the network performance by passively eavesdropping on OAM
messages. The advantage of the methods described in this document messages. The advantage of the methods described in this document
is that the marking bits are the only information that is is that the marking bits are the only information that is
exchanged between the network devices. Therefore, passive exchanged between the network devices. Therefore, Passive
eavesdropping to data plane traffic does not allow attackers to eavesdropping on data-plane traffic does not allow attackers to
gain information about the network performance. gain information about the network performance.
o Harm to the measurement: the measurements could be harmed by o Harm to the Measurement: the measurements could be harmed by
routers altering the marking of the packets, or by an attacker routers altering the marking of the packets or by an attacker
injecting artificial traffic. Authentication techniques, such as injecting artificial traffic. Authentication techniques, such as
digital signatures, may be used where appropriate to guard against digital signatures, may be used where appropriate to guard against
injected traffic attacks. Since the measurement itself may be injected traffic attacks. Since the measurement itself may be
affected by routers (or other network devices) along the path of affected by routers (or other network devices) along the path of
IP packets intentionally altering the value of marking bits of IP packets intentionally altering the value of marking bits of
packets, as mentioned above, the mechanism specified in this packets, as mentioned above, the mechanism specified in this
document can be applied just in the context of a controlled document can be applied just in the context of a controlled
domain, and thus the routers (or other network devices) are domain; thus, the routers (or other network devices) are locally
locally administered and this type of attack can be avoided. In administered and this type of attack can be avoided. In addition,
addition, an attacker can't gain information about network an attacker can't gain information about network performance from
performance from a single monitoring point, and must use a single monitoring point; it must use synchronized monitoring
synchronized monitoring points at multiple points on the path, points at multiple points on the path, because they have to do the
because they have to do the same kind of measurement and same kind of measurement and aggregation that Service Providers
aggregation that Service Providers using Alternate Marking must using Alternate Marking must do.
do.
The privacy concerns of network measurement are limited because the The privacy concerns of network measurement are limited because the
method only relies on information contained in the header or method only relies on information contained in the header or
encapsulation without any release of user data.Although information encapsulation without any release of user data. Although information
in the header or encapsulation is metadata that can be used to in the header or encapsulation is metadata that can be used to
compromise the privacy of users, the limited marking technique in compromise the privacy of users, the limited marking technique in
this document seems unlikely to substantially increase the existing this document seems unlikely to substantially increase the existing
privacy risks from header or encapsulation metadata.It might be privacy risks from header or encapsulation metadata. It might be
theoretically possible to modulate the marking to serve as a covert theoretically possible to modulate the marking to serve as a covert
channel, but it would have a very low data rate if it is to avoid channel, but it would have a very low data rate if it is to avoid
adversely affecting the measurement systems that monitor the marking. adversely affecting the measurement systems that monitor the marking.
Delay attacks are another potential threat in the context of this Delay attacks are another potential threat in the context of this
document. Delay measurement is performed using a specific packet in document. Delay measurement is performed using a specific packet in
each block, marked by a dedicated color bit. Therefore, a man-in- each block, marked by a dedicated color bit. Therefore, a
the-middle attacker can selectively induce synthetic delay only to man-in-the-middle attacker can selectively induce synthetic delay
delay-colored packets, causing systematic error in the delay only to delay-colored packets, causing systematic error in the delay
measurements. As discussed in previous sections, the methods measurements. As discussed in previous sections, the methods
described in this document rely on an underlying time synchronization described in this document rely on an underlying time synchronization
protocol. Thus, by attacking the time protocol an attacker can protocol. Thus, by attacking the time protocol, an attacker can
potentially compromise the integrity of the measurement. A detailed potentially compromise the integrity of the measurement. A detailed
discussion about the threats against time protocols and how to discussion about the threats against time protocols and how to
mitigate them is presented in RFC 7384 [RFC7384]. mitigate them is presented in RFC 7384 [RFC7384].
9. IANA Considerations 10. References
There are no IANA actions required.
10. Acknowledgements
The previous IETF drafts about this technique were:
[I-D.cociglio-mboned-multicast-pm] and [I-D.tempia-opsawg-p3m].
The authors would like to thank Alberto Tempia Bonda, Domenico
Laforgia, Daniele Accetta and Mario Bianchetti for their contribution
to the definition and the implementation of the method.
The authors would also thank Spencer Dawkins, Carlos Pignataro, Brian
Haberman and Eric Vyncke for their assistance and their detailed and
precious reviews.
11. References
11.1. Normative References 10.1. Normative References
[IEEE-1588] [IEEE-1588]
IEEE 1588-2008, "IEEE Standard for a Precision Clock IEEE, "IEEE Standard for a Precision Clock Synchronization
Synchronization Protocol for Networked Measurement and Protocol for Networked Measurement and Control Systems",
Control Systems", July 2008. IEEE Std 1588-2008.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997, DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>. <https://www.rfc-editor.org/info/rfc2119>.
[RFC3393] Demichelis, C. and P. Chimento, "IP Packet Delay Variation [RFC3393] Demichelis, C. and P. Chimento, "IP Packet Delay Variation
Metric for IP Performance Metrics (IPPM)", RFC 3393, Metric for IP Performance Metrics (IPPM)", RFC 3393,
DOI 10.17487/RFC3393, November 2002, DOI 10.17487/RFC3393, November 2002,
<https://www.rfc-editor.org/info/rfc3393>. <https://www.rfc-editor.org/info/rfc3393>.
skipping to change at page 29, line 14 skipping to change at page 29, line 24
[RFC7680] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton, [RFC7680] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton,
Ed., "A One-Way Loss Metric for IP Performance Metrics Ed., "A One-Way Loss Metric for IP Performance Metrics
(IPPM)", STD 82, RFC 7680, DOI 10.17487/RFC7680, January (IPPM)", STD 82, RFC 7680, DOI 10.17487/RFC7680, January
2016, <https://www.rfc-editor.org/info/rfc7680>. 2016, <https://www.rfc-editor.org/info/rfc7680>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>. May 2017, <https://www.rfc-editor.org/info/rfc8174>.
11.2. Informative References 10.2. Informative References
[I-D.bryant-mpls-sfl-framework] [ALT-MM-AMP]
Bryant, S., Chen, M., Li, Z., Swallow, G., Sivabalan, S., Fioccola, G., Clemm, A., Bryant, S., Cociglio, M.,
and G. Mirsky, "Synonymous Flow Label Framework", draft- Chandramouli, M., and A. Capello, "Alternate Marking
bryant-mpls-sfl-framework-05 (work in progress), June Extension to Active Measurement Protocol", Work in
Progress, draft-fioccola-ippm-alt-mark-active-01, March
2017. 2017.
[I-D.chen-ippm-coloring-based-ipfpm-framework] [ALT-MM-SLA]
Chen, M., Zheng, L., Mirsky, G., Fioccola, G., and T. Fioccola, G., Clemm, A., Cociglio, M., Chandramouli, M.,
Mizrahi, "IP Flow Performance Measurement Framework", and A. Capello, "Alternate Marking Extension to Cisco SLA
draft-chen-ippm-coloring-based-ipfpm-framework-06 (work in Protocol RFC6812", Work in Progress, draft-fioccola-ippm-
progress), March 2016. rfc6812-alt-mark-ext-01, March 2016.
[I-D.chen-ippm-ipfpm-report] [COLORING] Chen, M., Zheng, L., Mirsky, G., Fioccola, G., and T.
Mizrahi, "IP Flow Performance Measurement Framework", Work
in Progress, draft-chen-ippm-coloring-based-ipfpm-
framework-06, March 2016.
[IP-FLOW-REPORT]
Chen, M., Zheng, L., and G. Mirsky, "IP Flow Performance Chen, M., Zheng, L., and G. Mirsky, "IP Flow Performance
Measurement Report", draft-chen-ippm-ipfpm-report-01 (work Measurement Report", Work in Progress, draft-chen-ippm-
in progress), April 2016. ipfpm-report-01, April 2016.
[I-D.cociglio-mboned-multicast-pm] [IP-MULTICAST-PM]
Cociglio, M., Capello, A., Bonda, A., and L. Castaldelli, Cociglio, M., Capello, A., Bonda, A., and L. Castaldelli,
"A method for IP multicast performance monitoring", draft- "A method for IP multicast performance monitoring", Work
cociglio-mboned-multicast-pm-01 (work in progress), in Progress, draft-cociglio-mboned-multicast-pm-01,
October 2010. October 2010.
[I-D.fioccola-ippm-alt-mark-active] [MPLS-FLOW]
Fioccola, G., Clemm, A., Bryant, S., Cociglio, M., Bryant, S., Pignataro, C., Chen, M., Li, Z., and G.
Chandramouli, M., and A. Capello, "Alternate Marking Mirsky, "MPLS Flow Identification Considerations", Work in
Extension to Active Measurement Protocol", draft-fioccola- Progress, draft-ietf-mpls-flow-ident-06, December 2017.
ippm-alt-mark-active-01 (work in progress), March 2017.
[I-D.fioccola-ippm-multipoint-alt-mark] [MULTIPOINT-ALT-MM]
Fioccola, G., Cociglio, M., Sapio, A., and R. Sisto, Fioccola, G., Cociglio, M., Sapio, A., and R. Sisto,
"Multipoint Alternate Marking method for passive and "Multipoint Alternate Marking method for passive and
hybrid performance monitoring", draft-fioccola-ippm- hybrid performance monitoring", Work in Progress,
multipoint-alt-mark-01 (work in progress), October 2017. draft-fioccola-ippm-multipoint-alt-mark-01, October 2017.
[I-D.fioccola-ippm-rfc6812-alt-mark-ext] [NVO3-ENCAPS]
Fioccola, G., Clemm, A., Cociglio, M., Chandramouli, M., Boutros, S., Ganga, I., Garg, P., Manur, R., Mizrahi, T.,
and A. Capello, "Alternate Marking Extension to Cisco SLA Mozes, D., Nordmark, E., Smith, M., Aldrin, S., and I.
Protocol RFC6812", draft-fioccola-ippm-rfc6812-alt-mark- Bagdonas, "NVO3 Encapsulation Considerations", Work in
ext-01 (work in progress), March 2016. Progress, draft-ietf-nvo3-encap-01, October 2017.
[I-D.ietf-bier-mpls-encapsulation] [OPSAWG-P3M]
Wijnands, I., Rosen, E., Dolganow, A., Tantsura, J., Capello, A., Cociglio, M., Castaldelli, L., and A. Bonda,
Aldrin, S., and I. Meilik, "Encapsulation for Bit Index "A packet based method for passive performance
Explicit Replication in MPLS and non-MPLS Networks", monitoring", Work in Progress, draft-tempia-opsawg-p3m-04,
draft-ietf-bier-mpls-encapsulation-12 (work in progress), February 2014.
October 2017.
[I-D.ietf-bier-pmmm-oam] [PM-MM-BIER]
Mirsky, G., Zheng, L., Chen, M., and G. Fioccola, Mirsky, G., Zheng, L., Chen, M., and G. Fioccola,
"Performance Measurement (PM) with Marking Method in Bit "Performance Measurement (PM) with Marking Method in Bit
Index Explicit Replication (BIER) Layer", draft-ietf-bier- Index Explicit Replication (BIER) Layer", Work in
pmmm-oam-03 (work in progress), October 2017. Progress, draft-ietf-bier-pmmm-oam-03, October 2017.
[I-D.ietf-mpls-flow-ident]
Bryant, S., Pignataro, C., Chen, M., Li, Z., and G.
Mirsky, "MPLS Flow Identification Considerations", draft-
ietf-mpls-flow-ident-05 (work in progress), July 2017.
[I-D.ietf-mpls-rfc6374-sfl]
Bryant, S., Chen, M., Li, Z., Swallow, G., Sivabalan, S.,
Mirsky, G., and G. Fioccola, "RFC6374 Synonymous Flow
Labels", draft-ietf-mpls-rfc6374-sfl-01 (work in
progress), December 2017.
[I-D.ietf-nvo3-encap] [RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M.
Boutros, S., Ganga, I., Garg, P., Manur, R., Mizrahi, T., Zekauskas, "A One-way Active Measurement Protocol
Mozes, D., Nordmark, E., Smith, M., Aldrin, S., and I. (OWAMP)", RFC 4656, DOI 10.17487/RFC4656, September 2006,
Bagdonas, "NVO3 Encapsulation Considerations", draft-ietf- <https://www.rfc-editor.org/info/rfc4656>.
nvo3-encap-01 (work in progress), October 2017.
[I-D.tempia-opsawg-p3m] [RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K., and J.
Capello, A., Cociglio, M., Castaldelli, L., and A. Bonda, Babiarz, "A Two-Way Active Measurement Protocol (TWAMP)",
"A packet based method for passive performance RFC 5357, DOI 10.17487/RFC5357, October 2008,
monitoring", draft-tempia-opsawg-p3m-04 (work in <https://www.rfc-editor.org/info/rfc5357>.
progress), February 2014.
[RFC5481] Morton, A. and B. Claise, "Packet Delay Variation [RFC5481] Morton, A. and B. Claise, "Packet Delay Variation
Applicability Statement", RFC 5481, DOI 10.17487/RFC5481, Applicability Statement", RFC 5481, DOI 10.17487/RFC5481,
March 2009, <https://www.rfc-editor.org/info/rfc5481>. March 2009, <https://www.rfc-editor.org/info/rfc5481>.
[RFC6374] Frost, D. and S. Bryant, "Packet Loss and Delay [RFC6374] Frost, D. and S. Bryant, "Packet Loss and Delay
Measurement for MPLS Networks", RFC 6374, Measurement for MPLS Networks", RFC 6374,
DOI 10.17487/RFC6374, September 2011, DOI 10.17487/RFC6374, September 2011,
<https://www.rfc-editor.org/info/rfc6374>. <https://www.rfc-editor.org/info/rfc6374>.
skipping to change at page 31, line 40 skipping to change at page 31, line 40
<https://www.rfc-editor.org/info/rfc7276>. <https://www.rfc-editor.org/info/rfc7276>.
[RFC7384] Mizrahi, T., "Security Requirements of Time Protocols in [RFC7384] Mizrahi, T., "Security Requirements of Time Protocols in
Packet Switched Networks", RFC 7384, DOI 10.17487/RFC7384, Packet Switched Networks", RFC 7384, DOI 10.17487/RFC7384,
October 2014, <https://www.rfc-editor.org/info/rfc7384>. October 2014, <https://www.rfc-editor.org/info/rfc7384>.
[RFC7799] Morton, A., "Active and Passive Metrics and Methods (with [RFC7799] Morton, A., "Active and Passive Metrics and Methods (with
Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799, Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799,
May 2016, <https://www.rfc-editor.org/info/rfc7799>. May 2016, <https://www.rfc-editor.org/info/rfc7799>.
[RFC8296] Wijnands, IJ., Ed., Rosen, E., Ed., Dolganow, A.,
Tantsura, J., Aldrin, S., and I. Meilik, "Encapsulation
for Bit Index Explicit Replication (BIER) in MPLS and Non-
MPLS Networks", RFC 8296, DOI 10.17487/RFC8296, January
2018, <https://www.rfc-editor.org/info/rfc8296>.
[SFL-FRAMEWORK]
Bryant, S., Chen, M., Li, Z., Swallow, G., Sivabalan, S.,
and G. Mirsky, "Synonymous Flow Label Framework", Work in
Progress, draft-ietf-mpls-sfl-framework-00, August 2017.
[SYN-FLOW-LABELS]
Bryant, S., Chen, M., Li, Z., Swallow, G., Sivabalan, S.,
Mirsky, G., and G. Fioccola, "RFC6374 Synonymous Flow
Labels", Work in Progress, draft-ietf-mpls-rfc6374-sfl-01,
December 2017.
Acknowledgements
The previous IETF specifications describing this technique were:
[IP-MULTICAST-PM] and [OPSAWG-P3M].
The authors would like to thank Alberto Tempia Bonda, Domenico
Laforgia, Daniele Accetta, and Mario Bianchetti for their
contribution to the definition and the implementation of the method.
The authors would also thank Spencer Dawkins, Carlos Pignataro, Brian
Haberman, and Eric Vyncke for their assistance and their detailed and
precious reviews.
Authors' Addresses Authors' Addresses
Giuseppe Fioccola (editor) Giuseppe Fioccola (editor)
Telecom Italia Telecom Italia
Via Reiss Romoli, 274 Via Reiss Romoli, 274
Torino 10148 Torino 10148
Italy Italy
Email: giuseppe.fioccola@telecomitalia.it Email: giuseppe.fioccola@telecomitalia.it
Alessandro Capello Alessandro Capello
Telecom Italia Telecom Italia
Via Reiss Romoli, 274 Via Reiss Romoli, 274
Torino 10148 Torino 10148
Italy Italy
Email: alessandro.capello@telecomitalia.it Email: alessandro.capello@telecomitalia.it
Mauro Cociglio Mauro Cociglio
Telecom Italia Telecom Italia
skipping to change at page 32, line 40 skipping to change at page 33, line 24
Email: mach.chen@huawei.com Email: mach.chen@huawei.com
Lianshu Zheng Lianshu Zheng
Huawei Technologies Huawei Technologies
Email: vero.zheng@huawei.com Email: vero.zheng@huawei.com
Greg Mirsky Greg Mirsky
ZTE ZTE
USA United States of America
Email: gregimirsky@gmail.com Email: gregimirsky@gmail.com
Tal Mizrahi Tal Mizrahi
Marvell Marvell
6 Hamada st. 6 Hamada St.
Yokneam Yokneam
Israel Israel
Email: talmi@marvell.com Email: talmi@marvell.com
 End of changes. 235 change blocks. 
682 lines changed or deleted 689 lines changed or added

This html diff was produced by rfcdiff 1.46. The latest version is available from http://tools.ietf.org/tools/rfcdiff/