draft-ietf-detnet-bounded-latency-01.txt   draft-ietf-detnet-bounded-latency-02.txt 
DetNet N. Finn DetNet N. Finn
Internet-Draft Huawei Technologies Co. Ltd Internet-Draft Huawei Technologies Co. Ltd
Intended status: Informational J-Y. Le Boudec Intended status: Informational J-Y. Le Boudec
Expires: May 7, 2020 E. Mohammadpour Expires: May 6, 2021 E. Mohammadpour
EPFL EPFL
J. Zhang J. Zhang
Huawei Technologies Co. Ltd Huawei Technologies Co. Ltd
B. Varga B. Varga
J. Farkas J. Farkas
Ericsson Ericsson
November 4, 2019 November 2, 2020
DetNet Bounded Latency DetNet Bounded Latency
draft-ietf-detnet-bounded-latency-01 draft-ietf-detnet-bounded-latency-02
Abstract Abstract
This document presents a timing model for Deterministic Networking This document presents a timing model for Deterministic Networking
(DetNet), so that existing and future standards can achieve the (DetNet), so that existing and future standards can achieve the
DetNet quality of service features of bounded latency and zero DetNet quality of service features of bounded latency and zero
congestion loss. It defines requirements for resource reservation congestion loss. It defines requirements for resource reservation
protocols or servers. It calls out queuing mechanisms, defined in protocols or servers. It calls out queuing mechanisms, defined in
other documents, that can provide the DetNet quality of service. other documents, that can provide the DetNet quality of service.
skipping to change at page 1, line 42 skipping to change at page 1, line 42
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on May 7, 2020. This Internet-Draft will expire on May 6, 2021.
Copyright Notice Copyright Notice
Copyright (c) 2019 IETF Trust and the persons identified as the Copyright (c) 2020 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Terminology and Definitions . . . . . . . . . . . . . . . . . 3 2. Terminology and Definitions . . . . . . . . . . . . . . . . . 3
3. DetNet bounded latency model . . . . . . . . . . . . . . . . 4 3. DetNet bounded latency model . . . . . . . . . . . . . . . . 3
3.1. Flow creation . . . . . . . . . . . . . . . . . . . . . . 4 3.1. Flow creation . . . . . . . . . . . . . . . . . . . . . . 4
3.1.1. Static flow latency calculation . . . . . . . . . . . 4 3.1.1. Static flow latency calculation . . . . . . . . . . . 4
3.1.2. Dynamic flow latency calculation . . . . . . . . . . 5 3.1.2. Dynamic flow latency calculation . . . . . . . . . . 5
3.2. Relay node model . . . . . . . . . . . . . . . . . . . . 6 3.2. Relay node model . . . . . . . . . . . . . . . . . . . . 6
4. Computing End-to-end Delay Bounds . . . . . . . . . . . . . . 8 4. Computing End-to-end Delay Bounds . . . . . . . . . . . . . . 8
4.1. Non-queuing delay bound . . . . . . . . . . . . . . . . . 8 4.1. Non-queuing delay bound . . . . . . . . . . . . . . . . . 8
4.2. Queuing delay bound . . . . . . . . . . . . . . . . . . . 9 4.2. Queuing delay bound . . . . . . . . . . . . . . . . . . . 8
4.2.1. Per-flow queuing mechanisms . . . . . . . . . . . . . 9 4.2.1. Per-flow queuing mechanisms . . . . . . . . . . . . . 9
4.2.2. Per-class queuing mechanisms . . . . . . . . . . . . 9 4.2.2. Per-class queuing mechanisms . . . . . . . . . . . . 9
4.3. Ingress considerations . . . . . . . . . . . . . . . . . 10 4.3. Ingress considerations . . . . . . . . . . . . . . . . . 10
4.4. Interspersed non-DetNet transit nodes . . . . . . . . . . 11 4.4. Interspersed non-DetNet transit nodes . . . . . . . . . . 11
5. Achieving zero congestion loss . . . . . . . . . . . . . . . 11 5. Achieving zero congestion loss . . . . . . . . . . . . . . . 11
6. Queuing techniques . . . . . . . . . . . . . . . . . . . . . 13 6. Queuing techniques . . . . . . . . . . . . . . . . . . . . . 12
6.1. Queuing data model . . . . . . . . . . . . . . . . . . . 13 6.1. Queuing data model . . . . . . . . . . . . . . . . . . . 12
6.2. Preemption . . . . . . . . . . . . . . . . . . . . . . . 15 6.2. Preemption . . . . . . . . . . . . . . . . . . . . . . . 14
6.3. Time-scheduled queuing . . . . . . . . . . . . . . . . . 15 6.3. Time Aware Shaper . . . . . . . . . . . . . . . . . . . . 15
6.4. Credit-Based Shaper with Asynchronous Traffic Shaping . . 16 6.4. Credit-Based Shaper with Asynchronous Traffic Shaping . . 15
6.4.1. Delay Bound Calculation . . . . . . . . . . . . . . . 18 6.4.1. Delay Bound Calculation . . . . . . . . . . . . . . . 17
6.4.2. Flow Admission . . . . . . . . . . . . . . . . . . . 19 6.4.2. Flow Admission . . . . . . . . . . . . . . . . . . . 18
6.5. IntServ . . . . . . . . . . . . . . . . . . . . . . . . . 20 6.5. IntServ . . . . . . . . . . . . . . . . . . . . . . . . . 19
6.6. Cyclic Queuing and Forwarding . . . . . . . . . . . . . . 23 6.6. Cyclic Queuing and Forwarding . . . . . . . . . . . . . . 20
6.6.1. CQF timing sequence . . . . . . . . . . . . . . . . . 24 7. References . . . . . . . . . . . . . . . . . . . . . . . . . 21
6.6.2. CQF latency calculation . . . . . . . . . . . . . . . 24 7.1. Normative References . . . . . . . . . . . . . . . . . . 21
7. References . . . . . . . . . . . . . . . . . . . . . . . . . 25 7.2. Informative References . . . . . . . . . . . . . . . . . 22
7.1. Normative References . . . . . . . . . . . . . . . . . . 25 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 23
7.2. Informative References . . . . . . . . . . . . . . . . . 26
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 27
1. Introduction 1. Introduction
The ability for IETF Deterministic Networking (DetNet) or IEEE 802.1 The ability for IETF Deterministic Networking (DetNet) or IEEE 802.1
Time-Sensitive Networking (TSN, [IEEE8021TSN]) to provide the DetNet Time-Sensitive Networking (TSN, [IEEE8021TSN]) to provide the DetNet
services of bounded latency and zero congestion loss depends upon A) services of bounded latency and zero congestion loss depends upon A)
configuring and allocating network resources for the exclusive use of configuring and allocating network resources for the exclusive use of
DetNet/TSN flows; B) identifying, in the data plane, the resources to DetNet/TSN flows; B) identifying, in the data plane, the resources to
be utilized by any given packet, and C) the detailed behavior of be utilized by any given packet, and C) the detailed behavior of
those resources, especially transmission queue selection, so that those resources, especially transmission queue selection, so that
latency bounds can be reliably assured. Thus, DetNet is an example latency bounds can be reliably assured. Thus, DetNet is an example
of an IntServ Guaranteed Quality of Service [RFC2212] of an IntServ Guaranteed Quality of Service [RFC2212]
As explained in [I-D.ietf-detnet-architecture], DetNet flows are As explained in [RFC8655], DetNet flows are characterized by 1) a
characterized by 1) a maximum bandwidth, guaranteed either by the maximum bandwidth, guaranteed either by the transmitter or by strict
transmitter or by strict input metering; and 2) a requirement for a input metering; and 2) a requirement for a guaranteed worst-case end-
guaranteed worst-case end-to-end latency. That latency guarantee, in to-end latency. That latency guarantee, in turn, provides the
turn, provides the opportunity for the network to supply enough opportunity for the network to supply enough buffer space to
buffer space to guarantee zero congestion loss. guarantee zero congestion loss.
To be of use to the applications identified in [RFC8578], it must be To be of use to the applications identified in [RFC8578], it must be
possible to calculate, before the transmission of a DetNet flow possible to calculate, before the transmission of a DetNet flow
commences, both the worst-case end-to-end network latency, and the commences, both the worst-case end-to-end network latency, and the
amount of buffer space required at each hop to ensure against amount of buffer space required at each hop to ensure against
congestion loss. congestion loss.
This document references specific queuing mechanisms, defined in This document references specific queuing mechanisms, defined in
other documents, that can be used to control packet transmission at other documents, that can be used to control packet transmission at
each output port and achieve the DetNet qualities of service. This each output port and achieve the DetNet qualities of service. This
skipping to change at page 3, line 46 skipping to change at page 3, line 44
DetNet service. DetNet service.
This document does not specify any resource reservation protocol or This document does not specify any resource reservation protocol or
server. It does not describe all of the requirements for that server. It does not describe all of the requirements for that
protocol or server. It does describe requirements for such resource protocol or server. It does describe requirements for such resource
reservation methods, and for queuing mechanisms that, if met, will reservation methods, and for queuing mechanisms that, if met, will
enable them to work together. enable them to work together.
2. Terminology and Definitions 2. Terminology and Definitions
This document uses the terms defined in This document uses the terms defined in [RFC8655].
[I-D.ietf-detnet-architecture].
3. DetNet bounded latency model 3. DetNet bounded latency model
3.1. Flow creation 3.1. Flow creation
This document assumes that following paradigm is used for This document assumes that following paradigm is used for
provisioning DetNet flows: provisioning DetNet flows:
1. Perform any configuration required by the DetNet transit nodes in 1. Perform any configuration required by the DetNet transit nodes in
the network for the classes of service to be offered, including the network for the classes of service to be offered, including
one or more classes of DetNet service. This configuration is one or more classes of DetNet service. This configuration is
done beforehand, and not tied to any particular flow. done beforehand, and not tied to any particular flow.
skipping to change at page 5, line 7 skipping to change at page 5, line 7
3.1.1. Static flow latency calculation 3.1.1. Static flow latency calculation
The static problem: The static problem:
Given a network and a set of DetNet flows, compute an end-to- Given a network and a set of DetNet flows, compute an end-to-
end latency bound (if computable) for each flow, and compute end latency bound (if computable) for each flow, and compute
the resources, particularly buffer space, required in each the resources, particularly buffer space, required in each
DetNet transit node to achieve zero congestion loss. DetNet transit node to achieve zero congestion loss.
In this calculation, all of the DetNet flows are known before the In this calculation, all of the DetNet flows are known before the
calculation commences. This problem is of interest to relatively calculation commences. This problem is of interest to relatively
static networks, or static parts of larger networks. It gives the static networks, or static parts of larger networks. It provides
best possible worst-case behavior. The calculations can be extended bounds on delay and buffer size. The calculations can be extended to
to provide global optimizations, such as altering the path of one provide global optimizations, such as altering the path of one DetNet
DetNet flow in order to make resources available to another DetNet flow in order to make resources available to another DetNet flow with
flow with tighter constraints. tighter constraints.
The static flow calculation is not limited only to static networks; The static flow calculation is not limited only to static networks;
the entire calculation for all flows can be repeated each time a new the entire calculation for all flows can be repeated each time a new
DetNet flow is created or deleted. If some already-established flow DetNet flow is created or deleted. If some already-established flow
would be pushed beyond its latency requirements by the new flow, then would be pushed beyond its latency requirements by the new flow, then
the new flow can be refused, or some other suitable action taken. the new flow can be refused, or some other suitable action taken.
This calculation may be more difficult to perform than that of the This calculation may be more difficult to perform than that of the
dynamic calculation (Section 3.1.2), because the flows passing dynamic calculation (Section 3.1.2), because the flows passing
through one port on a DetNet transit node affect each others' through one port on a DetNet transit node affect each others'
skipping to change at page 5, line 47 skipping to change at page 5, line 47
particularly buffer space, required in each DetNet transit particularly buffer space, required in each DetNet transit
node to achieve zero congestion loss. node to achieve zero congestion loss.
This calculation is dynamic, in the sense that flows can be added or This calculation is dynamic, in the sense that flows can be added or
deleted at any time, with a minimum of computation effort, and deleted at any time, with a minimum of computation effort, and
without affecting the guarantees already given to other flows. without affecting the guarantees already given to other flows.
The choice of queuing methods is critical to the applicability of the The choice of queuing methods is critical to the applicability of the
dynamic calculation. Some queuing methods (e.g. CQF, Section 6.6) dynamic calculation. Some queuing methods (e.g. CQF, Section 6.6)
make it easy to configure bounds on the network's capacity, and to make it easy to configure bounds on the network's capacity, and to
make independent calculations for each flow. [[E:The rest of this make independent calculations for each flow. Some other queuing
paragraph should be changed.]] Other queuing methods (e.g., methods (e.g. strict priority with the credit-based shaper defined in
transmission selection by strict priority), make this calculation [IEEE8021Q] section 8.6.8.2) can be used for dynamic flow creation,
impossible, because the worst case for one flow cannot be computed but yield poorer latency and buffer space guarantees than when that
without complete knowledge of all other flows. Other queuing methods same queuing method is used for static flow creation (Section 3.1.1).
(e.g. the credit-based shaper defined in [IEEE8021Q] section 8.6.8.2)
can be used for dynamic flow creation, but yield poorer latency and
buffer space guarantees than when that same queuing method is used
for static flow creation (Section 3.1.1).
[[E:proposed replacement: Some other queuing methods (e.g. strict
priority with the credit-based shaper defined in [IEEE8021Q] section
8.6.8.2) can be used for dynamic flow creation, but yield poorer
latency and buffer space guarantees than when that same queuing
method is used for static flow creation (Section 3.1.1).]]
3.2. Relay node model 3.2. Relay node model
A model for the operation of a DetNet transit node is required, in A model for the operation of a DetNet transit node is required, in
order to define the latency and buffer calculations. In Figure 1 we order to define the latency and buffer calculations. In Figure 1 we
see a breakdown of the per-hop latency experienced by a packet see a breakdown of the per-hop latency experienced by a packet
passing through a DetNet transit node, in terms that are suitable for passing through a DetNet transit node, in terms that are suitable for
computing both hop-by-hop latency and per-hop buffer requirements. computing both hop-by-hop latency and per-hop buffer requirements.
DetNet transit node A DetNet transit node B DetNet transit node A DetNet transit node B
skipping to change at page 7, line 16 skipping to change at page 7, line 6
multiplexed connection. This causes variations in the output multiplexed connection. This causes variations in the output
delay that are hard for the forwarding node to predict or control. delay that are hard for the forwarding node to predict or control.
2. Link delay 2. Link delay
The time taken from the transmission of the first bit of the The time taken from the transmission of the first bit of the
packet to the reception of the last bit, assuming that the packet to the reception of the last bit, assuming that the
transmission is not suspended by a preemption event. This delay transmission is not suspended by a preemption event. This delay
has two components, the first-bit-out to first-bit-in delay and has two components, the first-bit-out to first-bit-in delay and
the first-bit-in to last-bit-in delay that varies with packet the first-bit-in to last-bit-in delay that varies with packet
size. The former is typically measured by the Precision Time size. The former is typically measured by the Precision Time
Protocol and is constant (see [I-D.ietf-detnet-architecture]). Protocol and is constant (see [RFC8655]). However, a virtual
However, a virtual "link" could exhibit a variable link delay. "link" could exhibit a variable link delay.
3. Preemption delay 3. Preemption delay
If the packet is interrupted in order to transmit another packet If the packet is interrupted in order to transmit another packet
or packets, (e.g. [IEEE8023] clause 99 frame preemption) an or packets, (e.g. [IEEE8023] clause 99 frame preemption) an
arbitrary delay can result. arbitrary delay can result.
4. Processing delay 4. Processing delay
This delay covers the time from the reception of the last bit of This delay covers the time from the reception of the last bit of
the packet to the time the packet is enqueued in the regulator the packet to the time the packet is enqueued in the regulator
(Queuing subsystem, if there is no regulation). This delay can be (Queuing subsystem, if there is no regulation). This delay can be
skipping to change at page 12, line 44 skipping to change at page 12, line 33
class that may be sent to this output port. This is counted in class that may be sent to this output port. This is counted in
data units. data units.
o max_delay456 be an upper bound, in seconds, on the sum of the o max_delay456 be an upper bound, in seconds, on the sum of the
processing delay (4) and the queuing delays (5,6) for a packet of processing delay (4) and the queuing delays (5,6) for a packet of
any class at this output port. any class at this output port.
Then a bound on the backlog of traffic of all classes in the queue at Then a bound on the backlog of traffic of all classes in the queue at
this output port is this output port is
[[E: The formula is not right; why do we need nb_classes to compute
backlog bound?]]
backlog_bound = ( nb_classes + nb_input_ports ) *
max_packet_length + total_in_rate* max_delay456
[[E: proposed general backlog bound:]]
backlog_bound = nb_input_ports * max_packet_length + backlog_bound = nb_input_ports * max_packet_length +
total_in_rate* max_delay456 total_in_rate* max_delay456
6. Queuing techniques 6. Queuing techniques
In this section, for simplicity of delay computation, we assume that
the T-SPEC or arrival curve [NetCalBook] for each flow at source is
leaky bucket. Also, at each relay node, the service for each queue
is abstracted with a guaranteed rate and a latency.
6.1. Queuing data model 6.1. Queuing data model
Sophisticated queuing mechanisms are available in Layer 3 (L3, see, Sophisticated queuing mechanisms are available in Layer 3 (L3, see,
e.g., [RFC7806] for an overview). In general, we assume that "Layer e.g., [RFC7806] for an overview). In general, we assume that "Layer
3" queues, shapers, meters, etc., are precisely the "regulators" 3" queues, shapers, meters, etc., are precisely the "regulators"
shown in Figure 1. The "queuing subsystems" in this figure are not shown in Figure 1. The "queuing subsystems" in this figure are not
the province solely of bridges; they are an essential part of any the province solely of bridges; they are an essential part of any
DetNet transit node. As illustrated by numerous implementation DetNet transit node. As illustrated by numerous implementation
examples, some of the "Layer 3" mechanisms described in documents examples, some of the "Layer 3" mechanisms described in documents
such as [RFC7806] are often integrated, in an implementation, with such as [RFC7806] are often integrated, in an implementation, with
skipping to change at page 15, line 35 skipping to change at page 14, line 48
6.2. Preemption 6.2. Preemption
In [IEEE8021Q] and [IEEE8023], the transmission of a frame can be In [IEEE8021Q] and [IEEE8023], the transmission of a frame can be
interrupted by one or more "express" frames, and then the interrupted interrupted by one or more "express" frames, and then the interrupted
frame can continue transmission. This frame preemption is modeled as frame can continue transmission. This frame preemption is modeled as
consisting of two MAC/PHY stacks, one for packets that can be consisting of two MAC/PHY stacks, one for packets that can be
interrupted, and one for packets that can interrupt the interruptible interrupted, and one for packets that can interrupt the interruptible
packets. The Class of Service (queue) determines which packets are packets. The Class of Service (queue) determines which packets are
which. Only one layer of preemption is supported -- a transmitter which. Only one layer of preemption is supported -- a transmitter
cannot have more than one interrupted frame in progress. DetNet cannot have more than one interrupted frame in progress. DetNet
flows typically pass through the interrupting MAC. Best-effort flows typically pass through the interrupting MAC. For those DetNet
queues pass through the interruptible MAC, and can thus be preempted. flows with T-SPEC, latency bound can be calculated by the methods
provided in the following sections that accounts for the affect of
preemption, according to the specific queuing mechanism that is used
in DetNet nodes. Best-effort queues pass through the interruptible
MAC, and can thus be preempted.
6.3. Time-scheduled queuing 6.3. Time Aware Shaper
In [IEEE8021Q], the notion of time-scheduling queue gates is In [IEEE8021Q], the notion of time-scheduling queue gates is
described in section 8.6.8.4. Below every output queue (the lower described in section 8.6.8.4. On each node, the transmission
row of queues in Figure 3) is a gate that permits or denies the queue selection for packets is controlled by time-synchronized gates; each
to present data for transmission selection. The gates are controlled output queue is associated with a gate. The gates can be either open
by a rotating schedule that can be locked to a clock that is or close. The states of the gates are determined by the gate control
synchronized with other DetNet transit nodes. The DetNet class of list (GCL). The GCL specifies the opening and closing times of the
service can be supplied by queuing mechanisms based on time, rather gates. Since the design of GCL should satisfy the requirement of
than the regulator model in Figure 3. Generally speaking, this time- latency upper bounds of all time-sensitive flows, those flows travers
aware scheduling can be used as a layer 2 time division multiplexing a network should have bounded latency, if the traffic and nodes are
(TDM) technique. conformant.
Consider the static configuration of a deterministic network. To
provide end-to-end latency guaranteed service, network nodes can
support time-based behavior, which is determined by gate control list
(GCL). GCL defines the gate operation, in open or closed state, with
associated timing for each traffic class queue. A time slice with
gate state "open" is called transmission window. The time-based
traffic scheduling must be coordinated among the DetNet transit nodes
along the path from sender to receiver, to control the transmission
of time-sensitive traffic.
Ideally all network devices are time synchronized and static GCL
configurations on all devices along the routed path are coordinated
to ensure that length of transmission window fits the assigned
frames, and no two time windows for DetNet traffic on the same port
overlap. (DetNet flows' windows can overlap with best-effort
windows, so that unused DetNet bandwidth is available to best-effort
traffic.) The processing delay, link delay and output delay in
transmitting are considered in GCL computation. Transmission window
for a certain flow may require that a time offset on consecutive hops
be selected to reduce queueing delay as much as possible. In this
case, TSN/DetNet frames transmit at the assigned transmission window
at every node through the routed path, with zero congestion loss and
bounded end-to-end latency. Then, the worst-case end-to-end latency
of the flow can be derived from GCL configuration. For a TSN or
DetNet frame, denote the transmission window on last hop closes at
gate_close_time_last_hop. Assuming talker supports scheduled traffic
behavior, it starts the transmission at gate_open_time_on_talker.
Then worst case end-to-end delay of this flow is bounded by
gate_close_time_last_hop - gate_open_time_on_talker +
link_delay_last_hop.
It should be noted that scheduled traffic service relies on a It should be noted that scheduled traffic service relies on a
synchronized network and coordinated GCL configuration. Synthesis of synchronized network and coordinated GCL configuration. Synthesis of
GCL on multiple nodes in network is a scheduling problem considering GCL on multiple nodes in network is a scheduling problem considering
all TSN/DetNet flows traversing the network, which is a non- all TSN/DetNet flows traversing the network, which is a non-
deterministic polynomial-time hard (NP-hard) problem. Also, at this deterministic polynomial-time hard (NP-hard) problem. Also, at this
writing, scheduled traffic service supports no more than eight writing, scheduled traffic service supports no more than eight
traffic classes, typically using up to seven priority classes and at traffic classes, typically using up to seven priority classes and at
least one best effort class. least one best effort class.
skipping to change at page 20, line 41 skipping to change at page 19, line 32
The choice of the static values of R and b_t at all nodes and classes The choice of the static values of R and b_t at all nodes and classes
must be done in a prior configuration phase; R controls the bandwidth must be done in a prior configuration phase; R controls the bandwidth
allocated to this class at this node, b_t affects the delay bound and allocated to this class at this node, b_t affects the delay bound and
the buffer requirement. R must satisfy the constraints given in the buffer requirement. R must satisfy the constraints given in
Annex L.1 of [IEEE8021Q]. Annex L.1 of [IEEE8021Q].
6.5. IntServ 6.5. IntServ
Integrated service (IntServ) is an architecture that specifies the Integrated service (IntServ) is an architecture that specifies the
elements to guarantee quality of service (QoS) on networks. [[E: The elements to guarantee quality of service (QoS) on networks.
rest of this paragraph is better not to be placed here; these should
be mentioned (is mentioned) in the introduction.]] To satisfied
guaranteed service, a flow must conform to a traffic specification
(T-spec), and reservation is made along a path, only if routers are
able to guarantee the required bandwidth and buffer.
[[E: The information about arrival and service curves can be shorter
with less detail. I put a proposed text after description of
these.]]
Consider the traffic model which conforms to token bucket regulator
(r, b), with
o Token bucket depth (b).
o Token bucket rate (r).
The traffic specification can be described as an arrival curve:
alpha(t) = b + rt
This token bucket regulator requires that, during any time window t,
the number of bit for the flow is limited by alpha(t) = b + rt.
If resource reservation on a path is applied, IntServ model of a
router can be described as a rate-latency service curve beta(t).
beta(t) = max(0, R(t-T))
It describes that bits might have to wait up to T before being served
with a rate greater or equal to R.
[[E: proposed text:
The flow, at the source, has a leaky bucket arrival curve with two The flow, at the source, has a leaky bucket arrival curve with two
parameters r as rate and b as bucket size, i.e., the amount of bits parameters r as rate and b as bucket size, i.e., the amount of bits
entering a node within a time interval t is bounded by r t + b. entering a node within a time interval t is bounded by r t + b.
If a resource reservation on a path is applied, a node provides a If a resource reservation on a path is applied, a node provides a
guaranteed rate R and maximum service latency of T. This can be guaranteed rate R and maximum service latency of T. This can be
interpreted in a way that the bits might have to wait up to T before interpreted in a way that the bits might have to wait up to T before
being served with a rate greater or equal to R. ]] being served with a rate greater or equal to R. The delay bound of
the flow traversing the node is T + b / R.
It should be noted that the guaranteed service rate R is a portion of
link's bandwidth. The selection of R is related to the specification
of flows traversing through the current node. For example, in strict
priority policy, considering a flow with priority i, its guaranteed
rate is R=c-sum(r_j), j<i, where c is the link bandwidth, r_j is the
token bucket rate for a flow j with priority higher than flow i. The
choice of T is also related to the specification of all the flows
traversing this node. For example, in a generalized processor
sharing (GPS) node, T = L / R + L_max/c, where L is the maximum
packet size for the flow, L_max is the maximum packet size in the
node across all flows. Other choice of R and T are also supported,
according to the specific scheduling of the node and flows traversing
this node.
As mentioned previously in this section, a delay bound and a buffer
size bound can be easily obtained by comparing arrival curve and
service curve. Backlog bound, or buffer bound, is the maximum
vertical derivation between curves alpha(t) and beta(t), which is
v=b+rT. Delay bound is the maximum horizontal derivation between
curves alpha(t) and beta(t), which is h = T+b/R. Graphical
illustration of the IntServ model is shown in Figure 5.
+ bit . *
| . *
| . *
| *
| * .
| * .
| * | . .. Service curve
*-----h-|---. ** Arrival curve
| v . h Delay_bound
| | . v Backlog_bound
| |.
+-------.--------------------+ time
Figure 5: Computation of backlog bound and delay bound. Note that
arrival and service curves are not necessary to be linear.
The output bound, or the next-hop arrival curve, is alpha_out(t) = b
+ rT + rt, where burstiness of the flow is increased by rT, compared
with the arrival curve.
We can calculate the end-to-end delay bound for a path including N
nodes, among which the i-th node offers service curve beta_i(t),
beta_i(t) = max(0, R_i(t-T_i)), i=1,...,N
By concatenating these IntServ nodes, an end-to-end service curve can
be computed as
beta_e2e (t) = max(0, R_e2e(t-T_e2e) )
where
R_e2e = min(R_1,..., R_N)
T_e2e = T_1 + ... + T_N Consider an IntServ path including a sequence of nodes, where the
i-th node provides a guaranteed rate R_i and maximum service latency
of T_i. Then, the end-to-end delay bound for a flow on this can be
calculated as sum(T_i) + b / min(R_i).
Similarly, delay bound, backlog bound and output bound can be If more information about the flow is known, e.g. the peak rate, the
computed by using the original arrival curve alpha(t) and delay bound is more complicated; the detail is available in
concatenated service curve beta_e2e(t). Section 1.4.1 of [NetCalBook].
6.6. Cyclic Queuing and Forwarding 6.6. Cyclic Queuing and Forwarding
Annex T of [IEEE8021Q] describes Cyclic Queuing and Forwarding (CQF), Annex T of [IEEE8021Q] describes Cyclic Queuing and Forwarding (CQF),
which provides bounded latency and zero congestion loss using the which provides bounded latency and zero congestion loss using the
time-scheduled gates of [IEEE8021Q] section 8.6.8.4. For a given time-scheduled gates of [IEEE8021Q] section 8.6.8.4. For a given
DetNet class of service, a set of two or three buffers is provided at DetNet class of service, a set of two or more buffers is provided at
the output queue layer of Figure 3. A cycle time T_c is configured the output queue layer of Figure 3. A cycle time T_c is configured
for each class c, and all of the buffer sets in a class swap buffers for each class c, and all of the buffer sets in a class swap buffers
simultaneously throughout the DetNet domain at that cycle rate, all simultaneously throughout the DetNet domain at that cycle rate, all
in phase. in phase. In such a mechanism, the regulator, mentioned in Figure 1,
is not required.
0 time --> 0.7 1 (units of T_c) 2 3
DetNet transit node A out port 1
| a <-DT->| b | c | d
+------------+------+-------------------+-------------------+--------
\_____ \_____
\_____ \_____ queue-to-queue delay = 1.3 T_c
\_____ \_____
\_____ \_____ DetNet transit node B
\_ \_ queue assignment, in
| | |<-DT->| port 2 to out 3 |
-------+-------------------+------------+------+-------------------+-
0.3 time--> 1.3 2.0 2.3 3.3
window to transfer
to buffer c ---> VVVVVVVVVVVV
if dead time not window to transfer
excessive VVVVVVVVVVVVVVVVVVV <--- to buffer d
DetNet transit node B out port 3
| a | b | c | d
+-------------------+-------------------+-------------------+--------
0 time--> 1 2 3
Figure 6: CQF timing diagram
Figure 6 shows two DetNet transit nodes A and B, including three
timelines for:
1. The output queues on port 1 in node A.
2. The input gate function ([IEEE8021Q], 8.6.5.1) that assigns
packets received on port 1 of transit node B to output queues on
port 2 of transit node B.
3. The output queues on port 2 of node B.
In this figure, the output ports on the two nodes are synchronized, In the case of two-buffer CQF, each class c has two buffers, namely
and a new buffer starts transmitting at each tick, shown as 0, 1, 2, buffer1 and buffer2. In a cycle (i) when buffer1 accumulates
... The output times shown for timelines 1 and 3 are the times at received packets from the node's reception ports, buffer2 transmits
which packets are selected for output, which is the start point of the already stored packets from the previous cycle (i-1). In the
the output time (1) of Figure 1. The queue assignments times on next cycle (i+1), buffer2 stores the received packets and buffer1
timeline 3 take place at the beginning of the queuing delay (6) of transmits the packets received in cycle (i). The duration of each
Figure 1. Time-based CQF, as described here, does not require any cycle is T_c.
regulator queues. In the shown in the figure, the total time [[E:
what is meant by total time? Does it mean a delay bound is 1.3
T_C?]] for delays (1) through (6) of Figure 1, is 1.3T_c. Of course,
any value is possible.
6.6.1. CQF timing sequence The per-hop latency is trivially determined by the cycle time T_c:
the packet transmitted from a node at a cycle (i), is transmitted
from the next node at cycle (i+1). Hence, the maximum delay
experienced by a given packet is from the beginning of cycle (i) to
the end of cycle (i+1), or 2T_c; also, the minimum delay is from the
end of cycle (i) to the beginning of cycle (i+1), i.e., zero. Then,
if the packet traverses h hops, the maximum delay is:
In general, as shown in Figure 6, the windows for buffer assignment (h+1) T_c
do not align perfectly with the windows for buffer transmission. The
input gates (the center timeline in Figure 6) must switch from using
one buffer to using another buffer in sync with the (delayed)
received data, at times offset by the dead time from the output
buffer switching (the bottom timeline in Figure 6).
If the dead time DT in Figure 6 is not excessive, then it is feasible and the minimum delay is:
to subtract the dead time from the cycle time Tc, and use the
remainder as the input window. In the example in Figure 6, packets
from node A buffer a can be transferred from the input port to node
B's buffer "c" during the window shown by the upper row "VVVV...".
Input must cease by time = 2.0, because that is when transit node B
starts transmitting the contents of buffer c. In this case, only two
output buffers are in use, one filling and one outputting.
If the dead time is too large (e.g., if the delays placed the middle (h-1) T_c
timeline's switching points at n+0.9, instead of n+0.3), three
buffers are used by node B. This case is shown by the lower row
"VVVV..." in Figure 6. In this case, node B places the data received
from node A buffer a into node B buffer d between the times 1.3 and
2.3 in Figure 6. Buffer b starts outputting at time = 2.0, while
buffer d is filling. Thus, three buffers are in use, one filling,
one waiting, and one emptying.
6.6.2. CQF latency calculation which gives a latency variation of 2T_c.
The per-hop latency is trivially determined by the wire delay and the The cycle length T_c should be carefully chosen; it needs to be large
queuing delay. Since the wire delay is either absorbed into the enough to accomodate all the DetNet traffic, plus at least one
queueing delay (dead time is small and two buffers are used) or maximum interfering packet, that can be received within one cycle.
padded out to a whole cycle time T_c (three buffers are used) the Also, the value of T_c includes a time interval, called dead time
per-hop latency is always an integral number of cycle times T_c, with (DT), which is the sum of the delays 1,2,3,4 defined in Figure 1.
a latency variation at the output of the final hop of T_c. The value of DT guarantees that the last packet of one cycle in a
node is fully delivered to a buffer of the next node is the same
cycle. A two-buffer CQF is recommended if DT is small compared to
T_c. For a large DT, CQF with more buffers can be used.
Ingress conditioning (Section 4.3) may be required if the source of a Ingress conditioning (Section 4.3) may be required if the source of a
DetNet flow does not, itself, employ CQF. DetNet flow does not, itself, employ CQF. Since there are no per-
flow parameters in the CQF technique, per-hop configuration is not
Note that there are no per-flow parameters in the CQF technique. required in the CQF forwarding nodes.
Therefore, there is no requirement for per-hop configuration when a
new DetNet flow is added to a network, except perhaps for ingress
checks to see that the transmitter does not exceed the contracted
bandwidth.
7. References 7. References
7.1. Normative References 7.1. Normative References
[I-D.ietf-detnet-architecture]
Finn, N., Thubert, P., Varga, B., and J. Farkas,
"Deterministic Networking Architecture", draft-ietf-
detnet-architecture-08 (work in progress), September 2018.
[I-D.ietf-detnet-ip] [I-D.ietf-detnet-ip]
Varga, B., Farkas, J., Berger, L., Fedyk, D., Malis, A., Varga, B., Farkas, J., Berger, L., Fedyk, D., Malis, A.,
Bryant, S., and J. Korhonen, "DetNet Data Plane: IP", and S. Bryant, "DetNet Data Plane: IP", draft-ietf-detnet-
draft-ietf-detnet-ip-00 (work in progress), May 2019. ip-05 (work in progress), February 2020.
[I-D.ietf-detnet-mpls] [I-D.ietf-detnet-mpls]
Varga, B., Farkas, J., Berger, L., Fedyk, D., Malis, A., Varga, B., Farkas, J., Berger, L., Fedyk, D., Malis, A.,
Bryant, S., and J. Korhonen, "DetNet Data Plane: MPLS", Bryant, S., and J. Korhonen, "DetNet Data Plane: MPLS",
draft-ietf-detnet-mpls-00 (work in progress), May 2019. draft-ietf-detnet-mpls-05 (work in progress), February
2020.
[RFC2212] Shenker, S., Partridge, C., and R. Guerin, "Specification [RFC2212] Shenker, S., Partridge, C., and R. Guerin, "Specification
of Guaranteed Quality of Service", RFC 2212, of Guaranteed Quality of Service", RFC 2212,
DOI 10.17487/RFC2212, September 1997, DOI 10.17487/RFC2212, September 1997,
<https://www.rfc-editor.org/info/rfc2212>. <https://www.rfc-editor.org/info/rfc2212>.
[RFC6658] Bryant, S., Ed., Martini, L., Swallow, G., and A. Malis, [RFC6658] Bryant, S., Ed., Martini, L., Swallow, G., and A. Malis,
"Packet Pseudowire Encapsulation over an MPLS PSN", "Packet Pseudowire Encapsulation over an MPLS PSN",
RFC 6658, DOI 10.17487/RFC6658, July 2012, RFC 6658, DOI 10.17487/RFC6658, July 2012,
<https://www.rfc-editor.org/info/rfc6658>. <https://www.rfc-editor.org/info/rfc6658>.
[RFC7806] Baker, F. and R. Pan, "On Queuing, Marking, and Dropping", [RFC7806] Baker, F. and R. Pan, "On Queuing, Marking, and Dropping",
RFC 7806, DOI 10.17487/RFC7806, April 2016, RFC 7806, DOI 10.17487/RFC7806, April 2016,
<https://www.rfc-editor.org/info/rfc7806>. <https://www.rfc-editor.org/info/rfc7806>.
[RFC8578] Grossman, E., Ed., "Deterministic Networking Use Cases", [RFC8578] Grossman, E., Ed., "Deterministic Networking Use Cases",
RFC 8578, DOI 10.17487/RFC8578, May 2019, RFC 8578, DOI 10.17487/RFC8578, May 2019,
<https://www.rfc-editor.org/info/rfc8578>. <https://www.rfc-editor.org/info/rfc8578>.
[RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas,
"Deterministic Networking Architecture", RFC 8655,
DOI 10.17487/RFC8655, October 2019,
<https://www.rfc-editor.org/info/rfc8655>.
7.2. Informative References 7.2. Informative References
[bennett2002delay] [bennett2002delay]
J.C.R. Bennett, K. Benson, A. Charny, W.F. Courtney, and J.C.R. Bennett, K. Benson, A. Charny, W.F. Courtney, and
J.-Y. Le Boudec, "Delay Jitter Bounds and Packet Scale J.-Y. Le Boudec, "Delay Jitter Bounds and Packet Scale
Rate Guarantee for Expedited Forwarding", Rate Guarantee for Expedited Forwarding",
<https://dl.acm.org/citation.cfm?id=581870>. <https://dl.acm.org/citation.cfm?id=581870>.
[charny2000delay] [charny2000delay]
A. Charny and J.-Y. Le Boudec, "Delay Bounds in a Network A. Charny and J.-Y. Le Boudec, "Delay Bounds in a Network
skipping to change at page 26, line 42 skipping to change at page 22, line 42
Task Group", <http://www.ieee802.org/1/>. Task Group", <http://www.ieee802.org/1/>.
[IEEE8023] [IEEE8023]
IEEE 802.3, "IEEE Std 802.3-2018: IEEE Standard for IEEE 802.3, "IEEE Std 802.3-2018: IEEE Standard for
Ethernet", 2018, Ethernet", 2018,
<http://ieeexplore.ieee.org/document/8457469>. <http://ieeexplore.ieee.org/document/8457469>.
[le_boudec_theory_2018] [le_boudec_theory_2018]
J.-Y. Le Boudec, "A Theory of Traffic Regulators for J.-Y. Le Boudec, "A Theory of Traffic Regulators for
Deterministic Networks with Application to Interleaved Deterministic Networks with Application to Interleaved
Regulators", <http://arxiv.org/abs/1801.08477/>. Regulators",
<https://ieeexplore.ieee.org/document/8519761>.
[NetCalBook] [NetCalBook]
Le Boudec, Jean-Yves, and Patrick Thiran, "Network J.-Y. Le Boudec and P. Thiran, "Network calculus: a theory
calculus: a theory of deterministic queuing systems for of deterministic queuing systems for the internet", 2001,
the internet", 2001, <https://arxiv.org/abs/1804.10608/>. <https://ica1www.epfl.ch/PS_files/NetCal.htm>.
[Specht2016UBS] [Specht2016UBS]
J. Specht and S. Samii, "Urgency-Based Scheduler for Time- J. Specht and S. Samii, "Urgency-Based Scheduler for Time-
Sensitive Switched Ethernet Networks", Sensitive Switched Ethernet Networks",
<https://ieeexplore.ieee.org/abstract/document/7557870>. <https://ieeexplore.ieee.org/abstract/document/7557870>.
[TSNwithATS] [TSNwithATS]
E. Mohammadpour, E. Stai, M. Mohiuddin, and J.-Y. Le E. Mohammadpour, E. Stai, M. Mohiuddin, and J.-Y. Le
Boudec, "End-to-end Latency and Backlog Bounds in Time- Boudec, "End-to-end Latency and Backlog Bounds in Time-
Sensitive Networking with Credit Based Shapers and Sensitive Networking with Credit Based Shapers and
skipping to change at page 27, line 38 skipping to change at page 24, line 4
Email: jean-yves.leboudec@epfl.ch Email: jean-yves.leboudec@epfl.ch
Ehsan Mohammadpour Ehsan Mohammadpour
EPFL EPFL
IC Station 14 IC Station 14
Lausanne EPFL 1015 Lausanne EPFL 1015
Switzerland Switzerland
Email: ehsan.mohammadpour@epfl.ch Email: ehsan.mohammadpour@epfl.ch
Jiayi Zhang Jiayi Zhang
Huawei Technologies Co. Ltd Huawei Technologies Co. Ltd
Q22, No.156 Beiqing Road Q27, No.156 Beiqing Road
Beijing 100095 Beijing 100095
China China
Email: zhangjiayi11@huawei.com Email: zhangjiayi11@huawei.com
Balazs Varga Balazs Varga
Ericsson Ericsson
Konyves Kalman krt. 11/B Konyves Kalman krt. 11/B
Budapest 1097 Budapest 1097
Hungary Hungary
Email: balazs.a.varga@ericsson.com Email: balazs.a.varga@ericsson.com
Janos Farkas Janos Farkas
Ericsson Ericsson
 End of changes. 42 change blocks. 
297 lines changed or deleted 119 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/