DetNet                                                     B. Varga, Ed.
Internet-Draft                                                 J. Farkas
Intended status: Standards Track                                Ericsson
Expires: November 6, 2019 April 29, 2020                                         A. Malis
                                                             Independent
                                                               S. Bryant
                                                     Huawei
                                                  Futurewei Technologies
                                                             J. Korhonen
                                                             May 5,
                                                                D. Fedyk
                                                 LabN Consulting, L.L.C.
                                                        October 27, 2019

   DetNet Data Plane: IEEE 802.1 Time Sensitive Networking over MPLS
                 draft-ietf-detnet-tsn-vpn-over-mpls-00
                 draft-ietf-detnet-tsn-vpn-over-mpls-01

Abstract

   This document specifies the Deterministic Networking data plane when
   TSN networks are interconnected over an a DetNet MPLS Packet Switched Networks. Network.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on November 6, 2019. April 29, 2020.

Copyright Notice

   Copyright (c) 2019 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (https://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Terminology . . . . . . . . . . . . . . . . . . . . . . . . .   2   3
     2.1.  Terms Used in This Document . . . . . . . . . . . . . . .   2   3
     2.2.  Abbreviations . . . . . . . . . . . . . . . . . . . . . .   3
   3.
     2.3.  Requirements Language . . . . . . . . . . . . . . . . . . . .   4
   4.
   3.  IEEE 802.1 TSN Over DetNet MPLS Data Plane Scenario . . . . .   4
   5.
   4.  DetNet MPLS Data Plane Considerations  . . . . . . . . . . . . . . . . . . .   6
     5.1.  End-System Specific Considerations
     4.1.  Overview  . . . . . . . . . . .   7
   6.  MPLS-Based DetNet Data Plane Solution . . . . . . . . . . . .   8
     6.1. .   6
     4.2.  TSN over DetNet Over MPLS Encapsulation Components  . . . . . . . .   8
     6.2. . . .   7
   5.  TSN over MPLS Data Plane Encapsulation Procedures . . . . . . . . .   9
       6.2.1. . . . .   8
     5.1.  Edge Node Processing TSN Procedures  . . . . . . . . . . . . . . . .   9
       6.2.2.  Layer 2 Addressing and QoS Considerations   8
     5.2.  Edge Node DetNet Service Proxy Procedures . . . . . .  10
   7.  Controller Plane (Management . .   9
     5.3.  Edge Node DetNet Service and Control)
       Considerations Forwarding Sub-Layer
           Procedures  . . . . . . . . . . . . . . . . . . . . . . .  11
   8.   9
   6.  Controller Plane (Management and Control) Considerations  . .  10
   7.  Security Considerations . . . . . . . . . . . . . . . . . . .  11
   9.
   8.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  11
   10.
   9.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .  11
   11.
   10. References  . . . . . . . . . . . . . . . . . . . . . . . . .  11
     11.1.
     10.1.  Normative References . . . . . . . . . . . . . . . . . .  11
     11.2.
     10.2.  Informative References . . . . . . . . . . . . . . . . .  13
   Appendix A.  Example of TSN over DetNet Data Plane Operation  . .  17  12
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  17  13

1.  Introduction

   [Editor's note: Introduction to

   The Time-Sensitive Networking Task Group (TSN TG) within IEEE 802.1
   Working Group deals with deterministic services through IEEE 802
   networks.  Deterministic Networking (DetNet) defined by IETF is a
   service that can be made specific to TSN over DetNet
   scenario.  Do we intend offered by a L3 network to cover both TSN over DetNet IP flows.  General
   background and TSN over concepts of DetNet MPLS?  Or this can be found in
   [I-D.ietf-detnet-architecture].

   This document is limited specifies the use of a DetNet MPLS network to
   interconnect TSN nodes/network segments.  DetNet MPLS scenarios?]. data plane is
   defined in [I-D.ietf-detnet-mpls].

2.  Terminology

   [Editor's note: text to be review what is really needed here.].

2.1.  Terms Used in This Document

   This document uses the terminology and concepts established in the
   DetNet architecture [I-D.ietf-detnet-architecture], [I-D.ietf-detnet-architecture] and the
   [I-D.ietf-detnet-data-plane-framework], and [I-D.ietf-detnet-mpls].
   The reader is assumed to be familiar with that document these documents and its their
   terminology.

2.2.  Abbreviations

   The following terminology is introduced abbreviations are used in this document:

   F-Label       A Detnet "forwarding" label that identifies the LSP
                 used to forward a

   AC            Attachment Circuit.

   CE            Customer Edge equipment.

   CoS           Class of Service.

   CW            Control Word.

   DetNet flow across an MPLS PSN, e.g.,
                 a hop-by-hop label used between label switching routers
                 (LSR).

   S-Label       A DetNet "service" label that is used between DetNet
                 nodes that implement also the DetNet service sub-layer
                 functions.  An S-Label is also used to identify a
                 DetNet flow at DetNet service sub-layer.

   d-CW          A DetNet Control Word (d-CW) is used for sequencing and
                 identifying duplicate packets of a DetNet flow at the
                 DetNet service sub-layer.

2.2.  Abbreviations

   [Editor's note: text to be cleaned up].

   The following abbreviations are used in this document:

   AC            Attachment Circuit.

   CE            Customer Edge equipment.

   CoS           Class of Service.

   CW            Control Word.

   DetNet        Deterministic Networking.

   DF        Deterministic Networking.

   DF            DetNet Flow.

   DN-IWF        DetNet Inter-Working Function.

   FRER          Frame Replication and Elimination for Redundancy (TSN
                 function).

   L2            Layer 2.

   L2VPN         Layer 2 Virtual Private Network.

   L3            Layer 3.

   LSR           Label Switching Router.

   MPLS          Multiprotocol Label Switching.

   MPLS-TE       Multiprotocol Label Switching - Traffic Engineering.

   MPLS-TP       Multiprotocol Label Switching - Transport Profile.

   MS-PW         Multi-Segment PseudoWire (MS-PW).

   NSP           Native Service Processing.

   OAM           Operations, Administration, and Maintenance.

   PE            Provider Edge.

   PEF           Packet Elimination Function.

   PRF           Packet Replication Function.

   PREOF         Packet Replication, Elimination and Ordering Functions.

   POF           Packet Ordering Function.

   PSN           Packet Switched Network.

   PW            PseudoWire.

   QoS           Quality of Service.

   S-PE          Switching Provider Edge.

   T-PE          Terminating Provider Edge.

   TSN           Time-Sensitive Network.

3.

2.3.  Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
   "OPTIONAL" in this document are to be interpreted as described in BCP
   14 [RFC2119] [RFC8174] when, and only when, they appear in all
   capitals, as shown here.

4.

3.  IEEE 802.1 TSN Over DetNet MPLS Data Plane Scenario

   [Author's note: review required on his part.]
    TSN             Edge          Transit        Edge        TSN
    End System      Node           Node          Node        End System
                   (T-PE)         (LSR)          (T-PE)

   +----------+  +.........+                   +.........+  +----------+
   |   Appl.  |<-:Svc Proxy:--End to End Svc.--:Svc Proxy:->|   Appl.  |
   +----------+  +---------+                   +---------+  +----------+
   |    TSN   |  |TSN| |Svc|<-- DetNet flow -->|Svc| |TSN|  |    TSN   |
   +----------+  +---+ +---+    +----------+   +---+ +---+  +----------+
   |Forwarding|  |Fwd| |Fwd|    |Forwarding|   |Fwd| |Fwd|  |Forwarding|
   +------.---+  +--.+ +-.-+    +---.----.-+   +--.+ +-.-+  +----.-----+
          :   Link  :    /  ,-----.  \   :  Link  :   /  ,-----.  \
          +.........+    +-[  Sub  ]-+   +........+   +-[  Sub  ]-+
                           [Network]                    [Network]
                            `-----'                      `-----'

           |<- TSN ->| |<------ DetNet MPLS ------>| |<-- TSN --->|

             Figure 1: A TSN over DetNet MPLS Enabled Network

   Figure 1 shows IEEE 802.1 TSN end stations operating over a TSN aware
   DetNet service running over an MPLS network.  DetNet Edge Nodes sit
   at the boundary of a DetNet domain.  They are responsible for mapping
   non-DetNet aware L2 traffic to DetNet services.  They also support
   the imposition and disposition of the required DetNet encapsulation.
   These are functionally similar to pseudowire (PW) Terminating
   Provider Edge (T-PE) nodes which use MPLS-TE LSPs.  In this example
   they understand and support IEEE 802.1
   TSN and Streams are able to map TSN
   flows into simple applicaions over DetNet flows.  The specific
   of this operation are discussed later in this document.

   Native

      TSN flow and           Edge          Transit         Edge          TSN
   End System       Node           Node           Node       End System
                   (T-PE)         (LSR)          (T-PE)

   +----------+                                             +----------+
   |   TSN    | <---------End to End TSN Service----------> |   TSN    |
   |  Applic. |                                             |  Applic. |
   +----------+  +.........+                   +.........+  +----------+
   |          |  | \S-Proxy:                   :S-Proxy/ |  |          |
   |   TSN    |  |   +.+---+<-- DetNet MPLS flow differ not only by the
   additional MPLS specific encapsulation, but DetNet MPLS flows have on
   each DetNet node an associated -->+---+ |   |  |   TSN    |
   |          |  |TSN| |Svc|                   |Svc| |TSN|  |          |
   +----------+  +---+ +---+    +----------+   +---+ +---+  +----------+
   |   L2     |  | L2| |Fwd|    |Forwarding|   |Fwd| |L2 |  |   L2     |
   +------.---+  +-.-+ +-.-+    +---.----.-+   +--.+ +-.-+  +---.------+
          :  Link  :     /  ,-----.  \   :  Link  :   /  ,-----. \
          +........+     +-[  Sub  ]-+   +........+   +-[  TSN  ]-+
                           [Network]                    [Network]
                            `-----'                      `-----'

                       |<------ DetNet specific data structure, what
   defines flow related characteristics and required forwarding
   functions. MPLS ------>|
           |<---------------------- TSN   --------------------->|

             Figure 1: A TSN over DetNet MPLS Enabled Network

   In this example, edge Nodes nodes provide a service proxy function that
   "associates" the DetNet flows and native flows (i.e., TSN Streams) at
   the edge of the DetNet domain.  This ensures that the DN Flow is properly
   served at the Edge  TSN streams are treated as App-flows
   for DetNet.  The whole DetNet domain behaves as a TSN relay node (and inside for
   the domain). TSN streams.  The service proxy behaves as a port of that TSN
   relay node.

   Figure 2 illustrates how DetNet can provide services for IEEE
   802.1TSN 802.1
   TSN end systems, CE1 and CE2, over a DetNet enabled MPLS network.
   Edge nodes, E1 and E2, insert and remove required DetNet data plane
   encapsulation.  The 'X' in the edge nodes and relay node, R1,
   represent a potential DetNet compound flow packet replication and
   elimination point.  This conceptually parallels L2VPN services, and
   could leverage existing related solutions as discussed below.

        TSN    |<------- End to End DetNet Service ------>|  TSN
       Service |         Transit          Transit         | Service
   TSN  (AC)   |        |<-Tnl->|        |<-Tnl->|        |  (AC)  TSN
   End    |    V        V    1  V        V   2   V        V   |    End
   System |    +--------+       +--------+       +--------+   |  System
   +---+  |    |   E1   |=======|   R1   |=======|   E2   |   |   +---+
   |   |--|----|._X_....|..DF1..|.._ _...|..DF3..|...._X_.|---|---|   |
   |CE1|  |    |    \   |       |   X    |       |   /    |   |   |CE2|
   |   |       |     \_.|..DF2..|._/ \_..|..DF4..|._/     |       |   |
   +---+       |        |=======|        |=======|        |       +---+
       ^       +--------+       +--------+       +--------+       ^
       |        Edge Node       Relay Node        Edge Node       |
       |          (T-PE)           (S-PE)          (T-PE)         |
       |                                                          |
       |<- TSN -> <------- TSN Over DetNet MPLS -------> <- TSN ->|
       |                                                          |
       |<--- Emulated
       |<-------- Time Sensitive Networking (TSN) Service --->| ------->|

       X   = Service protection
       DFx = DetNet member flow x over a TE LSP

                    Figure 2: IEEE 802.1TSN Over DetNet

5.

4.  DetNet MPLS Data Plane Considerations

   [Editor's note: Needs clean up, what is relevant for TSN over DetNet
   scenarios.].

   This section provides informative considerations related to providing

4.1.  Overview

   The basic approach defined in [I-D.ietf-detnet-mpls] supports the
   DetNet service to flows which are identified sub-layer based on their header
   information.  At a high level, the following are provided on a per
   flow basis:

   Eliminating contention loss existing pseudowire (PW)
   encapsulations and jitter reduction:

      Use of allocated resources (queuing, policing, shaping) to ensure
      that mechanisms, and supports the congestion-related loss DetNet forwarding
   sub-layer based on existing MPLS Traffic Engineering encapsulations
   and latency/jitter requirements
      of mechanisms.

   A node operating on a DetNet flow are met.

   Explicit routes:

      Use of in the Detnet service sub-layer,
   i.e.  a specific path for node processing a flow.  This limits misordering and
      bounds latency.

   Service protection:

      Which in DetNet packet which has the case S-Label as top
   of this document primarily relates to
      replication and elimination.  Changing stack uses the explicit path after local context associated with that S-Label, for
   example a
      failure is detected in order received F-Label, to restore delivery of the required determine what local DetNet service characteristics is also possible.  Path changes,
      even in the case of failure recovery, can lead
   operation(s) are applied to that packet.  An S-Label may be unique
   when taken from the out of order
      delivery of data.

   Load sharing:

      Generally, distributing packets platform label space [RFC3031], which would
   enable correct DetNet flow identification regardless of which input
   interface or LSP the same packet arrives on.  The service sub-layer
   functions (i.e., PREOF) use a DetNet flow over
      multiple paths is not recommended.  Such load sharing, e.g., via
      ECMP or UCMP, impacts ordering and possibly jitter.

   Troubleshooting:

      For example, to support identification of misbehaving flows.

   Recognize flow(s) for analytics:

      For example, increase counters.

   Correlate events with flows:

      For example, unexpected loss. control word (d-CW).

   The DetNet MPLS data plane also allows for the aggregation of DetNet
   flows, e.g., via builds on MPLS hierarchical LSPs, Traffic Engineering
   encapsulations and mechanisms to improved scaling.  When provide a forwarding sub-layer that
   is responsible for providing resource allocation and explicit routes.

   The forwarding sub-layer is supported by one or more forwarding
   labels (F-Labels).

   DetNet flows are aggregated, transit edge/relay nodes provide are DetNet service to sub-layer aware,
   understand the
   aggregate particular needs of DetNet flows and not on a per-DetNet flow basis.  In this case, nodes
   performing aggregation will ensure that per-flow service requirements
   are achieved.

5.1.  End-System Specific Considerations

   Data-flows requiring provide both
   DetNet service are generated and terminated on
   end-systems.  Encapsulation depends on application forwarding sub-layer functions.  They add, remove
   and its
   preferences.  In a DetNet MPLS domain the DN functions use the process d-CWs, S-Labels and F-Labels to provide F-labels as needed.  MPLS enabled
   DetNet services.  However, an
   application may exchange further flow related parameters (e.g., time-
   stamp), which are not provided nodes can enhance the reliability of delivery by DN functions.

   Specifics related to non-MPLS DetNet end station behavior are out
   side enabling the scope
   replication of this document.  For example, details on support for packets where multiple copies, possibly over multiple
   paths, are forwarded through the DetNet IP data flows domain.  They can be found in [I-D.ietf-detnet-dp-sol-ip].
   This document is also useful
   eliminate surplus previously replicated copies of DetNet packets.
   MPLS (DetNet) nodes also include DetNet forwarding sub-layer
   functions, support for end stations that map IP flows notably explicit routes, and resources
   allocation to eliminate (or reduce) congestion loss and jitter.

   DetNet flows.

   As transit nodes reside wholly within a general rule, DetNet MPLS domains are capable of forwarding any DetNet MPLS flows domain, and the also
   provide DetNet domain does not mandate forwarding sub-layer functions in accordance with the end-
   system or edge system
   performance required by a DetNet flow carried over an LSP.  Unlike
   other DetNet node types, transit nodes provide no service sub-layer
   processing.

4.2.  TSN over DetNet MPLS Encapsulation

   The basic encapsulation format.  Unless there approach is to treat a proxy
   of some form present, end-systems peer with similar end-systems using
   the same application encapsulation format.  For example, TSN Stream as an app-
   flow from the DetNet MPLS perspective.  The corresponding example
   shown in Figure 3, IP applications peer with IP applications and Ethernet
   L2VPN applications peer with Ethernet L2VPN applications.

             +-----+ 3.

                /->     +------+  +------+  +------+   TSN      ^ ^
                |       |  X   |                               +-----+
             +-----+  |  X   |  | Eth  X   |<- Appli    : :
     App-Flow <-+       +------+  +------+  +------+   cation   : :(1)
      for MPLS  |               ________        +-----+
             +-----+     _____    /        \       |TSN-L2|  |TSN-L2|  |TSN-L2|            : v
                \-> +---+======+--+======+--+======+-----+      :
                        | Eth d-CW |
                    \   /     \__/          \___   +-----+
                     \ /                        \ /
                      0======== tunnel-1 ========0_  |                             \
                       \ d-CW |
                       0========= tunnel-2 =========0
                      / \                        __/ \
               +-----+   \__ DetNet MPLS domain /     \  |  X d-CW |      \         __       /       +-----+
               +-----+       \_______/  \_____/            :
     DetNet-MPLS        +------+  +------+  +------+            :(2)
                        |Labels|  |Labels|  |Labels|            v
                    +---+======+--+======+--+======+-----+
     Link/Sub-Network   |  X  L2  |  |  IP TSN  |  |                                 +-----+
               +-----+ UDP  |
                        +------+  +------+  +------+
                                            |  IP  |
                                                       +-----+

             Figure 3: End-Systems and The DetNet MPLS Domain

6.  MPLS-Based DetNet Data Plane Solution

   [Editor's note: Needs clean up.  Text should focus on Edge node
   related topics.].

6.1.
                                            +------+
                                            |  L2  |
                                            +------+
         (1) TSN Stream
         (2) DetNet Over MPLS Encapsulation Components

   To carry DetNet Flow

           Figure 3: Example TSN over MPLS Encapsulation Formats

   In the following is required:

   1.  A method of identifying figure, "Application" indicates the MPLS application payload type.

   2.  A method of identifying
   carried by the DetNet flow group to TSN network.  "MPLS App-Flow" indicates that the processing
       element.

   3.  A method of distinguishing DetNet OAM packets TSN
   Stream is the payload from the perspective of the DetNet MPLS data
       packets.

   4.
   plane defined in [I-D.ietf-detnet-mpls].  A method of carrying the single DetNet sequence number. MPLS flow
   can aggregate multiple TSN Streams.

5.  A suitable LSP to deliver the packet to the egress PE.

   6.  A method  TSN over MPLS Data Plane Procedures

   Description of carrying queuing Edge Nodes procedures and forwarding indication.

   In this design an MPLS service label (the S-Label), similar to a
   pseudowire (PW) label [RFC3985], is used to identify both the functions for TSN over
   DetNet
   flow identity and the payload MPLS payload type satisfying (1) scenario follows the concept of [RFC3985] and
   (2) in covers the list above.  OAM traffic discrimination happens through
   Edge Nodes components shown on Figure 1.  In this section the use
   following procedures of the Associated Channel method described in [RFC4385].  The DetNet sequence number is carried in the Edge Nodes are described:

   o  TSN related (Section 5.1)

   o  DetNet Control word which
   carries the Data/OAM discriminator.  To simplify implementation Service Proxy (Section 5.2)

   o  DetNet service and
   to maximize interoperability two sequence number sizes are supported: forwarding sub-layer (Section 5.3)

5.1.  Edge Node TSN Procedures

   The Time-Sensitive Networking (TSN) Task Group of the IEEE 802.1
   Working Group have defined (and are defining) a 16 bit sequence number of amendments
   to IEEE 802.1Q [IEEE8021Q] that provide zero congestion loss and
   bounded latency in bridged networks.  IEEE 802.1CB [IEEE8021CB]
   defines packet replication and elimination functions for a 28 bit sequence number. TSN
   network.

   TSN specific functions are executed on the data received by the PE
   from the CE before presentation to the DetNet PW for transmission
   across the DetNet domain, or on the data received from a DetNet PW by
   a PE before it is output on the Attachment Circuit (AC).

   TSN specific function(s) of Edge Nodes (T-PE) are belonging to the
   native service processing (NSP) [RFC3985] block.  This is similar to
   other functionalities being defined by standard bodies other than the
   IETF (for example in case of Ethernet: stripping, overwriting or
   adding VLAN tags, etc.).  Depending on the TSN role of the Edge Node
   in the end-to-end TSN service selected TSN functions must be
   supported.

   Implementations of this document SHALL use management and control
   information to ensure TSN specific functions of the Edge Node
   according to the expectations of the connected TSN network.

5.2.  Edge Node DetNet Service Proxy Procedures

   The 16 bit Service Proxy function maps between App-flows and DetNet flows.
   The DetNet Edge Node TSN function MUST support the TSN Stream
   identification functions and the related managed objects as defined
   in IEEE 802.1CB [IEEE8021CB] and IEEE P802.1CBdb [IEEEP8021CBdb] to
   recognize the App-flow related packets.  The Service Proxy presents
   TSN Streams as an App-flow to a DetNet Flow.

   Implementations of this document SHALL use management and control
   information to map a TSN Stream to a DetNet flow.  N:1 mapping
   (aggregating multiple TSN Streams in a single DetNet flow) SHALL be
   supported.  The management or control function that provisions flow
   mapping SHALL ensure that adequate resources are allocated and
   configured to provide proper service requirements of the mapped
   flows.

   Due to the (intentional) similarities of the DetNet PREOF and TSN
   FRER functions service protection function interworking is possible
   between the TSN and the DetNet domains.  Such service protection
   interworking scenarios MAY require to copy sequence number fields
   from TSN (L2) to PW (MPLS) encapsulation.  However, such interworking
   is out-of-scope in this document and left for further study.

   A MPLS DetNet flow is needed configured to support some types carry any number of legacy clients. TSN flows.
   The 28 bit DetNet flow specific bandwidth profile SHOULD match the required
   bandwidth of the App-flow aggregate.

5.3.  Edge Node DetNet Service and Forwarding Sub-Layer Procedures

   In the design of [I-D.ietf-detnet-mpls] an MPLS service label (the
   S-Label), similar to a pseudowire (PW) label [RFC3985], is used to
   identify both the DetNet flow identity and the payload MPLS payload
   type.  The DetNet sequence number is used carried in situations where it the DetNet Control
   word (d-CW) which carries the Data/OAM discriminator as well.  In
   [I-D.ietf-detnet-mpls] two sequence number sizes are supported: a 16
   bit sequence number and a 28 bit sequence number.

   PREOF functions and the provided service recovery is
   necessary ensure that in high speed networks available only
   within the DetNet domain as the DetNet flow-ID and the DetNet
   sequence number
   space does not wrap whilst packets are not valid outside the DetNet network.  MPLS
   (DetNet) Edge node terminates all related information elements
   encoded in flight. the MPLS labels.

   The LSP used to forward the DetNet packet may be of any type (MPLS-
   LDP, MPLS-TE, MPLS-TP [RFC5921], or MPLS-SR
   [I-D.ietf-spring-segment-routing-mpls]).  The LSP (F-Label) label
   and/or the S-Label may be used to indicate the queue processing as
   well as the forwarding parameters.  Note that the possible use of
   Penultimate Hop Popping (PHP) means that the only label in a received
   label stack may be the S-Label.

6.2.  TSN over MPLS Data

   For further details see [I-D.ietf-detnet-mpls].

6.  Controller Plane Encapsulation

6.2.1.  Edge Node Processing

   An edge node is resposable for matching ingress packets to the
   service they require and encapsulating them accordingly.An edge node
   may participate in the packet replication (Management and duplication
   elimination.

   The DetNet-aware forwarder selects the egress Control) Considerations

   TSN Stream(s) to DetNet member flow
   segment based on the flow identification.  The mapping related information are required
   only for the service proxy function of ingress
   DetNet member flow segment to egress DetNet member flow segment may
   be statically or dynamically configured.  Additionally MPLS (DetNet) Edge nodes.
   From the DetNet-
   aware forwarder does duplicate frame elimination Data Plane perspective there is no practical difference
   based on the flow
   identification and the sequence number combination.  The packet
   replication is also done within the DetNet-aware forwarder.  During
   elimination and the replication process the sequence number origin of the
   DetNet member flow MUST be preserved and copied to the egress mapping related information (management
   plane or control plane).

   MPLS DetNet Edge nodes are member flow.

   The internal design of a relay node is out of scope of this document.
   However the reader's attention is drawn to the need to make any PREOF
   state available to the packet processor(s) dealing with packets to
   which both the PREOF functions must be applied, DetNet domain and to maintain that state
   is such as way that it is available to the packet processor operation
   on the next packet in the DetNet flow (which may be a duplicate, a
   late packet, or
   connected TSN network.  From the next packet in sequence.

   [Editor's note: I think TSN network perspective the rest of this section belongs in MPLS
   (DetNet) Edge node has a new
   "802.1 "TSN relay node" role, so TSN (island Interconnect) over DetNet MPLS" section.]

   This may specific
   management and control plane functionalities must be done implemented.
   There are many similarities in the management plane techniques used
   in DetNet layer, or where the native service
   processing (NSP) [RFC3985] is IEEE 802.1CB [IEEE8021CB] capable, the
   packet replication and duplicate elimination MAY entirely be done in TSN, but that is not the NSP, bypassing case for the DetNet flow encapsulation control plane
   protocols.  For example, RSVP-TE and logic entirely.
   This enables operating over unmodified implementations MSRP behaves differently.
   Therefore management and
   deployments.  The NSP approach works only control plane design is an important aspect
   of scenarios, where mapping between edge nodes DetNet and
   cannot make use of relay nodes.

   The NSP approach TSN is useful end to end tunnel and for for "island
   interconnect" scenarios.  However, when there required.

   Note that, as the DetNet network is just a need to do PREOF
   in a middle portion of the network, such plain edge end to edge operation is not
   sufficient.

   The extended forwarder MAY copy the sequencing information end
   TSN path (i.e., single hop from the
   native DetNet packet into the DetNet sequence number field and vice
   versa.  If Ethernet perspective), some
   parameters (e.g., delay) may differ significantly.  Since there is no existing sequencing information available in
   the native packet or
   interworking function the forwarder chose not bandwidth of DetNet network is assumed to copy
   be set large enough to handle all TSN Flows it from will support.  At the
   native packet, then
   egress of the extended forwarder MUST maintain a sequence
   number counter for each DetNet flow (indexed by Detnet Domain the DetNet flow
   identification).

6.2.2.  Layer 2 Addressing and QoS Considerations

   [Editor's NOTE: review MPLS headers are stripped and simplify this section if possible.]

   The Time-Sensitive Networking (TSN) Task Group of the IEEE 802.1
   Working Group have defined (and are defining) TSN
   flow continues on as a number of amendments normal TSN flow.

   In order to IEEE 802.1Q [IEEE8021Q] that provide zero congestion loss and
   bounded latency in bridged networks.  IEEE 802.1CB [IEEE8021CB]
   defines packet replication and elimination functions that should
   prove both compatible with and useful to, DetNet networks.

   As is the case for DetNet, use a Layer 2 DetNet network node such as a bridge
   may need to identify the interconnect TSN segments, TSN
   specific information must be converted to DetNet flow domain specific
   ones.  TSN Stream ID(s) and stream(s) related parameters/requirements
   must be converted to which a packet
   belongs in order DetNet flow-ID and flow related parameters/
   requirements.

   In some case it may be challenging to provide determine some egress node
   related information.  For example, it may be not trivial to locate
   the TSN/DetNet QoS for that packet.  It
   also will likely need egress point/interface of a CoS marking, such as TSN Streams with a multicast
   destination MAC address.  Such scenarios may require interaction
   between control and management plane functions and between DetNet and
   TSN domains.

   Mapping between DetNet flow identifiers and TSN Stream identifiers,
   if not provided explicitly, can be done by the priority field service proxy function
   of an
   IEEE Std 802.1Q VLAN tag, to give MPLS (DetNet) Edge node locally based on information provided
   for configuration of the packet proper service.

   Although TSN Stream identification functions (e.g.,
   Mask-and-Match Stream identification).

   Triggering the setup/modification of a DetNet flow identification methods described in IEEE 802.1CB
   [IEEE8021CB] the DetNet
   network is an example where management and/or control plane
   interactions are flexible, required between the DetNet and in fact, include IP 5-tuple
   identification methods, the baseline TSN standards assume that every
   Ethernet frame belonging to a network.

   Configuration of TSN stream (i.e.  DetNet flow) carries
   a multicast destination MAC address that is unique to that flow
   within specific functions (e.g., FRER) inside the bridged TSN
   network over which it is carried.  Furthermore,
   IEEE 802.1CB [IEEE8021CB] describes three methods by which a packet
   sequence number can TSN domain specific decision and may not be encoded visible in an Ethernet frame.

   Ensuring that
   the proper Ethernet VLAN tag priority and destination
   MAC address DetNet domain.  Service protection interworking scenarios are used on a DetNet/TSN packet may require
   left for further
   clarification of the customary L2/L3 transformations carried out by
   routers and edge label switches.  Edge nodes may also have to move
   sequence number fields among Layer 2, PW, and IP encapsulations. study.

7.  Controller Plane (Management and Control) Considerations

   [Editor's note: requires considerations related to TSN over DetNet.].

8.  Security Considerations

   The security

   Security considerations of for DetNet in general are discussed described in
   [I-D.ietf-detnet-architecture] and [I-D.sdt-detnet-security].  Other detail in
   [I-D.ietf-detnet-security].  General security considerations will be added are
   described in a future version [I-D.ietf-detnet-architecture].  DetNet MPLS data plane
   specific considerations are summarized in [I-D.ietf-detnet-mpls].
   The primary considerations for the data plane is to maintain
   integrity of this
   draft.

9. data and delivery of the associated DetNet service
   traversing the DetNet network.  Application flows can be protected
   through whatever means is provided by the underlying technology.  For
   example, encryption may be used, such as that provided by IPSec
   [RFC4301] for IP flows and/or by an underlying sub-net using MACSec
   [IEEE802.1AE-2018] for IP over Ethernet (Layer-2) flows.

8.  IANA Considerations

   This document makes no IANA requests.

10.

9.  Acknowledgements

   Thanks for

   The authors wish to thank Norman Finn and Finn, Lou Berger Berger, Craig Gunther,
   Christophe Mangin and Jouni Korhonen for their comments and
   contributions.

11. various contributions
   to this work.

10.  References

11.1.

10.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

   [RFC2211]  Wroclawski, J., "Specification of the Controlled-Load
              Network Element Service", RFC 2211, DOI 10.17487/RFC2211,
              September 1997, <https://www.rfc-editor.org/info/rfc2211>.

   [RFC2212]  Shenker, S., Partridge, C., and R. Guerin, "Specification
              of Guaranteed Quality of Service", RFC 2212,
              DOI 10.17487/RFC2212, September 1997,
              <https://www.rfc-editor.org/info/rfc2212>.

   [RFC3031]  Rosen, E., Viswanathan, A., and R. Callon, "Multiprotocol
              Label Switching Architecture", RFC 3031,
              DOI 10.17487/RFC3031, January 2001,
              <https://www.rfc-editor.org/info/rfc3031>.

   [RFC3032]  Rosen, E., Tappan, D., Fedorkow, G., Rekhter, Y.,
              Farinacci, D., Li, T., and A. Conta, "MPLS Label Stack
              Encoding", RFC 3032, DOI 10.17487/RFC3032, January 2001,
              <https://www.rfc-editor.org/info/rfc3032>.

   [RFC3209]  Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V.,
              and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP
              Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001,
              <https://www.rfc-editor.org/info/rfc3209>.

   [RFC3270]  Le Faucheur, F., Wu, L., Davie, B., Davari, S., Vaananen,
              P., Krishnan, R., Cheval, P., and J. Heinanen, "Multi-
              Protocol Label Switching (MPLS) Support of Differentiated
              Services", RFC 3270, DOI 10.17487/RFC3270, May 2002,
              <https://www.rfc-editor.org/info/rfc3270>.

   [RFC3443]  Agarwal, P. and B. Akyol, "Time To Live (TTL) Processing
              in Multi-Protocol Label Switching (MPLS) Networks",
              RFC 3443, DOI 10.17487/RFC3443, January 2003,
              <https://www.rfc-editor.org/info/rfc3443>.

   [RFC3473]  Berger, L., Ed., "Generalized Multi-Protocol Label
              Switching (GMPLS) Signaling Resource ReserVation Protocol-
              Traffic Engineering (RSVP-TE) Extensions", RFC 3473,
              DOI 10.17487/RFC3473, January 2003,
              <https://www.rfc-editor.org/info/rfc3473>.

   [RFC4206]  Kompella, K. and Y. Rekhter, "Label Switched Paths (LSP)
              Hierarchy with Generalized Multi-Protocol Label Switching
              (GMPLS) Traffic Engineering (TE)", RFC 4206,
              DOI 10.17487/RFC4206, October 2005,
              <https://www.rfc-editor.org/info/rfc4206>.

   [RFC4385]  Bryant, S., Swallow, G., Martini, L., and D. McPherson,
              "Pseudowire Emulation Edge-to-Edge (PWE3) Control Word for
              Use over an MPLS PSN", RFC 4385, DOI 10.17487/RFC4385,
              February 2006, <https://www.rfc-editor.org/info/rfc4385>.

   [RFC5085]  Nadeau, T., Ed. and C. Pignataro, Ed., "Pseudowire Virtual
              Circuit Connectivity Verification (VCCV): A Control
              Channel for Pseudowires", RFC 5085, DOI 10.17487/RFC5085,
              December 2007, <https://www.rfc-editor.org/info/rfc5085>.

   [RFC5129]  Davie, B., Briscoe, B., and J. Tay, "Explicit Congestion
              Marking in MPLS", RFC 5129, DOI 10.17487/RFC5129, January
              2008, <https://www.rfc-editor.org/info/rfc5129>.

   [RFC5462]  Andersson, L. and R. Asati, "Multiprotocol Label Switching
              (MPLS) Label Stack Entry: "EXP" Field Renamed to "Traffic
              Class" Field", RFC 5462, DOI 10.17487/RFC5462, February
              2009, <https://www.rfc-editor.org/info/rfc5462>.

   [RFC7510]  Xu, X., Sheth, N., Yong, L., Callon, R., and D. Black,
              "Encapsulating MPLS in UDP", RFC 7510,
              DOI 10.17487/RFC7510, April 2015,
              <https://www.rfc-editor.org/info/rfc7510>.

   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
              2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
              May 2017, <https://www.rfc-editor.org/info/rfc8174>.

11.2.

10.2.  Informative References

   [G.8275.1]
              International Telecommunication Union, "Precision time
              protocol telecom profile for phase/time synchronization
              with full timing support from the network", ITU-T
              G.8275.1/Y.1369.1 G.8275.1, June 2016,
              <https://www.itu.int/rec/T-REC-G.8275.1/en>.

   [G.8275.2]
              International Telecommunication Union, "Precision time
              protocol telecom profile for phase/time synchronization
              with partial timing support from the network", ITU-T
              G.8275.2/Y.1369.2 G.8275.2, June 2016,
              <https://www.itu.int/rec/T-REC-G.8275.2/en>.

   [I-D.ietf-detnet-architecture]
              Finn, N., Thubert, P., Varga, B., and J. Farkas,
              "Deterministic Networking Architecture", draft-ietf-
              detnet-architecture-12
              detnet-architecture-13 (work in progress), March May 2019.

   [I-D.ietf-detnet-dp-sol-ip]
              Korhonen, J.,

   [I-D.ietf-detnet-data-plane-framework]
              Varga, B., Farkas, J., Berger, L., Fedyk, D., Malis, A.,
              Bryant, S., and J. Korhonen, "DetNet IP Data Plane
              Encapsulation", 2018.

   [I-D.ietf-detnet-flow-information-model]
              Farkas, J.,
              Framework", draft-ietf-detnet-data-plane-framework-02
              (work in progress), September 2019.

   [I-D.ietf-detnet-mpls]
              Varga, B., Cummings, R., Farkas, J., Berger, L., Fedyk, D., Malis, A.,
              Bryant, S., and Y. Jiang, J. Korhonen, "DetNet
              Flow Information Model", draft-ietf-detnet-flow-
              information-model-03 Data Plane: MPLS",
              draft-ietf-detnet-mpls-01 (work in progress), March July 2019.

   [I-D.ietf-pce-pcep-extension-for-pce-controller]
              Zhao, Q., Li, Z., Negi, M., and C. Zhou, "PCEP Procedures

   [I-D.ietf-detnet-security]
              Mizrahi, T., Grossman, E., Hacker, A., Das, S., Dowdell,
              J., Austad, H., Stanton, K., and Protocol Extensions for Using PCE as a Central
              Controller (PCECC) of LSPs", draft-ietf-pce-pcep-
              extension-for-pce-controller-01 N. Finn, "Deterministic
              Networking (DetNet) Security Considerations", draft-ietf-
              detnet-security-05 (work in progress),
              February August 2019.

   [I-D.ietf-spring-segment-routing-mpls]
              Bashandy, A., Filsfils, C., Previdi, S., Decraene, B.,
              Litkowski, S., and R. Shakir, "Segment Routing with MPLS
              data plane", draft-ietf-spring-segment-routing-mpls-22
              (work in progress), May 2019.

   [I-D.sdt-detnet-security]
              Mizrahi, T., Grossman, E., Hacker, A., Das, S.,
              "Deterministic Networking (DetNet) Security
              Considerations, draft-sdt-detnet-security, work in
              progress", 2017.

   [IEEE1588]
              IEEE,

   [IEEE802.1AE-2018]
              IEEE Standards Association, "IEEE 1588 Standard for a Precision Clock
              Synchronization Protocol for Networked Measurement and
              Control Systems Version 2", 2008. Std 802.1AE-2018 MAC
              Security (MACsec)", 2018,
              <https://ieeexplore.ieee.org/document/8585421>.

   [IEEE8021CB]
              Finn, N., "Draft Standard for Local and metropolitan area
              networks - Seamless Redundancy", IEEE P802.1CB
              /D2.1 P802.1CB, December 2015,
              <http://www.ieee802.org/1/files/private/cb-drafts/
              d2/802-1CB-d2-1.pdf>.
              <http://www.ieee802.org/1/files/private/cb-drafts/d2/802-
              1CB-d2-1.pdf>.

   [IEEE8021Q]
              IEEE 802.1, "Standard for Local and metropolitan area
              networks--Bridges and Bridged Networks (IEEE Std 802.1Q-
              2014)", 2014, <http://standards.ieee.org/about/get/>.

   [RFC2205]  Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S.
              Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1
              Functional Specification", RFC 2205, DOI 10.17487/RFC2205,
              September 1997, <https://www.rfc-editor.org/info/rfc2205>.

   [RFC2474]  Nichols, K., Blake, S., Baker, F., and D. Black,
              "Definition of the Differentiated Services Field (DS
              Field) in the IPv4 and IPv6 Headers", RFC 2474,
              DOI 10.17487/RFC2474, December 1998,
              <https://www.rfc-editor.org/info/rfc2474>.

   [RFC3272]  Awduche, D., Chiu, A., Elwalid, A., Widjaja, I., and X.
              Xiao, "Overview and Principles of Internet Traffic
              Engineering", RFC 3272, DOI 10.17487/RFC3272, May 2002,
              <https://www.rfc-editor.org/info/rfc3272>.

   [IEEEP8021CBdb]
              Mangin, C., "Extended Stream identification functions",
              IEEE P802.1CBdb /D0.2 P802.1CBdb, August 2019,
              <http://www.ieee802.org/1/files/private/cb-drafts/d2/802-
              1CB-d2-1.pdf>.

   [RFC3985]  Bryant, S., Ed. and P. Pate, Ed., "Pseudo Wire Emulation
              Edge-to-Edge (PWE3) Architecture", RFC 3985,
              DOI 10.17487/RFC3985, March 2005,
              <https://www.rfc-editor.org/info/rfc3985>.

   [RFC4448]  Martini, L., Ed., Rosen, E., El-Aawar, N., and G. Heron,
              "Encapsulation Methods for Transport of Ethernet over MPLS
              Networks", RFC 4448, DOI 10.17487/RFC4448, April 2006,
              <https://www.rfc-editor.org/info/rfc4448>.

   [RFC4872]  Lang, J., Ed., Rekhter, Y., Ed., and D. Papadimitriou,
              Ed., "RSVP-TE Extensions in Support of End-to-End
              Generalized Multi-Protocol Label Switching (GMPLS)
              Recovery", RFC 4872, DOI 10.17487/RFC4872, May 2007,
              <https://www.rfc-editor.org/info/rfc4872>.

   [RFC4873]  Berger, L., Bryskin, I., Papadimitriou, D., and A. Farrel,
              "GMPLS Segment Recovery", RFC 4873, DOI 10.17487/RFC4873,
              May 2007, <https://www.rfc-editor.org/info/rfc4873>.

   [RFC4875]  Aggarwal, R., Ed., Papadimitriou, D., Ed., and S.
              Yasukawa, Ed., "Extensions to Resource Reservation
              Protocol - Traffic Engineering (RSVP-TE) for Point-to-
              Multipoint TE Label Switched Paths (LSPs)", RFC 4875,
              DOI 10.17487/RFC4875, May 2007,
              <https://www.rfc-editor.org/info/rfc4875>.

   [RFC5440]  Vasseur, JP., Ed. and JL. Le Roux, Ed., "Path Computation
              Element (PCE) Communication Protocol (PCEP)", RFC 5440,
              DOI 10.17487/RFC5440, March 2009,
              <https://www.rfc-editor.org/info/rfc5440>.

   [RFC5586]  Bocci, M., Ed., Vigoureux, M., Ed., and

   [RFC4301]  Kent, S. Bryant, Ed.,
              "MPLS Generic Associated Channel", RFC 5586,
              DOI 10.17487/RFC5586, June 2009,
              <https://www.rfc-editor.org/info/rfc5586>.

   [RFC5654]  Niven-Jenkins, B., Ed., Brungard, D., Ed., Betts, M., Ed.,
              Sprecher, N., and S. Ueno, "Requirements of an MPLS
              Transport Profile", K. Seo, "Security Architecture for the
              Internet Protocol", RFC 5654, 4301, DOI 10.17487/RFC5654,
              September 2009, <https://www.rfc-editor.org/info/rfc5654>. 10.17487/RFC4301,
              December 2005, <https://www.rfc-editor.org/info/rfc4301>.

   [RFC5921]  Bocci, M., Ed., Bryant, S., Ed., Frost, D., Ed., Levrau,
              L., and L. Berger, "A Framework for MPLS in Transport
              Networks", RFC 5921, DOI 10.17487/RFC5921, July 2010,
              <https://www.rfc-editor.org/info/rfc5921>.

   [RFC6003]  Papadimitriou, D., "Ethernet Traffic Parameters",
              RFC 6003, DOI 10.17487/RFC6003, October 2010,
              <https://www.rfc-editor.org/info/rfc6003>.

   [RFC6006]  Zhao, Q., Ed., King, D., Ed., Verhaeghe, F., Takeda, T.,
              Ali, Z., and J. Meuric, "Extensions to the Path
              Computation Element Communication Protocol (PCEP) for
              Point-to-Multipoint Traffic Engineering Label Switched
              Paths", RFC 6006, DOI 10.17487/RFC6006, September 2010,
              <https://www.rfc-editor.org/info/rfc6006>.

   [RFC6073]  Martini, L., Metz, C., Nadeau, T., Bocci, M., and M.
              Aissaoui, "Segmented Pseudowire", RFC 6073,
              DOI 10.17487/RFC6073, January 2011,
              <https://www.rfc-editor.org/info/rfc6073>.

   [RFC6387]  Takacs, A., Berger, L., Caviglia, D., Fedyk, D., and J.
              Meuric, "GMPLS Asymmetric Bandwidth Bidirectional Label
              Switched Paths (LSPs)", RFC 6387, DOI 10.17487/RFC6387,
              September 2011, <https://www.rfc-editor.org/info/rfc6387>.

   [RFC6790]  Kompella, K., Drake, J., Amante, S., Henderickx, W., and
              L. Yong, "The Use of Entropy Labels in MPLS Forwarding",
              RFC 6790, DOI 10.17487/RFC6790, November 2012,
              <https://www.rfc-editor.org/info/rfc6790>.

   [RFC7551]  Zhang, F., Ed., Jing, R., and R. Gandhi, Ed., "RSVP-TE
              Extensions for Associated Bidirectional Label Switched
              Paths (LSPs)", RFC 7551, DOI 10.17487/RFC7551, May 2015,
              <https://www.rfc-editor.org/info/rfc7551>.

   [RFC7950]  Bjorklund, M., Ed., "The YANG 1.1 Data Modeling Language",
              RFC 7950, DOI 10.17487/RFC7950, August 2016,
              <https://www.rfc-editor.org/info/rfc7950>.

   [RFC8040]  Bierman, A., Bjorklund, M., and K. Watsen, "RESTCONF
              Protocol", RFC 8040, DOI 10.17487/RFC8040, January 2017,
              <https://www.rfc-editor.org/info/rfc8040>.

   [RFC8169]  Mirsky, G., Ruffini, S., Gray, E., Drake, J., Bryant, S.,
              and A. Vainshtein, "Residence Time Measurement in MPLS
              Networks", RFC 8169, DOI 10.17487/RFC8169, May 2017,
              <https://www.rfc-editor.org/info/rfc8169>.

Appendix A.  Example of TSN over DetNet Data Plane Operation

   [Editor's note: Add a simplified example of DetNet data plane and how
   labels etc work in the case of TSN over DetNet MPLS and utilizing
   e.g., PREOF.]

Authors' Addresses

   Balazs Varga (editor)
   Ericsson
   Magyar Tudosok krt. 11.
   Budapest  1117
   Hungary

   Email: balazs.a.varga@ericsson.com
   Janos Farkas
   Ericsson
   Magyar Tudosok krt. 11.
   Budapest  1117
   Hungary

   Email: janos.farkas@ericsson.com

   Andrew G. Malis
   Huawei Technologies
   Independent

   Email: agmalis@gmail.com

   Stewart Bryant
   Huawei
   Futurewei Technologies

   Email: stewart.bryant@gmail.com

   Jouni Korhonen

   Don Fedyk
   LabN Consulting, L.L.C.

   Email: jouni.nospam@gmail.com dfedyk@labn.net