<?xml version="1.0" encoding="US-ASCII"?> encoding="UTF-8"?>
<!DOCTYPE rfc SYSTEM "rfc2629.dtd" [
]>
<?xml-stylesheet type='text/xsl' href='rfc2629.xslt' ?>
<?rfc toc="yes"?>
<?rfc symrefs="yes"?>
<?rfc sortrefs="yes"?>
<?rfc iprnotified="no"?>
<?rfc strict="yes"?>
<?rfc compact="yes"?>
<?rfc subcompact="no"?> "rfc2629-xhtml.ent">

<rfc category="std" xmlns:xi="http://www.w3.org/2001/XInclude"
     docName="draft-ietf-detnet-mpls-13" number="8964" ipr="trust200902"
         submissionType="IETF">
     submissionType="IETF" category="std" consensus="true" obsoletes=""
     updates="" xml:lang="en" tocInclude="true" symRefs="true" sortRefs="true"
     version="3">

  <!-- xml2rfc v2v3 conversion 3.3.0 -->
  <front>
    <title abbrev="DetNet Data Plane: MPLS">
     DetNet
     Deterministic Networking (DetNet) Data Plane: MPLS</title>
     <seriesInfo name="RFC" value="8964"/>
    <author role="editor" fullname="Bal&aacute;zs fullname="Balázs Varga" initials="B." surname="Varga">
      <organization>Ericsson</organization>
      <address>
        <postal>
          <street>Magyar Tudosok krt. 11.</street>
          <city>Budapest</city>
          <country>Hungary</country>
          <code>1117</code>
        </postal>
        <email>balazs.a.varga@ericsson.com</email>
      </address>
    </author>
    <author fullname="J&aacute;nos fullname="János Farkas" initials="J." surname="Farkas">
      <organization>Ericsson</organization>
      <address>
        <postal>
          <street>Magyar Tudosok krt. 11.</street>
          <city>Budapest</city>
          <country>Hungary</country>
          <code>1117</code>
        </postal>
        <email>janos.farkas@ericsson.com</email>
      </address>
    </author>
    <author fullname="Lou Berger" initials="L." surname="Berger">
      <organization>LabN Consulting, L.L.C.</organization>
      <address>
        <email>lberger@labn.net</email>
      </address>
    </author>
    <author fullname="Andrew G. Malis" initials="A.G." initials="A." surname="Malis">
      <organization>Malis Consulting</organization>
      <address>
        <email>agmalis@gmail.com</email>
      </address>
    </author>

    <author fullname="Stewart Bryant" initials="S." surname="Bryant">
      <organization>Futurewei Technologies</organization>
      <address>
        <email>stewart.bryant@gmail.com</email>
        <email>sb@stewartbryant.com</email>
      </address>
    </author>
    <author fullname="Jouni Korhonen" initials="J." surname="Korhonen">
      <!--organization abbrev="Nordic">Nordic Semiconductor</organization-->
      <address>        <email>jouni.nospam@gmail.com</email>
      </address>
    </author>

  <!--author fullname="Donald Fauntleroy Duck" initials="D. F." surname="Duck">
   <organization abbrev="Royal Bros.">Royal Bros.</organization>
   <address>
    <postal>
     <street>13 Paradise Road</street>
     <city>Duckburg</city>
     <region>Calisota</region>
     <country>USA</country>
    </postal>
   </address>
  </author-->
    <date /> year="2021" month="January"/>
    <workgroup>DetNet</workgroup>

    <abstract>
      <t>
     This document specifies the Deterministic Networking (DetNet) data plane
     when operating over an MPLS Packet Switched Network.  It leverages
     existing pseudowire (PW) encapsulations and MPLS Traffic Engineering
     (MPLS-TE) encapsulations and mechanisms.  This document builds on the
     DetNet Architecture architecture and Data Plane Framework. data plane framework.
      </t>
    </abstract>
  </front>
  <middle>
    <section title="Introduction" anchor="sec_intro">
<!-- Note: There are no dedicated section to procedures like "DetNet IP Data Plane Procedures" here. Do we need it like in DetNet-IP document ??? --> anchor="sec_intro" numbered="true" toc="default">
      <name>Introduction</name>

  <t>
        Deterministic Networking (DetNet) is a service that can be offered by a
	network to DetNet flows.
        DetNet provides a capability for the delivery of data flows with
        extremely low packet loss rates and bounded end-to-end delivery
        latency.
	General background and concepts of DetNet can be found in the DetNet
	Architecture
	architecture <xref target="RFC8655"/>. target="RFC8655" format="default"/>.
      </t>
      <t>
   The purpose of this document is to describe the use of the MPLS
   data plane to establish and support DetNet flows.
   The DetNet Architecture architecture models the DetNet related DetNet-related data plane
   functions as being decomposed into two sub-layers: a service sub-layer and a
   forwarding sub-layer.  The service sub-layer is used to provide
   DetNet service functions functions, such as protection and reordering.  At the
   DetNet data plane plane, a new set of functions (PREOF) (Packet Replication, Elimination
   and Ordering Functions (PREOF)) provide the service
   sub-layer tasks specific tasks. to the service
   sub-layer. The forwarding sub-layer is used to provide
   forwarding assurance (low loss, assured latency, and limited
   out-of-order delivery). The use of the functionalities of the DetNet
   service sub-layer and the DetNet forwarding sub-layer require
   careful design and control by the  controller plane Controller Plane in addition to the
   DetNet specific
   DetNet-specific use of MPLS encapsulation as specified by this
   document.
      </t>
      <t>
    This document specifies the DetNet data plane operation and the on-wire
    encapsulation of DetNet flows over an MPLS-based Packet Switched Network
    (PSN) using the service reference model.  MPLS encapsulation
    already provides a solid foundation of building blocks to enable the DetNet
    service and forwarding sub-layer functions.  MPLS encapsulated

MPLS-encapsulated DetNet can
    be carried over a variety of different network technologies that can
    provide the DetNet required level of service. service required for DetNet.  However, the specific
    details of how DetNet MPLS is carried over different network technologies
    are out of scope of for this document.
      </t>
      <t>
    MPLS encapsulated
    MPLS-encapsulated DetNet flows can carry different types of
    traffic.  The details of the types of traffic that are carried in
    DetNet are also out of scope of for this document.  An example of IP
    using DetNet MPLS sub-networks can be found in <xref
    target="I-D.ietf-detnet-ip"/>. target="RFC8939"
    format="default"/>. DetNet MPLS may use an associated controller
    and Operations, Administration, and Maintenance (OAM) functions
    that are defined outside of this document.
      </t>
      <t>
    Background information common to all data planes for DetNet
    can be found in the DetNet Data
    Plane Framework data plane framework <xref target="I-D.ietf-detnet-data-plane-framework"/>. target="RFC8938" format="default"/>.
      </t>
    </section>
    <section title="Terminology"> numbered="true" toc="default">
      <name>Terminology</name>
      <section title="Terms numbered="true" toc="default">
        <name>Terms Used in This Document"> Document</name>
        <t>
   This document uses the terminology established in the DetNet
   architecture <xref target="RFC8655"/> target="RFC8655" format="default"/> and
   the DetNet Data Plane Framework data plane framework <xref
   target="I-D.ietf-detnet-data-plane-framework"/>. target="RFC8938" format="default"/>. The reader is
   assumed to be familiar with these documents, any terminology
   defined therein therein, and basic MPLS related MPLS-related terminologies in
   <xref target="RFC3031"/>. target="RFC3031" format="default"/>.
        </t>
        <t>
   The following terminology is introduced in this document:
   <list style="hanging" hangIndent="14">
     <t hangText="F-Label">
        </t>
        <dl newline="false" spacing="normal" indent="14">
          <dt>F-Label</dt>
          <dd>
       A Detnet DetNet "forwarding" label that identifies the LSP Label Switched Path (LSP) used to
       forward a DetNet flow across an MPLS PSN, i.e., a hop-by-hop
       label used between label switching routers (LSR).
     </t>

     <t hangText="S-Label"> Label Switching Routers (LSRs).
     </dd>
          <dt>S-Label</dt>
          <dd>
       A DetNet "service" label that is used between DetNet nodes that
       implement the DetNet service sub-layer functions. An S-Label
       is used to identify a DetNet flow at the DetNet service
       sub-layer at a receiving DetNet node.
     </t>

     <t hangText="A-Label">
     </dd>
          <dt>A-Label</dt>
          <dd>
       A special case of an S-Label, whose aggregation properties are known only at
       the aggregation and deaggregation end-points.
     </t>

    <t hangText="d-CW"> end points.
     </dd>
          <dt>d-CW</dt>
          <dd>
      A DetNet Control Word (d-CW) that is used for sequencing information
      of a DetNet flow at the DetNet
      service sub-layer.
    </t>
    </list>
   </t>
    </dd>
        </dl>
      </section>
      <section title="Abbreviations"> numbered="true" toc="default">
        <name>Abbreviations</name>
        <t>
   The following abbreviations are used in this document:
   <list style="hanging" hangIndent="14">
    <t hangText="CoS">Class of Service.</t>
    <t hangText="CW">Control Word.</t>
    <t hangText="DetNet">Deterministic Networking.</t>
    <t hangText="LSR">Label
        </t>
        <dl newline="false" spacing="normal" indent="14">
          <dt>CoS</dt>
          <dd>Class of Service</dd>
          <dt>CW</dt>
          <dd>Control Word</dd>
          <dt>DetNet</dt>
          <dd>Deterministic Networking</dd>
          <dt>LSR</dt>
          <dd>Label Switching Router.</t>
    <t hangText="MPLS">Multiprotocol Router</dd>
          <dt>MPLS</dt>
          <dd>Multiprotocol Label Switching.</t>
    <t hangText="MPLS-TE">Multiprotocol Switching</dd>
          <dt>MPLS-TE</dt>
          <dd>Multiprotocol Label Switching - Traffic Engineering.</t>
    <t hangText="MPLS-TP">Multiprotocol Engineering</dd>
          <dt>MPLS-TP</dt>
          <dd>Multiprotocol Label Switching - Transport Profile.</t>
    <t hangText="OAM">Operations, Profile</dd>
          <dt>OAM</dt>
          <dd>Operations, Administration, and Maintenance.</t>
    <t hangText="PE">Provider Edge.</t>
    <t hangText="PEF">Packet Maintenance</dd>
          <dt>PE</dt>
          <dd>Provider Edge</dd>
          <dt>PEF</dt>
          <dd>Packet Elimination Function.</t>
    <t hangText="PRF">Packet Function</dd>
          <dt>PRF</dt>
          <dd>Packet Replication Function.</t>
    <t hangText="PREOF">Packet Function</dd>
          <dt>PREOF</dt>
          <dd>Packet Replication, Elimination and Ordering Functions.</t>
    <t hangText="POF">Packet Functions</dd>
          <dt>POF</dt>
          <dd>Packet Ordering Function.</t>
    <t hangText="PSN">Packet Function</dd>
          <dt>PSN</dt>
          <dd>Packet Switched Network.</t>
    <t hangText="PW">PseudoWire.</t>
    <t hangText="QoS">Quality of Service.</t>
    <t hangText="S-PE">Switching Network</dd>
          <dt>PW</dt>
          <dd>Pseudowire</dd>
          <dt>QoS</dt>
          <dd>Quality of Service</dd>
          <dt>S-PE</dt>
          <dd>Switching Provider Edge.</t>
    <t hangText="T-PE">Terminating Edge</dd>
          <dt>T-PE</dt>
          <dd>Terminating Provider Edge.</t>
    <t hangText="TSN">Time-Sensitive Network.</t>
   </list>
  </t> Edge</dd>
          <dt>TSN</dt>
          <dd>Time-Sensitive Networking</dd>
        </dl>
      </section>
      <section title="Requirements Language"> numbered="true" toc="default">
        <name>Requirements Language</name>
        <t>
    The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
    "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", "<bcp14>MUST</bcp14>", "<bcp14>MUST NOT</bcp14>", "<bcp14>REQUIRED</bcp14>", "<bcp14>SHALL</bcp14>", "<bcp14>SHALL
    NOT</bcp14>", "<bcp14>SHOULD</bcp14>", "<bcp14>SHOULD NOT</bcp14>", "<bcp14>RECOMMENDED</bcp14>", "<bcp14>NOT RECOMMENDED</bcp14>",
    "<bcp14>MAY</bcp14>", and
    "OPTIONAL" "<bcp14>OPTIONAL</bcp14>" in this document are to be interpreted as
    described in
    BCP 14 BCP&nbsp;14 <xref target="RFC2119"/> <xref target="RFC8174"/>
    when, and only when, they appear in all capitals, as shown here.
        </t>
      </section>
    </section>  <!-- end of terminology -->
 <section title="DetNet anchor="sec_dt_dp" numbered="true" toc="default">
      <name>Overview of the DetNet MPLS Data Plane Overview" anchor="sec_dt_dp"> Plane</name>
      <section title="Layers anchor="sec_lay_dt_dp" numbered="true" toc="default">
        <name>Layers of DetNet Data Plane" anchor="sec_lay_dt_dp"> Plane</name>
        <t>
    MPLS provides a wide range of capabilities that can be utilised utilized
    by DetNet.  A straight forward straight-forward approach utilizing MPLS for a
    DetNet service sub-layer is based on the existing pseudowire (PW)
    encapsulations and by utilizing utilizes existing MPLS Traffic Engineering MPLS-TE
    encapsulations and mechanisms.
    Background on PWs can be found in <xref target="RFC3985"/> target="RFC3985"
    format="default"/>, <xref target="RFC3032"/>, and <xref
    target="RFC3031"/>. target="RFC3031"
    format="default"/>.
    Background on MPLS Traffic Engineering MPLS-TE can be
    found in <xref target="RFC3272"/> target="RFC3272" format="default"/> and <xref target="RFC3209"/>. target="RFC3209" format="default"/>.
        </t>
        <figure anchor="dn_mpls_dp_approach" align="center"
   title="DetNet anchor="dn_mpls_dp_approach">
          <name>DetNet Adaptation to MPLS Data Plane"> Plane</name>
          <artwork align="center"><![CDATA[ align="center" name="" type="" alt=""><![CDATA[
                   DetNet        MPLS
                     .
Bottom of Stack      .
(inner label)    +------------+
                 |  Service   | d-CW, S-Label (A-Label)
                 +------------+
                 | Forwarding | F-Label(s)
                 +------------+
Top of Stack         .
(outer label)        .
    ]]></artwork>
        </figure>
        <t>
    The DetNet MPLS data plane representation is illustrated in
    <xref target="dn_mpls_dp_approach"/>. target="dn_mpls_dp_approach" format="default"/>.
    The service sub-layer includes a DetNet control word Control Word (d-CW) and
    an identifying service label (S-Label).  The DetNet control word Control Word
    (d-CW) conforms to the Generic PW MPLS Control Word (PWMCW)
    defined in <xref target="RFC4385"/>. target="RFC4385" format="default"/>. An aggregation label (A-Label) is
    a special case of S-Label used for aggregation.
        </t>
        <t>
    A node operating on a received DetNet flow at the Detnet DetNet service
    sub-layer uses the local context associated with a received S-Label,
    i.e., a received F-Label, to determine which local DetNet
    operation(s) are applied to that packet.
    An S-Label may be taken from the platform label space
    <xref target="RFC3031"/>, target="RFC3031" format="default"/>, making it unique, unique and enabling DetNet
    flow identification regardless of which input interface or
    LSP the packet arrives on. It is important to note that S-Label
    values are driven by the receiver, not the sender.
        </t>
        <t>
    The DetNet forwarding sub-layer is supported by zero or more
    forwarding labels (F-Labels). MPLS Traffic Engineering MPLS-TE
    encapsulations and mechanisms can be utilized to provide a
    forwarding sub-layer that is responsible for providing resource
    allocation and explicit routes.
        </t>
      </section>
      <section title="DetNet anchor="sec_mpls_dt_dp_scen" numbered="true" toc="default">
        <name>DetNet MPLS Data Plane Scenarios"
           anchor="sec_mpls_dt_dp_scen"> Scenarios</name>
        <figure align="center" anchor="fig_dn_mpls_detnet"
              title="A anchor="fig_dn_mpls_detnet">
          <name>A DetNet MPLS Network">
        <artwork><![CDATA[ Network</name>
          <artwork name="" type="" align="left" alt=""><![CDATA[
DetNet MPLS       Relay       Transit         Relay       DetNet MPLS
End System        Node         Node           Node        End System
   (T-PE)        (S-PE)       (LSR)          (S-PE)         (T-PE)
+----------+                                             +----------+
|   Appl.  |<------------ End to End End-to-End Service ----------->|   Appl.  |
+----------+   +---------+                 +---------+   +----------+
| Service  |<--| Service |-- DetNet flow --| Service |-->| Service  |
+----------+   +---------+  +----------+   +---------+   +----------+
|Forwarding|   |Fwd| |Fwd|  |Forwarding|   |Fwd| |Fwd|   |Forwarding|
+-------.--+   +-.-+ +-.-+  +----.---.-+   +-.-+ +-.-+   +---.------+
        :  Link  :    /  ,-----.  \   : Link :    /  ,-----.  \
        +........+    +-[  Sub Sub-  ]-+   +......+    +-[  Sub Sub-  ]-+
                        [Network]                   [Network]
                         `-----'                     `-----'
        |<- LSP -->| |<-------- LSP -----------| |<--- LSP -->|

        |<----------------- DetNet MPLS --------------------->|
        ]]></artwork>
        </figure>
        <t>
        <xref target="fig_dn_mpls_detnet"/> target="fig_dn_mpls_detnet" format="default"/> illustrates
        a hypothetical DetNet MPLS-only network
        composed of DetNet aware MPLS enabled DetNet-aware MPLS-enabled end systems, systems operating over a
        DetNet aware
        DetNet-aware MPLS network.  In this figure, the relay nodes are PE
        devices that define the MPLS LSP boundaries boundaries, and the transit nodes
        are LSRs.
        </t>
        <t>
        DetNet end systems and relay nodes understand the particular needs
        of DetNet flows and provide both DetNet service and forwarding
        sub-layer functions.  In the case of MPLS, DetNet service-aware nodes add, remove remove,
        and process d-CWs, S-Labels S-Labels, and F-labels F-Labels as needed.
        DetNet MPLS nodes provide functionality analogous to T-PEs when
        they sit at the edge of an MPLS domain, domain and S-PEs when they are
        in the middle of an MPLS domain, domain; see <xref
        target="RFC6073"/>. target="RFC6073" format="default"/>.
        </t>
        <t>
        In a DetNet MPLS network, transit nodes may be DetNet service
        aware DetNet-service-aware or may be DetNet unaware
	DetNet-unaware MPLS Label Switching Routers
        (LSRs).  In this latter case, such LSRs would be unaware of the
        special requirements of the DetNet service sub-layer, sub-layer but would
        still provide traffic engineering functions and the QoS
        capabilities needed to ensure that the (TE) LSPs meet the service
        requirements of the carried DetNet flows.
        </t>
        <t>
        The application of DetNet using MPLS supports a number of
	control
        plane/management and management plane types. These types support certain MPLS
	data plane deployments. For example example, RSVP-TE, MPLS-TP, or MPLS Segment
        Routing (when extended to support resource allocation) are all valid
        MPLS deployment cases.
        </t>
        <t>
        <xref target="fig_pw_detnet"/> target="fig_pw_detnet" format="default"/> illustrates how an end-to-end MPLS-based
        DetNet service is provided in a more detail.
	In this figure, the
        customer end systems, CE1
        Customer Edge (CE1 and CE2, CE2) are able to send and receive MPLS
        encapsulated
	MPLS-encapsulated DetNet flows, and R1, R2 R2, and R3 are relay nodes in	the
        middle of a DetNet network.  The 'X' in the end systems, systems and relay
        nodes represents potential DetNet compound flow packet replication and
        elimination points.  In this example, service protection is supported
        utilizing at least two DetNet member flows and TE LSPs.  For a
        unidirectional flow, R1 supports PRF PRF, and R3 supports PEF and POF.  Note
        that the relay nodes may change the underlying forwarding sub-layer,
        for example example, tunneling MPLS over IEEE 802.1 TSN <xref
        target="I-D.ietf-detnet-mpls-over-tsn"/>,
	target="I-D.ietf-detnet-mpls-over-tsn" format="default"/> or simply
	over interconnect interconnected network links.
        </t>
        <figure align="center" anchor="fig_pw_detnet"
              title="MPLS-Based anchor="fig_pw_detnet">
          <name>MPLS-Based Native DetNet">
        <artwork><![CDATA[ DetNet</name>
          <artwork name="" type="" align="left" alt=""><![CDATA[
        DetNet                                        DetNet
DetNet  Service        Transit          Transit       Service  DetNet
MPLS    |             |<-Tnl->|        |<-Tnl->|            |  MPLS
End     |             V   1   V        V   2   V            |  End
System  |    +--------+       +--------+       +--------+   |  System
+---+   |    |   R1   |=======|   R2   |=======|   R3   |   |  +---+
|  X...DFa...|._X_....|..DF1..|.__ ___.|..DF3..|...._X_.|.DFa..|.X |
|CE1|========|    \   |       |   X    |       |   /    |======|CE2|
|   |   |    |     \_.|..DF2..|._/ \__.|..DF4..|._/     |   |  |   |
+---+        |        |=======|        |=======|        |      +---+
    ^        +--------+       +--------+       +--------+      ^
    |        Relay Node       Relay Node       Relay Node      |
    |          (S-PE)           (S-PE)           (S-PE)        |
    |                                                          |
    |<---------------------- DetNet MPLS --------------------->|
    |                                                          |
    |<--------------- End to End End-to-End DetNet Service -------------->|

    -------------------------- Data Flow ------------------------->

X   =
        ]]></artwork>
        </figure>
	<dl newline="false" spacing="normal" indent="6">
	  <dt>X</dt>
	  <dd>- Optional service protection (none, PRF, PREOF, PEF/POF)
DFx = PEF/POF)</dd>
	  <dt>DFx</dt>
	  <dd>- DetNet member flow x over a TE LSP
        ]]></artwork>
      </figure> LSP</dd>
	</dl>
      </section>
    </section>  <!-- end of data plane overview -->

<!-- ================================================================= -->
<!-- =================== MPLS Encap considerations ================ -->
<!-- ================================================================= -->
<section title="MPLS-Based anchor="dn-dt-solution" numbered="true" toc="default">
      <name>MPLS-Based DetNet Data Plane Solution" anchor="dn-dt-solution"> Solution</name>
      <section title="DetNet Over anchor="dn-MPLS-en-comps" numbered="true" toc="default">
        <name>DetNet over MPLS Encapsulation Components" anchor="dn-MPLS-en-comps"> Components</name>
        <t>
   To carry DetNet over MPLS MPLS, the following is required:

  <list style="numbers">
   <t>A

        </t>
        <ol spacing="normal" type="1">
	  <li>A method of identifying the MPLS payload type.</t>
   <t>A type.</li>
          <li>A method of identifying the DetNet flow(s) to the processing element.</t>
   <t>A element.</li>
          <li>A method of distinguishing DetNet OAM packets from DetNet data packets.</t>
   <t>A packets.</li>
          <li>A method of carrying the DetNet sequence number.</t>
   <t>A number.</li>
          <li>A suitable LSP to deliver the packet to the egress PE.</t>
   <t>A PE.</li>
          <li>A method of carrying queuing and forwarding indication.</t>
  </list>
  </t> indication.</li>
        </ol>

        <t>
   In this design design, an MPLS service label (the S-Label), S-Label) is similar to a
   pseudowire (PW) label <xref target="RFC3985"/>, target="RFC3985" format="default"/> and
   is used to identify both the
   DetNet flow identity and the payload MPLS payload type satisfying
   (1) and (2) in the list above.
   OAM traffic discrimination
   happens through the use of the Associated Channel method described in
   <xref target="RFC4385"/>. target="RFC4385" format="default"/>.

   The DetNet sequence number is carried in
   the DetNet Control word Word, which also carries the Data/OAM discriminator.  To
   simplify implementation and to maximize interoperability interoperability, two sequence
   number sizes are supported: a 16 bit 16-bit sequence number and a 28 bit 28-bit
   sequence number.  The 16 bit 16-bit sequence number is needed to support
   some types of legacy clients. The 28 bit 28-bit sequence number is used in
   situations where it is necessary to ensure that that, in high speed networks high-speed networks,
   the sequence number space does not wrap whilst packets are in flight.
        </t>
        <t>
   The LSP used to forward the DetNet packet is not restricted regarding any
   method used for establishing that LSP (for example, MPLS-LDP, MPLS-TE,
   MPLS-TP <xref target="RFC5921"/>, MPLS-SR target="RFC5921" format="default"/>, MPLS Segment Routing
   <xref target="RFC8660"/>, target="RFC8660" format="default"/>, etc.).  The LSP (F-Label) label(s)
   and/or F-Label(s) and the
   S-Label may be used alone or together to indicate the required queue
   processing as well as the forwarding parameters.  Note that the possible
   use of Penultimate Hop Popping (PHP) means that the S-Label may be the only
   label received at the terminating DetNet service.
        </t>
      </section>
      <section title="MPLS anchor="pwSolution" numbered="true" toc="default">
        <name>MPLS Data Plane Encapsulation" anchor="pwSolution"> Encapsulation</name>
        <t>
    <xref target="fig_pw_mpls"/> target="fig_pw_mpls" format="default"/> illustrates a DetNet data plane MPLS
    encapsulation.  The MPLS-based encapsulation of the DetNet flows
    is well suited for the scenarios described in
    <xref target="I-D.ietf-detnet-data-plane-framework"/>. target="RFC8938" format="default"/>.
    Furthermore, an end-to-end DetNet
    service
    service, i.e., native DetNet deployment (see <xref
    target="sec_mpls_dt_dp_scen"/>)
    target="sec_mpls_dt_dp_scen" format="default"/>), is also possible if
    DetNet end systems are capable of initiating and termination MPLS encapsulated
    packets.
   </t>
   <t>
    The terminating
    MPLS-encapsulated packets.</t>
        <t>The MPLS-based DetNet data plane encapsulation consists of:
    <list style="symbols">
     <t> of:</t>
        <ul spacing="normal">

          <li>A DetNet control word Control Word (d-CW) containing sequencing information for
	  packet replication and duplicate elimination purposes, and the OAM indicator.</t>
     <t>
	  indicator.</li>
          <li>A DetNet service Label label (S-Label) that identifies a DetNet flow at
	  the receiving DetNet service sub-layer processing node.
     </t>
     <t>
      Zero node.</li>
          <li>Zero or more Detnet DetNet MPLS  Forwarding forwarding label(s) (F-Label) used to
          direct the packet along the label
      switched path Label Switched Path (LSP) to the next
          DetNet service sub-layer processing node along the path.  When Penultimate Hop Popping
          PHP is in
      use use, there may be no label F-Label in the
          protocol stack on the final hop.
     </t>
     <t>
      The hop.</li>
          <li>The necessary data-link encapsulation is then applied prior to
	  transmission over the physical
      media.
     </t>
    </list>
   </t> media.</li>
        </ul>
        <figure title="Encapsulation anchor="fig_pw_mpls">
          <name>Encapsulation of a DetNet App-Flow in an MPLS PSN" anchor="fig_pw_mpls"> PSN</name>
          <artwork align="center"><![CDATA[ align="center" name="" type="" alt=""><![CDATA[
  DetNet MPLS-based encapsulation

+---------------------------------+
|                                 |
|         DetNet App-Flow         |
|         Payload  Packet         |
|                                 |
+---------------------------------+ <--\
|       DetNet Control Word       |    |
+---------------------------------+    +--> DetNet data plane
|           S-Label               |    |    MPLS encapsulation
+---------------------------------+    |
|         [ F-Label(s) ]          |    |
+---------------------------------+ <--/
|           Data-Link             |
+---------------------------------+
|           Physical              |
+---------------------------------+
]]>
    </artwork></figure>
    ]]></artwork>
        </figure>
        <section title="DetNet anchor="dn-sn" numbered="true" toc="default">
          <name>DetNet Control Word and the DetNet Sequence Number"
           anchor="dn-sn"> Number</name>
          <t>
   A DetNet control word Control Word (d-CW) conforms to the Generic PW MPLS Control
   Word (PWMCW) defined in <xref target="RFC4385"/>. target="RFC4385" format="default"/>. The d-CW formatted
   as shown in <xref target="fig_detnet_cw"/> MUST target="fig_detnet_cw" format="default"/> <bcp14>MUST</bcp14> be present in all
   DetNet packets containing app-flow App-flow data.
   This format of the d-CW was created in order (1) to (1) allow larger sequence number
   space to avoid sequence number rollover frequency in some applications
   and (2) to allow sequence numbering systems that include the value zero
   as a valid sequence number, which simplifies implementation.
          </t>
          <figure title="DetNet Control Word" anchor="fig_detnet_cw">
            <name>DetNet Control Word</name>
            <artwork align="center"><![CDATA[ align="center" name="" type="" alt=""><![CDATA[
 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0 0 0 0|                Sequence Number                        |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
]]>
    </artwork></figure>

  <t>
    <list style="hanging">
      <t hangText="(bits
    ]]></artwork>
          </figure>
          <dl newline="true" spacing="normal">
            <dt>(bits 0 to 3)">
        <vspace blankLines="1"/>
        Per 3)</dt>
            <dd>Per <xref target="RFC4385"/>, MUST target="RFC4385" format="default"/>,
	    <bcp14>MUST</bcp14> be set to zero (0).
      </t>
      <t hangText="Sequence (0).</dd>
            <dt>Sequence Number (bits 4 to 31)">
        <vspace blankLines="1"/>
        An 31)</dt>
            <dd>An unsigned value implementing the DetNet sequence number. The
	    sequence number space is a circular one with no restriction on the
	    initial value.
      </t>
    </list>
  </t> value.</dd>
          </dl>
          <t>
   A separate sequence number space MUST <bcp14>MUST</bcp14> be maintained by
   the node that adds the d-CW for each DetNet app-flow, App-flow, i.e., DetNet service.
   The following sequence number Sequence Number field lengths MUST <bcp14>MUST</bcp14> be supported:
   <list style="bullets">
     <t>0 bits</t>
     <t>16 bits</t>
     <t>28 bits</t>
   </list>
   The
          </t>
          <ul spacing="normal">
            <li>0 bits</li>
            <li>16 bits</li>
            <li>28 bits</li>
          </ul>
          <t>The sequence number length MUST <bcp14>MUST</bcp14> be provisioned on
	  a per
   Detnet service per-DetNet-service basis via configuration, i.e., via the controller plane
	  Controller Plane described in <xref target="I-D.ietf-detnet-data-plane-framework"/>. target="RFC8938"
	  format="default"/>.
          </t>
          <t>
    A 0 bit sequence number 0-bit Sequence Number field length indicates that there is no
    DetNet sequence number used for the flow.  When the length is zero,
    the sequence number Sequence Number field MUST <bcp14>MUST</bcp14> be set to zero (0) on all packets
    sent for the flow.
          </t>
          <t>
    When the sequence number Sequence Number field length is 16 or 28 bits for a flow,
    the sequence number MUST <bcp14>MUST</bcp14> be incremented by one for each new app-flow App-flow
    packet sent. When the field length is 16 bits, d-CW bits 4 to 15
    MUST
    <bcp14>MUST</bcp14> be set to zero (0). The values carried in this field can wrap wrap,
    and it is important to note that zero (0) is a valid field value.
    For example, where the sequence number size is 16 bits, the sequence
    will contain: 65535, 0, 1, where zero (0) is an ordinary
    sequence number.
          </t>
          <t> It is important to note that this document differs from <xref
    target="RFC4448"/>
	  target="RFC4448" format="default"/>, where a sequence number of zero
	  (0) is used to indicate that the sequence number check algorithm is
	  not used.
          </t>
  <t>
    The
          <t>The sequence number is optionally used during receive processing processing,
	  as described below in Sections <xref target="pef-requirements"/> target="pef-requirements"
	  format="counter"/> and <xref
    target="pof-requirements"/>. target="pof-requirements"
	  format="counter"/>.
          </t>
        </section>
        <section anchor="flow-identification" title="S-Labels"> numbered="true" toc="default">
          <name>S-Labels</name>
          <t>
    A DetNet flow at the DetNet service sub-layer is identified by
    an S-Label.  MPLS-aware DetNet end systems
    and edge nodes, which are by definition MPLS ingress and egress
    nodes, MUST <bcp14>MUST</bcp14> add (push) and remove (pop) a DetNet
    service-specific d-CW and S-Label.  Relay nodes MAY <bcp14>MAY</bcp14>
    swap S-Label values when processing a DetNet flow, i.e., incoming and
	outgoing S-Labels of a DetNet flow can be different.
          </t>
          <t>
    S-Label values MUST <bcp14>MUST</bcp14> be provisioned per DetNet service via
    configuration, i.e., via
    the controller plane Controller Plane described in
    <xref target="I-D.ietf-detnet-data-plane-framework"/>. target="RFC8938" format="default"/>.
    Note that S-Labels provide identification at the downstream
    DetNet service sub-layer receiver, not the sender.  As such,
    S-Labels MUST <bcp14>MUST</bcp14> be allocated by the entity that controls the service
    sub-layer receiving a node's label space, space and MAY <bcp14>MAY</bcp14> be allocated from
    the platform label space <xref target="RFC3031"/>. target="RFC3031" format="default"/>.
    Because S-Labels are local to each node node, rather than being a
    global identifier within a domain, they must be advertised to their
    upstream DetNet service-aware peer nodes (i.e., a DetNet MPLS End
    System end
    system or a DetNet Relay relay or Edge Node) edge node) and interpreted in the
    context of their received F-Label(s).
    In some PREOF topologies, the node performing replication will be
    sending to multiple nodes performing PEF or POF, POF and may need to
    send different S-Label values for the different member flows for the
    same DetNet service.
          </t>
          <t>
    An S-Label will normally be at the bottom of the label stack once
    the last F-Label is removed, immediately preceding the d-CW.
    To support OAM at the service sub-layer level OAM, level, an OAM Associated Channel
    Header (ACH) <xref target="RFC4385"/> target="RFC4385" format="default"/>
    together with a Generic Associated Channel Label (GAL) <xref
    target="RFC5586"/> MAY
    target="RFC5586" format="default"/> <bcp14>MAY</bcp14> be used in place of
    a d-CW.
          </t>
          <t>
    Similarly, an Entropy Label Indicator/Entropy Indicator (ELI) and Entropy Label (ELI/EL) (EL) <xref
    target="RFC6790"/> MAY
    target="RFC6790" format="default"/> <bcp14>MAY</bcp14> be carried below
    the S-Label in the label stack in
    networks where DetNet flows would otherwise receive ECMP treatment.  When
    ELs are used, the same EL value SHOULD <bcp14>SHOULD</bcp14> be used for all of the packets
    sent using a specific S-Label to force the flow to follow the same path.
    However, as outlined in <xref
    target="I-D.ietf-detnet-data-plane-framework"/> target="RFC8938" format="default"/>, the use of ECMP for DetNet
    flows is NOT RECOMMENDED. <bcp14>NOT RECOMMENDED</bcp14>. ECMP MAY <bcp14>MAY</bcp14> be used for non-DetNet flows within a
    DetNet domain.
          </t>
          <t>
     When receiving a DetNet MPLS packet, an implementation MUST <bcp14>MUST</bcp14> identify
     the DetNet service associated with the incoming packet based on the
     S-Label.  When a node is using platform labels for S-Labels, no
     additional information is needed needed, as the S-label S-Label uniquely identifies
     the DetNet service.  In the case where platform labels are not used, zero
     or more F-Labels proceeding the S-Label MUST <bcp14>MUST</bcp14> be used together with
     the S-Label to uniquely identify the DetNet service associated with a
     received packet.
     The incoming interface MAY <bcp14>MAY</bcp14> also be used together with
     any present F-Label(s) and the S-Label to uniquely identify an
     incoming DetNet service, for example, in the case where PHP is used.
     Note that the choice to use the platform label space
     for an S-Label or an S-Label plus one or more F-Labels to identify
     DetNet services is a local implementation choice, with one caveat.  When one
     or more F-labels, F-Labels, or the incoming interface, is needed together with an
     S-Label to uniquely identify a service, the controller plane Controller Plane must ensure that
     incoming DetNet MPLS packets arrive with the needed information
     (F-label(s)
     (F-Label(s) and/or the incoming interface) and provision the needed
     information. The provisioned information MUST <bcp14>MUST</bcp14> then be used to
       identify incoming DetNet service based on the combination of S-Label
       and F-Label(s) or the incoming interface.
          </t>
          <t>
     The use of platform labels for S-Labels matches other pseudowire
     encapsulations for consistency consistency, but there is no hard requirement in
     this regard.
          </t>
   <t>
     Implementation
          <t>Implementation details of PREOF functions are out of scope for
	  this document. <xref target="IEEE802.1CB-2017"/> target="IEEE802.1CB-2017" format="default"/>
	  defines equivalent replication and elimination
	 specific elimination-specific aspects,
	  which can be applied to PRF and PEF.
   </t> PEF.</t>
          <section anchor="prf-requirements" title="Packet numbered="true" toc="default">
            <name>Packet Replication Function
                                             Processing"> Processing</name>
            <t>
       The Packet Replication Function (PRF) function MAY <bcp14>MAY</bcp14> be supported
       by an implementation for outgoing DetNet flows. The use of the PRF
       for a particular DetNet service MUST <bcp14>MUST</bcp14> be provisioned via configuration,
       i.e., via the controller plane Controller Plane described in
       <xref target="I-D.ietf-detnet-data-plane-framework"/>. target="RFC8938" format="default"/>.  When replication
       is configured, the same app-flow App-flow data will be sent over multiple
       outgoing DetNet member flows using forwarding sub-layer LSPs.
       An S-Label value MUST <bcp14>MUST</bcp14> be configured per outgoing member flow.
       The same d-CW field value MUST <bcp14>MUST</bcp14> be used on all outgoing member flows for
       each replicated MPLS packet.
<!--   PRF MUST NOT be used with DetNet services configured
       with a d-CW sequence number field length of 0 bits.
       [Note: What if multiple receivers? We may use the PRF for serving multiple end-points.] -->
            </t>
          </section>
          <section anchor="pef-requirements" title="Packet numbered="true" toc="default">
            <name>Packet Elimination Function
                                             Processing"> Processing</name>
            <t>
       Implementations MAY <bcp14>MAY</bcp14> support the Packet Elimination Function (PEF)
       for received DetNet MPLS flows.  When supported, use of the PEF
       for a particular DetNet service MUST <bcp14>MUST</bcp14> be provisioned via configuration,
       i.e., via the controller plane Controller Plane described in
       <xref target="I-D.ietf-detnet-data-plane-framework"/>. target="RFC8938" format="default"/>.
            </t>
            <t>
	   After a DetNet service is identified for a received DetNet MPLS packet,
	   as described above, if PEF is configured for that DetNet service,
	   duplicate (replicated) instances of a particular sequence number MUST <bcp14>MUST</bcp14> be
	   discarded.
	   The specific mechanisms used for an implementation to identify which
       received packets are duplicates and which are new is an
       implementation choice.   Note that that, per <xref target="dn-sn"/> target="dn-sn" format="default"/>,
       the sequence number Sequence Number field length may be 16 or 28 bits, and the
       field value can wrap. PEF MUST NOT <bcp14>MUST NOT</bcp14> be used with DetNet flows configured
       with a d-CW sequence number Sequence Number field length of 0 bits.
            </t>
            <t>
	   An implementation MAY <bcp14>MAY</bcp14> constrain the maximum number of sequence numbers
	   that are tracked on either a platform-wide or per flow per-flow basis.
	   Some implementations MAY <bcp14>MAY</bcp14> support the provisioning of
       the maximum number of sequence numbers that are tracked on
       either a platform-wide or per flow per-flow basis.
            </t>
          </section>
          <section anchor="pof-requirements" title="Packet numbered="true" toc="default">
            <name>Packet Ordering Function
                                             Processing"> Processing</name>
            <t>
       A function that is related to in-order delivery is the Packet Ordering Function
       (POF).  Implementations MAY <bcp14>MAY</bcp14> support POF. When
       supported, use of the POF for a particular DetNet service MUST <bcp14>MUST</bcp14> be
       provisioned via configuration, i.e., via the controller plane Controller Plane
       described by
       <xref target="I-D.ietf-detnet-data-plane-framework"/>. target="RFC8938" format="default"/>.
       Implementations MAY <bcp14>MAY</bcp14> require that PEF and POF be used in combination.
       There is no requirement related to the order of execution of the Packet
       Elimination PEF and Ordering Functions POF in an implementation.
            </t>
            <t>
	   After a DetNet service is identified for a received DetNet MPLS
	   packet, as described above, if POF is configured for that DetNet
	   service, packets
	   MUST <bcp14>MUST</bcp14> be processed in the order
	   indicated in the received d-CW sequence
	   number Sequence Number field, which may not
	   be in the order the packets are received.
As defined in <xref target="dn-sn"/>
	   target="dn-sn" format="default"/>, the sequence number Sequence Number field length
	   may be 16 or 28 bits, the sequence number is incremented by one (1) for each new
	   MPLS packet sent for a particular DetNet service, and the field
	   value can wrap.  The specific mechanisms used for an implementation
	   to identify the order of received packets is an implementation
	   choice.
            </t>
            <t>
       An implementation MAY <bcp14>MAY</bcp14> constrain the maximum number of out of order out-of-order
	   packets that can be processed, processed on either a platform-wide or per flow per-flow
	   basis.  The number of out of order out-of-order packets that can be processed also
	   impacts the latency of a flow.
            </t>
            <t>
	   The latency impact on the system resources needed to support a
	   specific DetNet flow will need to be evaluated by the controller plane Controller Plane
	   based on that flow's traffic specification.  An example traffic
	   specification that can be used with MPLS with Traffic Engineering
	   (MPLS-TE)
	   MPLS-TE can be found in <xref target="RFC2212"/>. target="RFC2212" format="default"/>.
            </t>
            <t>
        DetNet implementations can use flow-specific requirements (e.g., maximum
        number of out-of-order packets, packets and maximum latency of the flow) for
        configuration of POF-related buffers.  POF implementation details
		are out-of-scope out of scope for this document document, and POF configuration parameters
		are implementation specific. The Controller Plane identifies and
		sets the POF configuration parameters.
            </t>
          </section>
        </section>
        <section anchor="f-labels" title="F-Labels"> numbered="true" toc="default">
          <name>F-Labels</name>
          <t>
       F-Labels support the DetNet forwarding sub-layer. F-Labels
       are used to provide LSP-based connectivity between DetNet service
       sub-layer processing nodes.
          </t>
          <section anchor="f-labels-ssl"
              title="Service Sub-Layer Related Processing"> numbered="true" toc="default">
            <name>Service Sub-Layer-Related Processing</name>
            <t>
          DetNet MPLS end systems, edge nodes nodes, and relay nodes may operate at
          the DetNet service sub-layer with understanding of DetNet services and their
          requirements.  As mentioned earlier, when operating at this layer layer,
          such nodes can push, pop pop, or swap (pop then push) S-Labels.  In all
          cases, the F-Label(s) used for a DetNet service are always replaced replaced, and
          the following procedures apply.
            </t>
            <t>
         When sending a DetNet flow, zero or more F-Labels MAY <bcp14>MAY</bcp14> be pushed
         on top of an S-Label by the node pushing an S-Label. The
         F-Label(s) to be pushed when sending a particular DetNet service MUST <bcp14>MUST</bcp14>
         be provisioned per outgoing S-Label via configuration, i.e., via the
         controller plane
         Controller Plane discussed in <xref target="I-D.ietf-detnet-data-plane-framework"/>. target="RFC8938" format="default"/>.
         F-Label(s) can also provide context for an S-Label. To
         allow for the omission of F-Label(s), an implementation SHOULD <bcp14>SHOULD</bcp14>
         also allow an outgoing interface to be configured per S-Label.
            </t>
            <t>
       Note,
       Note that when PRF
       is supported, the same app-flow App-flow data will be sent over multiple
       outgoing  DetNet member flows using forwarding sub-layer
       LSPs. This means that an
       implementation may be sending different sets of
       F-Labels per DetNet member flow, each with a different S-Label.
            </t>
            <t>
       When a single set of F-Labels is provisioned for a particular
       outgoing S-Label, that set of F-labels MUST F-Labels <bcp14>MUST</bcp14> be pushed after
       the S-Label is pushed.
       The outgoing packet is then forwarded forwarded, as
       described below in <xref target="f-labels-all"/>. target="f-labels-all" format="default"/>. When a single
       outgoing interface is provisioned, the outgoing packet is then
       forwarded
       forwarded, as described below in <xref target="f-labels-all"/>. target="f-labels-all" format="default"/>.
            </t>
            <t>
       When multiple sets of outgoing F-Labels or interfaces are provisioned for
       a particular DetNet service (i.e., for PRF), a copy of the outgoing packet, including
       the pushed member flow-specific S-Label, MUST <bcp14>MUST</bcp14> be made per F-label F-Label set and outgoing
       interface. Each set of provisioned F-Labels are then pushed onto
       a copy of the packet.  Each copy is then forwarded forwarded, as described
       below in <xref target="f-labels-all"/>. target="f-labels-all" format="default"/>.
            </t>
            <t>
       As described in the previous section, when receiving a DetNet MPLS flow, an implementation
       identifies the DetNet service associated with the incoming packet based
       on the S-Label.  When a node is using platform labels for
       S-Labels, any F-Labels can be popped popped, and the S-label S-Label uniquely
       identifies the DetNet service.  In the case where platform labels are
       not used, incoming F-Label(s) and, optionally, the incoming interface MUST <bcp14>MUST</bcp14> also be provisioned for a
       DetNet service.
            </t>
          </section>
          <section anchor="f-labels-all"
              title="Common numbered="true" toc="default">
            <name>Common F-Label Processing"> Processing</name>
            <t>
         All DetNet aware DetNet-aware MPLS nodes process F-Labels as needed to meet
         the service requirements of the DetNet flow or flows carried in
         the LSPs represented by the F-Labels.  This includes normal
         push, pop pop, and swap operations.  Such processing is essentially
         the same type of processing provided for TE LSPs, although the
         specific service parameters, or traffic specification, can
         differ.  When the DetNet service parameters of the DetNet flow or
         flows carried in an LSP represented by an F-Label can be met by
         an existing TE mechanism, the forwarding sub-layer processing
         node MAY <bcp14>MAY</bcp14> be a DetNet unaware, DetNet-unaware, i.e., standard, MPLS LSR.  Such
         TE LSPs may provide LSP forwarding service as defined in, but
         not limited to, limited, to the following: <xref target="RFC3209"/>, target="RFC3209" format="default"/>, <xref
         target="RFC3270"/>,
	 target="RFC3270" format="default"/>, <xref target="RFC3272"/>, target="RFC3272"
	 format="default"/>, <xref
         target="RFC3473"/>, target="RFC3473" format="default"/>, <xref target="RFC4875"/>,
	 target="RFC4875" format="default"/>, <xref
         target="RFC5440"/>, target="RFC5440"
	 format="default"/>, and <xref target="RFC8306"/>. target="RFC8306" format="default"/>.
            </t>
            <t>
         More specifically, as mentioned above, the DetNet forwarding
         sub-layer provides explicit routes and allocated resources, and
         F-Labels are used to map to each.  Explicit routes are
         supported based on the topmost (outermost) F-Label that is
         pushed or swapped and the LSP that corresponds to this label.
         This topmost (outgoing) label MUST <bcp14>MUST</bcp14> be associated with a
         provisioned outgoing interface and, for non-point-to-point
         outgoing interfaces, a next hop next-hop LSR.  Note that this
         information MUST <bcp14>MUST</bcp14> be provisioned via configuration or the
         controller plane.
         Controller Plane.  In the previously mentioned special case
         where there are no added F-labels F-Labels and the outgoing interface is
         not a point-to-point interface, the outgoing interface MUST <bcp14>MUST</bcp14>
         also be associated with a next hop next-hop LSR.
            </t>
            <t>
         Resources may be allocated in a hierarchical fashion per each LSP
         that is represented by each F-Label. Each LSP MAY <bcp14>MAY</bcp14> be
         provisioned with a service parameter that dictates the
         specific traffic treatment to be received by the traffic
         carried over that LSP.  Implementations of this document MUST <bcp14>MUST</bcp14>
         ensure that traffic carried over each LSP represented by one or more
         F-Labels receives the traffic treatment provisioned for that
         LSP.  Typical mechanisms used to provide different treatment to
         different flows includes include the allocation of system resources
         (such as queues and buffers) and provisioning of related
         parameters (such as shaping, shaping and policing) that may be found in
         implementations of <xref target="RFC2205">Resource the Resource ReSerVation Protocol
       (RSVP)</xref> (RSVP) <xref
	 target="RFC2205" format="default"/> and RSVP-TE <xref target="RFC3209"/> and
	 target="RFC3209" format="default"/>
       <xref target="RFC3473"/>. target="RFC3473" format="default"/>. Support can also be
         provided via an underlying network technology technology, such IEEE802.1 as IEEE 802.1
         TSN <xref target="I-D.ietf-detnet-mpls-over-tsn"/>. target="I-D.ietf-detnet-mpls-over-tsn" format="default"/>.
         The specific mechanisms selected by a DetNet node to ensure DetNet
         service delivery requirements are met for supported DetNet
         flows is outside the scope of this document.
            </t>
            <t>
         Packets that are marked in a way that do not correspond to
         allocated resources, e.g., lack a provisioned F-Label, can
         disrupt the QoS offered to properly reserved DetNet flows by
         using resources allocated to the reserved flows.  Therefore,
         the network nodes of a DetNet network:
         <list style="symbols">
           <t>
             MUST
            </t>
            <ul spacing="normal">
              <li>
             <bcp14>MUST</bcp14> defend the DetNet QoS by discarding or remarking (to
             an allocated DetNet flow or non-competing noncompeting non-DetNet flow)
             packets received that are not associated with a completed
             resource allocation.
           </t>
           <t>
             MUST NOT
           </li>
              <li>
             <bcp14>MUST NOT</bcp14> use a DetNet allocated resource, e.g. e.g., a queue or
             shaper reserved for DetNet flows, for any packet that does
             match the corresponding DetNet flow.
           </t>
           <t>
             MUST
           </li>
              <li>
             <bcp14>MUST</bcp14> ensure a QoS flow does not exceed its allocated
             resources or provisioned service level,
           </t>
           <t>
             MUST level.
           </li>
              <li>
             <bcp14>MUST</bcp14> ensure a CoS flow or service class does not impact the
             service delivered to other flows.  This requirement is
             similar to the requirement for MPLS LSRs that CoS LSPs do
             not impact the resources allocated to TE LSPs, e.g., via
             <xref target="RFC3473"/>.
           </t>
         </list>
       </t> target="RFC3473" format="default"/>.
           </li>
            </ul>
            <t>
         Subsequent sections provide additional considerations related
         to CoS (<xref target="CoS"/>), target="CoS" format="default"/>), QoS (<xref target="QoS"/>) target="QoS" format="default"/>), and
         aggregation (<xref target="FAG"/>). target="FAG" format="default"/>).
            </t>
          </section>
        </section>
      </section>
      <section anchor="oam-indication" title="OAM Indication">
     <!-- LB: why only type 1, if keep it, need conformance language --> numbered="true" toc="default">
        <name>OAM Indication</name>

   <t>
    OAM follows the procedures set out in <xref target="RFC5085"/> target="RFC5085" format="default"/> with the
    restriction that only Virtual Circuit Connectivity Verification
    (VCCV) type 1 is supported.</t>
        <t>As shown in Figure 3 of <xref target="RFC5085"/> target="RFC5085" format="default"/>, when the first nibble of
    the d-CW is 0x0 0x0, the payload following the d-CW is normal user
    data. However, when the first nibble of the d-CW is 0x1, the
    payload that follows the d-CW is an OAM payload with the OAM
    type indicated by the value in the d-CW Channel Type field.</t>

        <t>The reader is referred to <xref target="RFC5085"/> target="RFC5085" format="default"/> for a more detailed
    description of the Associated Channel mechanism, mechanism and to the
    DetNet work on OAM <xref target="I-D.ietf-detnet-mpls-oam" format="default"/> for more information about DetNet OAM.
        </t>
        <t> Additional considerations on DetNet-specific OAM are subjects
   for further study.
        </t>
      </section>
      <section anchor="FAG" title="Flow Aggregation"> numbered="true" toc="default">
        <name>Flow Aggregation</name>
        <t>
    The ability to aggregate individual flows, flows and their associated
    resource control, control into a larger aggregate is an important technique
    for improving scaling of control in the data, management management, and control
    planes.  The DetNet data plane allows for the aggregation of DetNet flows, flows
    to improved scaling. There are two methods of supporting flow
    aggregation covered in this section.
        </t>
        <t>
    The resource control and management aspects of aggregation
    (including the configuration of queuing, shaping, and policing) are
    the responsibility of the DetNet controller plane Controller Plane and is are out of scope of for this
    documents.
    document.  It is also the responsibility of the controller
    plane Controller
    Plane to ensure that consistent aggregation methods are used.
        </t>
        <section anchor="aggregation-at-the-lsp" title="Aggregation Via numbered="true" toc="default">
          <name>Aggregation via LSP Hierarchy"> Hierarchy</name>
          <t>
     DetNet flows forwarded via MPLS can leverage MPLS-TE's existing
     support for hierarchical LSPs (H-LSPs), (H-LSPs); see <xref target="RFC4206"/>. target="RFC4206" format="default"/>.  H-LSPs are
     typically used to aggregate control and resources, resources; they may also be
     used to provide OAM or protection for the aggregated LSPs.  Arbitrary
     levels of aggregation naturally falls fall out of the definition for
     hierarchy and the MPLS label stack <xref target="RFC3032"/>. target="RFC3032" format="default"/>.  DetNet nodes which that
     support aggregation (LSP hierarchy) map one or more LSPs (labels)
     into and from an H-LSP.
     Both carried LSPs and H-LSPs may or may not
     use the TC Traffic Class (TC) field, i.e., L-LSPs (Label-Only-Inferred-PSC LSPs)
     or E-LSPs (EXP-Inferred-PSC LSPs <xref
     target="RFC3270" format="default"/>, which were renamed to "Explicitly TC-encoded-PSC LSPs" in <xref target="RFC3270"/>. target="RFC5462" format="default" sectionFormat="of" section="2.2"/>).  Such nodes will need to
     ensure that individual LSPs and H-LSPs receive the traffic
     treatment required to ensure the required
     DetNet service is preserved.
          </t>
          <t>
     Additional details of the traffic control capabilities needed at a
     DetNet-aware node may be covered in the new service definitions
     mentioned above or in separate future documents.  Controller plane Plane
     mechanisms will also need to ensure that the service required on
     the aggregate flow are provided, which may include the discarding
     or remarking mentioned in the previous sections.
          </t>
        </section>
        <section anchor="aggregating-detnet-flows-as-a-new-detnet-flow"
             title="Aggregating numbered="true" toc="default">
          <name>Aggregating DetNet Flows as a new New DetNet flow"> Flow</name>
          <t>
     An aggregate can be built by layering DetNet flows, including both
     their S-Label and, when present, F-Labels and (when present) F-Labels, as shown below:
          </t>
          <figure title="DetNet anchor="fig_detnet_agg_flow">
            <name>DetNet Aggregation as a new New DetNet Flow" anchor="fig_detnet_agg_flow">

<artwork><![CDATA[ Flow</name>
            <artwork name="" type="" align="left" alt=""><![CDATA[
+---------------------------------+
|                                 |
|           DetNet Flow           |
|         Payload  Packet         |
|                                 |
+---------------------------------+ <--\
|       DetNet Control Word       |    |
+=================================+    |
|            S-Label              |    |
+---------------------------------+    |
|         [ F-Label(s) ]          |    +----DetNet data plane
+---------------------------------+    |
|       DetNet Control Word       |    |
+=================================+    |
|            A-Label              |    |
+---------------------------------+    |
|           F-Label(s)            | <--/
+---------------------------------+
|           Data-Link             |
+---------------------------------+
|           Physical              |
+---------------------------------+
]]></artwork></figure>
]]></artwork>
          </figure>
          <t>
     Both the aggregation label, which is referred to as an A-Label, and the individual flow's S-Label
     have their MPLS S bit
     set indicating the bottom of stack, and the d-CW allows the PREOF
     to work.  An A-Label is a special case of an S-Label, whose
     properties are known only at the aggregation and deaggregation
     end-points.
     end points.
          </t>
          <t>
     It is a property of the A-Label that what follows is a d-CW
     followed by an MPLS label stack.  A
     relay node processing the A-Label would not know the underlying
     payload type, and the A-Label would be processed as a normal
     S-Label. This would only be known to a node that was a peer of the
     node imposing the S-Label. However However, there is no real need for it to
     know the payload type during aggregation processing.
          </t>
          <t>
      As in the previous section, nodes supporting this type of
      aggregation will need to ensure that individual and aggregated
      flows receive the traffic treatment required to ensure the
      required DetNet service is preserved. Also, it is the controller
      plane's Controller
      Plane's responsibility to ensure that the service required on
      the aggregate flow are is properly provisioned.
          </t>
        </section>
      </section>
      <section title="Service numbered="true" toc="default">
        <name>Service Sub-Layer Considerations"> Considerations</name>
        <t>
     The internal procedures for edge and relay node internal procedures nodes related to PREOF are
     implementation specific.  The order of a packet elimination or
     replication is out of scope in for this specification.
        </t>
        <t>
     It is important that the DetNet layer is configured such that a
     DetNet node never receives its own replicated packets. If it were
     to receive such packets packets, the replication function would make the
     loop more destructive of bandwidth than a conventional unicast
     loop.  Ultimately  Ultimately, the TTL in the S-Label will cause the packet to
     die during a transient loop, but given the sensitivity of applications
     to packet latency latency, the impact on the DetNet application would be
     severe.
     To avoid the problem of a transient forwarding loop, changes to an
     LSP supporting DetNet MUST <bcp14>MUST</bcp14> be loop-free.
        </t>
        <section title="Edge anchor="sec_t_pe" numbered="true" toc="default">
          <name>Edge Node Processing" anchor="sec_t_pe"> Processing</name>
          <t>
       A DetNet Edge edge node operates in the DetNet forwarding sub-layer
       and service sub-layer.  An edge node is responsible for matching
       incoming packets to the service they require and encapsulating
       them accordingly. An edge node may perform PRF, PEF, and or and/or POF.
       Details on encapsulation can be found in <xref
       target="pwSolution"/>; target="pwSolution"
       format="default"/>; details on PRF can be found in <xref
       target="prf-requirements"/>;
       target="prf-requirements" format="default"/>; details on PEF can be
       found in <xref
       target="pef-requirements"/>; target="pef-requirements" format="default"/>; and
       details on POF can be found in
       <xref target="pof-requirements"/>. target="pof-requirements" format="default"/>.
          </t>
        </section>
        <section title="Relay anchor="sec_s_pe" numbered="true" toc="default">
          <name>Relay Node Processing" anchor="sec_s_pe"> Processing</name>
          <t>
       A DetNet Relay relay node operates in the DetNet forwarding sub-layer and service sub-layer.
For
       DetNet using MPLS forwarding related MPLS, forwarding-related processing is performed on the F-Label.  This
       processing is done within an extended forwarder function. Whether an
       incoming DetNet flow receives DetNet specific DetNet-specific processing depends
       on how the forwarding is programmed.  Some relay nodes may be DetNet
       service aware for certain DetNet services, while while, for other DetNet services services,
	   these nodes may perform as unmodified LSRs that only understand
       how to switch MPLS-TE LSPs, i.e., as a transit node, node;
       see <xref target="FAG"/>. target="FAG" format="default"/>.  Again, this is entirely up to
       how the forwarding has been programmed.
          </t>
          <t>
	   During the elimination and replication process process, the
       sequence number of an incoming DetNet packet MUST <bcp14>MUST</bcp14> be preserved and
       carried in the corresponding outgoing DetNet
       packet.  For example, a relay node that performs both PEF and PRF
       first performs PEF on incoming packets to create a compound
       flow. It then performs PRF and copies the app-flow App-flow data and the
       d-CW into packets for each outgoing DetNet member flow.
          </t>
          <t>
       The internal design of a relay node is out of scope of for this
       document. However However, the reader's attention is drawn to the need to
       make any PREOF state available to the packet processor(s) dealing
       with packets to which the PREOF functions must be applied, applied and to
       maintain that state is in such a way that it is available to the
       packet processor operation on the next packet in the DetNet flow
       (which may be a duplicate, a late packet, or the next packet in
       sequence).
          </t>
        </section>
      </section>
      <section title="Forwarding numbered="true" toc="default">
        <name>Forwarding Sub-Layer Considerations">
   <!-- maybe move this to be under section 5? --> Considerations</name>

   <section title="Class anchor="CoS" numbered="true" toc="default">
          <name>Class of Service" anchor="CoS"> Service</name>
          <t>
       Class and quality of service, i.e., CoS Service (CoS) and QoS, Quality of Service (QoS) are terms that
       are often used interchangeably and confused with each other.
In the context of DetNet, CoS is used to refer to mechanisms that provide traffic forwarding
traffic-forwarding treatment based on aggregate group
       basis non-flow-specific traffic classification,
and QoS is used to refer to mechanisms that provide traffic
       forwarding traffic-forwarding
treatment based on a specific DetNet flow basis. flow-specific traffic classification.
       Examples of existing network level network-level CoS mechanisms include
       DiffServ
       Differentiated Services (Diffserv), which is enabled by the IP header differentiated services
       code point Differentiated Services
       Code Point (DSCP) field <xref target="RFC2474"/> target="RFC2474" format="default"/> and MPLS label
       traffic class
       Traffic Class field <xref target="RFC5462"/>, target="RFC5462" format="default"/> and at Layer-2,
       Layer 2 by IEEE 802.1p priority code point Priority Code Point (PCP).
          </t>
          <t>
       CoS for DetNet flows carried in PWs and MPLS is provided using
       the existing MPLS Differentiated Services (DiffServ) (Diffserv) architecture
       <xref target="RFC3270"/>. target="RFC3270" format="default"/>.  Both E-LSP and L-LSP MPLS DiffServ Diffserv
       modes MAY <bcp14>MAY</bcp14> be used to support DetNet flows.  The Traffic Class
       field (formerly the EXP field) of an MPLS label follows the
       definition of <xref target="RFC5462"/> target="RFC5462" format="default"/> and <xref
       target="RFC3270"/>.
       target="RFC3270" format="default"/>.  The Uniform, Pipe, and Short Pipe DiffServ
       Diffserv tunneling and TTL processing models are described in <xref
       target="RFC3270"/>
       target="RFC3270" format="default"/> and <xref target="RFC3443"/> target="RFC3443"
       format="default"/> and MAY <bcp14>MAY</bcp14> be used for MPLS LSPs
       supporting DetNet flows. MPLS Explicit Congestion Notification (ECN) MAY
       <bcp14>MAY</bcp14> also be used used, as defined in ECN <xref target="RFC5129"/>
       target="RFC5129" format="default"/> and updated by <xref
       target="RFC5462"/>.
       target="RFC5462" format="default"/>.
          </t>
        </section>
        <section title="Quality anchor="QoS" numbered="true" toc="default">
          <name>Quality of Service" anchor="QoS"> Service</name>
          <t>
       In addition to explicit routes, routes and packet replication and
       elimination, described
       elimination (described in <xref target="dn-dt-solution"/> above, target="dn-dt-solution" format="default"/> above),
       DetNet provides zero congestion loss and bounded latency and
       jitter.  As described in <xref
       target="RFC8655"/>, target="RFC8655" format="default"/>, there are different
       mechanisms that may be used separately or in combination to
       deliver a zero congestion loss service.  This includes Quality of
       Service (QoS)
       QoS mechanisms at the MPLS layer, that which may be combined
       with the mechanisms defined by the underlying network layer layer, such
       as 802.1TSN. IEEE 802.1 TSN.
          </t>
          <t>
       Quality of Service (QoS)
       QoS mechanisms for flow specific flow-specific traffic
       treatment typically includes include a guarantee/agreement for the
       service,
       service and allocation of resources to support the service.
       Example QoS mechanisms include discrete resource allocation,
       admission control, flow identification and isolation, and
       sometimes path control, traffic protection, shaping, policing policing, and
       remarking. Example protocols that support QoS control include
       <xref target="RFC2205">Resource the
       Resource ReSerVation Protocol
       (RSVP)</xref> (RSVP)
       <xref target="RFC2205" format="default"/> and RSVP-TE <xref target="RFC3209"/> and target="RFC3209" format="default"/>
       <xref target="RFC3473"/>. target="RFC3473" format="default"/>.  The existing MPLS mechanisms defined
       to support CoS <xref target="RFC3270"/> target="RFC3270" format="default"/> can also be used to
       reserve resources for specific traffic classes.
          </t>
          <t>
       A baseline set of QoS capabilities for DetNet flows carried in PWs
       and MPLS can be provided by MPLS with Traffic Engineering (MPLS-TE) MPLS-TE
       <xref target="RFC3209"/> and target="RFC3209" format="default"/> <xref target="RFC3473"/>. target="RFC3473" format="default"/>.  TE LSPs can
       also support explicit routes (path pinning).  Current service
       definitions for packet TE LSPs can be found in "Specification of
       the Controlled Load Quality of Service", Controlled-Load Network Element Service" <xref target="RFC2211"/>, target="RFC2211" format="default"/>,
       "Specification of Guaranteed Quality of Service", Service" <xref
       target="RFC2212"/>,
       target="RFC2212" format="default"/>, and "Ethernet Traffic Parameters", Parameters"
       <xref
       target="RFC6003"/>. target="RFC6003" format="default"/>. Additional service
       definitions are expected in
       future documents to support the full range of DetNet services.
       In all cases, the existing label-based marking mechanisms defined
       for TE-LSPs TE LSPs and even E-LSPs are use used to support the identification
       of flows requiring DetNet QoS.
          </t>
        </section>
      </section>
    </section>

<!-- ===================================================================== -->

<section anchor="mc_summary"
         title="Management numbered="true" toc="default">
      <name>Management and Control Information Summary"> Summary</name>
      <t>
    The specific information needed for the processing of each DetNet
    service depends on the DetNet node type and the functions being
    provided on the node.  This section summarizes this information based on the DetNet
    sub-layer and if the DetNet traffic is being sent or received.  All
    DetNet node types are DetNet forwarding sub-layer aware, while all
    but transit nodes are service sub-layer aware.  This is shown in <xref
    target="fig_dn_mpls_detnet"/>.
    target="fig_dn_mpls_detnet" format="default"/>.
      </t>
  <!-- LB: this seems duplicative

  <t>
        For DetNet management there are a number of approaches that could be used to provide
        explicit routes and resource allocation in the MPLS forwarding sub-layer:
        <list style="symbols">
          <t>
            The path could be explicitly set up by a controller which
            calculates the path and
            explicitly configures each node along that path with the
            appropriate forwarding and resource allocation information.
          </t>
          <t>
            The path could be set up using RSVP-TE signaling.
          </t>
          <t>
            The path could be implemented using
            MPLS-based segment routing when extended to support resource
            allocation.
          </t>
        </list>
        </t>
  -->
  <t>
        Much like other MPLS labels,
        Much like other MPLS labels, there are a number of alternatives
        available for DetNet S-Label and F-Label advertisement to an
        upstream peer node. These include distributed signaling
        protocols such (such as RSVP-TE, RSVP-TE), centralized label distribution via a
        controller that manages both the sender and the receiver using
        NETCONF/YANG, the
        Network Configuration Protocol (NETCONF) and YANG, BGP, PCEP, the Path Computation
	Element Communication Protocol (PCEP), etc., and hybrid combinations of the
        two.  The details of the controller plane Controller Plane solution required for
        the label distribution and the management of the label number
        space are out of scope of for this document.
        There are particular
        Particular DetNet considerations and requirements that
        are discussed in <xref target="I-D.ietf-detnet-data-plane-framework"/>. target="RFC8938" format="default"/>.
    Conformance language is not used in the summary summary, since it applies to
    future mechanisms mechanisms, such as those that may be provided in signaling
    and YANG models, e.g., <xref target="I-D.ietf-detnet-yang"/>. target="I-D.ietf-detnet-yang" format="default"/>.
      </t>
      <section anchor="mc_summary_ssl"
         title="Service numbered="true" toc="default">
        <name>Service Sub-Layer Information Summary"> Summary</name>
        <t>
    The following summarizes the information that is needed (on a per-service
    basis) on nodes that are service sub-layer aware nodes that and transmit DetNet
    MPLS traffic, on a per service basis:
    <list style="symbols">
      <t>
        App-Flow traffic:
        </t>
        <ul spacing="normal">
          <li>App-flow identification information, e.g., IP information as
	  defined in <xref target="I-D.ietf-detnet-ip-over-mpls"/>.  Note, target="I-D.ietf-detnet-ip-over-mpls"
	  format="default"/>.  Note that this information is not needed on DetNet
	  relay nodes.
      </t>
      <t>
        The nodes.</li>
          <li>The sequence number length to be used for the service.  Valid
	  values include 0, 16 16, and 28 bits. 0 bits cannot be used when PEF or
	  POF is configured for the service.
      </t>
      <t>
        If service.</li>
          <li>If PRF is to be provided for the service.
      </t>
      <t>
        The service.</li>
          <li>The outgoing S-Label for each of the service's outgoing DetNet
	  (member) flows.
      </t>
      <t>
        The flows.</li>
          <li>The forwarding sub-layer information associated with the output
	  of the service sub-layer.  Note that when the PRF function is
	  provisioned, this information is per DetNet member flow.
        Logically Logically,
	  the forwarding sub-layer information is a pointer to further details
	  of transmission of Detnet DetNet flows at the forwarding sub-layer.
      </t>
    </list>
  </t> sub-layer.</li>
        </ul>
        <t>
    The following summarizes the information that is needed (on a per-service basis) on nodes that are service
    sub-layer aware nodes that and receive DetNet MPLS traffic, on a per
    service basis:
    <list style="symbols">
      <t>
        The traffic:
        </t>
        <ul spacing="normal">
          <li>The forwarding sub-layer information associated with the input
	  of the service sub-layer.  Note that when the PEF function is
	  provisioned, this information is per DetNet member flow.
        Logically Logically,
	  the forwarding sub-layer information is a pointer to further details
	  of the reception of Detnet DetNet flows at the forwarding sub-layer or
        A-Label.
      </t>
      <t>
        The
	  A-Label.</li>
          <li>The incoming S-Label for the service.
      </t>
      <t>
        If service.</li>
          <li>If PEF or POF is to be provided for the service.
      </t>
      <t>
        The service.</li>
          <li>The sequence number length to be used for the service.  Valid
	  values included 0, 16 16, and 28 bits. 0 bits cannot be used when PEF or
	  POF are configured for the service.
      </t>
      <t>
        App-Flow service.</li>
          <li>App-flow identification information, e.g., IP information as
	  defined in <xref target="I-D.ietf-detnet-ip-over-mpls"/>.  Note, target="I-D.ietf-detnet-ip-over-mpls"
	  format="default"/>.  Note that this information is not needed on DetNet
	  relay nodes.
      </t>
    </list>
  </t> nodes.</li>
        </ul>
        <section anchor="mc_summary_ssl_agg"
         title="Service numbered="true" toc="default">
          <name>Service Aggregation Information Summary"> Summary</name>
          <t>
      Nodes performing aggregation using A-Labels, per Section <xref
      target="aggregating-detnet-flows-as-a-new-detnet-flow"/>, target="aggregating-detnet-flows-as-a-new-detnet-flow" format="default"/>, require
      the additional information summarized in this section.
          </t>
          <t>
      The following summarizes the additional information that is needed on
      a node that sends aggregated flows using A-Labels:
      <list style="symbols">
        <t>
          The
          </t>
          <ul spacing="normal">
            <li>The S-Labels or F-Labels that are to be carried over each
	    aggregated service.
        </t>
        <t>The service.</li>
            <li>The A-Label associated with each aggregated service.</t>
        <t>The service.</li>
            <li>The other S-Label information summarized above.</t>
     </list>
    </t> above.</li>
          </ul>
          <t>

      On the receiving node, the A-Label provides the forwarding context
      of an incoming interface or an F-Label and is used in subsequent
      service or forwarding sub-layer receive processing, as
      appropriated.
      appropriate.  The related additional configuration that may be
      provided is discussed elsewhere in this section.
          </t>
        </section>
      </section>
      <section anchor="mc_summary_fsl"
         title="Forwarding numbered="true" toc="default">
        <name>Forwarding Sub-Layer Information Summary"> Summary</name>

        <t>
      The following summarizes the information that is needed (on a per-forwarding-sub-layer-flow basis) on nodes that are
      forwarding sub-layer aware nodes that and send DetNet MPLS traffic, on
      a per forwarding sub-layer flow basis:
      <list style="symbols">
        <t> traffic:
        </t>
        <ul spacing="normal">
          <li>
          The outgoing F-Label stack to be pushed. The stack may include
          H-LSP labels.
        </t>
        <t> labels.</li>
          <li>
          The traffic parameters associated with a specific label in
          the stack.  Note that there may be multiple sets of traffic
          parameters associated with specific labels in the stack, e.g.,
          when H-LSPs are used.
        </t>
        <t> used.</li>
          <li>
          Outgoing interface and, for unicast traffic, the next hop
          information.
        </t>
        <t>
          Sub-network specific next-hop
          information.</li>
          <li>
          Sub-network-specific parameters on a technology specific technology-specific
          basis. For example, see <xref
          target="I-D.ietf-detnet-mpls-over-tsn"/>.
        </t>
      </list>
    </t> target="I-D.ietf-detnet-mpls-over-tsn"
	  format="default"/>.</li>
        </ul>
        <t>
      The following summarizes the information that is needed (on a
      per-forwarding-sub-layer-flow basis) on nodes that are forwarding
      sub-layer aware nodes that and receive DetNet MPLS traffic, on
      a per forwarding sub-layer flow basis:
      <list style="symbols">
        <t> traffic:
        </t>
        <ul spacing="normal">
          <li>
          The incoming interface.  For some implementations and
          deployment scenarios, this information may not be needed.
        </t>
        <t> needed.</li>
          <li>
          The incoming F-Label stack to be popped. The stack may include
          H-LSP labels.
        </t>
        <t> labels.</li>
          <li>
          How the incoming forwarding sub-layer flow is to be handled,
          i.e., forwarded as a transit node, node or provided to the service
          sub-layer.
        </t>
      </list>
    </t>
          sub-layer.</li>
        </ul>
        <t>
    It is the responsibility of the DetNet controller plane Controller Plane to
    properly provision both flow identification information and
    the flow-specific resources needed to provided provide the traffic
    treatment needed to meet each flow's service requirements.
    This applies for aggregated and individual flows.
        </t>
      </section>
    </section>
<!-- ===================================================================== -->

<section title="Security Considerations"> numbered="true" toc="default">
      <name>Security Considerations</name>
      <t>
   Detailed security considerations for DetNet are cataloged in
   <xref target="I-D.ietf-detnet-security"/>, target="I-D.ietf-detnet-security" format="default"/>, and more general security considerations
   are described in <xref target="RFC8655"/>. target="RFC8655" format="default"/>. This section
   considers
   exclusively considers security considerations which that are specific to the DetNet
   MPLS data plane. The considerations raised related to MPLS networks
   in general in <xref target="RFC5920"/> target="RFC5920" format="default"/> are equally applicable to
   the
   the DetNet MPLS data plane.
      </t>
      <t>
   Security aspects which that are unique to DetNet are those whose aim is to
   protect the support of specific quality of service QoS aspects of DetNet, which are
   primarily to deliver data flows with extremely low packet loss rates
   and bounded end-to-end delivery latency.

       Achieving such loss rates and bounded latency may not be possible
       in the face of a highly capable adversary, such as the one
       envisioned by the Internet Threat Model of BCP 72 <xref target="RFC3552"/> that can
       arbitrarily drop or delay any or all traffic.  In order to
       present meaningful security considerations, we consider a
       somewhat weaker attacker who does not control the physical links
       of the DetNet domain, domain but may have the ability to control a
       network node within the boundary of the DetNet domain.
      </t>
      <t>
    An additional consideration for the DetNet data plane is to maintain
    integrity of data and delivery of the associated DetNet service traversing
    the DetNet network.
    Application flows can be protected through whatever means are
    provided by the underlying technology. For example, encryption may be
    used, such as that provided by IPsec <xref target="RFC4301"/> target="RFC4301" format="default"/> for IP
    flows and/or by an underlying sub-net sub-network using MACSec MACsec
    <xref target="IEEE802.1AE-2018"/> target="IEEE802.1AE-2018" format="default"/> for IP over Ethernet (Layer-2)
    (Layer 2) flows.
	MPLS doesn't provide any native security services to account for
	confidentiality and integrity.
      </t>
      <t>
    From a data plane perspective perspective, this document does not add or modify any
    application header information.
      </t>
      <t>
    At the management and control level level, DetNet flows are identified on a
    per-flow basis, which may provide controller plane Controller Plane
    attackers with additional information about the data flows (when
    compared to controller planes Controller Planes that do not include per-flow identification).
    This is an inherent property of DetNet which that has security
    implications that should be considered when determining if DetNet is
    a suitable technology for any given use case.
      </t>
      <t>
    To provide uninterrupted availability of the DetNet
    service, provisions can be made against DOS DoS attacks and delay
    attacks. To protect against DOS DoS attacks, excess traffic due to
    malicious or malfunctioning devices is prevented or mitigated
    through the use of existing mechanisms, for example example, by policing and
	shaping incoming traffic. To prevent DetNet packets from
	having their delay manipulated by an external entity, precautions need
	to be taken to ensure that all devices on an LSP are those intended to
	be there by the network operator and that they are well behaved. In
	addition to physical security, technical methods methods,  such as authentication
	and authorization of network equipment and the instrumentation and
	monitoring of the LSP packet delay delay, may be used. If a delay attack is
	suspected, traffic may be moved to an alternate path, for example example,
	through changing the LSP or management of the PREOF configuration.
      </t>
    </section>
    <section anchor="iana" title="IANA Considerations">
  <t>
   This numbered="true" toc="default">
      <name>IANA Considerations</name>
      <t>This document makes has no IANA requests.
  </t> actions.</t>
    </section>

  </middle>
  <back>

<displayreference target="I-D.ietf-detnet-ip-over-mpls" to="DetNet-IP-over-MPLS"/>
<displayreference target="I-D.ietf-detnet-mpls-over-tsn" to="DetNet-MPLS-over-TSN"/>
<displayreference target="I-D.ietf-detnet-security" to="DetNet-Security"/>
<displayreference target="I-D.ietf-detnet-yang" to="DetNet-YANG"/>
<displayreference target="I-D.ietf-detnet-mpls-oam" to="DetNet-MPLS-OAM"/>

    <references>
      <name>References</name>
      <references>
        <name>Normative References</name>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.2119.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.2211.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.2212.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.3031.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.3032.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.3209.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.3270.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.3443.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.3473.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.4206.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.5129.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.5085.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.5462.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.5586.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.4385.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.6790.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8174.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8655.xml"/>

<xi:include
href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8938.xml"/>

      </references>
      <references>
        <name>Informative References</name>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.2205.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.2474.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.3272.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.3552.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.3985.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.4448.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.4875.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.4301.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.5440.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.5920.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.5921.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.6073.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.6003.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8306.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8660.xml"/>
        <xi:include
href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8939.xml"/>

<xi:include href="https://datatracker.ietf.org/doc/bibxml3/reference.I-D.draft-ietf-detnet-mpls-oam.xml"/>

<reference anchor='I-D.ietf-detnet-ip-over-mpls'>
<front>
<title>DetNet Data Plane: IP over MPLS</title>
<author initials='B' surname='Varga' fullname='Balazs Varga' role='editor'>
    <organization />
</author>
<author initials='L' surname='Berger' fullname='Lou Berger'>
    <organization />
</author>
<author initials='D' surname='Fedyk' fullname='Don Fedyk'>
    <organization />
</author>
<author initials='S' surname='Bryant' fullname='Stewart Bryant'>
    <organization />
</author>
<author initials='J' surname='Korhonen' fullname='Jouni Korhonen'>
    <organization />
</author>
<date month='October' day='11' year='2020' />
</front>
<seriesInfo name='Internet-Draft' value='draft-ietf-detnet-ip-over-mpls-09' />
<format type='TXT'
       target='http://www.ietf.org/internet-drafts/draft-ietf-detnet-ip-over-mpls-09.txt' />
</reference>

<reference anchor='I-D.ietf-detnet-mpls-over-tsn'>
<front>
<title>DetNet Data Plane: MPLS over IEEE 802.1 Time Sensitive Networking (TSN)</title>
<author initials='B' surname='Varga' fullname='Balazs Varga' role='editor'>
    <organization />
</author>
<author initials='J' surname='Farkas' fullname='Janos Farkas'>
    <organization />
</author>
<author initials='A' surname='Malis' fullname='Andrew Malis'>
    <organization />
</author>
<author initials='S' surname='Bryant' fullname='Stewart Bryant'>
    <organization />
</author>
<date month='December' day="13" year='2020' />
</front>
<seriesInfo name='Internet-Draft' value='draft-ietf-detnet-mpls-over-tsn-05' />
<format type='TXT'
        target='http://www.ietf.org/internet-drafts/draft-ietf-detnet-mpls-over-tsn-05.txt' />
</reference>

<reference anchor='I-D.ietf-detnet-security'>
<front>
<title>Deterministic Networking (DetNet) Security Considerations</title>
<author initials='E' surname='Grossman' fullname='Ethan Grossman' role='editor'>
    <organization />
</author>
<author initials='T' surname='Mizrahi' fullname='Tal Mizrahi'>
    <organization />
</author>
<author initials='A' surname='Hacker' fullname='Andrew Hacker'>
    <organization />
</author>
<date month='December' day="11" year='2020' />
</front>
<seriesInfo name='Internet-Draft' value='draft-ietf-detnet-security-13' />
<format type='TXT'
        target='http://www.ietf.org/internet-drafts/draft-ietf-detnet-security-13.txt' />
</reference>

        <xi:include href="https://datatracker.ietf.org/doc/bibxml3/reference.I-D.ietf-detnet-yang.xml"/>

        <reference anchor="IEEE802.1AE-2018" target="https://ieeexplore.ieee.org/document/8585421">
          <front>
            <title>IEEE Standard for Local and metropolitan area
	    networks-Media Access Control (MAC) Security</title>
            <author>
              <organization>IEEE</organization>
            </author>
            <date month="December" year="2018"/>
          </front>
	  <seriesInfo name="IEEE" value="802.1AE-2018"/>
	  <seriesInfo name="DOI" value="10.1109/IEEESTD.2018.8585421"/>
        </reference>

        <reference anchor="IEEE802.1CB-2017" target="https://ieeexplore.ieee.org/document/8091139">
          <front>
            <title>IEEE Standard for Local and metropolitan area networks--
	    Frame Replication and Elimination for Reliability</title>
            <author>
              <organization>IEEE</organization>
            </author>
            <date month="October" year="2017"/>
          </front>
	  <seriesInfo name="IEEE" value="802.1CB-2017"/>
	  <seriesInfo name="DOI" value="10.1109/IEEESTD.2017.8091139"/>
        </reference>
      </references>
    </references>
    <section anchor="acks" title="Acknowledgements"> numbered="false" toc="default">
      <name>Acknowledgements</name>
      <t>
                The authors wish to thank Pat Thaler, Norman Finn, Loa Anderson, David Black,
                Rodney Cummings, Ethan Grossman, Tal Mizrahi, David Mozes, Craig Gunther,
                George Swallow, Yuanlong Jiang, Jeong-dong Ryoo <contact fullname="Pat Thaler"/>,
		<contact fullname="Norman Finn"/>, <contact fullname="Loa
		Anderson"/>, <contact fullname="David Black"/>,
                <contact fullname="Rodney Cummings"/>, <contact
		fullname="Ethan Grossman"/>, <contact fullname="Tal
		Mizrahi"/>, <contact fullname="David Mozes"/>, <contact
		fullname="Craig Gunther"/>,
                <contact fullname="George Swallow"/>, <contact
		fullname="Yuanlong Jiang"/>, <contact fullname="Jeong-dong
		Ryoo"/>, and Carlos <contact fullname="Carlos J. Bernardos Bernardos"/> for their
                various contributions to this work.
      </t>
    </section>
    <section anchor="Contributors" title="Contributors"> numbered="false" toc="default">
      <name>Contributors</name>
      <t>
    RFC7322 limits the number of authors listed on the front page of a
    draft to a maximum of 5.
   The editor of this document wishes to thank and acknowledge the
   following author for contributing text person who contributed substantially to the content of this draft.
   document and should be considered a coauthor:
      </t>

  <figure> <artwork><![CDATA[
   Don Fedyk
   LabN
<contact fullname="Don Fedyk">
   <organization>LabN Consulting, L.L.C.
   Email: dfedyk@labn.net
   ]]></artwork>
  </figure> L.L.C.</organization>
   <address>
     <postal/>
     <email>dfedyk@labn.net</email>
   </address>
</contact>
    </section>

</middle>

<back>
  <references title="Normative References">
   <?rfc include="reference.RFC.2119"?>
   <?rfc include="reference.RFC.2211"?>
   <?rfc include="reference.RFC.2212"?>
   <?rfc include="reference.RFC.3031"?>
   <?rfc include="reference.RFC.3032"?>
   <?rfc include="reference.RFC.3209"?>
   <?rfc include="reference.RFC.3270"?>
   <?rfc include="reference.RFC.3443"?>
   <?rfc include="reference.RFC.3473"?>
   <?rfc include="reference.RFC.4206"?>
   <?rfc include="reference.RFC.5129"?>
   <?rfc include="reference.RFC.5085"?>
   <?rfc include="reference.RFC.5462"?>
   <?rfc include="reference.RFC.5586"?>
   <?rfc include="reference.RFC.4385"?>
   <?rfc include="reference.RFC.6790"?>
   <?rfc include="reference.RFC.8174"?>
   <?rfc include="reference.RFC.8655"?>
   <?rfc include="reference.I-D.ietf-detnet-data-plane-framework"?>
  </references>
  <references title="Informative References">
   <?rfc include="reference.RFC.2205"?>
   <?rfc include="reference.RFC.2474"?>
   <?rfc include="reference.RFC.3272"?>
   <?rfc include="reference.RFC.3985"?>
   <?rfc include="reference.RFC.4448"?>
   <?rfc include="reference.RFC.4875"?>
   <?rfc include="reference.RFC.4301"?>
   <?rfc include="reference.RFC.5440"?>
   <?rfc include="reference.RFC.5920"?>
   <?rfc include="reference.RFC.5921"?>
   <?rfc include="reference.RFC.6073"?>
   <?rfc include="reference.RFC.6003"?>
   <?rfc include="reference.RFC.8306"?>
   <?rfc include="reference.RFC.8660"?>
   <?rfc include="reference.I-D.ietf-detnet-ip"?>
   <?rfc include="reference.I-D.ietf-detnet-ip-over-mpls"?>
   <?rfc include="reference.I-D.ietf-detnet-mpls-over-tsn"?>
   <?rfc include="reference.I-D.ietf-detnet-security"?>
   <?rfc include="reference.I-D.ietf-detnet-yang"?>
   <reference anchor="IEEE802.1AE-2018"
      target="https://ieeexplore.ieee.org/document/8585421">
      <front>
        <title>IEEE Std 802.1AE-2018 MAC Security (MACsec)</title>
        <author>
          <organization>IEEE Standards Association</organization>
        </author>
        <date year="2018" />
      </front>
    </reference>
   <reference anchor="IEEE802.1CB-2017"
      target="https://ieeexplore.ieee.org/document/8091139">
      <front>
        <title>IEEE Std 802.1CB-2017 Frame Replication and Elimination for
		Reliability</title>
        <author>
          <organization>IEEE Standards Association</organization>
        </author>
        <date year="2017" />
      </front>
    </reference>

     </references>

  </back>
</rfc>