<?xml version="1.0" encoding="US-ASCII"?>
<!-- This template is for creating an Internet Draft using xml2rfc,
 which is available here: http://xml.resource.org. --> version='1.0' encoding='utf-8'?>
<!DOCTYPE rfc SYSTEM "rfc2629.dtd" [
]>
<?xml-stylesheet type='text/xsl' href='rfc2629.xslt' ?> "rfc2629-xhtml.ent">

<rfc xmlns:xi="http://www.w3.org/2001/XInclude" number="8626" docName="draft-ietf-detnet-architecture-13" submissionType="IETF" category="std" consensus="yes" ipr="trust200902" docName="draft-ietf-detnet-architecture-13"> obsoletes="" updates="" xml:lang="en" sortRefs="true" tocInclude="true" symRefs="true" version="3">
  <!-- xml2rfc v2v3 conversion 2.31.0 -->
  <?xml-stylesheet type='text/xsl' href='rfc2629.xslt' ?>

  <?rfc sortrefs="yes"?>
  <?rfc iprnotified-"no" ?>
  <?rfc authorship="yes"?>
  <?rfc tocappendix="yes"?>
  <?rfc strict="yes" ?>
  <!-- give errors regarding ID-nits and DTD validation -->
  <!-- control the table of contents (ToC) -->
  <?rfc toc="yes"?>
  <!-- generate a ToC -->
  <?rfc tocdepth="4"?>
  <!-- the number of levels of subsections in ToC. default: 3 -->
  <!-- control references -->
  <?rfc symrefs="yes"?>
  <!-- use symbolic references tags, i.e, [RFC2119] instead of [1] -->
  <?rfc sortrefs="yes" ?>
  <!-- sort the reference entries alphabetically -->
  <!-- control vertical white space
   (using these PIs as follows is recommended by the RFC Editor) -->
  <?rfc compact="no" ?>
  <front>
    <title>Deterministic Networking Architecture</title>
    <seriesInfo name="RFC" value="8626"/>
    <author initials="N" surname="Finn" fullname="Norman Finn" > Finn">
      <organization>
    Huawei
      </organization>
      <address>
        <postal>
          <street>3101 Rio Way</street>
          <city>Spring Valley</city>
          <region>California</region>
          <code>91977</code>
		<country>US</country>
          <country>United States of America</country>
        </postal>
        <phone>+1 925 980 6430</phone>
		<email>norman.finn@mail01.huawei.com</email>
        <email>nfinn@nfinnconsulting.com</email>
      </address>
    </author>
    <author initials="P" surname="Thubert" fullname="Pascal Thubert">
      <organization abbrev="Cisco">
		Cisco Systems
      </organization>
      <address>
        <postal>
          <street>Village d'Entreprises Green Side</street>
          <street>400, Avenue de Roumanille</street>
          <street>Batiment T3</street>
          <city>Biot - Sophia Antipolis</city>
          <code>06410</code>
		  <country>FRANCE</country>
          <country>France</country>
        </postal>
        <phone>+33 4 97 23 26 34</phone>
        <email>pthubert@cisco.com</email>
      </address>
    </author>
    <author fullname="Bal&aacute;zs fullname="Balázs Varga" initials="B." surname="Varga">
      <organization>Ericsson</organization>
      <address>
        <postal>
          <street>Magyar tudosok korutja 11</street>
          <city>Budapest</city>
          <country>Hungary</country>
          <code>1117</code>
        </postal>
        <email>balazs.a.varga@ericsson.com</email>
      </address>
    </author>
    <author fullname="J&aacute;nos fullname="János Farkas" initials="J." surname="Farkas">
      <organization>Ericsson</organization>
      <address>
        <postal>
          <street>Magyar tudosok korutja 11</street>
          <city>Budapest</city>
          <country>Hungary</country>
          <code>1117</code>
        </postal>
        <email>janos.farkas@ericsson.com</email>
      </address>
    </author>
	<date/>
    <date year="2019" month="September"/>
    <area>Internet</area>
    <workgroup>DetNet</workgroup>
    <keyword>&gt;TSN, Bounded Lantency, Reliable Networking, Available Networking</keyword>
    <abstract>
      <t>
		This document provides the overall architecture for
		Deterministic Networking (DetNet), which provides a capability
		to carry specified unicast or multicast data flows for
		real-time applications with extremely low data loss rates and
		bounded latency within a network domain.  Techniques used include:
		include 1) reserving data plane data-plane resources for individual (or
		aggregated) DetNet flows in some or all of the intermediate
		nodes along the path of the flow; flow, 2) providing explicit routes
		for DetNet flows that do not immediately change with the
		network topology; topology, and 3) distributing data from DetNet flow
		packets over time and/or space to ensure delivery of each
		packet's data in spite of the loss of a path.  DetNet operates
		at the IP layer and delivers service over lower layer lower-layer
		technologies such as MPLS and IEEE 802.1 Time-Sensitive
		Networking (TSN). (TSN) as defined by IEEE 802.1.
      </t>
    </abstract>
  </front>
  <middle>

	<!-- **************************************************************** -->
	<!-- **************************************************************** -->
	<!-- **************************************************************** -->
	<!-- **************************************************************** -->
    <section anchor='introduction' title="Introduction"> anchor="introduction" numbered="true" toc="default">
      <name>Introduction</name>
      <t>
        This document provides the overall architecture for Deterministic
        Networking (DetNet), which provides a capability for the delivery of
        data flows with extremely low packet loss rates and bounded end-to-end
        delivery latency.  DetNet is for networks that are under a single
        administrative control or within a closed group of administrative
        control; these include campus-wide networks and private WANs. DetNet
        is not for large groups of domains such as the Internet.
	  </t><t>  </t>
      <t>
        DetNet operates at the IP layer and delivers service over
        lower layer lower-layer
        technologies such as MPLS and IEEE 802.1 Time-Sensitive Networking
        (TSN).

DetNet accomplishes these goals provides a reliable and available service by dedicating network
resources such as link bandwidth and buffer space to DetNet flows and/or
classes of DetNet flows, and by replicating packets along multiple paths.
Unused reserved resources are available to non-DetNet packets as long as all
guarantees are fulfilled.
	  </t><t>  </t>
      <t> The <xref target="I-D.ietf-detnet-problem-statement">Deterministic target="RFC8557" format="default">"Deterministic
Networking Problem Statement</xref> Statement"</xref> introduces Deterministic Networking, DetNet, and <xref target="I-D.ietf-detnet-use-cases">Deterministic target="RFC8578" format="default">"Deterministic Networking Use Cases</xref> Cases"</xref> summarizes the
need for it.  See <xref target="I-D.ietf-detnet-dp-sol-mpls"/> and
		<xref target="I-D.ietf-detnet-dp-sol-ip"/> target="I-D.ietf-detnet-data-plane-framework" format="default"/> for specific techniques
that can be used to identify DetNet flows and assign them to specific paths
through a network.
	  </t><t>  </t>
      <t> A goal of DetNet is a converged network in all respects
respects, including the convergence of sensitive non-IP networks onto a common
network infrastructure.  The presence of DetNet flows does not preclude
non-DetNet flows, and the benefits offered DetNet flows should not, except in
extreme cases, prevent existing Quality of
		Service Quality-of-Service (QoS) mechanisms from
operating in a normal fashion, subject to the bandwidth required for the
DetNet flows.  A single source-destination pair can trade both DetNet and
non-DetNet flows.  End systems and applications need not instantiate special
interfaces for DetNet flows.  Networks are not restricted to certain
topologies; connectivity is not restricted.  Any application that generates a
data flow that can be usefully characterized as having a maximum bandwidth
should be able to take advantage of DetNet, as long as the necessary resources
can be reserved.

Reservations can be made by the application itself, via network management,
centrally by an application's controller, or by other means, e.g., for instance, by
placing on-demand reservation via a dynamic control plane
		(e.g., distributed Control Plane, e.g.,
leveraging the Resource Reservation Protocol (RSVP) <xref target="RFC2205"/>). target="RFC2205" format="default"/>.

QoS requirements of DetNet flows can be met if all network
nodes in a DetNet domain implement DetNet capabilities. DetNet nodes can be
interconnected with different sub-network technologies (<xref target="sec_dt_dp"/>), target="sec_dt_dp" format="default"/>) where the nodes of the subnet are not DetNet aware
(<xref target="netref"/>).
	  </t><t> target="netref" format="default"/>).  </t>
      <t> Many applications that are intended to be
served by Deterministic Networking DetNet require the ability to synchronize the clocks in end systems
to a sub-microsecond accuracy.  Some of the queue control queue-control techniques defined
in <xref target="QueuingModels"/> target="QueuingModels" format="default"/> also require time synchronization among
network nodes.  The means used to achieve time synchronization are not
addressed in this document.  DetNet can accommodate various time synchronization
time-synchronization techniques and profiles that are defined elsewhere to
address the needs of different market segments.
      </t>
    </section>
    <section title="Terminology">
	  <section title="Terms used numbered="true" toc="default">
      <name>Terminology</name>
      <section numbered="true" toc="default">
        <name>Terms Used in this document"> This Document</name>
        <t>
			The following terms are used in the context of DetNet in this document:
			<list hangIndent="8" style="hanging">
			  <t hangText="allocation"><vspace blankLines="0"/>
				  Resources are dedicated
        </t>
        <dl newline="true" spacing="normal" indent="3">
          <dt>allocation</dt>
          <dd>
			  The dedication of resources
			  to support a DetNet flow. Depending on an
			  implementation, the resource may be reused by
			  non-DetNet flows when it is not used by the DetNet
			  flow.
			  </t>
			  <t hangText="App-flow"><vspace blankLines="0"/>
			  </dd>
          <dt>App-flow</dt>
          <dd>
				The payload (data) carried over a DetNet service.
			  </t>
			  <t hangText="DetNet
			  </dd>
          <dt>DetNet compound flow and DetNet member flow"><vspace blankLines="0"/> flow</dt>
          <dd> A DetNet compound
			  flow is a DetNet flow that has been separated into
			  multiple duplicate DetNet member flows for service
			  protection at the DetNet service sub-layer.  Member
			  flows are merged back into a single DetNet compound
			  flow such that there are no duplicate packets.
			  "Compound" and "member" are strictly relative to
			  each other, not absolutes; a DetNet compound flow
			  comprising multiple DetNet member flows can, in
			  turn, be a member of a higher-order compound.
			  </t>
			  <t hangText="DetNet destination"><vspace blankLines="0"/>
			  </dd>
          <dt>DetNet destination</dt>
          <dd>
				An end system capable of terminating a DetNet flow.
			  </t>
			  <t hangText="DetNet domain"><vspace blankLines="0"/>
			  </dd>
          <dt>DetNet domain</dt>
          <dd>
			  The portion of a network that
			  is DetNet aware.  It includes end systems and DetNet
			  nodes.
			  </t>
			  <t hangText="DetNet
			  </dd>
          <dt>DetNet edge node"><vspace blankLines="0"/> node</dt>
          <dd> An instance
			  of a DetNet relay node that acts as a source and/or
			  destination at the DetNet service sub-layer. For
			  example, it can include a DetNet service sub-layer
			  proxy function for DetNet service protection (e.g.,
			  the addition or removal of packet sequencing
			  information) for one or more end systems, it can start
			  or starts or terminates terminate resource allocation at the DetNet
			  forwarding sub-layer, or aggregates it can aggregate DetNet services
			  into new DetNet flows.  It is analogous to a Label
			  Edge Router (LER) or a Provider Edge (PE) router.
			  </t>
			  <t hangText="DetNet flow"><vspace blankLines="0"/>
			  </dd>
          <dt>DetNet flow</dt>
          <dd>
				A DetNet flow is a sequence of packets which conform that conforms uniquely
				to a flow identifier, identifier and to which the DetNet service is to be
				provided. It includes any DetNet headers added to support the
				DetNet service and forwarding sub-layers.
			  </t>
			  <t hangText="DetNet
			  </dd>
          <dt>DetNet forwarding sub-layer"><vspace blankLines="0"/> sub-layer</dt>
          <dd>
                           DetNet functionality is divided into two sub-layers.  One of
                           them is the DetNet forwarding sub-layer, which optionally
				  provides resource allocation for DetNet flows over paths
				  provided by the underlying network.
			  </t>
			  <t hangText="DetNet
			  </dd>
          <dt>DetNet intermediate node"><vspace blankLines="0"/> node</dt>
          <dd>
				  A DetNet relay node or DetNet transit node.
			  </t>
			  <t hangText="DetNet node"><vspace blankLines="0"/>
			  </dd>
          <dt>DetNet node</dt>
          <dd> A
			   DetNet edge node, a DetNet relay
			  node, or a DetNet transit node.
			  </t>
			  <t hangText="DetNet
			  </dd>
          <dt>DetNet relay node"><vspace blankLines="0"/> node</dt>
          <dd> A DetNet
			  node including that includes a service
			  sub-layer function that interconnects different
			  DetNet forwarding sub-layer paths to provide service
			  protection.  A DetNet relay node participates in the
			  DetNet service sub-layer.  It typically incorporates
			  DetNet forwarding sub-layer functions as well, in
			  which case it is collocated with a transit node.
			  </t>
			  <t hangText="DetNet
			  </dd>
          <dt>DetNet service sub-layer"><vspace blankLines="0"/> sub-layer</dt>
          <dd>
           DetNet functionality is divided into two sub-layers.  One of
           them is the DetNet service sub-layer, at which a DetNet service,
				  e.g.,
           service protection (e.g., service protection) is provided.
			  </t>
			  <t hangText="DetNet
			  </dd>
          <dt>DetNet service proxy"><vspace blankLines="0"/>
				  Maps proxy</dt>
          <dd>
		          A proxy that maps between App-flows and DetNet flows.
			  </t>
			  <t hangText="DetNet source"><vspace blankLines="0"/>
			  </dd>
          <dt>DetNet source</dt>
          <dd>
				  An end system capable of originating a DetNet flow.
			  </t>
			  <t hangText="DetNet system"><vspace blankLines="0"/>
			  </dd>
          <dt>DetNet system</dt>
          <dd>
				  A DetNet aware DetNet-aware end system, transit node, or relay node.
				  "DetNet" may be omitted in some text.
			  </t>
			  <t hangText="DetNet
			  </dd>
          <dt>DetNet transit node"><vspace blankLines="0"/> node</dt>
          <dd>
                          A DetNet node node, operating at the DetNet forwarding sub-layer,
			  that utilizes link layer link-layer and/or network layer network-layer
			  switching across multiple links and/or sub-networks
			  to provide paths for DetNet service sub-layer
			  functions.
				  Typically  It typically provides resource allocation
			  over those paths.  An MPLS LSR Label Switch Router (LSR) is an example of a
			  DetNet transit node.
			  </t>
			  <t hangText="DetNet-UNI"><vspace blankLines="0"/>
			  </dd>
          <dt>DetNet-UNI</dt>
          <dd>
			  A User-to-Network Interface (UNI) with DetNet specific DetNet-specific
			  functionalities. It is a packet-based reference
			  point and may provide multiple functions like
			  encapsulation, status, synchronization, etc.
			  </t>
			  <t hangText="end system"><vspace blankLines="0"/>
			  </dd>
          <dt>end system</dt>
          <dd>
			  Commonly called a "host" in IETF documents, the RFC series and an
			  "end station" is in IEEE 802 documents. standards. End systems of
			  interest to this document are either sources or
			  destinations of DetNet flows.
				  And end system flows, and they may or may
			  not be aware of DetNet forwarding sub-layer aware sub-layers or
			  DetNet service sub-layer aware.
			  </t>
			  <t hangText="link"><vspace blankLines="0"/> sub-layers.
			  </dd>
          <dt>link</dt>
          <dd>
			  A connection between two DetNet nodes.  It may be
			  composed of a physical link or a sub-network
			  technology that can provide appropriate traffic
			  delivery for DetNet flows.
			  </t>
    		  <t hangText="PEF">
     		    A Packet
			  </dd>
          <dt>Packet Elimination Function (PEF) (PEF)</dt>
          <dd>
     		    A function that eliminates duplicate
     		    copies of packets to prevent excess packets flooding the
     		    network or duplicate packets being sent out of the DetNet
     		    domain.  A PEF can be implemented by a DetNet edge node, a
     		    DetNet relay node, or an end system.
   		      </t>

    		  <t hangText="PRF">
    		    A Packet
   		      </dd>
          <dt>Packet Replication Function (PRF) (PRF)</dt>
          <dd>
    		    A function that replicates DetNet flow
    		    packets and forwards them to one or more next hops in the
    		    DetNet domain.  The number of packet copies sent to the
    		    next hops is a parameter specific to the DetNet flow
				specific parameter at the point
    		    of replication. A PRF can be implemented by a DetNet edge
    		    node, a DetNet relay node, or an end system.
    		  </t>

    		  <t hangText="PREOF">
    		    Collective
    		  </dd>
          <dt>PREOF</dt>
          <dd>
    		    A collective name for Packet Replication, Elimination, and Ordering Functions.
    		  </t>

    		  <t hangText="POF">
    		    A Packet
    		  </dd>
          <dt>Packet Ordering Function (POF) re-orders (POF)</dt>
          <dd>
    		    A function that reorders packets within
    		    a DetNet flow that are received out of order.  This
    		    function can be implemented by a DetNet edge node, a
    		    DetNet relay node, or an end system.
    		  </t>

			  <t hangText="reservation"><vspace blankLines="0"/>
    		  </dd>
          <dt>reservation</dt>
          <dd>
			  The set of resources allocated between a source and
			  one or more destinations through DetNet nodes and
			  subnets associated with a DetNet flow, flow in order to provide
			  the provisioned DetNet service.
			  </t>
			</list>
		  </t>
			  </dd>
        </dl>
      </section>
      <section title="IEEE 802.1 numbered="true" toc="default">
        <name>Dictionary of Terms Used by TSN to DetNet dictionary"> and DetNet</name>
        <t>
			  This section also serves as a dictionary for translating from the
			  terms used by the Time-Sensitive Networking (TSN) Task Group
			  <xref target="IEEE802.1TSNTG"/> target="IEEE802.1TSNTG" format="default"/> of the IEEE 802.1 WG to those of
			  the DetNet WG.
			  <list hangIndent="8" style="hanging">
				  <t hangText="Listener"><vspace blankLines="0"/> Deterministic Networking (detnet) WG of the IETF.
        </t>
        <dl newline="true" spacing="normal" indent="3">
          <dt>Listener</dt>
          <dd>
					  The term used by IEEE 802.1 term for a destination of a DetNet flow.
				  </t>
				  <t hangText="relay system"><vspace blankLines="0"/>
				  </dd>
          <dt>Relay system</dt>
          <dd>
					  The term used by IEEE
					  802.1 term for a DetNet intermediate node.
				  </t>
				  <t hangText="Stream"><vspace blankLines="0"/>
				  </dd>
          <dt>Stream</dt>
          <dd>
					  The term used by IEEE 802.1 term for a DetNet flow.
				  </t>
				  <t hangText="Talker"><vspace blankLines="0"/>
				  </dd>
          <dt>Talker</dt>
          <dd>
					  The term used by IEEE
					  802.1 term for the source of a DetNet flow.
				  </t>
			  </list>
		  </t>
				  </dd>
        </dl>
      </section>
    </section>
    <section anchor="ProvidingQoS" title="Providing numbered="true" toc="default">
      <name>Providing the DetNet Quality of Service"> Service</name>
      <section anchor="DefiningGoals" title="Primary goals defining numbered="true" toc="default">
        <name>Primary Goals Defining the DetNet QoS"> QoS</name>
        <t>
			The DetNet Quality of Service QoS can be expressed in terms of:
			<list style="symbols">
			<t>
        </t>
        <ul spacing="normal">
          <li>
			Minimum and maximum end-to-end latency from source to destination;
			destination, timely delivery, and bounded jitter
			(packet delay variation) derived from these
			constraints.
			</t>
			<t>
			</li>
          <li>
			Packet loss ratio, ratio under various assumptions as to the operational
			states of the nodes and links.
			</t><t>
			</li>
          <li>
			An upper bound on out-of-order packet delivery. It is worth noting
			that some DetNet applications are unable to tolerate any
			out-of-order delivery.
            </t>
			</list>
		</t><t>
            </li>
        </ul>
        <t>
			It is a distinction of DetNet that it is concerned solely with
			worst-case values for the end-to-end latency, jitter, and
			misordering. Average, mean, or typical values are of little
			interest, because they do not affect the ability of a real-time
			system to perform its tasks. In general, a trivial priority-based
			queuing scheme will give better average latency to a data flow than
			DetNet; however, it may not be a suitable option for DetNet because
			of its worst-case latency.
		</t><t>
        </t>
        <t>
			Three techniques are used by DetNet to provide these qualities of service:
			<list style="symbols">
				<t>
        </t>
        <ul spacing="normal">
          <li>
					Resource allocation (<xref target="Zero"/>).
				</t><t> target="Zero" format="default"/>)
				</li>
          <li>
					Service protection (<xref target="srvcprot"/>).
				</t><t> target="srvcprot" format="default"/>)
				</li>
          <li>
					Explicit routes (<xref target="pinned"/>).
				</t>
			</list>
		</t><t> target="pinned" format="default"/>)
				</li>
        </ul>
        <t>
            Resource allocation operates by assigning resources, e.g., buffer
            space or link bandwidth, to a DetNet flow (or flow aggregate) along
            its path. Resource allocation greatly reduces, or even eliminates
            entirely, packet loss due to output packet contention within the
            network, but it can only be supplied to a DetNet flow that is limited
            at the source to a maximum packet size and transmission rate.

			As DetNet flows are assumed to be rate-limited rate limited and DetNet is designed
			to provide sufficient allocated resources (including provisioned
			capacity), the use of transport layer transport-layer congestion control
			<xref target="RFC2914"/> target="RFC2914" format="default"/> for App-flows is not required; however,
			if resources are allocated appropriately, use of congestion control
			should not impact transmission negatively.

		</t><t>

        </t>
        <t> Resource allocation addresses two of the DetNet QoS requirements: latency
and packet loss. Given that DetNet nodes have a finite amount of buffer space,
resource allocation necessarily results in a maximum end-to-end
latency. It Resource allocation also addresses contention related contention-related packet loss.
		</t><t>
</t>
        <t> Other important contribution contributions to packet loss are random media errors
and equipment failures.  Service protection is the name for the mechanisms
used by DetNet to address these losses.  The mechanisms employed are
constrained by the requirement need to meet the users' latency requirements. Packet
replication and elimination (<xref target="srvcprot"/>) target="Seamless" format="default"/>) and packet encoding
(<xref target="PacketEncoding"/>) target="PacketEncoding" format="default"/>) are described in this document to provide
service protection; others protection, but other mechanisms may also be found. For instance,
packet encoding can be used to provide service protection against random media
errors, while packet replication and elimination can be used to provide
service protection against equipment failures. This mechanism distributes the
contents of DetNet flows over multiple paths in time and/or space, so that the
loss of some of the paths does need not cause the loss of any packets.
		</t><t>
</t>
        <t> The paths are typically (but not necessarily) explicit routes, routes so that
they do not normally suffer temporary interruptions caused by the convergence
of routing or bridging protocols.
		</t><t>  </t>
        <t> These three techniques can be applied independently, giving individually or applied together; it
results that eight possible combinations, including none (no DetNet), although some combinations are
possible. Some combinations, however, are of wider utility than others.  This
separation keeps the protocol stack coherent and maximizes interoperability
with existing and developing standards in this (IETF) the IETF and other Standards
Development Organizations.  Some  The following are examples of typical expected
combinations:
			<list style="symbols">
				<t>
					Explicit
        </t>
        <ul spacing="normal">
          <li>
	The combination of explicit routes plus and service protection are exactly is the techniques technique
	employed by seamless redundancy mechanisms applied on a ring topology
					as described, topology,
	e.g., as described in <xref target="IEC62439-3-2016"/>. target="IEC-62439-3" format="default"/>. In this
	example, explicit routes are achieved by limiting the physical
	topology of the network to a ring. Sequentialization, replication, and
	duplicate elimination are facilitated by packet tags added at the
	front or the end of Ethernet frames. <xref target="RFC8227"/> target="RFC8227" format="default"/> provides
	another example in the context of MPLS.
				</t><t>  </li>
          <li> Resource allocation
	alone was originally offered by IEEE 802.1 Audio Video bridging Bridging as defined by IEEE 802.1 <xref target="IEEE802.1BA"/>. target="IEEE802.1BA" format="default"/>.  As long as the network suffers no failures,
	packet loss due to output packet contention can be eliminated through
	the use of a reservation protocol (e.g., the Multiple Stream Registration
	Protocol <xref target="IEEE802.1Q-2018"/>), target="IEEE802.1Q" format="default"/>), shapers in every bridge,
	and proper dimensioning.
				</t><t>  </li>
          <li> Using all three together gives
	maximum protection.
				</t>
			</list>
		</t><t>
				</li>
        </ul>
        <t> There are, of course, simpler methods available (and employed, employed today) to
achieve levels of latency and packet loss that are satisfactory for many
applications.  Prioritization and over-provisioning is one such technique.
However, these methods generally work best in the absence of any significant
amount of non-critical noncritical traffic in the network (if, indeed, such traffic is
supported at all), or all). They may also work only if the critical traffic constitutes only a small portion of
the network's theoretical capacity, or work only if all systems are functioning properly,
or in the absence of if actions by end systems that disrupt the network's operations.
		</t><t>
operations are absent.  </t>
        <t> There are any number of methods in use, defined, or in progress for
accomplishing each of the above techniques.  It is expected that this the DetNet Architecture
architecture defined in this document will assist various vendors, users, and/or "vertical" Standards
Development Organizations (dedicated to a single industry) to make in making selections
among the available means of implementing DetNet networks.
        </t>
      </section>
      <section anchor="Techniques" title="Mechanisms numbered="true" toc="default">
        <name>Mechanisms to achieve Achieve DetNet QoS"> QoS</name>
        <section anchor="Zero" title="Resource allocation"> numbered="true" toc="default">
          <name>Resource Allocation</name>
          <section anchor="ZeroL" title="Eliminate contention loss"> numbered="true" toc="default">
            <name>Eliminate Contention Loss</name>
            <t>
			The primary means by which DetNet achieves its QoS
			assurances is to reduce, or even completely eliminate eliminate,
			packet loss due to output packet contention within a
			DetNet node as a cause of packet loss.  This can be
			achieved only by the provision of sufficient buffer
			storage at each node through the network to ensure
			that no packets are dropped due to a lack of buffer
			storage.  Note that App-flows are generally not
			expected to be responsive to implicit <xref target="RFC2914"/> target="RFC2914" format="default"/> or explicit congestion notification
			<xref target="RFC3168"/>.

		</t><t> target="RFC3168" format="default"/>.

            </t>
            <t> Ensuring adequate buffering requires, in turn, that
		the source, source and every DetNet node along the path to the
		destination (or nearly every node, node; see <xref target="Incomplete"/>) target="Incomplete" format="default"/>) be careful to regulate its output to
		not exceed the data rate for any DetNet flow, except for brief
		periods when making up for interfering traffic.  Any packet
		sent ahead of its time potentially adds to the number of
		buffers required by the next hop next-hop DetNet node and may thus
		exceed the resources allocated for a particular DetNet
		flow. Furthermore, rate limiting, e.g., limiting (e.g., using traffic
			policing policing)
		and shaping functions, e.g., functions (e.g., shaping as defined in <xref target="RFC2475"/>, target="RFC2475" format="default"/>) at the
		ingress of the DetNet domain must be applied. This is needed
		for meeting the requirements of DetNet flows as well as for
		protecting non-DetNet traffic from potentially misbehaving
		DetNet traffic sources. Note that large buffers have some issues, see,
		issues (see, e.g., <xref target="BUFFERBLOAT"/>.
		</t><t> target="BUFFERBLOAT" format="default"/>).  </t>
            <t> The low-level mechanisms described in <xref target="QueuingModels"/> target="QueuingModels" format="default"/>
provide the necessary regulation of transmissions by an end system or DetNet
node to provide resource allocation.  The allocation of the bandwidth and
buffers for a DetNet flow requires provisioning.  A DetNet node may have other
resources requiring allocation and/or scheduling, scheduling that might otherwise be
over-subscribed and trigger the rejection of a reservation.
            </t>
          </section>
          <section anchor="Jitterless" title="Jitter Reduction"> numbered="true" toc="default">
            <name>Jitter Reduction</name>
            <t>
       A core objective of DetNet is to enable the convergence of sensitive non-IP networks
       onto a common network infrastructure. This requires the accurate emulation
       of currently deployed mission-specific networks, which which,
       for example example, rely on point-to-point analog (e.g., 4-20mA modulation) and
       serial-digital cables (or buses) for highly reliable, synchronized synchronized, and
       jitter-free communications. While the latency of analog transmissions is
       basically the speed of light, legacy serial links are usually slow (in the
       order of Kbps) compared to, say, Gigabit Ethernet, and some latency is usually
       acceptable. What is not acceptable is the introduction of excessive jitter,
       which may, for instance, affect the stability of control systems.
            </t>
            <t>Applications that are designed to operate on serial links usually do
       not provide services to recover the jitter, because jitter simply does not
       exist there. DetNet flows are generally expected to be delivered in-order in order,
       and the precise time of reception influences the processes. In order to
       converge such existing applications,
       there is a desire to emulate all properties of the serial cable, such
       as clock transportation, perfect flow isolation isolation, and fixed latency. While minimal
       jitter (in the form of specifying minimum, as well as maximum, end-to-end latency)
       is supported by DetNet, there are practical limitations on packet-based networks
       in this regard. In general, users
       are encouraged to use a combination of:
       <list style="symbols">
		 <t>
            </t>
            <ul spacing="normal">
              <li>
			 Sub-microsecond time synchronization among all source and destination
			 end systems, and
		 </t><t>
		 </li>
              <li>
			 Time-of-execution fields in the application packets.
		 </t>
       </list>
       </t><t>
		 </li>
            </ul>
            <t>
	     Jitter reduction is provided by the mechanisms described in <xref target="QueuingModels"/> target="QueuingModels" format="default"/>
	     that also provide resource allocation.
            </t>
          </section>
        </section>
        <section anchor="srvcprot" title="Service Protection"> numbered="true" toc="default">
          <name>Service Protection</name>
          <t>
	    Service protection aims to mitigate or eliminate packet loss due
	    to equipment failures, including random media and/or memory
	    faults. These types of packet loss can be greatly reduced by
	    spreading the data over multiple disjoint forwarding
	    paths. Various service protection methods are described in <xref target="RFC6372"/>, target="RFC6372" format="default"/>, e.g., 1+1 linear protection.
		This section describes the  The functional
	    details of an additional method are described in <xref target="Seamless"/>, target="Seamless" format="default"/>, which can be implemented as described in
	    <xref target="PacketEncoding"/> target="PacketEncoding" format="default"/> or as specified in <xref target="I-D.ietf-detnet-dp-sol-mpls"/> target="I-D.ietf-detnet-mpls" format="default"/> in order to provide 1+n hitless
	    protection. The appropriate service protection mechanism depends
	    on the scenario and the requirements.
          </t>
          <section anchor="inorder" title="In-Order Delivery"> numbered="true" toc="default">
            <name>In-Order Delivery</name>
            <t>
        Out-of-order packet delivery can be a side effect of service
        protection.  Packets delivered out-of-order out of order impact the amount of
        buffering needed at the destination to properly process the received
        data. Such packets also influence the jitter of a flow. The guarantees
        of a DetNet service includes include a maximum allowed amount of misordering as a
	constraint. Zero misordering would be a valid service constraint to
        reflect that the end system(s) of the flow cannot tolerate any
        out-of-order delivery. A DetNet Packet Ordering
		Functionality Function (POF)
        (<xref target="Seamless"/>) target="Seamless" format="default"/>) can be used to provide in-order delivery.
            </t>
          </section>
          <section anchor="Seamless" title="Packet numbered="true" toc="default">
            <name>Packet Replication and Elimination"> Elimination</name>
            <t>
         This section describes a service protection method that sends copies
         of the same packets over multiple paths.
		</t><t>  </t>
            <t> The DetNet service
         sub-layer includes the packet replication (PRF), the
		    packet elimination (PEF), PRF, PEF, and the packet ordering functionality (POF) POF for use in DetNet edge,
         relay node, and end system end-system packet processing.  These functions can be
         enabled in a DetNet edge node, relay
			node node, or end system. The
         collective name for all three functions is Packet Replication,
         Elimination, and Ordering Functions (PREOF).  The packet replication
         and elimination service protection method altogether involves four
         capabilities:
			<list style="symbols">
			  <t>
				Providing sequencing
            </t>
            <ul spacing="normal">
              <li>
				Sequencing information is provided to the
				packets of a DetNet compound flow.  This may
				be done by adding a sequence number or time
				stamp as part of DetNet, or it may be inherent
				in the packet, e.g., in a higher layer protocol, higher-layer
				protocol or associated to other physical
				properties such as the precise time (and radio
				channel) of reception of the packet.  This is
				typically done once, at or near the source.
			  </t><t>
				</li>
              <li> The Packet Replication Function (PRF) PRF
				replicates these packets into multiple DetNet
				member flows and typically sends them along
				multiple different paths to the
				destination(s), e.g., over the explicit routes of
				described in <xref target="pinned"/>. target="pinned" format="default"/>. The location
				within a DetNet node, node and the mechanism used
				for the PRF is are left open for implementations.
			  </t><t>
				</li>
              <li> The Packet Elimination Function (PEF) PEF
				eliminates duplicate packets of a DetNet flow
				based on the sequencing information and a
				history of received packets. The output of the
				PEF is always a single packet.  This may be
				done at any DetNet node along the path to save
				network resources further downstream, in
				particular if multiple
				Replication replication points
				exist. But the most common case is to perform
				this operation at the very edge of the DetNet
				network, preferably in or near the
				receiver. The location within a DetNet node, node
				and the mechanism used for the PEF is left open
				for implementations.
			  </t><t>  </li>
              <li> The Packet Ordering Function (POF)
				POF uses the sequencing
				information to re-order reorder a DetNet flow's packets
				that are received out of order.
			  </t>
			</list>
		</t><t>
			  </li>
            </ul>
            <t> The order in which a DetNet node applies PEF, POF, and
		PRF to a DetNet flow is left open for implementations.
		</t><t>
            </t>
            <t> Some service protection mechanisms rely on switching from one flow to
another when a failure of a flow is detected. Contrarily, packet replication
and elimination combines the DetNet member flows sent along multiple different paths,
paths and performs a packet-by-packet selection of which to discard, e.g.,
based on sequencing information.
		</t><t>
			In
</t>
            <t>In the simplest case, this amounts to 1) replicating each packet in a
source that has two interfaces, interfaces and 2) conveying them through the network, network along
separate (Shared Risk Link Group (SRLG) disjoint) paths, paths to the similarly
dual-homed destinations, destinations that 3) reorder the packets and 4) discard the extras.
duplicates. This ensures that one path
remains, even if some DetNet intermediate node fails.  The sequencing
information can also be used for loss detection and for re-ordering.
		</t><t> reordering.  </t>
            <t>
DetNet relay nodes in the network can provide replication and elimination
facilities at various points in the network, network so that multiple failures can be
accommodated.
		</t><t>  </t>
            <t> This is shown in <xref target="FigSeamless"/>, target="FigSeamless" format="default"/>, where the two relay nodes
each replicate (R) the DetNet flow on input, sending the DetNet member flows
to both the other relay node and to the end system, and eliminate duplicates
(E) on the output interface to the right-hand end system.  Any one link in the
network can fail, and the DetNet compound flow can still get through.
Furthermore, two links can fail, as long as they are in different segments of
the network.
            </t>
            <figure align="center" anchor="FigSeamless"
title="Packet replication anchor="FigSeamless">
              <name>Packet Replication and elimination"> Elimination</name>
              <artwork align="left"><![CDATA[ align="left" name="" type="" alt=""><![CDATA[
             > > > > > > > > > relay > > > > > > > >
            > /------------+ R node E +------------\ >
           > /                  v + ^               \ >
   end    R +                   v | ^                + E end
   system   +                   v | ^                +   system
           > \                  v + ^               / >
            > \------------+ R relay E +-----------/ >
             > > > > > > > > >  node > > > > > > > >
]]></artwork> >]]></artwork>
            </figure>
		<t>
			Packet
            <t>Packet replication and elimination does not react to and correct failures;
it is entirely passive.  Thus, intermittent failures, mistakenly created
packet filters, or misrouted data is handled just the same as the equipment
failures that are handled by typical routing and bridging protocols.
		</t><t>
			If  </t>
            <t>If member flows that take different-length paths through the network are
combined, a merge point may require extra buffering to equalize the delays
over the different paths.  This equalization ensures that the resultant
compound flow will not exceed its contracted bandwidth even after one or the other of the
paths is restored after a failure. The extra buffering can be also used to
provide in-order delivery.
            </t>
          </section>
          <section anchor="PacketEncoding" title="Packet encoding numbered="true" toc="default">
            <name>Packet Encoding for service protection"> Service Protection</name>
            <t>
              There are methods for using multiple paths to provide service protection
              that involve encoding the information in a
              packet belonging to a DetNet flow into multiple transmission units,
              combining information from multiple packets into any given transmission unit.
              Such techniques, also known as "network coding",
              can be used as a DetNet service protection technique.
            </t>
          </section>
        </section>
        <section anchor="pinned" title="Explicit routes"> numbered="true" toc="default">
          <name>Explicit Routes</name>
          <t>
In networks controlled by typical dynamic control protocols such as IS-IS or
OSPF, a network topology event in one part of the network can impact, at least
briefly, the delivery of data in parts of the network remote from the failure
or recovery event. Even the use of redundant paths through a network, e.g., as
defined by <xref target="RFC6372"/> do target="RFC6372" format="default"/>, does not eliminate the chances of packet
loss. Furthermore, out-of-order packet delivery can be a side effect of route
changes.
		</t><t>  </t>
          <t> Many real-time networks rely on physical rings of two-port
devices, with a relatively simple ring control protocol.  This supports
redundant paths for service protection with a minimum of wiring.  As an
additional benefit, ring topologies can often utilize different topology
management protocols than from those used for a mesh network, with a consequent
reduction in the response time to topology changes.  Of course, this comes at
some cost in terms of increased hop count, and thus latency, for the typical
path.
		</t><t>  </t>
          <t> In order to get the advantages of low hop count and still ensure against
even very brief losses of connectivity, DetNet employs explicit routes, routes where
the path taken by a given DetNet flow does not change, at least immediately, not
immediately and likely not at all, in response to network topology events.
Service protection (<xref target="srvcprot"/> or (see Sections <xref target="PacketEncoding"/>) target="srvcprot" format="counter"/>
and <xref target="PacketEncoding" format="counter"/>) over explicit routes
provides a high likelihood of continuous connectivity.  Explicit routes can be
established in various ways, e.g., with RSVP-TE <xref target="RFC3209"/>, target="RFC3209" format="default"/>, with
Segment Routing (SR) <xref target="RFC8402"/>, target="RFC8402" format="default"/>, via a Software
			Defined Networking SDN approach <xref target="RFC8453"/>, target="RFC8453" format="default"/>, with IS-IS <xref target="RFC7813"/>, target="RFC7813" format="default"/>, etc.  Explicit routes
are typically used in MPLS TE LSPs.
		</t><t> (Traffic Engineering) Label Switched Paths
(LSPs). </t>
          <t> Out-of-order packet delivery can be a side effect of distributing a single
flow over multiple paths, especially when there is a change from one path to
another when combining the flow. This is irrespective of the distribution
method used, used and also applies to service protection over explicit routes. As
described in <xref target="inorder"/>, target="inorder" format="default"/>, out-of-order packets influence the
jitter of a flow and impact the amount of buffering needed to process the
data; therefore, the guarantees of a DetNet service includes include a maximum
			allowed amount
of misordering as a constraint. The use of explicit routes helps to provide in-order delivery
because there is no immediate route change with the network topology, but the
changes are plannable as they are between the different explicit routes.

          </t>
        </section>
      </section>
      <section anchor="MoreGoals" title="Secondary goals numbered="true" toc="default">
        <name>Secondary Goals for DetNet"> DetNet</name>
        <t>
        Many applications require DetNet to provide additional services, including coexistence with
        other QoS mechanisms <xref target="Coexistence"/> (<xref target="Coexistence" format="default"/>) and protection against misbehaving transmitters
        <xref target="FaultMitigation"/>.
        (<xref target="FaultMitigation" format="default"/>).
        </t>
        <section anchor="Coexistence" title="Coexistence numbered="true" toc="default">
          <name>Coexistence with normal traffic"> Normal Traffic</name>
          <t>
              A DetNet network supports the dedication of a high proportion of
              the network bandwidth to DetNet flows.  But, no matter how much
              is dedicated for DetNet flows, it is a goal of DetNet to coexist
              with existing Class of Service Class-of-Service schemes (e.g., DiffServ).  It is
              also important that non-DetNet traffic not disrupt the DetNet
              flow, of course (see Sections <xref target="FaultMitigation"/> target="FaultMitigation" format="counter"/> and <xref target="SecurityConsiderations"/>). target="SecurityConsiderations" format="counter"/>).  For these reasons:
              <list style="symbols">
                  <t>
                      Bandwidth
          </t>
          <ul spacing="normal">
            <li>Bandwidth (transmission opportunities) not utilized by a DetNet flow is
available to non-DetNet packets (though not to other DetNet flows).
                  </t><t>  </li>
            <li> DetNet flows can be shaped or scheduled, in order to ensure that the
highest-priority non-DetNet packet is also ensured a worst-case latency.
                  </t><t>  </li>
            <li> When transmission opportunities for DetNet flows are scheduled in detail, then
the algorithm constructing the schedule should leave sufficient
opportunities for non-DetNet packets to satisfy the needs of the users of the
network.  Detailed scheduling can also permit the time-shared use of buffer
resources by different DetNet flows.
                  </t>
              </list>
          </t><t>
                  </li>
          </ul>
          <t> Starvation of non-DetNet traffic must be avoided, e.g., for example, by
          traffic policing and shaping functions (e.g., <xref target="RFC2475"/>). target="RFC2475" format="default"/>). Thus, the net effect of the presence of DetNet
          flows in a network on the non-DetNet flows is primarily a reduction
          in the available bandwidth.
          </t>
        </section>
        <section anchor="FaultMitigation" title="Fault Mitigation"> numbered="true" toc="default">
          <name>Fault Mitigation</name>
          <t>
              Robust real-time systems require reducing the number of possible
              failures. Filters and policers should be used in a DetNet
              network to detect if DetNet packets are received on the wrong
              interface, or at the wrong time, or in too great a volume.
              Furthermore, filters and policers can take actions to discard
              the offending packets or flows, or trigger shutting down the
              offending flow or the offending interface.
          </t><t>  </t>
          <t> It is also
              essential that filters and service remarking be employed at the
              network edge to prevent non-DetNet packets from being mistaken
              for DetNet packets, packets and thus impinging on the resources
              allocated to DetNet packets. In particular, sending DetNet
              traffic into networks that have not been provisioned in advance
              to handle that DetNet traffic has to be treated as a fault.  The
              use of egress traffic filters, or equivalent mechanisms, to
              prevent this from happening are strongly recommended at the
              edges of a DetNet networks and DetNet supporting networks.  In
              this context, the term 'provisioned' has a broad meaning, e.g.,
              provisioning could be performed via an administrative decision
              that the downstream network has the available capacity to carry
              the DetNet traffic that is being sent into it.
          </t><t>  </t>
          <t>

   			  Note that the sending of App-flows that do not use transport layer
   			  transport-layer congestion control per <xref target="RFC2914"/> target="RFC2914" format="default"/> into a network that is not
   			  provisioned to handle such DetNet traffic has to be treated
   			  as a fault and prevented. PRF generated PRF-generated DetNet
   			  member flows also need to be treated as not using transport layer
   			  transport-layer congestion control even if the
   			  original App-flow supports transport layer transport-layer
   			  congestion control because PREOF can remove
   			  congestion indications at the PEF and thereby hide
   			  such indications (e.g., drops, ECN markings,
   			  increased latency) from end systems.

          </t><t>

          </t>
          <t> The mechanisms to support these requirements are both data plane Data
          Plane and implementation specific.  Data plane  Solutions that are data-plane
          specific solutions will be specified in the relevant data plane data-plane solution
          document. There also exist techniques, at present and/or in various
          stages of standardization, that can support these fault mitigation fault-mitigation
          tasks that deliver a high probability that misbehaving systems will
          have zero impact on well-behaved DetNet flows, except flows with the exception, of
          course, for of the receiving interface(s) immediately downstream of from
          the misbehaving device.  Examples of such techniques include traffic
          policing and shaping functions (e.g., those described in <xref target="RFC2475"/>) and target="RFC2475" format="default"/>),
          separating flows into per-flow rate-limited queues, and potentially apply
          applying active queue management <xref target="RFC7567"/>. target="RFC7567" format="default"/>.
          </t>
        </section>
      </section>
    </section>
    <section anchor="arch" title="DetNet Architecture"> numbered="true" toc="default">
      <name>DetNet Architecture</name>
      <section anchor="Stacks" title="DetNet stack model"> numbered="true" toc="default">
        <name>DetNet Stack Model</name>
        <t>
		  DetNet functionality (<xref target="ProvidingQoS"/>) target="ProvidingQoS" format="default"/>) is implemented
		  in two adjacent sub-layers in the protocol stack: the DetNet service
		  sub-layer and the DetNet forwarding sub-layer. The DetNet service sub-layer
		  provides DetNet service, e.g., service protection,  to higher layers
		  in the protocol stack and applications. The DetNet forwarding sub-layer
		  supports DetNet service in the underlying network, e.g., by
		  providing explicit routes and resource allocation to DetNet flows.
        </t>
        <section anchor="StackModel" title="Representative numbered="true" toc="default">
          <name>Representative Protocol Stack Model"> Model</name>
          <t>
                <xref target="ProtStack1"/> target="ProtStack1" format="default"/> illustrates a conceptual DetNet data plane data-plane layering model.
                One may compare it to that in <xref target="IEEE802.1CB"/>, target="IEEE802.1CB" format="default"/>, Annex C.
          </t>
          <figure align="center" anchor="ProtStack1"
title="DetNet data plane protocol stack"> anchor="ProtStack1">
            <name>DetNet Data-Plane Protocol Stack</name>
            <artwork align="center"><![CDATA[ align="center" name="" type="" alt=""><![CDATA[
   |  packets going  |        ^  packets coming   ^
   v down the stack  v        |   up the stack    |
+-----------------------+   +-----------------------+
|        Source         |   |      Destination      |
+-----------------------+   +-----------------------+
|   Service sub-layer:  |   |   Service sub-layer:  |
|   Packet sequencing   |   | Duplicate elimination |
|    Flow replication   |   |      Flow merging     |
|    Packet encoding    |   |    Packet decoding    |
+-----------------------+   +-----------------------+
| Forwarding sub-layer: |   | Forwarding sub-layer: |
|  Resource allocation  |   |  Resource allocation  |
|    Explicit routes    |   |    Explicit routes    |
+-----------------------+   +-----------------------+
|     Lower layers      |   |     Lower layers      |
+-----------------------+   +-----------------------+
            v                           ^
             \_________________________/
]]></artwork>
          </figure>
          <t>
                Not all sub-layers are required for any given application, or even for any
                given network. The functionality shown in <xref target="ProtStack1"/> target="ProtStack1" format="default"/> is:
                <list hangIndent="8" style="hanging">
                    <t hangText="Application"><vspace blankLines="0"/>
          </t>
          <dl newline="true" spacing="normal" indent="3">
            <dt>Application</dt>
            <dd>
                        Shown as "source" and "destination" in the diagram.
                    </t>
                    <t hangText="Packet sequencing"><vspace blankLines="0"/>
                    </dd>
            <dt>Packet sequencing</dt>
            <dd>
                    As part of the DetNet service protection, sub-layer, the packet
                    sequencing function supplies the sequence number for
                    packet replication and elimination for DetNet service
                    protection (<xref target="srvcprot"/>), thus  peers
                        with Duplicate target="Seamless" format="default"/>); thus, its peer is
                    duplicate elimination.  This sub-layer is not needed if a higher layer
                    higher-layer protocol is expected to perform any packet
                    sequencing and duplicate elimination required by the
                    DetNet flow replication.
                    </t>
                    <t hangText="Duplicate elimination"><vspace blankLines="0"/>
                    </dd>
            <dt>Duplicate elimination</dt>
            <dd>
                        As part of the DetNet service sub-layer, based on the sequenced sequence number
                        supplied by its peer, packet sequencing,
                        Duplicate peer (packet sequencing),
                        duplicate elimination discards any duplicate packets generated by DetNet
                        flow replication.  It can operate on member flows, compound flows, or both.
                        The replication may also be inferred from other
                        information such as the precise time of reception in a scheduled network.
                        The duplicate elimination sub-layer may also perform resequencing of packets
                        to restore packet order in a
                        flow that was disrupted by the loss of packets on one or another of
                        the multiple paths taken.
                    </t>
                    <t hangText="Flow replication"><vspace blankLines="0"/>
                    </dd>
            <dt>Flow replication</dt>
            <dd> As
                    part of DetNet service protection, packets that belong to
                    a DetNet compound flow are replicated into two or more
                    DetNet member flows.  This function is separate from
                    packet sequencing.  Flow replication can be an explicit
                    replication and remarking of packets, packets or can be performed
                    by, for example, techniques similar to ordinary multicast
                    replication, albeit with resource allocation implications.
                        Peers with
                    Its peer is DetNet flow merging.
                    </t>
                    <t hangText="Flow merging"><vspace blankLines="0"/>
                    </dd>
            <dt>Flow merging</dt>
            <dd> As
                    part of the DetNet service protection, merges sub-layer, the flow merging
                    function combines DetNet member flows together for packets
                    coming up the stack belonging to a specific DetNet
                    compound flow.
                        Peers with DetNet flow replication. DetNet flow merging, together with packet
                    sequencing, duplicate elimination, and DetNet flow
                    replication perform packet replication and elimination
                    (<xref target="srvcprot"/>).
                    </t>
                    <t hangText="Packet encoding"><vspace blankLines="0"/> target="srvcprot" format="default"/>). Its peer is DetNet flow
                    replication.
                    </dd>
            <dt>Packet encoding</dt>
            <dd> As
                    part of DetNet service protection, as an alternative to
                    packet sequencing and flow replication, packet encoding
                    combines the information in multiple DetNet packets,
                    perhaps from different DetNet compound flows, and
                    transmits that information in packets on different DetNet
                    member Flows.  Peers with Packet flows.  Its peer is packet decoding.
                    </t>
                    <t hangText="Packet decoding"><vspace blankLines="0"/>
                    </dd>
            <dt>Packet decoding</dt>
            <dd> As
                    part of DetNet service protection, as an alternative to
                    flow merging and duplicate elimination, packet decoding
                    takes packets from different DetNet member
                        flows, flows and
                    computes from those packets the original DetNet packets
                    from the compound flows input to packet encoding.  Peers with Packet  Its
                    peer is packet encoding.
                    </t>
                    <t hangText="Resource allocation"><vspace blankLines="0"/>
                    </dd>
            <dt>Resource allocation</dt>
            <dd>
                    The DetNet forwarding sub-layer provides resource
                    allocation. See <xref target="QueuingModels"/>. target="QueuingModels" format="default"/>.  The
                    actual queuing and shaping mechanisms are typically
                    provided by the underlying subnet.  These can be closely
                    associated with the means of providing paths for DetNet
                    flows.  The path and the resource allocation are conflated
                    in this figure.
                    </t>
					<t hangText="Explicit routes"><vspace blankLines="0"/>
                        The
                    </dd>
            <dt>Explicit routes</dt>
            <dd>
		Explicit routes are arrangements of fixed paths operated at
		the DetNet forwarding sub-layer provides mechanisms to ensure that fixed paths are provided
                        for DetNet flows. These explicit paths avoid the determined in advance
		to avoid the impact of network convergence.
                    </t>

                </list>
            </t> convergence on DetNet flows.
                    </dd>
          </dl>
          <t>
            Operations, Administration, and Maintenance (OAM) leverages
            in-band and out-of-band signaling that validates whether the
            service is effectively obtained within QoS constraints.  OAM is
            not shown in <xref target="ProtStack1"/>; target="ProtStack1" format="default"/>; it may reside in any
            number of the layers.  OAM can involve specific tagging added in
            the packets for tracing implementation or network configuration
            errors; traceability enables to find finding whether a packet is a
            replica, which DetNet relay node performed the replication, and
            which segment was intended for the replica. Active and hybrid OAM
            methods require additional bandwidth to perform fault management
            and performance monitoring of the DetNet domain. OAM may, for
            instance, generate special test probes or add OAM information into
            the data packet.
          </t>
          <t>
                The packet sequencing and replication and elimination functions may be
                performed either at the source and destination ends of a
                DetNet compound flow may be performed either
                in the end system or in a DetNet relay node.
          </t>
        </section>
        <section title="DetNet Data Plane Overview" anchor="sec_dt_dp"> anchor="sec_dt_dp" numbered="true" toc="default">
          <name>DetNet Data-Plane Overview</name>
          <t>
            A "Deterministic Network" will be composed of DetNet enabled DetNet-enabled end
            systems, DetNet edge nodes, and DetNet relay nodes, which
            collectively deliver DetNet services.  DetNet relay and edge nodes
            are interconnected via DetNet transit nodes (e.g., LSRs) LSRs), which
            support DetNet, DetNet but are not DetNet service aware. All DetNet nodes
            are connected to sub-networks, where a point-to-point link is also
            considered as a simple sub-network. These sub-networks will provide DetNet compatible
            DetNet-compatible service for support of DetNet traffic.  Examples
            of sub-network technologies include MPLS TE, IEEE 802.1 TSN as defined by
            IEEE 802.1, and
            OTN. OTN (Optical Transport Network).  Of course, multi-layer
            multilayer DetNet systems may also be possible, where one DetNet
            appears as a sub-network, sub-network and provides service to, to a higher layer higher-layer
            DetNet system. A simple DetNet concept network is shown in <xref target="fig_detnet"/>. target="fig_detnet" format="default"/>.

Note that in this and following figures figures, "Forwarding" and "Fwd" refer to the
DetNet forwarding sub-layer, and "Service" and "Svc" refer to the DetNet
service sub-layer,
            which sub-layer; both of these sub-layers are described in detail in <xref target="Stacks"/>. target="StackModel" format="default"/>.
          </t>
          <figure anchor="fig_detnet" align="center"
title="A anchor="fig_detnet">
            <name>A Simple DetNet Enabled Network"> DetNet-Enabled Network</name>
            <artwork align="center"><![CDATA[ align="center" name="" type="" alt=""><![CDATA[
TSN               Edge        Transit         Relay        DetNet
End System        Node         Node           Node        End System

+----------+   +.........+                               +----------+
|  Appl.   |<--:Svc Proxy:-- End to End End-to-End Service -------->|  Appl.   |
+----------+   +---------+                 +---------+   +----------+
|   TSN    |   |TSN| |Svc|<- DetNet flow --: Service :-->| Service  |
+----------+   +---+ +---+   +--------+    +---------+   +----------+
|Forwarding|   |Fwd| |Fwd|   |  Fwd   |    |Fwd| |Fwd|   |Forwarding|
+-------.--+   +-.-+ +-.-+   +--.----.+    +-.-+ +-.-+   +---.------+
        :  Link  :    /  ,-----. \   : Link  :    /  ,-----.  \
        +........+    +-[  Sub  Sub- ]-+  +.......+    +-[  Sub  Sub- ]-+
                        [Network]                   [Network]
                        [network]                   [network]
                         `-----'                     `-----'
]]></artwork>
          </figure>
          <t>
            DetNet data plane Data Plane is divided into two sub-layers: the DetNet
            service sub-layer and the DetNet forwarding sub-layer. This helps
            to explore and evaluate various combinations of the data
			plane data-plane
            solutions available. Some of them are illustrated in <xref target="fig_adaptation"/>. target="fig_adaptation" format="default"/>.  This separation of DetNet sub-layers,
            while helpful, should not be considered as a formal requirement.
            For example, some technologies may violate these strict sub-layers
            and still be able to deliver a DetNet service.
          </t>
          <figure anchor="fig_adaptation" align="center"
title="DetNet adaptation anchor="fig_adaptation">
            <name>DetNet Adaptation to data plane"> Data Plane</name>
            <artwork align="center"><![CDATA[ align="center" name="" type="" alt=""><![CDATA[
              .
              .
+-----------------------------+
|  DetNet Service sub-layer   | PW, UDP, GRE
+-----------------------------+
| DetNet Forwarding sub-layer | IPv6, IPv4, MPLS TE LSPs, MPLS SR
+-----------------------------+
              .
              .
]]></artwork>
          </figure>
          <t>
            In some networking scenarios, the end system initially provides a
            DetNet flow encapsulation, which contains all information needed
            by DetNet nodes (e.g., DetNet flow based on the Real-time Transport
            Protocol (RTP) <xref target="RFC3550"/> based
            DetNet flow target="RFC3550" format="default"/> that is carried over a
            native UDP/IP network or PseudoWire). pseudowire (PW)). In other scenarios, the
            encapsulation formats might differ significantly.
          </t>
          <t>
            There are many valid options to create a data plane data-plane solution for DetNet
            traffic by selecting a technology approach for the DetNet service sub-layer and
            also selecting a technology approach for the DetNet forwarding sub-layer. There are
            a large number of valid combinations.
          </t>
          <t>
            One of the most fundamental differences between different
            potential
            data plane data-plane options is the basic headers used by DetNet
            nodes.  For example, the basic service can be delivered based on
            an MPLS label or an IP header. This decision impacts the basic
            forwarding logic for the DetNet service sub-layer. Note that in
            both cases, IP addresses are used to address DetNet nodes.  The
            selected DetNet forwarding sub-layer technology also needs to be
            mapped to the sub-net subnet technology used to interconnect DetNet
            nodes. For example, DetNet flows will need to be mapped to TSN
            Streams.
          </t>
        </section>
        <section anchor="netref" title="Network reference model"> numbered="true" toc="default">
          <name>Network Reference Model</name>
          <t> <xref target="fig_DetNetservice"/> target="fig_DetNetservice" format="default"/> shows another view of the
			DetNet service related service-related reference points and main components.
          </t>
          <figure anchor="fig_DetNetservice" align="center"
title="DetNet anchor="fig_DetNetservice">
            <name>DetNet Service Reference Model (multi-domain)">
<artwork><![CDATA[ (Multidomain)</name>
            <artwork name="" type="" align="left" alt=""><![CDATA[
DetNet                                                     DetNet
end system                                                 end system
End System                                                 End System
   _                                                             _
  / \     +----DetNet-UNI (U)                                   / \
 /App\    |                                                    /App\
/-----\   |                                                   /-----\
| NIC |   v         ________                                  | NIC |
+--+--+   _____    /        \             DetNet-UNI (U) --+  +--+--+
   |     /     \__/          \                             |     |
   |    / +----+    +----+    \_____                       |     |
   |   /  |    |    |    |          \_______               |     |
   +------U PE +----+ P  +----+             \          _   v     |
       |  |    |    |    |    |              |     ___/ \        |
       |  +--+-+    +----+    |       +----+ |    /      \_      |
       \     |                |       |    | |   /         \     |
        \    |   +----+    +--+-+  +--+PE  |------         U-----+
         \   |   |    |    |    |  |  |    | |   \_      _/
          \  +---+ P  +----+ P  +--+  +----+ |     \____/
           \___  |    |    |    |           /
               \ +----+__  +----+     DetNet-1    DetNet-2
   |            \_____/  \___________/                           |
   |                                                             |
   |      |     End-to-End service Service         |     |         |     |
   <------------------------------------------------------------->
   |      |     DetNet service Service             |     |         |     |
   |      <------------------------------------------------>     |
   |      |                                |     |         |     |
]]></artwork>
          </figure>
          <t>DetNet User Network User-to-Network Interfaces (DetNet-UNIs) ("U" in <xref target="fig_DetNetservice"/>) target="fig_DetNetservice" format="default"/>) are assumed in this document to be
            packet-based reference points and provide connectivity over the
            packet network. A DetNet-UNI may provide multiple functions, e.g., functions. For
	    example, it may add may:
</t>
          <ul spacing="normal">
            <li>add encapsulation specific to networking technology specific encapsulation to the DetNet flows if necessary; it may provide necessary,
</li>
            <li>provide status of the availability of the resources associated with a reservation; it may provide reservation,
</li>
            <li>provide a synchronization service for the end system; it may carry system, or
</li>
            <li>carry enough signaling to place the reservation in a network without a controller,
controller or if in a network where the controller only deals with the network
but not the end systems.
</li>
          </ul>
          <t>
           Internal reference points of end systems (between the application
           and the NIC) Network Interface Card (NIC)) are more challenging from the
           control perspective perspective, and they may have extra requirements (e.g.,
           in-order delivery is expected in end system internal reference
           points, whereas it is considered optional over the DetNet-UNI).</t>
        </section>
      </section>
      <section anchor="netrefsys" title="DetNet systems"> numbered="true" toc="default">
        <name>DetNet Systems</name>
        <section anchor="es" title="End system"> numbered="true" toc="default">
          <name>End System</name>
          <t>
		The traffic characteristics of an App-flow can be CBR
		(constant bit rate) or VBR (variable bit rate) and can have Layer-1 or Layer-2
		Layer 1, Layer 2, or Layer-3 Layer 3 encapsulation (e.g., TDM
		(time-division multiplexing), multiplexing) Ethernet, IP). These
		characteristics are considered as input for resource
		reservation and might be simplified to ensure determinism
		during packet forwarding (e.g., making reservations for the
		peak rate of VBR traffic, etc.).
          </t>
          <t>
		An end system may or may not be aware of the DetNet forwarding sub-layer aware
		or DetNet service sub-layer aware. sub-layer. That is, an end
		system may or may not contain DetNet specific DetNet-specific
		functionality. End systems with DetNet functionalities may
		have the same or different forwarding sub-layer as the
		connected DetNet domain. Categorization of end systems are
		shown in <xref target="fig_endsys2"/>. target="fig_endsys2" format="default"/>.
          </t>
          <figure anchor="fig_endsys2" align="center"
title="Categorization anchor="fig_endsys2">
            <name>Categorization of end systems">
<artwork><![CDATA[ End Systems</name>
            <artwork name="" type="" align="left" alt=""><![CDATA[
             End system
                 |
                 |
                 |  DetNet aware ?
                / \
        +------<   >------+
     NO |       \ /       | YES
        |        v        |
 DetNet unaware
 DetNet-unaware           |
   End system             |
                          | Service/Forwarding
                          |  sub-layer
                         / \  aware ?
               +--------<   >-------------+
       f-aware |         \ /              | s-aware
               |          v               |
               |          | both          |
               |          |               |
       DetNet f-aware     |        DetNet s-aware
         End system       |         End system
                          v
                    DetNet sf-aware
                      End system
]]></artwork>
          </figure>
          <t>
		Note
		The following are some known use case examples for end systems:
	     <list style="symbols">
			<t>
          </t>
          <ul spacing="normal">
            <li> DetNet unaware: The classic case requiring service proxies.</t>
			<t> proxies.</li>
            <li> DetNet f-aware: A system that is aware of the DetNet forwarding sub-layer aware system.
			 sub-layer. It knows about some TSN
			functions (e.g., reservation), reservation) but not about service
			protection. </t>
			<t> </li>
            <li> DetNet s-aware: A system that is aware of the DetNet service sub-layer aware system.
			sub-layer. It supplies sequence numbers, numbers but
			doesn't know about resource allocation. </t>
			<t> </li>
            <li> DetNet sf-aware: A full fully functioning DetNet end system, it
			system. It has DetNet functionalities and usually the
			same forwarding paradigm as the connected DetNet
			domain. It can be treated as an integral part of the
			DetNet domain. </t>
		</list>
		</t> </li>
          </ul>
        </section>
        <section anchor="ertn" title="DetNet edge, relay, numbered="true" toc="default">
          <name>DetNet Edge, Relay, and transit nodes"> Transit Nodes</name>
          <t>
            As shown in <xref target="fig_detnet"/>, target="fig_detnet" format="default"/>, DetNet edge nodes
            providing proxy service and DetNet relay nodes providing the
            DetNet service sub-layer are
            DetNet-aware, DetNet aware, and DetNet transit
            nodes need only be aware of the DetNet forwarding sub-layer.
          </t><t>
          </t>
          <t> In general, if a DetNet flow passes through one or more
            DetNet-unaware network nodes between two DetNet nodes providing
            the DetNet forwarding sub-layer for that flow, there is a
            potential for disruption or failure of the DetNet QoS.  A network
            administrator needs to 1) ensure that the DetNet-unaware network
            nodes are configured to minimize the chances of packet loss and
            delay,
            delay and 2) provision enough extra buffer space in the DetNet
            transit node following the DetNet-unaware network nodes to absorb
            the induced latency variations.
          </t>
        </section>
      </section>
      <section anchor="DetNetFlows" title="DetNet flows"> numbered="true" toc="default">
        <name>DetNet Flows</name>
        <section anchor="DetNetFlowsTypes" title="DetNet flow types"> numbered="true" toc="default">
          <name>DetNet Flow Types</name>
          <t>
 A DetNet flow can have different formats while its packets are forwarded
 between the peer end systems depending on the type of the end
 systems. Corresponding to the end system types, the following possible types / formats
 types/formats of a DetNet flow are distinguished in this document. The
 different flow types have different requirements to DetNet nodes.
                 <list style="symbols">
                    <t>
          </t>
          <ul spacing="normal">
            <li> App-flow: the The payload (data) carried over a DetNet
                    flow between DetNet unaware DetNet-unaware end systems. An app-flow App-flow does
                    not contain any DetNet related DetNet-related attributes and does not
                    imply any specific requirement on DetNet nodes.</t>
                    <t> nodes.</li>
            <li> DetNet-f-flow: The specific format of a DetNet flow. It
                    only requires the resource allocation features provided by
                    the DetNet forwarding sub-layer. </t>
                     <t> </li>
            <li> DetNet-s-flow: The specific format of a DetNet flow. It
                     only requires the service protection feature ensured by
                     the DetNet service sub-layer. </t>
                     <t> </li>
            <li> DetNet-sf-flow: The specific format of a DetNet
                     flow. It requires both the DetNet service sub-layer and
                     the DetNet forwarding sub-layer functions during
                     forwarding. </t>
                 </list>
              </t> </li>
          </ul>
        </section>
        <section anchor="FlowLimits" title="Source transmission behavior"> numbered="true" toc="default">
          <name>Source Transmission Behavior</name>
          <t>
                  For the purposes of resource allocation,
                  DetNet flows can be synchronous or asynchronous.
                  In synchronous DetNet flows, at least the DetNet nodes (and possibly
                  the end systems) are closely time
                  synchronized, typically to better than 1 microsecond.  By transmitting
                  packets from different DetNet flows or classes of DetNet flows at different times,
                  using repeating schedules synchronized among the DetNet nodes, resources
                  such as buffers and link bandwidth can be shared over the time domain
                  among different DetNet flows.  There is a tradeoff trade-off among techniques for
                  synchronous DetNet flows between the burden of fine-grained scheduling and the
                  benefit of reducing the required resources, especially buffer space.
              </t><t>
          </t>
          <t>
                  In contrast, asynchronous DetNet flows are not coordinated with a fine-grained
                  schedule, so relay and end systems must assume worst-case interference
                  among DetNet flows contending for buffer resources.
                  Asynchronous DetNet flows are characterized by:
                  <list style="symbols">
                      <t>
          </t>
          <ul spacing="normal">
            <li>
                          A maximum packet size;
                      </t><t>
                      </li>
            <li>
                          An observation interval; and
                      </t><t>
                      </li>
            <li>
                          A maximum number of transmissions during that observation interval.
                      </t>
                  </list>
              </t><t>
                      </li>
          </ul>
          <t> These parameters, together with knowledge of the
              protocol stack used (and thus the size of the various headers
              added to a packet), provide the bandwidth that is needed for the
              DetNet flow.
              </t><t>  </t>
          <t> The source is required not to exceed these
              limits in order to obtain DetNet service.  If the source
              transmits less data than this limit allows, then the unused resource resource,
              such as link
                  bandwidth bandwidth, can be made available by the DetNet
              system to non-DetNet packets as long as all guarantees are
              fulfilled.  However, making those resources available to DetNet
              packets in other DetNet flows would serve no purpose.  Those
              other DetNet flows have their own dedicated resources, on the
              assumption that all DetNet flows can use all of their resources
              over a long period of time.

              </t><t>

          </t>
          <t>
     There is no expectation in DetNet for App-flows to be responsive to
     congestion control <xref target="RFC2914"/> target="RFC2914" format="default"/> or explicit congestion
     notification <xref target="RFC3168"/>. target="RFC3168" format="default"/>. The assumption is that a DetNet
     flow, to be useful, must be delivered in its entirety.  That is, while
     any useful application is written to expect a certain number of lost
     packets, the real-time applications of interest to DetNet demand that the
     loss of data due to the network is a rare event.
              </t><t>  </t>
          <t> Although DetNet strives to minimize the changes required of an
 application to allow it to shift from a special-purpose digital network to an
 Internet Protocol network, one fundamental shift in the behavior of network
 applications is impossible to avoid: the reservation of resources before the
 application starts.  In the first place, a network cannot deliver finite
 latency and practically zero packet loss to an arbitrarily high offered load.
 Secondly, achieving practically zero packet loss for DetNet flows means that
 DetNet nodes have to dedicate buffer resources to specific DetNet flows or to
 classes of DetNet flows.  The requirements of each reservation have to be
 translated into the parameters that control each DetNet system's queuing,
 shaping, and scheduling functions functions, and they have to be delivered to the DetNet nodes and end
 systems.
               </t><t>  </t>
          <t> All nodes in a DetNet domain are expected to support the data behavior
required to deliver a particular DetNet service. If a node itself is not
DetNet service aware, the DetNet nodes that are adjacent to such non-DetNet aware nodes them must ensure
that the node that is non-DetNet aware node is provisioned to appropriately support
the DetNet service. For example, an IEEE 802.1 a TSN node (as defined by IEEE 802.1) may be used to
interconnect DetNet aware DetNet-aware nodes, and these DetNet nodes can map DetNet flows
to 802.1 TSN flows. Another As another example, an MPLS-TE or TP MPLS-TP (Transport Profile) domain may be used to
interconnect
				  DetNet aware DetNet-aware nodes, and these DetNet nodes can map DetNet flows
to TE LSPs LSPs, which can provide the QoS requirements of the DetNet service.
          </t>
        </section>
        <section anchor="Incomplete" title="Incomplete Networks"> numbered="true" toc="default">
          <name>Incomplete Networks</name>
          <t>
                  The presence in the network of intermediate nodes or subnets
                  that are not fully capable of offering DetNet services
                  complicates the ability of the intermediate nodes and/or
                  controller to allocate resources, as extra buffering must be
                  allocated at points downstream from the non-DetNet
                  intermediate node for a DetNet flow. This extra buffering
                  may increase latency and/or jitter.
          </t>
        </section>
      </section>
      <section anchor="te" title="Traffic numbered="true" toc="default">
        <name>Traffic Engineering for DetNet"> DetNet</name>
        <t>
         <xref target="TEAS">Traffic target="TEAS" format="default">Traffic Engineering Architecture and Signaling (TEAS)
          </xref> defines traffic-engineering architectures for generic applicability
         across packet and non-packet nonpacket networks.
         From a TEAS perspective, Traffic Engineering (TE) refers to techniques
         that enable operators to control how specific traffic flows are treated
         within their networks.
        </t>
        <t>
         Because if of its very nature of establishing explicit optimized paths,
         Deterministic Networking
         DetNet can be seen as a new, specialized branch of
         Traffic Engineering,
         TE, and it inherits its architecture with a separation
         into planes.
         </t><t>
        </t>
        <t>
         The Deterministic Networking DetNet architecture is thus composed
         of three planes, planes: a (User) Application Plane, a Controller Plane, and a
         Network Plane, which Plane. This echoes that the composition of Figure 1 of
         <xref target="RFC7426"> Software-Defined target="RFC7426" format="default">"Software-Defined Networking (SDN):
         Layers and Architecture Terminology</xref>, Terminology"</xref> and the Controllers controllers
		 identified in <xref target="RFC8453"/> target="RFC8453" format="default"/> and <xref target="RFC7149"/>. target="RFC7149" format="default"/>.
        </t>
        <section anchor="appplane" title="The numbered="true" toc="default">
          <name>The Application Plane"> Plane</name>
          <t>
         Per <xref target="RFC7426"/>, target="RFC7426" format="default"/>,
         the Application Plane includes both applications and services. In particular,
         the Application Plane incorporates the User Agent, a specialized application
         that interacts with the end user / and operator and performs requests for
         Deterministic Networking
         DetNet services via an abstract Flow Management Entity,
         (FME) Entity
         (FME), which may or may not be collocated with (one of) the end systems.
          </t>
          <t>At the Application Plane, a management interface enables
		the negotiation of flows between end systems. An abstraction
		of the flow called a Traffic Specification (TSpec) provides
		the representation. This abstraction is used to place a
		reservation over the (Northbound) Service Interface and within
		the Application plane. Plane.  It is associated with an abstraction
		of location, such as IP addresses and DNS names, to identify
		the end systems and possibly specify DetNet nodes.
          </t>
        </section>
        <section anchor="ctrlplane" title="The numbered="true" toc="default">
          <name>The Controller Plane"> Plane</name>
          <t>
         The Controller Plane corresponds to the aggregation of the Control
         and Management Planes in <xref target="RFC7426"/>, target="RFC7426" format="default"/>, though Common
         Control and Measurement Plane (CCAMP) (as defined by the CCAMP
	 Working Group <xref target="CCAMP"/> target="CCAMP" format="default"/>) makes an
         additional distinction between management and measurement.  When the
         logical separation of the Control, Measurement Measurement, and other Management
         entities is not relevant, the term Controller Plane "Controller Plane" is used for
         simplicity to represent them all, and the term Controller "Controller Plane
         Function (CPF) (CPF)" refers to any device operating in that plane, whether is
         it is a Path Computation Element (PCE) <xref target="RFC4655"/>, or target="RFC4655" format="default"/>, a
         Network Management entity Entity (NME), or a distributed control plane. protocol.

         The CPF is a core element of a controller, in charge of computing Deterministic
         deterministic paths to be applied in the Network Plane.
          </t>
          <t>
         A (Northbound) Service Interface enables applications in the Application
         Plane to communicate with the entities in the Controller Plane as
		 illustrated in <xref target="NorthSouth"/>. target="NorthSouth" format="default"/>.
          </t>
          <t>
         One or more CPF(s) CPFs collaborate to implement the requests from the FME
         as Per-Flow Per-Hop Behaviors per-flow, per-hop behaviors installed in the DetNet nodes for each
         individual flow. The CPFs place each flow along a deterministic sequence
         arrangement of DetNet nodes so as to respect per-flow constraints such
         as security and latency, and to optimize the overall result for metrics
         such as an abstract aggregated cost. The deterministic sequence arrangement can
         typically be more complex than a direct sequence arrangement and include
         redundant paths, paths with one or more packet replication and elimination
         points. Scaling to larger networks is discussed in <xref target="Scaling"/>. target="Scaling" format="default"/>.
          </t>
        </section>
        <section anchor="netplane" title="The numbered="true" toc="default">
          <name>The Network Plane"><t> Plane</name>
          <t>

The Network Plane represents the network devices and protocols as a whole,
regardless of the Layer layer at which the network devices operate. It includes Forwarding the
Data Plane (data plane), Application, and Operational Plane (e.g., OAM) aspects.
</t>
      <t>
         The network
          <t>The Network Plane comprises the Network Interface Cards (NIC) (NICs) in the
end systems, which are typically IP hosts, and DetNet nodes, which
are typically IP routers and MPLS switches.
         Network-to-Network Interfaces such as used for Traffic Engineering
         path reservation in <xref target="RFC5921"/>,
         as well as User-to-Network Interfaces (UNI) such as provided by
         the Local Management Interface (LMI) between network and end systems,
         are both part of the Network Plane, both in the control plane and
		 the data plane.
</t>
          <t>
         A Southbound (Network) Interface enables the entities in the Controller
         Plane to communicate with devices in the Network Plane as illustrated
		 in <xref target="NorthSouth"/>. target="NorthSouth" format="default"/>. This interface leverages and extends
		 TEAS to describe the physical topology and resources in the Network
		 Plane.
          </t>
          <figure align="center" anchor="NorthSouth"
title="Northbound anchor="NorthSouth">
            <name>Northbound and Southbound interfaces"> Interfaces</name>
            <artwork align="left"><![CDATA[ align="left" name="" type="" alt=""><![CDATA[
    End                                                     End
    System                                               System

   -+-+-+-+-+-+-+ Northbound -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-

             CPF         CPF              CPF              CPF

   -+-+-+-+-+-+-+ Southbound -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-

              DetNet     DetNet     DetNet     DetNet
               Node       Node       Node       Node
    NIC                                                     NIC
              DetNet     DetNet     DetNet     DetNet
               Node       Node       Node       Node
]]></artwork>
          </figure>
          <t>
The DetNet nodes (and possibly the end systems NIC) systems' NICs) expose their capabilities
and physical resources to the controller (the CPF), CPF) and update the CPFs with
their dynamic perception of the
			topology, topology across the Southbound Interface. In
return, the CPFs set the per-flow paths up, providing a Flow Characterization
that is more tightly coupled to the DetNet node
			Operation operation than a TSpec.
		</t><t>
</t>
          <t> At the Network plane, Plane, DetNet nodes may exchange information regarding
the state of the paths, between adjacent DetNet nodes and possibly with the
end systems, and forward packets within constraints associated to each flow,
or, when unable to do so, perform a last resort last-resort operation such as drop or
declassify.
		</t><t>  </t>
          <t> This document focuses on the Southbound interface and the
operation of the Network Plane.
          </t>
        </section>
      </section>
      <section anchor="QueuingModels" title="Queuing, numbered="true" toc="default">
        <name>Queuing, Shaping, Scheduling, and Preemption"> Preemption</name>
        <t> DetNet achieves bounded delivery latency by reserving bandwidth and buffer
resources at each DetNet node along the path of the DetNet flow.  The
reservation itself is not sufficient, however.  Implementors and users of a
number of proprietary and standard real-time networks have found that
standards for specific data plane data-plane techniques are required to enable these
assurances to be made in a multi-vendor multivendor network.  The fundamental reason is
that latency variation in one DetNet system results in the need for extra
buffer space in the next-hop DetNet system(s), which in turn, turn increases the
worst-case per-hop latency.
		</t><t>  </t>
        <t> Standard queuing and transmission selection transmission-selection algorithms allow traffic engineering
		  <xref target="te"/> TE
(<xref target="te" format="default"/>) to compute the latency contribution of each
DetNet node to the end-to-end latency, to compute the amount of buffer space
required in each DetNet node for each incremental DetNet flow, and most
importantly, to translate from a flow specification to a set of values for the
managed objects that control each relay or end system.  For example, the IEEE
802.1 WG has specified (and is specifying) a set of queuing, shaping, and
scheduling algorithms that enable each DetNet node, and/or a central
controller, to compute these values.  These algorithms include:
		  <list style="symbols">
			<t>
        </t>
        <ul spacing="normal">
          <li>
  A credit-based shaper <xref target="IEEE802.1Qav"/> (superseded by target="IEEE802.1Qav" format="default"/> (incorporated to <xref target="IEEE802.1Q-2018"/>).
			</t><t> target="IEEE802.1Q" format="default"/>).  </li>
          <li> Time-gated queues governed by a rotating time schedule based on
synchronized time <xref target="IEEE802.1Qbv"/> (superseded by target="IEEE802.1Qbv" format="default"/> (incorporated to <xref target="IEEE802.1Q-2018"/>).
			</t><t> target="IEEE802.1Q" format="default"/>).
  </li>
          <li> Synchronized double (or triple) buffers driven by synchronized time ticks.
<xref target="IEEE802.1Qch"/> (superseded by target="IEEE802.1Qch" format="default"/> (incorporated to <xref target="IEEE802.1Q-2018"/>).
			</t><t>
			  Pre-emption target="IEEE802.1Q" format="default"/>).  </li>
          <li> Preemption of an Ethernet packet in transmission by a packet with a more
stringent latency requirement, followed by the resumption of the preempted
packet <xref target="IEEE802.1Qbu"/> (superseded by target="IEEE802.1Qbu" format="default"/> (incorporated to <xref target="IEEE802.1Q-2018"/>), target="IEEE802.1Q" format="default"/>) <xref target="IEEE802.3br"/> (superseded by target="IEEE802.3br" format="default"/> (incorporated to <xref target="IEEE802.3-2018"/>).
			</t>
		  </list>
		</t><t> target="IEEE802.3" format="default"/>).
			</li>
        </ul>
        <t> While these techniques are currently embedded in
		Ethernet <xref target="IEEE802.3-2018"/> target="IEEE802.3" format="default"/> and bridging
		standards, we can note that they are all, except perhaps for
		packet preemption, equally applicable to other media other than Ethernet,
		Ethernet and to routers as well as bridges. Other media may
		have its their own methods, see, methods (see, e.g., <xref target="I-D.ietf-6tisch-architecture"/>, target="I-D.ietf-6tisch-architecture" format="default"/> and <xref target="RFC7554"/>. target="RFC7554" format="default"/>).
		Further techniques are defined by the IETF, e.g., IETF (e.g., <xref target="RFC8289"/> target="RFC8289" format="default"/> and <xref target="RFC8033"/>. target="RFC8033" format="default"/>).  DetNet may
		include such definitions in the future, future or may define how
		these techniques can be used by DetNet nodes.
        </t>
      </section>
      <section anchor="ServInst" title="Service instance"> numbered="true" toc="default">
        <name>Service Instance</name>
        <t> A Service service instance represents all the functions required
		on a DetNet node to allow the end-to-end service between the
		UNIs.
        </t>
        <t> The DetNet network general reference model is shown in <xref target="fig_DetNetrefmodel"/> target="fig_DetNetrefmodel" format="default"/> for a DetNet service scenario (i.e., between two
DetNet-UNIs). In this figure, end systems ("A" and "B") are connected directly
to the edge nodes of an IP/MPLS network ("PE1" and "PE2"). End systems
participating in DetNet communication may require connectivity before setting
up an App-flow that requires the DetNet service. Such a connectivity related connectivity-related
service instance and the one dedicated for DetNet service share the same
access. Packets belonging to a DetNet flow are selected by a filter configured
on the access ("F1" and "F2"). As a result, data flow specific data-flow-specific access
("access-A + F1" and "access-B + F2") are is terminated in the flow specific flow-specific
service instance ("SI-1" and "SI-2"). A tunnel is used to provide connectivity
between the service instances.
        </t>
        <t> The tunnel is exclusively used for the packets of the DetNet flow between "SI-1" and "SI-2". The service instances are configured to implement DetNet functions and a flow specific flow-specific DetNet forwarding. The service instance and the tunnel may or may not be shared by multiple DetNet flows. Sharing the service instance by multiple DetNet flows requires properly populated forwarding tables of the service instance.
        </t>
        <figure anchor="fig_DetNetrefmodel" align="center"
title="DetNet network general reference model">
<artwork><![CDATA[ anchor="fig_DetNetrefmodel">
          <name>DetNet Network General Reference Model</name>
          <artwork name="" type="" align="left" alt=""><![CDATA[
          access-A                                     access-B
           <----->    <-------- tunnel ---------->     <----->

              +---------+        ___  _        +---------+
End system    |  +----+ |       /   \/ \_      | +----+  | End system
    "A" -------F1+    | |      /         \     | |    +F2----- "B"
              |  |    +========+ IP/MPLS +=======+    |  |
              |  |SI-1| |      \__  Net._/     | |SI-2|  |
              |  +----+ |         \____/       | +----+  |
              |PE1      |                      |      PE2|
              +---------+                      +---------+

]]></artwork>                      +---------+]]></artwork>
        </figure>
        <t> The tunnel between the service instances may have some special
characteristics. For example, in case of a DetNet L3 service, there are
differences in the usage of the PW for DetNet traffic compared to the network
model described in <xref target="RFC6658"/>. target="RFC6658" format="default"/>. In the DetNet scenario, the PW is
likely to be used exclusively by the DetNet flow, whereas <xref target="RFC6658"/> target="RFC6658" format="default"/> states: "The
</t>
<blockquote>
The packet PW appears as a single point-to-point link to the client
layer.  Network-layer adjacency formation and maintenance between the
client equipment equipments will follow the normal practice needed to support
the required relationship in the client layer ... layer.
&br;...
&br;
This packet PseudoWire pseudowire is used to transport all of the required Layer-2 layer 2 and Layer-3
layer 3 protocols between LSR1 and LSR2". LSR2.
</blockquote>

<t>
Further details are network technology specific and can be found in <xref target="I-D.ietf-detnet-dp-sol-mpls"/> and <xref target="I-D.ietf-detnet-dp-sol-ip"/>. target="I-D.ietf-detnet-data-plane-framework" format="default"/>.
        </t>
      </section>
      <section anchor="FlowIDatTechBord" title="Flow identification numbered="true" toc="default">
        <name>Flow Identification at technology borders"> Technology Borders</name>
        <t>This section discusses what needs to be done at technology
	     borders including Ethernet as one of the technologies. Flow
	     identification for MPLS and IP data
		 planes Data Planes are described in <xref target="I-D.ietf-detnet-dp-sol-mpls"/> target="I-D.ietf-detnet-mpls" format="default"/> and <xref target="I-D.ietf-detnet-dp-sol-ip"/>, target="I-D.ietf-detnet-ip" format="default"/>,
	     respectively.
        </t>
        <section anchor="Relayering" title="Exporting flow identification"> numbered="true" toc="default">
          <name>Exporting Flow Identification</name>
          <t>
		  A DetNet node may need to map specific flows to lower layer lower-layer
		  flows (or Streams) in order to provide specific queuing and
		  shaping services for specific flows.  For example:
		  <list style="symbols">
			  <t>
          </t>
          <ul spacing="normal">
            <li>
				A non-IP, strictly L2 source end system X may be sending multiple flows
				to the same L2 destination end system Y.  Those flows may include
				DetNet flows with different QoS requirements, requirements and may include non-DetNet
				flows.
			  </t><t>
			  </li>
            <li>
				A router may be sending any number of flows to another router.
				Again, those flows may include
				DetNet flows with different QoS requirements, requirements and may include non-DetNet
				flows.
			  </t><t>
			  </li>
            <li>
				Two routers may be separated by bridges.  For these bridges to perform
				any required per-flow queuing and shaping, they must be able to identify
				the individual flows.
			  </t><t>
			  </li>
            <li>
				A Label Edge Router (LER) may have a Label Switched
				Path (LSP) set up for handling traffic
				destined for a particular IP address carrying only non-DetNet flows.  If
				a DetNet flow to that same address is requested, a separate LSP may be
				needed,
				needed in order that for all of the Label Switch Routers (LSRs) along the
				path to the destination to give that flow special queuing and shaping.
			  </t>
		  </list>
		</t><t>
			  </li>
          </ul>
          <t> The need for a lower-layer node to be aware of
		individual higher-layer flows is not unique to DetNet.  But,
		given the endless complexity of layering and relayering over
		tunnels that is available to network designers, DetNet needs
		to provide a model for flow identification that is better than
		packet inspection.  That is not to say that packet inspection
		to Layer-4 Layer 4 or Layer-5 Layer 5 addresses will not be used, used or the
		capability standardized; but, however, there are alternatives.
		</t><t>  </t>
          <t>
		A DetNet relay node can connect DetNet flows on different
		paths using different flow identification methods.  For
		example:
		  <list style="symbols">
			<t>
          </t>
          <ul spacing="normal">
            <li>
			  A single unicast DetNet flow passing from router A through a bridged network
			  to router B may be assigned a TSN Stream identifier that is
			  unique within that bridged network.  The bridges can then identify the
			  flow without accessing higher-layer headers.  Of course, the receiving router
			  must recognize and accept that TSN Stream.
			</t><t>
			</li>
            <li>
			  A DetNet flow passing from LSR A to LSR B may be assigned a different
			  label than that used for other flows to the same IP destination.
			</t>
		  </list>
		</t><t>
			</li>
          </ul>
          <t>
		  In any of the above cases, it is possible that an existing DetNet flow can
		  be an aggregate carrying multiple other DetNet flows.  (Not flows (not to be confused
		  with DetNet compound vs. member flows.) flows).  Of course, this requires that the
		  aggregate DetNet flow be provisioned properly to carry the aggregated flows.
		</t><t>
          </t>
          <t>
		  Thus, rather than packet inspection, there is the option to export
		  higher-layer information to the lower layer.  The requirement to support
		  one or the other method for flow identification (or both) is a
		  complexity that is part of DetNet control models.
          </t>
        </section>
        <section anchor="FlowAttrMapping" title="Flow attribute mapping numbered="true" toc="default">
          <name>Flow Attribute Mapping between layers"> Layers</name>
          <t>Forwarding of packets of DetNet flows over multiple
		technology domains may require that lower layers are aware of
		specific flows of higher layers. Such an "exporting of flow
		identification" is needed each time when the forwarding
		paradigm is changed on the forwarding path (e.g., two LSRs are
		interconnected by a an L2 bridged domain, etc.). The three
		representative forwarding methods considered for deterministic networking DetNet
		are:
		<list style="symbols">
			<t>IP routing</t>
			<t>MPLS
          </t>
          <ul spacing="normal">
            <li>IP routing</li>
            <li>MPLS label switching</t>
			<t>Ethernet bridging</t>
		</list> switching</li>
            <li>Ethernet bridging</li>
          </ul>
          <t>
		A packet with corresponding Flow-IDs is illustrated in <xref target="fig_FlowIDStack"/>, target="fig_FlowIDStack" format="default"/>,
		which also indicates where each Flow-ID can be added or removed.
          </t>
          <figure anchor="fig_FlowIDStack" align="center"
title="Packet anchor="fig_FlowIDStack">
            <name>Packet with multiple Flow-IDs">
<artwork><![CDATA[ Multiple Flow-IDs</name>
            <artwork name="" type="" align="left" alt=""><![CDATA[
    add/remove     add/remove
    Eth Flow-ID    IP Flow-ID
        |             |
        v             v
     +-----------------------------------------------------------+
     |      |      |      |                                      |
     | Eth  | MPLS |  IP  |     Application data                 |
     |      |      |      |                                      |
     +-----------------------------------------------------------+
               ^
               |
           add/remove
          MPLS Flow-ID
]]></artwork>
          </figure>
          <t> The additional (domain specific) (domain-specific) Flow-ID can be
		<list style="symbols">
			<t>created be:
          </t>
          <ul spacing="normal">
            <li>created by a domain specific domain-specific function or </t>
			<t>derived </li>
            <li>derived from the Flow-ID added to the App-flow. </t>
		</list> </li>
          </ul>
          <t>
		The Flow-ID must be unique inside a given domain. Note that the Flow-ID added to the App-flow is still present in the packet, but some nodes may lack the function to recognize it; that's why the additional Flow-ID is added.
          </t>
        </section>
        <section anchor="FlowIDMapEx" title="Flow-ID mapping examples"> numbered="true" toc="default">
          <name>Flow-ID Mapping Examples</name>
          <t> IP nodes and MPLS nodes are assumed to be configured to push such an additional (domain specific) (domain-specific) Flow-ID when sending traffic to an Ethernet switch (as shown in the examples below).
          </t>
          <t> <xref target="fig_ippushflowid"/> target="fig_ippushflowid" format="default"/> shows a scenario where an IP end system ("IP-A") is connected via two Ethernet switches ("ETH-n") to an IP router ("IP-1").
          </t>
          <figure  anchor="fig_ippushflowid" align="center"
title="IP nodes interconnected anchor="fig_ippushflowid">
            <name>IP Nodes Interconnected by an Ethernet domain">
<artwork><![CDATA[ Domain</name>
            <artwork name="" type="" align="left" alt=""><![CDATA[
                                  IP domain
               <-----------------------------------------------

        +======+                                       +======+
        |L3-ID |                                       |L3-ID |
        +======+  /\                           +-----+ +======+
                 /  \       Forward as         |     |
                /IP-A\      per ETH-ID         |IP-1 |      Recognize
Push  ------>  +-+----+         |              +---+-+  <----- ETH-ID
ETH-ID           |         +----+-----+            |
                 |         v          v            |
                 |      +-----+    +-----+         |
                 +------+     |    |     +---------+
        +......+        |ETH-1+----+ETH-2|           +======+
        .L3-ID .        +-----+    +-----+           |L3-ID |
        +======+             +......+                +======+
        |ETH-ID|             .L3-ID .                |ETH-ID|
        +======+             +======+                +------+
                             |ETH-ID|
                             +======+

                          Ethernet domain
                        <---------------->
]]></artwork>
          </figure>
          <t> End system "IP-A" uses the original App-flow specific App-flow-specific ID
		("L3-ID"), but as it is connected to an Ethernet domain
		domain, it has to push an Ethernet-domain specific flow-ID Ethernet-domain-specific
		 Flow-ID ("ETH-ID") before sending the packet to "ETH-1" node.
		"ETH-1". Ethernet switch "ETH-1" can recognize the data flow
		based on the "ETH-ID" "ETH-ID", and it does forwarding toward
		"ETH-2". "ETH-2" switches the packet toward the IP
		router. "IP-1" must be configured to receive the Ethernet Flow-ID specific
		Flow-ID-specific multicast
		flow, but (as it is an L3
		node) it decodes the data flow ID based on the "L3-ID" fields
		of the received packet.
          </t>
          <t> <xref target="fig_mplspushflowid"/> target="fig_mplspushflowid" format="default"/> shows a scenario where MPLS domain nodes ("PE-n" and "P-m") are connected via two Ethernet switches ("ETH-n").
          </t>
          <figure  anchor="fig_mplspushflowid" align="center"
title="MPLS nodes interconnected anchor="fig_mplspushflowid">
            <name>MPLS Nodes Interconnected by an Ethernet domain">
<artwork><![CDATA[ Domain</name>
            <artwork name="" type="" align="left" alt=""><![CDATA[
                                 MPLS domain
               <----------------------------------------------->

    +=======+                                  +=======+
    |MPLS-ID|                                  |MPLS-ID|
    +=======+  +-----+                 +-----+ +=======+ +-----+
               |     |   Forward as    |     |           |     |
               |PE-1 |   per ETH-ID    | P-2 +-----------+ PE-2|
Push   ----->  +-+---+        |        +---+-+           +-----+
ETH-ID           |      +-----+----+       |  \ Recognize
                 |      v          v       |   +-- ETH-ID
                 |   +-----+    +-----+    |
                 +---+     |    |     +----+
        +.......+    |ETH-1+----+ETH-2|   +=======+
        .MPLS-ID.    +-----+    +-----+   |MPLS-ID|
        +=======+                         +=======+
        |ETH-ID |         +.......+       |ETH-ID |
        +=======+         .MPLS-ID.       +-------+
                          +=======+
                          |ETH-ID |
                          +=======+
                       Ethernet domain
                     <---------------->
]]></artwork>
          </figure>
          <t> "PE-1" uses the MPLS specific MPLS-specific ID ("MPLS-ID"), but as it
		is connected to an Ethernet domain domain, it has to push an Ethernet-domain specific flow-ID
		Ethernet-domain-specific Flow-ID
		 ("ETH-ID") before sending the
		packet to "ETH-1". Ethernet switch "ETH-1" can recognize the
		data flow based on the "ETH-ID" "ETH-ID", and it does forwarding toward
		"ETH-2". "ETH-2" switches the packet toward the MPLS node
		("P-2"). "P-2" must be configured to receive the Ethernet Flow-ID specific
		Flow-ID-specific multicast flow, but (as it is an MPLS node)
		it decodes the data flow ID based on the "MPLS-ID" fields of
		the received packet.
          </t>
          <t>One can appreciate from the above example that, when the means used for DetNet flow identification
            is altered or exported, the means for encoding the sequence number information must similarly
            be altered or exported.
          </t>
        </section>
      </section>
      <section anchor="Advertising" title="Advertising resources, capabilities numbered="true" toc="default">
        <name>Advertising Resources, Capabilities, and adjacencies"> Adjacencies</name>
        <t>
		  Provisioning of DetNet requires knowledge about:
		  <list style="symbols"><t>
        </t>
        <ul spacing="normal">
          <li>
			Details of the DetNet system's capabilities that are required in order to
			accurately allocate that DetNet system's resources, as well as other DetNet systems'
			resources.  This includes, for example, which specific queuing and
			shaping algorithms are implemented (<xref target="QueuingModels"/>), target="QueuingModels" format="default"/>),
			the number of buffers dedicated for DetNet allocation, and the worst-case
			forwarding delay and misordering.
		  </t><t>
		  </li>
          <li>
			The actual state of a DetNet node's DetNet resources.
		  </t><t>
		  </li>
          <li>
			The identity of the DetNet system's neighbors, neighbors and the characteristics of the
			link(s) between the DetNet systems, including the latency of
			the links (in nanoseconds).
		  </t>
		  </list>
		</t>
		  </li>
        </ul>
      </section>
      <section anchor="Scaling" title="Scaling numbered="true" toc="default">
        <name>Scaling to larger networks"> Larger Networks</name>
        <t>
		  Reservations for individual DetNet flows require considerable state information in
		  each DetNet node, especially when adequate fault mitigation
		  (<xref target="FaultMitigation"/>) target="FaultMitigation" format="default"/>) is required.  The DetNet data plane, Data Plane, in order to
		  support larger numbers of DetNet flows, must support the aggregation of DetNet flows.
		  Such aggregated flows can be viewed by the DetNet nodes' data plane Data Plane
		  largely as individual DetNet flows.  Without such aggregation, the per-relay
		  system may limit the scale of DetNet networks. Example techniques that may be used
		  include MPLS hierarchy and IP DiffServ Code Points (DSCPs).
        </t>
      </section>
      <section anchor="Compatibility" title="Compatibility numbered="true" toc="default">
        <name>Compatibility with Layer-2"> Layer 2</name>
        <t>
 Standards providing similar capabilities for bridged networks (only) have
 been and are being generated in the IEEE 802 LAN/MAN Standards Committee.
 The present architecture describes an abstract model that can be applicable
 both at Layer-2 Layer 2 and Layer-3, Layer 3, and over links not defined by IEEE 802.
		</t><t>
			DetNet enabled  </t>
        <t> DetNet-enabled end systems and DetNet nodes can be interconnected by
sub-networks, i.e., Layer-2 Layer 2 technologies.  These sub-networks will provide
DetNet compatible service for support of DetNet traffic.  Examples of
sub-network technologies include MPLS TE, 802.1 TSN, TSN as defined by IEEE 802.1, and a point-to-point OTN
link.  Of course, multi-layer multilayer DetNet systems may be possible too, where one
DetNet appears as a sub-network, sub-network and provides service to, to a higher layer higher-layer DetNet
system.
        </t>
      </section>
    </section>
    <section anchor="SecurityConsiderations" title="Security Considerations"> numbered="true" toc="default">
      <name>Security Considerations</name>
      <t>
		  Security considerations for DetNet are described in detail
		  in <xref target="I-D.ietf-detnet-security"/>. target="I-D.ietf-detnet-security" format="default"/>. This section considers
		  exclusively security considerations which that are specific to the
		  DetNet architecture.
        </t><t>  </t>
      <t> Security aspects which that are
		  unique to DetNet are those whose aim is to provide the
		  specific quality of service QoS aspects of DetNet, which are
		  primarily to deliver data flows with extremely low packet
		  loss rates and bounded end-to-end delivery latency. A DetNet
		  may be implemented using MPLS and/or IP (including both v4
		  and v6) technologies, technologies and thus inherits the security
		  properties of those technologies at both the data plane Data Plane and
		  the control plane.
        </t><t> Controller Plane.  </t>
      <t> Security considerations for
		  DetNet are constrained (compared to, for example, the open
		  Internet) because DetNet is defined to operate only within a
		  single administrative domain (see Section 1). <xref target="introduction"/>). The primary
		  considerations are to secure the request and control of
		  DetNet resources, maintain confidentiality of data
		  traversing the DetNet, and provide the availability of the
		  DetNet quality of service.
        </t><t> QoS.  </t>
      <t> To secure the request
		  and control of DetNet resources, authentication and
		  authorization can be used for each device connected to a
		  DetNet domain, most importantly to network controller
		  devices. Control of a DetNet network may be centralized or
		  distributed (within a single administrative domain). In the
		  case of centralized control, precedent for security
		  considerations as defined for Abstraction and Control of
		  Traffic Engineered Networks (ACTN) can be found in <xref target="RFC8453"/>, Section 9.
		  target="RFC8453" sectionFormat ="of" section="9"/>. In the case of distributed
		  control protocols, DetNet security is expected to be
		  provided by the security properties of the protocols in
		  use. In any case, the result is that manipulation of
		  administratively configurable parameters is limited to
		  authorized entities.
        </t><t>  </t>
      <t> To maintain confidentiality of
		  data traversing the DetNet, application flows can be
		  protected through whatever means is provided by the
		  underlying technology. For example, encryption may be used,
		  such as that provided by IPSec IPsec <xref target="RFC4301"/> target="RFC4301" format="default"/>, for
		  IP flows and by MACSec <xref target="IEEE802.1AE-2018"/> target="IEEE802.1AE" format="default"/> for
		  Ethernet
		  (Layer-2) (Layer 2) flows.
        </t><t>  </t>
      <t> DetNet flows are
		  identified on a per-flow basis, which may provide attackers
		  with additional information about the data flows (when
		  compared to networks that do not include per-flow
		  identification).  This is an inherent property of DetNet which
		  that has security implications that should be considered
		  when determining if DetNet is a suitable technology for any
		  given use case.
        </t><t>  </t>
      <t> To provide uninterrupted
		  availability of the DetNet quality of
		  service, QoS, provisions
		  can be made against DOS DoS attacks and delay attacks. To
		  protect against DOS DoS attacks, excess traffic due to malicious
		  or malfunctioning devices can be prevented or mitigated, for example
		  example, through the use of traffic admission control
		  applied at the input of a DetNet domain, domain as described in
		  <xref target="Zero"/>, target="Zero" format="default"/> and through the fault mitigation fault-mitigation
		  methods described in <xref target="FaultMitigation"/>. target="FaultMitigation" format="default"/>. To
		  prevent DetNet packets from being delayed by an entity
		  external to a DetNet domain, DetNet technology definition
		  can allow for the mitigation of
		  Man-In-The-Middle man-in-the-middle attacks,
		  for example example, through use of authentication and authorization
		  of devices within the DetNet domain.
        </t><t>  </t>
      <t> Because DetNet
		  mechanisms or applications that rely on DetNet can make
		  heavy use of methods that require precise time
		  synchronization, the accuracy, availability, and integrity
		  of time synchronization is of critical importance. Extensive
		  discussion of this topic can be found in <xref target="RFC7384"/>.
		</t><t> target="RFC7384" format="default"/>.  </t>
      <t> DetNet use cases are known to
		  have widely divergent security requirements. The intent of
		  this section is to provide a baseline for security
		  considerations which that are common to all DetNet designs and
		  implementations, without burdening individual designs with
		  specifics of security infrastructure which that may not be
		  germane to the given use case. Designers and implementers implementors of
		  DetNet systems are expected to take use case specific use-case-specific considerations
		  into account in their DetNet designs
		  and implementations.
      </t>
    </section>
    <section title="Privacy Considerations"> numbered="true" toc="default">
      <name>Privacy Considerations</name>
      <t>
		DetNet provides a Quality of Service (QoS), QoS, and the generic
		considerations for such mechanisms apply. In particular, such markings
		allow for an attacker to correlate flows or to select particular types
		of flow for more detailed inspection.
	  </t><t>
      </t>
      <t>
		However, the requirement for every (or almost every) node along the path of
		a DetNet flow to identify DetNet flows may present an additional attack
		surface for privacy, privacy should the DetNet paradigm be found useful in broader
		environments.
      </t>
    </section>
    <section title="IANA Considerations"> numbered="true" toc="default">
      <name>IANA Considerations</name>
      <t>This document does not require an action from IANA.
	  </t>
	</section>

	<section title="Acknowledgements">
	  <t>The authors wish to thank Lou Berger, David Black, Stewart Bryant,
	  Rodney Cummings, Ethan Grossman, Craig Gunther, Marcel Kiessling,
	  Rudy Klecka, Jouni Korhonen, Erik Nordmark, Shitanshu Shah,
	  Wilfried Steiner, George Swallow, Michael Johas Teener, Pat Thaler,
	  Thomas Watteyne, Patrick Wetterwald, Karl Weber, Anca Zamfir,
	  for their various contributions to this work. has no IANA actions.
      </t>
    </section>
  </middle>
  <back>
	<references title='Informative References'>

	  <?rfc include='reference.I-D.ietf-6tisch-architecture'?>
	  <?rfc include='reference.I-D.ietf-detnet-problem-statement'?>
	  <?rfc include='reference.I-D.ietf-detnet-use-cases'?>
	  <?rfc include='reference.I-D.ietf-detnet-dp-sol-mpls'?>
	  <?rfc include='reference.I-D.ietf-detnet-dp-sol-ip'?>
	  <?rfc include='reference.I-D.ietf-detnet-security'?>
	  <?rfc include='reference.RFC.2205'?>
	  <?rfc include='reference.RFC.2475'?>
	  <?rfc include='reference.RFC.2914'?>
	  <?rfc include='reference.RFC.3168'?>
	  <?rfc include='reference.RFC.3209'?>
      <?rfc include='reference.RFC.3550'?>
	  <?rfc include='reference.RFC.4301'?>
	  <?rfc include='reference.RFC.4655'?>
	  <?rfc include='reference.RFC.5921'?>
	  <?rfc include='reference.RFC.6372'?>
	  <?rfc include='reference.RFC.6658'?>
	  <?rfc include='reference.RFC.7149'?>
	  <?rfc include='reference.RFC.7384'?>
	  <?rfc include='reference.RFC.7426'?>
	  <?rfc include='reference.RFC.7554'?>
	  <?rfc include='reference.RFC.7567'?>
	  <?rfc include='reference.RFC.7813'?>
	  <?rfc include='reference.RFC.8033'?>
	  <?rfc include='reference.RFC.8227'?>
	  <?rfc include='reference.RFC.8289'?>
	  <?rfc include='reference.RFC.8402'?>
	  <?rfc include='reference.RFC.8453'?>

      <!-- I-D.ietf-detnet-security-05: I-D Exists  -->
<displayreference target="I-D.ietf-detnet-security" to="DETNET-SECURITY"/>

      <!-- draft-ietf-detnet-ip-01: I-D exists -->
<displayreference target="I-D.ietf-detnet-ip" to="DETNET-IP"/>

  <!-- draft-ietf-detnet-data-plane-framework-02: I-D exists-->
<displayreference target="I-D.ietf-detnet-data-plane-framework" to="DETNET-FRAMEWORK"/>

   <!-- draft-ietf-detnet-mpls-01: I-D exists-->
<displayreference target="I-D.ietf-detnet-mpls" to="DETNET-MPLS"/>

      <!-- I-D.ietf-6tisch-architecture-26: I-D exists-->
<displayreference target="I-D.ietf-6tisch-architecture" to="TSCH-ARCH"/>

    <references>
      <name>Informative References</name>

<xi:include href="https://www.rfc-editor.org/refs/bibxml3/reference.I-D.ietf-detnet-security.xml"/>

<xi:include href="https://www.rfc-editor.org/refs/bibxml3/reference.I-D.ietf-detnet-ip.xml"/>

<xi:include href="https://www.rfc-editor.org/refs/bibxml3/reference.I-D.ietf-detnet-data-plane-framework.xml"/>

<xi:include href="https://www.rfc-editor.org/refs/bibxml3/reference.I-D.ietf-detnet-mpls.xml"/>

<xi:include href="https://www.rfc-editor.org/refs/bibxml3/reference.I-D.ietf-6tisch-architecture.xml"/>

<!-- draft-ietf-detnet-use-cases now RFC 8578-->

<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8578.xml"/>

<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8557.xml"/>

<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.2205.xml"/>

<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.2475.xml"/>

<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.2914.xml"/>

<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.3168.xml"/>

<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.3209.xml"/>

<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.3550.xml"/>

<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.4301.xml"/>

<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.4655.xml"/>

<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.6372.xml"/>

<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.6658.xml"/>

<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7149.xml"/>

<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7384.xml"/>

<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7426.xml"/>

<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7554.xml"/>

<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7567.xml"/>

<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7813.xml"/>

<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8033.xml"/>

<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8227.xml"/>

<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8289.xml"/>

<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8402.xml"/>

<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8453.xml"/>

      <reference anchor="IEC62439-3-2016" anchor="IEC-62439-3" target="https://webstore.iec.ch/publication/24447">
        <front>
		  <title>IEC 62439-3:2016 Industrial
          <title>Industrial communication networks - High
		     availability automation networks - Part 3: Parallel Redundancy
		     Protocol (PRP) and High-availability Seamless Redundancy (HSR)
          </title>
          <seriesInfo name="TC 65 /" value="SC 65C"/>
          <seriesInfo name="IEC" value="62439-3:2016"/>
          <author>
			<organization>International Electrotechnical Commission (IEC)
			   TC 65/SC 65C - Industrial networks</organization>
            <organization>IEC</organization>
          </author>
          <date year="2016" /> month="March" year="2016"/>
        </front>
      </reference>
      <reference anchor="IEEE802.1AE-2018" anchor="IEEE802.1AE" target="https://ieeexplore.ieee.org/document/8585421">
        <front>
          <title>IEEE Std 802.1AE-2018 MAC Security (MACsec)</title> Standard for Local and metropolitan area
		  networks-Media Access Control (MAC) Security</title>
          <seriesInfo name="DOI" value="10.1109/IEEESTD.2018.8585421"/>
          <seriesInfo name="IEEE" value="802.1AE-2018"/>
          <author>
			<organization>IEEE Standards Association</organization>
            <organization>IEEE</organization>
          </author>
		  <date year="2018" />
          <date/>
        </front>
      </reference>
      <reference anchor="IEEE802.1CB"
		target="https://ieeexplore.ieee.org/document/8091139/"> target="https://ieeexplore.ieee.org/document/8091139">
        <front>
          <title>IEEE Std 802.1CB-2017 Frame Standard for Local and metropolitan area
		  networks--Frame Replication and Elimination for Reliability</title>
          <seriesInfo name="DOI" value="10.1109/IEEESTD.2017.8091139"/>
          <seriesInfo name="IEEE" value="802.1CB-2017"/>
          <author>
			<organization>IEEE Standards Association</organization>
            <organization>IEEE</organization>
          </author>
		  <date year="2017" />
          <date/>
        </front>
      </reference>
      <reference anchor="IEEE802.1Q-2018" anchor="IEEE802.1Q" target="https://ieeexplore.ieee.org/document/8403927">
        <front>
          <title>IEEE Std 802.1Q-2018 Bridges Standard for Local and Metropolitan Area
	      Network--Bridges and Bridged Networks</title>
          <seriesInfo name="DOI" value="10.1109/IEEESTD.2018.8403927"/>
          <seriesInfo name="IEEE" value="802.1Q-2018"/>
          <author>
                  <organization>IEEE Standards Association</organization>
            <organization>IEEE</organization>
          </author>
              <date year="2018" />
          <date/>
        </front>
      </reference>
      <reference anchor="IEEE802.1Qav"
		  target="https://ieeexplore.ieee.org/document/5375704/"> target="https://ieeexplore.ieee.org/document/5375704">
        <front>
          <title>IEEE Std 802.1Qav-2009 Bridges Standard for Local and Bridged Metropolitan Area Networks -
	      Virtual Bridged Local Area Networks Amendment 12: Forwarding and
	      Queuing Enhancements for Time-Sensitive Streams</title>
          <seriesInfo name="DOI" value="10.1109/IEEESTD.2009.5375704"/>
          <seriesInfo name="IEEE" value="802.1Qav-2009"/>
          <author>
				  <organization>IEEE Standards Association</organization>
            <organization>IEEE</organization>
          </author>
			  <date year="2009" />
          <date/>
        </front>
      </reference>
      <reference anchor="IEEE802.1Qbu"
		  target="https://ieeexplore.ieee.org/document/7553415/"> target="https://ieeexplore.ieee.org/document/7553415">
        <front>
          <title>IEEE Std 802.1Qbu-2016 Standard for Local and metropolitan area networks --
	      Bridges and Bridged Networks - -- Amendment 26: Frame Preemption</title>
          <seriesInfo name="DOI" value=" 10.1109/IEEESTD.2016.7553415"/>
          <seriesInfo name="IEEE" value="802.1Qbu-2016"/>
          <author>
				  <organization>IEEE Standards Association</organization>
            <organization>IEEE</organization>
          </author>
			  <date year="2016" />
          <date/>
        </front>
      </reference>
      <reference anchor="IEEE802.1Qbv"
		  target="https://ieeexplore.ieee.org/document/7572858/"> target="https://ieeexplore.ieee.org/document/7440741">
        <front>
          <title>IEEE Std 802.1Qbv-2015 Standard for Local and metropolitan area networks --
	      Bridges and Bridged Networks - Amendment 25: Enhancements for
	      Scheduled Traffic</title>
          <seriesInfo name="DOI" value="10.1109/IEEESTD.2016.7572858"/>
          <seriesInfo name="IEEE" value="802.1Qbv-2015"/>
          <author>
				  <organization>IEEE Standards Association</organization>
            <organization>IEEE</organization>
          </author>
			  <date year="2015" />
          <date/>
        </front>
      </reference>
      <reference anchor="IEEE802.1BA"
		target="https://ieeexplore.ieee.org/document/6032690/"> target="https://ieeexplore.ieee.org/document/6032690">
        <front>
          <title>IEEE Std 802.1BA-2011 Audio Standard for Local and metropolitan area
		  networks--Audio Video Bridging (AVB) Systems</title>
          <seriesInfo name="DOI" value="10.1109/IEEESTD.2011.6032690"/>
          <seriesInfo name="IEEE" value="802.1BA-2011"/>
          <author>
			<organization>IEEE Standards Association</organization>
            <organization>IEEE</organization>
          </author>
		  <date year="2011" />
          <date/>
        </front>
      </reference>
      <reference anchor="IEEE802.1Qch"
		  target="https://ieeexplore.ieee.org/document/7961303/"> target="https://ieeexplore.ieee.org/document/7961303">
        <front>
          <title>IEEE Std 802.1Qch-2017 Bridges Standard for Local and metropolitan area
			  networks--Bridges and Bridged Networks - Amendment Networks--Amendment
			  29: Cyclic Queuing and Forwarding</title>
          <seriesInfo name="DOI" value="10.1109/IEEESTD.2017.7961303"/>
          <seriesInfo name="IEEE" value="802.1Qch-2017"/>
          <author>
				  <organization>IEEE Standards Association</organization>
            <organization>IEEE</organization>
          </author>
			  <date year="2017" />
          <date/>
        </front>
      </reference>
      <reference anchor="IEEE802.3-2018" anchor="IEEE802.3" target="https://ieeexplore.ieee.org/document/8457469">
        <front>
          <title>IEEE Std 802.3-2018 Standard for Ethernet</title>
          <seriesInfo name="DOI" value="10.1109/IEEESTD.2018.8457469"/>
          <seriesInfo name="IEEE" value="802.3-2018"/>
          <author>
				  <organization>IEEE Standards Association</organization>
            <organization>IEEE</organization>
          </author>
			  <date year="2018" />
          <date/>
        </front>
      </reference>
      <reference anchor="IEEE802.3br"
		  target="http://ieeexplore.ieee.org/document/7900321/"> target="https://ieeexplore.ieee.org/document/7900321">
        <front>
          <title>IEEE Std 802.3br-2016 Standard for Ethernet Amendment 5:
			  Specification and Management Parameters for
			  Interspersing Express Traffic</title>
          <seriesInfo name="DOI" value="10.1109/IEEESTD.2016.7900321"/>
          <seriesInfo name="IEEE" value="802.3br"/>
          <author>
				  <organization>IEEE Standards Association</organization>
            <organization>IEEE</organization>
          </author>
			  <date year="2016" />
          <date/>
        </front>
      </reference>
      <reference anchor="IEEE802.1TSNTG" target="http://www.ieee802.org/1/tsn"> target="https://1.ieee802.org/tsn/">
        <front>
		  <title>IEEE 802.1 Time-Sensitive
          <title>Time-Sensitive Networking (TSN) Task Group</title>
          <author>
			<organization>IEEE Standards Association</organization>
            <organization>IEEE</organization>
          </author>
		  <date></date>
          <date/>
        </front>
      </reference>
      <reference anchor="TEAS" target="https://datatracker.ietf.org/doc/charter-ietf-teas/">
        <front>
          <title>Traffic Engineering Architecture and Signaling Working Group</title> (teas)</title>
          <author>
            <organization>IETF</organization>
          </author>
            <date></date>
          <date/>
        </front>
      </reference>
      <reference anchor="CCAMP" target="https://datatracker.ietf.org/doc/charter-ietf-ccamp/"> target="https://datatracker.ietf.org/wg/ccamp/charter/">
        <front>
          <title>Common Control and Measurement Plane Working Group</title> (ccamp)</title>
          <author>
            <organization>IETF</organization>
          </author>
            <date></date>
          <date/>
        </front>
      </reference>
      <reference anchor="BUFFERBLOAT">
        <front>
          <title>Bufferbloat: Dark Buffers in the Internet</title>
          <seriesInfo name="DOI" value="10.1145/2063176.2063196"/>
          <seriesInfo name="Communications of the ACM," value="Volume 55, Issue 1"/>
          <author fullname="J. Gettys" initials="J." surname="Gettys"></author> surname="Gettys" fullname="Jim Gettys">
            <organization/>
          </author>
          <author fullname="K. Nichols" initials="K." surname="Nichols"></author> surname="Nichols" fullname="Kathleen Nichols">
            <organization/>
          </author>
          <date month="January" year="2012" /> year="2012"/>
        </front>

        <format octets="Communications of the ACM, Volume 55 Issue 1, Pages 57-65 " target="https://dl.acm.org/citation.cfm?id=2063176.2063196" type="PDF" />
      </reference>
    </references>
    <section numbered="false" toc="default">
      <name>Acknowledgements</name>
      <t>The authors wish to thank Lou Berger, David Black, Stewart
	  Bryant, Rodney Cummings, Ethan Grossman, Craig Gunther, Marcel
	  Kiessling, Rudy Klecka, Jouni Korhonen, Erik Nordmark, Shitanshu
	  Shah, Wilfried Steiner, George Swallow, Michael Johas Teener, Pat
	  Thaler, Thomas Watteyne, Patrick Wetterwald, Karl Weber, and Anca
	  Zamfir for their various contributions to this work.
      </t>
    </section>
  </back>
</rfc>