<?xmlversion="1.0" encoding="US-ASCII"?> <!-- This template is for creating an Internet Draft using xml2rfc, which is available here: http://xml.resource.org. --> <!DOCTYPE rfc SYSTEM "rfc2629.dtd" [ ]> <?xml-stylesheet type='text/xsl' href='rfc2629.xslt' ?>version='1.0' encoding='utf-8'?> <rfc xmlns:xi="http://www.w3.org/2001/XInclude" version="3" category="std" consensus="true" docName="draft-ietf-detnet-architecture-13" indexInclude="true" ipr="trust200902"docName="draft-ietf-detnet-architecture-13"> <?xml-stylesheet type='text/xsl' href='rfc2629.xslt' ?> <?rfc sortrefs="yes"?> <?rfc iprnotified-"no" ?> <?rfc authorship="yes"?> <?rfc tocappendix="yes"?> <?rfc strict="yes" ?> <!-- give errors regarding ID-nits and DTD validation --> <!-- control the table of contents (ToC) --> <?rfc toc="yes"?> <!-- generate a ToC --> <?rfc tocdepth="4"?> <!-- the number of levels of subsections in ToC. default: 3 --> <!-- control references --> <?rfc symrefs="yes"?> <!-- use symbolic references tags, i.e, [RFC2119] instead of [1] --> <?rfc sortrefs="yes" ?> <!-- sort the reference entries alphabetically --> <!-- control vertical white space (using these PIs as follows is recommended by the RFC Editor) --> <?rfc compact="no" ?>number="8655" prepTime="2019-10-11T11:14:10" scripts="Common,Latin" sortRefs="true" submissionType="IETF" symRefs="true" tocDepth="3" tocInclude="true" xml:lang="en"> <link href="https://datatracker.ietf.org/doc/draft-ietf-detnet-architecture-13" rel="prev"/> <link href="https://dx.doi.org/10.17487/rfc8655" rel="alternate"/> <link href="urn:issn:2070-1721" rel="alternate"/> <front> <title>Deterministic Networking Architecture</title> <seriesInfo name="RFC" value="8655" stream="IETF"/> <author initials="N" surname="Finn" fullname="NormanFinn" > <organization> Huawei </organization>Finn"> <organization showOnFrontPage="true">Huawei</organization> <address> <postal> <street>3101 Rio Way</street> <city>Spring Valley</city> <region>California</region> <code>91977</code><country>US</country><country>United States of America</country> </postal> <phone>+1 925 980 6430</phone><email>norman.finn@mail01.huawei.com</email><email>nfinn@nfinnconsulting.com</email> </address> </author> <author initials="P" surname="Thubert" fullname="Pascal Thubert"> <organizationabbrev="Cisco">abbrev="Cisco" showOnFrontPage="true"> Cisco Systems </organization> <address> <postal> <street>Village d'Entreprises Green Side</street> <street>400, Avenue de Roumanille</street><street>Batiment<street> Batiment T3</street> <city>Biot - Sophia Antipolis</city> <code>06410</code><country>FRANCE</country><country>France</country> </postal> <phone>+33 4 97 23 26 34</phone> <email>pthubert@cisco.com</email> </address> </author> <authorfullname="Balázsfullname="Balázs Varga" initials="B." surname="Varga"><organization>Ericsson</organization><organization showOnFrontPage="true">Ericsson</organization> <address> <postal> <street>Magyar tudosok korutja 11</street> <city>Budapest</city> <country>Hungary</country> <code>1117</code> </postal> <email>balazs.a.varga@ericsson.com</email> </address> </author> <authorfullname="Jánosfullname="János Farkas" initials="J." surname="Farkas"><organization>Ericsson</organization><organization showOnFrontPage="true">Ericsson</organization> <address> <postal> <street>Magyar tudosok korutja 11</street> <city>Budapest</city> <country>Hungary</country> <code>1117</code> </postal> <email>janos.farkas@ericsson.com</email> </address> </author><date/><date month="10" year="2019"/> <area>Internet</area> <workgroup>DetNet</workgroup><abstract> <t><keyword>TSN, Bounded Lantency, Reliable Networking, Available Networking</keyword> <abstract pn="section-abstract"> <t pn="section-abstract-1"> This document provides the overall architecture for Deterministic Networking (DetNet), which provides a capability to carry specified unicast or multicast data flows for real-time applications with extremely low data loss rates and bounded latency within a network domain. Techniques usedinclude:include 1) reservingdata planedata-plane resources for individual (or aggregated) DetNet flows in some or all of the intermediate nodes along the path of theflow;flow, 2) providing explicit routes for DetNet flows that do not immediately change with the networktopology;topology, and 3) distributing data from DetNet flow packets over time and/or space to ensure delivery of each packet's data in spite of the loss of a path. DetNet operates at the IP layer and delivers service overlower layerlower-layer technologies such as MPLS andIEEE 802.1Time-Sensitive Networking(TSN).(TSN) as defined by IEEE 802.1. </t> </abstract></front> <middle> <!-- **************************************************************** --> <!-- **************************************************************** --> <!-- **************************************************************** --> <!-- **************************************************************** --> <section anchor='introduction' title="Introduction"> <t><boilerplate> <section anchor="status-of-memo" numbered="false" removeInRFC="false" toc="exclude" pn="section-boilerplate.1"> <name slugifiedName="name-status-of-this-memo">Status of This Memo</name> <t pn="section-boilerplate.1-1"> This is an Internet Standards Track document. </t> <t pn="section-boilerplate.1-2"> This documentprovides the overall architecture for Deterministic Networking (DetNet), which providesis acapability forproduct of thedeliveryInternet Engineering Task Force (IETF). It represents the consensus ofdata flows with extremely low packet loss ratesthe IETF community. It has received public review andbounded end-to-end delivery latency. DetNet ishas been approved fornetworks that are under a single administrative control or within a closed group of administrative control; these include campus-wide networks and private WANs. DetNetpublication by the Internet Engineering Steering Group (IESG). Further information on Internet Standards isnot for large groupsavailable in Section 2 ofdomains such asRFC 7841. </t> <t pn="section-boilerplate.1-3"> Information about theInternet. </t><t> DetNet operatescurrent status of this document, any errata, and how to provide feedback on it may be obtained atthe IP layer<eref target="https://www.rfc-editor.org/info/rfc8655"/>. </t> </section> <section anchor="copyright" numbered="false" removeInRFC="false" toc="exclude" pn="section-boilerplate.2"> <name slugifiedName="name-copyright-notice">Copyright Notice</name> <t pn="section-boilerplate.2-1"> Copyright (c) 2019 IETF Trust anddelivers service over lower layer technologies suchthe persons identified asMPLSthe document authors. All rights reserved. </t> <t pn="section-boilerplate.2-2"> This document is subject to BCP 78 andIEEE 802.1 Time-Sensitive Networking (TSN). DetNet accomplishesthe IETF Trust's Legal Provisions Relating to IETF Documents (<eref target="https://trustee.ietf.org/license-info"/>) in effect on the date of publication of this document. Please review thesegoals by dedicating network resources suchdocuments carefully, aslink bandwidththey describe your rights andbuffer spacerestrictions with respect toDetNet flows and/or classes of DetNet flows, and bythis document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. </t> </section> <section anchor="toc" numbered="false" removeInRFC="false" toc="exclude" pn="section-boilerplate.3"> <name slugifiedName="name-table-of-contents">Table of Contents</name> <ul bare="true" empty="true" indent="2" spacing="compact" pn="section-boilerplate.3-1"> <li pn="section-boilerplate.3-1.1"> <t keepWithNext="true" pn="section-boilerplate.3-1.1.1"><xref derivedContent="1" format="counter" sectionFormat="of" target="section-1"/>. <xref derivedContent="Introduction" format="title" sectionFormat="of" target="name-introduction">Introduction</xref></t> </li> <li pn="section-boilerplate.3-1.2"> <t keepWithNext="true" pn="section-boilerplate.3-1.2.1"><xref derivedContent="2" format="counter" sectionFormat="of" target="section-2"/>. <xref derivedContent="Terminology" format="title" sectionFormat="of" target="name-terminology">Terminology</xref></t> <ul bare="true" empty="true" indent="2" spacing="compact" pn="section-boilerplate.3-1.2.2"> <li pn="section-boilerplate.3-1.2.2.1"> <t keepWithNext="true" pn="section-boilerplate.3-1.2.2.1.1"><xref derivedContent="2.1" format="counter" sectionFormat="of" target="section-2.1"/>. <xref derivedContent="Terms Used in This Document" format="title" sectionFormat="of" target="name-terms-used-in-this-document">Terms Used in This Document</xref></t> </li> <li pn="section-boilerplate.3-1.2.2.2"> <t keepWithNext="true" pn="section-boilerplate.3-1.2.2.2.1"><xref derivedContent="2.2" format="counter" sectionFormat="of" target="section-2.2"/>. <xref derivedContent="Dictionary of Terms Used by TSN and DetNet" format="title" sectionFormat="of" target="name-dictionary-of-terms-used-by">Dictionary of Terms Used by TSN and DetNet</xref></t> </li> </ul> </li> <li pn="section-boilerplate.3-1.3"> <t keepWithNext="true" pn="section-boilerplate.3-1.3.1"><xref derivedContent="3" format="counter" sectionFormat="of" target="section-3"/>. <xref derivedContent="Providing the DetNet Quality of Service" format="title" sectionFormat="of" target="name-providing-the-detnet-qualit">Providing the DetNet Quality of Service</xref></t> <ul bare="true" empty="true" indent="2" spacing="compact" pn="section-boilerplate.3-1.3.2"> <li pn="section-boilerplate.3-1.3.2.1"> <t keepWithNext="true" pn="section-boilerplate.3-1.3.2.1.1"><xref derivedContent="3.1" format="counter" sectionFormat="of" target="section-3.1"/>. <xref derivedContent="Primary Goals Defining the DetNet QoS" format="title" sectionFormat="of" target="name-primary-goals-defining-the-">Primary Goals Defining the DetNet QoS</xref></t> </li> <li pn="section-boilerplate.3-1.3.2.2"> <t keepWithNext="true" pn="section-boilerplate.3-1.3.2.2.1"><xref derivedContent="3.2" format="counter" sectionFormat="of" target="section-3.2"/>. <xref derivedContent="Mechanisms to Achieve DetNet QoS" format="title" sectionFormat="of" target="name-mechanisms-to-achieve-detne">Mechanisms to Achieve DetNet QoS</xref></t> <ul bare="true" empty="true" indent="2" spacing="compact" pn="section-boilerplate.3-1.3.2.2.2"> <li pn="section-boilerplate.3-1.3.2.2.2.1"> <t keepWithNext="true" pn="section-boilerplate.3-1.3.2.2.2.1.1"><xref derivedContent="3.2.1" format="counter" sectionFormat="of" target="section-3.2.1"/>. <xref derivedContent="Resource Allocation" format="title" sectionFormat="of" target="name-resource-allocation">Resource Allocation</xref></t> </li> <li pn="section-boilerplate.3-1.3.2.2.2.2"> <t keepWithNext="true" pn="section-boilerplate.3-1.3.2.2.2.2.1"><xref derivedContent="3.2.2" format="counter" sectionFormat="of" target="section-3.2.2"/>. <xref derivedContent="Service Protection" format="title" sectionFormat="of" target="name-service-protection">Service Protection</xref></t> </li> <li pn="section-boilerplate.3-1.3.2.2.2.3"> <t keepWithNext="true" pn="section-boilerplate.3-1.3.2.2.2.3.1"><xref derivedContent="3.2.3" format="counter" sectionFormat="of" target="section-3.2.3"/>. <xref derivedContent="Explicit Routes" format="title" sectionFormat="of" target="name-explicit-routes">Explicit Routes</xref></t> </li> </ul> </li> <li pn="section-boilerplate.3-1.3.2.3"> <t keepWithNext="true" pn="section-boilerplate.3-1.3.2.3.1"><xref derivedContent="3.3" format="counter" sectionFormat="of" target="section-3.3"/>. <xref derivedContent="Secondary Goals for DetNet" format="title" sectionFormat="of" target="name-secondary-goals-for-detnet">Secondary Goals for DetNet</xref></t> <ul bare="true" empty="true" indent="2" spacing="compact" pn="section-boilerplate.3-1.3.2.3.2"> <li pn="section-boilerplate.3-1.3.2.3.2.1"> <t keepWithNext="true" pn="section-boilerplate.3-1.3.2.3.2.1.1"><xref derivedContent="3.3.1" format="counter" sectionFormat="of" target="section-3.3.1"/>. <xref derivedContent="Coexistence with Normal Traffic" format="title" sectionFormat="of" target="name-coexistence-with-normal-tra">Coexistence with Normal Traffic</xref></t> </li> <li pn="section-boilerplate.3-1.3.2.3.2.2"> <t keepWithNext="true" pn="section-boilerplate.3-1.3.2.3.2.2.1"><xref derivedContent="3.3.2" format="counter" sectionFormat="of" target="section-3.3.2"/>. <xref derivedContent="Fault Mitigation" format="title" sectionFormat="of" target="name-fault-mitigation">Fault Mitigation</xref></t> </li> </ul> </li> </ul> </li> <li pn="section-boilerplate.3-1.4"> <t keepWithNext="true" pn="section-boilerplate.3-1.4.1"><xref derivedContent="4" format="counter" sectionFormat="of" target="section-4"/>. <xref derivedContent="DetNet Architecture" format="title" sectionFormat="of" target="name-detnet-architecture">DetNet Architecture</xref></t> <ul bare="true" empty="true" indent="2" spacing="compact" pn="section-boilerplate.3-1.4.2"> <li pn="section-boilerplate.3-1.4.2.1"> <t keepWithNext="true" pn="section-boilerplate.3-1.4.2.1.1"><xref derivedContent="4.1" format="counter" sectionFormat="of" target="section-4.1"/>. <xref derivedContent="DetNet Stack Model" format="title" sectionFormat="of" target="name-detnet-stack-model">DetNet Stack Model</xref></t> <ul bare="true" empty="true" indent="2" spacing="compact" pn="section-boilerplate.3-1.4.2.1.2"> <li pn="section-boilerplate.3-1.4.2.1.2.1"> <t keepWithNext="true" pn="section-boilerplate.3-1.4.2.1.2.1.1"><xref derivedContent="4.1.1" format="counter" sectionFormat="of" target="section-4.1.1"/>. <xref derivedContent="Representative Protocol Stack Model" format="title" sectionFormat="of" target="name-representative-protocol-sta">Representative Protocol Stack Model</xref></t> </li> <li pn="section-boilerplate.3-1.4.2.1.2.2"> <t keepWithNext="true" pn="section-boilerplate.3-1.4.2.1.2.2.1"><xref derivedContent="4.1.2" format="counter" sectionFormat="of" target="section-4.1.2"/>. <xref derivedContent="DetNet Data-Plane Overview" format="title" sectionFormat="of" target="name-detnet-data-plane-overview">DetNet Data-Plane Overview</xref></t> </li> <li pn="section-boilerplate.3-1.4.2.1.2.3"> <t keepWithNext="true" pn="section-boilerplate.3-1.4.2.1.2.3.1"><xref derivedContent="4.1.3" format="counter" sectionFormat="of" target="section-4.1.3"/>. <xref derivedContent="Network Reference Model" format="title" sectionFormat="of" target="name-network-reference-model">Network Reference Model</xref></t> </li> </ul> </li> <li pn="section-boilerplate.3-1.4.2.2"> <t keepWithNext="true" pn="section-boilerplate.3-1.4.2.2.1"><xref derivedContent="4.2" format="counter" sectionFormat="of" target="section-4.2"/>. <xref derivedContent="DetNet Systems" format="title" sectionFormat="of" target="name-detnet-systems">DetNet Systems</xref></t> <ul bare="true" empty="true" indent="2" spacing="compact" pn="section-boilerplate.3-1.4.2.2.2"> <li pn="section-boilerplate.3-1.4.2.2.2.1"> <t keepWithNext="true" pn="section-boilerplate.3-1.4.2.2.2.1.1"><xref derivedContent="4.2.1" format="counter" sectionFormat="of" target="section-4.2.1"/>. <xref derivedContent="End System" format="title" sectionFormat="of" target="name-end-system">End System</xref></t> </li> <li pn="section-boilerplate.3-1.4.2.2.2.2"> <t keepWithNext="true" pn="section-boilerplate.3-1.4.2.2.2.2.1"><xref derivedContent="4.2.2" format="counter" sectionFormat="of" target="section-4.2.2"/>. <xref derivedContent="DetNet Edge, Relay, and Transit Nodes" format="title" sectionFormat="of" target="name-detnet-edge-relay-and-trans">DetNet Edge, Relay, and Transit Nodes</xref></t> </li> </ul> </li> <li pn="section-boilerplate.3-1.4.2.3"> <t keepWithNext="true" pn="section-boilerplate.3-1.4.2.3.1"><xref derivedContent="4.3" format="counter" sectionFormat="of" target="section-4.3"/>. <xref derivedContent="DetNet Flows" format="title" sectionFormat="of" target="name-detnet-flows">DetNet Flows</xref></t> <ul bare="true" empty="true" indent="2" spacing="compact" pn="section-boilerplate.3-1.4.2.3.2"> <li pn="section-boilerplate.3-1.4.2.3.2.1"> <t keepWithNext="true" pn="section-boilerplate.3-1.4.2.3.2.1.1"><xref derivedContent="4.3.1" format="counter" sectionFormat="of" target="section-4.3.1"/>. <xref derivedContent="DetNet Flow Types" format="title" sectionFormat="of" target="name-detnet-flow-types">DetNet Flow Types</xref></t> </li> <li pn="section-boilerplate.3-1.4.2.3.2.2"> <t keepWithNext="true" pn="section-boilerplate.3-1.4.2.3.2.2.1"><xref derivedContent="4.3.2" format="counter" sectionFormat="of" target="section-4.3.2"/>. <xref derivedContent="Source Transmission Behavior" format="title" sectionFormat="of" target="name-source-transmission-behavio">Source Transmission Behavior</xref></t> </li> <li pn="section-boilerplate.3-1.4.2.3.2.3"> <t keepWithNext="true" pn="section-boilerplate.3-1.4.2.3.2.3.1"><xref derivedContent="4.3.3" format="counter" sectionFormat="of" target="section-4.3.3"/>. <xref derivedContent="Incomplete Networks" format="title" sectionFormat="of" target="name-incomplete-networks">Incomplete Networks</xref></t> </li> </ul> </li> <li pn="section-boilerplate.3-1.4.2.4"> <t keepWithNext="true" pn="section-boilerplate.3-1.4.2.4.1"><xref derivedContent="4.4" format="counter" sectionFormat="of" target="section-4.4"/>. <xref derivedContent="Traffic Engineering for DetNet" format="title" sectionFormat="of" target="name-traffic-engineering-for-det">Traffic Engineering for DetNet</xref></t> <ul bare="true" empty="true" indent="2" spacing="compact" pn="section-boilerplate.3-1.4.2.4.2"> <li pn="section-boilerplate.3-1.4.2.4.2.1"> <t keepWithNext="true" pn="section-boilerplate.3-1.4.2.4.2.1.1"><xref derivedContent="4.4.1" format="counter" sectionFormat="of" target="section-4.4.1"/>. <xref derivedContent="The Application Plane" format="title" sectionFormat="of" target="name-the-application-plane">The Application Plane</xref></t> </li> <li pn="section-boilerplate.3-1.4.2.4.2.2"> <t keepWithNext="true" pn="section-boilerplate.3-1.4.2.4.2.2.1"><xref derivedContent="4.4.2" format="counter" sectionFormat="of" target="section-4.4.2"/>. <xref derivedContent="The Controller Plane" format="title" sectionFormat="of" target="name-the-controller-plane">The Controller Plane</xref></t> </li> <li pn="section-boilerplate.3-1.4.2.4.2.3"> <t keepWithNext="true" pn="section-boilerplate.3-1.4.2.4.2.3.1"><xref derivedContent="4.4.3" format="counter" sectionFormat="of" target="section-4.4.3"/>. <xref derivedContent="The Network Plane" format="title" sectionFormat="of" target="name-the-network-plane">The Network Plane</xref></t> </li> </ul> </li> <li pn="section-boilerplate.3-1.4.2.5"> <t keepWithNext="true" pn="section-boilerplate.3-1.4.2.5.1"><xref derivedContent="4.5" format="counter" sectionFormat="of" target="section-4.5"/>. <xref derivedContent="Queuing, Shaping, Scheduling, and Preemption" format="title" sectionFormat="of" target="name-queuing-shaping-scheduling-">Queuing, Shaping, Scheduling, and Preemption</xref></t> </li> <li pn="section-boilerplate.3-1.4.2.6"> <t keepWithNext="true" pn="section-boilerplate.3-1.4.2.6.1"><xref derivedContent="4.6" format="counter" sectionFormat="of" target="section-4.6"/>. <xref derivedContent="Service Instance" format="title" sectionFormat="of" target="name-service-instance">Service Instance</xref></t> </li> <li pn="section-boilerplate.3-1.4.2.7"> <t keepWithNext="true" pn="section-boilerplate.3-1.4.2.7.1"><xref derivedContent="4.7" format="counter" sectionFormat="of" target="section-4.7"/>. <xref derivedContent="Flow Identification at Technology Borders" format="title" sectionFormat="of" target="name-flow-identification-at-tech">Flow Identification at Technology Borders</xref></t> <ul bare="true" empty="true" indent="2" spacing="compact" pn="section-boilerplate.3-1.4.2.7.2"> <li pn="section-boilerplate.3-1.4.2.7.2.1"> <t keepWithNext="true" pn="section-boilerplate.3-1.4.2.7.2.1.1"><xref derivedContent="4.7.1" format="counter" sectionFormat="of" target="section-4.7.1"/>. <xref derivedContent="Exporting Flow Identification" format="title" sectionFormat="of" target="name-exporting-flow-identificati">Exporting Flow Identification</xref></t> </li> <li pn="section-boilerplate.3-1.4.2.7.2.2"> <t keepWithNext="true" pn="section-boilerplate.3-1.4.2.7.2.2.1"><xref derivedContent="4.7.2" format="counter" sectionFormat="of" target="section-4.7.2"/>. <xref derivedContent="Flow Attribute Mapping between Layers" format="title" sectionFormat="of" target="name-flow-attribute-mapping-betw">Flow Attribute Mapping between Layers</xref></t> </li> <li pn="section-boilerplate.3-1.4.2.7.2.3"> <t keepWithNext="true" pn="section-boilerplate.3-1.4.2.7.2.3.1"><xref derivedContent="4.7.3" format="counter" sectionFormat="of" target="section-4.7.3"/>. <xref derivedContent="Flow-ID Mapping Examples" format="title" sectionFormat="of" target="name-flow-id-mapping-examples">Flow-ID Mapping Examples</xref></t> </li> </ul> </li> <li pn="section-boilerplate.3-1.4.2.8"> <t keepWithNext="true" pn="section-boilerplate.3-1.4.2.8.1"><xref derivedContent="4.8" format="counter" sectionFormat="of" target="section-4.8"/>. <xref derivedContent="Advertising Resources, Capabilities, and Adjacencies" format="title" sectionFormat="of" target="name-advertising-resources-capab">Advertising Resources, Capabilities, and Adjacencies</xref></t> </li> <li pn="section-boilerplate.3-1.4.2.9"> <t keepWithNext="true" pn="section-boilerplate.3-1.4.2.9.1"><xref derivedContent="4.9" format="counter" sectionFormat="of" target="section-4.9"/>. <xref derivedContent="Scaling to Larger Networks" format="title" sectionFormat="of" target="name-scaling-to-larger-networks">Scaling to Larger Networks</xref></t> </li> <li pn="section-boilerplate.3-1.4.2.10"> <t keepWithNext="true" pn="section-boilerplate.3-1.4.2.10.1"><xref derivedContent="4.10" format="counter" sectionFormat="of" target="section-4.10"/>. <xref derivedContent="Compatibility with Layer 2" format="title" sectionFormat="of" target="name-compatibility-with-layer-2">Compatibility with Layer 2</xref></t> </li> </ul> </li> <li pn="section-boilerplate.3-1.5"> <t keepWithNext="true" pn="section-boilerplate.3-1.5.1"><xref derivedContent="5" format="counter" sectionFormat="of" target="section-5"/>. <xref derivedContent="Security Considerations" format="title" sectionFormat="of" target="name-security-considerations">Security Considerations</xref></t> </li> <li pn="section-boilerplate.3-1.6"> <t keepWithNext="true" pn="section-boilerplate.3-1.6.1"><xref derivedContent="6" format="counter" sectionFormat="of" target="section-6"/>. <xref derivedContent="Privacy Considerations" format="title" sectionFormat="of" target="name-privacy-considerations">Privacy Considerations</xref></t> </li> <li pn="section-boilerplate.3-1.7"> <t keepWithNext="true" pn="section-boilerplate.3-1.7.1"><xref derivedContent="7" format="counter" sectionFormat="of" target="section-7"/>. <xref derivedContent="IANA Considerations" format="title" sectionFormat="of" target="name-iana-considerations">IANA Considerations</xref></t> </li> <li pn="section-boilerplate.3-1.8"> <t keepWithNext="true" pn="section-boilerplate.3-1.8.1"><xref derivedContent="8" format="counter" sectionFormat="of" target="section-8"/>. <xref derivedContent="Informative References" format="title" sectionFormat="of" target="name-informative-references">Informative References</xref></t> </li> <li pn="section-boilerplate.3-1.9"> <t keepWithNext="true" pn="section-boilerplate.3-1.9.1"><xref derivedContent="" format="none" sectionFormat="of" target="section-appendix.a"/><xref derivedContent="Acknowledgements" format="title" sectionFormat="of" target="name-acknowledgements">Acknowledgements</xref></t> </li> <li pn="section-boilerplate.3-1.10"> <t keepWithNext="true" pn="section-boilerplate.3-1.10.1"><xref derivedContent="" format="none" sectionFormat="of" target="section-appendix.b"/><xref derivedContent="Authors' Addresses" format="title" sectionFormat="of" target="name-authors-addresses">Authors' Addresses</xref></t> </li> </ul> </section> </boilerplate> </front> <middle> <section anchor="introduction" numbered="true" toc="include" removeInRFC="false" pn="section-1"> <name slugifiedName="name-introduction">Introduction</name> <t pn="section-1-1"> This document provides the overall architecture for Deterministic Networking (DetNet), which provides a capability for the delivery of data flows with extremely low packet loss rates and bounded end-to-end delivery latency. DetNet is for networks that are under a single administrative control or within a closed group of administrative control; these include campus-wide networks and private WANs. DetNet is not for large groups of domains such as the Internet. </t> <t pn="section-1-2"> DetNet operates at the IP layer and delivers service over lower-layer technologies such as MPLS and IEEE 802.1 Time-Sensitive Networking (TSN). DetNet provides a reliable and available service by dedicating network resources such as link bandwidth and buffer space to DetNet flows and/or classes of DetNet flows, and by replicating packets along multiple paths. Unused reserved resources are available to non-DetNet packets as long as all guarantees are fulfilled.</t><t></t> <t pn="section-1-3"> The <xreftarget="I-D.ietf-detnet-problem-statement">Deterministictarget="RFC8557" format="default" sectionFormat="of" derivedContent="RFC8557">"Deterministic Networking ProblemStatement</xref>Statement"</xref> introducesDeterministic Networking,DetNet, and <xreftarget="I-D.ietf-detnet-use-cases">Deterministictarget="RFC8578" format="default" sectionFormat="of" derivedContent="RFC8578">"Deterministic Networking UseCases</xref>Cases"</xref> summarizes the need for it. See <xreftarget="I-D.ietf-detnet-dp-sol-mpls"/> and <xref target="I-D.ietf-detnet-dp-sol-ip"/>target="I-D.ietf-detnet-data-plane-framework" format="default" sectionFormat="of" derivedContent="DETNET-FRAMEWORK"/> for specific techniques that can be used to identify DetNet flows and assign them to specific paths through a network.</t><t></t> <t pn="section-1-4"> A goal of DetNet is a converged network in allrespectsrespects, including the convergence of sensitive non-IP networks onto a common network infrastructure. The presence of DetNet flows does not preclude non-DetNet flows, and the benefits offered DetNet flows should not, except in extreme cases, prevent existingQuality of ServiceQuality-of-Service (QoS) mechanisms from operating in a normal fashion, subject to the bandwidth required for the DetNet flows. A single source-destination pair can trade both DetNet and non-DetNet flows. End systems and applications need not instantiate special interfaces for DetNet flows. Networks are not restricted to certain topologies; connectivity is not restricted. Any application that generates a data flow that can be usefully characterized as having a maximum bandwidth should be able to take advantage of DetNet, as long as the necessary resources can be reserved. Reservations can be made by the application itself, via network management, centrally by an application's controller, or by other means,e.g.,for instance, by placing on-demand reservation via adynamic control plane (e.g.,distributed Control Plane, e.g., leveraging the Resource Reservation Protocol (RSVP) <xreftarget="RFC2205"/>).target="RFC2205" format="default" sectionFormat="of" derivedContent="RFC2205"/>. QoS requirements of DetNet flows can be met if all network nodes in a DetNet domain implement DetNet capabilities. DetNet nodes can be interconnected with different sub-network technologies (<xreftarget="sec_dt_dp"/>),target="sec_dt_dp" format="default" sectionFormat="of" derivedContent="Section 4.1.2"/>) where the nodes of the subnet are not DetNet aware (<xreftarget="netref"/>). </t><t>target="netref" format="default" sectionFormat="of" derivedContent="Section 4.1.3"/>). </t> <t pn="section-1-5"> Many applications that are intended to be served byDeterministic NetworkingDetNet require the ability to synchronize the clocks in end systems to a sub-microsecond accuracy. Some of thequeue controlqueue-control techniques defined in <xreftarget="QueuingModels"/>target="QueuingModels" format="default" sectionFormat="of" derivedContent="Section 4.5"/> also require time synchronization among network nodes. The means used to achieve time synchronization are not addressed in this document. DetNet can accommodate varioustime synchronizationtime-synchronization techniques and profiles that are defined elsewhere to address the needs of different market segments. </t> </section> <sectiontitle="Terminology">numbered="true" toc="include" removeInRFC="false" pn="section-2"> <name slugifiedName="name-terminology">Terminology</name> <sectiontitle="Terms usednumbered="true" toc="include" removeInRFC="false" pn="section-2.1"> <name slugifiedName="name-terms-used-in-this-document">Terms Used inthis document"> <t>This Document</name> <t pn="section-2.1-1"> The following terms are used in the context of DetNet in this document:<list hangIndent="8" style="hanging"> <t hangText="allocation"><vspace blankLines="0"/> Resources are dedicated</t> <dl newline="true" spacing="normal" indent="3" pn="section-2.1-2"> <dt pn="section-2.1-2.1">allocation</dt> <dd pn="section-2.1-2.2"> The dedication of resources to support a DetNet flow. Depending on an implementation, the resource may be reused by non-DetNet flows when it is not used by the DetNet flow.</t> <t hangText="App-flow"><vspace blankLines="0"/></dd> <dt pn="section-2.1-2.3">App-flow</dt> <dd pn="section-2.1-2.4"> The payload (data) carried over a DetNet service.</t> <t hangText="DetNet</dd> <dt pn="section-2.1-2.5">DetNet compound flow and DetNet memberflow"><vspace blankLines="0"/>flow</dt> <dd pn="section-2.1-2.6"> A DetNet compound flow is a DetNet flow that has been separated into multiple duplicate DetNet member flows for service protection at the DetNet service sub-layer. Member flows are merged back into a single DetNet compound flow such that there are no duplicate packets. "Compound" and "member" are strictly relative to each other, not absolutes; a DetNet compound flow comprising multiple DetNet member flows can, in turn, be a member of a higher-order compound.</t> <t hangText="DetNet destination"><vspace blankLines="0"/></dd> <dt pn="section-2.1-2.7">DetNet destination</dt> <dd pn="section-2.1-2.8"> An end system capable of terminating a DetNet flow.</t> <t hangText="DetNet domain"><vspace blankLines="0"/></dd> <dt pn="section-2.1-2.9">DetNet domain</dt> <dd pn="section-2.1-2.10"> The portion of a network that is DetNet aware. It includes end systems and DetNet nodes.</t> <t hangText="DetNet</dd> <dt pn="section-2.1-2.11">DetNet edgenode"><vspace blankLines="0"/>node</dt> <dd pn="section-2.1-2.12"> An instance of a DetNet relay node that acts as a source and/or destination at the DetNet service sub-layer. For example, it can include a DetNet service sub-layer proxy function for DetNet service protection (e.g., the addition or removal of packet sequencing information) for one or more end systems, it can start orstarts or terminatesterminate resource allocation at the DetNet forwarding sub-layer, oraggregatesit can aggregate DetNet services into new DetNet flows. It is analogous to a Label Edge Router (LER) or a Provider Edge (PE) router.</t> <t hangText="DetNet flow"><vspace blankLines="0"/></dd> <dt pn="section-2.1-2.13">DetNet flow</dt> <dd pn="section-2.1-2.14"> ADetNet flow is asequence of packetswhich conformthat conforms uniquely to a flowidentifier,identifier and to which the DetNet service is to be provided. It includes any DetNet headers added to support the DetNet service and forwarding sub-layers.</t> <t hangText="DetNet</dd> <dt pn="section-2.1-2.15">DetNet forwardingsub-layer"><vspace blankLines="0"/>sub-layer</dt> <dd pn="section-2.1-2.16"> DetNet functionality is divided into two sub-layers. One of them is the DetNet forwarding sub-layer, which optionally provides resource allocation for DetNet flows over paths provided by the underlying network.</t> <t hangText="DetNet</dd> <dt pn="section-2.1-2.17">DetNet intermediatenode"><vspace blankLines="0"/>node</dt> <dd pn="section-2.1-2.18"> A DetNet relay node or DetNet transit node.</t> <t hangText="DetNet node"><vspace blankLines="0"/></dd> <dt pn="section-2.1-2.19">DetNet node</dt> <dd pn="section-2.1-2.20"> A DetNet edge node, a DetNet relay node, or a DetNet transit node.</t> <t hangText="DetNet</dd> <dt pn="section-2.1-2.21">DetNet relaynode"><vspace blankLines="0"/>node</dt> <dd pn="section-2.1-2.22"> A DetNet nodeincludingthat includes a service sub-layer function that interconnects different DetNet forwarding sub-layer paths to provide service protection. A DetNet relay node participates in the DetNet service sub-layer. It typically incorporates DetNet forwarding sub-layer functions as well, in which case it is collocated with a transit node.</t> <t hangText="DetNet service sub-layer"><vspace blankLines="0"/></dd> <dt pn="section-2.1-2.23">DetNet service sub-layer</dt> <dd pn="section-2.1-2.24"> DetNet functionality is divided into two sub-layers. One of them is the DetNet service sub-layer, at which a DetNetservice, e.g.,serviceprotection(e.g., service protection) is provided.</t> <t hangText="DetNet service proxy"><vspace blankLines="0"/> Maps</dd> <dt pn="section-2.1-2.25">DetNet service proxy</dt> <dd pn="section-2.1-2.26"> A proxy that maps between App-flows and DetNet flows.</t> <t hangText="DetNet source"><vspace blankLines="0"/></dd> <dt pn="section-2.1-2.27">DetNet source</dt> <dd pn="section-2.1-2.28"> An end system capable of originating a DetNet flow.</t> <t hangText="DetNet system"><vspace blankLines="0"/></dd> <dt pn="section-2.1-2.29">DetNet system</dt> <dd pn="section-2.1-2.30"> ADetNet awareDetNet-aware end system, transit node, or relay node. "DetNet" may be omitted in some text.</t> <t hangText="DetNet</dd> <dt pn="section-2.1-2.31">DetNet transitnode"><vspace blankLines="0"/>node</dt> <dd pn="section-2.1-2.32"> A DetNetnodenode, operating at the DetNet forwarding sub-layer, that utilizeslink layerlink-layer and/ornetwork layernetwork-layer switching across multiple links and/or sub-networks to provide paths for DetNet service sub-layer functions.TypicallyIt typically provides resource allocation over those paths. An MPLSLSRLabel Switch Router (LSR) is an example of a DetNet transit node.</t> <t hangText="DetNet-UNI"><vspace blankLines="0"/></dd> <dt pn="section-2.1-2.33">DetNet-UNI</dt> <dd pn="section-2.1-2.34"> A User-to-Network Interface (UNI) withDetNet specificDetNet-specific functionalities. It is a packet-based reference point and may provide multiple functions like encapsulation, status, synchronization, etc.</t> <t hangText="end system"><vspace blankLines="0"/></dd> <dt pn="section-2.1-2.35">end system</dt> <dd pn="section-2.1-2.36"> Commonly called a "host" inIETF documents,the RFC series and an "end station"isin IEEE 802documents.standards. End systems of interest to this document are either sources or destinations of DetNetflows. And end systemflows, and they may or may not be aware of DetNet forwardingsub-layer awaresub-layers or DetNet servicesub-layer aware. </t> <t hangText="link"><vspace blankLines="0"/>sub-layers. </dd> <dt pn="section-2.1-2.37">link</dt> <dd pn="section-2.1-2.38"> A connection between two DetNet nodes. It may be composed of a physical link or a sub-network technology that can provide appropriate traffic delivery for DetNet flows.</t> <t hangText="PEF"> A Packet</dd> <dt pn="section-2.1-2.39">Packet Elimination Function(PEF)(PEF)</dt> <dd pn="section-2.1-2.40"> A function that eliminates duplicate copies of packets to prevent excess packets flooding the network or duplicate packets being sent out of the DetNet domain. A PEF can be implemented by a DetNet edge node, a DetNet relay node, or an end system.</t> <t hangText="PRF"> A Packet</dd> <dt pn="section-2.1-2.41">Packet Replication Function(PRF)(PRF)</dt> <dd pn="section-2.1-2.42"> A function that replicates DetNet flow packets and forwards them to one or more next hops in the DetNet domain. The number of packet copies sent to the next hops is a parameter specific to the DetNet flowspecific parameterat the point of replication. A PRF can be implemented by a DetNet edge node, a DetNet relay node, or an end system.</t> <t hangText="PREOF"> Collective</dd> <dt pn="section-2.1-2.43">PREOF</dt> <dd pn="section-2.1-2.44"> A collective name for Packet Replication, Elimination, and Ordering Functions.</t> <t hangText="POF"> A Packet</dd> <dt pn="section-2.1-2.45">Packet Ordering Function(POF) re-orders(POF)</dt> <dd pn="section-2.1-2.46"> A function that reorders packets within a DetNet flow that are received out of order. This function can be implemented by a DetNet edge node, a DetNet relay node, or an end system.</t> <t hangText="reservation"><vspace blankLines="0"/></dd> <dt pn="section-2.1-2.47">reservation</dt> <dd pn="section-2.1-2.48"> The set of resources allocated between a source and one or more destinations through DetNet nodes and subnets associated with a DetNetflow,flow in order to provide the provisioned DetNet service.</t> </list> </t></dd> </dl> </section> <sectiontitle="IEEE 802.1numbered="true" toc="include" removeInRFC="false" pn="section-2.2"> <name slugifiedName="name-dictionary-of-terms-used-by">Dictionary of Terms Used by TSNto DetNet dictionary"> <t>and DetNet</name> <t pn="section-2.2-1"> This sectionalsoserves as a dictionary for translatingfromthe terms used by the Time-Sensitive Networking (TSN) Task Group <xreftarget="IEEE802.1TSNTG"/>target="IEEE802.1TSNTG" format="default" sectionFormat="of" derivedContent="IEEE802.1TSNTG"/> of the IEEE 802.1 WG to those of theDetNet WG. <list hangIndent="8" style="hanging"> <t hangText="Listener"><vspace blankLines="0"/>Deterministic Networking (detnet) WG of the IETF. </t> <dl newline="true" spacing="normal" indent="3" pn="section-2.2-2"> <dt pn="section-2.2-2.1">Listener</dt> <dd pn="section-2.2-2.2"> The term used by IEEE 802.1termfor a destination of a DetNet flow.</t> <t hangText="relay system"><vspace blankLines="0"/></dd> <dt pn="section-2.2-2.3">Relay system</dt> <dd pn="section-2.2-2.4"> The term used by IEEE 802.1termfor a DetNet intermediate node.</t> <t hangText="Stream"><vspace blankLines="0"/></dd> <dt pn="section-2.2-2.5">Stream</dt> <dd pn="section-2.2-2.6"> The term used by IEEE 802.1termfor a DetNet flow.</t> <t hangText="Talker"><vspace blankLines="0"/></dd> <dt pn="section-2.2-2.7">Talker</dt> <dd pn="section-2.2-2.8"> The term used by IEEE 802.1termfor the source of a DetNet flow.</t> </list> </t></dd> </dl> </section> </section> <section anchor="ProvidingQoS"title="Providingnumbered="true" toc="include" removeInRFC="false" pn="section-3"> <name slugifiedName="name-providing-the-detnet-qualit">Providing the DetNet Quality ofService">Service</name> <section anchor="DefiningGoals"title="Primary goals definingnumbered="true" toc="include" removeInRFC="false" pn="section-3.1"> <name slugifiedName="name-primary-goals-defining-the-">Primary Goals Defining the DetNetQoS"> <t>QoS</name> <t pn="section-3.1-1"> The DetNetQuality of ServiceQoS can be expressed in terms of:<list style="symbols"> <t></t> <ul spacing="normal" bare="false" empty="false" pn="section-3.1-2"> <li pn="section-3.1-2.1"> Minimum and maximum end-to-end latency from source todestination;destination, timely delivery, and bounded jitter (packet delay variation) derived from these constraints.</t> <t></li> <li pn="section-3.1-2.2"> Packet lossratio,ratio under various assumptions as to the operational states of the nodes and links.</t><t></li> <li pn="section-3.1-2.3"> An upper bound on out-of-order packet delivery. It is worth noting that some DetNet applications are unable to tolerate any out-of-order delivery.</t> </list> </t><t></li> </ul> <t pn="section-3.1-3"> It is a distinction of DetNet that it is concerned solely with worst-case values for the end-to-end latency, jitter, and misordering. Average, mean, or typical values are of little interest, because they do not affect the ability of a real-time system to perform its tasks. In general, a trivial priority-based queuing scheme will give better average latency to a data flow than DetNet; however, it may not be a suitable option for DetNet because of its worst-case latency.</t><t></t> <t pn="section-3.1-4"> Three techniques are used by DetNet to provide these qualities of service:<list style="symbols"> <t></t> <ul spacing="normal" bare="false" empty="false" pn="section-3.1-5"> <li pn="section-3.1-5.1"> Resource allocation (<xreftarget="Zero"/>). </t><t>target="Zero" format="default" sectionFormat="of" derivedContent="Section 3.2.1"/>) </li> <li pn="section-3.1-5.2"> Service protection (<xreftarget="srvcprot"/>). </t><t>target="srvcprot" format="default" sectionFormat="of" derivedContent="Section 3.2.2"/>) </li> <li pn="section-3.1-5.3"> Explicit routes (<xreftarget="pinned"/>). </t> </list> </t><t>target="pinned" format="default" sectionFormat="of" derivedContent="Section 3.2.3"/>) </li> </ul> <t pn="section-3.1-6"> Resource allocation operates by assigning resources, e.g., buffer space or link bandwidth, to a DetNet flow (or flow aggregate) along its path. Resource allocation greatly reduces, or even eliminates entirely, packet loss due to output packet contention within the network, but it can only be supplied to a DetNet flow that is limited at the source to a maximum packet size and transmission rate. As DetNet flows are assumed to berate-limitedrate limited and DetNet is designed to provide sufficient allocated resources (including provisioned capacity), the use oftransport layertransport-layer congestion control <xreftarget="RFC2914"/>target="RFC2914" format="default" sectionFormat="of" derivedContent="RFC2914"/> for App-flows is not required; however, if resources are allocated appropriately, use of congestion control should not impact transmission negatively.</t><t></t> <t pn="section-3.1-7"> Resource allocation addresses two of the DetNet QoS requirements: latency and packet loss. Given that DetNet nodes have a finite amount of buffer space, resource allocation necessarily results in a maximum end-to-end latency.It alsoResource allocation also addressescontention relatedcontention-related packet loss.</t><t></t> <t pn="section-3.1-8"> Other importantcontributioncontributions to packet loss are random media errors and equipment failures. Service protection is the name for the mechanisms used by DetNet to address these losses. The mechanisms employed are constrained by therequirementneed to meet the users' latency requirements. Packet replication and elimination (<xreftarget="srvcprot"/>)target="Seamless" format="default" sectionFormat="of" derivedContent="Section 3.2.2.2"/>) and packet encoding (<xreftarget="PacketEncoding"/>)target="PacketEncoding" format="default" sectionFormat="of" derivedContent="Section 3.2.2.3"/>) are described in this document to provide serviceprotection; othersprotection, but other mechanisms may also be found. For instance, packet encoding can be used to provide service protection against random media errors, while packet replication and elimination can be used to provide service protection against equipment failures. This mechanism distributes the contents of DetNet flows over multiple paths in time and/or space, so that the loss of some of the paths does need not cause the loss of any packets.</t><t></t> <t pn="section-3.1-9"> The paths are typically (but not necessarily) explicitroutes,routes so that they do not normally suffer temporary interruptions caused by the convergence of routing or bridging protocols.</t><t></t> <t pn="section-3.1-10"> These three techniques can be appliedindependently, givingindividually or applied together; it results that eightpossiblecombinations, including none (no DetNet),although some combinationsare possible. Some combinations, however, are of wider utility than others. This separation keeps the protocol stack coherent and maximizes interoperability with existing and developing standards inthis (IETF)the IETF and other Standards Development Organizations.SomeThe following are examples of typical expected combinations:<list style="symbols"> <t> Explicit</t> <ul spacing="normal" bare="false" empty="false" pn="section-3.1-11"> <li pn="section-3.1-11.1"> The combination of explicit routesplusand service protectionare exactlyis thetechniquestechnique employed by seamless redundancy mechanisms applied on a ringtopology as described,topology, e.g., as described in <xreftarget="IEC62439-3-2016"/>.target="IEC-62439-3" format="default" sectionFormat="of" derivedContent="IEC-62439-3"/>. In this example, explicit routes are achieved by limiting the physical topology of the network to a ring. Sequentialization, replication, and duplicate elimination are facilitated by packet tags added at the front or the end of Ethernet frames. <xreftarget="RFC8227"/>target="RFC8227" format="default" sectionFormat="of" derivedContent="RFC8227"/> provides another example in the context of MPLS.</t><t></li> <li pn="section-3.1-11.2"> Resource allocation alone was originally offered byIEEE 802.1Audio VideobridgingBridging as defined by IEEE 802.1 <xreftarget="IEEE802.1BA"/>.target="IEEE802.1BA" format="default" sectionFormat="of" derivedContent="IEEE802.1BA"/>. As long as the network suffers no failures, packet loss due to output packet contention can be eliminated through the use of a reservation protocol (e.g., the Multiple Stream Registration Protocol <xreftarget="IEEE802.1Q-2018"/>),target="IEEE802.1Q" format="default" sectionFormat="of" derivedContent="IEEE802.1Q"/>), shapers in every bridge, and proper dimensioning.</t><t></li> <li pn="section-3.1-11.3"> Using all three together gives maximum protection.</t> </list> </t><t></li> </ul> <t pn="section-3.1-12"> There are, of course, simpler methods available (andemployed,employed today) to achieve levels of latency and packet loss that are satisfactory for many applications. Prioritization and over-provisioning is one such technique. However, these methods generally work best in the absence of any significant amount ofnon-criticalnoncritical traffic in the network (if, indeed, such traffic is supported atall), orall). They may also work only if the critical traffic constitutes only a small portion of the network's theoretical capacity,or work onlyif all systems are functioning properly, orin the absence ofif actions by end systems that disrupt the network'soperations. </t><t>operations are absent. </t> <t pn="section-3.1-13"> There are any number of methods in use, defined, or in progress for accomplishing each of the above techniques. It is expected thatthisthe DetNetArchitecturearchitecture defined in this document will assist various vendors, users, and/or "vertical" Standards Development Organizations (dedicated to a single industry)to makein making selections among the available means of implementing DetNet networks. </t> </section> <section anchor="Techniques"title="Mechanismsnumbered="true" toc="include" removeInRFC="false" pn="section-3.2"> <name slugifiedName="name-mechanisms-to-achieve-detne">Mechanisms toachieveAchieve DetNetQoS">QoS</name> <section anchor="Zero"title="Resource allocation">numbered="true" toc="include" removeInRFC="false" pn="section-3.2.1"> <name slugifiedName="name-resource-allocation">Resource Allocation</name> <section anchor="ZeroL"title="Eliminate contention loss"> <t>numbered="true" toc="exclude" removeInRFC="false" pn="section-3.2.1.1"> <name slugifiedName="name-eliminate-contention-loss">Eliminate Contention Loss</name> <t pn="section-3.2.1.1-1"> The primary means by which DetNet achieves its QoS assurances is to reduce, or even completelyeliminateeliminate, packet loss due to output packet contention within a DetNet node as a cause of packet loss. This can be achieved only by the provision of sufficient buffer storage at each node through the network to ensure that no packets are dropped due to a lack of buffer storage. Note that App-flows are generally not expected to be responsive to implicit <xreftarget="RFC2914"/>target="RFC2914" format="default" sectionFormat="of" derivedContent="RFC2914"/> or explicit congestion notification <xreftarget="RFC3168"/>. </t><t>target="RFC3168" format="default" sectionFormat="of" derivedContent="RFC3168"/>. </t> <t pn="section-3.2.1.1-2"> Ensuring adequate buffering requires, in turn, that thesource,source and every DetNet node along the path to the destination (or nearly everynode,node; see <xreftarget="Incomplete"/>)target="Incomplete" format="default" sectionFormat="of" derivedContent="Section 4.3.3"/>) be careful to regulate its output to not exceed the data rate for any DetNet flow, except for brief periods when making up for interfering traffic. Any packet sent ahead of its time potentially adds to the number of buffers required by thenext hopnext-hop DetNet node and may thus exceed the resources allocated for a particular DetNet flow. Furthermore, ratelimiting, e.g.,limiting (e.g., using trafficpolicingpolicing) and shapingfunctions, e.g.,functions (e.g., shaping as defined in <xreftarget="RFC2475"/>,target="RFC2475" format="default" sectionFormat="of" derivedContent="RFC2475"/>) at the ingress of the DetNet domain must be applied. This is needed for meeting the requirements of DetNet flows as well as for protecting non-DetNet traffic from potentially misbehaving DetNet traffic sources. Note that large buffers have someissues, see,issues (see, e.g., <xreftarget="BUFFERBLOAT"/>. </t><t>target="BUFFERBLOAT" format="default" sectionFormat="of" derivedContent="BUFFERBLOAT"/>). </t> <t pn="section-3.2.1.1-3"> The low-level mechanisms described in <xreftarget="QueuingModels"/>target="QueuingModels" format="default" sectionFormat="of" derivedContent="Section 4.5"/> provide the necessary regulation of transmissions by an end system or DetNet node to provide resource allocation. The allocation of the bandwidth and buffers for a DetNet flow requires provisioning. A DetNet node may have other resources requiring allocation and/orscheduling,scheduling that might otherwise be over-subscribed and trigger the rejection of a reservation. </t> </section> <section anchor="Jitterless"title="Jitter Reduction"> <t>numbered="true" toc="exclude" removeInRFC="false" pn="section-3.2.1.2"> <name slugifiedName="name-jitter-reduction">Jitter Reduction</name> <t pn="section-3.2.1.2-1"> A core objective of DetNet is to enable the convergence of sensitive non-IP networks onto a common network infrastructure. This requires the accurate emulation of currently deployed mission-specific networks,whichwhich, forexampleexample, rely on point-to-point analog (e.g., 4-20mA modulation) and serial-digital cables (or buses) for highly reliable,synchronizedsynchronized, and jitter-free communications. While the latency of analog transmissions is basically the speed of light, legacy serial links are usually slow (in the order of Kbps) compared to, say, Gigabit Ethernet, and some latency is usually acceptable. What is not acceptable is the introduction of excessive jitter, which may, for instance, affect the stability of control systems. </t><t>Applications<t pn="section-3.2.1.2-2">Applications that are designed to operate on serial links usually do not provide services to recover the jitter, because jitter simply does not exist there. DetNet flows are generally expected to be deliveredin-orderin order, and the precise time of reception influences the processes. In order to converge such existing applications, there is a desire to emulate all properties of the serial cable, such as clock transportation, perfect flowisolationisolation, and fixed latency. While minimal jitter (in the form of specifying minimum, as well as maximum, end-to-end latency) is supported by DetNet, there are practical limitations on packet-based networks in this regard. In general, users are encouraged to use a combination of:<list style="symbols"> <t></t> <ul spacing="normal" bare="false" empty="false" pn="section-3.2.1.2-3"> <li pn="section-3.2.1.2-3.1"> Sub-microsecond time synchronization among all source and destination end systems, and</t><t></li> <li pn="section-3.2.1.2-3.2"> Time-of-execution fields in the application packets.</t> </list> </t><t></li> </ul> <t pn="section-3.2.1.2-4"> Jitter reduction is provided by the mechanisms described in <xreftarget="QueuingModels"/>target="QueuingModels" format="default" sectionFormat="of" derivedContent="Section 4.5"/> that also provide resource allocation. </t> </section> </section> <section anchor="srvcprot"title="Service Protection"> <t>numbered="true" toc="include" removeInRFC="false" pn="section-3.2.2"> <name slugifiedName="name-service-protection">Service Protection</name> <t pn="section-3.2.2-1"> Service protection aims to mitigate or eliminate packet loss due to equipment failures, including random media and/or memory faults. These types of packet loss can be greatly reduced by spreading the data over multiple disjoint forwarding paths. Various service protection methods are described in <xreftarget="RFC6372"/>,target="RFC6372" format="default" sectionFormat="of" derivedContent="RFC6372"/>, e.g., 1+1 linear protection.This section describes theThe functional details of an additional method are described in <xreftarget="Seamless"/>,target="Seamless" format="default" sectionFormat="of" derivedContent="Section 3.2.2.2"/>, which can be implemented as described in <xreftarget="PacketEncoding"/>target="PacketEncoding" format="default" sectionFormat="of" derivedContent="Section 3.2.2.3"/> or as specified in <xreftarget="I-D.ietf-detnet-dp-sol-mpls"/>target="I-D.ietf-detnet-mpls" format="default" sectionFormat="of" derivedContent="DETNET-MPLS"/> in order to provide 1+n hitless protection. The appropriate service protection mechanism depends on the scenario and the requirements. </t> <section anchor="inorder"title="In-Order Delivery"> <t>numbered="true" toc="exclude" removeInRFC="false" pn="section-3.2.2.1"> <name slugifiedName="name-in-order-delivery">In-Order Delivery</name> <t pn="section-3.2.2.1-1"> Out-of-order packet delivery can be a side effect of service protection. Packets deliveredout-of-orderout of order impact the amount of buffering needed at the destination to properly process the received data. Such packets also influence the jitter of a flow. The guarantees of a DetNet serviceincludesinclude a maximumallowedamount of misordering as a constraint. Zero misordering would be a valid service constraint to reflect that the end system(s) of the flow cannot tolerate any out-of-order delivery. A DetNet Packet OrderingFunctionalityFunction (POF) (<xreftarget="Seamless"/>)target="Seamless" format="default" sectionFormat="of" derivedContent="Section 3.2.2.2"/>) can be used to provide in-order delivery. </t> </section> <section anchor="Seamless"title="Packetnumbered="true" toc="exclude" removeInRFC="false" pn="section-3.2.2.2"> <name slugifiedName="name-packet-replication-and-elim">Packet Replication andElimination"> <t>Elimination</name> <t pn="section-3.2.2.2-1"> This section describes a service protection method that sends copies of the same packets over multiple paths.</t><t></t> <t pn="section-3.2.2.2-2"> The DetNet service sub-layer includes thepacket replication (PRF), the packet elimination (PEF),PRF, PEF, andthe packet ordering functionality (POF)POF for use in DetNet edge, relay node, andend systemend-system packet processing. These functions can be enabled in a DetNet edge node, relaynodenode, or end system. The collective name for all three functions is Packet Replication, Elimination, and Ordering Functions (PREOF). The packet replication and elimination service protection method altogether involves four capabilities:<list style="symbols"> <t> Providing sequencing</t> <ul spacing="normal" bare="false" empty="false" pn="section-3.2.2.2-3"> <li pn="section-3.2.2.2-3.1"> Sequencing information is provided to the packets of a DetNet compound flow. This may be done by adding a sequence number or time stamp as part of DetNet, or it may be inherent in the packet, e.g., in ahigher layer protocol,higher-layer protocol or associated to other physical properties such as the precise time (and radio channel) of reception of the packet. This is typically done once, at or near the source.</t><t></li> <li pn="section-3.2.2.2-3.2"> ThePacket Replication Function (PRF)PRF replicates these packets into multiple DetNet member flows and typically sends them along multiple different paths to the destination(s), e.g., over the explicit routesofdescribed in <xreftarget="pinned"/>.target="pinned" format="default" sectionFormat="of" derivedContent="Section 3.2.3"/>. The location within a DetNetnode,node and the mechanism used for the PRFisare left open for implementations.</t><t></li> <li pn="section-3.2.2.2-3.3"> ThePacket Elimination Function (PEF)PEF eliminates duplicate packets of a DetNet flow based on the sequencing information and a history of received packets. The output of the PEF is always a single packet. This may be done at any DetNet node along the path to save network resources further downstream, in particular if multipleReplicationreplication points exist. But the most common case is to perform this operation at the very edge of the DetNet network, preferably in or near the receiver. The location within a DetNetnode,node and the mechanism used for the PEF is left open for implementations.</t><t></li> <li pn="section-3.2.2.2-3.4"> ThePacket Ordering Function (POF)POF uses the sequencing information tore-orderreorder a DetNet flow's packets that are received out of order.</t> </list> </t><t></li> </ul> <t pn="section-3.2.2.2-4"> The order in which a DetNet node applies PEF, POF, and PRF to a DetNet flow is left open for implementations.</t><t></t> <t pn="section-3.2.2.2-5"> Some service protection mechanisms rely on switching from one flow to another when a failure of a flow is detected. Contrarily, packet replication and elimination combines the DetNet member flows sent along multiple differentpaths,paths and performs a packet-by-packet selection of which to discard, e.g., based on sequencing information.</t><t> In</t> <t pn="section-3.2.2.2-6">In the simplest case, this amounts to 1) replicating each packet in a source that has twointerfaces,interfaces and 2) conveying them through thenetwork,network along separate (Shared Risk Link Group (SRLG) disjoint)paths,paths to the similarly dual-homeddestinations,destinations that 3) reorder the packets and 4) discard theextras.duplicates. This ensures that one path remains, even if some DetNet intermediate node fails. The sequencing information can also be used for loss detection and forre-ordering. </t><t>reordering. </t> <t pn="section-3.2.2.2-7"> DetNet relay nodes in the network can provide replication and elimination facilities at various points in thenetwork,network so that multiple failures can be accommodated.</t><t></t> <t pn="section-3.2.2.2-8"> This is shown in <xreftarget="FigSeamless"/>,target="FigSeamless" format="default" sectionFormat="of" derivedContent="Figure 1"/>, where the two relay nodes each replicate (R) the DetNet flow on input, sending the DetNet member flows to both the other relay node and to the end system, and eliminate duplicates (E) on the output interface to the right-hand end system. Any one link in the network can fail, and the DetNet compound flow can still get through. Furthermore, two links can fail, as long as they are in different segments of the network. </t> <figurealign="center"anchor="FigSeamless"title="Packet replicationalign="left" suppress-title="false" pn="figure-1"> <name slugifiedName="name-packet-replication-and-elimi">Packet Replication andelimination">Elimination</name> <artworkalign="left"><![CDATA[ > > > > > > > > >align="left" name="" type="" alt="" pn="section-3.2.2.2-9.1"> > > > > > > > > > relay> > > > > > > > >> > > > > > > > > /------------+ R node E +------------\> >> > / v + ^ \>> end R + v | ^ + E end system + v | ^ + system>> \ v + ^ /> >> > \------------+ R relay E +-----------/> > > > > > > > > >> > > > > > > > > > node> > > > > > > > ]]></artwork>> > > > > > > ></artwork> </figure><t> Packet<t pn="section-3.2.2.2-10">Packet replication and elimination does not react to and correct failures; it is entirely passive. Thus, intermittent failures, mistakenly created packet filters, or misrouted data is handled just the same as the equipment failures that are handled by typical routing and bridging protocols.</t><t> If</t> <t pn="section-3.2.2.2-11">If member flows that take different-length paths through the network are combined, a merge point may require extra buffering to equalize the delays over the different paths. This equalization ensures that the resultant compound flow will not exceed its contracted bandwidth even after oneor the otherof the paths is restored after a failure. The extra buffering can be also used to provide in-order delivery. </t> </section> <section anchor="PacketEncoding"title="Packet encodingnumbered="true" toc="exclude" removeInRFC="false" pn="section-3.2.2.3"> <name slugifiedName="name-packet-encoding-for-service">Packet Encoding forservice protection"> <t>Service Protection</name> <t pn="section-3.2.2.3-1"> There are methods for using multiple paths to provide service protection that involve encoding the information in a packet belonging to a DetNet flow into multiple transmission units, combining information from multiple packets into any given transmission unit. Such techniques, also known as "network coding", can be used as a DetNet service protection technique. </t> </section> </section> <section anchor="pinned"title="Explicit routes"> <t>numbered="true" toc="include" removeInRFC="false" pn="section-3.2.3"> <name slugifiedName="name-explicit-routes">Explicit Routes</name> <t pn="section-3.2.3-1"> In networks controlled by typical dynamic control protocols such as IS-IS or OSPF, a network topology event in one part of the network can impact, at least briefly, the delivery of data in parts of the network remote from the failure or recovery event. Even the use of redundant paths through a network, e.g., as defined by <xreftarget="RFC6372"/> dotarget="RFC6372" format="default" sectionFormat="of" derivedContent="RFC6372"/>, does not eliminate the chances of packet loss. Furthermore, out-of-order packet delivery can be a side effect of route changes.</t><t></t> <t pn="section-3.2.3-2"> Many real-time networks rely on physical rings of two-port devices, with a relatively simple ring control protocol. This supports redundant paths for service protection with a minimum of wiring. As an additional benefit, ring topologies can often utilize different topology management protocolsthanfrom those used for a mesh network, with a consequent reduction in the response time to topology changes. Of course, this comes at some cost in terms of increased hop count, and thus latency, for the typical path.</t><t></t> <t pn="section-3.2.3-3"> In order to get the advantages of low hop count and still ensure against even very brief losses of connectivity, DetNet employs explicitroutes,routes where the path taken by a given DetNet flow does not change, at leastimmediately,not immediately and likely not at all, in response to network topology events. Service protection(<xref target="srvcprot"/> or(see Sections <xref target="srvcprot" format="counter" sectionFormat="of" derivedContent="3.2.2"/> and <xreftarget="PacketEncoding"/>)target="PacketEncoding" format="counter" sectionFormat="of" derivedContent="3.2.2.3"/>) over explicit routes provides a high likelihood of continuous connectivity. Explicit routes can be established in various ways, e.g., with RSVP-TE <xreftarget="RFC3209"/>,target="RFC3209" format="default" sectionFormat="of" derivedContent="RFC3209"/>, with Segment Routing (SR) <xreftarget="RFC8402"/>,target="RFC8402" format="default" sectionFormat="of" derivedContent="RFC8402"/>, via aSoftware Defined NetworkingSDN approach <xreftarget="RFC8453"/>,target="RFC8453" format="default" sectionFormat="of" derivedContent="RFC8453"/>, with IS-IS <xreftarget="RFC7813"/>,target="RFC7813" format="default" sectionFormat="of" derivedContent="RFC7813"/>, etc. Explicit routes are typically used in MPLS TELSPs. </t><t>(Traffic Engineering) Label Switched Paths (LSPs). </t> <t pn="section-3.2.3-4"> Out-of-order packet delivery can be a side effect of distributing a single flow over multiple paths, especially when there is a change from one path to another when combining the flow. This is irrespective of the distribution methodused,used and also applies to service protection over explicit routes. As described in <xreftarget="inorder"/>,target="inorder" format="default" sectionFormat="of" derivedContent="Section 3.2.2.1"/>, out-of-order packets influence the jitter of a flow and impact the amount of buffering needed to process the data; therefore, the guarantees of a DetNet serviceincludesinclude a maximumallowedamount of misordering as a constraint. The use of explicit routes helps to provide in-order delivery because there is no immediate route change with the network topology, but the changes are plannable as they are between the different explicit routes. </t> </section> </section> <section anchor="MoreGoals"title="Secondary goalsnumbered="true" toc="include" removeInRFC="false" pn="section-3.3"> <name slugifiedName="name-secondary-goals-for-detnet">Secondary Goals forDetNet"> <t>DetNet</name> <t pn="section-3.3-1"> Many applications require DetNet to provide additional services, including coexistence with other QoS mechanisms<xref target="Coexistence"/>(<xref target="Coexistence" format="default" sectionFormat="of" derivedContent="Section 3.3.1"/>) and protection against misbehaving transmitters<xref target="FaultMitigation"/>.(<xref target="FaultMitigation" format="default" sectionFormat="of" derivedContent="Section 3.3.2"/>). </t> <section anchor="Coexistence"title="Coexistencenumbered="true" toc="include" removeInRFC="false" pn="section-3.3.1"> <name slugifiedName="name-coexistence-with-normal-tra">Coexistence withnormal traffic"> <t>Normal Traffic</name> <t pn="section-3.3.1-1"> A DetNet network supports the dedication of a high proportion of the network bandwidth to DetNet flows. But, no matter how much is dedicated for DetNet flows, it is a goal of DetNet to coexist with existingClass of ServiceClass-of-Service schemes (e.g., DiffServ). It is also important that non-DetNet traffic not disrupt the DetNet flow, of course (see Sections <xreftarget="FaultMitigation"/>target="FaultMitigation" format="counter" sectionFormat="of" derivedContent="3.3.2"/> and <xreftarget="SecurityConsiderations"/>).target="SecurityConsiderations" format="counter" sectionFormat="of" derivedContent="5"/>). For these reasons:<list style="symbols"> <t> Bandwidth</t> <ul spacing="normal" bare="false" empty="false" pn="section-3.3.1-2"> <li pn="section-3.3.1-2.1">Bandwidth (transmission opportunities) not utilized by a DetNet flow is available to non-DetNet packets (though not to other DetNet flows).</t><t></li> <li pn="section-3.3.1-2.2"> DetNet flows can be shaped or scheduled, in order to ensure that the highest-priority non-DetNet packet is also ensured a worst-case latency.</t><t></li> <li pn="section-3.3.1-2.3"> When transmission opportunities for DetNet flows are scheduled in detail,thenthe algorithm constructing the schedule should leave sufficient opportunities for non-DetNet packets to satisfy the needs of the users of the network. Detailed scheduling can also permit the time-shared use of buffer resources by different DetNet flows.</t> </list> </t><t></li> </ul> <t pn="section-3.3.1-3"> Starvation of non-DetNet traffic must be avoided,e.g.,for example, by traffic policing and shaping functions (e.g., <xreftarget="RFC2475"/>).target="RFC2475" format="default" sectionFormat="of" derivedContent="RFC2475"/>). Thus, the net effect of the presence of DetNet flows in a network on the non-DetNet flows is primarily a reduction in the available bandwidth. </t> </section> <section anchor="FaultMitigation"title="Fault Mitigation"> <t>numbered="true" toc="include" removeInRFC="false" pn="section-3.3.2"> <name slugifiedName="name-fault-mitigation">Fault Mitigation</name> <t pn="section-3.3.2-1"> Robust real-time systems require reducing the number of possible failures. Filters and policers should be used in a DetNet network to detect if DetNet packets are received on the wrong interface,orat the wrong time, or in too great a volume. Furthermore, filters and policers can take actions to discard the offending packets or flows, or trigger shutting down the offending flow or the offending interface.</t><t></t> <t pn="section-3.3.2-2"> It is also essential that filters and service remarking be employed at the network edge to prevent non-DetNet packets from being mistaken for DetNetpackets,packets and thus impinging on the resources allocated to DetNet packets. In particular, sending DetNet traffic into networks that have not been provisioned in advance to handle that DetNet traffic has to be treated as a fault. The use of egress traffic filters, or equivalent mechanisms, to prevent this from happening are strongly recommended at the edges ofaDetNet networks and DetNet supporting networks. In this context, the term 'provisioned' has a broad meaning, e.g., provisioning could be performed via an administrative decision that the downstream network has the available capacity to carry the DetNet traffic that is being sent into it.</t><t></t> <t pn="section-3.3.2-3"> Note that the sending of App-flows that do not usetransport layertransport-layer congestion control per <xreftarget="RFC2914"/>target="RFC2914" format="default" sectionFormat="of" derivedContent="RFC2914"/> into a network that is not provisioned to handle suchDetNettraffic has to be treated as a fault and prevented.PRF generatedPRF-generated DetNet member flows also need to be treated as not usingtransport layertransport-layer congestion control even if the original App-flow supportstransport layertransport-layer congestion control because PREOF can remove congestion indications at the PEF and thereby hide such indications (e.g., drops, ECN markings, increased latency) from end systems.</t><t></t> <t pn="section-3.3.2-4"> The mechanisms to support these requirements are bothdata planeData Plane and implementation specific.Data planeSolutions that are data-plane specificsolutionswill be specified in the relevantdata planedata-plane solution document. There also exist techniques, at present and/or in various stages of standardization, that can support thesefault mitigationfault-mitigation tasks that deliver a high probability that misbehaving systems will have zero impact on well-behaved DetNetflows, exceptflows with the exception, of course,forof the receiving interface(s) immediately downstreamoffrom the misbehaving device. Examples of such techniques include traffic policing and shaping functions (e.g., those described in <xreftarget="RFC2475"/>) andtarget="RFC2475" format="default" sectionFormat="of" derivedContent="RFC2475"/>), separating flows into per-flow rate-limited queues, and potentiallyapplyapplying active queue management <xreftarget="RFC7567"/>.target="RFC7567" format="default" sectionFormat="of" derivedContent="RFC7567"/>. </t> </section> </section> </section> <section anchor="arch"title="DetNet Architecture">numbered="true" toc="include" removeInRFC="false" pn="section-4"> <name slugifiedName="name-detnet-architecture">DetNet Architecture</name> <section anchor="Stacks"title="DetNet stack model"> <t>numbered="true" toc="include" removeInRFC="false" pn="section-4.1"> <name slugifiedName="name-detnet-stack-model">DetNet Stack Model</name> <t pn="section-4.1-1"> DetNet functionality (<xreftarget="ProvidingQoS"/>)target="ProvidingQoS" format="default" sectionFormat="of" derivedContent="Section 3"/>) is implemented in two adjacent sub-layers in the protocol stack: the DetNet service sub-layer and the DetNet forwarding sub-layer. The DetNet service sub-layer provides DetNet service, e.g., service protection, to higher layers in the protocol stack and applications. The DetNet forwarding sub-layer supports DetNet service in the underlying network, e.g., by providing explicit routes and resource allocation to DetNet flows. </t> <section anchor="StackModel"title="Representativenumbered="true" toc="include" removeInRFC="false" pn="section-4.1.1"> <name slugifiedName="name-representative-protocol-sta">Representative Protocol StackModel"> <t> <xref target="ProtStack1"/>Model</name> <t pn="section-4.1.1-1"><xref target="ProtStack1" format="default" sectionFormat="of" derivedContent="Figure 2"/> illustrates a conceptual DetNetdata planedata-plane layering model. One may compare it to that in <xreftarget="IEEE802.1CB"/>,target="IEEE802.1CB" format="default" sectionFormat="of" derivedContent="IEEE802.1CB"/>, Annex C. </t> <figurealign="center"anchor="ProtStack1"title="DetNet data plane protocol stack">align="left" suppress-title="false" pn="figure-2"> <name slugifiedName="name-detnet-data-plane-protocol-">DetNet Data-Plane Protocol Stack</name> <artworkalign="center"><![CDATA[align="center" name="" type="" alt="" pn="section-4.1.1-2.1"> | packets going | ^ packets coming ^ v down the stack v | up the stack | +-----------------------+ +-----------------------+ | Source | | Destination | +-----------------------+ +-----------------------+ | Service sub-layer: | | Service sub-layer: | | Packet sequencing | | Duplicate elimination | | Flow replication | | Flow merging | | Packet encoding | | Packet decoding | +-----------------------+ +-----------------------+ | Forwarding sub-layer: | | Forwarding sub-layer: | | Resource allocation | | Resource allocation | | Explicit routes | | Explicit routes | +-----------------------+ +-----------------------+ | Lower layers | | Lower layers | +-----------------------+ +-----------------------+ v ^ \_________________________/]]></artwork></artwork> </figure><t><t pn="section-4.1.1-3"> Not all sub-layers are required for any given application, or even for any given network. The functionality shown in <xreftarget="ProtStack1"/>target="ProtStack1" format="default" sectionFormat="of" derivedContent="Figure 2"/> is:<list hangIndent="8" style="hanging"> <t hangText="Application"><vspace blankLines="0"/></t> <dl newline="true" spacing="normal" indent="3" pn="section-4.1.1-4"> <dt pn="section-4.1.1-4.1">Application</dt> <dd pn="section-4.1.1-4.2"> Shown as "source" and "destination" in the diagram.</t> <t hangText="Packet sequencing"><vspace blankLines="0"/></dd> <dt pn="section-4.1.1-4.3">Packet sequencing</dt> <dd pn="section-4.1.1-4.4"> As part of the DetNet serviceprotection,sub-layer, the packet sequencing function supplies the sequence number for packet replication and elimination for DetNet service protection (<xreftarget="srvcprot"/>), thus peers with Duplicatetarget="Seamless" format="default" sectionFormat="of" derivedContent="Section 3.2.2.2"/>); thus, its peer is duplicate elimination. This sub-layer is not needed if ahigher layerhigher-layer protocol is expected to perform any packet sequencing and duplicate elimination required by the DetNet flow replication.</t> <t hangText="Duplicate elimination"><vspace blankLines="0"/></dd> <dt pn="section-4.1.1-4.5">Duplicate elimination</dt> <dd pn="section-4.1.1-4.6"> As part of the DetNet service sub-layer, based on thesequencedsequence number supplied by itspeer, packet sequencing, Duplicatepeer (packet sequencing), duplicate elimination discards any duplicate packets generated by DetNet flow replication. It can operate on member flows, compound flows, or both. The replication may also be inferred from other information such as the precise time of reception in a scheduled network. The duplicate elimination sub-layer may also perform resequencing of packets to restore packet order in a flow that was disrupted by the loss of packets on one or another of the multiple paths taken.</t> <t hangText="Flow replication"><vspace blankLines="0"/></dd> <dt pn="section-4.1.1-4.7">Flow replication</dt> <dd pn="section-4.1.1-4.8"> As part of DetNet service protection, packets that belong to a DetNet compound flow are replicated into two or more DetNet member flows. This function is separate from packet sequencing. Flow replication can be an explicit replication and remarking ofpackets,packets or can be performed by, for example, techniques similar to ordinary multicast replication, albeit with resource allocation implications.Peers withIts peer is DetNet flow merging.</t> <t hangText="Flow merging"><vspace blankLines="0"/></dd> <dt pn="section-4.1.1-4.9">Flow merging</dt> <dd pn="section-4.1.1-4.10"> As part of the DetNet serviceprotection, mergessub-layer, the flow merging function combines DetNet member flows together for packets coming up the stack belonging to a specific DetNet compound flow.Peers with DetNet flow replication.DetNet flow merging, together with packet sequencing, duplicate elimination, and DetNet flow replication perform packet replication and elimination (<xreftarget="srvcprot"/>). </t> <t hangText="Packet encoding"><vspace blankLines="0"/>target="srvcprot" format="default" sectionFormat="of" derivedContent="Section 3.2.2"/>). Its peer is DetNet flow replication. </dd> <dt pn="section-4.1.1-4.11">Packet encoding</dt> <dd pn="section-4.1.1-4.12"> As part of DetNet service protection, as an alternative to packet sequencing and flow replication, packet encoding combines the information in multiple DetNet packets, perhaps from different DetNet compound flows, and transmits that information in packets on different DetNet memberFlows. Peers with Packetflows. Its peer is packet decoding.</t> <t hangText="Packet decoding"><vspace blankLines="0"/></dd> <dt pn="section-4.1.1-4.13">Packet decoding</dt> <dd pn="section-4.1.1-4.14"> As part of DetNet service protection, as an alternative to flow merging and duplicate elimination, packet decoding takes packets from different DetNet memberflows,flows and computes from those packets the original DetNet packets from the compound flows input to packet encoding.Peers with PacketIts peer is packet encoding.</t> <t hangText="Resource allocation"><vspace blankLines="0"/></dd> <dt pn="section-4.1.1-4.15">Resource allocation</dt> <dd pn="section-4.1.1-4.16"> The DetNet forwarding sub-layer provides resource allocation. See <xreftarget="QueuingModels"/>.target="QueuingModels" format="default" sectionFormat="of" derivedContent="Section 4.5"/>. The actual queuing and shaping mechanisms are typically provided by the underlying subnet. These can be closely associated with the means of providing paths for DetNet flows. The path and the resource allocation are conflated in this figure.</t> <t hangText="Explicit routes"><vspace blankLines="0"/> The</dd> <dt pn="section-4.1.1-4.17">Explicit routes</dt> <dd pn="section-4.1.1-4.18"> Explicit routes are arrangements of fixed paths operated at the DetNet forwarding sub-layerprovides mechanisms to ensurethatfixed pathsareprovided for DetNet flows. These explicit pathsdetermined in advance to avoid the impact of networkconvergence. </t> </list> </t> <t>convergence on DetNet flows. </dd> </dl> <t pn="section-4.1.1-5"> Operations, Administration, and Maintenance (OAM) leverages in-band and out-of-band signaling that validates whether the service is effectively obtained within QoS constraints. OAM is not shown in <xreftarget="ProtStack1"/>;target="ProtStack1" format="default" sectionFormat="of" derivedContent="Figure 2"/>; it may reside in any number of the layers. OAM can involve specific tagging added in the packets for tracing implementation or network configuration errors; traceability enablesto findfinding whether a packet is a replica, which DetNet relay node performed the replication, and which segment was intended for the replica. Active and hybrid OAM methods require additional bandwidth to perform fault management and performance monitoring of the DetNet domain. OAM may, for instance, generate special test probes or add OAM information into the data packet. </t><t><t pn="section-4.1.1-6"> The packetsequencing andreplication and elimination functions may be performed either at the source and destination ends of a DetNet compound flowmay be performed either in the end systemor in a DetNet relay node. </t> </section> <sectiontitle="DetNet Data Plane Overview" anchor="sec_dt_dp"> <t>anchor="sec_dt_dp" numbered="true" toc="include" removeInRFC="false" pn="section-4.1.2"> <name slugifiedName="name-detnet-data-plane-overview">DetNet Data-Plane Overview</name> <t pn="section-4.1.2-1"> A "Deterministic Network" will be composed ofDetNet enabledDetNet-enabled end systems, DetNet edge nodes, and DetNet relay nodes, which collectively deliver DetNet services. DetNet relay and edge nodes are interconnected via DetNet transit nodes (e.g.,LSRs)LSRs), which supportDetNet,DetNet but are not DetNet service aware. All DetNet nodes are connected to sub-networks, where a point-to-point link is also consideredasa simple sub-network. These sub-networkswillprovideDetNet compatibleDetNet-compatible service for support of DetNet traffic. Examples of sub-network technologies include MPLS TE,IEEE 802.1TSN as defined by IEEE 802.1, andOTN.OTN (Optical Transport Network). Of course,multi-layermultilayer DetNet systems may also be possible, where one DetNet appears as asub-network,sub-network and provides serviceto,to ahigher layerhigher-layer DetNet system. A simple DetNet concept network is shown in <xreftarget="fig_detnet"/>.target="fig_detnet" format="default" sectionFormat="of" derivedContent="Figure 3"/>. Note that in this and followingfiguresfigures, "Forwarding" and "Fwd" refer to the DetNet forwarding sub-layer, and "Service" and "Svc" refer to the DetNet servicesub-layer, whichsub-layer; both of these sub-layers are described in detail in <xreftarget="Stacks"/>.target="StackModel" format="default" sectionFormat="of" derivedContent="Section 4.1.1"/>. </t> <figure anchor="fig_detnet"align="center" title="Aalign="left" suppress-title="false" pn="figure-3"> <name slugifiedName="name-a-simple-detnet-enabled-net">A SimpleDetNet Enabled Network">DetNet-Enabled Network</name> <artworkalign="center"><![CDATA[align="center" name="" type="" alt="" pn="section-4.1.2-2.1"> TSN Edge Transit Relay DetNet End System Node Node Node End System +----------+ +.........+ +----------+ | Appl.|<--:Svc|<--:Svc Proxy:--End to EndEnd-to-End Service-------->|-------->| Appl. | +----------+ +---------+ +---------+ +----------+ | TSN | |TSN||Svc|<-|Svc|<- DetNet flow --: Service:-->|:-->| Service | +----------+ +---+ +---+ +--------+ +---------+ +----------+ |Forwarding| |Fwd| |Fwd| | Fwd | |Fwd| |Fwd| |Forwarding| +-------.--+ +-.-+ +-.-+ +--.----.+ +-.-+ +-.-+ +---.------+ : Link : / ,-----. \ : Link : / ,-----. \ +........+ +-[SubSub- ]-+ +.......+ +-[SubSub- ]-+[Network] [Network][network] [network] `-----' `-----']]></artwork></artwork> </figure><t><t pn="section-4.1.2-3"> DetNetdata planeData Plane is divided into two sub-layers: the DetNet service sub-layer and the DetNet forwarding sub-layer. This helps to explore and evaluate various combinations of thedata planedata-plane solutions available. Some of them are illustrated in <xreftarget="fig_adaptation"/>.target="fig_adaptation" format="default" sectionFormat="of" derivedContent="Figure 4"/>. This separation of DetNet sub-layers, while helpful, should not be consideredasa formal requirement. For example, some technologies may violate these strict sub-layers and still be able to deliver a DetNet service. </t> <figure anchor="fig_adaptation"align="center" title="DetNet adaptationalign="left" suppress-title="false" pn="figure-4"> <name slugifiedName="name-detnet-adaptation-to-data-p">DetNet Adaptation todata plane">Data Plane</name> <artworkalign="center"><![CDATA[align="center" name="" type="" alt="" pn="section-4.1.2-4.1"> . . +-----------------------------+ | DetNet Service sub-layer | PW, UDP, GRE +-----------------------------+ | DetNet Forwarding sub-layer | IPv6, IPv4, MPLS TE LSPs, MPLS SR +-----------------------------+ . .]]></artwork></artwork> </figure><t><t pn="section-4.1.2-5"> In some networking scenarios, the end system initially provides a DetNet flow encapsulation, which contains all information needed by DetNet nodes (e.g., DetNet flow based on the Real-time Transport Protocol (RTP) <xreftarget="RFC3550"/> based DetNet flowtarget="RFC3550" format="default" sectionFormat="of" derivedContent="RFC3550"/> that is carried over a native UDP/IP network orPseudoWire).pseudowire (PW)). In other scenarios, the encapsulation formats might differ significantly. </t><t><t pn="section-4.1.2-6"> There are many valid options to create adata planedata-plane solution for DetNet traffic by selecting a technology approach for the DetNet service sub-layer and also selecting a technology approach for the DetNet forwarding sub-layer. There are a large number of valid combinations. </t><t><t pn="section-4.1.2-7"> One of the most fundamental differences between different potentialdata planedata-plane options is the basic headers used by DetNet nodes. For example, the basic service can be delivered based on an MPLS label or an IP header. This decision impacts the basic forwarding logic for the DetNet service sub-layer. Note that in both cases, IP addresses are used to address DetNet nodes. The selected DetNet forwarding sub-layer technology also needs to be mapped to thesub-netsubnet technology used to interconnect DetNet nodes. For example, DetNet flows will need to be mapped to TSN Streams. </t> </section> <section anchor="netref"title="Network reference model"> <t> <xref target="fig_DetNetservice"/>numbered="true" toc="include" removeInRFC="false" pn="section-4.1.3"> <name slugifiedName="name-network-reference-model">Network Reference Model</name> <t pn="section-4.1.3-1"><xref target="fig_DetNetservice" format="default" sectionFormat="of" derivedContent="Figure 5"/> shows another view of the DetNetservice relatedservice-related reference points and main components. </t> <figure anchor="fig_DetNetservice"align="center" title="DetNetalign="left" suppress-title="false" pn="figure-5"> <name slugifiedName="name-detnet-service-reference-mo">DetNet Service Reference Model(multi-domain)"> <artwork><![CDATA[(Multidomain)</name> <artwork name="" type="" align="left" alt="" pn="section-4.1.3-2.1"> DetNet DetNetend system end systemEnd System End System _ _ / \ +----DetNet-UNI (U) / \ /App\ | /App\ /-----\ | /-----\ | NIC | v ________ | NIC | +--+--+ _____ / \ DetNet-UNI (U) --+ +--+--+ | / \__/ \ | | | / +----+ +----+ \_____ | | | / | | | | \_______ | | +------U PE +----+ P +----+ \ _ v | | | | | | | | ___/ \ | | +--+-+ +----+ | +----+ | / \_ | \ | | | | | / \ | \ | +----+ +--+-+ +--+PE |------ U-----+ \ | | | | | | | | | \_ _/ \ +---+ P +----+ P +--+ +----+ | \____/ \___ | | | | / \ +----+__ +----+ DetNet-1 DetNet-2 | \_____/ \___________/ | | | | | End-to-EndserviceService | | | |<-------------------------------------------------------------><-------------------------------------------------------------> | | DetNetserviceService | | | | |<------------------------------------------------><------------------------------------------------> | | | | | | |]]></artwork></artwork> </figure><t>DetNet User Network<t pn="section-4.1.3-3">DetNet User-to-Network Interfaces (DetNet-UNIs) ("U" in <xreftarget="fig_DetNetservice"/>)target="fig_DetNetservice" format="default" sectionFormat="of" derivedContent="Figure 5"/>) are assumed in this document to be packet-based reference points and provide connectivity over the packet network. A DetNet-UNI may provide multiplefunctions, e.g.,functions. For example, itmay addmay: </t> <ul spacing="normal" bare="false" empty="false" pn="section-4.1.3-4"> <li pn="section-4.1.3-4.1">add encapsulation specific to networking technologyspecific encapsulationto the DetNet flows ifnecessary; it may providenecessary, </li> <li pn="section-4.1.3-4.2">provide status of the availability of the resources associated with areservation; it may providereservation, </li> <li pn="section-4.1.3-4.3">provide a synchronization service for the endsystem; it may carrysystem, or </li> <li pn="section-4.1.3-4.4">carry enough signaling to place the reservation in a network without acontroller,controller orifin a network where the controller only deals with the network but not the end systems. </li> </ul> <t pn="section-4.1.3-5"> Internal reference points of end systems (between the application and theNIC)Network Interface Card (NIC)) are more challenging from the controlperspectiveperspective, and they may have extra requirements (e.g., in-order delivery is expected in end system internal reference points, whereas it is considered optional over the DetNet-UNI).</t> </section> </section> <section anchor="netrefsys"title="DetNet systems">numbered="true" toc="include" removeInRFC="false" pn="section-4.2"> <name slugifiedName="name-detnet-systems">DetNet Systems</name> <section anchor="es"title="End system"> <t>numbered="true" toc="include" removeInRFC="false" pn="section-4.2.1"> <name slugifiedName="name-end-system">End System</name> <t pn="section-4.2.1-1"> The traffic characteristics of an App-flow can be CBR (constant bit rate) or VBR (variable bit rate) and can haveLayer-1 or Layer-2Layer 1, Layer 2, orLayer-3Layer 3 encapsulation (e.g., TDM (time-divisionmultiplexing),multiplexing) Ethernet, IP). These characteristics are considered as input for resource reservation and might be simplified to ensure determinism during packet forwarding (e.g., making reservations for the peak rate of VBR traffic, etc.). </t><t><t pn="section-4.2.1-2"> An end system may or may not be aware of the DetNet forwarding sub-layerawareor DetNet servicesub-layer aware.sub-layer. That is, an end system may or may not containDetNet specificDetNet-specific functionality. End systems with DetNet functionalities may have the same or different forwarding sub-layer as the connected DetNet domain. Categorization of end systems are shown in <xreftarget="fig_endsys2"/>.target="fig_endsys2" format="default" sectionFormat="of" derivedContent="Figure 6"/>. </t> <figure anchor="fig_endsys2"align="center" title="Categorizationalign="left" suppress-title="false" pn="figure-6"> <name slugifiedName="name-categorization-of-end-syste">Categorization ofend systems"> <artwork><![CDATA[End Systems</name> <artwork name="" type="" align="left" alt="" pn="section-4.2.1-3.1"> End system | | | DetNet aware ? / \+------< >------++------< >------+ NO | \ / | YES | v |DetNet unawareDetNet-unaware | End system | | Service/Forwarding | sub-layer / \ aware ?+--------< >-------------++--------< >-------------+ f-aware | \ / | s-aware | v | | | both | | | | DetNet f-aware | DetNet s-aware End system | End system v DetNet sf-aware End system]]></artwork></artwork> </figure><t> Note<t pn="section-4.2.1-4"> The following are some known use case examples for end systems:<list style="symbols"> <t> DetNet unaware:</t> <dl spacing="normal" newline="true" indent="3" pn="section-4.2.1-5"> <dt pn="section-4.2.1-5.1">DetNet unaware</dt> <dd pn="section-4.2.1-5.2"> The classic case requiring serviceproxies.</t> <t> DetNet f-aware: Aproxies.</dd> <dt pn="section-4.2.1-5.3">DetNet f-aware</dt> <dd pn="section-4.2.1-5.4">A system that is aware of the DetNet forwardingsub-layer aware system.sub-layer. It knows about some TSN functions (e.g.,reservation),reservation) but not about service protection.</t> <t> DetNet s-aware:</dd> <dt pn="section-4.2.1-5.5">DetNet s-aware</dt> <dd pn="section-4.2.1-5.6"> A system that is aware of the DetNet servicesub-layer aware system.sub-layer. It supplies sequencenumbers,numbers but doesn't know about resource allocation.</t> <t> DetNet sf-aware: A full</dd> <dt pn="section-4.2.1-5.7">DetNet sf-aware</dt> <dd pn="section-4.2.1-5.8">A fully functioning DetNet endsystem, itsystem. It has DetNet functionalities and usually the same forwarding paradigm as the connected DetNet domain. It can be treated as an integral part of the DetNet domain.</t> </list> </t></dd> </dl> </section> <section anchor="ertn"title="DetNet edge, relay,numbered="true" toc="include" removeInRFC="false" pn="section-4.2.2"> <name slugifiedName="name-detnet-edge-relay-and-trans">DetNet Edge, Relay, andtransit nodes"> <t>Transit Nodes</name> <t pn="section-4.2.2-1"> As shown in <xreftarget="fig_detnet"/>,target="fig_detnet" format="default" sectionFormat="of" derivedContent="Figure 3"/>, DetNet edge nodes providing proxy service and DetNet relay nodes providing the DetNet service sub-layer areDetNet-aware,DetNet aware, and DetNet transit nodes need only be aware of the DetNet forwarding sub-layer.</t><t></t> <t pn="section-4.2.2-2"> In general, if a DetNet flow passes through one or more DetNet-unaware network nodes between two DetNet nodes providing the DetNet forwarding sub-layer for that flow, there is a potential for disruption or failure of the DetNet QoS. A network administrator needs to 1) ensure that the DetNet-unaware network nodes are configured to minimize the chances of packet loss anddelay,delay and 2) provision enough extra buffer space in the DetNet transit node following the DetNet-unaware network nodes to absorb the induced latency variations. </t> </section> </section> <section anchor="DetNetFlows"title="DetNet flows">numbered="true" toc="include" removeInRFC="false" pn="section-4.3"> <name slugifiedName="name-detnet-flows">DetNet Flows</name> <section anchor="DetNetFlowsTypes"title="DetNet flow types"> <t>numbered="true" toc="include" removeInRFC="false" pn="section-4.3.1"> <name slugifiedName="name-detnet-flow-types">DetNet Flow Types</name> <t pn="section-4.3.1-1"> A DetNet flow can have different formats while its packets are forwarded between the peer end systems depending on the type of the end systems. Corresponding to the end system types, the following possibletypes / formatstypes/formats of a DetNet flow are distinguished in this document. The different flow types have different requirements to DetNet nodes.<list style="symbols"> <t> App-flow: the</t> <dl spacing="normal" newline="true" indent="3" pn="section-4.3.1-2"> <dt pn="section-4.3.1-2.1">App-flow</dt> <dd pn="section-4.3.1-2.2">The payload (data) carried over a DetNet flow betweenDetNet unawareDetNet-unaware end systems. Anapp-flowApp-flow does not contain anyDetNet relatedDetNet-related attributes and does not imply any specific requirement on DetNetnodes.</t> <t> DetNet-f-flow:nodes.</dd> <dt pn="section-4.3.1-2.3">DetNet-f-flow</dt> <dd pn="section-4.3.1-2.4">The specific format of a DetNet flow. It only requires the resource allocation features provided by the DetNet forwarding sub-layer.</t> <t> DetNet-s-flow:</dd> <dt pn="section-4.3.1-2.5">DetNet-s-flow</dt> <dd pn="section-4.3.1-2.6">The specific format of a DetNet flow. It only requires the service protection feature ensured by the DetNet service sub-layer.</t> <t> DetNet-sf-flow:</dd> <dt pn="section-4.3.1-2.7"> DetNet-sf-flow</dt> <dd pn="section-4.3.1-2.8">The specific format of a DetNet flow. It requires both the DetNet service sub-layer and the DetNet forwarding sub-layer functions during forwarding.</t> </list> </t></dd> </dl> </section> <section anchor="FlowLimits"title="Source transmission behavior"> <t>numbered="true" toc="include" removeInRFC="false" pn="section-4.3.2"> <name slugifiedName="name-source-transmission-behavio">Source Transmission Behavior</name> <t pn="section-4.3.2-1"> For the purposes of resource allocation, DetNet flows can be synchronous or asynchronous. In synchronous DetNet flows, at least the DetNet nodes (and possibly the end systems) are closely time synchronized, typically to better than 1 microsecond. By transmitting packets from different DetNet flows or classes of DetNet flows at different times, using repeating schedules synchronized among the DetNet nodes, resources such as buffers and link bandwidth can be shared over the time domain among different DetNet flows. There is atradeofftrade-off among techniques for synchronous DetNet flows between the burden of fine-grained scheduling and the benefit of reducing the required resources, especially buffer space.</t><t></t> <t pn="section-4.3.2-2"> In contrast, asynchronous DetNet flows are not coordinated with a fine-grained schedule, so relay and end systems must assume worst-case interference among DetNet flows contending for buffer resources. Asynchronous DetNet flows are characterized by:<list style="symbols"> <t></t> <ul spacing="normal" bare="false" empty="false" pn="section-4.3.2-3"> <li pn="section-4.3.2-3.1"> A maximum packet size;</t><t></li> <li pn="section-4.3.2-3.2"> An observation interval; and</t><t></li> <li pn="section-4.3.2-3.3"> A maximum number of transmissions during that observation interval.</t> </list> </t><t></li> </ul> <t pn="section-4.3.2-4"> These parameters, together with knowledge of the protocol stack used (and thus the size of the various headers added to a packet), provide the bandwidth that is needed for the DetNet flow.</t><t></t> <t pn="section-4.3.2-5"> The source is required not to exceed these limits in order to obtain DetNet service. If the source transmits less data than this limit allows, then the unusedresourceresource, such as linkbandwidthbandwidth, can be made available by the DetNet system to non-DetNet packets as long as all guarantees are fulfilled. However, making those resources available to DetNet packets in other DetNet flows would serve no purpose. Those other DetNet flows have their own dedicated resources, on the assumption that all DetNet flows can use all of their resources over a long period of time.</t><t></t> <t pn="section-4.3.2-6"> There is no expectation in DetNet for App-flows to be responsive to congestion control <xreftarget="RFC2914"/>target="RFC2914" format="default" sectionFormat="of" derivedContent="RFC2914"/> or explicit congestion notification <xreftarget="RFC3168"/>.target="RFC3168" format="default" sectionFormat="of" derivedContent="RFC3168"/>. The assumption is that a DetNet flow, to be useful, must be delivered in its entirety. That is, while any useful application is written to expect a certain number of lost packets, the real-time applications of interest to DetNet demand that the loss of data due to the network is a rare event.</t><t></t> <t pn="section-4.3.2-7"> Although DetNet strives to minimize the changes required of an application to allow it to shift from a special-purpose digital network to an Internet Protocol network, one fundamental shift in the behavior of network applications is impossible to avoid: the reservation of resources before the application starts. In the first place, a network cannot deliver finite latency and practically zero packet loss to an arbitrarily high offered load. Secondly, achieving practically zero packet loss for DetNet flows means that DetNet nodes have to dedicate buffer resources to specific DetNet flows or to classes of DetNet flows. The requirements of each reservation have to be translated into the parameters that control each DetNet system's queuing, shaping, and schedulingfunctionsfunctions, and they have to be delivered to the DetNet nodes and end systems.</t><t></t> <t pn="section-4.3.2-8"> All nodes in a DetNet domain are expected to support the data behavior required to deliver a particular DetNet service. If a node itself is not DetNet service aware, the DetNet nodes that are adjacent tosuch non-DetNet aware nodesthem must ensure that the node that is non-DetNet awarenodeis provisioned to appropriately support the DetNet service. For example,an IEEE 802.1a TSN node (as defined by IEEE 802.1) may be used to interconnectDetNet awareDetNet-aware nodes, and these DetNet nodes can map DetNet flows to 802.1 TSN flows.AnotherAs another example, an MPLS-TE orTPMPLS-TP (Transport Profile) domain may be used to interconnectDetNet awareDetNet-aware nodes, and these DetNet nodes can map DetNet flows to TELSPsLSPs, which can provide the QoS requirements of the DetNet service. </t> </section> <section anchor="Incomplete"title="Incomplete Networks"> <t>numbered="true" toc="include" removeInRFC="false" pn="section-4.3.3"> <name slugifiedName="name-incomplete-networks">Incomplete Networks</name> <t pn="section-4.3.3-1"> The presence in the network of intermediate nodes or subnets that are not fully capable of offering DetNet services complicates the ability of the intermediate nodes and/or controller to allocate resources, as extra buffering must be allocated at points downstream from the non-DetNet intermediate node for a DetNet flow. This extra buffering may increase latency and/or jitter. </t> </section> </section> <section anchor="te"title="Trafficnumbered="true" toc="include" removeInRFC="false" pn="section-4.4"> <name slugifiedName="name-traffic-engineering-for-det">Traffic Engineering forDetNet"> <t> <xref target="TEAS">TrafficDetNet</name> <t pn="section-4.4-1"><xref target="TEAS" format="default" sectionFormat="of" derivedContent="TEAS">Traffic Engineering Architecture and Signaling (TEAS) </xref> defines traffic-engineering architectures for generic applicability across packet andnon-packetnonpacket networks. From a TEAS perspective, Traffic Engineering (TE) refers to techniques that enable operators to control how specific traffic flows are treated within their networks. </t><t><t pn="section-4.4-2"> Becauseifof its very nature of establishing explicit optimized paths,Deterministic NetworkingDetNet can be seen as a new, specialized branch ofTraffic Engineering,TE, and it inherits its architecture with a separation into planes.</t><t></t> <t pn="section-4.4-3"> TheDeterministic NetworkingDetNet architecture is thus composed of threeplanes,planes: a (User) Application Plane, a Controller Plane, and a NetworkPlane, whichPlane. This echoesthatthe composition of Figure 1 of <xreftarget="RFC7426"> Software-Definedtarget="RFC7426" format="default" sectionFormat="of" derivedContent="RFC7426">"Software-Defined Networking (SDN): Layers and ArchitectureTerminology</xref>,Terminology"</xref> and theControllerscontrollers identified in <xreftarget="RFC8453"/>target="RFC8453" format="default" sectionFormat="of" derivedContent="RFC8453"/> and <xreftarget="RFC7149"/>.target="RFC7149" format="default" sectionFormat="of" derivedContent="RFC7149"/>. </t> <section anchor="appplane"title="Thenumbered="true" toc="include" removeInRFC="false" pn="section-4.4.1"> <name slugifiedName="name-the-application-plane">The ApplicationPlane"> <t>Plane</name> <t pn="section-4.4.1-1"> Per <xreftarget="RFC7426"/>,target="RFC7426" format="default" sectionFormat="of" derivedContent="RFC7426"/>, the Application Plane includes both applications and services. In particular, the Application Plane incorporates the User Agent, a specialized application that interacts with the end user/and operator and performs requests forDeterministic NetworkingDetNet services via an abstract Flow ManagementEntity, (FME)Entity (FME), which may or may not be collocated with (one of) the end systems. </t><t>At<t pn="section-4.4.1-2">At the Application Plane, a management interface enables the negotiation of flows between end systems. An abstraction of the flow called a Traffic Specification (TSpec) provides the representation. This abstraction is used to place a reservation over the (Northbound) Service Interface and within the Applicationplane.Plane. It is associated with an abstraction of location, such as IP addresses and DNS names, to identify the end systems and possibly specify DetNet nodes. </t> </section> <section anchor="ctrlplane"title="Thenumbered="true" toc="include" removeInRFC="false" pn="section-4.4.2"> <name slugifiedName="name-the-controller-plane">The ControllerPlane"> <t>Plane</name> <t pn="section-4.4.2-1"> The Controller Plane corresponds to the aggregation of the Control and Management Planes in <xreftarget="RFC7426"/>,target="RFC7426" format="default" sectionFormat="of" derivedContent="RFC7426"/>, though Common Control and Measurement Plane (CCAMP) (as defined by the CCAMP Working Group <xreftarget="CCAMP"/>target="CCAMP" format="default" sectionFormat="of" derivedContent="CCAMP"/>) makes an additional distinction between management and measurement. When the logical separation of the Control,MeasurementMeasurement, and other Management entities is not relevant, the termController Plane"Controller Plane" is used for simplicity to represent them all, and the termController"Controller Plane Function(CPF)(CPF)" refers to any device operating in that plane, whetherisit is a Path Computation Element (PCE) <xreftarget="RFC4655"/>, ortarget="RFC4655" format="default" sectionFormat="of" derivedContent="RFC4655"/>, a Network ManagemententityEntity (NME), or a distributed controlplane.protocol. The CPF is a core element of a controller, in charge of computingDeterministicdeterministic paths to be applied in the Network Plane. </t><t><t pn="section-4.4.2-2"> A (Northbound) Service Interface enables applications in the Application Plane to communicate with the entities in the Controller Plane as illustrated in <xreftarget="NorthSouth"/>.target="NorthSouth" format="default" sectionFormat="of" derivedContent="Figure 7"/>. </t><t><t pn="section-4.4.2-3"> One or moreCPF(s)CPFs collaborate to implement the requests from the FME asPer-Flow Per-Hop Behaviorsper-flow, per-hop behaviors installed in the DetNet nodes for each individual flow. The CPFs place each flow along a deterministicsequencearrangement of DetNet nodes so as to respect per-flow constraints such as security and latency, and to optimize the overall result for metrics such as an abstract aggregated cost. The deterministicsequencearrangement can typically be more complex than a directsequencearrangement and include redundantpaths,paths with one or more packet replication and elimination points. Scaling to larger networks is discussed in <xreftarget="Scaling"/>.target="Scaling" format="default" sectionFormat="of" derivedContent="Section 4.9"/>. </t> </section> <section anchor="netplane"title="Thenumbered="true" toc="include" removeInRFC="false" pn="section-4.4.3"> <name slugifiedName="name-the-network-plane">The NetworkPlane"><t>Plane</name> <t pn="section-4.4.3-1"> The Network Plane represents the network devices and protocols as a whole, regardless of theLayerlayer at which the network devices operate. It includesForwardingthe Data Plane(data plane), Application,and Operational Plane (e.g., OAM) aspects. </t><t> The network<t pn="section-4.4.3-2">The Network Plane comprises the Network Interface Cards(NIC)(NICs) in the end systems, which are typically IP hosts, and DetNet nodes, which are typically IP routers and MPLS switches.Network-to-Network Interfaces such as used for Traffic Engineering path reservation in <xref target="RFC5921"/>, as well as User-to-Network Interfaces (UNI) such as provided by the Local Management Interface (LMI) between network and end systems, are both part of the Network Plane, both in the control plane and the data plane.</t><t><t pn="section-4.4.3-3"> A Southbound (Network) Interface enables the entities in the Controller Plane to communicate with devices in the Network Plane as illustrated in <xreftarget="NorthSouth"/>.target="NorthSouth" format="default" sectionFormat="of" derivedContent="Figure 7"/>. This interface leverages and extends TEAS to describe the physical topology and resources in the Network Plane. </t> <figurealign="center"anchor="NorthSouth"title="Northboundalign="left" suppress-title="false" pn="figure-7"> <name slugifiedName="name-northbound-and-southbound-i">Northbound and Southboundinterfaces">Interfaces</name> <artworkalign="left"><![CDATA[align="left" name="" type="" alt="" pn="section-4.4.3-4.1"> End End System System -+-+-+-+-+-+-+ Northbound -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- CPF CPF CPF CPF -+-+-+-+-+-+-+ Southbound -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- DetNet DetNet DetNet DetNet Node Node Node Node NIC NIC DetNet DetNet DetNet DetNet Node Node Node Node]]></artwork></artwork> </figure><t><t pn="section-4.4.3-5"> The DetNet nodes (and possibly the endsystems NIC)systems' NICs) expose their capabilities and physical resources to the controller (theCPF),CPF) and update the CPFs with their dynamic perception of thetopology,topology across the Southbound Interface. In return, the CPFs set the per-flow paths up, providing a Flow Characterization that is more tightly coupled to the DetNet nodeOperationoperation than a TSpec.</t><t></t> <t pn="section-4.4.3-6"> At the Networkplane,Plane, DetNet nodes may exchange information regarding the state of the paths, between adjacent DetNet nodes and possibly with the end systems, and forward packets within constraints associated to each flow, or, when unable to do so, perform alast resortlast-resort operation such as drop or declassify.</t><t></t> <t pn="section-4.4.3-7"> This document focuses on the Southbound interface and the operation of the Network Plane. </t> </section> </section> <section anchor="QueuingModels"title="Queuing,numbered="true" toc="include" removeInRFC="false" pn="section-4.5"> <name slugifiedName="name-queuing-shaping-scheduling-">Queuing, Shaping, Scheduling, andPreemption"> <t>Preemption</name> <t pn="section-4.5-1"> DetNet achieves bounded delivery latency by reserving bandwidth and buffer resources at each DetNet node along the path of the DetNet flow. The reservation itself is not sufficient, however. Implementors and users of a number of proprietary and standard real-time networks have found that standards for specificdata planedata-plane techniques are required to enable these assurances to be made in amulti-vendormultivendor network. The fundamental reason is that latency variation in one DetNet system results in the need for extra buffer space in the next-hop DetNet system(s), which inturn,turn increases the worst-case per-hop latency.</t><t></t> <t pn="section-4.5-2"> Standard queuing andtransmission selectiontransmission-selection algorithms allowtraffic engineering <xref target="te"/>TE (<xref target="te" format="default" sectionFormat="of" derivedContent="Section 4.4"/>) to compute the latency contribution of each DetNet node to the end-to-end latency, to compute the amount of buffer space required in each DetNet node for each incremental DetNet flow, and most importantly, to translate from a flow specification to a set of values for the managed objects that control each relay or end system. For example, the IEEE 802.1 WG has specified (and is specifying) a set of queuing, shaping, and scheduling algorithms that enable each DetNet node, and/or a central controller, to compute these values. These algorithms include:<list style="symbols"> <t></t> <ul spacing="normal" bare="false" empty="false" pn="section-4.5-3"> <li pn="section-4.5-3.1"> A credit-based shaper <xreftarget="IEEE802.1Qav"/> (superseded bytarget="IEEE802.1Qav" format="default" sectionFormat="of" derivedContent="IEEE802.1Qav"/> (incorporated to <xreftarget="IEEE802.1Q-2018"/>). </t><t>target="IEEE802.1Q" format="default" sectionFormat="of" derivedContent="IEEE802.1Q"/>). </li> <li pn="section-4.5-3.2"> Time-gated queues governed by a rotating time schedule based on synchronized time <xreftarget="IEEE802.1Qbv"/> (superseded bytarget="IEEE802.1Qbv" format="default" sectionFormat="of" derivedContent="IEEE802.1Qbv"/> (incorporated to <xreftarget="IEEE802.1Q-2018"/>). </t><t>target="IEEE802.1Q" format="default" sectionFormat="of" derivedContent="IEEE802.1Q"/>). </li> <li pn="section-4.5-3.3"> Synchronized double (or triple) buffers driven by synchronized time ticks. <xreftarget="IEEE802.1Qch"/> (superseded bytarget="IEEE802.1Qch" format="default" sectionFormat="of" derivedContent="IEEE802.1Qch"/> (incorporated to <xreftarget="IEEE802.1Q-2018"/>). </t><t> Pre-emptiontarget="IEEE802.1Q" format="default" sectionFormat="of" derivedContent="IEEE802.1Q"/>). </li> <li pn="section-4.5-3.4"> Preemption of an Ethernet packet in transmission by a packet with a more stringent latency requirement, followed by the resumption of the preempted packet <xreftarget="IEEE802.1Qbu"/> (superseded by <xref target="IEEE802.1Q-2018"/>), <xref target="IEEE802.3br"/> (superseded by <xref target="IEEE802.3-2018"/>). </t> </list> </t><t>target="IEEE802.1Qbu" format="default" sectionFormat="of" derivedContent="IEEE802.1Qbu"/> (incorporated to <xref target="IEEE802.1Q" format="default" sectionFormat="of" derivedContent="IEEE802.1Q"/>) <xref target="IEEE802.3br" format="default" sectionFormat="of" derivedContent="IEEE802.3br"/> (incorporated to <xref target="IEEE802.3" format="default" sectionFormat="of" derivedContent="IEEE802.3"/>). </li> </ul> <t pn="section-4.5-4"> While these techniques are currently embedded in Ethernet <xreftarget="IEEE802.3-2018"/>target="IEEE802.3" format="default" sectionFormat="of" derivedContent="IEEE802.3"/> and bridging standards, we can note that they are all, except perhaps for packet preemption, equally applicable toothermedia other thanEthernet,Ethernet and to routers as well as bridges. Other media may haveitstheir ownmethods, see,methods (see, e.g., <xreftarget="I-D.ietf-6tisch-architecture"/>,target="I-D.ietf-6tisch-architecture" format="default" sectionFormat="of" derivedContent="TSCH-ARCH"/> and <xreftarget="RFC7554"/>.target="RFC7554" format="default" sectionFormat="of" derivedContent="RFC7554"/>). Further techniques are defined by theIETF, e.g.,IETF (e.g., <xreftarget="RFC8289"/>target="RFC8289" format="default" sectionFormat="of" derivedContent="RFC8289"/> and <xreftarget="RFC8033"/>.target="RFC8033" format="default" sectionFormat="of" derivedContent="RFC8033"/>). DetNet may include such definitions in thefuture,future or may define how these techniques can be used by DetNet nodes. </t> </section> <section anchor="ServInst"title="Service instance"> <t>numbered="true" toc="include" removeInRFC="false" pn="section-4.6"> <name slugifiedName="name-service-instance">Service Instance</name> <t pn="section-4.6-1"> AServiceservice instance represents all the functions required on a DetNet node to allow the end-to-end service between the UNIs. </t><t><t pn="section-4.6-2"> The DetNet network general reference model is shown in <xreftarget="fig_DetNetrefmodel"/>target="fig_DetNetrefmodel" format="default" sectionFormat="of" derivedContent="Figure 8"/> for a DetNet service scenario (i.e., between two DetNet-UNIs). In this figure, end systems ("A" and "B") are connected directly to the edge nodes of an IP/MPLS network ("PE1" and "PE2"). End systems participating in DetNet communication may require connectivity before setting up an App-flow that requires the DetNet service. Such aconnectivity relatedconnectivity-related service instance and the one dedicated for DetNet service share the same access. Packets belonging to a DetNet flow are selected by a filter configured on the access ("F1" and "F2"). As a result,data flow specificdata-flow-specific access ("access-A + F1" and "access-B + F2")areis terminated in theflow specificflow-specific service instance ("SI-1" and "SI-2"). A tunnel is used to provide connectivity between the service instances. </t><t><t pn="section-4.6-3"> The tunnel is exclusively used for the packets of the DetNet flow between "SI-1" and "SI-2". The service instances are configured to implement DetNet functions and aflow specificflow-specific DetNet forwarding. The service instance and the tunnel may or may not be shared by multiple DetNet flows. Sharing the service instance by multiple DetNet flows requires properly populated forwarding tables of the service instance. </t> <figure anchor="fig_DetNetrefmodel"align="center" title="DetNet network general reference model"> <artwork><![CDATA[align="left" suppress-title="false" pn="figure-8"> <name slugifiedName="name-detnet-network-general-refe">DetNet Network General Reference Model</name> <artwork name="" type="" align="left" alt="" pn="section-4.6-4.1"> access-A access-B<-----> <--------<-----> <-------- tunnel----------> <----->----------> <-----> +---------+ ___ _ +---------+ End system | +----+ | / \/ \_ | +----+ | End system "A" -------F1+ | | / \ | | +F2----- "B" | | +========+ IP/MPLS +=======+ | | | |SI-1| | \__ Net._/ | |SI-2| | | +----+ | \____/ | +----+ | |PE1 | | PE2| +---------++---------+ ]]></artwork>+---------+</artwork> </figure><t><t pn="section-4.6-5"> The tunnel between the service instances may have some special characteristics. For example, in case of a DetNet L3 service, there are differences in the usage of the PW for DetNet traffic compared to the network model described in <xreftarget="RFC6658"/>.target="RFC6658" format="default" sectionFormat="of" derivedContent="RFC6658"/>. In the DetNet scenario, the PW is likely to be used exclusively by the DetNet flow, whereas <xreftarget="RFC6658"/>target="RFC6658" format="default" sectionFormat="of" derivedContent="RFC6658"/> states:"The</t> <blockquote pn="section-4.6-6"> The packet PW appears as a single point-to-point link to the client layer. Network-layer adjacency formation and maintenance between the clientequipmentequipments will follow the normal practice needed to support the required relationship in the clientlayer ...layer. </blockquote> <t pn="section-4.6-7">and</t> <blockquote pn="section-4.6-8"> This packetPseudoWirepseudowire is used to transport all of the requiredLayer-2layer 2 andLayer-3layer 3 protocols between LSR1 andLSR2".LSR2. </blockquote> <t pn="section-4.6-9"> Further details are network technology specific and can be found in <xreftarget="I-D.ietf-detnet-dp-sol-mpls"/> and <xref target="I-D.ietf-detnet-dp-sol-ip"/>.target="I-D.ietf-detnet-data-plane-framework" format="default" sectionFormat="of" derivedContent="DETNET-FRAMEWORK"/>. </t> </section> <section anchor="FlowIDatTechBord"title="Flow identificationnumbered="true" toc="include" removeInRFC="false" pn="section-4.7"> <name slugifiedName="name-flow-identification-at-tech">Flow Identification attechnology borders"> <t>ThisTechnology Borders</name> <t pn="section-4.7-1">This section discusses what needs to be done at technology borders including Ethernet as one of the technologies. Flow identification for MPLS and IPdata planesData Planes are described in <xreftarget="I-D.ietf-detnet-dp-sol-mpls"/>target="I-D.ietf-detnet-mpls" format="default" sectionFormat="of" derivedContent="DETNET-MPLS"/> and <xreftarget="I-D.ietf-detnet-dp-sol-ip"/>,target="I-D.ietf-detnet-ip" format="default" sectionFormat="of" derivedContent="DETNET-IP"/>, respectively. </t> <section anchor="Relayering"title="Exporting flow identification"> <t>numbered="true" toc="include" removeInRFC="false" pn="section-4.7.1"> <name slugifiedName="name-exporting-flow-identificati">Exporting Flow Identification</name> <t pn="section-4.7.1-1"> A DetNet node may need to map specific flows tolower layerlower-layer flows (or Streams) in order to provide specific queuing and shaping services for specific flows. For example:<list style="symbols"> <t></t> <ul spacing="normal" bare="false" empty="false" pn="section-4.7.1-2"> <li pn="section-4.7.1-2.1"> A non-IP, strictly L2 source end system X may be sending multiple flows to the same L2 destination end system Y. Those flows may include DetNet flows with different QoSrequirements,requirements and may include non-DetNet flows.</t><t></li> <li pn="section-4.7.1-2.2"> A router may be sending any number of flows to another router. Again, those flows may include DetNet flows with different QoSrequirements,requirements and may include non-DetNet flows.</t><t></li> <li pn="section-4.7.1-2.3"> Two routers may be separated by bridges. For these bridges to perform any required per-flow queuing and shaping, they must be able to identify the individual flows.</t><t></li> <li pn="section-4.7.1-2.4"> A Label Edge Router (LER) may have a Label Switched Path (LSP) set up for handling traffic destined for a particular IP address carrying only non-DetNet flows. If a DetNet flow to that same address is requested, a separate LSP may beneeded,needed in orderthatfor all of the Label Switch Routers (LSRs) along the path to the destination to give that flow special queuing and shaping.</t> </list> </t><t></li> </ul> <t pn="section-4.7.1-3"> The need for a lower-layer node to be aware of individual higher-layer flows is not unique to DetNet. But, given the endless complexity of layering and relayering over tunnels that is available to network designers, DetNet needs to provide a model for flow identification that is better than packet inspection. That is not to say that packet inspection toLayer-4Layer 4 orLayer-5Layer 5 addresses will not beused,used or the capability standardized;but,however, there are alternatives.</t><t></t> <t pn="section-4.7.1-4"> A DetNet relay node can connect DetNet flows on different paths using different flow identification methods. For example:<list style="symbols"> <t></t> <ul spacing="normal" bare="false" empty="false" pn="section-4.7.1-5"> <li pn="section-4.7.1-5.1"> A single unicast DetNet flow passing from router A through a bridged network to router B may be assigned a TSN Stream identifier that is unique within that bridged network. The bridges can then identify the flow without accessing higher-layer headers. Of course, the receiving router must recognize and accept that TSN Stream.</t><t></li> <li pn="section-4.7.1-5.2"> A DetNet flow passing from LSR A to LSR B may be assigned a different label than that used for other flows to the same IP destination.</t> </list> </t><t></li> </ul> <t pn="section-4.7.1-6"> In any of the above cases, it is possible that an existing DetNet flow can be an aggregate carrying multiple other DetNetflows. (Notflows (not to be confused with DetNet compound vs. memberflows.)flows). Of course, this requires that the aggregate DetNet flow be provisioned properly to carry the aggregated flows.</t><t></t> <t pn="section-4.7.1-7"> Thus, rather than packet inspection, there is the option to export higher-layer information to the lower layer. The requirement to support one or the other method for flow identification (or both) is a complexity that is part of DetNet control models. </t> </section> <section anchor="FlowAttrMapping"title="Flow attribute mappingnumbered="true" toc="include" removeInRFC="false" pn="section-4.7.2"> <name slugifiedName="name-flow-attribute-mapping-betw">Flow Attribute Mapping betweenlayers"> <t>ForwardingLayers</name> <t pn="section-4.7.2-1">Forwarding of packets of DetNet flows over multiple technology domains may require that lower layers are aware of specific flows of higher layers. Such an "exporting of flow identification" is needed each time when the forwarding paradigm is changed on the forwarding path (e.g., two LSRs are interconnected byaan L2 bridged domain, etc.). The three representative forwarding methods considered fordeterministic networkingDetNet are:<list style="symbols"> <t>IP routing</t> <t>MPLS</t> <ul spacing="normal" bare="false" empty="false" pn="section-4.7.2-2"> <li pn="section-4.7.2-2.1">IP routing</li> <li pn="section-4.7.2-2.2">MPLS labelswitching</t> <t>Ethernet bridging</t> </list>switching</li> <li pn="section-4.7.2-2.3">Ethernet bridging</li> </ul> <t pn="section-4.7.2-3"> A packet with corresponding Flow-IDs is illustrated in <xreftarget="fig_FlowIDStack"/>,target="fig_FlowIDStack" format="default" sectionFormat="of" derivedContent="Figure 9"/>, which also indicates where each Flow-ID can be added or removed. </t> <figure anchor="fig_FlowIDStack"align="center" title="Packetalign="left" suppress-title="false" pn="figure-9"> <name slugifiedName="name-packet-with-multiple-flow-i">Packet withmultiple Flow-IDs"> <artwork><![CDATA[Multiple Flow-IDs</name> <artwork name="" type="" align="left" alt="" pn="section-4.7.2-4.1"> add/remove add/remove Eth Flow-ID IP Flow-ID | | v v +-----------------------------------------------------------+ | | | | | | Eth | MPLS | IP | Application data | | | | | | +-----------------------------------------------------------+ ^ | add/remove MPLS Flow-ID]]></artwork></artwork> </figure><t><t pn="section-4.7.2-5"> The additional(domain specific)(domain-specific) Flow-ID canbe <list style="symbols"> <t>createdbe: </t> <ul spacing="normal" bare="false" empty="false" pn="section-4.7.2-6"> <li pn="section-4.7.2-6.1">created by adomain specificdomain-specific function or</t> <t>derived</li> <li pn="section-4.7.2-6.2">derived from the Flow-ID added to the App-flow.</t> </list></li> </ul> <t pn="section-4.7.2-7"> The Flow-ID must be unique inside a given domain. Note that the Flow-ID added to the App-flow is still present in the packet, but some nodes may lack the function to recognize it; that's why the additional Flow-ID is added. </t> </section> <section anchor="FlowIDMapEx"title="Flow-ID mapping examples"> <t>numbered="true" toc="include" removeInRFC="false" pn="section-4.7.3"> <name slugifiedName="name-flow-id-mapping-examples">Flow-ID Mapping Examples</name> <t pn="section-4.7.3-1"> IP nodes and MPLS nodes are assumed to be configured to push such an additional(domain specific)(domain-specific) Flow-ID when sending traffic to an Ethernet switch (as shown in the examples below). </t><t> <xref target="fig_ippushflowid"/><t pn="section-4.7.3-2"><xref target="fig_ippushflowid" format="default" sectionFormat="of" derivedContent="Figure 10"/> shows a scenario where an IP end system ("IP-A") is connected via two Ethernet switches ("ETH-n") to an IP router ("IP-1"). </t> <figure anchor="fig_ippushflowid"align="center" title="IP nodes interconnectedalign="left" suppress-title="false" pn="figure-10"> <name slugifiedName="name-ip-nodes-interconnected-by-">IP Nodes Interconnected by an Ethernetdomain"> <artwork><![CDATA[Domain</name> <artwork name="" type="" align="left" alt="" pn="section-4.7.3-3.1"> IP domain<-----------------------------------------------<----------------------------------------------- +======+ +======+ |L3-ID | |L3-ID | +======+ /\ +-----+ +======+ / \ Forward as | | /IP-A\ per ETH-ID |IP-1 | Recognize Push------>------> +-+----+ | +---+-+<-----<----- ETH-ID ETH-ID | +----+-----+ | | v v | | +-----+ +-----+ | +------+ | | +---------+ +......+ |ETH-1+----+ETH-2| +======+ .L3-ID . +-----+ +-----+ |L3-ID | +======+ +......+ +======+ |ETH-ID| .L3-ID . |ETH-ID| +======+ +======+ +------+ |ETH-ID| +======+ Ethernet domain<----------------> ]]></artwork><----------------> </artwork> </figure><t><t pn="section-4.7.3-4"> End system "IP-A" uses the originalApp-flow specificApp-flow-specific ID ("L3-ID"), but as it is connected to an Ethernetdomaindomain, it has to push anEthernet-domain specific flow-IDEthernet-domain-specific Flow-ID ("ETH-ID") before sending the packet to"ETH-1" node."ETH-1". Ethernet switch "ETH-1" can recognize the data flow based on the"ETH-ID""ETH-ID", and it does forwarding toward "ETH-2". "ETH-2" switches the packet toward the IP router. "IP-1" must be configured to receive the EthernetFlow-ID specificFlow-ID-specific multicast flow, but (as it is an L3 node) it decodes the data flow ID based on the "L3-ID" fields of the received packet. </t><t> <xref target="fig_mplspushflowid"/><t pn="section-4.7.3-5"><xref target="fig_mplspushflowid" format="default" sectionFormat="of" derivedContent="Figure 11"/> shows a scenario where MPLS domain nodes ("PE-n" and "P-m") are connected via two Ethernet switches ("ETH-n"). </t> <figure anchor="fig_mplspushflowid"align="center" title="MPLS nodes interconnectedalign="left" suppress-title="false" pn="figure-11"> <name slugifiedName="name-mpls-nodes-interconnected-b">MPLS Nodes Interconnected by an Ethernetdomain"> <artwork><![CDATA[Domain</name> <artwork name="" type="" align="left" alt="" pn="section-4.7.3-6.1"> MPLS domain<-----------------------------------------------><-----------------------------------------------> +=======+ +=======+ |MPLS-ID| |MPLS-ID| +=======+ +-----+ +-----+ +=======+ +-----+ | | Forward as | | | | |PE-1 | per ETH-ID | P-2 +-----------+ PE-2| Push----->-----> +-+---+ | +---+-+ +-----+ ETH-ID | +-----+----+ | \ Recognize | v v | +-- ETH-ID | +-----+ +-----+ | +---+ | | +----+ +.......+ |ETH-1+----+ETH-2| +=======+ .MPLS-ID. +-----+ +-----+ |MPLS-ID| +=======+ +=======+ |ETH-ID | +.......+ |ETH-ID | +=======+ .MPLS-ID. +-------+ +=======+ |ETH-ID | +=======+ Ethernet domain<----------------> ]]></artwork><----------------> </artwork> </figure><t><t pn="section-4.7.3-7"> "PE-1" uses theMPLS specificMPLS-specific ID ("MPLS-ID"), but as it is connected to an Ethernetdomaindomain, it has to push anEthernet-domain specific flow-IDEthernet-domain-specific Flow-ID ("ETH-ID") before sending the packet to "ETH-1". Ethernet switch "ETH-1" can recognize the data flow based on the"ETH-ID""ETH-ID", and it does forwarding toward "ETH-2". "ETH-2" switches the packet toward the MPLS node ("P-2"). "P-2" must be configured to receive the EthernetFlow-ID specificFlow-ID-specific multicast flow, but (as it is an MPLS node) it decodes the data flow ID based on the "MPLS-ID" fields of the received packet. </t><t>One<t pn="section-4.7.3-8">One can appreciate from the above example that, when the means used for DetNet flow identification is altered or exported, the means for encoding the sequence number information must similarly be altered or exported. </t> </section> </section> <section anchor="Advertising"title="Advertising resources, capabilitiesnumbered="true" toc="include" removeInRFC="false" pn="section-4.8"> <name slugifiedName="name-advertising-resources-capab">Advertising Resources, Capabilities, andadjacencies"> <t>Adjacencies</name> <t pn="section-4.8-1"> Provisioning of DetNet requires knowledge about:<list style="symbols"><t></t> <ul spacing="normal" bare="false" empty="false" pn="section-4.8-2"> <li pn="section-4.8-2.1"> Details of the DetNet system's capabilities that are required in order to accurately allocate that DetNet system's resources, as well as other DetNet systems' resources. This includes, for example, which specific queuing and shaping algorithms are implemented (<xreftarget="QueuingModels"/>),target="QueuingModels" format="default" sectionFormat="of" derivedContent="Section 4.5"/>), the number of buffers dedicated for DetNet allocation, and the worst-case forwarding delay and misordering.</t><t></li> <li pn="section-4.8-2.2"> The actual state of a DetNet node's DetNet resources.</t><t></li> <li pn="section-4.8-2.3"> The identity of the DetNet system'sneighbors,neighbors and the characteristics of the link(s) between the DetNet systems, including the latency of the links (in nanoseconds).</t> </list> </t></li> </ul> </section> <section anchor="Scaling"title="Scalingnumbered="true" toc="include" removeInRFC="false" pn="section-4.9"> <name slugifiedName="name-scaling-to-larger-networks">Scaling tolarger networks"> <t>Larger Networks</name> <t pn="section-4.9-1"> Reservations for individual DetNet flows require considerable state information in each DetNet node, especially when adequate fault mitigation (<xreftarget="FaultMitigation"/>)target="FaultMitigation" format="default" sectionFormat="of" derivedContent="Section 3.3.2"/>) is required. The DetNetdata plane,Data Plane, in order to support larger numbers of DetNet flows, must support the aggregation of DetNet flows. Such aggregated flows can be viewed by the DetNet nodes'data planeData Plane largely as individual DetNet flows. Without such aggregation, the per-relay system may limit the scale of DetNet networks. Example techniques that may be used include MPLS hierarchy and IP DiffServ Code Points (DSCPs). </t> </section> <section anchor="Compatibility"title="Compatibilitynumbered="true" toc="include" removeInRFC="false" pn="section-4.10"> <name slugifiedName="name-compatibility-with-layer-2">Compatibility withLayer-2"> <t>Layer 2</name> <t pn="section-4.10-1"> Standards providing similar capabilities for bridged networks (only) have been and are being generated in the IEEE 802 LAN/MAN Standards Committee. The present architecture describes an abstract model that can be applicable both atLayer-2Layer 2 andLayer-3,Layer 3, and over links not defined by IEEE 802.</t><t> DetNet enabled</t> <t pn="section-4.10-2"> DetNet-enabled end systems and DetNet nodes can be interconnected by sub-networks, i.e.,Layer-2Layer 2 technologies. These sub-networks will provide DetNet compatible service for support of DetNet traffic. Examples of sub-network technologies include MPLS TE,802.1 TSN,TSN as defined by IEEE 802.1, and a point-to-point OTN link. Of course,multi-layermultilayer DetNet systems may be possible too, where one DetNet appears as asub-network,sub-network and provides serviceto,to ahigher layerhigher-layer DetNet system. </t> </section> </section> <section anchor="SecurityConsiderations"title="Security Considerations"> <t>numbered="true" toc="include" removeInRFC="false" pn="section-5"> <name slugifiedName="name-security-considerations">Security Considerations</name> <t pn="section-5-1"> Security considerations for DetNet are described in detail in <xreftarget="I-D.ietf-detnet-security"/>.target="I-D.ietf-detnet-security" format="default" sectionFormat="of" derivedContent="DETNET-SECURITY"/>. This section considers exclusively security considerationswhichthat are specific to the DetNet architecture.</t><t></t> <t pn="section-5-2"> Security aspectswhichthat are unique to DetNet are those whose aim is to provide the specificquality of serviceQoS aspects of DetNet, which are primarily to deliver data flows with extremely low packet loss rates and bounded end-to-end delivery latency. A DetNet may be implemented using MPLS and/or IP (including both v4 and v6)technologies,technologies and thus inherits the security properties of those technologies at both thedata planeData Plane and thecontrol plane. </t><t>Controller Plane. </t> <t pn="section-5-3"> Security considerations for DetNet are constrained (compared to, for example, the open Internet) because DetNet is defined to operate only within a single administrative domain (seeSection 1).<xref target="introduction" format="default" sectionFormat="of" derivedContent="Section 1"/>). The primary considerations are to secure the request and control of DetNet resources, maintain confidentiality of data traversing the DetNet, and provide the availability of the DetNetquality of service. </t><t>QoS. </t> <t pn="section-5-4"> To secure the request and control of DetNet resources, authentication and authorization can be used for each device connected to a DetNet domain, most importantly to network controller devices. Control of a DetNet network may be centralized or distributed (within a single administrative domain). In the case of centralized control, precedent for security considerations as defined for Abstraction and Control of Traffic Engineered Networks (ACTN) can be found in <xreftarget="RFC8453"/>, Section 9.target="RFC8453" sectionFormat="of" section="9" format="default" derivedLink="https://rfc-editor.org/rfc/rfc8453#section-9" derivedContent="RFC8453"/>. In the case of distributed control protocols, DetNet security is expected to be provided by the security properties of the protocols in use. In any case, the result is that manipulation of administratively configurable parameters is limited to authorized entities.</t><t></t> <t pn="section-5-5"> To maintain confidentiality of data traversing the DetNet, application flows can be protected through whatever means is provided by the underlying technology. For example, encryption may be used, such as that provided byIPSecIPsec <xreftarget="RFC4301"/>target="RFC4301" format="default" sectionFormat="of" derivedContent="RFC4301"/>, for IP flows and by MACSec <xreftarget="IEEE802.1AE-2018"/>target="IEEE802.1AE" format="default" sectionFormat="of" derivedContent="IEEE802.1AE"/> for Ethernet(Layer-2)(Layer 2) flows.</t><t></t> <t pn="section-5-6"> DetNet flows are identified on a per-flow basis, which may provide attackers with additional information about the data flows (when compared to networks that do not include per-flow identification). This is an inherent property of DetNetwhichthat has security implications that should be considered when determining if DetNet is a suitable technology for any given use case.</t><t></t> <t pn="section-5-7"> To provide uninterrupted availability of the DetNetquality of service,QoS, provisions can be made againstDOSDoS attacks and delay attacks. To protect againstDOSDoS attacks, excess traffic due to malicious or malfunctioning devices can be prevented or mitigated, forexampleexample, through the use of traffic admission control applied at the input of a DetNetdomain,domain as described in <xreftarget="Zero"/>,target="Zero" format="default" sectionFormat="of" derivedContent="Section 3.2.1"/> and through thefault mitigationfault-mitigation methods described in <xreftarget="FaultMitigation"/>.target="FaultMitigation" format="default" sectionFormat="of" derivedContent="Section 3.3.2"/>. To prevent DetNet packets from being delayed by an entity external to a DetNet domain, DetNet technology definition can allow for the mitigation ofMan-In-The-Middleman-in-the-middle attacks, forexampleexample, through use of authentication and authorization of devices within the DetNet domain.</t><t></t> <t pn="section-5-8"> Because DetNet mechanisms or applications that rely on DetNet can make heavy use of methods that require precise time synchronization, the accuracy, availability, and integrity of time synchronization is of critical importance. Extensive discussion of this topic can be found in <xreftarget="RFC7384"/>. </t><t>target="RFC7384" format="default" sectionFormat="of" derivedContent="RFC7384"/>. </t> <t pn="section-5-9"> DetNet use cases are known to have widely divergent security requirements. The intent of this section is to provide a baseline for security considerationswhichthat are common to all DetNet designs and implementations, without burdening individual designs with specifics of security infrastructurewhichthat may not be germane to the given use case. Designers andimplementersimplementors of DetNet systems are expected to takeuse case specificuse-case-specific considerations into account in their DetNet designs and implementations. </t> </section> <sectiontitle="Privacy Considerations"> <t>numbered="true" toc="include" removeInRFC="false" pn="section-6"> <name slugifiedName="name-privacy-considerations">Privacy Considerations</name> <t pn="section-6-1"> DetNet provides aQuality of Service (QoS),QoS, and the generic considerations for such mechanisms apply. In particular, such markings allow for an attacker to correlate flows or to select particular types of flow for more detailed inspection.</t><t></t> <t pn="section-6-2"> However, the requirement for every (or almost every) node along the path of a DetNet flow to identify DetNet flows may present an additional attack surface forprivacy,privacy should the DetNet paradigm be found useful in broader environments. </t> </section> <sectiontitle="IANA Considerations"> <t>Thisnumbered="true" toc="include" removeInRFC="false" pn="section-7"> <name slugifiedName="name-iana-considerations">IANA Considerations</name> <t pn="section-7-1">This documentdoes not require an action from IANA. </t> </section> <section title="Acknowledgements"> <t>The authors wish to thank Lou Berger, David Black, Stewart Bryant, Rodney Cummings, Ethan Grossman, Craig Gunther, Marcel Kiessling, Rudy Klecka, Jouni Korhonen, Erik Nordmark, Shitanshu Shah, Wilfried Steiner, George Swallow, Michael Johas Teener, Pat Thaler, Thomas Watteyne, Patrick Wetterwald, Karl Weber, Anca Zamfir, for their various contributions to this work.has no IANA actions. </t> </section> </middle> <back> <displayreference target="I-D.ietf-detnet-security" to="DETNET-SECURITY"/> <displayreference target="I-D.ietf-detnet-ip" to="DETNET-IP"/> <displayreference target="I-D.ietf-detnet-data-plane-framework" to="DETNET-FRAMEWORK"/> <displayreference target="I-D.ietf-detnet-mpls" to="DETNET-MPLS"/> <displayreference target="I-D.ietf-6tisch-architecture" to="TSCH-ARCH"/> <referencestitle='Informative References'> <?rfc include='reference.I-D.ietf-6tisch-architecture'?> <?rfc include='reference.I-D.ietf-detnet-problem-statement'?> <?rfc include='reference.I-D.ietf-detnet-use-cases'?> <?rfc include='reference.I-D.ietf-detnet-dp-sol-mpls'?> <?rfc include='reference.I-D.ietf-detnet-dp-sol-ip'?> <?rfc include='reference.I-D.ietf-detnet-security'?> <?rfc include='reference.RFC.2205'?> <?rfc include='reference.RFC.2475'?> <?rfc include='reference.RFC.2914'?> <?rfc include='reference.RFC.3168'?> <?rfc include='reference.RFC.3209'?> <?rfc include='reference.RFC.3550'?> <?rfc include='reference.RFC.4301'?> <?rfc include='reference.RFC.4655'?> <?rfc include='reference.RFC.5921'?> <?rfc include='reference.RFC.6372'?> <?rfc include='reference.RFC.6658'?> <?rfc include='reference.RFC.7149'?> <?rfc include='reference.RFC.7384'?> <?rfc include='reference.RFC.7426'?> <?rfc include='reference.RFC.7554'?> <?rfc include='reference.RFC.7567'?> <?rfc include='reference.RFC.7813'?> <?rfc include='reference.RFC.8033'?> <?rfc include='reference.RFC.8227'?> <?rfc include='reference.RFC.8289'?> <?rfc include='reference.RFC.8402'?> <?rfc include='reference.RFC.8453'?> <reference anchor="IEC62439-3-2016" target="https://webstore.iec.ch/publication/24447"> <front> <title>IEC 62439-3:2016 Industrial communication networks - High availability automation networks - Part 3: Parallel Redundancy Protocol (PRP) and High-availability Seamless Redundancy (HSR) </title> <author> <organization>International Electrotechnical Commission (IEC) TC 65/SC 65C - Industrial networks</organization> </author> <date year="2016" /> </front> </reference> <reference anchor="IEEE802.1AE-2018" target="https://ieeexplore.ieee.org/document/8585421"> <front> <title>IEEE Std 802.1AE-2018 MAC Security (MACsec)</title> <author> <organization>IEEE Standards Association</organization> </author> <date year="2018" /> </front> </reference>pn="section-8"> <name slugifiedName="name-informative-references">Informative References</name> <referenceanchor="IEEE802.1CB" target="https://ieeexplore.ieee.org/document/8091139/">anchor="BUFFERBLOAT" quoteTitle="true" target="https://doi.org/10.1145/2063176.2063196" derivedAnchor="BUFFERBLOAT"> <front><title>IEEE Std 802.1CB-2017 Frame Replication and Elimination for Reliability</title> <author> <organization>IEEE Standards Association</organization><title>Bufferbloat: Dark Buffers in the Internet</title> <seriesInfo name="DOI" value="10.1145/2063176.2063196"/> <seriesInfo name="Communications of the ACM," value="Volume 55, Issue 1"/> <author initials="J." surname="Gettys" fullname="Jim Gettys"> <organization showOnFrontPage="true"/> </author> <author initials="K." surname="Nichols" fullname="Kathleen Nichols"> <organization showOnFrontPage="true"/> </author> <dateyear="2017" />month="January" year="2012"/> </front> </reference> <referenceanchor="IEEE802.1Q-2018" target="https://ieeexplore.ieee.org/document/8403927">anchor="CCAMP" target="https://datatracker.ietf.org/wg/ccamp/charter/" quoteTitle="true" derivedAnchor="CCAMP"> <front><title>IEEE Std 802.1Q-2018 Bridges<title>Common Control andBridged Networks</title>Measurement Plane (ccamp)</title> <author><organization>IEEE Standards Association</organization><organization showOnFrontPage="true">IETF</organization> </author><date year="2018" /><date/> </front> </reference> <referenceanchor="IEEE802.1Qav" target="https://ieeexplore.ieee.org/document/5375704/">anchor="I-D.ietf-detnet-data-plane-framework" quoteTitle="true" target="https://tools.ietf.org/html/draft-ietf-detnet-data-plane-framework-02" derivedAnchor="DETNET-FRAMEWORK"> <front><title>IEEE Std 802.1Qav-2009 Bridges and Bridged Networks - Amendment 12: Forwarding and Queuing Enhancements for Time-Sensitive Streams</title> <author> <organization>IEEE Standards Association</organization><title>DetNet Data Plane Framework</title> <author initials="B" surname="Varga" fullname="Balazs Varga"> <organization showOnFrontPage="true"/> </author> <author initials="J" surname="Farkas" fullname="Janos Farkas"> <organization showOnFrontPage="true"/> </author> <author initials="L" surname="Berger" fullname="Lou Berger"> <organization showOnFrontPage="true"/> </author> <author initials="D" surname="Fedyk" fullname="Don Fedyk"> <organization showOnFrontPage="true"/> </author> <author initials="A" surname="Malis" fullname="Andrew Malis"> <organization showOnFrontPage="true"/> </author> <author initials="S" surname="Bryant" fullname="Stewart Bryant"> <organization showOnFrontPage="true"/> </author> <author initials="J" surname="Korhonen" fullname="Jouni Korhonen"> <organization showOnFrontPage="true"/> </author> <dateyear="2009" />month="September" day="13" year="2019"/> <abstract> <t>This document provides an overall framework for the Deterministic Networking data plane. It covers concepts and considerations that are generally common to any Deterministic Networking data plane specification.</t> </abstract> </front> <seriesInfo name="Internet-Draft" value="draft-ietf-detnet-data-plane-framework-02"/> <format type="TXT" target="http://www.ietf.org/internet-drafts/draft-ietf-detnet-data-plane-framework-02.txt"/> <refcontent>Work in Progress</refcontent> </reference> <referenceanchor="IEEE802.1Qbu" target="https://ieeexplore.ieee.org/document/7553415/">anchor="I-D.ietf-detnet-ip" quoteTitle="true" target="https://tools.ietf.org/html/draft-ietf-detnet-ip-01" derivedAnchor="DETNET-IP"> <front><title>IEEE Std 802.1Qbu-2016 Bridges and Bridged Networks - Amendment 26: Frame Preemption</title> <author> <organization>IEEE Standards Association</organization><title>DetNet Data Plane: IP</title> <author initials="B" surname="Varga" fullname="Balazs Varga"> <organization showOnFrontPage="true"/> </author> <author initials="J" surname="Farkas" fullname="Janos Farkas"> <organization showOnFrontPage="true"/> </author> <author initials="L" surname="Berger" fullname="Lou Berger"> <organization showOnFrontPage="true"/> </author> <author initials="D" surname="Fedyk" fullname="Don Fedyk"> <organization showOnFrontPage="true"/> </author> <author initials="A" surname="Malis" fullname="Andrew Malis"> <organization showOnFrontPage="true"/> </author> <author initials="S" surname="Bryant" fullname="Stewart Bryant"> <organization showOnFrontPage="true"/> </author> <author initials="J" surname="Korhonen" fullname="Jouni Korhonen"> <organization showOnFrontPage="true"/> </author> <dateyear="2016" />month="July" day="1" year="2019"/> <abstract> <t>This document specifies the Deterministic Networking data plane when operating in an IP packet switched network.</t> </abstract> </front> <seriesInfo name="Internet-Draft" value="draft-ietf-detnet-ip-01"/> <format type="TXT" target="http://www.ietf.org/internet-drafts/draft-ietf-detnet-ip-01.txt"/> <refcontent>Work in Progress</refcontent> </reference> <referenceanchor="IEEE802.1Qbv" target="https://ieeexplore.ieee.org/document/7572858/">anchor="I-D.ietf-detnet-mpls" quoteTitle="true" target="https://tools.ietf.org/html/draft-ietf-detnet-mpls-01" derivedAnchor="DETNET-MPLS"> <front><title>IEEE Std 802.1Qbv-2015 Bridges and Bridged Networks - Amendment 25: Enhancements for Scheduled Traffic</title> <author> <organization>IEEE Standards Association</organization><title>DetNet Data Plane: MPLS</title> <author initials="B" surname="Varga" fullname="Balazs Varga"> <organization showOnFrontPage="true"/> </author> <author initials="J" surname="Farkas" fullname="Janos Farkas"> <organization showOnFrontPage="true"/> </author> <author initials="L" surname="Berger" fullname="Lou Berger"> <organization showOnFrontPage="true"/> </author> <author initials="D" surname="Fedyk" fullname="Don Fedyk"> <organization showOnFrontPage="true"/> </author> <author initials="A" surname="Malis" fullname="Andrew Malis"> <organization showOnFrontPage="true"/> </author> <author initials="S" surname="Bryant" fullname="Stewart Bryant"> <organization showOnFrontPage="true"/> </author> <author initials="J" surname="Korhonen" fullname="Jouni Korhonen"> <organization showOnFrontPage="true"/> </author> <dateyear="2015" />month="July" day="1" year="2019"/> <abstract> <t>This document specifies the Deterministic Networking data plane when operating over an MPLS Packet Switched Networks.</t> </abstract> </front> <seriesInfo name="Internet-Draft" value="draft-ietf-detnet-mpls-01"/> <format type="TXT" target="http://www.ietf.org/internet-drafts/draft-ietf-detnet-mpls-01.txt"/> <refcontent>Work in Progress</refcontent> </reference> <referenceanchor="IEEE802.1BA" target="https://ieeexplore.ieee.org/document/6032690/">anchor="I-D.ietf-detnet-security" quoteTitle="true" target="https://tools.ietf.org/html/draft-ietf-detnet-security-05" derivedAnchor="DETNET-SECURITY"> <front><title>IEEE Std 802.1BA-2011 Audio Video Bridging (AVB) Systems</title> <author> <organization>IEEE Standards Association</organization><title>Deterministic Networking (DetNet) Security Considerations</title> <author initials="T" surname="Mizrahi" fullname="Tal Mizrahi"> <organization showOnFrontPage="true"/> </author> <author initials="E" surname="Grossman" fullname="Ethan Grossman"> <organization showOnFrontPage="true"/> </author> <author initials="A" surname="Hacker" fullname="Andrew Hacker"> <organization showOnFrontPage="true"/> </author> <author initials="S" surname="Das" fullname="Subir Das"> <organization showOnFrontPage="true"/> </author> <author initials="J" surname="Dowdell" fullname="John Dowdell"> <organization showOnFrontPage="true"/> </author> <author initials="H" surname="Austad" fullname="Henrik Austad"> <organization showOnFrontPage="true"/> </author> <author initials="K" surname="Stanton" fullname="Kevin Stanton"> <organization showOnFrontPage="true"/> </author> <author initials="N" surname="Finn" fullname="Norman Finn"> <organization showOnFrontPage="true"/> </author> <dateyear="2011" />month="August" day="29" year="2019"/> <abstract> <t>A deterministic network is one that can carry data flows for real- time applications with extremely low data loss rates and bounded latency. Deterministic networks have been successfully deployed in real-time operational technology (OT) applications for some years (for example [ARINC664P7]). However, such networks are typically isolated from external access, and thus the security threat from external attackers is low. IETF Deterministic Networking (DetNet) specifies a set of technologies that enable creation of deterministic networks on IP-based networks of potentially wide area (on the scale of a corporate network) potentially bringing the OT network into contact with Information Technology (IT) traffic and security threats that lie outside of a tightly controlled and bounded area (such as the internals of an aircraft). These DetNet technologies have not previously been deployed together on a wide area IP-based network, and thus can present security considerations that may be new to IP- based wide area network designers. This draft, intended for use by DetNet network designers, provides insight into these security considerations. In addition, this draft collects all security- related statements from the various DetNet drafts (Architecture, Use Cases, etc) into a single location Section 8.</t> </abstract> </front> <seriesInfo name="Internet-Draft" value="draft-ietf-detnet-security-05"/> <format type="TXT" target="http://www.ietf.org/internet-drafts/draft-ietf-detnet-security-05.txt"/> <refcontent>Work in Progress</refcontent> </reference> <reference anchor="IEC-62439-3" target="https://webstore.iec.ch/publication/24447" quoteTitle="true" derivedAnchor="IEC-62439-3"> <front> <title>Industrial communication networks - High availability automation networks - Part 3: Parallel Redundancy Protocol (PRP) and High-availability Seamless Redundancy (HSR) </title> <seriesInfo name="TC 65 /" value="SC 65C"/> <seriesInfo name="IEC" value="62439-3:2016"/> <author> <organization showOnFrontPage="true">IEC</organization> </author> <date month="March" year="2016"/> </front> </reference> <reference anchor="IEEE802.1AE" target="https://ieeexplore.ieee.org/document/8585421" quoteTitle="true" derivedAnchor="IEEE802.1AE"> <front> <title>IEEE Standard for Local and metropolitan area networks-Media Access Control (MAC) Security</title> <seriesInfo name="IEEE" value="802.1AE-2018"/> <author> <organization showOnFrontPage="true">IEEE</organization> </author> </front> </reference> <reference anchor="IEEE802.1BA" target="https://ieeexplore.ieee.org/document/6032690" quoteTitle="true" derivedAnchor="IEEE802.1BA"> <front> <title>IEEE Standard for Local and metropolitan area networks--Audio Video Bridging (AVB) Systems</title> <seriesInfo name="IEEE" value="802.1BA-2011"/> <author> <organization showOnFrontPage="true">IEEE</organization> </author> </front> </reference> <reference anchor="IEEE802.1CB" target="https://ieeexplore.ieee.org/document/8091139" quoteTitle="true" derivedAnchor="IEEE802.1CB"> <front> <title>IEEE Standard for Local and metropolitan area networks--Frame Replication and Elimination for Reliability</title> <seriesInfo name="DOI" value="10.1109/IEEESTD.2017.8091139"/> <seriesInfo name="IEEE" value="802.1CB-2017"/> <author> <organization showOnFrontPage="true">IEEE</organization> </author> <date/> </front> </reference> <reference anchor="IEEE802.1Q" target="https://ieeexplore.ieee.org/document/8403927" quoteTitle="true" derivedAnchor="IEEE802.1Q"> <front> <title>IEEE Standard for Local and Metropolitan Area Network--Bridges and Bridged Networks</title> <seriesInfo name="IEEE" value="802.1Q-2018"/> <author> <organization showOnFrontPage="true">IEEE</organization> </author> </front> </reference> <reference anchor="IEEE802.1Qav" target="https://ieeexplore.ieee.org/document/5375704" quoteTitle="true" derivedAnchor="IEEE802.1Qav"> <front> <title>IEEE Standard for Local and Metropolitan Area Networks - Virtual Bridged Local Area Networks Amendment 12: Forwarding and Queuing Enhancements for Time-Sensitive Streams</title> <seriesInfo name="IEEE" value="802.1Qav-2009"/> <author> <organization showOnFrontPage="true">IEEE</organization> </author> </front> </reference> <reference anchor="IEEE802.1Qbu" target="https://ieeexplore.ieee.org/document/7553415" quoteTitle="true" derivedAnchor="IEEE802.1Qbu"> <front> <title>IEEE Standard for Local and metropolitan area networks -- Bridges and Bridged Networks -- Amendment 26: Frame Preemption</title> <seriesInfo name="IEEE" value="802.1Qbu-2016"/> <author> <organization showOnFrontPage="true">IEEE</organization> </author> </front> </reference> <reference anchor="IEEE802.1Qbv" target="https://ieeexplore.ieee.org/document/7440741" quoteTitle="true" derivedAnchor="IEEE802.1Qbv"> <front> <title>IEEE Standard for Local and metropolitan area networks -- Bridges and Bridged Networks - Amendment 25: Enhancements for Scheduled Traffic</title> <seriesInfo name="IEEE" value="802.1Qbv-2015"/> <author> <organization showOnFrontPage="true">IEEE</organization> </author> </front> </reference> <reference anchor="IEEE802.1Qch" target="https://ieeexplore.ieee.org/document/7961303" quoteTitle="true" derivedAnchor="IEEE802.1Qch"> <front> <title>IEEE Standard for Local and metropolitan area networks--Bridges and Bridged Networks--Amendment 29: Cyclic Queuing and Forwarding</title> <seriesInfo name="IEEE" value="802.1Qch-2017"/> <author> <organization showOnFrontPage="true">IEEE</organization> </author> </front> </reference> <reference anchor="IEEE802.1TSNTG" target="https://1.ieee802.org/tsn/" quoteTitle="true" derivedAnchor="IEEE802.1TSNTG"> <front> <title>Time-Sensitive Networking (TSN) Task Group</title> <author> <organization showOnFrontPage="true">IEEE</organization> </author> </front> </reference> <reference anchor="IEEE802.3" target="https://ieeexplore.ieee.org/document/8457469" quoteTitle="true" derivedAnchor="IEEE802.3"> <front> <title>IEEE Standard for Ethernet</title> <seriesInfo name="IEEE" value="802.3-2018"/> <author> <organization showOnFrontPage="true">IEEE</organization> </author> </front> </reference> <reference anchor="IEEE802.3br" target="https://ieeexplore.ieee.org/document/7900321" quoteTitle="true" derivedAnchor="IEEE802.3br"> <front> <title>IEEE Standard for Ethernet Amendment 5: Specification and Management Parameters for Interspersing Express Traffic</title> <seriesInfo name="IEEE" value="802.3br"/> <author> <organization showOnFrontPage="true">IEEE</organization> </author> </front> </reference> <reference anchor="RFC2205" target="https://www.rfc-editor.org/info/rfc2205" quoteTitle="true" derivedAnchor="RFC2205"> <front> <title>Resource ReSerVation Protocol (RSVP) -- Version 1 Functional Specification</title> <author initials="R." surname="Braden" fullname="R. Braden" role="editor"> <organization showOnFrontPage="true"/> </author> <author initials="L." surname="Zhang" fullname="L. Zhang"> <organization showOnFrontPage="true"/> </author> <author initials="S." surname="Berson" fullname="S. Berson"> <organization showOnFrontPage="true"/> </author> <author initials="S." surname="Herzog" fullname="S. Herzog"> <organization showOnFrontPage="true"/> </author> <author initials="S." surname="Jamin" fullname="S. Jamin"> <organization showOnFrontPage="true"/> </author> <date year="1997" month="September"/> <abstract> <t>This memo describes version 1 of RSVP, a resource reservation setup protocol designed for an integrated services Internet. RSVP provides receiver-initiated setup of resource reservations for multicast or unicast data flows, with good scaling and robustness properties. [STANDARDS-TRACK]</t> </abstract> </front> <seriesInfo name="RFC" value="2205"/> <seriesInfo name="DOI" value="10.17487/RFC2205"/> </reference> <reference anchor="RFC2475" target="https://www.rfc-editor.org/info/rfc2475" quoteTitle="true" derivedAnchor="RFC2475"> <front> <title>An Architecture for Differentiated Services</title> <author initials="S." surname="Blake" fullname="S. Blake"> <organization showOnFrontPage="true"/> </author> <author initials="D." surname="Black" fullname="D. Black"> <organization showOnFrontPage="true"/> </author> <author initials="M." surname="Carlson" fullname="M. Carlson"> <organization showOnFrontPage="true"/> </author> <author initials="E." surname="Davies" fullname="E. Davies"> <organization showOnFrontPage="true"/> </author> <author initials="Z." surname="Wang" fullname="Z. Wang"> <organization showOnFrontPage="true"/> </author> <author initials="W." surname="Weiss" fullname="W. Weiss"> <organization showOnFrontPage="true"/> </author> <date year="1998" month="December"/> <abstract> <t>This document defines an architecture for implementing scalable service differentiation in the Internet. This memo provides information for the Internet community.</t> </abstract> </front> <seriesInfo name="RFC" value="2475"/> <seriesInfo name="DOI" value="10.17487/RFC2475"/> </reference> <reference anchor="RFC2914" target="https://www.rfc-editor.org/info/rfc2914" quoteTitle="true" derivedAnchor="RFC2914"> <front> <title>Congestion Control Principles</title> <author initials="S." surname="Floyd" fullname="S. Floyd"> <organization showOnFrontPage="true"/> </author> <date year="2000" month="September"/> <abstract> <t>The goal of this document is to explain the need for congestion control in the Internet, and to discuss what constitutes correct congestion control. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements.</t> </abstract> </front> <seriesInfo name="BCP" value="41"/> <seriesInfo name="RFC" value="2914"/> <seriesInfo name="DOI" value="10.17487/RFC2914"/> </reference> <reference anchor="RFC3168" target="https://www.rfc-editor.org/info/rfc3168" quoteTitle="true" derivedAnchor="RFC3168"> <front> <title>The Addition of Explicit Congestion Notification (ECN) to IP</title> <author initials="K." surname="Ramakrishnan" fullname="K. Ramakrishnan"> <organization showOnFrontPage="true"/> </author> <author initials="S." surname="Floyd" fullname="S. Floyd"> <organization showOnFrontPage="true"/> </author> <author initials="D." surname="Black" fullname="D. Black"> <organization showOnFrontPage="true"/> </author> <date year="2001" month="September"/> <abstract> <t>This memo specifies the incorporation of ECN (Explicit Congestion Notification) to TCP and IP, including ECN's use of two bits in the IP header. [STANDARDS-TRACK]</t> </abstract> </front> <seriesInfo name="RFC" value="3168"/> <seriesInfo name="DOI" value="10.17487/RFC3168"/> </reference> <reference anchor="RFC3209" target="https://www.rfc-editor.org/info/rfc3209" quoteTitle="true" derivedAnchor="RFC3209"> <front> <title>RSVP-TE: Extensions to RSVP for LSP Tunnels</title> <author initials="D." surname="Awduche" fullname="D. Awduche"> <organization showOnFrontPage="true"/> </author> <author initials="L." surname="Berger" fullname="L. Berger"> <organization showOnFrontPage="true"/> </author> <author initials="D." surname="Gan" fullname="D. Gan"> <organization showOnFrontPage="true"/> </author> <author initials="T." surname="Li" fullname="T. Li"> <organization showOnFrontPage="true"/> </author> <author initials="V." surname="Srinivasan" fullname="V. Srinivasan"> <organization showOnFrontPage="true"/> </author> <author initials="G." surname="Swallow" fullname="G. Swallow"> <organization showOnFrontPage="true"/> </author> <date year="2001" month="December"/> <abstract> <t>This document describes the use of RSVP (Resource Reservation Protocol), including all the necessary extensions, to establish label-switched paths (LSPs) in MPLS (Multi-Protocol Label Switching). Since the flow along an LSP is completely identified by the label applied at the ingress node of the path, these paths may be treated as tunnels. A key application of LSP tunnels is traffic engineering with MPLS as specified in RFC 2702. [STANDARDS-TRACK]</t> </abstract> </front> <seriesInfo name="RFC" value="3209"/> <seriesInfo name="DOI" value="10.17487/RFC3209"/> </reference> <reference anchor="RFC3550" target="https://www.rfc-editor.org/info/rfc3550" quoteTitle="true" derivedAnchor="RFC3550"> <front> <title>RTP: A Transport Protocol for Real-Time Applications</title> <author initials="H." surname="Schulzrinne" fullname="H. Schulzrinne"> <organization showOnFrontPage="true"/> </author> <author initials="S." surname="Casner" fullname="S. Casner"> <organization showOnFrontPage="true"/> </author> <author initials="R." surname="Frederick" fullname="R. Frederick"> <organization showOnFrontPage="true"/> </author> <author initials="V." surname="Jacobson" fullname="V. Jacobson"> <organization showOnFrontPage="true"/> </author> <date year="2003" month="July"/> <abstract> <t>This memorandum describes RTP, the real-time transport protocol. RTP provides end-to-end network transport functions suitable for applications transmitting real-time data, such as audio, video or simulation data, over multicast or unicast network services. RTP does not address resource reservation and does not guarantee quality-of- service for real-time services. The data transport is augmented by a control protocol (RTCP) to allow monitoring of the data delivery in a manner scalable to large multicast networks, and to provide minimal control and identification functionality. RTP and RTCP are designed to be independent of the underlying transport and network layers. The protocol supports the use of RTP-level translators and mixers. Most of the text in this memorandum is identical to RFC 1889 which it obsoletes. There are no changes in the packet formats on the wire, only changes to the rules and algorithms governing how the protocol is used. The biggest change is an enhancement to the scalable timer algorithm for calculating when to send RTCP packets in order to minimize transmission in excess of the intended rate when many participants join a session simultaneously. [STANDARDS-TRACK]</t> </abstract> </front> <seriesInfo name="STD" value="64"/> <seriesInfo name="RFC" value="3550"/> <seriesInfo name="DOI" value="10.17487/RFC3550"/> </reference> <reference anchor="RFC4301" target="https://www.rfc-editor.org/info/rfc4301" quoteTitle="true" derivedAnchor="RFC4301"> <front> <title>Security Architecture for the Internet Protocol</title> <author initials="S." surname="Kent" fullname="S. Kent"> <organization showOnFrontPage="true"/> </author> <author initials="K." surname="Seo" fullname="K. Seo"> <organization showOnFrontPage="true"/> </author> <date year="2005" month="December"/> <abstract> <t>This document describes an updated version of the "Security Architecture for IP", which is designed to provide security services for traffic at the IP layer. This document obsoletes RFC 2401 (November 1998). [STANDARDS-TRACK]</t> </abstract> </front> <seriesInfo name="RFC" value="4301"/> <seriesInfo name="DOI" value="10.17487/RFC4301"/> </reference> <reference anchor="RFC4655" target="https://www.rfc-editor.org/info/rfc4655" quoteTitle="true" derivedAnchor="RFC4655"> <front> <title>A Path Computation Element (PCE)-Based Architecture</title> <author initials="A." surname="Farrel" fullname="A. Farrel"> <organization showOnFrontPage="true"/> </author> <author initials="J.-P." surname="Vasseur" fullname="J.-P. Vasseur"> <organization showOnFrontPage="true"/> </author> <author initials="J." surname="Ash" fullname="J. Ash"> <organization showOnFrontPage="true"/> </author> <date year="2006" month="August"/> <abstract> <t>Constraint-based path computation is a fundamental building block for traffic engineering systems such as Multiprotocol Label Switching (MPLS) and Generalized Multiprotocol Label Switching (GMPLS) networks. Path computation in large, multi-domain, multi-region, or multi-layer networks is complex and may require special computational components and cooperation between the different network domains.</t> <t>This document specifies the architecture for a Path Computation Element (PCE)-based model to address this problem space. This document does not attempt to provide a detailed description of all the architectural components, but rather it describes a set of building blocks for the PCE architecture from which solutions may be constructed. This memo provides information for the Internet community.</t> </abstract> </front> <seriesInfo name="RFC" value="4655"/> <seriesInfo name="DOI" value="10.17487/RFC4655"/> </reference> <reference anchor="RFC6372" target="https://www.rfc-editor.org/info/rfc6372" quoteTitle="true" derivedAnchor="RFC6372"> <front> <title>MPLS Transport Profile (MPLS-TP) Survivability Framework</title> <author initials="N." surname="Sprecher" fullname="N. Sprecher" role="editor"> <organization showOnFrontPage="true"/> </author> <author initials="A." surname="Farrel" fullname="A. Farrel" role="editor"> <organization showOnFrontPage="true"/> </author> <date year="2011" month="September"/> <abstract> <t>Network survivability is the ability of a network to recover traffic delivery following failure or degradation of network resources. Survivability is critical for the delivery of guaranteed network services, such as those subject to strict Service Level Agreements (SLAs) that place maximum bounds on the length of time that services may be degraded or unavailable.</t> <t>The Transport Profile of Multiprotocol Label Switching (MPLS-TP) is a packet-based transport technology based on the MPLS data plane that reuses many aspects of the MPLS management and control planes.</t> <t>This document comprises a framework for the provision of survivability in an MPLS-TP network; it describes recovery elements, types, methods, and topological considerations. To enable data-plane recovery, survivability may be supported by the control plane, management plane, and by Operations, Administration, and Maintenance (OAM) functions. This document describes mechanisms for recovering MPLS-TP Label Switched Paths (LSPs). A detailed description of pseudowire recovery in MPLS-TP networks is beyond the scope of this document.</t> <t>This document is a product of a joint Internet Engineering Task Force (IETF) / International Telecommunication Union Telecommunication Standardization Sector (ITU-T) effort to include an MPLS Transport Profile within the IETF MPLS and Pseudowire Emulation Edge-to-Edge (PWE3) architectures to support the capabilities and functionalities of a packet-based transport network as defined by the ITU-T. </t> <t>This document is not an Internet Standards Track specification; it is published for informational purposes.</t> </abstract> </front> <seriesInfo name="RFC" value="6372"/> <seriesInfo name="DOI" value="10.17487/RFC6372"/> </reference> <reference anchor="RFC6658" target="https://www.rfc-editor.org/info/rfc6658" quoteTitle="true" derivedAnchor="RFC6658"> <front> <title>Packet Pseudowire Encapsulation over an MPLS PSN</title> <author initials="S." surname="Bryant" fullname="S. Bryant" role="editor"> <organization showOnFrontPage="true"/> </author> <author initials="L." surname="Martini" fullname="L. Martini"> <organization showOnFrontPage="true"/> </author> <author initials="G." surname="Swallow" fullname="G. Swallow"> <organization showOnFrontPage="true"/> </author> <author initials="A." surname="Malis" fullname="A. Malis"> <organization showOnFrontPage="true"/> </author> <date year="2012" month="July"/> <abstract> <t>This document describes a pseudowire mechanism that is used to transport a packet service over an MPLS PSN in the case where the client Label Switching Router (LSR) and the server Provider Edge equipments are co-resident in the same equipment. This pseudowire mechanism may be used to carry all of the required layer 2 and layer 3 protocols between the pair of client LSRs. [STANDARDS-TRACK]</t> </abstract> </front> <seriesInfo name="RFC" value="6658"/> <seriesInfo name="DOI" value="10.17487/RFC6658"/> </reference> <reference anchor="RFC7149" target="https://www.rfc-editor.org/info/rfc7149" quoteTitle="true" derivedAnchor="RFC7149"> <front> <title>Software-Defined Networking: A Perspective from within a Service Provider Environment</title> <author initials="M." surname="Boucadair" fullname="M. Boucadair"> <organization showOnFrontPage="true"/> </author> <author initials="C." surname="Jacquenet" fullname="C. Jacquenet"> <organization showOnFrontPage="true"/> </author> <date year="2014" month="March"/> <abstract> <t>Software-Defined Networking (SDN) has been one of the major buzz words of the networking industry for the past couple of years. And yet, no clear definition of what SDN actually covers has been broadly admitted so far. This document aims to clarify the SDN landscape by providing a perspective on requirements, issues, and other considerations about SDN, as seen from within a service provider environment.</t> <t>It is not meant to endlessly discuss what SDN truly means but rather to suggest a functional taxonomy of the techniques that can be used under an SDN umbrella and to elaborate on the various pending issues the combined activation of such techniques inevitably raises. As such, a definition of SDN is only mentioned for the sake of clarification.</t> </abstract> </front> <seriesInfo name="RFC" value="7149"/> <seriesInfo name="DOI" value="10.17487/RFC7149"/> </reference> <reference anchor="RFC7384" target="https://www.rfc-editor.org/info/rfc7384" quoteTitle="true" derivedAnchor="RFC7384"> <front> <title>Security Requirements of Time Protocols in Packet Switched Networks</title> <author initials="T." surname="Mizrahi" fullname="T. Mizrahi"> <organization showOnFrontPage="true"/> </author> <date year="2014" month="October"/> <abstract> <t>As time and frequency distribution protocols are becoming increasingly common and widely deployed, concern about their exposure to various security threats is increasing. This document defines a set of security requirements for time protocols, focusing on the Precision Time Protocol (PTP) and the Network Time Protocol (NTP). This document also discusses the security impacts of time protocol practices, the performance implications of external security practices on time protocols, and the dependencies between other security services and time synchronization.</t> </abstract> </front> <seriesInfo name="RFC" value="7384"/> <seriesInfo name="DOI" value="10.17487/RFC7384"/> </reference> <reference anchor="RFC7426" target="https://www.rfc-editor.org/info/rfc7426" quoteTitle="true" derivedAnchor="RFC7426"> <front> <title>Software-Defined Networking (SDN): Layers and Architecture Terminology</title> <author initials="E." surname="Haleplidis" fullname="E. Haleplidis" role="editor"> <organization showOnFrontPage="true"/> </author> <author initials="K." surname="Pentikousis" fullname="K. Pentikousis" role="editor"> <organization showOnFrontPage="true"/> </author> <author initials="S." surname="Denazis" fullname="S. Denazis"> <organization showOnFrontPage="true"/> </author> <author initials="J." surname="Hadi Salim" fullname="J. Hadi Salim"> <organization showOnFrontPage="true"/> </author> <author initials="D." surname="Meyer" fullname="D. Meyer"> <organization showOnFrontPage="true"/> </author> <author initials="O." surname="Koufopavlou" fullname="O. Koufopavlou"> <organization showOnFrontPage="true"/> </author> <date year="2015" month="January"/> <abstract> <t>Software-Defined Networking (SDN) refers to a new approach for network programmability, that is, the capacity to initialize, control, change, and manage network behavior dynamically via open interfaces. SDN emphasizes the role of software in running networks through the introduction of an abstraction for the data forwarding plane and, by doing so, separates it from the control plane. This separation allows faster innovation cycles at both planes as experience has already shown. However, there is increasing confusion as to what exactly SDN is, what the layer structure is in an SDN architecture, and how layers interface with each other. This document, a product of the IRTF Software-Defined Networking Research Group (SDNRG), addresses these questions and provides a concise reference for the SDN research community based on relevant peer-reviewed literature, the RFC series, and relevant documents by other standards organizations.</t> </abstract> </front> <seriesInfo name="RFC" value="7426"/> <seriesInfo name="DOI" value="10.17487/RFC7426"/> </reference> <reference anchor="RFC7554" target="https://www.rfc-editor.org/info/rfc7554" quoteTitle="true" derivedAnchor="RFC7554"> <front> <title>Using IEEE 802.15.4e Time-Slotted Channel Hopping (TSCH) in the Internet of Things (IoT): Problem Statement</title> <author initials="T." surname="Watteyne" fullname="T. Watteyne" role="editor"> <organization showOnFrontPage="true"/> </author> <author initials="M." surname="Palattella" fullname="M. Palattella"> <organization showOnFrontPage="true"/> </author> <author initials="L." surname="Grieco" fullname="L. Grieco"> <organization showOnFrontPage="true"/> </author> <date year="2015" month="May"/> <abstract> <t>This document describes the environment, problem statement, and goals for using the Time-Slotted Channel Hopping (TSCH) Medium Access Control (MAC) protocol of IEEE 802.14.4e in the context of Low-Power and Lossy Networks (LLNs). The set of goals enumerated in this document form an initial set only.</t> </abstract> </front> <seriesInfo name="RFC" value="7554"/> <seriesInfo name="DOI" value="10.17487/RFC7554"/> </reference> <reference anchor="RFC7567" target="https://www.rfc-editor.org/info/rfc7567" quoteTitle="true" derivedAnchor="RFC7567"> <front> <title>IETF Recommendations Regarding Active Queue Management</title> <author initials="F." surname="Baker" fullname="F. Baker" role="editor"> <organization showOnFrontPage="true"/> </author> <author initials="G." surname="Fairhurst" fullname="G. Fairhurst" role="editor"> <organization showOnFrontPage="true"/> </author> <date year="2015" month="July"/> <abstract> <t>This memo presents recommendations to the Internet community concerning measures to improve and preserve Internet performance. It presents a strong recommendation for testing, standardization, and widespread deployment of active queue management (AQM) in network devices to improve the performance of today's Internet. It also urges a concerted effort of research, measurement, and ultimate deployment of AQM mechanisms to protect the Internet from flows that are not sufficiently responsive to congestion notification.</t> <t>Based on 15 years of experience and new research, this document replaces the recommendations of RFC 2309.</t> </abstract> </front> <seriesInfo name="BCP" value="197"/> <seriesInfo name="RFC" value="7567"/> <seriesInfo name="DOI" value="10.17487/RFC7567"/> </reference> <reference anchor="RFC7813" target="https://www.rfc-editor.org/info/rfc7813" quoteTitle="true" derivedAnchor="RFC7813"> <front> <title>IS-IS Path Control and Reservation</title> <author initials="J." surname="Farkas" fullname="J. Farkas" role="editor"> <organization showOnFrontPage="true"/> </author> <author initials="N." surname="Bragg" fullname="N. Bragg"> <organization showOnFrontPage="true"/> </author> <author initials="P." surname="Unbehagen" fullname="P. Unbehagen"> <organization showOnFrontPage="true"/> </author> <author initials="G." surname="Parsons" fullname="G. Parsons"> <organization showOnFrontPage="true"/> </author> <author initials="P." surname="Ashwood-Smith" fullname="P. Ashwood-Smith"> <organization showOnFrontPage="true"/> </author> <author initials="C." surname="Bowers" fullname="C. Bowers"> <organization showOnFrontPage="true"/> </author> <date year="2016" month="June"/> <abstract> <t>IEEE 802.1Qca Path Control and Reservation (PCR) specifies explicit path control via IS-IS in Layer 2 networks in order to move beyond the shortest path capabilities provided by IEEE 802.1aq Shortest Path Bridging (SPB). IS-IS PCR provides capabilities for the establishment and control of explicit forwarding trees in a Layer 2 network domain. This document specifies the sub-TLVs for IS-IS PCR.</t> </abstract> </front> <seriesInfo name="RFC" value="7813"/> <seriesInfo name="DOI" value="10.17487/RFC7813"/> </reference> <reference anchor="RFC8033" target="https://www.rfc-editor.org/info/rfc8033" quoteTitle="true" derivedAnchor="RFC8033"> <front> <title>Proportional Integral Controller Enhanced (PIE): A Lightweight Control Scheme to Address the Bufferbloat Problem</title> <author initials="R." surname="Pan" fullname="R. Pan"> <organization showOnFrontPage="true"/> </author> <author initials="P." surname="Natarajan" fullname="P. Natarajan"> <organization showOnFrontPage="true"/> </author> <author initials="F." surname="Baker" fullname="F. Baker"> <organization showOnFrontPage="true"/> </author> <author initials="G." surname="White" fullname="G. White"> <organization showOnFrontPage="true"/> </author> <date year="2017" month="February"/> <abstract> <t>Bufferbloat is a phenomenon in which excess buffers in the network cause high latency and latency variation. As more and more interactive applications (e.g., voice over IP, real-time video streaming, and financial transactions) run in the Internet, high latency and latency variation degrade application performance. There is a pressing need to design intelligent queue management schemes that can control latency and latency variation, and hence provide desirable quality of service to users.</t> <t>This document presents a lightweight active queue management design called "PIE" (Proportional Integral controller Enhanced) that can effectively control the average queuing latency to a target value. Simulation results, theoretical analysis, and Linux testbed results have shown that PIE can ensure low latency and achieve high link utilization under various congestion situations. The design does not require per-packet timestamps, so it incurs very little overhead and is simple enough to implement in both hardware and software.</t> </abstract> </front> <seriesInfo name="RFC" value="8033"/> <seriesInfo name="DOI" value="10.17487/RFC8033"/> </reference> <reference anchor="RFC8227" target="https://www.rfc-editor.org/info/rfc8227" quoteTitle="true" derivedAnchor="RFC8227"> <front> <title>MPLS-TP Shared-Ring Protection (MSRP) Mechanism for Ring Topology</title> <author initials="W." surname="Cheng" fullname="W. Cheng"> <organization showOnFrontPage="true"/> </author> <author initials="L." surname="Wang" fullname="L. Wang"> <organization showOnFrontPage="true"/> </author> <author initials="H." surname="Li" fullname="H. Li"> <organization showOnFrontPage="true"/> </author> <author initials="H." surname="van Helvoort" fullname="H. van Helvoort"> <organization showOnFrontPage="true"/> </author> <author initials="J." surname="Dong" fullname="J. Dong"> <organization showOnFrontPage="true"/> </author> <date year="2017" month="August"/> <abstract> <t>This document describes requirements, architecture, and solutions for MPLS-TP Shared-Ring Protection (MSRP) in a ring topology for point- to-point (P2P) services. The MSRP mechanism is described to meet the ring protection requirements as described in RFC 5654. This document defines the Ring Protection Switching (RPS) protocol that is used to coordinate the protection behavior of the nodes on an MPLS ring.</t> </abstract> </front> <seriesInfo name="RFC" value="8227"/> <seriesInfo name="DOI" value="10.17487/RFC8227"/> </reference> <reference anchor="RFC8289" target="https://www.rfc-editor.org/info/rfc8289" quoteTitle="true" derivedAnchor="RFC8289"> <front> <title>Controlled Delay Active Queue Management</title> <author initials="K." surname="Nichols" fullname="K. Nichols"> <organization showOnFrontPage="true"/> </author> <author initials="V." surname="Jacobson" fullname="V. Jacobson"> <organization showOnFrontPage="true"/> </author> <author initials="A." surname="McGregor" fullname="A. McGregor" role="editor"> <organization showOnFrontPage="true"/> </author> <author initials="J." surname="Iyengar" fullname="J. Iyengar" role="editor"> <organization showOnFrontPage="true"/> </author> <date year="2018" month="January"/> <abstract> <t>This document describes CoDel (Controlled Delay) -- a general framework that controls bufferbloat-generated excess delay in modern networking environments. CoDel consists of an estimator, a setpoint, and a control loop. It requires no configuration in normal Internet deployments.</t> </abstract> </front> <seriesInfo name="RFC" value="8289"/> <seriesInfo name="DOI" value="10.17487/RFC8289"/> </reference> <reference anchor="RFC8402" target="https://www.rfc-editor.org/info/rfc8402" quoteTitle="true" derivedAnchor="RFC8402"> <front> <title>Segment Routing Architecture</title> <author initials="C." surname="Filsfils" fullname="C. Filsfils" role="editor"> <organization showOnFrontPage="true"/> </author> <author initials="S." surname="Previdi" fullname="S. Previdi" role="editor"> <organization showOnFrontPage="true"/> </author> <author initials="L." surname="Ginsberg" fullname="L. Ginsberg"> <organization showOnFrontPage="true"/> </author> <author initials="B." surname="Decraene" fullname="B. Decraene"> <organization showOnFrontPage="true"/> </author> <author initials="S." surname="Litkowski" fullname="S. Litkowski"> <organization showOnFrontPage="true"/> </author> <author initials="R." surname="Shakir" fullname="R. Shakir"> <organization showOnFrontPage="true"/> </author> <date year="2018" month="July"/> <abstract> <t>Segment Routing (SR) leverages the source routing paradigm. A node steers a packet through an ordered list of instructions, called "segments". A segment can represent any instruction, topological or service based. A segment can have a semantic local to an SR node or global within an SR domain. SR provides a mechanism that allows a flow to be restricted to a specific topological path, while maintaining per-flow state only at the ingress node(s) to the SR domain.</t> <t>SR can be directly applied to the MPLS architecture with no change to the forwarding plane. A segment is encoded as an MPLS label. An ordered list of segments is encoded as a stack of labels. The segment to process is on the top of the stack. Upon completion of a segment, the related label is popped from the stack.</t> <t>SR can be applied to the IPv6 architecture, with a new type of routing header. A segment is encoded as an IPv6 address. An ordered list of segments is encoded as an ordered list of IPv6 addresses in the routing header. The active segment is indicated by the Destination Address (DA) of the packet. The next active segment is indicated by a pointer in the new routing header.</t> </abstract> </front> <seriesInfo name="RFC" value="8402"/> <seriesInfo name="DOI" value="10.17487/RFC8402"/> </reference> <referenceanchor="IEEE802.1Qch" target="https://ieeexplore.ieee.org/document/7961303/">anchor="RFC8453" target="https://www.rfc-editor.org/info/rfc8453" quoteTitle="true" derivedAnchor="RFC8453"> <front><title>IEEE Std 802.1Qch-2017 Bridges<title>Framework for Abstraction andBridgedControl of TE Networks- Amendment 29: Cyclic Queuing and Forwarding</title> <author> <organization>IEEE Standards Association</organization>(ACTN)</title> <author initials="D." surname="Ceccarelli" fullname="D. Ceccarelli" role="editor"> <organization showOnFrontPage="true"/> </author><date year="2017" /> </front> </reference> <reference anchor="IEEE802.3-2018" target="https://ieeexplore.ieee.org/document/8457469"> <front> <title>IEEE Std 802.3-2018 Standard for Ethernet</title> <author> <organization>IEEE Standards Association</organization><author initials="Y." surname="Lee" fullname="Y. Lee" role="editor"> <organization showOnFrontPage="true"/> </author> <date year="2018"/>month="August"/> <abstract> <t>Traffic Engineered (TE) networks have a variety of mechanisms to facilitate the separation of the data plane and control plane. They also have a range of management and provisioning protocols to configure and activate network resources. These mechanisms represent key technologies for enabling flexible and dynamic networking. The term "Traffic Engineered network" refers to a network that uses any connection-oriented technology under the control of a distributed or centralized control plane to support dynamic provisioning of end-to- end connectivity.</t> <t>Abstraction of network resources is a technique that can be applied to a single network domain or across multiple domains to create a single virtualized network that is under the control of a network operator or the customer of the operator that actually owns the network resources.</t> <t>This document provides a framework for Abstraction and Control of TE Networks (ACTN) to support virtual network services and connectivity services.</t> </abstract> </front> <seriesInfo name="RFC" value="8453"/> <seriesInfo name="DOI" value="10.17487/RFC8453"/> </reference> <referenceanchor="IEEE802.3br" target="http://ieeexplore.ieee.org/document/7900321/">anchor="RFC8557" target="https://www.rfc-editor.org/info/rfc8557" quoteTitle="true" derivedAnchor="RFC8557"> <front><title>IEEE Std 802.3br-2016 Standard for Ethernet Amendment 5: Specification and Management Parameters for Interspersing Express Traffic</title> <author> <organization>IEEE Standards Association</organization><title>Deterministic Networking Problem Statement</title> <author initials="N." surname="Finn" fullname="N. Finn"> <organization showOnFrontPage="true"/> </author> <author initials="P." surname="Thubert" fullname="P. Thubert"> <organization showOnFrontPage="true"/> </author> <dateyear="2016" />year="2019" month="May"/> <abstract> <t>This paper documents the needs in various industries to establish multi-hop paths for characterized flows with deterministic properties.</t> </abstract> </front> <seriesInfo name="RFC" value="8557"/> <seriesInfo name="DOI" value="10.17487/RFC8557"/> </reference> <referenceanchor="IEEE802.1TSNTG" target="http://www.ieee802.org/1/tsn">anchor="RFC8578" target="https://www.rfc-editor.org/info/rfc8578" quoteTitle="true" derivedAnchor="RFC8578"> <front><title>IEEE 802.1 Time-Sensitive<title>Deterministic Networking Use Cases</title> <author initials="E." surname="Grossman" fullname="E. Grossman" role="editor"> <organization showOnFrontPage="true"/> </author> <date year="2019" month="May"/> <abstract> <t>This document presents use cases for diverse industries that have in common a need for "deterministic flows". "Deterministic" in this context means that such flows provide guaranteed bandwidth, bounded latency, and other properties germane to the transport of time-sensitive data. These use cases differ notably in their network topologies and specific desired behavior, providing as a group broad industry context for Deterministic NetworkingTask Group</title> <author> <organization>IEEE Standards Association</organization> </author> <date></date>(DetNet). For each use case, this document will identify the use case, identify representative solutions used today, and describe potential improvements that DetNet can enable.</t> </abstract> </front> <seriesInfo name="RFC" value="8578"/> <seriesInfo name="DOI" value="10.17487/RFC8578"/> </reference> <reference anchor="TEAS"target="https://datatracker.ietf.org/doc/charter-ietf-teas/">target="https://datatracker.ietf.org/doc/charter-ietf-teas/" quoteTitle="true" derivedAnchor="TEAS"> <front> <title>Traffic Engineering Architecture and SignalingWorking Group</title> <author> <organization>IETF</organization> </author> <date></date> </front> </reference> <reference anchor="CCAMP" target="https://datatracker.ietf.org/doc/charter-ietf-ccamp/"> <front> <title>Common Control and Measurement Plane Working Group</title>(teas)</title> <author><organization>IETF</organization><organization showOnFrontPage="true">IETF</organization> </author><date></date><date/> </front> </reference> <referenceanchor="BUFFERBLOAT">anchor="I-D.ietf-6tisch-architecture" quoteTitle="true" target="https://tools.ietf.org/html/draft-ietf-6tisch-architecture-26" derivedAnchor="TSCH-ARCH"> <front><title>Bufferbloat: Dark Buffers in<title>An Architecture for IPv6 over theInternet</title> <author fullname="J. Gettys" initials="J." surname="Gettys"></author>TSCH mode of IEEE 802.15.4</title> <authorfullname="K. Nichols" initials="K." surname="Nichols"></author>initials="P" surname="Thubert" fullname="Pascal Thubert"> <organization showOnFrontPage="true"/> </author> <datemonth="January" year="2012" />month="August" day="27" year="2019"/> <abstract> <t>This document describes a network architecture that provides low- latency, low-jitter and high-reliability packet delivery. It combines a high-speed powered backbone and subnetworks using IEEE 802.15.4 time-slotted channel hopping (TSCH) to meet the requirements of LowPower wireless deterministic applications.</t> </abstract> </front> <seriesInfo name="Internet-Draft" value="draft-ietf-6tisch-architecture-26"/> <formatoctets="Communications of the ACM, Volume 55 Issue 1, Pages 57-65 " target="https://dl.acm.org/citation.cfm?id=2063176.2063196" type="PDF" />type="TXT" target="http://www.ietf.org/internet-drafts/draft-ietf-6tisch-architecture-26.txt"/> <refcontent>Work in Progress</refcontent> </reference> </references> <section numbered="false" toc="include" removeInRFC="false" pn="section-appendix.a"> <name slugifiedName="name-acknowledgements">Acknowledgements</name> <t pn="section-appendix.a-1">The authors wish to thank Lou Berger, David Black, Stewart Bryant, Rodney Cummings, Ethan Grossman, Craig Gunther, Marcel Kiessling, Rudy Klecka, Jouni Korhonen, Erik Nordmark, Shitanshu Shah, Wilfried Steiner, George Swallow, Michael Johas Teener, Pat Thaler, Thomas Watteyne, Patrick Wetterwald, Karl Weber, and Anca Zamfir for their various contributions to this work. </t> </section> <section anchor="authors-addresses" numbered="false" removeInRFC="false" toc="include" pn="section-appendix.b"> <name slugifiedName="name-authors-addresses">Authors' Addresses</name> <author initials="N" surname="Finn" fullname="Norman Finn"> <organization showOnFrontPage="true">Huawei</organization> <address> <postal> <street>3101 Rio Way</street> <city>Spring Valley</city> <region>California</region> <code>91977</code> <country>United States of America</country> </postal> <phone>+1 925 980 6430</phone> <email>nfinn@nfinnconsulting.com</email> </address> </author> <author initials="P" surname="Thubert" fullname="Pascal Thubert"> <organization abbrev="Cisco" showOnFrontPage="true"> Cisco Systems </organization> <address> <postal> <street>Village d'Entreprises Green Side</street> <street>400, Avenue de Roumanille</street> <street> Batiment T3</street> <city>Biot - Sophia Antipolis</city> <code>06410</code> <country>France</country> </postal> <phone>+33 4 97 23 26 34</phone> <email>pthubert@cisco.com</email> </address> </author> <author fullname="Balázs Varga" initials="B." surname="Varga"> <organization showOnFrontPage="true">Ericsson</organization> <address> <postal> <street>Magyar tudosok korutja 11</street> <city>Budapest</city> <country>Hungary</country> <code>1117</code> </postal> <email>balazs.a.varga@ericsson.com</email> </address> </author> <author fullname="János Farkas" initials="J." surname="Farkas"> <organization showOnFrontPage="true">Ericsson</organization> <address> <postal> <street>Magyar tudosok korutja 11</street> <city>Budapest</city> <country>Hungary</country> <code>1117</code> </postal> <email>janos.farkas@ericsson.com</email> </address> </author> </section> </back> </rfc>