PCE Working GroupInternet Engineering Task Force (IETF) D. DhodyInternet-DraftRequest for Comments: 8637 Huawei Technologies Category: Informational Y. LeeIntended status: Informational HuaweiISSN: 2070-1721 Futurewei TechnologiesExpires: November 17, 2019D. Ceccarelli EricssonMay 16,July 2019 Applicability of the Path Computation Element (PCE) to the Abstraction and Control of TE Networks (ACTN)draft-ietf-pce-applicability-actn-12Abstract Abstraction and Control of TE Networks (ACTN) refers to the set of virtual network (VN) operations needed to orchestrate,controlcontrol, and manage large-scalemulti-domainmultidomain TE networks so as to facilitate network programmability, automation, efficient resource sharing, and end-to-end virtualservice awareservice-aware connectivity and network function virtualization services. The Path Computation Element (PCE) is a component, application, or network node that is capable of computing a network path or route based on a network graph and applying computational constraints. The PCE serves requests from Path Computation Clients (PCCs) that communicate with it over a local API or using the Path Computation Element Communication Protocol (PCEP). This document examines the applicability of PCE to the ACTN framework. Status of This Memo ThisInternet-Draftdocument issubmitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documentsnot an Internet Standards Track specification; it is published for informational purposes. This document is a product of the Internet Engineering Task Force (IETF).Note that other groups may also distribute working documents as Internet-Drafts. The listIt represents the consensus ofcurrent Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draftthe IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Not all documentsvalidapproved by the IESG are candidates fora maximumany level of Internet Standard; see Section 2 of RFC 7841. Information about the current status ofsix monthsthis document, any errata, and how to provide feedback on it may beupdated, replaced, or obsoleted by other documentsobtained atany time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on November 17, 2019.https://www.rfc-editor.org/info/rfc8637. Copyright Notice Copyright (c) 2019 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 2. Background Information . . . . . . . . . . . . . . . . . . . 3 2.1. Path Computation Element (PCE) . . . . . . . . . . . . . 3 2.1.1. Role of PCE in SDN . . . . . . . . . . . . . . . . . 4 2.1.2. PCE inMulti-domainMultidomain andMulti-layerMultilayer Deployments . . . . 4 2.1.3. Relationship toPCE BasedPCE-Based Central Control . . . . . . 5 2.2. Abstraction and Control of TE Networks (ACTN) . . . . . . 5 3. Architectural Considerations . . . . . . . . . . . . . . . . 7 3.1.Multi-DomainMultidomain Coordination via Hierarchy . . . . . . . . . 7 3.2. Abstraction . . . . . . . . . . . . . . . . . . . . . . . 8 3.3. Customer Mapping . . . . . . . . . . . . . . . . . . . . 9 3.4. Virtual Service Coordination . . . . . . . . . . . . . . 10 4. Interface Considerations . . . . . . . . . . . . . . . . . . 10 5. Realizing ACTN with PCE (and PCEP) . . . . . . . . . . . . . 11 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . .1514 7. Security Considerations . . . . . . . . . . . . . . . . . . .1514 8.AcknowledgmentsReferences . . . . . . . . . . . . . . . . . . . . . . .16 9.. . 15 8.1. Normative References . . . . . . . . . . . . . . . . . . 16 8.2. Informative References . . . . . . .16 9.1. Normative References. . . . . . . . . . 16 Appendix A. Additional Information . . . . . . . .16 9.2. Informative References. . . . . . . 20 Acknowledgments . . . . . . . . . .17 Appendix A. Additional Information. . . . . . . . . . . . . . .2120 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 21 1. Introduction Abstraction and Control of TE Networks (ACTN) [RFC8453] refers to the set of virtual network (VN) operations needed to orchestrate,controlcontrol, and manage large-scalemulti-domainmultidomain TE networks so as to facilitate network programmability, automation, efficient resource sharing, and end-to-end virtualservice awareservice-aware connectivity and network function virtualization services. The Path Computation Element (PCE) [RFC4655] is a component, application, or network node that is capable of computing a network path or route based on a network graph and applying computational constraints. The PCE serves requests from Path Computation Clients (PCCs) that communicate with it over a local API or using the Path Computation Element Communication Protocol (PCEP). This document examines the PCE and ACTN architecture and describes how PCE architecture is applicable to ACTN. It also lists the PCEP extensions that are needed to use PCEP as an ACTN interface. This document also identifies any gaps inPCEP,PCEP that exist at the time of publication of this document. Further, ACTN, statefulH-PCE [I-D.ietf-pce-stateful-hpce],Hierarchical PCE (H-PCE) [PCE-HPCE], and PCE as a central controller (PCECC) [RFC8283] are based on the same basic hierarchy framework and are thus compatible with each other. 2. Background Information 2.1. Path Computation Element (PCE) The Path Computation Element Communication Protocol (PCEP) [RFC5440] provides mechanisms for Path Computation Clients (PCCs) to request a Path Computation Element (PCE) [RFC4655] to perform path computations. The ability to compute shortest constrained TE LSPs in Multiprotocol Label Switching (MPLS) and Generalized MPLS (GMPLS) networks across multiple domains has been identified as a key motivation for PCE development. A stateful PCE [RFC8231] is capable of considering, for the purposes of path computation, not only the network state in terms of links and nodes (referred to as the Traffic Engineering Database orTED)TED), but also the status of active services (previously computed paths), and currently reserved resources, stored in the Label Switched Paths Database (LSP-DB). [RFC8051] describes general considerations for a stateful PCE deployment and examines its applicability andbenefits,benefits as well as its challenges and limitations through a number of use cases. [RFC8231] describes a set of extensions to PCEP to provide stateful control. A stateful PCE has access to not only the information carried by the network's Interior Gateway Protocol (IGP), but also the set of active paths and their reserved resources for its computations. The additional state allows the PCE to compute constrained paths while considering individual LSPs and their interactions. [RFC8281] describes the setup,maintenancemaintenance, and teardown of PCE-initiated LSPs under the stateful PCE model. [RFC8231] also describes the active stateful PCE. The active PCE functionality allows a PCE to reroute an existing LSP or make changes to the attributes of an existing LSP, or a PCC to delegate control of specific LSPs to a new PCE. 2.1.1. Role of PCE in SDN Software-Defined Networking (SDN) [RFC7149] refers to a separation between the control elements and the forwarding components so that software running in a centralizedsystemsystem, called a controller, can act to program the devices in the network to behave in specific ways. A required element in an SDN architecture is a component that plans how the network resources will be used and how the devices will be programmed. It is possible to view this component as performing specific computations to place flows within the network given knowledge of the availability of network resources, how other forwarding devices are programmed, and the way that other flows are routed. It is concluded in[RFC7399],[RFC7399] that this is the same function that a PCE might offer in a network operated using a dynamic control plane. This is the function and purpose of a PCE, and the way that a PCE integrates into a wider network control system including SDN is presented in Application-Based Network Operation (ABNO) [RFC7491]. 2.1.2. PCE inMulti-domainMultidomain andMulti-layerMultilayer Deployments Computing paths across largemulti-domainmultidomain environments requires special computational components and cooperation between entities in different domains capable of complex path computation. The PCE provides an architecture and a set of functional components to address this problem space. A PCE may be used to compute end-to-end paths acrossmulti-domainmultidomain environments using a per-domain path computation technique [RFC5152]. TheBackward Recursive PCE basedBackward-Recursive PCE-based path computation (BRPC) mechanism [RFC5441] defines a PCE-based path computation procedure to computeinter-domain constrainedinterdomain-constrained MPLS and GMPLS TE networks. However, per-domain technique assumes that the sequence of domains to be crossed from source to destination is known, either fixed by the network operator or obtained by other means. BRPC can work best with a known domain sequence, and it will also work nicely with a small set of interconnected domains. However, it doesn't work well forisa large set of interconnected domains. [RFC6805] describes a Hierarchical PCE (H-PCE) architecturewhichthat can be used for computing end-to-end paths forinter-domaininterdomain MPLS Traffic Engineering (TE) and GMPLS Label Switched Paths (LSPs) when the domain sequence is not known. Within the Hierarchical PCE (H-PCE) architecture, the Parent PCE (P-PCE) is used to compute amulti- domainmultidomain path based on the domain connectivity information. A Child PCE (C-PCE) may be responsible for a single domain or multipledomains,domains; it is used to compute theintra-domainintradomain path based on its domain topology information.[I-D.ietf-pce-stateful-hpce] state[PCE-HPCE] states the considerations for stateful PCEs inhierarchicalHierarchical PCE architecture. In particular, the behavior changes andadditionsadds to the existing stateful PCE mechanisms (including PCE- initiated LSP setup and active PCE usage) in the context of networks using the H-PCE architecture. [RFC5623] describes a framework for applying the PCE-based architecture tointer-layer tointerlayer (G)MPLS TE. It provides suggestions for the deployment of PCE in support ofmulti-layermultilayer networks. It also describes the relationship between PCE and a functional component in charge of the control and management of the Virtual Network Topology (VNT)[RFC5212],[RFC5212] called the VNT Manager (VNTM). 2.1.3. Relationship toPCE BasedPCE-Based Central Control [RFC8283] introduces the architecture for PCE as a central controller(PCECC),(PCECC); it further examines the motivations and applicability for PCEP as a southboundinterface,interface (SBI) and introduces the implications for the protocol. Section 2.1.3 of [RFC8283]describedescribes a hierarchy of PCE-basedcontrollercontrollers as per theHierarchy ofPCEframeworkHierarchy Framework defined in [RFC6805]. 2.2. Abstraction and Control of TE Networks (ACTN) [RFC8453] describes the high-level ACTN requirements and the architecture model forACTNACTN, including the entities Customer Network Controller (CNC),Multi-domainMultidomain Service Coordinator (MDSC), and Provisioning Network Controller (PNC) and their interfaces. The ACTN reference architecture is shown in Figure11, which is reproduced here from [RFC8453] for convenience. [RFC8453] remains the definitive reference for the ACTN architecture. As depicted in Figure 1, the ACTN architecture identifies a three-tier hierarchy. +---------+ +---------+ +---------+ | CNC | | CNC | | CNC | +---------+ +---------+ +---------+ \ | / \ | / Boundary =============\==============|==============/============ Between \ | / Customer & ------- | CMI ------- Network Operator \ | / +---------------+ | MDSC | +---------------+ / | \ ------------ | MPI ------------- / | \ +-------+ +-------+ +-------+ | PNC | | PNC | | PNC | +-------+ +-------+ +-------+ | SBI / | / \ | / | SBI / \ --------- ----- | / \ ( ) ( ) | / \ - Control - ( Phys. ) | / ----- ( Plane ) ( Net ) | / ( ) ( Physical ) ----- | / ( Phys. ) ( Network ) ----- ----- ( Net ) - - ( ) ( ) ----- ( ) ( Phys. ) ( Phys. ) --------- ( Net ) ( Net ) ----- ----- CMI - (CNC-MDSC Interface) MPI - (MDSC-PNC Interface) SBI - (Southbound Interface) Figure 1: ACTN Hierarchy There are two interfaces with respect to the MDSC: one north of the MDSC (the CNC-MDSC Interface: CMI),(CMI)), and one south (the MDSC-PNC Interface: MPI).(MPI)). A hierarchy of MDSCs is possible with a recursive MPI interface. [RFC8454] provides an information model for ACTN interfaces. 3. Architectural Considerations The ACTN architecture [RFC8453] is based on the hierarchy and recursiveness of controllers. It defines three types of controllers (depending on the functionalities they implement). The main functionalitiesare -are: oMulti-domainMultidomain coordination o Abstraction o Customer mapping/translation o Virtual service coordination Section 3 of [RFC8453] describes these functions. It should be noted that this document lists all possible ways in which PCE could be used for each of the above functions, but all functions are not required to be implemented via PCE. Similarly, this document presents the ways in which PCEP could be used as the communications medium between functional components. Operators may choose to use the PCEP formulti-domainmultidomain coordination via statefulH-PCE,H-PCE but alternatively use Network Configuration Protocol (NETCONF) [RFC6241], RESTCONF [RFC8040], or BGP - Link State (BGP-LS) [RFC7752] to get access to the topology and support abstraction function. 3.1.Multi-DomainMultidomain Coordination via Hierarchy With the definition of domain being"everythingeverything that is under the control of the single logicalcontroller",controller, as per [RFC8453], it is needed both to have a control entity that oversees the specific aspects of the different domains and to build a single abstracted end-to-end network topology in order to coordinate end-to-end path computation and path/service provisioning. The MDSC in ACTN framework realizes this function by coordinating the per-domain PNCs in a hierarchy of controllers. It also needs to detach from the underlying network technology and express customer concerns by business needs. [RFC6805] and[I-D.ietf-pce-stateful-hpce][PCE-HPCE] describe a hierarchy of PCEs with the Parent PCE coordinatingmulti-domainmultidomain path computation function between Child PCEs. It is easy to see how these principles align, and thus how the stateful H-PCE architecture can be used to realize ACTN. Theper domainper-domain stitched LSP in the Hierarchical stateful PCE architecture, described in Section 3.3.1 of[I-D.ietf-pce-stateful-hpce][PCE-HPCE], is well suited formulti-domainmultidomain coordination function. This includes domain sequenceselection; End- to-Endselection, End-to-End (E2E) pathcomputation; Controller (PCE) initiatedcomputation, and controller-initiated (PCE-initiated) path setup and reporting. This is also applicable tomulti-layermultilayer coordination in case of IP+optical networks.[I-D.litkowski-pce-state-sync][PCE-STATE-SYNC] describes the procedures to allow a stateful communication between PCEs for varioususe-cases.use cases. The procedures and extensions are also applicable to Child and Parent PCE communication and are thus useful for ACTN as well. 3.2. Abstraction To realize ACTN, an abstracted view of the underlying network resources needs to be built. This includes global network-wide abstracted topology based on the underlying network resources of each domain. This also includes abstract topology created as per the customer service connectivity requests and represented as a VN slice allocated to each customer. In order to compute and provide optimal paths, PCEs require an accurate and timely Traffic Engineering Database (TED).TraditionallyTraditionally, this TED has been obtained from alink statelink-state (LS) routing protocol supporting traffic engineering extensions. PCE may construct its TED by participating in the IGP ([RFC3630] and [RFC5305] for MPLS-TE; [RFC4203] and [RFC5307] for GMPLS). An alternative is offered by BGP-LS [RFC7752]. In case of H-PCE [RFC6805], the Parent PCE needs to build the domain topology map of the child domains and their interconnectivity. [RFC6805] and[I-D.ietf-pce-inter-area-as-applicability][PCE-INTER-AREA] suggest that BGP-LS could be used as a "northbound" TE advertisement from the Child PCE to the Parent PCE.[I-D.dhodylee-pce-pcep-ls][PCEP-LS] proposes another approach for learning and maintaining the Link-State and TE information as an alternative to IGPs and BGP flooding, using PCEP itself. The Child PCE can use this mechanism to transport Link-State and TE information from Child PCE to a Parent PCE using PCEP. In ACTN, there is a need to control the level of abstraction based on the deployment scenario and business relationship between the controllers. The mechanism used to disseminate information from the PNC (Child PCE) to the MDSC (Parent PCE) should support abstraction. [RFC8453] describes a few alternative approaches of abstraction. The resulting abstracted topology can be encoded using the PCEP-LS mechanisms[I-D.dhodylee-pce-pcep-ls][PCEP-LS] and its optical network extension[I-D.lee-pce-pcep-ls-optical].[PCEP-OPTICAL]. PCEP-LS is an attractive option when the operator would wish to have a singlecontrol planecontrol-plane protocol (PCEP) to achieve ACTN functions. [RFC8453] discusses two ways to build abstract topology from an MDSC standpoint with interaction with PNCs. The primary method is called automatic generation of abstract topology by configuration. With this method, automatic generation is based on the abstraction/ summarization of the whole domain by the PNC and its advertisement on the MPI. The secondary method is called on-demand generation of supplementary topology via Path Compute Request/Reply. This method may be needed to obtain further complementary information such as potential connectivity from Child PCEs in order to facilitate an end- to-end path provisioning. PCEP is well suited to support both methods. 3.3. Customer Mapping In ACTN, there is a need to map customer virtual network (VN) requirements into a network provisioning request to the PNC. That is, the customer requests/commands are mapped by the MDSC into network provisioning requests that can be sent to the PNC. Specifically, the MDSC provides mapping and translation of a customer's service request into a set of parameters that are specific to a network type and technology such that the network configuration process is made possible. [RFC8281] describes the setup,maintenancemaintenance, and teardown of PCE- initiated LSPs under the stateful PCE model, without the need for local configuration on the PCC, thus allowing for a dynamic network that is centrally controlled and deployed. To instantiate or delete an LSP, the PCE sends the Path Computation LSP Initiate Request (PCInitiate) message to the PCC. As described in[I-D.ietf-pce-stateful-hpce],[PCE-HPCE], forinter-domaininterdomain LSP in Hierarchical PCE architecture, the initiation operations can be carried out at the Parent PCE. Inwhichthis case, after Parent PCE finishes the E2E path computation, it can send the PCInitiate message to the ChildPCE,PCE; the Child PCE further propagates the initiate request to the Label Switching Router (LSR). The customer request is received by the MDSC (ParentPCE) andPCE), and, based on the business logic, global abstracted topology, networkconditionsconditions, and local policy, the MDSC (Parent PCE) translates this intopera per- domain LSP initiation request that a PNC (Child PCE) can understand and act on. This can be done via the PCInitiate message. PCEP extensions for associating opaque policy between PCEP peer[I-D.ietf-pce-association-policy][ASSOC-POLICY] can be used. 3.4. Virtual Service Coordination Virtual service coordination function in ACTN incorporates customer service-related information into the virtual network service operations in order to seamlessly operate virtual networks while meetingcustomer'scustomers' service requirements.[I-D.leedhody-pce-vn-association][PCEP-VN] describes the need for associating a set of LSPs with a VN "construct" to facilitate VN operations in PCE architecture. This association allows the PCEs to identify which LSPs belong to a certain VN. This association based on VN is useful for various optimizations at the VNlevellevel, which can be applied to all the LSPs that are part of the VN slice. During path computation, the impact of a path for an LSP is compared against the paths of other LSPs in the VN. This is tomake sure that the overallensure optimization and SLAofattainment for the VN rather thanoffor a single LSP. Similarly, duringre-optimization,reoptimization, advanced path computationalgorithmalgorithms and optimizationtechniquetechniques can be considered for all the LSPs belonging to a VN/customer and optimize them all together. 4. Interface Considerations As per [RFC8453], to allow virtualization andmulti-domainmultidomain coordination, the network has to provide open, programmableinterfaces,interfaces in which customer applications can create,replacereplace, and modify virtual network resources and services in an interactive,flexibleflexible, and dynamic fashion while having no impact on other customers. The two ACTN interfaces are-as follows: o The CNC-MDSC Interface (CMI) is an interface between a Customer Network Controller and aMulti-DomainMultidomain Service Coordinator. It requests the creation of the network resources,topologytopology, or services for the applications. The MDSC may also report potential network topology availability if queried for current capability from the Customer Network Controller. o The MDSC-PNC Interface (MPI) is an interface between aMulti- DomainMultidomain Service Coordinator and a Provisioning Network Controller. It communicates the creation request, if required, of new connectivity of bandwidth changes in the physicalnetwork,network via the PNC. Inmulti-domainmultidomain environments, the MDSC needs to establish multiple MPIs, one for each PNC, as there are multiple PNCs responsible for its domain control. In the case of a hierarchy of MDSCs, the MPI is applied recursively. From an abstraction point of view, thetop level MDSCtop-level MDSC, which interfaces theCNCCNC, operates on a higher level of abstraction (i.e., less granular level) than thelower level MSDCs.lower-level MDSCs. PCEP is especially suitable on the MPI as it meets the requirement and the functions as set out in the ACTN framework [RFC8453]. Its recursive nature is well suited via themulti-levelmultilevel hierarchy of PCE. PCEP can also be applied to the CMI as the CNC can be a path computation client while the MDSC can be a path computation server. Section 5 describes how PCE and PCEP could help realize ACTN on the MPI. 5. Realizing ACTN with PCE (and PCEP) As per the example in Figure 2, there are 4 domains, each withitstheir own PNC andanMDSC on top. The PNC and MDSC need PCE as an important function. The PNC (or Child PCE) already uses PCEP to communicate to the network device. It can utilize the PCEP as the MPI to communicate between controllers too. ****** ..........*MDSC*.............................. . ****** .. MPI . . . . . . . . . . . . . . . . . . . . . . . . . v v v . ****** ****** ****** . *PNC1* *PNC2* *PNC4* . ****** ****** ****** . +---------------+ +---------------+ +---------------+ . |A |----| |----| C| . | | | | | | . |DOMAIN 1 |----|DOMAIN 2 |----|DOMAIN 4 | . +------------B13+ +---------------+ +B43------------+ . \ / . \ ****** / . \ *PNC3*<............/..................... \ ****** / \+---------------+/ B31 B34 | | |DOMAIN 3 B| +---------------+ MDSC -> Parent PCE PNC -> Child PCE MPI -> PCEP Figure 2: ACTN with PCE o Building Domain Topology at MDSC: PNC (or Child PCE) needs to have the TED to compute the path in its domain. As described in Section 3.2, it can learn the topology via IGP or BGP-LS. PCEP-LS is also a proposed mechanism to carry link state and traffic engineering information within PCEP. A mechanism to carry abstracted topology while hidingtechnology specifictechnology-specific information between PNC and MDSC is described in[I-D.dhodylee-pce-pcep-ls].[PCEP-LS]. At the end of thisstepstep, the MDSC (or Parent PCE) has the abstracted topology from each of itsPNCPNCs (or Child PCE). This could be as simple as a domain topology map as described in[RFC6805][RFC6805], or it can have full topology information of all domains. The latter is notscalablescalable, andthusthus, an abstracted topology of each domain interconnected byinter-domaininterdomain links is the most common case. * Topology Change: When the PNC learns of any topology change, the PNC needs to decide if the change needs to be notified to the MDSC. This is dependent on the level of abstraction between the MDSC and the PNC. o VN Instantiate: When an MDSC is requested to instantiate a VN, the minimal information that is required would be a VN identifier and a set of end points. Various path computation, setupconstraintsconstraints, and objective functions may also be provided. In PCE terms, a VN Instantiate can be considered as a set of paths belonging to the same VN. As described in Section 3.4 and[I-D.leedhody-pce-vn-association][PCEP-VN], the VN association can help in identifying the set of paths that belong to a VN. The rest of theinformationinformation, like the endpoints,constraintsconstraints, and objective function(OF)(OF), is already defined in PCEP in terms of a single path. * Path Computation: As per the example in Figure 2, the VN instantiate requires twoend to endend-to-end paths between (A in Domain 1 to B in Domain 3) and (A in Domain 1 to C in Domain 4). The MDSC (or Parent PCE) triggers theend to endend-to-end path computation for these two paths. MDSC can do path computation based on the abstracted domain topology that it alreadyhashas, or it may use the H-PCE procedures (Section 3.1) using the PCReq and PCRep messages to get theend to endend-to-end path with the help of the Child PCEs (PNC). Either way, the resultant E2E paths may be broken into per-domain paths. * A-B: (A-B13,B13-B31,B31-B) * A-C: (A-B13,B13-B31,B31-B34,B34-B43,B43-C) *Per DomainPer-Domain Path Instantiation: Based on the above path computation, MDSC can issue the path instantiation request to each PNC via PCInitiate message (see[I-D.ietf-pce-stateful-hpce][PCE-HPCE] and[I-D.leedhody-pce-vn-association]).[PCEP-VN]). A suitable stitching mechanism would be used to stitch theseper domainper-domain LSPs. One such mechanism is described in[I-D.dugeon-pce-stateful-interdomain],[PCE-INTERDOMAIN], where PCEP is extended to support stitching in stateful H-PCE context. *Per DomainPer-Domain Path Report: Each PNC should report the status of the per-domain LSP to the MDSC via PCRpt message, as per theHierarchyhierarchy of statefulPCE ([I-D.ietf-pce-stateful-hpce]).PCEs ([PCE-HPCE]). The status of theend to endend-to-end LSP (A-B and A-C) is made up when all theper domain LSPper-domain LSPs are reported up by the PNCs. * Delegation: It is suggested that theper domainper-domain LSPs are delegated to respectivePNC,PNCs so that they can control the path and attributes based on the conditions of each domainnetwork conditions.network. * State Synchronization: The state needs to be synchronized between the Parent PCE and Child PCE. The mechanism described in[I-D.litkowski-pce-state-sync][PCE-STATE-SYNC] can be used. o VN Modify: MDSC is requested to modify a VN, forexampleexample, the bandwidth for VN is increased. This may trigger path computation at MDSC as described in the previous step and can trigger an update to an existingper-intra-domainper-intradomain path (via PCUpd message) or the creation (or deletion) of a per-domain path (via PCInitiate message). As described in[I-D.ietf-pce-stateful-hpce],[PCE-HPCE], this should be done in make-before-break fashion. o VN Delete: MDSC is requested to delete a VN, in this case, based on the E2Epathspaths, and the resulting per-domain paths need to be removed (via PCInitiate message). o VN Update (based on network changes): Any change in the per-domain LSP is reported to the MDSC (via PCRpt message) as per[I-D.ietf-pce-stateful-hpce].[PCE-HPCE]. This may result in changes in the E2E path or VN status. This may also trigger are-optimizationreoptimization leading to a new per-domain path, an update to an existing path, or the deletion of the path. o VN Protection: The VN protection/restorationrequirements,requirements need to be applied to each E2E path as well as eachper domainper-domain path. The MDSC needs to play a crucial role in coordinating the right protection/restoration policy across each PNC. The existing protection/restoration mechanism of PCEP can be applied on each path. o In case a PNC generates an abstract topology towards the MDSC, the PCInitiate/PCUpd messages from the MDSC to a PNC will contain a path with abstract nodes and links. A PNC would need to take that as an input for path computation to get a path with physical nodes and links. Similarly, a PNC would convert the path received from the device (with physical nodes and links) into an abstract path (based on the abstract topology generated before with abstract nodes and links) and report it to the MDSC. 6. IANA Considerations This documentmakeshas norequests forIANAaction.actions. 7. Security Considerations Various security considerations for PCEP are described in [RFC5440] and [RFC8253]. Security considerations as stated inSectionSections 10.1,Section10.6, andSection10.7 of [RFC5440] continue to apply on PCEP when used as an ACTN interface. Further, this document lists various extensions of PCEP that areapplicable,applicable; each of them specify various security considerationswhichthat continue to apply here. The ACTN framework described in [RFC8453] defines key components and interfaces for managedtraffic engineeredtraffic-engineered networks. It also lists various security considerations such as request and control of resources, confidentially of the information, and availability offunctionfunction, which should be taken into consideration. As per [RFC8453], securing the request and control of resources, confidentiality of the information, and availability of function should all be critical security considerations when deploying and operating ACTN platforms. From a security and reliability perspective, ACTN may encounter many risks such as malicious attack and rogue elements attempting to connect to various ACTN components (with PCE being one of them). Furthermore, some ACTN components represent a single point of failure and threatvectorvector, and must also manage policy conflicts and eavesdropping of communication between different ACTN components. [RFC8453] further states that all protocols used to realize the ACTN framework should have rich security features, and customer,applicationapplication, and network data should be stored in encrypted data stores. When PCEP is used as an ACTN interface, the security of PCEP provided by Transport Layer Security (TLS) [RFC8253], as per the recommendations and best current practices in [RFC7525] (unless explicitly set aside in [RFC8253]), is used. As per [RFC8453], regarding the MPI, aPKI- basedPKI-based mechanism is suggested, such as building a TLS or HTTPS connection between the MDSC and PNCs, to ensure trust between the physical network layer control components and the MDSC. Which MDSC the PNC exports topology information to, and the level of detail (full or abstracted), should also be authenticated, and specific access restrictions and topology views should be configurable and/or policy based. When PCEP is used in MPI, the securityfunctionsfunctions, as per[RFC8253][RFC8253], are used to fulfill these requirements. As per [RFC8453], regarding the CMI, suitable authentication and authorization of each CNC connecting to the MDSC will be required. If PCEP is used in CMI, the securityfunctionsfunctions, as per[RFC8253][RFC8253], can be used to support peer authentication, message encryption, and integrity checks. 8.Acknowledgments The authors would like to thank Jonathan Hardwick for the inspiration behind this document. Further thanks to Avantika for her comments with suggested text. Thanks to Adrian Farrel and Daniel King for their substantial reviews. Thanks to Yingzhen Qu for RTGDIR review. Thanks to Rifaat Shekh-Yusef for SECDIR review. Thanks to Meral Shirazipour for GENART review. Thanks to Roman Danyliw and Barry Leiba for IESG review comments. Thanks to Deborah Brungard for being the responsible AD. 9.References9.1.8.1. Normative References [RFC4655] Farrel, A., Vasseur, J., and J. Ash, "A Path Computation Element (PCE)-Based Architecture", RFC 4655, DOI 10.17487/RFC4655, August 2006, <https://www.rfc-editor.org/info/rfc4655>. [RFC5440] Vasseur, JP., Ed. and JL. Le Roux, Ed., "Path Computation Element (PCE) Communication Protocol (PCEP)", RFC 5440, DOI 10.17487/RFC5440, March 2009, <https://www.rfc-editor.org/info/rfc5440>. [RFC6805] King, D., Ed. and A. Farrel, Ed., "The Application of the Path Computation Element Architecture to the Determination of a Sequence of Domains in MPLS and GMPLS", RFC 6805, DOI 10.17487/RFC6805, November 2012, <https://www.rfc-editor.org/info/rfc6805>. [RFC8453] Ceccarelli, D., Ed. and Y. Lee, Ed., "Framework for Abstraction and Control of TE Networks (ACTN)", RFC 8453, DOI 10.17487/RFC8453, August 2018, <https://www.rfc-editor.org/info/rfc8453>.9.2.8.2. Informative References[RFC3630] Katz, D., Kompella, K.,[ASSOC-POLICY] Litkowski, S., Sivabalan, S., Tantsura, J., Hardwick, J., andD. Yeung, "Traffic Engineering (TE) Extensions to OSPF Version 2", RFC 3630, DOI 10.17487/RFC3630, September 2003, <https://www.rfc-editor.org/info/rfc3630>. [RFC4203] Kompella, K., Ed.M. Negi, "Path Computation Element communication Protocol extension for associating Policies andY. Rekhter, Ed., "OSPF ExtensionsLSPs", Work inSupport of Generalized Multi-Protocol Label Switching (GMPLS)", RFC 4203, DOI 10.17487/RFC4203, October 2005, <https://www.rfc-editor.org/info/rfc4203>. [RFC5152] Vasseur, JP., Ed., Ayyangar, A., Ed.,Progress, draft-ietf-pce-association-policy-05, February 2019. [EXP] Casellas, R., Vilalta, R., Martinez, R., Munoz, R., Zheng, H., andR. Zhang, "A Per-Domain PathY. Lee, "Experimental Validation of the ACTN architecture for flexi-grid optical networks using Active Stateful Hierarchical PCEs", 19th International Conference on Transparent Optical Networks (ICTON), DOI 10.5281/zenodo.832903, July 2017, <https://zenodo.org/record/832904>. [PCE-HPCE] Dhody, D., Lee, Y., Ceccarelli, D., Shin, J., and D. King, "Hierarchical Stateful Path Computation Element (PCE).", Work in Progress, draft-ietf-pce-stateful-hpce-10, June 2019. [PCE-INTER-AREA] King, D. and H. Zheng, "Applicability of the Path Computation Element to Interarea and Inter-AS MPLS and GMPLS Traffic Engineering", Work in Progress, draft-ietf- pce-inter-area-as-applicability-07, December 2018. [PCE-INTERDOMAIN] Dugeon, O., Meuric, J., Lee, Y., and D. Ceccarelli, "PCEP Extension for Stateful Interdomain Tunnels", Work in Progress, draft-dugeon-pce-stateful-interdomain-02, March 2019. [PCE-STATE-SYNC] Litkowski, S., Sivabalan, S., Li, C., and H. Zheng, "Inter Stateful Path Computation Element (PCE) Communication Procedures.", Work in Progress, draft-litkowski-pce-state- sync-05, March 2019. [PCEP-LS] Dhody, D., Lee, Y., and D. Ceccarelli, "PCEP Extension for Distribution of Link-State and TE Information.", Work in Progress, draft-dhodylee-pce-pcep-ls-13, February 2019. [PCEP-OPTICAL] Lee, Y., Zheng, H., Ceccarelli, D., Wang, W., Park, P., and B. Yoon, "PCEP Extension for Distribution of Link- State and TE information for Optical Networks", Work in Progress, draft-lee-pce-pcep-ls-optical-07, March 2019. [PCEP-VN] Lee, Y., Zhang, X., and D. Ceccarelli, "Path Computation Element communication Protocol (PCEP) extensions for Establishing Relationships between sets of LSPs and Virtual Networks", Work in Progress, draft-leedhody-pce- vn-association-08, June 2019. [RFC3630] Katz, D., Kompella, K., and D. Yeung, "Traffic Engineering (TE) Extensions to OSPF Version 2", RFC 3630, DOI 10.17487/RFC3630, September 2003, <https://www.rfc-editor.org/info/rfc3630>. [RFC4203] Kompella, K., Ed. and Y. Rekhter, Ed., "OSPF Extensions in Support of Generalized Multi-Protocol Label Switching (GMPLS)", RFC 4203, DOI 10.17487/RFC4203, October 2005, <https://www.rfc-editor.org/info/rfc4203>. [RFC5152] Vasseur, JP., Ed., Ayyangar, A., Ed., and R. Zhang, "A Per-Domain Path Computation Method for Establishing Inter- Domain Traffic Engineering (TE) Label Switched Paths (LSPs)", RFC 5152, DOI 10.17487/RFC5152, February 2008, <https://www.rfc-editor.org/info/rfc5152>. [RFC5212] Shiomoto, K., Papadimitriou, D., Le Roux, JL., Vigoureux, M., and D. Brungard, "Requirements for GMPLS-Based Multi- Region and Multi-Layer Networks (MRN/MLN)", RFC 5212, DOI 10.17487/RFC5212, July 2008, <https://www.rfc-editor.org/info/rfc5212>. [RFC5305] Li, T. and H. Smit, "IS-IS Extensions for Traffic Engineering", RFC 5305, DOI 10.17487/RFC5305, October 2008, <https://www.rfc-editor.org/info/rfc5305>. [RFC5307] Kompella, K., Ed. and Y. Rekhter, Ed., "IS-IS Extensions in Support of Generalized Multi-Protocol Label Switching (GMPLS)", RFC 5307, DOI 10.17487/RFC5307, October 2008, <https://www.rfc-editor.org/info/rfc5307>. [RFC5441] Vasseur, JP., Ed., Zhang, R., Bitar, N., and JL. Le Roux, "A Backward-Recursive PCE-Based Computation (BRPC) Procedure to Compute Shortest Constrained Inter-Domain Traffic Engineering Label Switched Paths", RFC 5441, DOI 10.17487/RFC5441, April 2009, <https://www.rfc-editor.org/info/rfc5441>. [RFC5623] Oki, E., Takeda, T., Le Roux, JL., and A. Farrel, "Framework for PCE-Based Inter-Layer MPLS and GMPLS Traffic Engineering", RFC 5623, DOI 10.17487/RFC5623, September 2009, <https://www.rfc-editor.org/info/rfc5623>. [RFC6241] Enns, R., Ed., Bjorklund, M., Ed., Schoenwaelder, J., Ed., and A. Bierman, Ed., "Network Configuration Protocol (NETCONF)", RFC 6241, DOI 10.17487/RFC6241, June 2011, <https://www.rfc-editor.org/info/rfc6241>. [RFC7149] Boucadair, M. and C. Jacquenet, "Software-Defined Networking: A Perspective from within a Service Provider Environment", RFC 7149, DOI 10.17487/RFC7149, March 2014, <https://www.rfc-editor.org/info/rfc7149>. [RFC7399] Farrel, A. and D. King, "Unanswered Questions in the Path Computation Element Architecture", RFC 7399, DOI 10.17487/RFC7399, October 2014, <https://www.rfc-editor.org/info/rfc7399>. [RFC7491] King, D. and A. Farrel, "A PCE-Based Architecture for Application-Based Network Operations", RFC 7491, DOI 10.17487/RFC7491, March 2015, <https://www.rfc-editor.org/info/rfc7491>. [RFC7525] Sheffer, Y., Holz, R., and P. Saint-Andre, "Recommendations for Secure Use of Transport Layer Security (TLS) and Datagram Transport Layer Security (DTLS)", BCP 195, RFC 7525, DOI 10.17487/RFC7525, May 2015, <https://www.rfc-editor.org/info/rfc7525>. [RFC7752] Gredler, H., Ed., Medved, J., Previdi, S., Farrel, A., and S. Ray, "North-Bound Distribution of Link-State and Traffic Engineering (TE) Information Using BGP", RFC 7752, DOI 10.17487/RFC7752, March 2016, <https://www.rfc-editor.org/info/rfc7752>. [RFC8040] Bierman, A., Bjorklund, M., and K. Watsen, "RESTCONF Protocol", RFC 8040, DOI 10.17487/RFC8040, January 2017, <https://www.rfc-editor.org/info/rfc8040>. [RFC8051] Zhang, X., Ed. and I. Minei, Ed., "Applicability of a Stateful Path Computation Element (PCE)", RFC 8051, DOI 10.17487/RFC8051, January 2017, <https://www.rfc-editor.org/info/rfc8051>. [RFC8231] Crabbe, E., Minei, I., Medved, J., and R. Varga, "Path Computation Element Communication Protocol (PCEP) Extensions for Stateful PCE", RFC 8231, DOI 10.17487/RFC8231, September 2017, <https://www.rfc-editor.org/info/rfc8231>. [RFC8253] Lopez, D., Gonzalez de Dios, O., Wu, Q., and D. Dhody, "PCEPS: Usage of TLS to Provide a Secure Transport for the Path Computation Element Communication Protocol (PCEP)", RFC 8253, DOI 10.17487/RFC8253, October 2017, <https://www.rfc-editor.org/info/rfc8253>. [RFC8281] Crabbe, E., Minei, I., Sivabalan, S., and R. Varga, "Path Computation Element Communication Protocol (PCEP) Extensions for PCE-Initiated LSP Setup in a Stateful PCE Model", RFC 8281, DOI 10.17487/RFC8281, December 2017, <https://www.rfc-editor.org/info/rfc8281>. [RFC8283] Farrel, A., Ed., Zhao, Q., Ed., Li, Z., and C. Zhou, "An Architecture for Use of PCE and the PCE Communication Protocol (PCEP) in a Network with Central Control", RFC 8283, DOI 10.17487/RFC8283, December 2017, <https://www.rfc-editor.org/info/rfc8283>. [RFC8454] Lee, Y., Belotti, S., Dhody, D., Ceccarelli, D., and B. Yoon, "Information Model for Abstraction and Control of TE Networks (ACTN)", RFC 8454, DOI 10.17487/RFC8454, September 2018, <https://www.rfc-editor.org/info/rfc8454>.[I-D.ietf-pce-stateful-hpce] Dhody, D., Lee, Y., Ceccarelli, D., Shin, J., King, D., and O. Dios, "Hierarchical Stateful Path Computation Element (PCE).", draft-ietf-pce-stateful-hpce-07 (work in progress), April 2019. [I-D.ietf-pce-inter-area-as-applicability] King, D. and H. Zheng, "Applicability of the Path Computation Element to Inter-Area and Inter-AS MPLS and GMPLS Traffic Engineering", draft-ietf-pce-inter-area-as- applicability-07 (work in progress), December 2018. [I-D.dhodylee-pce-pcep-ls] Dhody, D., Lee, Y., and D. Ceccarelli, "PCEP Extension for Distribution of Link-State and TE Information.", draft- dhodylee-pce-pcep-ls-13 (work in progress), February 2019. [I-D.lee-pce-pcep-ls-optical] Lee, Y., Zheng, H., Ceccarelli, D., weiw@bupt.edu.cn, w., Park, P., and B. Yoon, "PCEP Extension for Distribution of Link-State and TE information for Optical Networks", draft-lee-pce-pcep-ls-optical-07 (work in progress), March 2019. [I-D.leedhody-pce-vn-association] Lee, Y., Zhang, X., and D. Ceccarelli, "PCEP Extensions for Establishing Relationships Between Sets of LSPs and Virtual Networks", draft-leedhody-pce-vn-association-07 (work in progress), February 2019. [I-D.litkowski-pce-state-sync] Litkowski, S., Sivabalan, S., Li, C., and H. Zheng, "Inter Stateful Path Computation Element (PCE) Communication Procedures.", draft-litkowski-pce-state-sync-05 (work in progress), March 2019. [I-D.ietf-pce-association-policy] Litkowski, S., Sivabalan, S., Tantsura, J., Hardwick, J., and M. Negi, "Path Computation Element communication Protocol extension for associating Policies and LSPs", draft-ietf-pce-association-policy-05 (work in progress), February 2019. [I-D.dugeon-pce-stateful-interdomain] Dugeon, O., Meuric, J., Lee, Y., and D. Ceccarelli, "PCEP Extension for Stateful Inter-Domain Tunnels", draft- dugeon-pce-stateful-interdomain-02 (work in progress), March 2019. [EXP] Casellas, R., Vilalta, R., Martinez, R., Munoz, R., Zheng, H., and Y. Lee, "Experimental Validation of the ACTN architecture for flexi-grid optical networks using Active Stateful Hierarchical PCEs", 19th International Conference on Transparent Optical Networks (ICTON) , July 2017, <http://www.cttc.es/publication/experimental-validation- of-the-actn-architecture-for-flexi-grid-optical-networks- using-active-stateful-hierarchical-pces/>.Appendix A. Additional Information In the paper [EXP], the application of the ACTN architecture is presented to demonstrate the control of amulti-domainmultidomain flexi-grid opticalnetwork,network by proposing,adoptingadopting, andextending -extending: o the Hierarchical active stateful PCE architectures and protocols o the PCEP protocol to support efficient and incrementallink statelink-state topological reporting, known as PCEP-LS o theper linkper-link partitioning of the optical spectrum based on variable-sized allocated frequency slots enabling network sharing and virtualization o the use of a model-based interface to dynamically request the instantiation of virtual networks for specificclients / tenants.clients/tenants. The design andtheimplementation of thetestbedtest bed are reported in order to validate the approach. Acknowledgments The authors would like to thank Jonathan Hardwick for the inspiration behind this document. Further thanks to Avantika for her comments with suggested text. Thanks to Adrian Farrel and Daniel King for their substantial reviews. Thanks to Yingzhen Qu for RTGDIR review. Thanks to Rifaat Shekh-Yusef for SECDIR review. Thanks to Meral Shirazipour for GENART review. Thanks to Roman Danyliw and Barry Leiba for IESG review comments. Thanks to Deborah Brungard for being the responsible AD. Authors' Addresses Dhruv Dhody Huawei Technologies Divyashree Techno Park, Whitefield Bangalore, Karnataka 560066 IndiaEMail:Email: dhruv.ietf@gmail.com Young LeeHuaweiFuturewei Technologies 5340 Legacy Drive,Building 3Suite 173 Plano, TX75023 USA EMail: leeyoung@huawei.com75024 United States of America Email: younglee.tx@gmail.com Daniele Ceccarelli Ericsson Torshamnsgatan,48 Stockholm SwedenEMail:Email: daniele.ceccarelli@ericsson.com