rfc9326.original   rfc9326.txt 
IPPM H. Song Internet Engineering Task Force (IETF) H. Song
Internet-Draft Futurewei Request for Comments: 9326 Futurewei
Intended status: Standards Track B. Gafni Category: Standards Track B. Gafni
Expires: 27 March 2023 Nvidia ISSN: 2070-1721 Nvidia
F. Brockners F. Brockners
Cisco Cisco
S. Bhandari S. Bhandari
Thoughtspot Thoughtspot
T. Mizrahi T. Mizrahi
Huawei Huawei
23 September 2022 October 2022
In-situ OAM Direct Exporting In Situ Operations, Administration, and Maintenance (IOAM) Direct
draft-ietf-ippm-ioam-direct-export-11 Exporting
Abstract Abstract
In-situ Operations, Administration, and Maintenance (IOAM) is used In situ Operations, Administration, and Maintenance (IOAM) is used
for recording and collecting operational and telemetry information. for recording and collecting operational and telemetry information.
Specifically, IOAM allows telemetry data to be pushed into data Specifically, IOAM allows telemetry data to be pushed into data
packets while they traverse the network. This document introduces a packets while they traverse the network. This document introduces a
new IOAM option type (denoted IOAM-Option-Type) called the Direct new IOAM option type (denoted IOAM-Option-Type) called the "IOAM
Export (DEX) Option-Type, which is used as a trigger for IOAM data to Direct Export (DEX) Option-Type". This Option-Type is used as a
be directly exported or locally aggregated without being pushed into trigger for IOAM data to be directly exported or locally aggregated
in-flight data packets. The exporting method and format are outside without being pushed into in-flight data packets. The exporting
the scope of this document. method and format are outside the scope of this document.
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This is an Internet Standards Track document.
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months This document is a product of the Internet Engineering Task Force
and may be updated, replaced, or obsoleted by other documents at any (IETF). It represents the consensus of the IETF community. It has
time. It is inappropriate to use Internet-Drafts as reference received public review and has been approved for publication by the
material or to cite them other than as "work in progress." Internet Engineering Steering Group (IESG). Further information on
Internet Standards is available in Section 2 of RFC 7841.
This Internet-Draft will expire on 27 March 2023. Information about the current status of this document, any errata,
and how to provide feedback on it may be obtained at
https://www.rfc-editor.org/info/rfc9326.
Copyright Notice Copyright Notice
Copyright (c) 2022 IETF Trust and the persons identified as the Copyright (c) 2022 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/ Provisions Relating to IETF Documents
license-info) in effect on the date of publication of this document. (https://trustee.ietf.org/license-info) in effect on the date of
Please review these documents carefully, as they describe your rights publication of this document. Please review these documents
and restrictions with respect to this document. Code Components carefully, as they describe your rights and restrictions with respect
extracted from this document must include Revised BSD License text as to this document. Code Components extracted from this document must
described in Section 4.e of the Trust Legal Provisions and are include Revised BSD License text as described in Section 4.e of the
provided without warranty as described in the Revised BSD License. Trust Legal Provisions and are provided without warranty as described
in the Revised BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 1. Introduction
2. Conventions . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Conventions
2.1. Requirement Language . . . . . . . . . . . . . . . . . . 3 2.1. Requirements Language
2.2. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3 2.2. Terminology
3. The Direct Exporting (DEX) IOAM-Option-Type . . . . . . . . . 3 3. The Direct Exporting (DEX) IOAM-Option-Type
3.1. Overview . . . . . . . . . . . . . . . . . . . . . . . . 4 3.1. Overview
3.1.1. DEX Packet Selection . . . . . . . . . . . . . . . . 5 3.1.1. DEX Packet Selection
3.1.2. Responding to the DEX Trigger . . . . . . . . . . . . 6 3.1.2. Responding to the DEX Trigger
3.2. The DEX Option-Type Format . . . . . . . . . . . . . . . 7 3.2. The DEX Option-Type Format
4. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 9 4. IANA Considerations
4.1. IOAM Type . . . . . . . . . . . . . . . . . . . . . . . . 9 4.1. IOAM Type
4.2. IOAM DEX Flags . . . . . . . . . . . . . . . . . . . . . 9 4.2. IOAM DEX Flags
4.3. IOAM DEX Extension-Flags . . . . . . . . . . . . . . . . 9 4.3. IOAM DEX Extension-Flags
5. Performance Considerations . . . . . . . . . . . . . . . . . 10 5. Performance Considerations
6. Security Considerations . . . . . . . . . . . . . . . . . . . 10 6. Security Considerations
7. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 12 7. References
8. References . . . . . . . . . . . . . . . . . . . . . . . . . 12 7.1. Normative References
8.1. Normative References . . . . . . . . . . . . . . . . . . 12 7.2. Informative References
8.2. Informative References . . . . . . . . . . . . . . . . . 12 Appendix A. Notes about the History of This Document
Appendix A. Notes About the History of this Document . . . . . . 14 Acknowledgments
Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Contributors
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 15 Authors' Addresses
1. Introduction 1. Introduction
IOAM [RFC9197] is used for monitoring traffic in the network, and for IOAM [RFC9197] is used for monitoring traffic in the network and for
incorporating IOAM data fields (denoted IOAM-Data-Fields) into in- incorporating IOAM data fields (denoted IOAM-Data-Fields) into in-
flight data packets. flight data packets.
IOAM makes use of four possible IOAM-Option-Types, defined in IOAM makes use of four possible IOAM-Option-Types, defined in
[RFC9197]: Pre-allocated Trace Option-Type, Incremental Trace Option- [RFC9197]: Pre-allocated Trace, Incremental Trace, Proof of Transit
Type, Proof of Transit (POT) Option-Type, and Edge-to-Edge Option- (POT), and Edge-to-Edge.
Type.
This document defines a new IOAM-Option-Type called the Direct Export This document defines a new IOAM-Option-Type called the "IOAM Direct
(DEX) Option-Type. This Option-Type is used as a trigger for IOAM Export (DEX) Option-Type". This Option-Type is used as a trigger for
nodes to locally aggregate and process IOAM data, and/or to export it IOAM nodes to locally aggregate and process IOAM data and/or to
to a receiving entity (or entities). Throughout the document this export it to a receiving entity (or entities). Throughout the
functionality is referred to as collection and/or exporting. A document, this functionality is referred to as "collection" and/or
"receiving entity" in this context is an entity that resides within "exporting". In this context, a "receiving entity" is an entity that
the IOAM domain such as a collector, analyzer, controller, resides within the IOAM domain such as a collector, analyzer,
decapsulating node, or a software module in one of the IOAM nodes. controller, decapsulating node, or software module in one of the IOAM
nodes.
Note that even though the IOAM-Option-Type is called "Direct Export", Note that even though the IOAM-Option-Type is called "Direct Export",
it depends on the deployment whether the receipt of a packet with DEX it depends on the deployment whether the receipt of a packet with a
Option-Type leads to the creation of another packet. Some DEX Option-Type leads to the creation of another packet. Some
deployments might simply use the packet with the DEX Option-Type to deployments might simply use the packet with the DEX Option-Type to
trigger local processing of OAM data. The functionality of this trigger local processing of Operations, Administration, and
local processing is not within the scope of this document. Maintenance (OAM) data. The functionality of this local processing
is not within the scope of this document.
2. Conventions 2. Conventions
2.1. Requirement Language 2.1. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP "OPTIONAL" in this document are to be interpreted as described in
14 [RFC2119] [RFC8174] when, and only when, they appear in all BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all
capitals, as shown here. capitals, as shown here.
2.2. Terminology 2.2. Terminology
Abbreviations used in this document: Abbreviations used in this document:
IOAM: In-situ Operations, Administration, and Maintenance IOAM: In situ Operations, Administration, and Maintenance
OAM: Operations, Administration, and Maintenance [RFC6291] OAM: Operations, Administration, and Maintenance [RFC6291]
DEX: Direct EXporting DEX: Direct Exporting
3. The Direct Exporting (DEX) IOAM-Option-Type 3. The Direct Exporting (DEX) IOAM-Option-Type
3.1. Overview 3.1. Overview
The DEX Option-Type is used as a trigger for collecting IOAM data The DEX Option-Type is used as a trigger for collecting IOAM data
locally or for exporting it to a receiving entity (or entities). locally or exporting it to a receiving entity (or entities).
Specifically, the DEX Option-Type can be used as a trigger for Specifically, the DEX Option-Type can be used as a trigger for
collecting IOAM data by an IOAM node and locally aggregating it; collecting IOAM data by an IOAM node and locally aggregating it;
thus, this aggregated data can be periodically pushed to a receiving thus, this aggregated data can be periodically pushed to a receiving
entity, or pulled by a receiving entity on-demand. entity or pulled by a receiving entity on-demand.
This Option-Type is incorporated into data packets by an IOAM This Option-Type is incorporated into data packets by an IOAM
encapsulating node, and removed by an IOAM decapsulating node, as encapsulating node and removed by an IOAM decapsulating node, as
illustrated in Figure 1. The Option-Type can be read but not illustrated in Figure 1. The Option-Type can be read, but not
modified by transit nodes. Note: the terms IOAM encapsulating, modified, by transit nodes. Note that the terms "IOAM encapsulating
decapsulating and transit nodes are as defined in [RFC9197]. node", "IOAM decapsulating node", and "IOAM transit node" are as
defined in [RFC9197].
^ ^
|Exported IOAM data |Exported IOAM data
| |
| |
| |
+--------------+------+-------+--------------+ +--------------+------+-------+--------------+
| | | | | | | |
| | | | | | | |
User +---+----+ +---+----+ +---+----+ +---+----+ User +---+----+ +---+----+ +---+----+ +---+----+
skipping to change at page 5, line 6 skipping to change at line 173
option and IOAM data IOAM data option and option and IOAM data IOAM data option and
export data export data export data export data
Figure 1: DEX Architecture Figure 1: DEX Architecture
The DEX Option-Type is used as a trigger to collect and/or export The DEX Option-Type is used as a trigger to collect and/or export
IOAM data. The trigger applies to transit nodes, the decapsulating IOAM data. The trigger applies to transit nodes, the decapsulating
node, and the encapsulating node: node, and the encapsulating node:
* An IOAM encapsulating node configured to incorporate the DEX * An IOAM encapsulating node configured to incorporate the DEX
Option-Type encapsulates (possibly a subset of) the packets it Option-Type encapsulates the packets (or possibly a subset of the
forwards with the DEX Option-Type, and MAY export and/or collect packets) it forwards with the DEX Option-Type and MAY export and/
the requested IOAM data immediately. Only IOAM encapsulating or collect the requested IOAM data immediately. Only IOAM
nodes are allowed to add the DEX Option-Type to a packet. An IOAM encapsulating nodes are allowed to add the DEX Option-Type to a
encapsulating node can generate probe packets that incorporate the packet. An IOAM encapsulating node can generate probe packets
DEX Option-Type. These probe packets can be generated that incorporate the DEX Option-Type. These probe packets can be
periodically or on-demand (for example triggered by the management generated periodically or on-demand (for example, triggered by the
plane). The specification of such probe packets is outside the management plane). The specification of such probe packets is
scope of this document. outside the scope of this document.
* A transit node that processes a packet with the DEX Option-Type * A transit node that processes a packet with the DEX Option-Type
MAY export and/or collect the requested IOAM data. MAY export and/or collect the requested IOAM data.
* An IOAM decapsulating node that processes a packet with the DEX * An IOAM decapsulating node that processes a packet with the DEX
Option-Type MAY export and/or collect the requested IOAM data, and Option-Type MAY export and/or collect the requested IOAM data and
MUST decapsulate the IOAM header. MUST decapsulate the IOAM header.
As in [RFC9197], the DEX Option-Type can be incorporated into all or As in [RFC9197], the DEX Option-Type can be incorporated into all or
a subset of the traffic that is forwarded by the encapsulating node, a subset of the traffic that is forwarded by the encapsulating node,
as further discussed in Section 3.1.1 below. Moreover, IOAM nodes as further discussed in Section 3.1.1. Moreover, IOAM nodes respond
respond to the DEX trigger by exporting and/or collecting IOAM data to the DEX trigger by exporting and/or collecting IOAM data either
either for all traversing packets that carry the DEX Option-Type, or for all traversing packets that carry the DEX Option-Type or
selectively only for a subset of these packets, as further discussed selectively only for a subset of these packets, as further discussed
in Section 3.1.2 below. in Section 3.1.2.
3.1.1. DEX Packet Selection 3.1.1. DEX Packet Selection
If an IOAM encapsulating node incorporates the DEX Option-Type into If an IOAM encapsulating node incorporates the DEX Option-Type into
all the traffic it forwards it may lead to an excessive amount of all the traffic it forwards, it may lead to an excessive amount of
exported data, which may overload the network and the receiving exported data, which may overload the network and the receiving
entity. Therefore, an IOAM encapsulating node that supports the DEX entity. Therefore, an IOAM encapsulating node that supports the DEX
Option-Type MUST support the ability to incorporate the DEX Option- Option-Type MUST support the ability to incorporate the DEX Option-
Type selectively into a subset of the packets that are forwarded by Type selectively into a subset of the packets that are forwarded by
it. the IOAM encapsulating node.
Various methods of packet selection and sampling have been previously Various methods of packet selection and sampling have been previously
defined, such as [RFC7014] and [RFC5475]. Similar techniques can be defined, such as [RFC7014] and [RFC5475]. Similar techniques can be
applied by an IOAM encapsulating node to apply DEX to a subset of the applied by an IOAM encapsulating node to apply DEX to a subset of the
forwarded traffic. forwarded traffic.
The subset of traffic that is forwarded or transmitted with a DEX The subset of traffic that is forwarded or transmitted with a DEX
Option-Type SHOULD NOT exceed 1/N of the interface capacity on any of Option-Type SHOULD NOT exceed 1/N of the interface capacity on any of
the IOAM encapsulating node's interfaces. It is noted that this the IOAM encapsulating node's interfaces. It is noted that this
requirement applies to the total traffic that incorporates a DEX requirement applies to the total traffic that incorporates a DEX
Option-Type, including traffic that is forwarded by the IOAM Option-Type, including traffic that is forwarded by the IOAM
encapsulating node and probe packets that are generated by the IOAM encapsulating node and probe packets that are generated by the IOAM
encapsulating node. In this context N is a parameter that can be encapsulating node. In this context, N is a parameter that can be
configurable by network operators. If there is an upper bound, M, on configurable by network operators. If there is an upper bound, M, on
the number of IOAM transit nodes in any path in the network, then it the number of IOAM transit nodes in any path in the network, then it
is RECOMMENDED to use an N such that N >> M (i.e., N is much greater is RECOMMENDED to use an N such that N >> M (i.e., N is much greater
than M). The rationale is that a packet that includes a DEX Option- than M). The rationale is that a packet that includes a DEX Option-
Type may trigger an exported packet from each IOAM transit node along Type may trigger an exported packet from each IOAM transit node along
the path for a total of M exported packets. Thus, if N >> M then the the path for a total of M exported packets. Thus, if N >> M, then
number of exported packets is significantly lower than the number of the number of exported packets is significantly lower than the number
data packets forwarded by the IOAM encapsulating node. If there is of data packets forwarded by the IOAM encapsulating node. If there
no prior knowledge about the network topology or size, it is is no prior knowledge about the network topology or size, it is
RECOMMENDED to use N>100. RECOMMENDED to use N>100.
3.1.2. Responding to the DEX Trigger 3.1.2. Responding to the DEX Trigger
The DEX Option-Type specifies which IOAM-Data-Fields should be The DEX Option-Type specifies which IOAM-Data-Fields should be
exported and/or collected, as specified in Section 3.2. As mentioned exported and/or collected, as specified in Section 3.2. As mentioned
above, the data can be locally collected, and optionally can be above, the data can be locally collected, aggregated, and/or exported
aggregated and exported to a receiving entity, either proactively or to a receiving entity proactively or on-demand. If IOAM data is
on-demand. If IOAM data is exported, the format and encapsulation of exported, the format and encapsulation of the packet that contains
the packet that contains the exported data is not within the scope of the exported data is not within the scope of the current document.
the current document. For example, the export format can be based on For example, the export format can be based on [IOAM-RAWEXPORT].
[I-D.spiegel-ippm-ioam-rawexport].
An IOAM node that performs DEX-triggered exporting MUST support the An IOAM node that performs DEX-triggered exporting MUST support the
ability to limit the rate of the exported packets. The rate of ability to limit the rate of the exported packets. The rate of
exported packets SHOULD be limited so that the number of exported exported packets SHOULD be limited so that the number of exported
packets is significantly lower than the number of packets that are packets is significantly lower than the number of packets that are
forwarded by the device. The exported data rate SHOULD NOT exceed 1/ forwarded by the device. The exported data rate SHOULD NOT exceed 1/
N of the interface capacity on any of the IOAM node's interfaces. It N of the interface capacity on any of the IOAM node's interfaces. It
is RECOMMENDED to use N>100. Depending on the IOAM node's is RECOMMENDED to use N>100. Depending on the IOAM node's
architecture considerations, the export rate may be limited to a architecture considerations, the export rate may be limited to a
lower number in order to avoid loading the IOAM node. An IOAM node lower number in order to avoid loading the IOAM node. An IOAM node
skipping to change at page 6, line 47 skipping to change at line 262
does not collect and/or export data due to the rate limits. does not collect and/or export data due to the rate limits.
IOAM nodes SHOULD NOT be configured to export packets over a path or IOAM nodes SHOULD NOT be configured to export packets over a path or
a tunnel that is subject to IOAM direct exporting. Furthermore, IOAM a tunnel that is subject to IOAM direct exporting. Furthermore, IOAM
encapsulating nodes that can identify a packet as an IOAM exported encapsulating nodes that can identify a packet as an IOAM exported
packet MUST NOT push a DEX Option-Type into such a packet. This packet MUST NOT push a DEX Option-Type into such a packet. This
requirement is intended to prevent nested exporting and/or exporting requirement is intended to prevent nested exporting and/or exporting
loops. loops.
A transit or decapsulating IOAM node that receives an unknown IOAM- A transit or decapsulating IOAM node that receives an unknown IOAM-
Option-Type ignores it (as defined in [RFC9197]), and specifically Option-Type ignores it (as defined in [RFC9197]); specifically, nodes
nodes that do not support the DEX Option-Type ignore it. Note that that do not support the DEX Option-Type ignore it. As per [RFC9197],
as per [RFC9197] a decapsulating node removes the IOAM encapsulation note that a decapsulating node removes the IOAM encapsulation and all
and all its IOAM-Option-Types. Specifically, this applies to the its IOAM-Option-Types. Specifically, this applies to the case where
case where one of these options is a (possibly unknown) DEX Option- one of these options is a (possibly unknown) DEX Option-Type. The
Type. The ability to skip over a (possibly unknown) DEX Option-Type ability to skip over a (possibly unknown) DEX Option-Type in the
in the parsing or in the decapsulation procedure is dependent on the parsing or in the decapsulation procedure is dependent on the
specific encapsulation, which is outside the scope of this document. specific encapsulation, which is outside the scope of this document.
For example, when IOAM is encapsulated in IPv6 For example, when IOAM is encapsulated in IPv6 [IOAM-IPV6-OPTIONS],
[I-D.ietf-ippm-ioam-ipv6-options] the DEX Option-Type is incorporated the DEX Option-Type is incorporated either in a Hop-by-Hop options
either in a Hop-by-Hop options header or in a Destination options header or in a Destination options header; thus, it can be skipped
header, and thus can be skipped using the length field in the options using the length field in the options header.
header.
3.2. The DEX Option-Type Format 3.2. The DEX Option-Type Format
The format of the DEX Option-Type is depicted in Figure 2. The The format of the DEX Option-Type is depicted in Figure 2. The
length of the DEX Option-Type is at least 8 octets. The DEX Option- length of the DEX Option-Type is at least 8 octets. The DEX Option-
Type MAY include one or more optional fields. The existence of the Type MAY include one or more optional fields. The existence of the
optional fields is indicated by the corresponding flags in the optional fields is indicated by the corresponding flags in the
Extension-Flags field. Two optional fields are defined in this Extension-Flags field. Two optional fields are defined in this
document, the Flow ID and the Sequence Number fields. Every optional document: the Flow ID and Sequence Number fields. Every optional
field MUST be exactly 4 octets long. Thus, the Extension-Flags field field MUST be exactly 4 octets long. Thus, the Extension-Flags field
explicitly indicates the length of the DEX Option-Type. Defining a explicitly indicates the length of the DEX Option-Type. Defining a
new optional field requires an allocation of a corresponding flag in new optional field requires an allocation of a corresponding flag in
the Extension-Flags field, as specified in Section 4.2. the Extension-Flags field, as specified in Section 4.2.
0 1 2 3 0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Namespace-ID | Flags |Extension-Flags| | Namespace-ID | Flags |Extension-Flags|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IOAM-Trace-Type | Reserved | | IOAM-Trace-Type | Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Flow ID (optional) | | Flow ID (Optional) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Sequence Number (Optional) | | Sequence Number (Optional) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 2: DEX Option-Type Format Figure 2: DEX Option-Type Format
Namespace-ID A 16-bit identifier of the IOAM namespace, as defined Namespace-ID:
in [RFC9197]. A 16-bit identifier of the IOAM namespace, as defined in
[RFC9197].
Flags An 8-bit field, comprised of 8 one-bit subfields.
Flags are allocated by IANA, as defined in
Section 4.2.
Extension-Flags An 8-bit field, comprised of 8 one-bit subfields. Flags:
An 8-bit field, comprised of 8 1-bit subfields. Flags are
allocated by IANA, as defined in Section 4.2.
Extension-Flags are allocated by IANA, as defined in Extension-Flags:
Section 4.3. Every bit in the Extension-Flag field An 8-bit field, comprised of 8 1-bit subfields. Extension-Flags
that is set to 1 indicates the existence of a are allocated by IANA, as defined in Section 4.3. Every bit in
corresponding optional 4-octet field. An IOAM node the Extension-Flag field that is set to 1 indicates the existence
that receives a DEX Option-Type with an unknown flag of a corresponding optional 4-octet field. An IOAM node that
set to 1 MUST ignore the corresponding optional receives a DEX Option-Type with an unknown flag set to 1 MUST
field. ignore the corresponding optional field.
IOAM-Trace-Type A 24-bit identifier which specifies which IOAM-Data- IOAM-Trace-Type:
Fields should be exported. The format of this field A 24-bit identifier that specifies which IOAM-Data-Fields should
is as defined in [RFC9197]. Specifically, the bit be exported. The format of this field is as defined in [RFC9197].
that corresponds to the Checksum Complement IOAM- Specifically, the bit that corresponds to the Checksum Complement
Data-Field SHOULD be assigned to be zero by the IOAM IOAM-Data-Field SHOULD be assigned to be zero by the IOAM
encapsulating node, and ignored by transit and encapsulating node and ignored by transit and decapsulating nodes.
decapsulating nodes. The reason for this is that the The reason for this is that the Checksum Complement is intended
Checksum Complement is intended for in-flight packet for in-flight packet modifications and is not relevant for direct
modifications and is not relevant for direct exporting.
exporting.
Reserved This field MUST be ignored by the receiver. Reserved:
This field MUST be ignored by the receiver.
Optional fields The optional fields, if present, reside after the Optional fields:
Reserved field. The order of the optional fields is The optional fields, if present, reside after the Reserved field.
according to the order of the respective bits, The order of the optional fields is according to the order of the
starting from the most significant bit, that are respective bits, starting from the most significant bit, that are
enabled in the Extension-Flags field. Each optional enabled in the Extension-Flags field. Each optional field is 4
field is 4 octets long. octets long.
Flow ID An optional 32-bit field representing the flow Flow ID:
identifier. If the actual Flow ID is shorter than 32 An optional 32-bit field representing the flow identifier. If
bits, it is zero padded in its most significant bits. the actual Flow ID is shorter than 32 bits, it is zero padded
The field is set at the encapsulating node. The Flow in its most significant bits. The field is set at the
ID can be used to correlate the exported data of the encapsulating node. The Flow ID can be used to correlate the
same flow from multiple nodes and from multiple exported data of the same flow from multiple nodes and from
packets. Flow ID values are expected to be allocated multiple packets. Flow ID values are expected to be allocated
in a way that avoids collisions. For example, random in a way that avoids collisions. For example, random
assignment of Flow ID values can be subject to assignment of Flow ID values can be subject to collisions,
collisions, while centralized allocation can avoid while centralized allocation can avoid this problem. The
this problem. The specification of the Flow ID specification of the Flow ID allocation method is not within
allocation method is not within the scope of this the scope of this document.
document.
Sequence Number An optional 32-bit sequence number starting from 0 Sequence Number:
and incremented by 1 for each packet from the same An optional 32-bit sequence number starting from 0 and
flow at the encapsulating node that includes the DEX incremented by 1 for each packet from the same flow at the
option. The Sequence Number, when combined with the encapsulating node that includes the DEX option. The Sequence
Flow ID, provides a convenient approach to correlate Number, when combined with the Flow ID, provides a convenient
the exported data from the same user packet. approach to correlate the exported data from the same user
packet.
4. IANA Considerations 4. IANA Considerations
4.1. IOAM Type 4.1. IOAM Type
The "IOAM Option-Type Registry" was defined in Section 7.1 of The "IOAM Option-Type" registry is defined in Section 7.1 of
[RFC9197]. IANA is requested to allocate the following code point [RFC9197]. IANA has allocated the following code point from the
from the "IOAM Option-Type Registry" as follows: "IOAM Option-Type" registry as follows:
TBD-type IOAM Direct Export (DEX) Option-Type Code Point: 4
If possible, IANA is requested to allocate code point 4 (TBD-type). Name IOAM Direct Export (DEX) Option-Type
The "Description" for the new option should be "Direct exporting" and
the "Reference" should be the current document. Description: Direct exporting
Reference: This document
4.2. IOAM DEX Flags 4.2. IOAM DEX Flags
IANA is requested to define an "IOAM DEX Flags" registry. This IANA has created the "IOAM DEX Flags" registry. This registry
registry includes 8 flag bits. Allocation is based on the "IETF includes 8 flag bits. Allocation is based on the "IETF Review"
Review" procedure, as defined in [RFC8126]. procedure defined in [RFC8126].
New registration requests MUST use the following template: New registration requests MUST use the following template:
Bit: Desired bit to be allocated in the 8 bit Flags field of the DEX Bit: Desired bit to be allocated in the 8-bit Flags field of the DEX
Option-Type. Option-Type.
Description: Brief description of the newly registered bit. Description: Brief description of the newly registered bit.
Reference: Reference to the document that defines the new bit. Reference: Reference to the document that defines the new bit.
4.3. IOAM DEX Extension-Flags 4.3. IOAM DEX Extension-Flags
IANA is requested to define an "IOAM DEX Extension-Flags" registry. IANA has created the "IOAM DEX Extension-Flags" registry. This
This registry includes 8 flag bits. Bit 0 (the most significant bit) registry includes 8 flag bits. Bit 0 (the most significant bit) and
and bit 1 in the registry are allocated by this document, and bit 1 in the registry are allocated by this document and described in
described in Section 3.2. Allocation of the other bits should be Section 3.2. Allocation of the other bits should be performed based
performed based on the "IETF Review" procedure, as defined in on the "IETF Review" procedure defined in [RFC8126].
[RFC8126].
Bit 0 "Flow ID [RFC XXXX] [RFC Editor: please replace with the RFC Bit 0: "Flow ID [RFC9326]"
number of the current document]"
Bit 1 "Sequence Number [RFC XXXX] [RFC Editor: please replace with Bit 1: "Sequence Number [RFC9326]"
the RFC number of the current document]"
New registration requests MUST use the following template: New registration requests MUST use the following template:
Bit: Desired bit to be allocated in the 8 bit Extension-Flags field Bit: Desired bit to be allocated in the 8-bit Extension-Flags field
of the DEX Option-Type. of the DEX Option-Type.
Description: Brief description of the newly registered bit. Description: Brief description of the newly registered bit.
Reference: Reference to the document that defines the new bit. Reference: Reference to the document that defines the new bit.
5. Performance Considerations 5. Performance Considerations
The DEX Option-Type triggers IOAM data to be collected and/or The DEX Option-Type triggers IOAM data to be collected and/or
exported packets to be exported to a receiving entity (or entities). exported packets to be exported to a receiving entity (or entities).
In some cases this may impact the receiving entity's performance, or In some cases, this may impact the receiving entity's performance or
the performance along the paths leading to it. the performance along the paths leading to it.
Therefore, the performance impact of these exported packets is Therefore, the performance impact of these exported packets is
limited by taking two measures: at the encapsulating nodes, by limited by taking two measures: at the encapsulating nodes by
selective DEX encapsulation (Section 3.1.1), and at the transit selective DEX encapsulation (Section 3.1.1) and at the transit nodes
nodes, by limiting exporting rate (Section 3.1.2). These two by limiting exporting rate (Section 3.1.2). These two measures
measures ensure that direct exporting is used at a rate that does not ensure that direct exporting is used at a rate that does not
significantly affect the network bandwidth, and does not overload the significantly affect the network bandwidth and does not overload the
receiving entity. Moreover, it is possible to load balance the receiving entity. Moreover, it is possible to load balance the
exported data among multiple receiving entities, although the exported data among multiple receiving entities, although the
exporting method is not within the scope of this document. exporting method is not within the scope of this document.
It should be noted that in some networks DEX data may be exported It should be noted that, in some networks, DEX data may be exported
over an out-of-band network, in which a large volume of exported over an out-of-band network in which a large volume of exported
traffic does not compromise user traffic. In this case an operator traffic does not compromise user traffic. In this case, an operator
may choose to disable the exporting rate limiting. may choose to disable the exporting rate limiting.
6. Security Considerations 6. Security Considerations
The security considerations of IOAM in general are discussed in The security considerations of IOAM in general are discussed in
[RFC9197]. Specifically, an attacker may try to use the [RFC9197]. Specifically, an attacker may try to use the
functionality that is defined in this document to attack the network. functionality that is defined in this document to attack the network.
An attacker may attempt to overload network devices by injecting An attacker may attempt to overload network devices by injecting
synthetic packets that include the DEX Option-Type. Similarly, an synthetic packets that include the DEX Option-Type. Similarly, an
on-path attacker may maliciously incorporate the DEX Option-Type into on-path attacker may maliciously incorporate the DEX Option-Type into
transit packets, or maliciously remove it from packets in which it is transit packets or maliciously remove it from packets in which it is
incorporated. incorporated.
Forcing DEX, either in synthetic packets or in transit packets may Forcing DEX, either in synthetic packets or in transit packets, may
overload the IOAM nodes and/or the receiving entity (or entities). overload the IOAM nodes and/or the receiving entity (or entities).
Since this mechanism affects multiple devices along the network path, Since this mechanism affects multiple devices along the network path,
it potentially amplifies the effect on the network bandwidth, on the it potentially amplifies the effect on the network bandwidth, the
storage of the devices that collect the data, and on the receiving storage of the devices that collect the data, and the receiving
entity's load. entity's load.
The amplification effect of DEX may be worse in wide area networks in The amplification effect of DEX may be worse in wide area networks in
which there are multiple IOAM domains. For example, if DEX is used which there are multiple IOAM-Domains. For example, if DEX is used
in IOAM domain 1 for exporting IOAM data to a receiving entity, then in IOAM-Domain 1 for exporting IOAM data to a receiving entity, then
the exported packets of domain 1 can be forwarded through IOAM domain the exported packets of IOAM-Domain 1 can be forwarded through IOAM-
2, in which they are subject to DEX. The exported packets of domain Domain 2, in which they are subject to DEX. In turn, the exported
2 may in turn be forwarded through another IOAM domain (or through packets of IOAM-Domain 2 may be forwarded through another IOAM domain
domain 1), and theoretically this recursive amplification may (or through IOAM-Domain 1); theoretically, this recursive
continue infinitely. amplification may continue infinitely.
In order to mitigate the attacks described above, the following In order to mitigate the attacks described above, the following
requirements (Section 3) have been defined: requirements (Section 3) have been defined:
* Selective DEX (Section 3.1.1) is applied by IOAM encapsulating * Selective DEX (Section 3.1.1) is applied by IOAM encapsulating
nodes in order to limit the potential impact of DEX attacks to a nodes in order to limit the potential impact of DEX attacks to a
small fraction of the traffic. small fraction of the traffic.
* Rate limiting of exported traffic (Section 3.1.2) is applied by * Rate limiting of exported traffic (Section 3.1.2) is applied by
IOAM nodes in order to prevent overloading attacks and in order to IOAM nodes in order to prevent overloading attacks and to
significantly limit the scale of amplification attacks. significantly limit the scale of amplification attacks.
* IOAM encapsulating nodes are required to avoid pushing the DEX * IOAM encapsulating nodes are required to avoid pushing the DEX
Option-Type into IOAM exported packets (Section 3.1.2), thus Option-Type into IOAM exported packets (Section 3.1.2), thus
preventing some of the amplification and export loop scenarios. preventing some of the amplification and export loop scenarios.
Although the exporting method is not within the scope of this Although the exporting method is not within the scope of this
document, any exporting method MUST secure the exported data from the document, any exporting method MUST secure the exported data from the
IOAM node to the receiving entity, in order to protect the IOAM node to the receiving entity in order to protect the
confidentiality and guarantee the integrity of the exported data. confidentiality and guarantee the integrity of the exported data.
Specifically, an IOAM node that performs DEX exporting MUST send the Specifically, an IOAM node that performs DEX exporting MUST send the
exported data to a pre-configured trusted receiving entity that is in exported data to a pre-configured trusted receiving entity that is in
the same IOAM domain as the exporting IOAM node. Furthermore, an the same IOAM-Domain as the exporting IOAM node. Furthermore, an
IOAM node MUST gain explicit consent to export data to a receiving IOAM node MUST gain explicit consent to export data to a receiving
entity before starting to send exported data. entity before starting to send exported data.
An attacker may keep track of the information sent in DEX headers as An attacker may keep track of the information sent in DEX headers as
a means of reconnaissance. This form of recon can be mitigated to a means of reconnaissance. This form of recon can be mitigated to
some extent by careful allocation of the Flow ID and Sequence Number some extent by careful allocation of the Flow ID and Sequence Number
space, in a way that does not compromise privacy aspects such as space in a way that does not compromise privacy aspects, such as
customer identities. customer identities.
The integrity of the DEX Option-Type can be protected through a The integrity of the DEX Option-Type can be protected through a
mechanism of the encapsulating protocol. While mechanism of the encapsulating protocol. While [IOAM-DATA-INTEGRITY]
[I-D.ietf-ippm-ioam-data-integrity] introduces an integrity introduces an integrity protection mechanism that protects the
protection mechanism that protects the integrity of IOAM-Data-Fields, integrity of IOAM-Data-Fields, the DEX Option-Type does not include
the DEX Option-Type does not include IOAM-Data-Fields, and therefore IOAM-Data-Fields; therefore, these integrity protection mechanisms
these integrity protection mechanisms are not applicable to the DEX are not applicable to the DEX Option-Type. As discussed in the
Option-Type. As discussed in the threat analysis of threat analysis of [IOAM-DATA-INTEGRITY], injection or modification
[I-D.ietf-ippm-ioam-data-integrity], injection or modification of of IOAM-Option-Type headers are threats that are not addressed in
IOAM-Option-Type headers are threats that are not addressed in IOAM. IOAM.
IOAM is assumed to be deployed in a restricted administrative domain, IOAM is assumed to be deployed in a restricted administrative domain,
thus limiting the scope of the threats above and their affect. This thus limiting the scope of the threats above and their effect. This
is a fundamental assumption with respect to the security aspects of is a fundamental assumption with respect to the security aspects of
IOAM, as further discussed in [RFC9197]. IOAM, as further discussed in [RFC9197].
7. Acknowledgments 7. References
The authors thank Martin Duke, Tommy Pauly, Meral Shirazipour, Colin
Perkins, Stephen Farrell, Linda Dunbar, Justin Iurman, Greg Mirsky,
and other members of the IPPM working group for many helpful
comments.
8. References
8.1. Normative References 7.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997, DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>. <https://www.rfc-editor.org/info/rfc2119>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>. May 2017, <https://www.rfc-editor.org/info/rfc8174>.
[RFC9197] Brockners, F., Ed., Bhandari, S., Ed., and T. Mizrahi, [RFC9197] Brockners, F., Ed., Bhandari, S., Ed., and T. Mizrahi,
Ed., "Data Fields for In Situ Operations, Administration, Ed., "Data Fields for In Situ Operations, Administration,
and Maintenance (IOAM)", RFC 9197, DOI 10.17487/RFC9197, and Maintenance (IOAM)", RFC 9197, DOI 10.17487/RFC9197,
May 2022, <https://www.rfc-editor.org/info/rfc9197>. May 2022, <https://www.rfc-editor.org/info/rfc9197>.
8.2. Informative References 7.2. Informative References
[I-D.ietf-ippm-ioam-data-integrity] [IOAM-DATA-INTEGRITY]
Brockners, F., Bhandari, S., Mizrahi, T., and J. Iurman, Brockners, F., Bhandari, S., Mizrahi, T., and J. Iurman,
"Integrity of In-situ OAM Data Fields", Work in Progress, "Integrity of In-situ OAM Data Fields", Work in Progress,
Internet-Draft, draft-ietf-ippm-ioam-data-integrity-02, 5 Internet-Draft, draft-ietf-ippm-ioam-data-integrity-02, 5
July 2022, <https://www.ietf.org/archive/id/draft-ietf- July 2022, <https://datatracker.ietf.org/doc/html/draft-
ippm-ioam-data-integrity-02.txt>. ietf-ippm-ioam-data-integrity-02>.
[I-D.ietf-ippm-ioam-flags]
Mizrahi, T., Brockners, F., Bhandari, S., Gafni, B., and
M. Spiegel, "In-situ OAM Loopback and Active Flags", Work
in Progress, Internet-Draft, draft-ietf-ippm-ioam-flags-
10, 18 August 2022, <https://www.ietf.org/archive/id/
draft-ietf-ippm-ioam-flags-10.txt>.
[I-D.ietf-ippm-ioam-ipv6-options] [IOAM-IPV6-OPTIONS]
Bhandari, S. and F. Brockners, "In-situ OAM IPv6 Options", Bhandari, S. and F. Brockners, "In-situ OAM IPv6 Options",
Work in Progress, Internet-Draft, draft-ietf-ippm-ioam- Work in Progress, Internet-Draft, draft-ietf-ippm-ioam-
ipv6-options-08, 16 June 2022, ipv6-options-09, 11 October 2022,
<https://www.ietf.org/archive/id/draft-ietf-ippm-ioam- <https://datatracker.ietf.org/doc/html/draft-ietf-ippm-
ipv6-options-08.txt>. ioam-ipv6-options-09>.
[I-D.song-ippm-postcard-based-telemetry]
Song, H., Mirsky, G., Filsfils, C., Abdelsalam, A., Zhou,
T., Li, Z., Graf, T., Mishra, G., Shin, J., and K. Lee,
"Marking-based Direct Export for On-path Telemetry", Work
in Progress, Internet-Draft, draft-song-ippm-postcard-
based-telemetry-14, 7 September 2022,
<https://www.ietf.org/archive/id/draft-song-ippm-postcard-
based-telemetry-14.txt>.
[I-D.spiegel-ippm-ioam-rawexport] [IOAM-RAWEXPORT]
Spiegel, M., Brockners, F., Bhandari, S., and R. Spiegel, M., Brockners, F., Bhandari, S., and R.
Sivakolundu, "In-situ OAM raw data export with IPFIX", Sivakolundu, "In-situ OAM raw data export with IPFIX",
Work in Progress, Internet-Draft, draft-spiegel-ippm-ioam- Work in Progress, Internet-Draft, draft-spiegel-ippm-ioam-
rawexport-06, 21 February 2022, rawexport-06, 21 February 2022,
<https://www.ietf.org/archive/id/draft-spiegel-ippm-ioam- <https://datatracker.ietf.org/doc/html/draft-spiegel-ippm-
rawexport-06.txt>. ioam-rawexport-06>.
[POSTCARD-BASED-TELEMETRY]
Song, H., Mirsky, G., Filsfils, C., Abdelsalam, A., Zhou,
T., Li, Z., Graf, T., Mishra, G. S., Shin, J., and K. Lee,
"Marking-based Direct Export for On-path Telemetry", Work
in Progress, Internet-Draft, draft-song-ippm-postcard-
based-telemetry-14, 7 September 2022,
<https://datatracker.ietf.org/doc/html/draft-song-ippm-
postcard-based-telemetry-14>.
[RFC5475] Zseby, T., Molina, M., Duffield, N., Niccolini, S., and F. [RFC5475] Zseby, T., Molina, M., Duffield, N., Niccolini, S., and F.
Raspall, "Sampling and Filtering Techniques for IP Packet Raspall, "Sampling and Filtering Techniques for IP Packet
Selection", RFC 5475, DOI 10.17487/RFC5475, March 2009, Selection", RFC 5475, DOI 10.17487/RFC5475, March 2009,
<https://www.rfc-editor.org/info/rfc5475>. <https://www.rfc-editor.org/info/rfc5475>.
[RFC6291] Andersson, L., van Helvoort, H., Bonica, R., Romascanu, [RFC6291] Andersson, L., van Helvoort, H., Bonica, R., Romascanu,
D., and S. Mansfield, "Guidelines for the Use of the "OAM" D., and S. Mansfield, "Guidelines for the Use of the "OAM"
Acronym in the IETF", BCP 161, RFC 6291, Acronym in the IETF", BCP 161, RFC 6291,
DOI 10.17487/RFC6291, June 2011, DOI 10.17487/RFC6291, June 2011,
skipping to change at page 14, line 5 skipping to change at line 578
[RFC7014] D'Antonio, S., Zseby, T., Henke, C., and L. Peluso, "Flow [RFC7014] D'Antonio, S., Zseby, T., Henke, C., and L. Peluso, "Flow
Selection Techniques", RFC 7014, DOI 10.17487/RFC7014, Selection Techniques", RFC 7014, DOI 10.17487/RFC7014,
September 2013, <https://www.rfc-editor.org/info/rfc7014>. September 2013, <https://www.rfc-editor.org/info/rfc7014>.
[RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for [RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for
Writing an IANA Considerations Section in RFCs", BCP 26, Writing an IANA Considerations Section in RFCs", BCP 26,
RFC 8126, DOI 10.17487/RFC8126, June 2017, RFC 8126, DOI 10.17487/RFC8126, June 2017,
<https://www.rfc-editor.org/info/rfc8126>. <https://www.rfc-editor.org/info/rfc8126>.
Appendix A. Notes About the History of this Document [RFC9322] Mizrahi, T., Brockners, F., Bhandari, S., Gafni, B., and
M. Spiegel, "In Situ Operations, Administration, and
Maintanence (IOAM) Loopback and Active Flags", RFC 9322,
DOI 10.17487/RFC9322, November 2022,
<https://www.rfc-editor.org/info/rfc9322>.
Appendix A. Notes about the History of This Document
This document evolved from combining some of the concepts of PBT-I This document evolved from combining some of the concepts of PBT-I
from [I-D.song-ippm-postcard-based-telemetry] with immediate from [POSTCARD-BASED-TELEMETRY] with immediate exporting from early
exporting from early versions of [I-D.ietf-ippm-ioam-flags]. versions of [RFC9322].
In order to help correlate and order the exported packets, it is In order to help correlate and order the exported packets, it is
possible to include the Hop_Lim/Node_ID IOAM-Data-Field in exported possible to include the Hop_Lim/Node_ID IOAM-Data-Field in exported
packets; if the IOAM-Trace-Type [RFC9197] has the Hop_Lim/Node_ID bit packets. If the IOAM-Trace-Type [RFC9197] has the Hop_Lim/Node_ID
set, then exported packets include the Hop_Lim/Node_ID IOAM-Data- bit set, then exported packets include the Hop_Lim/Node_ID IOAM-Data-
Field, which contains the TTL/Hop Limit value from a lower layer Field, which contains the TTL/Hop Limit value from a lower layer
protocol. An alternative approach was considered during the design protocol. An alternative approach was considered during the design
of this document, according to which a 1-octet Hop Count field would of this document, according to which a 1-octet Hop Count field would
be included in the DEX header (presumably by claiming some space from be included in the DEX header (presumably by claiming some space from
the Flags field). The Hop Limit would starts from 0 at the the Flags field). The Hop Limit would start from 0 at the
encapsulating node and be incremented by each IOAM transit node that encapsulating node and be incremented by each IOAM transit node that
supports the DEX Option-Type. In this approach the Hop Count field supports the DEX Option-Type. In this approach, the Hop Count field
value would also be included in the exported packet. value would also be included in the exported packet.
Acknowledgments
The authors thank Martin Duke, Tommy Pauly, Meral Shirazipour, Colin
Perkins, Stephen Farrell, Linda Dunbar, Justin Iurman, Greg Mirsky,
and other members of the IPPM working group for many helpful
comments.
Contributors Contributors
The Editors would like to recognize the contributions of the The Editors would like to recognize the contributions of the
following individuals to this document. following individuals to this document.
Tianran Zhou Tianran Zhou
Huawei Huawei
156 Beiqing Rd. 156 Beiqing Rd.
Beijing 100095 Beijing
China 100095
China
Email: zhoutianran@huawei.com Email: zhoutianran@huawei.com
Zhenbin Li
Huawei
156 Beiqing Rd.
Beijing 100095
China
Email: lizhenbin@huawei.com
Ramesh Sivakolundu Zhenbin Li
Cisco Systems, Inc. Huawei
170 West Tasman Dr. 156 Beiqing Rd.
SAN JOSE, CA 95134 Beijing
U.S.A. 100095
China
Email: lizhenbin@huawei.com
Email: sramesh@cisco.com Ramesh Sivakolundu
Cisco Systems, Inc.
170 West Tasman Dr.
San Jose, CA 95134
United States of America
Email: sramesh@cisco.com
Authors' Addresses Authors' Addresses
Haoyu Song Haoyu Song
Futurewei Futurewei
2330 Central Expressway 2330 Central Expressway
Santa Clara, 95050 Santa Clara, 95050
United States of America United States of America
Email: haoyu.song@futurewei.com Email: haoyu.song@futurewei.com
Barak Gafni Barak Gafni
Nvidia Nvidia
350 Oakmead Parkway, Suite 100 Suite 100
Sunnyvale, CA 350 Oakmead Parkway
Sunnyvale, CA 94085
United States of America
Email: gbarak@nvidia.com Email: gbarak@nvidia.com
Frank Brockners Frank Brockners
Cisco Systems, Inc. Cisco Systems, Inc.
Hansaallee 249, 3rd Floor Hansaallee 249
40549 DUESSELDORF 40549 Duesseldorf
Germany Germany
Email: fbrockne@cisco.com Email: fbrockne@cisco.com
Shwetha Bhandari Shwetha Bhandari
Thoughtspot Thoughtspot
3rd Floor, Indiqube Orion, 24th Main Rd, Garden Layout, HSR Layout 3rd Floor, Indiqube Orion, Garden Layout, HSR Layout
Bangalore, KARNATAKA 560 102 24th Main Rd
Bangalore 560 102
Karnataka
India India
Email: shwetha.bhandari@thoughtspot.com Email: shwetha.bhandari@thoughtspot.com
Tal Mizrahi Tal Mizrahi
Huawei Huawei
8-2 Matam 8-2 Matam
Haifa 3190501 Haifa 3190501
Israel Israel
Email: tal.mizrahi.phd@gmail.com Email: tal.mizrahi.phd@gmail.com
 End of changes. 88 change blocks. 
298 lines changed or deleted 297 lines changed or added

This html diff was produced by rfcdiff 1.48.