rfc8911.original   rfc8911.txt 
Network Working Group M. Bagnulo Internet Engineering Task Force (IETF) M. Bagnulo
Internet-Draft UC3M Request for Comments: 8911 UC3M
Intended status: Standards Track B. Claise Category: Standards Track B. Claise
Expires: September 10, 2020 Cisco Systems, Inc. ISSN: 2070-1721 Huawei
P. Eardley P. Eardley
BT BT
A. Morton A. Morton
AT&T Labs AT&T Labs
A. Akhter A. Akhter
Consultant Consultant
March 9, 2020 November 2021
Registry for Performance Metrics Registry for Performance Metrics
draft-ietf-ippm-metric-registry-24
Abstract Abstract
This document defines the format for the IANA Performance Metrics This document defines the format for the IANA Registry of Performance
Registry. This document also gives a set of guidelines for Metrics. This document also gives a set of guidelines for Registered
Registered Performance Metric requesters and reviewers. Performance Metric requesters and reviewers.
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This is an Internet Standards Track document.
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months This document is a product of the Internet Engineering Task Force
and may be updated, replaced, or obsoleted by other documents at any (IETF). It represents the consensus of the IETF community. It has
time. It is inappropriate to use Internet-Drafts as reference received public review and has been approved for publication by the
material or to cite them other than as "work in progress." Internet Engineering Steering Group (IESG). Further information on
Internet Standards is available in Section 2 of RFC 7841.
This Internet-Draft will expire on September 10, 2020. Information about the current status of this document, any errata,
and how to provide feedback on it may be obtained at
https://www.rfc-editor.org/info/rfc8911.
Copyright Notice Copyright Notice
Copyright (c) 2020 IETF Trust and the persons identified as the Copyright (c) 2021 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Revised BSD License text as described in Section 4.e of the
the Trust Legal Provisions and are provided without warranty as Trust Legal Provisions and are provided without warranty as described
described in the Simplified BSD License. in the Revised BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 1. Introduction
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 2. Terminology
3. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3. Scope
4. Motivation for a Performance Metrics Registry . . . . . . . . 8 4. Motivations for the Performance Metrics Registry
4.1. Interoperability . . . . . . . . . . . . . . . . . . . . 8 4.1. Interoperability
4.2. Single point of reference for Performance Metrics . . . . 9 4.2. Single Point of Reference for Performance Metrics
4.3. Side benefits . . . . . . . . . . . . . . . . . . . . . . 9 4.3. Side Benefits
5. Criteria for Performance Metrics Registration . . . . . . . . 9 5. Criteria for Performance Metrics Registration
6. Performance Metric Registry: Prior attempt . . . . . . . . . 10 6. Performance Metrics Registry: Prior Attempt
6.1. Why this Attempt Should Succeed . . . . . . . . . . . . . 11 6.1. Why This Attempt Should Succeed
7. Definition of the Performance Metric Registry . . . . . . . . 11 7. Definition of the Performance Metrics Registry
7.1. Summary Category . . . . . . . . . . . . . . . . . . . . 13 7.1. Summary Category
7.1.1. Identifier . . . . . . . . . . . . . . . . . . . . . 13 7.1.1. Identifier
7.1.2. Name . . . . . . . . . . . . . . . . . . . . . . . . 13 7.1.2. Name
7.1.3. URI . . . . . . . . . . . . . . . . . . . . . . . . . 17 7.1.3. URI
7.1.4. Description . . . . . . . . . . . . . . . . . . . . . 17 7.1.4. Description
7.1.5. Reference . . . . . . . . . . . . . . . . . . . . . . 17 7.1.5. Reference
7.1.6. Change Controller . . . . . . . . . . . . . . . . . . 17 7.1.6. Change Controller
7.1.7. Version (of Registry Format) . . . . . . . . . . . . 18 7.1.7. Version (of Registry Format)
7.2. Metric Definition Category . . . . . . . . . . . . . . . 18 7.2. Metric Definition Category
7.2.1. Reference Definition . . . . . . . . . . . . . . . . 18 7.2.1. Reference Definition
7.2.2. Fixed Parameters . . . . . . . . . . . . . . . . . . 18 7.2.2. Fixed Parameters
7.3. Method of Measurement Category . . . . . . . . . . . . . 19 7.3. Method of Measurement Category
7.3.1. Reference Method . . . . . . . . . . . . . . . . . . 19 7.3.1. Reference Method
7.3.2. Packet Stream Generation . . . . . . . . . . . . . . 19 7.3.2. Packet Stream Generation
7.3.3. Traffic Filter . . . . . . . . . . . . . . . . . . . 20 7.3.3. Traffic Filter
7.3.4. Sampling Distribution . . . . . . . . . . . . . . . . 20 7.3.4. Sampling Distribution
7.3.5. Run-time Parameters . . . . . . . . . . . . . . . . . 21 7.3.5. Runtime Parameters
7.3.6. Role . . . . . . . . . . . . . . . . . . . . . . . . 22 7.3.6. Role
7.4. Output Category . . . . . . . . . . . . . . . . . . . . . 22 7.4. Output Category
7.4.1. Type . . . . . . . . . . . . . . . . . . . . . . . . 22 7.4.1. Type
7.4.2. Reference Definition . . . . . . . . . . . . . . . . 23 7.4.2. Reference Definition
7.4.3. Metric Units . . . . . . . . . . . . . . . . . . . . 23 7.4.3. Metric Units
7.4.4. Calibration . . . . . . . . . . . . . . . . . . . . . 23 7.4.4. Calibration
7.5. Administrative information . . . . . . . . . . . . . . . 24 7.5. Administrative Information
7.5.1. Status . . . . . . . . . . . . . . . . . . . . . . . 24 7.5.1. Status
7.5.2. Requester . . . . . . . . . . . . . . . . . . . . . . 24 7.5.2. Requester
7.5.3. Revision . . . . . . . . . . . . . . . . . . . . . . 24 7.5.3. Revision
7.5.4. Revision Date . . . . . . . . . . . . . . . . . . . . 24 7.5.4. Revision Date
7.6. Comments and Remarks . . . . . . . . . . . . . . . . . . 24 7.6. Comments and Remarks
8. Processes for Managing the Performance Metrics Registry Group
8. Processes for Managing the Performance Metric Registry Group 24 8.1. Adding New Performance Metrics to the Performance Metrics
8.1. Adding new Performance Metrics to the Performance Metrics Registry
Registry . . . . . . . . . . . . . . . . . . . . . . . . 25 8.2. Backward-Compatible Revision of Registered Performance
8.2. Revising Registered Performance Metrics . . . . . . . . . 26 Metrics
8.3. Deprecating Registered Performance Metrics . . . . . . . 28 8.3. Non-Backward-Compatible Deprecation of Registered
9. Security considerations . . . . . . . . . . . . . . . . . . . 28 Performance Metrics
10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 29 8.4. Obsolete Registry Entries
10.1. Registry Group . . . . . . . . . . . . . . . . . . . . . 29 8.5. Registry Format Version and Future Changes/Extensions
10.2. Performance Metric Name Elements . . . . . . . . . . . . 29 9. Security Considerations
10.3. New Performance Metrics Registry . . . . . . . . . . . . 30 10. IANA Considerations
11. Blank Registry Template . . . . . . . . . . . . . . . . . . . 32 10.1. Registry Group
11.1. Summary . . . . . . . . . . . . . . . . . . . . . . . . 32 10.2. Performance Metrics Name Elements
11.1.1. ID (Identifier) . . . . . . . . . . . . . . . . . . 32 10.3. New Performance Metrics Registry
11.1.2. Name . . . . . . . . . . . . . . . . . . . . . . . . 32 11. Blank Registry Template
11.1.3. URI . . . . . . . . . . . . . . . . . . . . . . . . 32 11.1. Summary
11.1.4. Description . . . . . . . . . . . . . . . . . . . . 32 11.1.1. ID (Identifier)
11.1.5. Change Controller . . . . . . . . . . . . . . . . . 32 11.1.2. Name
11.1.6. Version (of Registry Format) . . . . . . . . . . . . 32 11.1.3. URI
11.2. Metric Definition . . . . . . . . . . . . . . . . . . . 32 11.1.4. Description
11.2.1. Reference Definition . . . . . . . . . . . . . . . . 32 11.1.5. Reference
11.2.2. Fixed Parameters . . . . . . . . . . . . . . . . . . 32 11.1.6. Change Controller
11.3. Method of Measurement . . . . . . . . . . . . . . . . . 33 11.1.7. Version (of Registry Format)
11.3.1. Reference Method . . . . . . . . . . . . . . . . . . 33 11.2. Metric Definition
11.3.2. Packet Stream Generation . . . . . . . . . . . . . . 33 11.2.1. Reference Definition
11.3.3. Traffic Filtering (observation) Details . . . . . . 33 11.2.2. Fixed Parameters
11.3.4. Sampling Distribution . . . . . . . . . . . . . . . 33 11.3. Method of Measurement
11.3.5. Run-time Parameters and Data Format . . . . . . . . 33 11.3.1. Reference Method
11.3.6. Roles . . . . . . . . . . . . . . . . . . . . . . . 33 11.3.2. Packet Stream Generation
11.4. Output . . . . . . . . . . . . . . . . . . . . . . . . . 33 11.3.3. Traffic Filtering (Observation) Details
11.4.1. Type . . . . . . . . . . . . . . . . . . . . . . . . 34 11.3.4. Sampling Distribution
11.4.2. Reference Definition . . . . . . . . . . . . . . . . 34 11.3.5. Runtime Parameters and Data Format
11.4.3. Metric Units . . . . . . . . . . . . . . . . . . . . 34 11.3.6. Roles
11.4.4. Calibration . . . . . . . . . . . . . . . . . . . . 34 11.4. Output
11.5. Administrative items . . . . . . . . . . . . . . . . . . 34 11.4.1. Type
11.5.1. Status . . . . . . . . . . . . . . . . . . . . . . . 34 11.4.2. Reference Definition
11.5.2. Requester . . . . . . . . . . . . . . . . . . . . . 34 11.4.3. Metric Units
11.5.3. Revision . . . . . . . . . . . . . . . . . . . . . . 34 11.4.4. Calibration
11.5.4. Revision Date . . . . . . . . . . . . . . . . . . . 34 11.5. Administrative Items
11.6. Comments and Remarks . . . . . . . . . . . . . . . . . . 34 11.5.1. Status
12. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 34 11.5.2. Requester
13. References . . . . . . . . . . . . . . . . . . . . . . . . . 35 11.5.3. Revision
13.1. Normative References . . . . . . . . . . . . . . . . . . 35 11.5.4. Revision Date
13.2. Informative References . . . . . . . . . . . . . . . . . 36 11.6. Comments and Remarks
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 37 12. References
12.1. Normative References
12.2. Informative References
Acknowledgments
Authors' Addresses
1. Introduction 1. Introduction
The IETF specifies and uses Performance Metrics of protocols and The IETF specifies and uses Performance Metrics of protocols and
applications transported over its protocols. Performance metrics are applications transported over its protocols. Performance Metrics are
important part of network operations using IETF protocols, and an important part of network operations using IETF protocols, and
[RFC6390] specifies guidelines for their development. [RFC6390] specifies guidelines for their development.
The definition and use of Performance Metrics in the IETF has been The definition and use of Performance Metrics in the IETF have been
fostered in various working groups (WG), most notably: fostered in various working groups (WGs). Most notably:
The "IP Performance Metrics" (IPPM) WG is the WG primarily * The "IP Performance Metrics" (IPPM) WG is the WG primarily
focusing on Performance Metrics definition at the IETF. focusing on Performance Metrics definition at the IETF.
The "Benchmarking Methodology" WG (BMWG) defines many Performance * The "Benchmarking Methodology" WG (BMWG) defines many Performance
Metrics for use in laboratory benchmarking of inter-networking Metrics for use in laboratory benchmarking of internetworking
technologies. technologies.
The "Metric Blocks for use with RTCP's Extended Report Framework" * The "Metric Blocks for use with RTCP's Extended Report Framework"
(XRBLOCK) WG (concluded) specified many Performance Metrics (XRBLOCK) WG (concluded) specified many Performance Metrics
related to "RTP Control Protocol Extended Reports (RTCP XR)" related to "RTP Control Protocol Extended Reports (RTCP XR)"
[RFC3611], which establishes a framework to allow new information [RFC3611], which establishes a framework to allow new information
to be conveyed in RTCP, supplementing the original report blocks to be conveyed in RTCP, supplementing the original report blocks
defined in "RTP: A Transport Protocol for Real-Time Applications", defined in "RTP: A Transport Protocol for Real-Time Applications"
[RFC3550]. [RFC3550].
The "IP Flow Information eXport" (IPFIX) concluded WG specified an * The "IP Flow Information eXport" (IPFIX) WG (concluded) specified
IANA process for new Information Elements. Some Performance an Internet Assigned Numbers Authority (IANA) process for new
Metrics related Information Elements are proposed on regular Information Elements. Some Information Elements related to
basis. Performance Metrics are proposed on a regular basis.
The "Performance Metrics for Other Layers" (PMOL) a concluded WG * The "Performance Metrics for Other Layers" (PMOL) WG (concluded)
defined some Performance Metrics related to Session Initiation defined some Performance Metrics related to Session Initiation
Protocol (SIP) voice quality [RFC6035]. Protocol (SIP) voice quality [RFC6035].
It is expected that more Performance Metrics will be defined in the It is expected that more Performance Metrics will be defined in the
future, not only IP-based metrics, but also metrics which are future -- not only IP-based metrics but also metrics that are
protocol-specific and application-specific. protocol specific and application specific.
Despite the importance of Performance Metrics, there are two related Despite the importance of Performance Metrics, there are two related
problems for the industry. First, ensuring that when one party problems for the industry:
requests another party to measure (or report or in some way act on) a
particular Performance Metric, then both parties have exactly the
same understanding of what Performance Metric is being referred to.
Second, discovering which Performance Metrics have been specified, to
avoid developing a new Performance Metric that is very similar, but
not quite inter-operable. These problems can be addressed by
creating a registry of performance metrics. The usual way in which
the IETF organizes registries is with Internet Assigned Numbers
Authority (IANA), and there is currently no Performance Metrics
Registry maintained by the IANA.
This document requests that IANA create and maintain a Performance * First, ensuring that when one party requests that another party
measure (or report or in some way act on) a particular Performance
Metric, both parties have exactly the same understanding of what
Performance Metric is being referred to.
* Second, discovering which Performance Metrics have been specified,
to avoid developing a new Performance Metric that is very similar
but not quite interoperable.
These problems can be addressed by creating a Registry for
Performance Metrics with the Internet Assigned Numbers Authority
(IANA). As such, this document defines the new IANA Registry for
Performance Metrics.
Per this document, IANA has created and now maintains the Performance
Metrics Registry, according to the maintenance procedures and the Metrics Registry, according to the maintenance procedures and the
Performance Metrics Registry format defined in this memo. The format defined in the sections below. The resulting Performance
resulting Performance Metrics Registry is for use by the IETF and Metrics Registry is for use by the IETF and others. Although the
others. Although the Registry formatting specifications herein are Registry formatting specifications herein are primarily for Registry
primarily for registry creation by IANA, any other organization that creation by IANA, any other organization that wishes to create a
wishes to create a performance metrics registry may use the same Performance Metrics Registry may use the same formatting
formatting specifications for their purposes. The authors make no specifications for their purposes. The authors make no guarantee of
guarantee of the registry format's applicability to any possible set the Registry format's applicability to any possible set of
of Performance Metrics envisaged by other organizations, but Performance Metrics envisaged by other organizations, but we
encourage others to apply it. In the remainder of this document, encourage others to apply it. In the remainder of this document,
unless we explicitly say otherwise, we will refer to the IANA- unless we explicitly say otherwise, we will refer to the IANA-
maintained Performance Metrics Registry as simply the Performance maintained Performance Metrics Registry as simply the Performance
Metrics Registry. Metrics Registry.
2. Terminology 2. Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP "OPTIONAL" in this document are to be interpreted as described in
14 [RFC2119] [RFC8174] when, and only when, they appear in all BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all
capitals, as shown here. capitals, as shown here.
Performance Metric: A Performance Metric is a quantitative measure Performance Metric: A quantitative measure of performance, targeted
of performance, targeted to an IETF-specified protocol or targeted to an IETF-specified protocol or targeted to an application
to an application transported over an IETF-specified protocol. transported over an IETF-specified protocol. Examples of
Examples of Performance Metrics are the FTP response time for a Performance Metrics are the FTP response time for a complete file
complete file download, the DNS response time to resolve the IP download, the DNS Response time to resolve the IP address(es), a
address(es), a database logging time, etc. This definition is database logging time, etc. This definition is consistent with
consistent with the definition of metric in [RFC2330] and broader the definition of a metric in [RFC2330] and broader than the
than the definition of performance metric in [RFC6390]. definition of a Performance Metric in [RFC6390].
Registered Performance Metric: A Registered Performance Metric is a Registered Performance Metric: A Performance Metric expressed as an
Performance Metric expressed as an entry in the Performance entry in the Performance Metrics Registry, administered by IANA.
Metrics Registry, administered by IANA. Such a performance metric Such a Performance Metric has met all of the Registry review
has met all the registry review criteria defined in this document criteria defined in this document in order to be included in the
in order to be included in the registry. Registry.
Performance Metrics Registry: The IANA registry containing Performance Metrics Registry: The IANA Registry containing
Registered Performance Metrics. Registered Performance Metrics.
Proprietary Registry: A set of metrics that are registered in a Proprietary Registry: A set of metrics that are registered in a
proprietary registry, as opposed to Performance Metrics Registry. proprietary Registry, as opposed to the Performance Metrics
Registry.
Performance Metrics Experts: The Performance Metrics Experts is a Performance Metrics Experts: A group of designated experts [RFC8126]
group of designated experts [RFC8126] selected by the IESG to selected by the IESG to validate the Performance Metrics before
validate the Performance Metrics before updating the Performance updating the Performance Metrics Registry. The Performance
Metrics Registry. The Performance Metrics Experts work closely Metrics Experts work closely with IANA.
with IANA.
Parameter: A Parameter is an input factor defined as a variable in Parameter: An input factor defined as a variable in the definition
the definition of a Performance Metric. A Parameter is a of a Performance Metric. A Parameter is a numerical or other
numerical or other specified factor forming one of a set that specified factor forming one of a set that defines a metric or
defines a metric or sets the conditions of its operation. All sets the conditions of its operation. All Parameters must be
Parameters must be known in order to make a measurement using a known in order to make a measurement using a metric and interpret
metric and interpret the results. There are two types of the results. There are two types of Parameters: Fixed and
Parameters: Fixed and Run-time parameters. For the Fixed Runtime. For the Fixed Parameters, the value of the variable is
Parameters, the value of the variable is specified in the specified in the Performance Metrics Registry Entry and different
Performance Metrics Registry entry and different Fixed Parameter Fixed Parameter values results in different Registered Performance
values results in different Registered Performance Metrics. For Metrics. For the Runtime Parameters, the value of the variable is
the Run-time Parameters, the value of the variable is defined when defined when the Metric Measurement Method is executed and a given
the metric measurement method is executed and a given Registered Registered Performance Metric supports multiple values for the
Performance Metric supports multiple values for the parameter. Parameter. Although Runtime Parameters do not change the
Although Run-time Parameters do not change the fundamental nature fundamental nature of the Performance Metric's definition, some
of the Performance Metric's definition, some have substantial have substantial influence on the network property being assessed
influence on the network property being assessed and and interpretation of the results.
interpretation of the results.
Note: Consider the case of packet loss in the following two | Note: Consider the case of packet loss in the following two
Active Measurement Method cases. The first case is packet loss | Active Measurement Method cases. The first case is packet loss
as background loss where the Run-time Parameter set includes a | as background loss where the Runtime Parameter set includes a
very sparse Poisson stream, and only characterizes the times | very sparse Poisson stream and only characterizes the times
when packets were lost. Actual user streams likely see much | when packets were lost. Actual user streams likely see much
higher loss at these times, due to tail drop or radio errors. | higher loss at these times, due to tail drop or radio errors.
The second case is packet loss as inverse of throughput where | The second case is packet loss ratio as the complimentary
the Run-time Parameter set includes a very dense, bursty | probability of delivery ratio where the Runtime Parameter set
stream, and characterizes the loss experienced by a stream that | includes a very dense, bursty stream, and characterizes the
approximates a user stream. These are both "loss metrics", but | loss experienced by a stream that approximates a user stream.
the difference in interpretation of the results is highly | These are both "Loss metrics", but the difference in
dependent on the Run-time Parameters (at least), to the extreme | interpretation of the results is highly dependent on the
where we are actually using loss to infer its compliment: | Runtime Parameters (at least), to the extreme where we are
delivered throughput. | actually using loss ratio to infer its complimentary
| probability: delivery ratio.
Active Measurement Method: Methods of Measurement conducted on Active Measurement Methods: Methods of Measurement conducted on
traffic which serves only the purpose of measurement and is traffic that serves only the purpose of measurement and is
generated for that reason alone, and whose traffic characteristics generated for that reason alone, and whose traffic characteristics
are known a priori. The complete definition of Active Methods is are known a priori. The complete definition of Active Methods is
specified in section 3.4 of[RFC7799]. Examples of Active specified in Section 3.4 of [RFC7799]. Examples of Active
Measurement Methods are the measurement methods for the One way Measurement Methods are the Measurement Methods for the one-way
delay metric defined in [RFC7679] and the one for round trip delay delay metric defined in [RFC7679] and the round-trip delay metric
defined in [RFC2681]. defined in [RFC2681].
Passive Measurement Method: Methods of Measurement conducted on Passive Measurement Methods: Methods of Measurement conducted on
network traffic, generated either from the end users or from network traffic, generated by either (1) the end users or
network elements that would exist regardless whether the (2) network elements that would exist regardless of whether the
measurement was being conducted or not. The complete definition measurement was being conducted or not. The complete definition
of Passive Methods is specified in section 3.6 of [RFC7799]. One of Passive Methods is specified in Section 3.6 of [RFC7799]. One
characteristic of Passive Measurement Methods is that sensitive characteristic of Passive Measurement Methods is that sensitive
information may be observed, and as a consequence, stored in the information may be observed and, as a consequence, stored in the
measurement system. measurement system.
Hybrid Measurement Method: Hybrid Methods are Methods of Measurement Hybrid Measurement Methods: Methods of Measurement that use a
that use a combination of Active Methods and Passive Methods, to combination of Active Methods and Passive Methods, to assess
assess Active Metrics, Passive Metrics, or new metrics derived Active Metrics, Passive Metrics, or new metrics derived from the
from the a priori knowledge and observations of the stream of a priori knowledge and observations of the stream of interest.
interest. The complete definition of Hybrid Methods is specified The complete definition of Hybrid Methods is specified in
in section 3.8 of [RFC7799]. Section 3.8 of [RFC7799].
3. Scope 3. Scope
This document is intended for two different audiences: This document is intended for two different audiences:
1. For those defining new Registered Performance Metrics, it 1. For those preparing a candidate Performance Metric, it provides
provides specifications and best practices to be used in deciding criteria that the proposal SHOULD meet (see Section 5). It also
which Registered Performance Metrics are useful for a measurement provides instructions for writing the text for each column of the
study, instructions for writing the text for each column of the candidate Performance Metric and the references required for the
Registered Performance Metrics, and information on the supporting new Performance Metrics Registry Entry (up to and including the
documentation required for the new Performance Metrics Registry publication of one or more immutable documents such as an RFC)
entry (up to and including the publication of one or more (see Section 7).
immutable documents such as an RFC).
2. For the appointed Performance Metrics Experts and for IANA 2. For the appointed Performance Metrics Experts and for IANA
personnel administering the new IANA Performance Metrics personnel administering the new IANA Performance Metrics
Registry, it defines a set of acceptance criteria against which Registry, it defines a set of acceptance criteria against which a
these proposed Registered Performance Metrics should be candidate Registered Performance Metric should be evaluated, and
evaluated. requirements for the composition of a candidate Performance
Metric Registry Entry.
In addition, this document may be useful for other organizations who Other organizations that standardize performance metrics are
are defining a Performance Metric registry of their own, and may re- encouraged to use the process defined in this memo to propose a
use the features of the Performance Metrics Registry defined in this candidate Registered Performance Metric. In addition, this document
document. may be useful for other organizations who are defining a Performance
Metrics Registry of their own and may reuse the features of the
Performance Metrics Registry defined in this document.
This Performance Metrics Registry is applicable to Performance This Performance Metrics Registry is applicable to Performance
Metrics issued from Active Measurement, Passive Measurement, and any Metrics derived from Active Measurement, Passive Measurement, and any
other form of Performance Metric. This registry is designed to other form of Performance Metric. This Registry is designed to
encompass Performance Metrics developed throughout the IETF and encompass Performance Metrics developed throughout the IETF and
especially for the technologies specified in the following working especially for the technologies specified in the following working
groups: IPPM, XRBLOCK, IPFIX, and BMWG. This document analyzes a groups: IPPM, XRBLOCK, IPFIX, and BMWG. This document analyzes a
prior attempt to set up a Performance Metrics Registry, and the prior attempt to set up a Performance Metrics Registry and the
reasons why this design was inadequate [RFC6248]. Finally, this reasons why this design was inadequate [RFC6248].
document gives a set of guidelines for requesters and expert
reviewers of candidate Registered Performance Metrics.
This document makes no attempt to populate the Performance Metrics [RFC8912] populates the new Registry with the initial set of entries.
Registry with initial entries; the related memo
[I-D.ietf-ippm-initial-registry] proposes the initial set of regsitry
entries.
4. Motivation for a Performance Metrics Registry 4. Motivations for the Performance Metrics Registry
In this section, we detail several motivations for the Performance In this section, we detail several motivations for the Performance
Metrics Registry. Metrics Registry.
4.1. Interoperability 4.1. Interoperability
As with any IETF registry, the primary intention is to manage As with any IETF Registry, the primary intention is to manage the
registration of identifiers for use within one or more protocols. In registration of Identifiers for use within one or more protocols. In
the particular case of the Performance Metrics Registry, there are the particular case of the Performance Metrics Registry, there are
two types of protocols that will use the Performance Metrics in the two types of protocols that will use the Performance Metrics in the
Performance Metrics Registry during their operation (by referring to Performance Metrics Registry during their operation (by referring to
the Index values): the index values):
o Control protocol: This type of protocol used to allow one entity Control Protocol: This type of protocol is used to allow one entity
to request another entity to perform a measurement using a to request that another entity perform a measurement using a
specific metric defined by the Performance Metrics Registry. One specific metric defined by the Performance Metrics Registry. One
particular example is the LMAP framework [RFC7594]. Using the particular example is the Large-scale Measurement of Broadband
LMAP terminology, the Performance Metrics Registry is used in the Performance (LMAP) framework [RFC7594]. Using the LMAP
LMAP Control protocol to allow a Controller to schedule a terminology, the Performance Metrics Registry is used in the LMAP
measurement task for one or more Measurement Agents. In order to Control Protocol to allow a Controller to schedule a Measurement
enable this use case, the entries of the Performance Metrics Task for one or more Measurement Agents. In order to enable this
Registry must be sufficiently defined to allow a Measurement Agent use case, the entries in the Performance Metrics Registry must be
implementation to trigger a specific measurement task upon the sufficiently defined to allow a Measurement Agent implementation
reception of a control protocol message. This requirement heavily to trigger a specific Measurement Task upon the reception of a
constrains the type of entries that are acceptable for the Control Protocol message. This requirement heavily constrains the
Performance Metrics Registry. types of entries that are acceptable for the Performance Metrics
Registry.
o Report protocol: This type of protocol is used to allow an entity Report Protocol: This type of protocol is used to allow an entity to
to report measurement results to another entity. By referencing report Measurement Results to another entity. By referencing to a
to a specific Performance Metrics Registry, it is possible to specific Registered Performance Metric, it is possible to properly
properly characterize the measurement result data being reported. characterize the Measurement Result data being reported. Using
Using the LMAP terminology, the Performance Metrics Registry is the LMAP terminology, the Performance Metrics Registry is used in
used in the Report protocol to allow a Measurement Agent to report the LMAP Report Protocol to allow a Measurement Agent to report
measurement results to a Collector. Measurement Results to a Collector.
It should be noted that the LMAP framework explicitly allows for It should be noted that the LMAP framework explicitly allows for
using not only the IANA-maintained Performance Metrics Registry but using not only the IANA-maintained Performance Metrics Registry but
also other registries containing Performance Metrics, either defined also other registries containing Performance Metrics, i.e., either
by other organizations or private ones. However, others who are (1) registries defined by other organizations or (2) private
creating Registries to be used in the context of an LMAP framework registries. However, others who are creating registries to be used
are encouraged to use the Registry format defined in this document, in the context of an LMAP framework are encouraged to use the
because this makes it easier for developers of LMAP Measurement Registry format defined in this document, because this makes it
Agents (MAs) to programmatically use information found in those other easier for developers of LMAP Measurement Agents to programmatically
Registries' entries. use information found in those other registries' entries.
4.2. Single point of reference for Performance Metrics 4.2. Single Point of Reference for Performance Metrics
A Performance Metrics Registry serves as a single point of reference A Performance Metrics Registry serves as a single point of reference
for Performance Metrics defined in different working groups in the for Performance Metrics defined in different working groups in the
IETF. As we mentioned earlier, there are several WGs that define IETF. As we mentioned earlier, there are several working groups that
Performance Metrics in the IETF and it is hard to keep track of all define Performance Metrics in the IETF, and it is hard to keep track
them. This results in multiple definitions of similar Performance of all of them. This results in multiple definitions of similar
Metrics that attempt to measure the same phenomena but in slightly Performance Metrics that attempt to measure the same phenomena but in
different (and incompatible) ways. Having a registry would allow the slightly different (and incompatible) ways. Having a Registry would
IETF community and others to have a single list of relevant allow the IETF community and others to have a single list of relevant
Performance Metrics defined by the IETF (and others, where Performance Metrics defined by the IETF (and others, where
appropriate). The single list is also an essential aspect of appropriate). The single list is also an essential aspect of
communication about Performance Metrics, where different entities communication about Performance Metrics, where different entities
that request measurements, execute measurements, and report the that request measurements, execute measurements, and report the
results can benefit from a common understanding of the referenced results can benefit from a common understanding of the referenced
Performance Metric. Performance Metric.
4.3. Side benefits 4.3. Side Benefits
There are a couple of side benefits of having such a registry. There are a couple of side benefits of having such a Registry.
First, the Performance Metrics Registry could serve as an inventory First, the Performance Metrics Registry could serve as an inventory
of useful and used Performance Metrics, that are normally supported of useful and used Performance Metrics that are normally supported by
by different implementations of measurement agents. Second, the different implementations of Measurement Agents. Second, the results
results of measurements using the Performance Metrics should be of measurements using the Performance Metrics should be comparable
comparable even if they are performed by different implementations even if they are performed by different implementations and in
and in different networks, as the Performance Metric is properly different networks, as the Performance Metric is properly defined.
defined. BCP 176 [RFC6576] examines whether the results produced by BCP 176 [RFC6576] examines whether the results produced by
independent implementations are equivalent in the context of independent implementations are equivalent in the context of
evaluating the completeness and clarity of metric specifications. evaluating the completeness and clarity of metric specifications.
This BCP defines the standards track advancement testing for (active) [RFC6576] is a BCP [RFC2026] that defines the Standards Track
IPPM metrics, and the same process will likely suffice to determine advancement testing for (Active) IPPM Metrics, and the same process
whether Registered Performance Metrics are sufficiently well will likely suffice to determine whether Registered Performance
specified to result in comparable (or equivalent) results. Metrics are sufficiently well specified to result in comparable (or
Registered Performance Metrics which have undergone such testing equivalent) results. If a Registered Performance Metric has
SHOULD be noted, with a reference to the test results. undergone such testing, this SHOULD be noted in "Comments and
Remarks" (see Section 7.6), with a reference to the test results.
5. Criteria for Performance Metrics Registration 5. Criteria for Performance Metrics Registration
It is neither possible nor desirable to populate the Performance It is neither possible nor desirable to populate the Performance
Metrics Registry with all combinations of Parameters of all Metrics Registry with all combinations of Parameters of all
Performance Metrics. The Registered Performance Metrics SHOULD be: Performance Metrics. A Registered Performance Metric SHOULD be:
1. interpretable by the user. 1. Interpretable by the human user.
2. implementable by the software or hardware designer, 2. Implementable by the software or hardware designer.
3. deployable by network operators, 3. Deployable by network operators.
4. accurate in terms of producing equivalent results, and for 4. Accurate in terms of producing equivalent results, and for
interoperability and deployment across vendors, interoperability and deployment across vendors.
5. Operationally useful, so that it has significant industry 5. Operationally useful, so that it has significant industry
interest and/or has seen deployment, interest and/or has seen deployment.
6. Sufficiently tightly defined, so that different values for the 6. Sufficiently tightly defined, so that different values for the
Run-time Parameters does not change the fundamental nature of the Runtime Parameters do not change the fundamental nature of the
measurement, nor change the practicality of its implementation. measurement or change the practicality of its implementation.
In essence, there needs to be evidence that a candidate Registered In essence, there needs to be evidence that (1) a candidate
Performance Metric has significant industry interest, or has seen Registered Performance Metric has significant industry interest or
deployment, and there is agreement that the candidate Registered has seen deployment and (2) there is agreement that the candidate
Performance Metric serves its intended purpose. Registered Performance Metric serves its intended purpose.
6. Performance Metric Registry: Prior attempt 6. Performance Metrics Registry: Prior Attempt
There was a previous attempt to define a metric registry RFC 4148 There was a previous attempt to define a Metrics Registry [RFC4148].
[RFC4148]. However, it was obsoleted by RFC 6248 [RFC6248] because However, it was obsoleted by [RFC6248] because it was "found to be
it was "found to be insufficiently detailed to uniquely identify IPPM insufficiently detailed to uniquely identify IPPM metrics... [there
metrics... [there was too much] variability possible when was too much] variability possible when characterizing a metric
characterizing a metric exactly" which led to the RFC4148 registry exactly", which led to the IPPM Metrics Registry defined in [RFC4148]
having "very few users, if any". having "very few users, if any."
A couple of interesting additional quotes from RFC 6248 [RFC6248] Three interesting additional quotes from [RFC6248] might help to
might help to understand the issues related to that registry. understand the issues related to that registry.
1. "It is not believed to be feasible or even useful to register 1. "It is not believed to be feasible or even useful to register
every possible combination of Type P, metric parameters, and every possible combination of Type P, metric parameters, and
Stream parameters using the current structure of the IPPM Metrics Stream parameters using the current structure of the IPPM Metrics
Registry." Registry."
2. "The registry structure has been found to be insufficiently 2. "The current registry structure has been found to be
detailed to uniquely identify IPPM metrics." insufficiently detailed to uniquely identify IPPM metrics."
3. "Despite apparent efforts to find current or even future users, 3. "Despite apparent efforts to find current or even future users,
no one responded to the call for interest in the RFC 4148 no one responded to the call for interest in the RFC 4148
registry during the second half of 2010." registry during the second half of 2010."
The current approach learns from this by tightly defining each The current approach learns from this by tightly defining each
Registered Performance Metric with only a few variable (Run-time) Registered Performance Metric with only a few variable (Runtime)
Parameters to be specified by the measurement designer, if any. The Parameters to be specified by the measurement designer, if any. The
idea is that entries in the Performance Metrics Registry stem from idea is that entries in the Performance Metrics Registry stem from
different measurement methods which require input (Run-time) different Measurement Methods that require input (Runtime) Parameters
parameters to set factors like source and destination addresses to set factors like Source and Destination addresses (which do not
(which do not change the fundamental nature of the measurement). The change the fundamental nature of the measurement). The downside of
downside of this approach is that it could result in a large number this approach is that it could result in a large number of entries in
of entries in the Performance Metrics Registry. There is agreement the Performance Metrics Registry. There is agreement that less is
that less is more in this context - it is better to have a reduced more in this context -- it is better to have a reduced set of useful
set of useful metrics rather than a large set of metrics, some with metrics rather than a large set of metrics, some with questionable
with questionable usefulness. usefulness.
6.1. Why this Attempt Should Succeed 6.1. Why This Attempt Should Succeed
As mentioned in the previous section, one of the main issues with the As mentioned in the previous section, one of the main issues with the
previous registry was that the metrics contained in the registry were previous Registry was that the metrics contained in the Registry were
too generic to be useful. This document specifies stricter criteria too generic to be useful. This document specifies stricter criteria
for performance metric registration (see section 5), and imposes a for Performance Metric registration (see Section 5) and imposes a
group of Performance Metrics Experts that will provide guidelines to group of Performance Metrics Experts that will provide guidelines to
assess if a Performance Metric is properly specified. assess if a Performance Metric is properly specified.
Another key difference between this attempt and the previous one is Another key difference between this attempt and the previous one is
that in this case there is at least one clear user for the that in this case there is at least one clear user for the
Performance Metrics Registry: the LMAP framework and protocol. Performance Metrics Registry: the LMAP framework and protocol.
Because the LMAP protocol will use the Performance Metrics Registry Because the LMAP protocol will use the Performance Metrics Registry
values in its operation, this actually helps to determine if a metric values in its operation, this actually helps to determine if a metric
is properly defined. In particular, since we expect that the LMAP is properly defined -- in particular, since we expect that the LMAP
control protocol will enable a controller to request a measurement Control Protocol will enable a Controller to request that a
agent to perform a measurement using a given metric by embedding the Measurement Agent perform a measurement using a given metric by
Performance Metrics Registry identifier in the protocol. Such a embedding the Performance Metrics Registry Identifier in the
metric and method are properly specified if they are defined well- protocol. Such a metric and method are properly specified if they
enough so that it is possible (and practical) to implement them in are defined well enough so that it is possible (and practical) to
the measurement agent. This was the failure of the previous attempt: implement them in the Measurement Agent. This was the failure of the
a registry entry with an undefined Type-P (section 13 of RFC 2330 previous attempt: a Registry Entry with an undefined Type-P
[RFC2330]) allows implementation to be ambiguous. (Section 13 of [RFC2330]) allows measurement results to vary
significantly.
7. Definition of the Performance Metric Registry 7. Definition of the Performance Metrics Registry
This Performance Metrics Registry is applicable to Performance This Performance Metrics Registry is applicable to Performance
Metrics used for Active Measurement, Passive Measurement, and any Metrics used for Active Measurement, Passive Measurement, and any
other form of Performance Measurement. Each category of measurement other form of Performance Measurement. Each category of measurement
has unique properties, so some of the columns defined below are not has unique properties, so some of the columns defined below are not
applicable for a given metric category. In this case, the column(s) applicable for a given metric category. In this case, the column(s)
SHOULD be populated with the "NA" value (Non Applicable). However, SHOULD be populated with the "N/A" value (Not Applicable). However,
the "NA" value MUST NOT be used by any metric in the following the "N/A" value MUST NOT be used by any metric in the following
columns: Identifier, Name, URI, Status, Requester, Revision, Revision columns: Identifier, Name, URI, Status, Requester, Revision, Revision
Date, Description. In the future, a new category of metrics could Date, Description. In the future, a new category of metrics could
require additional columns, and adding new columns is a recognized require additional columns, and adding new columns is a recognized
form of registry extension. The specification defining the new form of Registry extension. The specification defining the new
column(s) MUST give general guidelines for populating the new column(s) MUST give general guidelines for populating the new
column(s) for existing entries. column(s) for existing entries.
The columns of the Performance Metrics Registry are defined below. The columns of the Performance Metrics Registry are defined below.
The columns are grouped into "Categories" to facilitate the use of The columns are grouped into "Categories" to facilitate the use of
the registry. Categories are described at the 7.x heading level, and the Registry. Categories are described at the "Section 7.x" heading
columns are at the 7.x.y heading level. The Figure below illustrates level, and columns are described at the "Section 7.x.y" heading
this organization. An entry (row) therefore gives a complete level. The figure below illustrates this organization. An entry
description of a Registered Performance Metric. (row) therefore gives a complete description of a Registered
Performance Metric.
Each column serves as a check-list item and helps to avoid omissions Each column serves as a checklist item and helps to avoid omissions
during registration and expert review. during registration and Expert Review [RFC8126].
======================================================================= Registry Categories and Columns are shown below in this format:
Legend:
Registry Categories and Columns are shown below as:
Category
------------------...
Column | Column |...
=======================================================================
Summary
Identifier | Name | URI | Desc. | Reference | Change Controller | Ver |
Metric Definition Category
Reference Definition | Fixed Parameters | ------------------...
Column | Column |...
Method of Measurement Summary
Reference | Packet | Traffic | Sampling | Run-time | Role | ----------------------------------------------------------------
Method | Stream | Filter | Distribution | Parameters | | Identifier | Name | URI | Desc. | Reference | Change | Ver |
| Generation | | | | | | Controller |
Output
Type | Reference | Units | Calibration |
| Definition | | |
Administrative Information Metric Definition
Status |Requester | Rev | Rev.Date | -----------------------------------------
Reference Definition | Fixed Parameters |
Comments and Remarks Method of Measurement
---------------------------------------------------------------------
Reference | Packet | Traffic | Sampling | Runtime | Role |
Method | Stream | Filter | Distribution | Parameters | |
| Generation |
Output
-----------------------------------------
Type | Reference | Units | Calibration |
| Definition | | |
Administrative Information
-------------------------------------
Status |Requester | Rev | Rev. Date |
Comments and Remarks
--------------------
There is a blank template of the Registry template provided in There is a blank template of the Registry template provided in
Section 11 of this memo. Section 11 of this memo.
7.1. Summary Category 7.1. Summary Category
7.1.1. Identifier 7.1.1. Identifier
A numeric identifier for the Registered Performance Metric. This This column provides a numeric Identifier for the Registered
identifier MUST be unique within the Performance Metrics Registry. Performance Metric. The Identifier of each Registered Performance
Metric MUST be unique. Note that revising a Metric according to the
process in Section 8.2 creates a new entry in the Performance Metrics
Registry with the same identifier.
The Registered Performance Metric unique identifier is an unbounded The Registered Performance Metric unique Identifier is an unbounded
integer (range 0 to infinity). integer (range 0 to infinity).
The Identifier 0 should be Reserved. The Identifier values from The Identifier 0 should be Reserved. The Identifier values from
64512 to 65536 are reserved for private or experimental use, and the 64512 to 65535 are reserved for private or experimental use, and the
user may encounter overlapping uses. user may encounter overlapping uses.
When adding newly Registered Performance Metrics to the Performance When adding new Registered Performance Metrics to the Performance
Metrics Registry, IANA SHOULD assign the lowest available identifier Metrics Registry, IANA SHOULD assign the lowest available Identifier
to the new Registered Performance Metric. to the new Registered Performance Metric.
If a Performance Metrics Expert providing review determines that If a Performance Metrics Expert providing review determines that
there is a reason to assign a specific numeric identifier, possibly there is a reason to assign a specific numeric Identifier, possibly
leaving a temporary gap in the numbering, then the Performance Expert leaving a temporary gap in the numbering, then the Performance
SHALL inform IANA of this decision. Metrics Expert SHALL inform IANA of this decision.
7.1.2. Name 7.1.2. Name
As the name of a Registered Performance Metric is the first thing a As the Name of a Registered Performance Metric is the first thing a
potential human implementor will use when determining whether it is potential human implementer will use when determining whether it is
suitable for their measurement study, it is important to be as suitable for their measurement study, it is important to be as
precise and descriptive as possible. In future, users will review precise and descriptive as possible. In the future, users will
the names to determine if the metric they want to measure has already review the Names to determine if the metric they want to measure has
been registered, or if a similar entry is available as a basis for already been registered, or if a similar entry is available, as a
creating a new entry. basis for creating a new entry.
Names are composed of the following elements, separated by an Names are composed of the following elements, separated by an
underscore character "_": underscore character "_":
MetricType_Method_SubTypeMethod_... Spec_Units_Output MetricType_Method_SubTypeMethod_... Spec_Units_Output
o MetricType: a combination of the directional properties and the MetricType: A combination of the directional properties and the
metric measured, such as and not limited to: metric measured, such as and not limited to:
RTDelay (Round Trip Delay) +-----------+--------------------------------------+
| RTDelay | Round-Trip Delay |
RTDNS (Response Time Domain Name Service) +-----------+--------------------------------------+
| RTDNS | Response Time Domain Name Service |
RLDNS (Response Loss Domain Name Service) +-----------+--------------------------------------+
| RLDNS | Response Loss Domain Name Service |
OWDelay (One Way Delay) +-----------+--------------------------------------+
RTLoss (Round Trip Loss) | OWDelay | One-Way Delay |
+-----------+--------------------------------------+
OWLoss (One Way Loss) | RTLoss | Round-Trip Loss |
+-----------+--------------------------------------+
OWPDV (One Way Packet Delay Variation) | OWLoss | One-Way Loss |
+-----------+--------------------------------------+
OWIPDV (One Way Inter-Packet Delay Variation) | OWPDV | One-Way Packet Delay Variation |
+-----------+--------------------------------------+
OWReorder (One Way Packet Reordering) | OWIPDV | One-Way Inter-Packet Delay Variation |
+-----------+--------------------------------------+
OWDuplic (One Way Packet Duplication) | OWReorder | One-Way Packet Reordering |
+-----------+--------------------------------------+
OWBTC (One Way Bulk Transport Capacity) | OWDuplic | One-Way Packet Duplication |
+-----------+--------------------------------------+
OWMBM (One Way Model Based Metric) | OWBTC | One-Way Bulk Transport Capacity |
+-----------+--------------------------------------+
SPMonitor (Single Point Monitor) | OWMBM | One-Way Model-Based Metric |
+-----------+--------------------------------------+
| SPMonitor | Single-Point Monitor |
+-----------+--------------------------------------+
| MPMonitor | Multi-Point Monitor |
+-----------+--------------------------------------+
MPMonitor (Multi-Point Monitor) Table 1
o Method: One of the methods defined in [RFC7799], such as and not Method: One of the methods defined in [RFC7799], such as and not
limited to: limited to:
Active (depends on a dedicated measurement packet stream and +-------------+----------------------------------------------+
observations of the stream) | Active | depends on a dedicated measurement packet |
| | stream and observations of the stream as |
Passive (depends *solely* on observation of one or more | | described in [RFC7799] |
existing packet streams) +-------------+----------------------------------------------+
| Passive | depends *solely* on observation of one or |
HybridType1 (observations on one stream that combine both | | more existing packet streams as described in |
active and passive methods) | | [RFC7799] |
+-------------+----------------------------------------------+
HybridType2 (observations on two or more streams that combine | HybridType1 | Hybrid Type I observations on one stream |
both active and passive methods) | | that combine both Active Methods and Passive |
| | Methods as described in [RFC7799] |
Spatial (Spatial Metric of RFC5644) +-------------+----------------------------------------------+
| HybridType2 | Hybrid Type II observations on two or more |
o SubTypeMethod: One or more sub-types to further describe the | | streams that combine both Active Methods and |
features of the entry, such as and not limited to: | | Passive Methods as described in [RFC7799] |
+-------------+----------------------------------------------+
ICMP (Internet Control Message Protocol) | Spatial | spatial metric as described in [RFC5644] |
+-------------+----------------------------------------------+
IP (Internet Protocol)
DSCPxx (where xx is replaced by a Diffserv code point)
UDP (User Datagram Protocol)
TCP (Transport Control Protocol)
QUIC (QUIC transport protocol)
HS (Hand-Shake, such as TCP's 3-way HS)
Poisson (Packet generation using Poisson distribution)
Periodic (Periodic packet generation)
SendOnRcv (Sender keeps one packet in-transit by sending when Table 2
previous packet arrives)
PayloadxxxxB (where xxxx is replaced by an integer, the number SubTypeMethod: One or more subtypes to further describe the features
of octets in the Payload)) of the entry, such as and not limited to:
SustainedBurst (Capacity test, worst case) +----------------+------------------------------------------------+
| ICMP | Internet Control Message Protocol |
+----------------+------------------------------------------------+
| IP | Internet Protocol |
+----------------+------------------------------------------------+
| DSCPxx | where xx is replaced by a Diffserv code point |
+----------------+------------------------------------------------+
| UDP | User Datagram Protocol |
+----------------+------------------------------------------------+
| TCP | Transport Control Protocol |
+----------------+------------------------------------------------+
| QUIC | QUIC transport protocol |
+----------------+------------------------------------------------+
| HS | Hand-Shake, such as TCP's 3-way HS |
+----------------+------------------------------------------------+
| Poisson | packet generation using Poisson distribution |
+----------------+------------------------------------------------+
| Periodic | periodic packet generation |
+----------------+------------------------------------------------+
| SendOnRcv | sender keeps one packet in transit by sending |
| | when previous packet arrives |
+----------------+------------------------------------------------+
| PayloadxxxxB | where xxxx is replaced by an integer, the |
| | number of octets or 8-bit Bytes in the Payload |
+----------------+------------------------------------------------+
| SustainedBurst | capacity test, worst case |
+----------------+------------------------------------------------+
| StandingQueue | test of bottleneck queue behavior |
+----------------+------------------------------------------------+
StandingQueue (test of bottleneck queue behavior) Table 3
SubTypeMethod values are separated by a hyphen "-" character, SubTypeMethod values are separated by a hyphen "-" character,
which indicates that they belong to this element, and that their which indicates that they belong to this element and that their
order is unimportant when considering name uniqueness. order is unimportant when considering Name uniqueness.
o Spec: An immutable document identifier combined with a document Spec: An immutable document Identifier combined with a document
section identifier. For RFCs, this consists of the RFC number and section Identifier. For RFCs, this consists of the RFC number and
major section number that specifies this Registry entry in the major section number that specifies this Registry Entry in the
form RFCXXXXsecY, such as RFC7799sec3. Note: the RFC number is form "RFCXXXXsecY", e.g., RFC7799sec3. Note: The RFC number is
not the Primary Reference specification for the metric definition, not the primary reference specification for the metric definition
such as [RFC7679] for One-way Delay; it will contain the (e.g., [RFC7679] as the primary reference specification for one-
placeholder "RFCXXXXsecY" until the RFC number is assigned to the way delay metrics); it will contain the placeholder "RFCXXXXsecY"
specifying document, and would remain blank in private registry until the RFC number is assigned to the specifying document and
entries without a corresponding RFC. Anticipating the "RFC10K" would remain blank in Private Registry Entries without a
problem, the number of the RFC continues to replace RFCXXXX corresponding RFC. Anticipating the "RFC10K" problem, the number
regardless of the number of digits in the RFC number. of the RFC continues to replace "RFCXXXX", regardless of the
Anticipating Registry Entries from other standards bodies, the number of digits in the RFC number. Anticipating Registry Entries
form of this Name Element MUST be proposed and reviewed for from other standards bodies, the form of this Name Element MUST be
consistency and uniqueness by the Expert Reviewer. proposed and reviewed for consistency and uniqueness by the Expert
Reviewer.
o Units: The units of measurement for the output, such as and not Units: The units of measurement for the output, such as and not
limited to: limited to:
Seconds +------------+----------------------------+
| Seconds | |
Ratio (unitless) +------------+----------------------------+
Percent (value multiplied by 100%) | Ratio | unitless |
+------------+----------------------------+
Logical (1 or 0) | Percent | value multiplied by 100% |
+------------+----------------------------+
Packets | Logical | 1 or 0 |
+------------+----------------------------+
BPS (Bits per Second) | Packets | |
+------------+----------------------------+
PPS (Packets per Second) | BPS | bits per second |
+------------+----------------------------+
EventTotal (for unit-less counts) | PPS | packets per second |
+------------+----------------------------+
Multiple (more than one type of unit) | EventTotal | for unitless counts |
+------------+----------------------------+
Enumerated (a list of outcomes) | Multiple | more than one type of unit |
+------------+----------------------------+
| Enumerated | a list of outcomes |
+------------+----------------------------+
| Unitless | |
+------------+----------------------------+
Unitless Table 4
o Output: The type of output resulting from measurement, such as and Output: The type of output resulting from measurement, such as and
not limited to: not limited to:
Singleton +--------------+------------------------------------+
| Singleton | |
Raw (multiple Singletons) +--------------+------------------------------------+
| Raw | multiple singletons |
Count +--------------+------------------------------------+
| Count | |
Minimum +--------------+------------------------------------+
| Minimum | |
Maximum +--------------+------------------------------------+
| Maximum | |
Median +--------------+------------------------------------+
| Median | |
Mean +--------------+------------------------------------+
| Mean | |
95Percentile (95th Percentile) +--------------+------------------------------------+
| 95Percentile | 95th percentile |
99Percentile (99th Percentile) +--------------+------------------------------------+
| 99Percentile | 99th percentile |
StdDev (Standard Deviation) +--------------+------------------------------------+
| StdDev | standard deviation |
Variance +--------------+------------------------------------+
| Variance | |
PFI (Pass, Fail, Inconclusive) +--------------+------------------------------------+
| PFI | pass, fail, inconclusive |
FlowRecords (descriptions of flows observed) +--------------+------------------------------------+
| FlowRecords | descriptions of flows observed |
LossRatio (lost packets to total packets, <=1) +--------------+------------------------------------+
| LossRatio | lost packets to total packets, <=1 |
+--------------+------------------------------------+
An example is: Table 5
RTDelay_Active_IP-UDP-Periodic_RFCXXXXsecY_Seconds_95Percentile An example, as described in Section 4 of [RFC8912], is
as described in section 4 of [I-D.ietf-ippm-initial-registry]. RTDelay_Active_IP-UDP-Periodic_RFC8912sec4_Seconds_95Percentile
Note that private registries following the format described here Note that private registries following the format described here
SHOULD use the prefix "Priv_" on any name to avoid unintended SHOULD use the prefix "Priv_" on any Name to avoid unintended
conflicts (further considerations are described in section 10). conflicts (further considerations are described in Section 10).
Private registry entries usually have no specifying RFC, thus the Private Registry Entries usually have no specifying RFC; thus, the
Spec: element has no clear interpretation. Spec: element has no clear interpretation.
7.1.3. URI 7.1.3. URI
The URIs column MUST contain a URL [RFC3986] that uniquely identifies The URI column MUST contain a URL [RFC3986] that uniquely identifies
and locates the metric entry so it is accessible through the and locates the Metric Entry so it is accessible through the
Internet. The URL points to a file containing all the human-readable Internet. The URL points to a file containing all of the human-
information for one registry entry. The URL SHALL reference a target readable information for one Registry Entry. The URL SHALL reference
file that is preferably HTML-formatted and contains URLs to a target file that is preferably HTML-formatted and contains URLs to
referenced sections of HTML-ized RFCs, or other reference referenced sections of HTMLized RFCs, or other reference
specifications. These target files for different entries can be more specifications. These target files for different entries can be more
easily edited and re-used when preparing new entries. The exact form easily edited and reused when preparing new entries. The exact form
of the URL for each target file, and the target file itself, will be of the URL for each target file, and the target file itself, will be
determined by IANA and reside on "iana.org". The major sections of determined by IANA and reside on <https://www.iana.org/>. Section 4
[I-D.ietf-ippm-initial-registry] provide an example of a target file of [RFC8912], as well as subsequent major sections of that document,
in HTML form (sections 4 and higher). provide an example of a target file in HTML form.
7.1.4. Description 7.1.4. Description
A Registered Performance Metric description is a written A Registered Performance Metric description is a written
representation of a particular Performance Metrics Registry entry. representation of a particular Performance Metrics Registry Entry.
It supplements the Registered Performance Metric name to help It supplements the Registered Performance Metric Name to help
Performance Metrics Registry users select relevant Registered Performance Metrics Registry users select relevant Registered
Performance Metrics. Performance Metrics.
7.1.5. Reference 7.1.5. Reference
This entry gives the specification containing the candidate registry This entry gives the specification containing the candidate Registry
entry which was reviewed and agreed, if such an RFC or other Entry that was reviewed and agreed upon, if such an RFC or other
specification exists. specification exists.
7.1.6. Change Controller 7.1.6. Change Controller
This entry names the entity responsible for approving revisions to This entry names the entity responsible for approving revisions to
the registry entry, and SHALL provide contact information (for an the Registry Entry and SHALL provide contact information (for an
individual, where appropriate). individual, where appropriate).
7.1.7. Version (of Registry Format) 7.1.7. Version (of Registry Format)
This entry gives the version number for the registry format used. This column gives the version number for the Registry format used, at
Formats complying with this memo MUST use 1.0. The version number the time the Performance Metric is registered. The format complying
SHALL NOT change unless a new RFC is published that changes the with this memo MUST use 1.0. A new RFC that changes the Registry
registry format. The version number of registry entries SHALL NOT format will designate a new version number corresponding to that
change unless the registry entry is updated (following procedures in format. The version number of Registry Entries SHALL NOT change
section 8). unless the Registry Entry is updated to reflect the Registry format
(following the procedures in Section 8).
7.2. Metric Definition Category 7.2. Metric Definition Category
This category includes columns to prompt all necessary details This category includes columns to prompt all necessary details
related to the metric definition, including the immutable document related to the metric definition, including the immutable document
reference and values of input factors, called fixed parameters, which reference and values of input factors, called "Fixed Parameters",
are left open in the immutable document, but have a particular value which are left open in the immutable document but have a particular
defined by the performance metric. value defined by the Performance Metric.
7.2.1. Reference Definition 7.2.1. Reference Definition
This entry provides a reference (or references) to the relevant This entry provides a reference (or references) to the relevant
section(s) of the document(s) that define the metric, as well as any sections of the document or documents that define the metric, as well
supplemental information needed to ensure an unambiguous definition as any supplemental information needed to ensure an unambiguous
for implementations. The reference needs to be an immutable definition for implementations. A given reference needs to be an
document, such as an RFC; for other standards bodies, it is likely to immutable document, such as an RFC; for other standards bodies, it is
be necessary to reference a specific, dated version of a likely to be necessary to reference a specific, dated version of a
specification. specification.
7.2.2. Fixed Parameters 7.2.2. Fixed Parameters
Fixed Parameters are Parameters whose value must be specified in the Fixed Parameters are Parameters whose values must be specified in the
Performance Metrics Registry. The measurement system uses these Performance Metrics Registry. The measurement system uses these
values. values.
Where referenced metrics supply a list of Parameters as part of their Where referenced metrics supply a list of Parameters as part of their
descriptive template, a sub-set of the Parameters will be designated descriptive template, a subset of the Parameters will be designated
as Fixed Parameters. As an example for active metrics, Fixed as Fixed Parameters. As an example for Active Metrics, Fixed
Parameters determine most or all of the IPPM Framework convention Parameters determine most or all of the IPPM framework convention
"packets of Type-P" as described in [RFC2330], such as transport "packets of Type-P" as described in [RFC2330], such as transport
protocol, payload length, TTL, etc. An example for passive metrics protocol, payload length, TTL, etc. An example for Passive Metrics
is for RTP packet loss calculation that relies on the validation of a is for an RTP packet loss calculation that relies on the validation
packet as RTP which is a multi-packet validation controlled by of a packet as RTP, which is a multi-packet validation controlled by
MIN_SEQUENTIAL as defined by [RFC3550]. Varying MIN_SEQUENTIAL the MIN_SEQUENTIAL variable as defined by [RFC3550]. Varying
values can alter the loss report and this value could be set as a MIN_SEQUENTIAL values can alter the loss report, and this variable
Fixed Parameter. could be set as a Fixed Parameter.
Parameters MUST have well-defined names. For human readers, the Parameters MUST have well-defined names. For human readers, the
hanging indent style is preferred, and any Parameter names and hanging-indent style is preferred, and any Parameter names and
definitions that do not appear in the Reference Method Specification definitions that do not appear in the Reference Method Specification
MUST appear in this column (or Run-time Parameters column). MUST appear in this column (or the Runtime Parameters column).
Parameters MUST have a well-specified data format. Parameters MUST have a well-specified data format.
A Parameter which is a Fixed Parameter for one Performance Metrics A Parameter that is a Fixed Parameter for one Performance Metrics
Registry entry may be designated as a Run-time Parameter for another Registry Entry may be designated as a Runtime Parameter for another
Performance Metrics Registry entry. Performance Metrics Registry Entry.
7.3. Method of Measurement Category 7.3. Method of Measurement Category
This category includes columns for references to relevant sections of This category includes columns for references to relevant sections of
the immutable document(s) and any supplemental information needed to the immutable document(s) and any supplemental information needed to
ensure an unambiguous method for implementations. ensure an unambiguous method for implementations.
7.3.1. Reference Method 7.3.1. Reference Method
This entry provides references to relevant sections of immutable This entry provides references to relevant sections of immutable
documents, such as RFC(s) (for other standards bodies, it is likely documents, such as RFC(s) (for other standards bodies, it is likely
to be necessary to reference a specific, dated version of a to be necessary to reference a specific, dated version of a
specification) describing the method of measurement, as well as any specification) describing the Method of Measurement, as well as any
supplemental information needed to ensure unambiguous interpretation supplemental information needed to ensure unambiguous interpretation
for implementations referring to the immutable document text. for implementations referring to the immutable document text.
Specifically, this section should include pointers to pseudocode or Specifically, this section should include pointers to pseudocode or
actual code that could be used for an unambiguous implementation. actual code that could be used for an unambiguous implementation.
7.3.2. Packet Stream Generation 7.3.2. Packet Stream Generation
This column applies to Performance Metrics that generate traffic as This column applies to Performance Metrics that generate traffic as
part of their Measurement Method, including but not necessarily part of their Measurement Method, including, but not necessarily
limited to Active metrics. The generated traffic is referred as a limited to, Active Metrics. The generated traffic is referred to as
stream and this column describes its characteristics. a "stream", and this column describes its characteristics.
Each entry for this column contains the following information: Each entry for this column contains the following information:
o Value: The name of the packet stream scheduling discipline Value: The name of the packet stream scheduling discipline
o Reference: the specification where the parameters of the stream Reference: The specification where the Parameters of the stream are
are defined defined
The packet generation stream may require parameters such as the The packet generation stream may require Parameters such as the
average packet rate and distribution truncation value for streams average packet rate and distribution truncation value for streams
with Poisson-distributed inter-packet sending times. In case such with Poisson-distributed inter-packet sending times. If such
parameters are needed, they should be included either in the Fixed Parameters are needed, they should be included in either the Fixed
parameter column or in the run time parameter column, depending on Parameters column or the Runtime Parameters column, depending on
whether they will be fixed or will be an input for the metric. whether they will be fixed or will be an input for the metric.
The simplest example of stream specification is Singleton scheduling The simplest example of stream specification is singleton scheduling
(see [RFC2330]), where a single atomic measurement is conducted. (see [RFC2330]), where a single atomic measurement is conducted.
Each atomic measurement could consist of sending a single packet Each atomic measurement could consist of sending a single packet
(such as a DNS request) or sending several packets (for example, to (such as a DNS request) or sending several packets (for example, to
request a webpage). Other streams support a series of atomic request a web page). Other streams support a series of atomic
measurements in a "sample", with a schedule defining the timing measurements using pairs of packets, where the packet stream follows
between each transmitted packet and subsequent measurement. a schedule defining the timing between transmitted packets, and an
Principally, two different streams are used in IPPM metrics, Poisson atomic measurement assesses the reception time between successive
distributed as described in [RFC2330] and Periodic as described in packets (e.g., a measurement of Inter-Packet Delay Variation). More
[RFC3432]. Both Poisson and Periodic have their own unique complex streams and measurement relationships are possible.
parameters, and the relevant set of parameters names and values Principally, two different streams are used in IPPM Metrics:
should be included either in the Fixed Parameters column or in the (1) Poisson, distributed as described in [RFC2330] and (2) periodic,
Run-time parameter column. as described in [RFC3432]. Both Poisson and periodic have their own
unique Parameters, and the relevant set of Parameter names and values
should be included in either the Fixed Parameters column or the
Runtime Parameters column.
7.3.3. Traffic Filter 7.3.3. Traffic Filter
This column applies to Performance Metrics that observe packets This column applies to Performance Metrics that observe packets
flowing through (the device with) the measurement agent i.e. that is flowing through (the device with) the Measurement Agent, i.e.,
not necessarily addressed to the measurement agent. This includes packets that are not necessarily addressed to the Measurement Agent.
but is not limited to Passive Metrics. The filter specifies the This includes, but is not limited to, Passive Metrics. The filter
traffic that is measured. This includes protocol field values/ specifies the traffic that is measured. This includes protocol field
ranges, such as address ranges, and flow or session identifiers. values/ranges, such as address ranges, and flow or session
Identifiers.
The traffic filter itself depends on needs of the metric itself and a The Traffic Filter itself depends on the needs of the metric itself
balance of an operator's measurement needs and a user's need for and a balance of an operator's measurement needs and a user's need
privacy. Mechanics for conveying the filter criteria might be the for privacy. Mechanics for conveying the filter criteria might be
BPF (Berkley Packet Filter) or PSAMP [RFC5475] Property Match the BPF (Berkeley Packet Filter) or PSAMP (Packet Sampling) [RFC5475]
Filtering which reuses IPFIX [RFC7012]. An example BPF string for Property Match Filtering, which reuses IPFIX [RFC7012]. An example
matching TCP/80 traffic to remote destination net 192.0.2.0/24 would BPF string for matching TCP/80 traffic to remote Destination net
be "dst net 192.0.2.0/24 and tcp dst port 80". More complex filter 192.0.2.0/24 would be "dst net 192.0.2.0/24 and tcp dst port 80".
engines might be supported by the implementation that might allow for More complex filter engines may allow for matching using Deep Packet
matching using Deep Packet Inspection (DPI) technology. Inspection (DPI) technology.
The traffic filter includes the following information: The Traffic Filter includes the following information:
Type: the type of traffic filter used, e.g. BPF, PSAMP, OpenFlow Type: The type of Traffic Filter used, e.g., BPF, PSAMP, OpenFlow
rule, etc. as defined by a normative reference rule, etc., as defined by a normative reference
Value: the actual set of rules expressed Value: The actual set of rules expressed
7.3.4. Sampling Distribution 7.3.4. Sampling Distribution
The sampling distribution defines out of all the packets that match The sampling distribution defines, out of all of the packets that
the traffic filter, which one of those are actually used for the match the Traffic Filter, which one or more of those packets are
measurement. One possibility is "all" which implies that all packets actually used for the measurement. One possibility is "all", which
matching the Traffic filter are considered, but there may be other implies that all packets matching the Traffic Filter are considered,
sampling strategies. It includes the following information: but there may be other sampling strategies. It includes the
following information:
Value: the name of the sampling distribution Value: The name of the sampling distribution
Reference definition: pointer to the specification where the Reference definition: Pointer to the specification where the
sampling distribution is properly defined. sampling distribution is properly defined
The sampling distribution may require parameters. In case such The sampling distribution may require Parameters. If such Parameters
parameters are needed, they should be included either in the Fixed are needed, they should be included in either the Fixed Parameters
parameter column or in the run time parameter column, depending on column or the Runtime Parameters column, depending on whether they
whether they will be fixed or will be an input for the metric. will be fixed or will be an input for the metric.
Sampling and Filtering Techniques for IP Packet Selection are PSAMP is documented in "Sampling and Filtering Techniques for IP
documented in the PSAMP (Packet Sampling) [RFC5475], while the Packet Selection" [RFC5475], while "A Framework for Packet Selection
Framework for Packet Selection and Reporting, [RFC5474] provides more and Reporting" [RFC5474] provides more background information. The
background information. The sampling distribution parameters might sampling distribution Parameters might be expressed in terms of the
be expressed in terms of the Information Model for Packet Sampling model described in "Information Model for Packet Sampling Exports"
Exports, [RFC5477], and the Flow Selection Techniques, [RFC7014]. [RFC5477] and the process provided in "Flow Selection Techniques"
[RFC7014].
7.3.5. Run-time Parameters 7.3.5. Runtime Parameters
Run-Time Parameters are Parameters that must be determined, In contrast to the Fixed Parameters, Runtime Parameters are
configured into the measurement system, and reported with the results Parameters that do not change the fundamental nature of the
for the context to be complete. However, the values of these measurement and their values are not specified in the Performance
parameters is not specified in the Performance Metrics Registry (like Metrics Registry. They are left as variables in the Registry, as an
the Fixed Parameters), rather these parameters are listed as an aid aid to the measurement system implementer or user. Their values are
to the measurement system implementer or user (they must be left as supplied on execution, configured into the measurement system, and
variables, and supplied on execution). reported with the Measurement Results (so that the context is
complete).
Where metrics supply a list of Parameters as part of their Where metrics supply a list of Parameters as part of their
descriptive template, a sub-set of the Parameters will be designated descriptive template, a subset of the Parameters will be designated
as Run-Time Parameters. as Runtime Parameters.
Parameters MUST have well defined names. For human readers, the Parameters MUST have well-defined names. For human readers, the
hanging indent style is preferred, and the names and definitions that hanging-indent style is preferred, and the names and definitions that
do not appear in the Reference Method Specification MUST appear in do not appear in the Reference Method Specification MUST appear in
this column. this column.
A Data Format for each Run-time Parameter MUST be specified in this A data format for each Runtime Parameter MUST be specified in this
column, to simplify the control and implementation of measurement column, to simplify the control and implementation of measurement
devices. For example, parameters that include an IPv4 address can be devices. For example, Parameters that include an IPv4 address can be
encoded as a 32 bit integer (i.e. binary base64 encoded value) or ip- encoded as a 32-bit integer (i.e., a binary base64-encoded value) or
address as defined in [RFC6991]. The actual encoding(s) used must be "ip-address" as defined in [RFC6991]. The actual encoding(s) used
explicitly defined for each Run-time parameter. IPv6 addresses and must be explicitly defined for each Runtime Parameter. IPv6
options MUST be accommodated, allowing Registered Metrics to be used addresses and options MUST be accommodated, allowing Registered
in that address family. Other address families are permissable. Performance Metrics to be used in that address family. Other address
families are permissible.
Examples of Run-time Parameters include IP addresses, measurement Examples of Runtime Parameters include IP addresses, measurement
point designations, start times and end times for measurement, and point designations, start times and end times for measurement, and
other information essential to the method of measurement. other information essential to the Method of Measurement.
7.3.6. Role 7.3.6. Role
In some methods of measurement, there may be several roles defined, In some Methods of Measurement, there may be several Roles defined,
e.g., for a one-way packet delay active measurement there is one e.g., for a one-way packet delay Active Measurement, there is one
measurement agent that generates the packets and another agent that Measurement Agent that generates the packets and another Agent that
receives the packets. This column contains the name of the Role(s) receives the packets. This column contains the name of the Role(s)
for this particular entry. In the one-way delay example above, there for this particular entry. In the one-way delay example above, there
should be two entries in the Role registry column, one for each Role should be two entries in the Registry's Role column, one for each
(Source and Destination). When a measurement agent is instructed to Role (Source and Destination). When a Measurement Agent is
perform the "Source" Role for one-way delay metric, the agent knows instructed to perform the "Source" Role for the one-way delay metric,
that it is required to generate packets. The values for this field the Agent knows that it is required to generate packets. The values
are defined in the reference method of measurement (and this for this field are defined in the Reference Method of Measurement
frequently results in abbreviated role names such as "Src"). (and this frequently results in abbreviated Role names such as
"Src").
When the Role column of a registry entry defines more than one Role, When the Role column of a Registry Entry defines more than one Role,
then the Role SHALL be treated as a Run-time Parameter and supplied the Role SHALL be treated as a Runtime Parameter and supplied for
for execution. It should be noted that the LMAP framework [RFC7594] execution. It should be noted that the LMAP framework [RFC7594]
distinguishes the Role from other Run-time Parameters, and defines a distinguishes the Role from other Runtime Parameters.
special parameter "Roles" inside the registry-grouping function list
in the LMAP YANG model[RFC8194].
7.4. Output Category 7.4. Output Category
For entries which involve a stream and many singleton measurements, a For entries that involve a stream and many singleton measurements, a
statistic may be specified in this column to summarize the results to statistic may be specified in this column to summarize the results to
a single value. If the complete set of measured singletons is a single value. If the complete set of measured singletons is
output, this will be specified here. output, this will be specified here.
Some metrics embed one specific statistic in the reference metric Some metrics embed one specific statistic in the reference metric
definition, while others allow several output types or statistics. definition, while others allow several output types or statistics.
7.4.1. Type 7.4.1. Type
This column contains the name of the output type. The output type This column contains the name of the output type. The output type
defines a single type of result that the metric produces. It can be defines a single type of result that the metric produces. It can be
the raw results (packet send times and singleton metrics), or it can the raw results (packet send times and singleton metrics), or it can
be a summary statistic. The specification of the output type MUST be a summary statistic. The specification of the output type MUST
define the format of the output. In some systems, format define the format of the output. In some systems, format
specifications will simplify both measurement implementation and specifications will simplify both measurement implementation and
collection/storage tasks. Note that if two different statistics are collection/storage tasks. Note that if two different statistics are
required from a single measurement (for example, both "Xth percentile required from a single measurement (for example, both "Xth percentile
mean" and "Raw"), then a new output type must be defined ("Xth mean" and "Raw"), then a new output type must be defined ("Xth
percentile mean AND Raw"). See the Naming section above for a list percentile mean AND Raw"). See Section 7.1.2 above for a list of
of Output Types. output types.
7.4.2. Reference Definition 7.4.2. Reference Definition
This column contains a pointer to the specification(s) where the This column contains a pointer to the specification(s) where the
output type and format are defined. output type and format are defined.
7.4.3. Metric Units 7.4.3. Metric Units
The measured results must be expressed using some standard dimension The measured results must be expressed using some standard dimension
or units of measure. This column provides the units. or units of measure. This column provides the units.
When a sample of singletons (see Section 11 of[RFC2330] for When a sample of singletons (see Section 11 of [RFC2330] for
definitions of these terms) is collected, this entry will specify the definitions of these terms) is collected, this entry will specify the
units for each measured value. units for each measured value.
7.4.4. Calibration 7.4.4. Calibration
Some specifications for Methods of Measurement include the Some specifications for Methods of Measurement include the ability to
possibility to perform an error calibration. Section 3.7.3 of perform an error calibration. Section 3.7.3 of [RFC7679] is one
[RFC7679] is one example. In the registry entry, this field will example. In the Registry Entry, this field will identify a method of
identify a method of calibration for the metric, and when available, calibration for the metric, and, when available, the measurement
the measurement system SHOULD perform the calibration when requested system SHOULD perform the calibration when requested and produce the
and produce the output with an indication that it is the result of a output with an indication that it is the result of a calibration
calibration method. In-situ calibration could be enabled with an method. In-situ calibration could be enabled with an internal
internal loopback that includes as much of the measurement system as loopback that includes as much of the measurement system as possible,
possible, performs address manipulation as needed, and provides some performs address manipulation as needed, and provides some form of
form of isolation (e.g., deterministic delay) to avoid send-receive isolation (e.g., deterministic delay) to avoid send-receive interface
interface contention. Some portion of the random and systematic contention. Some portion of the random and systematic error can be
error can be characterized this way. characterized in this way.
For one-way delay measurements, the error calibration must include an For one-way delay measurements, the error calibration must include an
assessment of the internal clock synchronization with its external assessment of the internal clock synchronization with its external
reference (this internal clock is supplying timestamps for reference (this internal clock is supplying timestamps for
measurement). In practice, the time offsets of clocks at both the measurement). In practice, the time offsets of clocks at both the
source and destination are needed to estimate the systematic error Source and Destination are needed to estimate the systematic error
due to imperfect clock synchronization (the time offsets are due to imperfect clock synchronization (the time offsets are
smoothed, thus the random variation is not usually represented in the smoothed; thus, the random variation is not usually represented in
results). the results).
Both internal loopback calibration and clock synchronization can be Both internal loopback calibration and clock synchronization can be
used to estimate the *available accuracy* of the Output Metric Units. used to estimate the *available accuracy* of the Output Metric Units.
For example, repeated loopback delay measurements will reveal the For example, repeated loopback delay measurements will reveal the
portion of the Output result resolution which is the result of system portion of the output result resolution that is the result of system
noise, and thus inaccurate. noise and is thus inaccurate.
7.5. Administrative information 7.5. Administrative Information
7.5.1. Status 7.5.1. Status
The status of the specification of this Registered Performance This entry indicates the status of the specification of this
Metric. Allowed values are 'current' and 'deprecated'. All newly Registered Performance Metric. Allowed values are 'Current',
defined Information Elements have 'current' status. 'Deprecated', and 'Obsolete'. All newly defined Registered
Performance Metrics have 'Current' Status.
7.5.2. Requester 7.5.2. Requester
The requester for the Registered Performance Metric. The requester This entry indicates the requester for the Registered Performance
MAY be a document, such as RFC, or person. Metric. The requester MAY be a document (such as an RFC) or a
person.
7.5.3. Revision 7.5.3. Revision
The revision number of a Registered Performance Metric, starting at 0 This entry indicates the revision number of a Registered Performance
for Registered Performance Metrics at time of definition and Metric, starting at 0 for Registered Performance Metrics at the time
incremented by one for each revision. of definition and incremented by one for each revision. However, in
the case of a non-backward-compatible revision, see Section 8.3.
7.5.4. Revision Date 7.5.4. Revision Date
The date of acceptance or the most recent revision for the Registered This entry indicates the date of acceptance of the most recent
Performance Metric. The date SHALL be determined by IANA and the revision for the Registered Performance Metric. The date SHALL be
reviewing Performance Metrics Expert. determined by IANA and the reviewing Performance Metrics Expert.
7.6. Comments and Remarks 7.6. Comments and Remarks
Besides providing additional details which do not appear in other Besides providing additional details that do not appear in other
categories, this open Category (single column) allows for unforeseen categories, this open category (single column) allows unforeseen
issues to be addressed by simply updating this informational entry. issues to be addressed by simply updating this informational entry.
8. Processes for Managing the Performance Metric Registry Group 8. Processes for Managing the Performance Metrics Registry Group
Once a Performance Metric or set of Performance Metrics has been Once a Performance Metric or set of Performance Metrics has been
identified for a given application, candidate Performance Metrics identified for a given application, candidate Performance Metrics
Registry entry specifications prepared in accordance with Section 7 Registry Entry specifications prepared in accordance with Section 7
should be submitted to IANA to follow the process for review by the should be submitted to IANA to follow the process for review by the
Performance Metric Experts, as defined below. This process is also Performance Metrics Experts, as defined below. This process is also
used for other changes to the Performance Metrics Registry, such as used for other changes to a Performance Metrics Registry Entry, such
deprecation or revision, as described later in this section. as deprecation or revision, as described later in this section.
It is desirable that the author(s) of a candidate Performance Metrics It is desirable that the author(s) of a candidate Performance Metrics
Registry entry seek review in the relevant IETF working group, or Registry Entry seek review in the relevant IETF working group or
offer the opportunity for review on the working group mailing list. offer the opportunity for review on the working group mailing list.
8.1. Adding new Performance Metrics to the Performance Metrics Registry 8.1. Adding New Performance Metrics to the Performance Metrics Registry
Requests to add Registered Performance Metrics in the Performance Requests to add Registered Performance Metrics in the Performance
Metrics Registry SHALL be submitted to IANA, which forwards the Metrics Registry SHALL be submitted to IANA, which forwards the
request to a designated group of experts (Performance Metric Experts) request to a designated group of experts (Performance Metrics
appointed by the IESG; these are the reviewers called for by the Experts) appointed by the IESG; these are the reviewers called for by
Specification Required [RFC8126] policy defined for the Performance the Specification Required policy [RFC8126] defined for the
Metrics Registry. The Performance Metric Experts review the request Performance Metrics Registry. The Performance Metrics Experts review
for such things as compliance with this document, compliance with the request for such things as compliance with this document,
other applicable Performance Metric-related RFCs, and consistency compliance with other applicable Performance Metrics-related RFCs,
with the currently defined set of Registered Performance Metrics. and consistency with the currently defined set of Registered
The most efficient path for submission begins with preparation of an Performance Metrics. The most efficient path for submission begins
Internet Draft containing the proposed Performance Metrics Registry with preparation of an Internet-Draft containing the proposed
entry using the template in Section 11, so that the submission Performance Metrics Registry Entry using the template in Section 11,
formatting will benefit from the normal IETF Internet Draft so that the submission formatting will benefit from the normal IETF
submission processing (including HTML-ization). Internet-Draft submission processing (including HTMLization).
Submission to IANA may be during IESG review (leading to IETF Submission to IANA may be during IESG review (leading to IETF
Standards Action), where an Internet Draft proposes one or more Standards Action), where an Internet-Draft proposes one or more
Registered Performance Metrics to be added to the Performance Metrics Registered Performance Metrics to be added to the Performance Metrics
Registry, including the text of the proposed Registered Performance Registry, including the text of the proposed Registered Performance
Metric(s). Metric(s).
If an RFC-to-be includes a Performance Metric and a proposed If an RFC-to-be includes a Performance Metric and a proposed
Performance Metrics Registry entry, but the Performance Metric Expert Performance Metrics Registry Entry but the Performance Metrics
review determines that one or more of the Section 5 criteria have not Expert's review determines that one or more of the criteria listed in
been met, then the proposed Performance Metrics Registry entry MUST Section 5 have not been met, then the proposed Performance Metrics
be removed from the text. Once evidence exists that the Performance Registry Entry MUST be removed from the text. Once evidence exists
Metric meets the criteria in section 5, the proposed Performance that the Performance Metric meets the criteria in Section 5, the
Metrics Registry entry SHOULD be submitted to IANA to be evaluated in proposed Performance Metrics Registry Entry SHOULD be submitted to
consultation with the Performance Metric Experts for registration at IANA to be evaluated in consultation with the Performance Metrics
that time. Experts for registration at that time.
Authors of proposed Registered Performance Metrics SHOULD review Authors of proposed Registered Performance Metrics SHOULD review
compliance with the specifications in this document to check their compliance with the specifications in this document to check their
submissions before sending them to IANA. submissions before sending them to IANA.
At least one Performance Metric Expert should endeavor to complete At least one Performance Metrics Expert should endeavor to complete
referred reviews in a timely manner. If the request is acceptable, referred reviews in a timely manner. If the request is acceptable,
the Performance Metric Experts signify their approval to IANA, and the Performance Metrics Experts signify their approval to IANA, and
IANA updates the Performance Metrics Registry. If the request is not IANA updates the Performance Metrics Registry. If the request is not
acceptable, the Performance Metric Experts MAY coordinate with the acceptable, the Performance Metrics Experts MAY coordinate with the
requester to change the request to be compliant, otherwise IANA SHALL requester to change the request so that it is compliant; otherwise,
coordinate resolution of issues on behalf of the expert. The IANA SHALL coordinate resolution of issues on behalf of the expert.
Performance Metric Experts MAY choose to reject clearly frivolous or The Performance Metrics Experts MAY choose to reject clearly
inappropriate change requests outright, but such exceptional frivolous or inappropriate change requests outright, but such
circumstances should be rare. exceptional circumstances should be rare.
This process should not in any way be construed as allowing the If the proposed Metric is unique in a significant way, in order to
Performance Metric Experts to overrule IETF consensus. Specifically, properly describe the Metric, it may be necessary to propose a new
any Registered Performance Metrics that were added to the Performance Name Element Registry, or (more likely) a new Entry in an existing
Metrics Registry with IETF consensus require IETF consensus for Name Element Registry. This proposal is part of the request for the
revision or deprecation. new Metric, so that it undergoes the same IANA review and approval
process.
Decisions by the Performance Metric Experts may be appealed as in Decisions by the Performance Metrics Experts may be appealed per
Section 7 of [RFC8126]. Section 10 of [RFC8126].
8.2. Revising Registered Performance Metrics 8.2. Backward-Compatible Revision of Registered Performance Metrics
A request for Revision is only permitted when the requested changes A request for revision is only permitted when the requested changes
maintain backward-compatibility with implementations of the prior maintain backward compatibility with implementations of the prior
Performance Metrics Registry entry describing a Registered Performance Metrics Registry Entry describing a Registered
Performance Metric (entries with lower revision numbers, but the same Performance Metric (entries with lower revision numbers but having
Identifier and Name). the same Identifier and Name).
The purpose of the Status field in the Performance Metrics Registry The purpose of the Status field in the Performance Metrics Registry
is to indicate whether the entry for a Registered Performance Metric is to indicate whether the entry for a Registered Performance Metric
is 'current' or 'deprecated'. is 'Current', 'Deprecated', or 'Obsolete'. The term 'deprecated' is
used when an entry is replaced, either with a backwards-compatible
revision (this sub-section) or with a non-backwards-compatible
revision (in Section 8.3).
In addition, no policy is defined for revising the Performance Metric In addition, no policy is defined for revising the Performance Metric
entries in the IANA Registry or addressing errors therein. To be Entries in the IANA Registry or addressing errors therein. To be
clear, changes and deprecations within the Performance Metrics clear, changes and deprecations within the Performance Metrics
Registry are not encouraged, and should be avoided to the extent Registry are not encouraged and should be avoided to the extent
possible. However, in recognition that change is inevitable, the possible. However, in recognition that change is inevitable, the
provisions of this section address the need for revisions. provisions of this section address the need for revisions.
Revisions are initiated by sending a candidate Registered Performance Revisions are initiated by sending a candidate Registered Performance
Metric definition to IANA, as in Section 8.1, identifying the Metric definition to IANA, per Section 8.1, identifying the existing
existing Performance Metrics Registry entry, and explaining how and Performance Metrics Registry Entry, and explaining how and why the
why the existing entry should be revised. existing entry should be revised.
The primary requirement in the definition of procedures for managing The primary requirement in the definition of procedures for managing
changes to existing Registered Performance Metrics is avoidance of changes to existing Registered Performance Metrics is avoidance of
measurement interoperability problems; the Performance Metric Experts measurement interoperability problems; the Performance Metrics
must work to maintain interoperability above all else. Changes to Experts must work to maintain interoperability above all else.
Registered Performance Metrics may only be done in an interoperable Changes to Registered Performance Metrics may only be done in an
way; necessary changes that cannot be done in a way to allow interoperable way; necessary changes that cannot be done in a way
interoperability with unchanged implementations MUST result in the that allows interoperability with unchanged implementations MUST
creation of a new Registered Performance Metric (with a new Name, result in the creation of a new Registered Performance Metric (with a
replacing the RFCXXXXsecY portion of the name) and possibly the new Name, replacing the RFCXXXXsecY portion of the Name) and possibly
deprecation of the earlier metric. the deprecation of the earlier metric.
A change to a Registered Performance Metric SHALL be determined to be A change to a Registered Performance Metric SHALL be determined to be
backward-compatible when: backward compatible when:
1. it involves the correction of an error that is obviously only 1. it involves the correction of an error that is obviously only
editorial; or editorial, or
2. it corrects an ambiguity in the Registered Performance Metric's 2. it corrects an ambiguity in the Registered Performance Metric's
definition, which itself leads to issues severe enough to prevent definition, which itself leads to issues severe enough to prevent
the Registered Performance Metric's usage as originally defined; the Registered Performance Metric's usage as originally defined,
or or
3. it corrects missing information in the metric definition without 3. it corrects missing information in the metric definition without
changing its meaning (e.g., the explicit definition of 'quantity' changing its meaning (e.g., the explicit definition of 'quantity'
semantics for numeric fields without a Data Type Semantics semantics for numeric fields without a Data Type Semantics
value); or value), or
4. it harmonizes with an external reference that was itself 4. it harmonizes with an external reference that was itself
corrected. corrected, or
If a Performance Metric revision is deemed permissible and backward- 5. if the current Registry format has been revised by adding a new
compatible by the Performance Metric Experts, according to the rules column that is not relevant to an existing Registered Performance
Metric (i.e., the new column can be safely filled in with "Not
Applicable").
If a Performance Metric revision is deemed permissible and backward
compatible by the Performance Metrics Experts, according to the rules
in this document, IANA SHOULD execute the change(s) in the in this document, IANA SHOULD execute the change(s) in the
Performance Metrics Registry. The requester of the change is Performance Metrics Registry. The requester of the change is
appended to the original requester in the Performance Metrics appended to the original requester in the Performance Metrics
Registry. The Name of the revised Registered Performance Metric, Registry. The Name of the revised Registered Performance Metric,
including the RFCXXXXsecY portion of the name, SHALL remain unchanged including the RFCXXXXsecY portion of the Name, SHALL remain unchanged
(even when the change is the result of IETF Standards Action; the even when the change is the result of IETF Standards Action. The
revised registry entry SHOULD reference the new immutable document, revised Registry Entry SHOULD reference the new immutable document,
such as an RFC or for other standards bodies, it is likely to be such as an RFC. For other standards bodies, it is likely to be
necessary to reference a specific, dated version of a specification, necessary to reference a specific, dated version of a specification,
in an appropriate category and column). in an appropriate category and column.
Each Registered Performance Metric in the Performance Metrics Each Registered Performance Metric in the Performance Metrics
Registry has a revision number, starting at zero. Each change to a Registry has a revision number, starting at zero. Each change to a
Registered Performance Metric following this process increments the Registered Performance Metric following this process increments the
revision number by one. revision number by one.
When a revised Registered Performance Metric is accepted into the When a revised Registered Performance Metric is accepted into the
Performance Metrics Registry, the date of acceptance of the most Performance Metrics Registry, the date of acceptance of the most
recent revision is placed into the revision Date column of the recent revision is placed into the Revision Date column of the
registry for that Registered Performance Metric. Registry for that Registered Performance Metric.
Where applicable, additions to Registered Performance Metrics in the Where applicable, additions to Registered Performance Metrics in the
form of text Comments or Remarks should include the date, but such form of text in the Comments or Remarks column should include the
additions may not constitute a revision according to this process. date, but such additions may not constitute a revision according to
this process.
Older version(s) of the updated metric entries are kept in the Older versions of the updated Metric Entries are kept in the Registry
registry for archival purposes. The older entries are kept with all for archival purposes. The older entries are kept with all fields
fields unmodified (version, revision date) except for the status unmodified (including Revision Date) except for the Status field,
field that SHALL be changed to "Deprecated". which SHALL be changed to 'Deprecated'.
8.3. Deprecating Registered Performance Metrics This process should not in any way be construed as allowing the
Performance Metrics Experts to overrule IETF consensus.
Specifically, any Registered Performance Metrics that were added to
the Performance Metrics Registry with IETF consensus require IETF
consensus for revision or deprecation.
Changes that are not permissible by the above criteria for Registered 8.3. Non-Backward-Compatible Deprecation of Registered Performance
Performance Metric's revision may only be handled by deprecation. A Metrics
Registered Performance Metric MAY be deprecated and replaced when:
This section describes how to make a non-backward-compatible update
to a Registered Performance Metric. A Registered Performance Metric
MAY be deprecated and replaced when:
1. the Registered Performance Metric definition has an error or 1. the Registered Performance Metric definition has an error or
shortcoming that cannot be permissibly changed as in Section 8.2 shortcoming that cannot be permissibly changed per Section 8.2
Revising Registered Performance Metrics; or ("Revising Registered Performance Metrics"), or
2. the deprecation harmonizes with an external reference that was 2. the deprecation harmonizes with an external reference that was
itself deprecated through that reference's accepted deprecation itself deprecated through that reference's accepted deprecation
method. method.
A request for deprecation is sent to IANA, which passes it to the A request for deprecation is sent to IANA, which passes it to the
Performance Metric Experts for review. When deprecating an Performance Metrics Experts for review. When deprecating a
Performance Metric, the Performance Metric description in the Performance Metric, the Performance Metric Description in the
Performance Metrics Registry must be updated to explain the Performance Metrics Registry MUST be updated to explain the
deprecation, as well as to refer to any new Performance Metrics deprecation, as well as to refer to the new Performance Metric
created to replace the deprecated Performance Metric. created to replace the deprecated Performance Metric.
The revision number of a Registered Performance Metric is incremented When a new, non-backward-compatible Performance Metric replaces a
upon deprecation, and the revision Date updated, as with any (now) deprecated metric, the revision number of the new Registered
revision. Performance Metric is incremented over the value in the deprecated
version, and the current date is entered as the Revision Date of the
new Registered Performance Metric.
The intentional use of deprecated Registered Performance Metrics The intentional use of deprecated Registered Performance Metrics
should result in a log entry or human-readable warning by the should result in a log entry or human-readable warning by the
respective application. respective application.
Names and Metric IDs of deprecated Registered Performance Metrics Names and Metric IDs of deprecated Registered Performance Metrics
must not be reused. must not be reused.
The deprecated entries are kept with all fields unmodified, except The deprecated entries are kept with all Administrative columns
the version, revision date, and the status field (changed to unmodified, except the Status field (which is changed to
"Deprecated"). 'Deprecated').
9. Security considerations 8.4. Obsolete Registry Entries
This draft defines a registry structure, and does not itself Existing Registry Entries may become obsolete over time due to:
1. the Registered Performance Metric is found to contain
considerable errors (and no one sees the value in the effort to
fix it), or
2. one or more critical References (or sections thereof) have been
designated obsolete by the SDO, or
3. other reasons brought to the attention of IANA and the Registry
Experts.
When a Performance Metric Registry Entry is declared obsolete, the
Performance Metric Description in the Performance Metrics Registry is
updated to explain the reasons the Entry is now obsolete and has not
been replaced (Deprecation always involves replacement).
Obsolete entries are kept with all Administrative columns unmodified,
except the Status field (which is changed to 'Obsolete').
8.5. Registry Format Version and Future Changes/Extensions
The Registry Format Version defined in this memo is 1.0, and
candidate Registry Entries complying with this memo MUST use 1.0.
The Registry Format can only be updated by publishing a new RFC with
the new format (Standards Action).
When a Registered Performance Metric is created or revised, then it
uses the most recent Registry Format Version.
Only one form of Registry extension is envisaged:
Adding columns, or both categories and columns, to accommodate
unanticipated aspects of new measurements and metric categories.
If the Performance Metrics Registry is extended in this way, the
version number of future entries complying with the extension SHALL
be incremented (in either the unit or the tenths digit, depending on
the degree of extension).
9. Security Considerations
This document defines a Registry structure and does not itself
introduce any new security considerations for the Internet. The introduce any new security considerations for the Internet. The
definition of Performance Metrics for this registry may introduce definition of Performance Metrics for this Registry may introduce
some security concerns, but the mandatory references should have some security concerns, but the mandatory references should have
their own considerations for security, and such definitions should be their own considerations for security, and such definitions should be
reviewed with security in mind if the security considerations are not reviewed with security in mind if the security considerations are not
covered by one or more reference standards. covered by one or more reference standards.
The aggregated results of the performance metrics described in this The aggregated results of the Performance Metrics described in this
registry might reveal network topology information that may be Registry might reveal network topology information that may be
considered sensitive. If such cases are found, then access control considered sensitive. If such cases are found, then access control
mechanisms should be applied. mechanisms should be applied.
10. IANA Considerations 10. IANA Considerations
With the background and processes described in earlier sections, this With the background and processes described in earlier sections, IANA
document requests the following IANA Actions. has taken the actions described below.
Editor's Note: Mock-ups of the implementation of this set of requests
have been prepared with IANA's help during development of this memo,
and have been captured in the Proceedings of IPPM working group
sessions. IANA is currently preparing a mock-up. A recent version
is available here: http://encrypted.net/IETFMetricsRegistry-106.html
10.1. Registry Group 10.1. Registry Group
The new registry group SHALL be named, "PERFORMANCE METRICS Group". The new Registry group is named Performance Metrics. This document
refers to it as the "Performance Metrics Group" or "Registry Group",
meaning all registrations appearing on
<https://www.iana.org/assignments/performance-metrics>
(https://www.iana.org/assignments/performance-metrics).
Registration Procedure: Specification Required For clarity, note that this document and [RFC8912] use the following
conventions to refer to the various IANA registries related to
Performance Metrics.
Reference: <This RFC> +===============+===========================+=====================+
| | RFC 8911 and RFC 8912 | IANA Web page |
+===============+===========================+=====================+
| Page Title | Performance Metrics Group | Performance Metrics |
+---------------+---------------------------+---------------------+
| Main Registry | Performance Metrics | Performance Metrics |
| | Registry | Registry |
+---------------+---------------------------+---------------------+
| Registry Row | Performance Metrics | registration (also |
| | Registry Entry | template) |
+---------------+---------------------------+---------------------+
Experts: Performance Metrics Experts Table 6
Note: TBD Registration Procedure: Specification Required
10.2. Performance Metric Name Elements Reference: RFC 8911
This document specifies the procedure for Performance Metrics Name Experts: Performance Metrics Experts
Element Registry setup. IANA is requested to create a new set of
registries for Performance Metric Name Elements called "Registered
Performance Metric Name Elements". Each Registry, whose names are
listed below:
MetricType: 10.2. Performance Metrics Name Elements
Method: This memo specifies and populates the Registries for the Performance
Metric Name Elements. The Name assigned to a Performance Metric
Registry Entry consists of multiple Elements separated by an "_"
(underscore), in the order defined in Section 7.1.2. IANA has
created the following registries, which contain the current set of
possibilities for each Element in the Performance Metric Name.
SubTypeMethod: MetricType
Spec: Method
Units: SubTypeMethod
Output: Spec
will contain the current set of possibilities for Performance Metrics Units
Registry Entry Names.
To populate the Registered Performance Metric Name Elements at Output
creation, the IANA is asked to use the lists of values for each name
element listed in Section 7.1.2. The Name Elements in each registry
are case-sensitive.
When preparing a Metric entry for Registration, the developer SHOULD At creation, IANA has populated the Registered Performance Metrics
choose Name elements from among the registered elements. However, if Name Elements using the lists of values for each Name Element listed
in Section 7.1.2. The Name Elements in each Registry are case
sensitive.
When preparing a Metric Entry for registration, the developer SHOULD
choose Name Elements from among the registered elements. However, if
the proposed metric is unique in a significant way, it may be the proposed metric is unique in a significant way, it may be
necessary to propose a new Name element to properly describe the necessary to propose a new Name Element to properly describe the
metric, as described below. metric, as described below.
A candidate Metric Entry RFC or immutable document for IANA and A candidate Metric Entry proposes a set of values for its Name
Expert Review would propose one or more new element values required Elements. These are reviewed by IANA and an Expert Reviewer. It is
to describe the unique entry, and the new name element(s) would be possible that a candidate Metric Entry proposes a new value for a
reviewed along with the metric entry. New assignments for Registered Name Element (that is, one that is not in the existing list of
Performance Metric Name Elements will be administered by IANA through possibilities), or even that it proposes a new Name Element. Such
Specification Required policy (which includes Expert Review) new assignments are administered by IANA through the Specification
[RFC8126], i.e., review by one of a group of experts, the Performance Required policy [RFC8126], which includes Expert Review (i.e., review
Metric Experts, who are appointed by the IESG upon recommendation of by one of a group of Performance Metrics Experts, who are appointed
the Transport Area Directors. by the IESG upon recommendation of the Transport Area Directors).
10.3. New Performance Metrics Registry 10.3. New Performance Metrics Registry
This document specifies the procedure for Performance Metrics This document specifies the Performance Metrics Registry. The
Registry setup. IANA is requested to create a new registry for Registry contains the following columns in the Summary category:
Performance Metrics called "Performance Metrics Registry". This
Registry will contain the following Summary columns:
Identifier: Identifier
Name: Name
URI: URI
Description: Description
Reference: Reference
Change Controller: Change Controller
Version: Version
Descriptions of these columns and additional information found in the Descriptions of these columns and additional information found in the
template for registry entries (categories and columns) are further template for Registry Entries (categories and columns) are further
defined in section Section 7. defined in Section 7.
The Identifier 0 should be Reserved. The Registered Performance The Identifier 0 should be Reserved. The Registered Performance
Metric unique identifier is an unbounded integer (range 0 to Metric unique Identifier is an unbounded integer (range 0 to
infinity). The Identifier values from 64512 to 65536 are reserved infinity). The Identifier values from 64512 to 65535 are reserved
for private or experimental use, and the user may encounter for private or experimental use, and the user may encounter
overlapping uses. When adding newly Registered Performance Metrics overlapping uses. When adding new Registered Performance Metrics to
to the Performance Metrics Registry, IANA SHOULD assign the lowest the Performance Metrics Registry, IANA SHOULD assign the lowest
available identifier to the new Registered Performance Metric. If a available Identifier to the new Registered Performance Metric. If a
Performance Metrics Expert providing review determines that there is Performance Metrics Expert providing review determines that there is
a reason to assign a specific numeric identifier, possibly leaving a a reason to assign a specific numeric Identifier, possibly leaving a
temporary gap in the numbering, then the Performance Expert SHALL temporary gap in the numbering, then the Performance Metrics Expert
inform IANA of this decision. SHALL inform IANA of this decision.
Names starting with the prefix Priv_ are reserved for private use, Names starting with the prefix "Priv_" are reserved for private use
and are not considered for registration. The "Name" column entries and are not considered for registration. The Name column entries are
are further defined in section Section 7. further defined in Section 7.
The "URI" column will have a URL to the full template of each The URI column will have a URL to each completed Registry Entry. The
registry entry. The Registry Entry text SHALL be HTML-ized to aid Registry Entry text SHALL be HTMLized to aid the reader (similar to
the reader, with links to reference RFCs (similar to the way that the way that Internet-Drafts are HTMLized, the same tool can perform
Internet Drafts are HTML-ized, the same tool can perform the the function), with links to referenced section(s) of an RFC or
function) or immutable document. another immutable document.
The "Reference" column will include an RFC number, an approved The Reference column will include an RFC number, an approved
specification designator from another standards body, or other specification designator from another standards body, or some other
immutable document. immutable document.
New assignments for Performance Metrics Registry will be administered New assignments for the Performance Metrics Registry will be
by IANA through Specification Required policty (which includes Expert administered by IANA through the Specification Required policy
Review) [RFC8126], i.e., review by one of a group of experts, the [RFC8126] (which includes Expert Review, i.e., review by one of a
Performance Metric Experts, who are appointed by the IESG upon group of experts -- in the case of this document, the Performance
recommendation of the Transport Area Directors, or by Standards Metrics Experts, who are appointed by the IESG upon recommendation of
Action. The experts can be initially drawn from the Working Group the Transport Area Directors) or by Standards Action. The experts
Chairs, document editors, and members of the Performance Metrics can be initially drawn from the Working Group Chairs, document
Directorate, among other sources of experts. editors, and members of the Performance Metrics Directorate, among
other sources of experts.
Extensions of the Performance Metrics Registry require IETF Standards Extensions to the Performance Metrics Registry require IETF Standards
Action. Only one form of registry extension is envisaged: Action. Only one form of Registry extension is envisaged:
1. Adding columns, or both categories and columns, to accommodate * Adding columns, or both categories and columns, to accommodate
unanticipated aspects of new measurements and metric categories. unanticipated aspects of new measurements and metric categories.
If the Performance Metrics Registry is extended in this way, the If the Performance Metrics Registry is extended in this way, the
Version number of future entries complying with the extension SHALL version number of future entries complying with the extension SHALL
be incremented (either in the unit or tenths digit, depending on the be incremented (in either the unit or the tenths digit, depending on
degree of extension. the degree of extension).
11. Blank Registry Template 11. Blank Registry Template
This section provides a blank template to help IANA and registry This section provides a blank template to help IANA and Registry
entry writers. Entry writers.
11.1. Summary 11.1. Summary
This category includes multiple indexes to the registry entry: the This category includes multiple indexes to the Registry Entry: the
element ID and metric name. element ID and Metric Name.
11.1.1. ID (Identifier) 11.1.1. ID (Identifier)
<insert a numeric identifier, an integer, TBD> <insert a numeric Identifier, an integer, TBD>
11.1.2. Name 11.1.2. Name
<insert name according to metric naming convention> <insert the Name, according to the metric naming convention>
11.1.3. URI 11.1.3. URI
URL: https://www.iana.org/ ... <name> URL: https://www.iana.org/performance-metrics/ ... <Name>
11.1.4. Description 11.1.4. Description
<provide a description> <provide a description>
11.1.5. Change Controller 11.1.5. Reference
11.1.6. Version (of Registry Format) <provide the RFC or other specification that contains the approved
candidate Registry Entry>
11.1.6. Change Controller
<provide information regarding the entity responsible for approving
revisions to the Registry Entry (including contact information for an
individual, where appropriate)>
11.1.7. Version (of Registry Format)
11.2. Metric Definition 11.2. Metric Definition
This category includes columns to prompt the entry of all necessary This category includes columns to prompt the entry of all necessary
details related to the metric definition, including the immutable details related to the metric definition, including the immutable
document reference and values of input factors, called fixed document reference and values of input factors, called "Fixed
parameters. Parameters".
11.2.1. Reference Definition 11.2.1. Reference Definition
<Full bibliographic reference to an immutable doc.> <provide a full bibliographic reference to an immutable document>
<specific section reference and additional clarifications, if needed> <provide a specific section reference and additional clarifications,
if needed>
11.2.2. Fixed Parameters 11.2.2. Fixed Parameters
<list and specify Fixed Parameters, input factors that must be <list and specify Fixed Parameters, input factors that must be
determined and embedded in the measurement system for use when determined and embedded in the measurement system for use when
needed> needed>
11.3. Method of Measurement 11.3. Method of Measurement
This category includes columns for references to relevant sections of This category includes columns for references to relevant sections of
the immutable documents(s) and any supplemental information needed to the immutable document(s) and any supplemental information needed to
ensure an unambiguous methods for implementations. ensure an unambiguous method for implementations.
11.3.1. Reference Method 11.3.1. Reference Method
<for metric, insert relevant section references and supplemental <for the metric, insert relevant section references and supplemental
info> info>
11.3.2. Packet Stream Generation 11.3.2. Packet Stream Generation
<list of generation parameters and section/spec references if needed> <provide a list of generation Parameters and section/spec references
if needed>
11.3.3. Traffic Filtering (observation) Details 11.3.3. Traffic Filtering (Observation) Details
The measured results based on a filtered version of the packets This category provides the filter details (when present), which
observed, and this section provides the filter details (when qualify the set of packets that contribute to the measured results
present). from among all packets observed.
<section reference>. <provide a section reference>
11.3.4. Sampling Distribution 11.3.4. Sampling Distribution
<insert time distribution details, or how this is diff from the <insert time distribution details, or how this is different from the
filter> filter>
11.3.5. Run-time Parameters and Data Format 11.3.5. Runtime Parameters and Data Format
Run-time Parameters are input factors that must be determined, Runtime Parameters are input factors that must be determined,
configured into the measurement system, and reported with the results configured into the measurement system, and reported with the results
for the context to be complete. for the context to be complete.
<list of run-time parameters, and their data formats> <provide a list of Runtime Parameters and their data formats>
11.3.6. Roles 11.3.6. Roles
<lists the names of the different roles from the measurement method> <list the names of the different Roles from the Measurement Method>
11.4. Output 11.4. Output
This category specifies all details of the Output of measurements This category specifies all details of the output of measurements
using the metric. using the metric.
11.4.1. Type 11.4.1. Type
<insert name of the output type, raw or a selected summary statistic> <insert the name of the output type -- raw results or a selected
summary statistic>
11.4.2. Reference Definition 11.4.2. Reference Definition
<describe the reference data format for each type of result> <describe the reference data format for each type of result>
11.4.3. Metric Units 11.4.3. Metric Units
<insert units for the measured results, and the reference <insert units for the measured results, and provide the reference
specification>. specification>
11.4.4. Calibration 11.4.4. Calibration
<insert information on calibration> <insert information on calibration>
11.5. Administrative items 11.5. Administrative Items
This category provides administrative information.
11.5.1. Status 11.5.1. Status
<current or deprecated> <provide status: 'Current' or 'Deprecated'>
11.5.2. Requester 11.5.2. Requester
<name or RFC, etc.> <provide a person's name, an RFC number, etc.>
11.5.3. Revision 11.5.3. Revision
<1.0> <provide the revision number: starts at 0>
11.5.4. Revision Date 11.5.4. Revision Date
<format YYYY-MM-DD> <provide the date, in YYYY-MM-DD format>
11.6. Comments and Remarks 11.6. Comments and Remarks
<Additional (Informational) details for this entry> <list any additional (informational) details for this entry>
12. Acknowledgments
Thanks to Brian Trammell and Bill Cerveny, IPPM chairs, for leading
some brainstorming sessions on this topic. Thanks to Barbara Stark
and Juergen Schoenwaelder for the detailed feedback and suggestions.
Thanks to Andrew McGregor for suggestions on metric naming. Thanks
to Michelle Cotton for her early IANA review, and to Amanda Barber
for answering questions related to the presentation of the registry
and accessibility of the complete template via URL. Thanks to Roni
Even for his review and suggestions to generalize the procedures.
Thanks to ~all the Area Directors for their reviews.
13. References 12. References
13.1. Normative References 12.1. Normative References
[RFC2026] Bradner, S., "The Internet Standards Process -- Revision [RFC2026] Bradner, S., "The Internet Standards Process -- Revision
3", BCP 9, RFC 2026, DOI 10.17487/RFC2026, October 1996, 3", BCP 9, RFC 2026, DOI 10.17487/RFC2026, October 1996,
<https://www.rfc-editor.org/info/rfc2026>. <https://www.rfc-editor.org/info/rfc2026>.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997, DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>. <https://www.rfc-editor.org/info/rfc2119>.
[RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis,
"Framework for IP Performance Metrics", RFC 2330, "Framework for IP Performance Metrics", RFC 2330,
DOI 10.17487/RFC2330, May 1998, DOI 10.17487/RFC2330, May 1998,
<https://www.rfc-editor.org/info/rfc2330>. <https://www.rfc-editor.org/info/rfc2330>.
[RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform
Resource Identifier (URI): Generic Syntax", STD 66, Resource Identifier (URI): Generic Syntax", STD 66,
RFC 3986, DOI 10.17487/RFC3986, January 2005, RFC 3986, DOI 10.17487/RFC3986, January 2005,
<https://www.rfc-editor.org/info/rfc3986>. <https://www.rfc-editor.org/info/rfc3986>.
[RFC5644] Stephan, E., Liang, L., and A. Morton, "IP Performance
Metrics (IPPM): Spatial and Multicast", RFC 5644,
DOI 10.17487/RFC5644, October 2009,
<https://www.rfc-editor.org/info/rfc5644>.
[RFC6390] Clark, A. and B. Claise, "Guidelines for Considering New [RFC6390] Clark, A. and B. Claise, "Guidelines for Considering New
Performance Metric Development", BCP 170, RFC 6390, Performance Metric Development", BCP 170, RFC 6390,
DOI 10.17487/RFC6390, October 2011, DOI 10.17487/RFC6390, October 2011,
<https://www.rfc-editor.org/info/rfc6390>. <https://www.rfc-editor.org/info/rfc6390>.
[RFC6576] Geib, R., Ed., Morton, A., Fardid, R., and A. Steinmitz, [RFC6576] Geib, R., Ed., Morton, A., Fardid, R., and A. Steinmitz,
"IP Performance Metrics (IPPM) Standard Advancement "IP Performance Metrics (IPPM) Standard Advancement
Testing", BCP 176, RFC 6576, DOI 10.17487/RFC6576, March Testing", BCP 176, RFC 6576, DOI 10.17487/RFC6576, March
2012, <https://www.rfc-editor.org/info/rfc6576>. 2012, <https://www.rfc-editor.org/info/rfc6576>.
skipping to change at page 36, line 5 skipping to change at line 1783
[RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for [RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for
Writing an IANA Considerations Section in RFCs", BCP 26, Writing an IANA Considerations Section in RFCs", BCP 26,
RFC 8126, DOI 10.17487/RFC8126, June 2017, RFC 8126, DOI 10.17487/RFC8126, June 2017,
<https://www.rfc-editor.org/info/rfc8126>. <https://www.rfc-editor.org/info/rfc8126>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>. May 2017, <https://www.rfc-editor.org/info/rfc8174>.
13.2. Informative References 12.2. Informative References
[I-D.ietf-ippm-initial-registry]
Morton, A., Bagnulo, M., Eardley, P., and K. D'Souza,
"Initial Performance Metrics Registry Entries", draft-
ietf-ippm-initial-registry-15 (work in progress), December
2019.
[RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip [RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip
Delay Metric for IPPM", RFC 2681, DOI 10.17487/RFC2681, Delay Metric for IPPM", RFC 2681, DOI 10.17487/RFC2681,
September 1999, <https://www.rfc-editor.org/info/rfc2681>. September 1999, <https://www.rfc-editor.org/info/rfc2681>.
[RFC3432] Raisanen, V., Grotefeld, G., and A. Morton, "Network [RFC3432] Raisanen, V., Grotefeld, G., and A. Morton, "Network
performance measurement with periodic streams", RFC 3432, performance measurement with periodic streams", RFC 3432,
DOI 10.17487/RFC3432, November 2002, DOI 10.17487/RFC3432, November 2002,
<https://www.rfc-editor.org/info/rfc3432>. <https://www.rfc-editor.org/info/rfc3432>.
skipping to change at page 37, line 39 skipping to change at line 1857
Aitken, P., and A. Akhter, "A Framework for Large-Scale Aitken, P., and A. Akhter, "A Framework for Large-Scale
Measurement of Broadband Performance (LMAP)", RFC 7594, Measurement of Broadband Performance (LMAP)", RFC 7594,
DOI 10.17487/RFC7594, September 2015, DOI 10.17487/RFC7594, September 2015,
<https://www.rfc-editor.org/info/rfc7594>. <https://www.rfc-editor.org/info/rfc7594>.
[RFC7679] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton, [RFC7679] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton,
Ed., "A One-Way Delay Metric for IP Performance Metrics Ed., "A One-Way Delay Metric for IP Performance Metrics
(IPPM)", STD 81, RFC 7679, DOI 10.17487/RFC7679, January (IPPM)", STD 81, RFC 7679, DOI 10.17487/RFC7679, January
2016, <https://www.rfc-editor.org/info/rfc7679>. 2016, <https://www.rfc-editor.org/info/rfc7679>.
[RFC8194] Schoenwaelder, J. and V. Bajpai, "A YANG Data Model for [RFC8912] Morton, A., Bagnulo, M., Eardley, P., and K. D'Souza,
LMAP Measurement Agents", RFC 8194, DOI 10.17487/RFC8194, "Initial Performance Metrics Registry Entries", RFC 8912,
August 2017, <https://www.rfc-editor.org/info/rfc8194>. DOI 10.17487/RFC8912, November 2021,
<https://www.rfc-editor.org/info/rfc8912>.
Acknowledgments
Thanks to Brian Trammell and Bill Cerveny, IPPM co-chairs during the
development of this memo, for leading several brainstorming sessions
on this topic. Thanks to Barbara Stark and Juergen Schoenwaelder for
the detailed feedback and suggestions. Thanks to Andrew McGregor for
suggestions on metric naming. Thanks to Michelle Cotton for her
early IANA review, and to Amanda Baber for answering questions
related to the presentation of the Registry and accessibility of the
complete template via URL. Thanks to Roni Even for his review and
suggestions to generalize the procedures. Thanks to all of the Area
Directors for their reviews.
Authors' Addresses Authors' Addresses
Marcelo Bagnulo Marcelo Bagnulo
Universidad Carlos III de Madrid Universidad Carlos III de Madrid
Av. Universidad 30 Av. Universidad 30
Leganes, Madrid 28911 28911 Leganes Madrid
SPAIN Spain
Phone: 34 91 6249500 Phone: 34 91 6249500
Email: marcelo@it.uc3m.es Email: marcelo@it.uc3m.es
URI: http://www.it.uc3m.es URI: http://www.it.uc3m.es
Benoit Claise Benoit Claise
Cisco Systems, Inc. Huawei
De Kleetlaan 6a b1
1831 Diegem
Belgium
Email: bclaise@cisco.com Email: benoit.claise@huawei.com
Philip Eardley Philip Eardley
BT BT
Adastral Park, Martlesham Heath Adastral Park, Martlesham Heath
Ipswich Ipswich
ENGLAND United Kingdom
Email: philip.eardley@bt.com Email: philip.eardley@bt.com
Al Morton Al Morton
AT&T Labs AT&T Labs
200 Laurel Avenue South 200 Laurel Avenue South
Middletown, NJ Middletown, NJ 07748
USA United States of America
Email: acmorton@att.com Email: acmorton@att.com
Aamer Akhter Aamer Akhter
Consultant Consultant
118 Timber Hitch 118 Timber Hitch
Cary, NC Cary, NC
USA United States of America
Email: aakhter@gmail.com Email: aakhter@gmail.com
 End of changes. 289 change blocks. 
956 lines changed or deleted 1093 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/