<?xmlversion="1.0" encoding="US-ASCII"?>version='1.0' encoding='utf-8'?> <!DOCTYPE rfcSYSTEM "rfc2629.dtd"> <?xml-stylesheet type='text/xsl' href='rfc2629.xslt' ?> <?rfc toc="yes" ?> <?rfc symrefs="yes" ?> <?rfc strict="no" ?> <?rfc strict="no" ?>[ <!ENTITY nbsp " "> <!ENTITY zwsp "​"> <!ENTITY nbhy "‑"> <!ENTITY wj "⁠"> ]> <rfc xmlns:xi="http://www.w3.org/2001/XInclude" category="std" docName="draft-ietf-ippm-metric-registry-24" number="8911" ipr="trust200902" obsoletes=""updates="">updates="" submissionType="IETF" consensus="true" xml:lang="en" tocInclude="true" symRefs="true" sortRefs="true" version="3"> <!-- xml2rfc v2v3 conversion 2.42.0 --> <front> <title abbrev="Registry for Performance Metrics">Registry for Performance Metrics</title> <seriesInfo name="RFC" value="8911"/> <author fullname="Marcelo Bagnulo" initials="M." surname="Bagnulo"> <organization abbrev="UC3M">Universidad Carlos III de Madrid</organization> <address> <postal> <street>Av. Universidad 30</street> <city>Leganes</city> <region>Madrid</region> <code>28911</code><country>SPAIN</country><country>Spain</country> </postal> <phone>34 91 6249500</phone> <email>marcelo@it.uc3m.es</email> <uri>http://www.it.uc3m.es</uri> </address> </author> <author fullname="Benoit Claise" initials="B." surname="Claise"><organization abbrev="Cisco Systems, Inc.">Cisco Systems, Inc.</organization><organization>Huawei</organization> <address><postal> <street>De Kleetlaan 6a b1</street> <city>1831 Diegem</city> <country>Belgium</country> </postal> <email>bclaise@cisco.com</email><postal/> <email>benoit.claise@huawei.com</email> </address> </author> <author fullname="Philip Eardley" initials="P." surname="Eardley"> <organization abbrev="BT">BT</organization> <address> <postal> <street>Adastral Park, Martlesham Heath</street> <city>Ipswich</city><country>ENGLAND</country><country>United Kingdom</country> </postal> <email>philip.eardley@bt.com</email> </address> </author> <author fullname="Al Morton" initials="A." surname="Morton"> <organization abbrev="AT&T Labs">AT&T Labs</organization> <address> <postal> <street>200 Laurel Avenue South</street><city>Middletown, NJ</city> <country>USA</country><city>Middletown</city> <region>NJ</region> <code>07748</code> <country>United States of America</country> </postal> <email>acmorton@att.com</email> </address> </author> <author fullname="Aamer Akhter" initials="A." surname="Akhter"> <organization abbrev="Consultant">Consultant</organization> <address> <postal> <street>118 Timber Hitch</street> <city>Cary</city> <region>NC</region> <code/><country>USA</country><country>United States of America</country> </postal> <email>aakhter@gmail.com</email> </address> </author> <dateday="9" month="March" year="2020"/>month="November" year="2021"/> <keyword>IPPM</keyword> <keyword>Loss</keyword> <keyword>Delay</keyword> <abstract> <t>This document defines the format for the IANA Registry of PerformanceMetrics Registry.Metrics. This document also gives a set of guidelines for Registered Performance Metric requesters and reviewers.</t> </abstract> </front> <middle> <sectiontitle="Introduction">numbered="true" toc="default"> <name>Introduction</name> <t>The IETF specifies and uses Performance Metrics of protocols and applications transported over its protocols. PerformancemetricsMetrics are an important part of network operations using IETF protocols, and <xreftarget="RFC6390"/>target="RFC6390" format="default"/> specifies guidelines for their development.</t> <t>The definition and use of Performance Metrics in the IETFhashave been fostered in various working groups(WG), most(WGs). Most notably:<list> <t>The</t> <ul empty="false" spacing="normal"> <li>The "IP Performance Metrics" (IPPM) WG is the WG primarily focusing on Performance Metrics definition at theIETF.</t> <t>TheIETF.</li> <li>The "Benchmarking Methodology" WG (BMWG) defines many Performance Metrics for use in laboratory benchmarking ofinter-networking technologies.</t> <t>Theinternetworking technologies.</li> <li>The "Metric Blocks for use with RTCP's Extended Report Framework" (XRBLOCK) WG (concluded) specified many Performance Metrics related to "RTP Control Protocol Extended Reports (RTCP XR)" <xreftarget="RFC3611"/>,target="RFC3611" format="default"/>, which establishes a framework to allow new information to be conveyed in RTCP, supplementing the original report blocks defined in "RTP: A Transport Protocol for Real-TimeApplications",Applications" <xreftarget="RFC3550"/>.</t> <t>Thetarget="RFC3550" format="default"/>.</li> <li>The "IP Flow Information eXport" (IPFIX)concludedWG (concluded) specified anIANAInternet Assigned Numbers Authority (IANA) process for new Information Elements. SomePerformance Metrics relatedInformation Elements related to Performance Metrics are proposed on a regularbasis.</t> <t>Thebasis.</li> <li>The "Performance Metrics for Other Layers" (PMOL)a concludedWG (concluded) defined some Performance Metrics related to Session Initiation Protocol (SIP) voice quality <xreftarget="RFC6035"/>.</t> </list></t>target="RFC6035" format="default"/>.</li> </ul> <t>It is expected that more Performance Metrics will be defined in thefuture,future -- not only IP-basedmetrics,metrics but also metricswhichthat areprotocol-specificprotocol specific andapplication-specific.</t>application specific.</t> <t>Despite the importance of Performance Metrics, there are two related problems for theindustry. First,industry:</t> <ul empty="false" spacing="normal"> <li>First, ensuring that when one party requests that another partytomeasure (or report or in some way act on) a particular Performance Metric,thenboth parties have exactly the same understanding of what Performance Metric is being referredto. Second,to.</li> <li>Second, discovering which Performance Metrics have been specified, to avoid developing a new Performance Metric that is verysimilar,similar but not quiteinter-operable. Theseinteroperable.</li> </ul> <t>These problems can be addressed by creating aregistry of performance metrics. The usual way in which the IETF organizes registries isRegistry for Performance Metrics with the Internet Assigned Numbers Authority(IANA), and there is currently no Performance Metrics Registry maintained by the IANA.</t> <t>This(IANA). As such, this documentrequests thatdefines the new IANAcreateRegistry for Performance Metrics.</t> <t>Per this document, IANA has created andmaintain anow maintains the Performance Metrics Registry, according to the maintenance procedures and thePerformance Metrics Registryformat defined inthis memo.the sections below. The resulting Performance Metrics Registry is for use by the IETF and others. Although the Registry formatting specifications herein are primarily forregistryRegistry creation by IANA, any other organization that wishes to create aperformance metrics registryPerformance Metrics Registry may use the same formatting specifications for their purposes. The authors make no guarantee of theregistryRegistry format's applicability to any possible set of Performance Metrics envisaged by other organizations, but we encourage others to apply it. In the remainder of this document, unless we explicitly say otherwise, we will refer to the IANA-maintained Performance Metrics Registry as simply the Performance Metrics Registry.</t> </section> <sectiontitle="Terminology">numbered="true" toc="default"> <name>Terminology</name> <t>The key words"MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY","<bcp14>MUST</bcp14>", "<bcp14>MUST NOT</bcp14>", "<bcp14>REQUIRED</bcp14>", "<bcp14>SHALL</bcp14>", "<bcp14>SHALL NOT</bcp14>", "<bcp14>SHOULD</bcp14>", "<bcp14>SHOULD NOT</bcp14>", "<bcp14>RECOMMENDED</bcp14>", "<bcp14>NOT RECOMMENDED</bcp14>", "<bcp14>MAY</bcp14>", and"OPTIONAL""<bcp14>OPTIONAL</bcp14>" in this document are to be interpreted as described inBCP 14BCP 14 <xref target="RFC2119"/> <xref target="RFC8174"/> when, and only when, they appear in all capitals, as shown here.</t><!-- <t>The terms Performance Metric is defined in <xref target="RFC6390"/>, and copied over in this document for the readers convenience.</t> --> <t><list style="hanging"> <t hangText="Performance Metric:">A Performance Metric is a<dl newline="false" spacing="normal"> <dt>Performance Metric:</dt> <dd>A quantitative measure of performance, targeted to an IETF-specified protocol or targeted to an application transported over an IETF-specified protocol. Examples of Performance Metrics are the FTP response time for a complete file download, the DNSresponseResponse time to resolve the IP address(es), a database logging time, etc. This definition is consistent with the definition of a metric in <xreftarget="RFC2330"/>target="RFC2330" format="default"/> and broader than the definition ofperformance metrica Performance Metric in <xreftarget="RFC6390"/>.</t> <t hangText="Registered Performance Metric:">A Registeredtarget="RFC6390" format="default"/>.</dd> <dt>Registered PerformanceMetric is aMetric:</dt> <dd>A Performance Metric expressed as an entry in the Performance Metrics Registry, administered by IANA. Such aperformance metricPerformance Metric has met all of theregistryRegistry review criteria defined in this document in order to be included in theregistry.</t> <t hangText="PerformanceRegistry.</dd> <dt>Performance MetricsRegistry:">TheRegistry:</dt> <dd>The IANAregistryRegistry containing Registered PerformanceMetrics.</t> <t hangText="Proprietary Registry:">AMetrics.</dd> <dt>Proprietary Registry:</dt> <dd>A set of metrics that are registered in a proprietaryregistry,Registry, as opposed to the Performance MetricsRegistry.</t> <t hangText="Performance Metrics Experts:">The PerformanceRegistry.</dd> <dt>Performance MetricsExperts is aExperts:</dt> <dd>A group of designated experts <xreftarget="RFC8126"/>target="RFC8126" format="default"/> selected by the IESG to validate the Performance Metrics before updating the Performance Metrics Registry. The Performance Metrics Experts work closely withIANA.</t> <!-- <t hangText="Performance Metrics Directorate:">The Performance Metrics Directorate is a directorate that provides guidance for Performance Metrics development in the IETF. The Performance Metrics Directorate should be composed of experts in the performance community, potentially selected from the IP Performance Metrics (IPPM), Benchmarking Methodology (BMWG), and Performance Metrics for Other Layers (PMOL) WGs.</t> --> <t hangText="Parameter:">A Parameter is anIANA.</dd> <dt>Parameter:</dt> <dd>An input factor defined as a variable in the definition of a Performance Metric. A Parameter is a numerical or other specified factor forming one of a set that defines a metric or sets the conditions of its operation. All Parameters must be known in order to make a measurement using a metric and interpret the results. There are two types of Parameters: Fixed andRun-time parameters.Runtime. For the Fixed Parameters, the value of the variable is specified in the Performance Metrics RegistryentryEntry and different Fixed Parameter values results in different Registered Performance Metrics. For theRun-timeRuntime Parameters, the value of the variable is defined when themetric measurement methodMetric Measurement Method is executed and a given Registered Performance Metric supports multiple values for theparameter.Parameter. AlthoughRun-timeRuntime Parameters do not change the fundamental nature of the Performance Metric's definition, some have substantial influence on the network property being assessed and interpretation of theresults. <list> <t>Note:results.</dd> </dl> <aside><t>Note: Consider the case of packet loss in the following two Active Measurement Method cases. The first case is packet loss as background loss where theRun-timeRuntime Parameter set includes a very sparse Poissonstream,stream and only characterizes the times when packets were lost. Actual user streams likely see much higher loss at these times, due to tail drop or radio errors. The second case is packet loss ratio asinversethe complimentary probability ofthroughputdelivery ratio where theRun-timeRuntime Parameter set includes a very dense, bursty stream, and characterizes the loss experienced by a stream that approximates a user stream. These are both"loss"Loss metrics", but the difference in interpretation of the results is highly dependent on theRun-timeRuntime Parameters (at least), to the extreme where we are actually using loss ratio to infer itscompliment: delivered throughput.</t> </list></t> <t hangText="Activecomplimentary probability: delivery ratio.</t></aside> <dl newline="false" spacing="normal"> <dt>Active MeasurementMethod:">MethodsMethods:</dt> <dd>Methods of Measurement conducted on trafficwhichthat serves only the purpose of measurement and is generated for that reason alone, and whose traffic characteristics are known a priori. The complete definition of Active Methods is specified insection 3.4 of<xref target="RFC7799"/>.<xref target="RFC7799" sectionFormat="of" section="3.4"/>. Examples of Active Measurement Methods are themeasurement methodsMeasurement Methods for theOne wayone-way delay metric defined in <xreftarget="RFC7679"/>target="RFC7679" format="default"/> and theone for round tripround-trip delay metric defined in <xreftarget="RFC2681"/>.</t> <t hangText="Passivetarget="RFC2681" format="default"/>.</dd> <dt>Passive MeasurementMethod:">MethodsMethods:</dt> <dd>Methods of Measurement conducted on network traffic, generated by eitherfrom the(1) the end users orfrom network(2) network elements that would exist regardless of whether the measurement was being conducted or not. The complete definition of Passive Methods is specified insection 3.6 of<xreftarget="RFC7799"/>.target="RFC7799" sectionFormat="of" section="3.6"/>. One characteristic of Passive Measurement Methods is that sensitive information may beobserved, andobserved and, as a consequence, stored in the measurementsystem.</t> <t hangText="Hybridsystem.</dd> <dt>Hybrid MeasurementMethod:">Hybrid Methods are MethodsMethods:</dt> <dd>Methods of Measurement that use a combination of Active Methods and Passive Methods, to assess Active Metrics, Passive Metrics, or new metrics derived from thea prioria priori knowledge and observations of the stream of interest. The complete definition of Hybrid Methods is specified insection 3.8 of <xref target="RFC7799"/>.<!-- Some examples include IPFIX<xreftarget="RFC4656"/>, PSAMP. [RFC 5470], [RFC 5476]--></t> </list></t>target="RFC7799" sectionFormat="of" section="3.8"/>. </dd> </dl> </section> <sectiontitle="Scope">numbered="true" toc="default"> <name>Scope</name> <t>This document is intended for two different audiences:</t><t><list style="numbers"> <t>For<ol spacing="normal" type="1"> <li>For thosedefining new Registeredpreparing a candidate PerformanceMetrics,Metric, it providesspecifications and best practices to be used in deciding which Registered Performance Metrics are useful for a measurement study,criteria that the proposal <bcp14>SHOULD</bcp14> meet (see <xref target="metrics-criteria"/>). It also provides instructions for writing the text for each column of theRegisteredcandidate PerformanceMetrics,Metric andinformation onthesupporting documentationreferences required for the new Performance Metrics RegistryentryEntry (up to and including the publication of one or more immutable documents such as anRFC).</t> <t>ForRFC) (see <xref target="columns"/>).</li> <li>For the appointed Performance Metrics Experts and for IANA personnel administering the new IANA Performance Metrics Registry, it defines a set of acceptance criteria against whichthese proposeda candidate Registered PerformanceMetricsMetric should beevaluated.</t> </list></t> <t>Inevaluated, and requirements for the composition of a candidate Performance Metric Registry Entry.</li> </ol> <t>Other organizations that standardize performance metrics are encouraged to use the process defined in this memo to propose a candidate Registered Performance Metric. In addition, this document may be useful for other organizations who are defining a PerformanceMetric registryMetrics Registry of theirown,own and mayre-usereuse the features of the Performance Metrics Registry defined in this document.</t> <t>This Performance Metrics Registry is applicable to Performance Metricsissuedderived from Active Measurement, Passive Measurement, and any other form of Performance Metric. ThisregistryRegistry is designed to encompass Performance Metrics developed throughout the IETF and especially for the technologies specified in the following working groups: IPPM, XRBLOCK, IPFIX, and BMWG. This document analyzes a prior attempt to set up a Performance MetricsRegistry,Registry and the reasons why this design was inadequate <xreftarget="RFC6248"/>. Finally, this document gives a set of guidelines for requesters and expert reviewers of candidate Registered Performance Metrics.</t> <t>This document makes no attempt to populatetarget="RFC6248" format="default"/>. </t> <t><xref target="RFC8912" format="default"/> populates thePerformance Metricsnew Registry withinitial entries; the related memo <xref target="I-D.ietf-ippm-initial-registry"/> proposesthe initial set ofregsitryentries.</t> </section> <sectiontitle="Motivationnumbered="true" toc="default"> <name>Motivations forathe Performance MetricsRegistry">Registry</name> <t>In this section, we detail several motivations for the Performance Metrics Registry.</t> <sectiontitle="Interoperability">numbered="true" toc="default"> <name>Interoperability</name> <t>As with any IETFregistry,Registry, the primary intention is to manage the registration ofidentifiersIdentifiers for use within one or more protocols. In the particular case of the Performance Metrics Registry, there are two types of protocols that will use the Performance Metrics in the Performance Metrics Registry during their operation (by referring to theIndexindex values):<list style="symbols"> <t>Control protocol: This</t> <dl newline="false" spacing="normal"> <dt>Control Protocol:</dt><dd>This type of protocol is used to allow one entity to request that another entitytoperform a measurement using a specific metric defined by the Performance Metrics Registry. One particular example is theLMAPLarge-scale Measurement of Broadband Performance (LMAP) framework <xreftarget="RFC7594"/>.target="RFC7594" format="default"/>. Using the LMAP terminology, the Performance Metrics Registry is used in the LMAP ControlprotocolProtocol to allow a Controller to schedule ameasurement taskMeasurement Task for one or more Measurement Agents. In order to enable this use case, the entriesofin the Performance Metrics Registry must be sufficiently defined to allow a Measurement Agent implementation to trigger a specificmeasurement taskMeasurement Task upon the reception of acontrol protocolControl Protocol message. This requirement heavily constrains thetypetypes of entries that are acceptable for the Performance MetricsRegistry. <!--Further considerations about this are captured in the Guidelines for metric registry allocations (cross reference to another section of this document or to a different document).--></t> <t>Report protocol: ThisRegistry.</dd> <dt>Report Protocol:</dt><dd>This type of protocol is used to allow an entity to reportmeasurement resultsMeasurement Results to another entity. By referencing to a specific Registered PerformanceMetrics Registry,Metric, it is possible to properly characterize themeasurement resultMeasurement Result data being reported. Using the LMAP terminology, the Performance Metrics Registry is used in the LMAP ReportprotocolProtocol to allow a Measurement Agent to reportmeasurement resultsMeasurement Results to aCollector.</t> </list>Collector.</dd> </dl> <t> It should be noted that the LMAP framework explicitly allows for using not only the IANA-maintained Performance Metrics Registry but also other registries containing Performance Metrics, i.e., either (1) registries defined by other organizations orprivate ones.(2) private registries. However, others who are creatingRegistriesregistries to be used in the context of an LMAP framework are encouraged to use the Registry format defined in this document, because this makes it easier for developers of LMAP Measurement Agents(MAs)to programmatically use information found in those otherRegistries'registries' entries.</t> </section> <sectiontitle="Single pointnumbered="true" toc="default"> <name>Single Point ofreferenceReference for PerformanceMetrics">Metrics</name> <t>A Performance Metrics Registry serves as a single point of reference for Performance Metrics defined in different working groups in the IETF. As we mentioned earlier, there are severalWGsworking groups that define Performance Metrics in theIETFIETF, and it is hard to keep track of all of them. This results in multiple definitions of similar Performance Metrics that attempt to measure the same phenomena but in slightly different (and incompatible) ways. Having aregistryRegistry would allow the IETF community and others to have a single list of relevant Performance Metrics defined by the IETF (and others, where appropriate). The single list is also an essential aspect of communication about Performance Metrics, where different entities that request measurements, execute measurements, and report the results can benefit from a common understanding of the referenced Performance Metric.</t> </section> <sectiontitle="Side benefits">numbered="true" toc="default"> <name>Side Benefits</name> <t>There are a couple of side benefits of having such aregistry.Registry. First, the Performance Metrics Registry could serve as an inventory of useful and used PerformanceMetrics,Metrics that are normally supported by different implementations ofmeasurement agents.Measurement Agents. Second, the results of measurements using the Performance Metrics should be comparable even if they are performed by different implementations and in different networks, as the Performance Metric is properly defined. BCP 176 <xreftarget="RFC6576"/>target="RFC6576" format="default"/> examines whether the results produced by independent implementations are equivalent in the context of evaluating the completeness and clarity of metric specifications.This<xref target="RFC6576"/> is a BCP <xref target="RFC2026"/> that defines thestandards trackStandards Track advancement testing for(active)(Active) IPPMmetrics,Metrics, and the same process will likely suffice to determine whether Registered Performance Metrics are sufficiently well specified to result in comparable (or equivalent) results. If a Registered PerformanceMetrics which haveMetric has undergone suchtesting SHOULDtesting, this <bcp14>SHOULD</bcp14> benoted,noted in "Comments and Remarks" (see <xref target="remarks"/>), with a reference to the test results.</t> </section> </section> <sectiontitle="Criteriaanchor="metrics-criteria" numbered="true" toc="default"> <name>Criteria for Performance MetricsRegistration">Registration</name> <t>It is neither possible nor desirable to populate the Performance Metrics Registry with all combinations of Parameters of all Performance Metrics.TheA Registered PerformanceMetrics SHOULDMetric <bcp14>SHOULD</bcp14> be:<list style="numbers"> <t>interpretable</t> <ol spacing="normal" type="1"> <li>Interpretable by theuser.</t> <t>implementablehuman user.</li> <li>Implementable by the software or hardwaredesigner,</t> <t>deployabledesigner.</li> <li>Deployable by networkoperators,</t> <t>accurateoperators.</li> <li>Accurate in terms of producing equivalent results, and for interoperability and deployment acrossvendors,</t> <t>Operationallyvendors.</li> <li>Operationally useful, so that it has significant industry interest and/or has seendeployment,</t> <t>Sufficientlydeployment.</li> <li>Sufficiently tightly defined, so that different values for theRun-timeRuntime Parametersdoesdo not change the fundamental nature of themeasurement, normeasurement or change the practicality of itsimplementation.</t> </list>Inimplementation.</li> </ol> <t>In essence, there needs to be evidence thata(1) a candidate Registered Performance Metric has significant industryinterest,interest or has seendeployment,deployment andthere(2) there is agreement that the candidate Registered Performance Metric serves its intended purpose.</t> </section> <!-- start here --> <sectiontitle="Performance Metricnumbered="true" toc="default"> <name>Performance Metrics Registry: Priorattempt">Attempt</name> <t>There was a previous attempt to define ametric registryMetrics Registry <xreftarget="RFC4148">RFC 4148</xref>.target="RFC4148" format="default"></xref>. However, it was obsoleted by <xreftarget="RFC6248">RFC 6248</xref>target="RFC6248" format="default"></xref> because it was <!-- Begin DNE --> "found to be insufficiently detailed to uniquely identify IPPM metrics... [there was too much] variability possible when characterizing a metricexactly"exactly", <!-- End DNE --> which led to theRFC4148 registryIPPM Metrics Registry defined in <xref target="RFC4148"/> having <!-- Begin DNE --> "very few users, ifany".</t> <t>A couple ofany." <!-- End DNE --></t> <t>Three interesting additional quotes from <xreftarget="RFC6248">RFC 6248</xref>target="RFC6248" format="default"></xref> might help to understand the issues related to that registry.<list style="numbers"> <t>"It</t> <!-- Begin DNE. Had to fix second item (and AQed it, since it's a repeat of text in the first paragraph of this section). --> <ol spacing="normal" type="1"> <li>"It is not believed to be feasible or even useful to register every possible combination of Type P, metric parameters, and Stream parameters using the current structure of the IPPM MetricsRegistry."</t> <t>"TheRegistry."</li> <li>"The current registry structure has been found to be insufficiently detailed to uniquely identify IPPMmetrics."</t> <t>"Despitemetrics."</li> <li>"Despite apparent efforts to find current or even future users, no one responded to the call for interest in the RFC 4148 registry during the second half of2010."</t> </list></t>2010."</li> <!-- End DNE --> </ol> <t>The current approach learns from this by tightly defining each Registered Performance Metric with only a few variable(Run-time)(Runtime) Parameters to be specified by the measurement designer, if any. The idea is that entries in the Performance Metrics Registry stem from differentmeasurement methods whichMeasurement Methods that require input(Run-time) parameters(Runtime) Parameters to set factors likesourceSource anddestinationDestination addresses (which do not change the fundamental nature of the measurement). The downside of this approach is that it could result in a large number of entries in the Performance Metrics Registry. There is agreement that less is more in this context--- it is better to have a reduced set of useful metrics rather than a large set of metrics, some withwithquestionable usefulness.</t> <sectiontitle="Why thisnumbered="true" toc="default"> <name>Why This Attempt ShouldSucceed">Succeed</name> <t>As mentioned in the previous section, one of the main issues with the previousregistryRegistry was that the metrics contained in theregistryRegistry were too generic to be useful. This document specifies stricter criteria forperformance metricPerformance Metric registration (seesection 5),<xref target="metrics-criteria"/>) and imposes a group of Performance Metrics Experts that will provide guidelines to assess if a Performance Metric is properly specified.</t> <t>Another key difference between this attempt and the previous one is that in this case there is at least one clear user for the Performance Metrics Registry: the LMAP framework and protocol. Because the LMAP protocol will use the Performance Metrics Registry values in its operation, this actually helps to determine if a metric is properlydefined. <!--The following sentence seems very awkward to me.-->Indefined -- in particular, since we expect that the LMAPcontrol protocolControl Protocol will enable acontrollerController to request that ameasurement agent toMeasurement Agent perform a measurement using a given metric by embedding the Performance Metrics RegistryidentifierIdentifier in the protocol. Such a metric and method are properly specified if they are definedwell-enoughwell enough so that it is possible (and practical) to implement them in themeasurement agent.Measurement Agent. This was the failure of the previous attempt: aregistry entryRegistry Entry with an undefined Type-P(section 13 of <xref target="RFC2330">RFC 2330</xref>)(<xref target="RFC2330" sectionFormat="of" section="13"/>) allowsimplementationmeasurement results tobe ambiguous.</t>vary significantly.</t> </section> </section> <section anchor="columns"title="Definitionnumbered="true" toc="default"> <name>Definition of the PerformanceMetric Registry">Metrics Registry</name> <t>This Performance Metrics Registry is applicable to Performance Metrics used for Active Measurement, Passive Measurement, and any other form of Performance Measurement. Each category of measurement has unique properties, so some of the columns defined below are not applicable for a given metric category. In this case, the column(s)SHOULD<bcp14>SHOULD</bcp14> be populated with the"NA""N/A" value(Non(Not Applicable). However, the"NA""N/A" valueMUST NOT<bcp14>MUST NOT</bcp14> be used by any metric in the following columns: Identifier, Name, URI, Status, Requester, Revision, Revision Date, Description. In the future, a new category of metrics could require additional columns, and adding new columns is a recognized form ofregistryRegistry extension. The specification defining the new column(s)MUST<bcp14>MUST</bcp14> give general guidelines for populating the new column(s) for existing entries.</t> <t>The columns of the Performance Metrics Registry are defined below. The columns are grouped into "Categories" to facilitate the use of theregistry.Registry. Categories are described at the7.x"Section 7.x" heading level, and columns are described at the7.x.y"Section 7.x.y" heading level. TheFigurefigure below illustrates this organization. An entry (row) therefore gives a complete description of a Registered Performance Metric.</t> <t>Each column serves as acheck-listchecklist item and helps to avoid omissions during registration andexpert review. <figure> <artwork><![CDATA[======================================================================= Legend: RegistryExpert Review <xref target="RFC8126"/>. </t> <t>Registry Categories and Columns are shown belowas:in this format:</t> <artwork name="" type="" align="left" alt=""><![CDATA[ Category ------------------... Column | Column |...=======================================================================]]></artwork> <artwork name="" type="" align="left" alt=""><![CDATA[ Summary---------------------------------------------------------------------------------------------------------------------------------------- Identifier | Name | URI | Desc. | Reference | ChangeController| Ver | | | | | | Controller | Metric Definition ----------------------------------------- Reference Definition | Fixed Parameters | Method of Measurement --------------------------------------------------------------------- Reference | Packet | Traffic | Sampling |Run-timeRuntime | Role | Method | Stream | Filter | Distribution | Parameters | | | Generation | Output ----------------------------------------- Type | Reference | Units | Calibration | | Definition | | | Administrative Information------------------------------------------------------------------------- Status |Requester | Rev |Rev.DateRev. Date | Comments and Remarks-------------------- ]]></artwork> </figure></t>--------------------]]></artwork> <t>There is a blank template of the Registry template provided inSection 11<xref target="blank-reg-template"/> of this memo.</t> <sectiontitle="Summary Category"> <section title="Identifier"> <t>Anumbered="true" toc="default"> <name>Summary Category</name> <section numbered="true" toc="default"> <name>Identifier</name> <t>This column provides a numericidentifierIdentifier for the Registered Performance Metric.This identifier MUSTThe Identifier of each Registered Performance Metric <bcp14>MUST</bcp14> beunique withinunique. Note that revising a Metric according to the process in <xref target="revise-reg-perf-metrics"/> creates a new entry in the Performance MetricsRegistry.</t>Registry with the same identifier.</t> <t>The Registered Performance Metric uniqueidentifierIdentifier is an unbounded integer (range 0 to infinity).</t> <t>The Identifier 0 should be Reserved. The Identifier values from 64512 to6553665535 are reserved for private or experimental use, and the user may encounter overlapping uses.</t> <t>When addingnewlynew Registered Performance Metrics to the Performance Metrics Registry, IANASHOULD<bcp14>SHOULD</bcp14> assign the lowest availableidentifierIdentifier to the new Registered Performance Metric.</t> <t>If a Performance Metrics Expert providing review determines that there is a reason to assign a specific numericidentifier,Identifier, possibly leaving a temporary gap in the numbering, then the Performance Metrics ExpertSHALL<bcp14>SHALL</bcp14> inform IANA of this decision.</t> </section> <sectiontitle="Name">anchor="name-sec7-1-2" numbered="true" toc="default"> <name>Name</name> <t>As thenameName of a Registered Performance Metric is the first thing a potential humanimplementorimplementer will use when determining whether it is suitable for their measurement study, it is important to be as precise and descriptive as possible. In the future, users will review thenamesNames to determine if the metric they want to measure has already been registered, or if a similar entry isavailableavailable, as a basis for creating a new entry.</t> <t>Names are composed of the following elements, separated by an underscore character "_":</t><t>MetricType_Method_SubTypeMethod_... Spec_Units_Output</t> <t><list style="symbols"> <t>MetricType: a<ul empty="true" spacing="normal"> <li>MetricType_Method_SubTypeMethod_... Spec_Units_Output</li> </ul> <dl newline="false" spacing="normal"> <dt>MetricType:</dt><dd>A combination of the directional properties and the metric measured, such as and not limitedto:<list style="empty"> <t>RTDelay (Round Trip Delay)</t> <t>RTDNS (Responseto:</dd> </dl> <table anchor="metric-type"> <tbody> <tr> <td>RTDelay</td> <td>Round-Trip Delay</td> </tr> <tr> <td>RTDNS</td> <td>Response Time Domain NameService)</t> <t>RLDNS (ResponseService</td> </tr> <tr> <td>RLDNS</td> <td>Response Loss Domain NameService)</t> <t>OWDelay (One Way Delay)</t> <t>RTLoss (Round Trip Loss)</t> <t>OWLoss (One Way Loss)</t> <t>OWPDV (One WayService</td> </tr> <tr> <td>OWDelay</td> <td>One-Way Delay</td> </tr> <tr> <td>RTLoss</td> <td>Round-Trip Loss</td> </tr> <tr> <td>OWLoss</td> <td>One-Way Loss</td> </tr> <tr> <td>OWPDV</td> <td>One-Way Packet DelayVariation)</t> <t>OWIPDV (One WayVariation</td> </tr> <tr> <td>OWIPDV</td> <td>One-Way Inter-Packet DelayVariation)</t> <t>OWReorder (One WayVariation</td> </tr> <tr> <td>OWReorder</td> <td>One-Way PacketReordering)</t> <t>OWDuplic (One WayReordering</td> </tr> <tr> <td>OWDuplic</td> <td>One-Way PacketDuplication)</t> <t>OWBTC (One WayDuplication</td> </tr> <tr> <td>OWBTC</td> <td>One-Way Bulk TransportCapacity)</t> <t>OWMBM (One Way Model Based Metric)</t> <t>SPMonitor (Single Point Monitor)</t> <t>MPMonitor (Multi-Point Monitor)</t> </list></t> <t>Method: OneCapacity</td> </tr> <tr> <td>OWMBM</td> <td>One-Way Model-Based Metric</td> </tr> <tr> <td>SPMonitor</td> <td>Single-Point Monitor</td> </tr> <tr> <td>MPMonitor</td> <td>Multi-Point Monitor</td> </tr> </tbody> </table> <dl newline="false" spacing="normal"> <dt>Method:</dt><dd>One of the methods defined in <xreftarget="RFC7799"/>,target="RFC7799" format="default"/>, such as and not limitedto:<list style="empty"> <t>Active (dependsto:</dd> </dl> <table anchor="methods"> <tbody> <tr> <td>Active</td> <td>depends on a dedicated measurement packet stream and observations of thestream)</t> <t>Passive (dependsstream as described in <xref target="RFC7799"/></td> </tr> <tr> <td>Passive</td> <td>depends *solely* on observation of one or more existing packetstreams)</t> <t>HybridType1 (observationsstreams as described in <xref target="RFC7799"/></td> </tr> <tr> <td>HybridType1</td> <td>Hybrid Type I observations on one stream that combine bothactiveActive Methods andpassive methods)</t> <t>HybridType2 (observationsPassive Methods as described in <xref target="RFC7799"/></td> </tr> <tr> <td>HybridType2</td> <td>Hybrid Type II observations on two or more streams that combine bothactiveActive Methods andpassive methods)</t> <t>Spatial (Spatial Metric of RFC5644)</t> </list></t> <t>SubTypeMethod: OnePassive Methods as described in <xref target="RFC7799"/></td> </tr> <tr> <td>Spatial</td> <td>spatial metric as described in <xref target="RFC5644"/></td> </tr> </tbody> </table> <dl newline="false" spacing="normal"> <dt>SubTypeMethod:</dt><dd>One or moresub-typessubtypes to further describe the features of the entry, such as and not limitedto:<list style="empty"> <t>ICMP (Internetto:</dd> </dl> <!-- Reviewer: Instances of "xxxx" and "RFCXXXX" are DNE. --> <table anchor="SubTypeMethod"> <tbody> <tr> <td>ICMP</td> <td>Internet Control MessageProtocol)</t> <t>IP (Internet Protocol)</t> <t>DSCPxx (whereProtocol</td> </tr> <tr> <td>IP</td> <td>Internet Protocol</td> </tr> <tr> <td>DSCPxx</td> <td>where xx is replaced by a Diffserv codepoint)</t> <t>UDP (Userpoint</td> </tr> <tr> <td>UDP</td> <td>User DatagramProtocol)</t> <t>TCP (TransportProtocol</td> </tr> <tr> <td>TCP</td> <td>Transport ControlProtocol)</t> <t>QUIC (QUICProtocol</td> </tr> <tr> <td>QUIC</td> <td>QUIC transportprotocol)</t> <t>HS (Hand-Shake,protocol</td> </tr> <tr> <td>HS</td> <td>Hand-Shake, such as TCP's 3-wayHS)</t> <t>Poisson (PacketHS</td> </tr> <tr> <td>Poisson</td> <td>packet generation using Poissondistribution)</t> <t>Periodic (Periodicdistribution</td> </tr> <tr> <td>Periodic</td> <td>periodic packetgeneration)</t> <t>SendOnRcv (Sendergeneration</td> </tr> <tr> <td>SendOnRcv</td> <td>sender keeps one packetin-transitin transit by sending when previous packetarrives)</t> <t>PayloadxxxxB (wherearrives</td> </tr> <tr> <td>PayloadxxxxB</td> <td>where xxxx is replaced by an integer, the number of octets or 8-bit Bytes in thePayload))</t> <t>SustainedBurst (CapacityPayload</td> </tr> <tr> <td>SustainedBurst</td> <td>capacity test, worstcase)</t> <t>StandingQueue (testcase</td> </tr> <tr> <td>StandingQueue</td> <td>test of bottleneck queuebehavior)</t> <t/> </list>SubTypeMethodbehavior</td> </tr> </tbody> </table> <t indent="3">SubTypeMethod values are separated by a hyphen "-" character, which indicates that they belong to thiselement,element and that their order is unimportant when consideringnameName uniqueness.</t><t>Spec: An<dl newline="false" spacing="normal"> <dt>Spec:</dt><dd>An immutable documentidentifierIdentifier combined with a document sectionidentifier.Identifier. For RFCs, this consists of the RFC number and major section number that specifies this RegistryentryEntry in the formRFCXXXXsecY, such as"RFCXXXXsecY", e.g., RFC7799sec3. Note:theThe RFC number is not thePrimary Referenceprimary reference specification for the metricdefinition, such asdefinition (e.g., <xreftarget="RFC7679"/>target="RFC7679" format="default"/> as the primary reference specification forOne-way Delay;one-way delay metrics); it will contain the placeholder "RFCXXXXsecY" until the RFC number is assigned to the specifyingdocument,document and would remain blank inprivate registry entriesPrivate Registry Entries without a corresponding RFC. Anticipating the "RFC10K" problem, the number of the RFC continues to replaceRFCXXXX"RFCXXXX", regardless of the number of digits in the RFC number. Anticipating Registry Entries from other standards bodies, the form of this Name ElementMUST<bcp14>MUST</bcp14> be proposed and reviewed for consistency and uniqueness by the ExpertReviewer.</t> <t>Units: TheReviewer.</dd> <dt>Units:</dt><dd><t>The units of measurement for the output, such as and not limitedto:<list style="empty"> <t>Seconds</t> <t>Ratio (unitless)</t> <t>Percent (valueto:</t></dd> </dl> <table anchor="units"> <tbody> <tr> <td>Seconds</td> <td></td> </tr> <tr> <td>Ratio</td> <td>unitless</td> </tr> <tr> <td>Percent</td> <td>value multiplied by100%)</t> <t>Logical (1 or 0)</t> <t>Packets</t> <t>BPS (Bits100%</td> </tr> <tr> <td>Logical</td> <td>1 or 0</td> </tr> <tr> <td>Packets</td> <td></td> </tr> <tr> <td>BPS</td> <td>bits perSecond)</t> <t>PPS (Packetssecond</td> </tr> <tr> <td>PPS</td> <td>packets perSecond)</t> <t>EventTotal (for unit-less counts)</t> <t>Multiple (moresecond</td> </tr> <tr> <td>EventTotal</td> <td>for unitless counts</td> </tr> <tr> <td>Multiple</td> <td>more than one type ofunit)</t> <t>Enumerated (aunit</td> </tr> <tr> <td>Enumerated</td> <td>a list ofoutcomes)</t> <t>Unitless</t> </list></t> <t>Output: Theoutcomes</td> </tr> <tr> <td>Unitless</td> <td></td> </tr> </tbody> </table> <dl newline="false" spacing="normal"> <dt>Output:</dt><dd>The type of output resulting from measurement, such as and not limitedto:<list style="empty"> <t>Singleton</t> <t>Raw (multiple Singletons)</t> <t>Count</t> <t>Minimum</t> <t>Maximum</t> <t>Median</t> <t>Mean</t> <t>95Percentile (95th Percentile)</t> <t>99Percentile (99th Percentile)</t> <t>StdDev (Standard Deviation)</t> <t>Variance</t> <t>PFI (Pass, Fail, Inconclusive)</t> <t>FlowRecords (descriptionsto:</dd> </dl> <table anchor="output"> <tbody> <tr> <td>Singleton</td> <td></td> </tr> <tr> <td>Raw</td> <td>multiple singletons</td> </tr> <tr> <td>Count</td> <td></td> </tr> <tr> <td>Minimum</td> <td></td> </tr> <tr> <td>Maximum</td> <td></td> </tr> <tr> <td>Median</td> <td></td> </tr> <tr> <td>Mean</td> <td></td> </tr> <tr> <td>95Percentile</td> <td>95th percentile</td> </tr> <tr> <td>99Percentile</td> <td>99th percentile</td> </tr> <tr> <td>StdDev</td> <td>standard deviation</td> </tr> <tr> <td>Variance</td> <td></td> </tr> <tr> <td>PFI</td> <td>pass, fail, inconclusive</td> </tr> <tr> <td>FlowRecords</td> <td>descriptions of flowsobserved)</t> <t>LossRatio (lostobserved</td> </tr> <tr> <td>LossRatio</td> <td>lost packets to total packets,<=1)</t> </list></t> </list>An example is:</t> <t>RTDelay_Active_IP-UDP-Periodic_RFCXXXXsecY_Seconds_95Percentile</t> <t>as<=1</td> </tr> </tbody> </table> <t>An example, as described insection 4 of<xreftarget="I-D.ietf-ippm-initial-registry"/>.</t>target="RFC8912" sectionFormat="of" section="4"/>, is</t> <ul empty="true" spacing="normal"> <li>RTDelay_Active_IP-UDP-Periodic_RFC8912sec4_Seconds_95Percentile</li> </ul> <t>Note that private registries following the format described hereSHOULD<bcp14>SHOULD</bcp14> use the prefix "Priv_" on anynameName to avoid unintended conflicts (further considerations are described insection 10).<xref target="iana-cons"/>). Privateregistry entriesRegistry Entries usually have no specifyingRFC, thusRFC; thus, the Spec: element has no clear interpretation.</t> </section> <sectiontitle="URI">numbered="true" toc="default"> <name>URI</name> <t>TheURIsURI columnMUST<bcp14>MUST</bcp14> contain a URL <xreftarget="RFC3986"/>target="RFC3986" format="default"/> that uniquely identifies and locates themetric entryMetric Entry so it is accessible through the Internet. The URL points to a file containing all of the human-readable information for oneregistry entry.Registry Entry. The URLSHALL<bcp14>SHALL</bcp14> reference a target file that is preferably HTML-formatted and contains URLs to referenced sections ofHTML-izedHTMLized RFCs, or other reference specifications. These target files for different entries can be more easily edited andre-usedreused when preparing new entries. The exact form of the URL for each target file, and the target file itself, will be determined by IANA and reside on"iana.org". The<eref target="https://www.iana.org/" brackets="angle"/>. <xref target="RFC8912" sectionFormat="of" section="4"/>, as well as subsequent major sections of<xref target="I-D.ietf-ippm-initial-registry"/>that document, provide an example of a target file in HTMLform (sections 4 and higher).</t>form.</t> </section> <sectiontitle="Description">numbered="true" toc="default"> <name>Description</name> <t>A Registered Performance Metric description is a written representation of a particular Performance Metrics Registryentry.Entry. It supplements the Registered Performance MetricnameName to help Performance Metrics Registry users select relevant Registered Performance Metrics.</t> </section> <sectiontitle="Reference">numbered="true" toc="default"> <name>Reference</name> <t>This entry gives the specification containing the candidateregistry entry whichRegistry Entry that was reviewed andagreed,agreed upon, if such an RFC or other specification exists.</t> </section> <sectiontitle="Change Controller">numbered="true" toc="default"> <name>Change Controller</name> <t>This entry names the entity responsible for approving revisions to theregistry entry,Registry Entry andSHALL<bcp14>SHALL</bcp14> provide contact information (for an individual, where appropriate).</t> </section> <sectiontitle="Versionnumbered="true" toc="default"> <name>Version (of RegistryFormat)">Format)</name> <t>Thisentrycolumn gives the version number for theregistryRegistry format used, at the time the Performance Metric is registered. The formatused. Formatscomplying with this memo MUST use 1.0.The version number SHALL NOT change unless aA new RFCis publishedthat changes theregistryRegistry format will designate a new version number corresponding to that format. The version number ofregistry entriesRegistry Entries SHALL NOT change unless theregistry entryRegistry Entry is updated to reflect the Registry format (following the procedures insection 8).</t><xref target="managing-perf-metric-grp"/>).</t> </section><!-- <section title="Reference Specification(s)"> <t>Registered Performance Metrics that follow the common columns must provide the reference specification(s) on which the Registered Performance Metric is based.</t> </section>--></section> <sectiontitle="Metricnumbered="true" toc="default"> <name>Metric DefinitionCategory">Category</name> <t>This category includes columns to prompt all necessary details related to the metric definition, including the immutable document reference and values of input factors, calledfixed parameters,"Fixed Parameters", which are left open in the immutabledocument,document but have a particular value defined by theperformance metric.</t>Performance Metric.</t> <sectiontitle="Reference Definition">numbered="true" toc="default"> <name>Reference Definition</name> <t>This entry provides a reference (or references) to the relevantsection(s)sections of thedocument(s)document or documents that define the metric, as well as any supplemental information needed to ensure an unambiguous definition for implementations.TheA given reference needs to be an immutable document, such as an RFC; for other standards bodies, it is likely to be necessary to reference a specific, dated version of a specification.</t> </section> <sectiontitle="Fixed Parameters">numbered="true" toc="default"> <name>Fixed Parameters</name> <t>Fixed Parameters are Parameters whosevaluevalues must be specified in the Performance Metrics Registry. The measurement system uses these values.</t> <t>Where referenced metrics supply a list of Parameters as part of their descriptive template, asub-setsubset of the Parameters will be designated as Fixed Parameters. As an example foractive metrics,Active Metrics, Fixed Parameters determine most or all of the IPPMFrameworkframework convention "packets of Type-P" as described in <xreftarget="RFC2330"/>,target="RFC2330" format="default"/>, such as transport protocol, payload length, TTL, etc. An example forpassive metricsPassive Metrics is for an RTP packet loss calculation that relies on the validation of a packet asRTPRTP, which is a multi-packet validation controlled by the MIN_SEQUENTIAL variable as defined by <xreftarget="RFC3550"/>.target="RFC3550" format="default"/>. Varying MIN_SEQUENTIAL values can alter the lossreportreport, and thisvaluevariable could be set as a Fixed Parameter.</t> <t>ParametersMUST<bcp14>MUST</bcp14> have well-defined names. For human readers, thehanging indenthanging-indent style is preferred, and any Parameter names and definitions that do not appear in the Reference Method SpecificationMUST<bcp14>MUST</bcp14> appear in this column (orRun-timethe Runtime Parameters column).</t> <t>ParametersMUST<bcp14>MUST</bcp14> have a well-specified data format.</t> <t>A Parameterwhichthat is a Fixed Parameter for one Performance Metrics RegistryentryEntry may be designated as aRun-timeRuntime Parameter for another Performance Metrics Registryentry.</t>Entry.</t> </section> </section> <sectiontitle="Methodnumbered="true" toc="default"> <name>Method of MeasurementCategory">Category</name> <t>This category includes columns for references to relevant sections of the immutable document(s) and any supplemental information needed to ensure an unambiguous method for implementations.</t> <sectiontitle="Reference Method">numbered="true" toc="default"> <name>Reference Method</name> <t>This entry provides references to relevant sections of immutable documents, such as RFC(s) (for other standards bodies, it is likely to be necessary to reference a specific, dated version of a specification) describing themethodMethod ofmeasurement,Measurement, as well as any supplemental information needed to ensure unambiguous interpretation for implementations referring to the immutable document text.</t> <t>Specifically, this section should include pointers to pseudocode or actual code that could be used for an unambiguous implementation.</t> </section> <sectiontitle="Packetnumbered="true" toc="default"> <name>Packet StreamGeneration">Generation</name> <t>This column applies to Performance Metrics that generate traffic as part of their Measurement Method,includingincluding, but not necessarily limitedtoto, Activemetrics.Metrics. The generated traffic is referred to as astream"stream", and this column describes its characteristics.</t><!-- <t>Some metrics, such as those intended for passive monitoring or RTCP and RTCP-XR metrics, will not specify an entry for this column.</t> --><t>Each entry for this column contains the following information:<list style="symbols"> <t>Value: The</t> <dl newline="false" spacing="normal"> <dt>Value:</dt><dd>The name of the packet stream schedulingdiscipline</t> <t>Reference: thediscipline</dd> <dt>Reference:</dt><dd>The specification where theparametersParameters of the stream aredefined</t> </list></t>defined</dd> </dl> <t>The packet generation stream may requireparametersParameters such as the average packet rate and distribution truncation value for streams with Poisson-distributed inter-packet sending times.In caseIf suchparametersParameters are needed, they should be includedeitherin either the FixedparameterParameters column orintherun time parameterRuntime Parameters column, depending on whether they will be fixed or will be an input for the metric.</t> <t>The simplest example of stream specification isSingletonsingleton scheduling (see <xreftarget="RFC2330"/>),target="RFC2330" format="default"/>), where a single atomic measurement is conducted. Each atomic measurement could consist of sending a single packet (such as a DNS request) or sending several packets (for example, to request awebpage).web page). Other streams support a series of atomic measurementsin a "sample", withusing pairs of packets, where the packet stream follows a schedule defining the timing betweeneachtransmittedpacketpackets, andsubsequent measurement.an atomic measurement assesses the reception time between successive packets (e.g., a measurement of Inter-Packet Delay Variation). More complex streams and measurement relationships are possible. Principally, two different streams are used in IPPMmetrics, PoissonMetrics: (1) Poisson, distributed as described in <xreftarget="RFC2330"/>target="RFC2330" format="default"/> andPeriodic(2) periodic, as described in <xreftarget="RFC3432"/>.target="RFC3432" format="default"/>. Both Poisson andPeriodicperiodic have their own uniqueparameters,Parameters, and the relevant set ofparametersParameter names and values should be includedeitherin either the Fixed Parameters column orintheRun-time parameterRuntime Parameters column.</t> </section> <sectiontitle="Traffic Filter">numbered="true" toc="default"> <name>Traffic Filter</name> <t>This column applies to Performance Metrics that observe packets flowing through (the device with) themeasurement agent i.e.Measurement Agent, i.e., packets thatisare not necessarily addressed to themeasurement agent.Measurement Agent. Thisincludesincludes, but is not limitedtoto, Passive Metrics. The filter specifies the traffic that is measured. This includes protocol field values/ranges, such as address ranges, and flow or sessionidentifiers.</t>Identifiers.</t> <t>Thetraffic filterTraffic Filter itself depends on the needs of the metric itself and a balance of an operator's measurement needs and a user's need for privacy. Mechanics for conveying the filter criteria might be the BPF(Berkley(Berkeley Packet Filter) or PSAMP (Packet Sampling) <xreftarget="RFC5475"/>target="RFC5475" format="default"/> Property MatchFilteringFiltering, which reuses IPFIX <xreftarget="RFC7012"/>.target="RFC7012" format="default"/>. An example BPF string for matching TCP/80 traffic to remotedestinationDestination net 192.0.2.0/24 would be "dst net 192.0.2.0/24 and tcp dst port 80". More complex filter enginesmight be supported by the implementation that mightmay allow for matching using Deep Packet Inspection (DPI) technology.</t> <t>Thetraffic filterTraffic Filter includes the following information:<list> <t>Type: the</t> <dl newline="false" spacing="normal"> <dt>Type:</dt><dd>The type oftraffic filterTraffic Filter used,e.g.e.g., BPF, PSAMP, OpenFlow rule,etc.etc., as defined by a normativereference</t> <t>Value: thereference</dd> <dt>Value:</dt><dd>The actual set of rulesexpressed</t> </list></t>expressed</dd> </dl> </section> <sectiontitle="Sampling Distribution"> <t><!--This sentence seems awkward to me.-->Thenumbered="true" toc="default"> <name>Sampling Distribution</name> <t>The sampling distributiondefinesdefines, out of all of the packets that match thetraffic filter,Traffic Filter, which one or more of those packets are actually used for the measurement. One possibility is"all""all", which implies that all packets matching the TrafficfilterFilter are considered, but there may be other sampling strategies. It includes the following information:<list> <t>Value: the</t> <dl newline="false" spacing="normal"> <dt>Value:</dt><dd>The name of the samplingdistribution</t> <t>Reference definition: pointerdistribution</dd> <dt>Reference definition:</dt><dd>Pointer to the specification where the sampling distribution is properlydefined.</t> </list></t>defined</dd> </dl> <t>The sampling distribution may requireparameters. In caseParameters. If suchparametersParameters are needed, they should be includedeitherin either the FixedparameterParameters column orintherun time parameterRuntime Parameters column, depending on whether they will be fixed or will be an input for the metric.</t><t>Sampling<t>PSAMP is documented in "Sampling and Filtering Techniques for IP PacketSelection are documented in the PSAMP (Packet Sampling)Selection" <xreftarget="RFC5475"/>,target="RFC5475" format="default"/>, whilethe"A Framework for Packet Selection andReporting,Reporting" <xreftarget="RFC5474"/>target="RFC5474" format="default"/> provides more background information. The sampling distributionparametersParameters might be expressed in terms of theInformationmodel described in "Information Model for Packet SamplingExports,Exports" <xreftarget="RFC5477"/>,target="RFC5477" format="default"/> and theFlowprocess provided in "Flow SelectionTechniques,Techniques" <xreftarget="RFC7014"/>.</t>target="RFC7014" format="default"/>.</t> </section> <sectiontitle="Run-time Parameters"> <t>Run-Timenumbered="true" toc="default"> <name>Runtime Parameters</name> <t>In contrast to the Fixed Parameters, Runtime Parameters are Parameters thatmust be determined, configured intodo not change the fundamental nature of the measurementsystem,andreported with the results for the context to be complete. However, thetheir valuesof these parameters isare not specified in the Performance MetricsRegistry (like the Fixed Parameters), rather these parametersRegistry. They arelistedleft as variables in the Registry, as an aid to the measurement system implementer oruser (they must be left as variables, anduser. Their values are supplied onexecution).</t>execution, configured into the measurement system, and reported with the Measurement Results (so that the context is complete).</t> <t>Where metrics supply a list of Parameters as part of their descriptive template, asub-setsubset of the Parameters will be designated asRun-TimeRuntime Parameters.</t> <t>ParametersMUST<bcp14>MUST</bcp14> havewell definedwell-defined names. For human readers, thehanging indenthanging-indent style is preferred, and the names and definitions that do not appear in the Reference Method SpecificationMUST<bcp14>MUST</bcp14> appear in this column.</t> <t>AData Formatdata format for eachRun-timeRuntime ParameterMUST<bcp14>MUST</bcp14> be specified in this column, to simplify the control and implementation of measurement devices. For example,parametersParameters that include an IPv4 address can be encoded as a32 bit32-bit integer(i.e.(i.e., a binarybase64 encodedbase64-encoded value) orip-address"ip&nbhy;address" as defined in <xreftarget="RFC6991"/>.target="RFC6991" format="default"/>. The actual encoding(s) used must be explicitly defined for eachRun-time parameter.Runtime Parameter. IPv6 addresses and optionsMUST<bcp14>MUST</bcp14> be accommodated, allowing Registered Performance Metrics to be used in that address family. Other address families arepermissable.</t>permissible.</t> <t>Examples ofRun-timeRuntime Parameters include IP addresses, measurement point designations, start times and end times for measurement, and other information essential to themethodMethod ofmeasurement.</t>Measurement.</t> </section> <sectiontitle="Role">numbered="true" toc="default"> <name>Role</name> <t>In somemethodsMethods ofmeasurement,Measurement, there may be severalrolesRoles defined, e.g., for a one-way packet delayactive measurementActive Measurement, there is onemeasurement agentMeasurement Agent that generates the packets and anotheragentAgent that receives the packets. This column contains the name of the Role(s) for this particular entry. In the one-way delay example above, there should be two entries in the Registry's Roleregistrycolumn, one for each Role (Source and Destination). When ameasurement agentMeasurement Agent is instructed to perform the "Source" Role for the one-way delay metric, theagentAgent knows that it is required to generate packets. The values for this field are defined in thereference methodReference Method ofmeasurementMeasurement (and this frequently results in abbreviatedroleRole names such as "Src").</t> <t>When the Role column of aregistry entryRegistry Entry defines more than one Role,thenthe RoleSHALL<bcp14>SHALL</bcp14> be treated as aRun-timeRuntime Parameter and supplied for execution. It should be noted that the LMAP framework <xreftarget="RFC7594"/>target="RFC7594" format="default"/> distinguishes the Role from otherRun-time Parameters, and defines a special parameter "Roles" inside the registry-grouping function list in the LMAP YANG model<xref target="RFC8194"/>.</t>Runtime Parameters.</t> </section> </section> <sectiontitle="Output Category">numbered="true" toc="default"> <name>Output Category</name> <t>For entrieswhichthat involve a stream and many singleton measurements, a statistic may be specified in this column to summarize the results to a single value. If the complete set of measured singletons is output, this will be specified here.</t> <t>Some metrics embed one specific statistic in the reference metric definition, while others allow several output types or statistics.</t> <sectiontitle="Type">numbered="true" toc="default"> <name>Type</name> <t>This column contains the name of the output type. The output type defines a single type of result that the metric produces. It can be the raw results (packet send times and singleton metrics), or it can be a summary statistic. The specification of the output typeMUST<bcp14>MUST</bcp14> define the format of the output. In some systems, format specifications will simplify both measurement implementation and collection/storage tasks. Note that if two different statistics are required from a single measurement (for example, both "Xth percentile mean" and "Raw"), then a new output type must be defined ("Xth percentile mean AND Raw"). Seethe Naming section<xref target="name-sec7-1-2"/> above for a list ofOutput Types.</t> </section> <!-- <section title="Data Format"> <t>This column provides the data format for the output. It is provided to simplify the communication with collection systems and implementation of measurement devices.</t>output types.</t> </section>--><sectiontitle="Reference Definition">numbered="true" toc="default"> <name>Reference Definition</name> <t>This column contains a pointer to the specification(s) where the output type and format are defined.</t> </section> <sectiontitle="Metric Units">numbered="true" toc="default"> <name>Metric Units</name> <t>The measured results must be expressed using some standard dimension or units of measure. This column provides the units.</t> <t>When a sample of singletons (seeSection 11 of<xref target="RFC2330"/><xref target="RFC2330" sectionFormat="of" section="11"/> for definitions of these terms) is collected, this entry will specify the units for each measured value.</t> </section> <sectiontitle="Calibration">numbered="true" toc="default"> <name>Calibration</name> <t>Some specifications for Methods of Measurement include thepossibilityability to perform an error calibration.Section 3.7.3 of<xreftarget="RFC7679"/>target="RFC7679" sectionFormat="of" section="3.7.3"/> is one example. In theregistry entry,Registry Entry, this field will identify a method of calibration for the metric,andand, when available, the measurement systemSHOULD<bcp14>SHOULD</bcp14> perform the calibration when requested and produce the output with an indication that it is the result of a calibration method. In-situ calibration could be enabled with an internal loopback that includes as much of the measurement system as possible, performs address manipulation as needed, and provides some form of isolation (e.g., deterministic delay) to avoid send-receive interface contention. Some portion of the random and systematic error can be characterized in this way.</t> <t>For one-way delay measurements, the error calibration must include an assessment of the internal clock synchronization with its external reference (this internal clock is supplying timestamps for measurement). In practice, the time offsets of clocks at both thesourceSource anddestinationDestination are needed to estimate the systematic error due to imperfect clock synchronization (the time offsets aresmoothed, thussmoothed; thus, the random variation is not usually represented in the results).</t> <t>Both internal loopback calibration and clock synchronization can be used to estimate the *available accuracy* of the Output Metric Units. For example, repeated loopback delay measurements will reveal the portion of theOutputoutput result resolutionwhichthat is the result of systemnoise,noise and is thus inaccurate.</t> </section> </section> <sectiontitle="Administrative information"> <section title="Status"> <t>Thenumbered="true" toc="default"> <name>Administrative Information</name> <section numbered="true" toc="default"> <name>Status</name> <t>This entry indicates the status of the specification of this Registered Performance Metric. Allowed values are'current''Current', 'Deprecated', and'deprecated'.‘Obsolete’. All newly definedInformation ElementsRegistered Performance Metrics have'current' status.</t>'Current' Status.</t> </section> <sectiontitle="Requester"> <t>Thenumbered="true" toc="default"> <name>Requester</name> <t>This entry indicates the requester for the Registered Performance Metric. The requesterMAY<bcp14>MAY</bcp14> be adocument, suchdocument (such asRFC,an RFC) or a person.</t> </section> <sectiontitle="Revision"> <t>Thenumbered="true" toc="default"> <name>Revision</name> <t>This entry indicates the revision number of a Registered Performance Metric, starting at 0 for Registered Performance Metrics at the time of definition and incremented by one for eachrevision.</t>revision. However, in the case of a non-backward-compatible revision, see <xref target="deprecating-metrics"/>.</t> </section> <sectiontitle="Revision Date"> <t>Thenumbered="true" toc="default"> <name>Revision Date</name> <t>This entry indicates the date of acceptanceorof the most recent revision for the Registered Performance Metric. The dateSHALL<bcp14>SHALL</bcp14> be determined by IANA and the reviewing Performance Metrics Expert.</t> </section> </section> <sectiontitle="Commentsanchor="remarks" numbered="true" toc="default"> <name>Comments andRemarks">Remarks</name> <t>Besides providing additional detailswhichthat do not appear in other categories, this openCategorycategory (single column) allowsforunforeseen issues to be addressed by simply updating this informational entry.</t> </section> </section> <sectiontitle="Processesanchor="managing-perf-metric-grp" numbered="true" toc="default"> <name>Processes for Managing the PerformanceMetricMetrics RegistryGroup">Group</name> <t>Once a Performance Metric or set of Performance Metrics has been identified for a given application, candidate Performance Metrics RegistryentryEntry specifications prepared in accordance with <xreftarget="columns"/>target="columns" format="default"/> should be submitted to IANA to follow the process for review by the PerformanceMetricMetrics Experts, as defined below. This process is also used for other changes tothea Performance MetricsRegistry,Registry Entry, such as deprecation or revision, as described later in this section.</t> <t>It is desirable that the author(s) of a candidate Performance Metrics RegistryentryEntry seek review in the relevant IETF workinggroup,group or offer the opportunity for review on the working group mailing list.</t> <sectiontitle="Adding newanchor="add-new-perf-metrics" numbered="true" toc="default"> <name>Adding New Performance Metrics to the Performance MetricsRegistry">Registry</name> <t>Requests to add Registered Performance Metrics in the Performance Metrics RegistrySHALL<bcp14>SHALL</bcp14> be submitted to IANA, which forwards the request to a designated group of experts (PerformanceMetricMetrics Experts) appointed by the IESG; these are the reviewers called for by the Specification Required<xref target="RFC8126"/>policy <xref target="RFC8126" format="default"/> defined for the Performance Metrics Registry. The PerformanceMetricMetrics Experts review the request for such things as compliance with this document, compliance with other applicable PerformanceMetric-relatedMetrics-related RFCs, and consistency with the currently defined set of Registered Performance Metrics. The most efficient path for submission begins with preparation of anInternet DraftInternet-Draft containing the proposed Performance Metrics RegistryentryEntry using the template inSection 11,<xref target="blank-reg-template"/>, so that the submission formatting will benefit from the normal IETFInternet DraftInternet-Draft submission processing (includingHTML-ization).</t>HTMLization).</t> <t>Submission to IANA may be during IESG review (leading to IETF Standards Action), where anInternet DraftInternet-Draft proposes one or more Registered Performance Metrics to be added to the Performance Metrics Registry, including the text of the proposed Registered PerformanceMetric(s).</t>Metric&wj;(s).</t> <t>If an RFC-to-be includes a Performance Metric and a proposed Performance Metrics Registryentry,Entry but the PerformanceMetric ExpertMetrics Expert's review determines that one or more of theSection 5criteria listed in <xref target="metrics-criteria"/> have not been met, then the proposed Performance Metrics Registryentry MUSTEntry <bcp14>MUST</bcp14> be removed from the text. Once evidence exists that the Performance Metric meets the criteria insection 5,<xref target="metrics-criteria"/>, the proposed Performance Metrics Registryentry SHOULDEntry <bcp14>SHOULD</bcp14> be submitted to IANA to be evaluated in consultation with the PerformanceMetricMetrics Experts for registration at that time.</t> <t>Authors of proposed Registered Performance MetricsSHOULD<bcp14>SHOULD</bcp14> review compliance with the specifications in this document to check their submissions before sending them to IANA.</t> <t>At least one PerformanceMetricMetrics Expert should endeavor to complete referred reviews in a timely manner. If the request is acceptable, the PerformanceMetricMetrics Experts signify their approval to IANA, and IANA updates the Performance Metrics Registry. If the request is not acceptable, the PerformanceMetricMetrics ExpertsMAY<bcp14>MAY</bcp14> coordinate with the requester to change the requestto be compliant, otherwiseso that it is compliant; otherwise, IANASHALL<bcp14>SHALL</bcp14> coordinate resolution of issues on behalf of the expert. The PerformanceMetricMetrics ExpertsMAY<bcp14>MAY</bcp14> choose to reject clearly frivolous or inappropriate change requests outright, but such exceptional circumstances should be rare.</t><t>This process should not in any way be construed as allowing<t>If thePerformanceproposed MetricExperts to overrule IETF consensus. Specifically, any Registered Performance Metrics that were addedis unique in a significant way, in order to properly describe thePerformance Metrics Registry with IETF consensus require IETF consensus for revision or deprecation.</t> <t>Decisions by the Performance Metric ExpertsMetric, it may beappealed as in Section 7 of <xref target="RFC8126"/>.</t> </section> <section title="Revising Registered Performance Metrics"> <!-- <t>Requestsnecessary torevise the Performance Metrics Registrypropose a new Name Element Registry, or (more likely) alinked sub-registry are submitted to IANA, which forwardsnew Entry in an existing Name Element Registry. This proposal is part of the requestto a designated group of experts (Performance Metric Experts) appointed byfor theIESG; these arenew Metric, so that it undergoes thereviewers called forsame IANA review and approval process. </t> <t>Decisions by theExpert Review [RFC5226] policy defined for thePerformance MetricsRegistry. The Performance MetricExpertsreview the request for such things as compliance with this document, compliance with other applicable Performance Metric-related RFCs, and consistency with the currently defined setmay be appealed per <xref target="RFC8126" sectionFormat="of" section="10"/>.</t> </section> <section anchor="revise-reg-perf-metrics" numbered="true" toc="default"> <name>Backward-Compatible Revision of Registered PerformanceMetrics.</t> -->Metrics</name> <t>A request forRevisionrevision is only permitted when the requested changes maintainbackward-compatibilitybackward compatibility with implementations of the prior Performance Metrics RegistryentryEntry describing a Registered Performance Metric (entries with lower revisionnumbers,numbers but having the same Identifier and Name).</t> <t>The purpose of the Status field in the Performance Metrics Registry is to indicate whether the entry for a Registered Performance Metric is'current''Current', 'Deprecated', or 'Obsolete'. The term 'deprecated' is used when an entry is replaced, either with a backwards-compatible revision (this sub-section) or'deprecated'.</t>with a non-backwards-compatible revision (in <xref target="deprecating-metrics"/>).</t> <t>In addition, no policy is defined for revising the Performance MetricentriesEntries in the IANA Registry or addressing errors therein. To be clear, changes and deprecations within the Performance Metrics Registry are notencouraged,encouraged and should be avoided to the extent possible. However, in recognition that change is inevitable, the provisions of this section address the need for revisions.</t> <t>Revisions are initiated by sending a candidate Registered Performance Metric definition to IANA,as in Section 8.1,per <xref target="add-new-perf-metrics"/>, identifying the existing Performance Metrics Registryentry,Entry, and explaining how and why the existing entry should be revised.</t> <t>The primary requirement in the definition of procedures for managing changes to existing Registered Performance Metrics is avoidance of measurement interoperability problems; the PerformanceMetricMetrics Experts must work to maintain interoperability above all else. Changes to Registered Performance Metrics may only be done in an interoperable way; necessary changes that cannot be done in a wayto allowthat allows interoperability with unchanged implementationsMUST<bcp14>MUST</bcp14> result in the creation of a new Registered Performance Metric (with a new Name, replacing the RFCXXXXsecY portion of thename)Name) and possibly the deprecation of the earlier metric.</t> <t>A change to a Registered Performance MetricSHALL<bcp14>SHALL</bcp14> be determined to bebackward-compatiblebackward compatible when:<list style="numbers"> <t>it</t> <ol spacing="normal" type="1"> <li>it involves the correction of an error that is obviously onlyeditorial; or</t> <t>iteditorial, or</li> <li>it corrects an ambiguity in the Registered Performance Metric's definition, which itself leads to issues severe enough to prevent the Registered Performance Metric's usage as originallydefined; or</t> <t>itdefined, or</li> <li>it corrects missing information in the metric definition without changing its meaning (e.g., the explicit definition of 'quantity' semantics for numeric fields without a Data Type Semanticsvalue); or</t> <t>itvalue), or</li> <li>it harmonizes with an external reference that was itselfcorrected.</t> <!-- <t>"BENOIT: NOTE THAT THERE ARE MORE RULES IN RFC 7013 SECTION 5 BUT THEY WOULD ONLY APPLY TO THE ACTIVE/PASSIVE DRAFTS. TO BE DISCUSSED."</t> --> </list></t>corrected, or</li> <li>if the current Registry format has been revised by adding a new column that is not relevant to an existing Registered Performance Metric (i.e., the new column can be safely filled in with “Not Applicable”).</li> </ol> <t>If a Performance Metric revision is deemed permissible andbackward-compatiblebackward compatible by the PerformanceMetricMetrics Experts, according to the rules in this document, IANASHOULD<bcp14>SHOULD</bcp14> execute the change(s) in the Performance Metrics Registry. The requester of the change is appended to the original requester in the Performance Metrics Registry. The Name of the revised Registered Performance Metric, including the RFCXXXXsecY portion of thename, SHALLName, <bcp14>SHALL</bcp14> remain unchanged(eveneven when the change is the result of IETF StandardsAction; theAction. The revisedregistry entry SHOULDRegistry Entry <bcp14>SHOULD</bcp14> reference the new immutable document, such as anRFC or forRFC. For other standards bodies, it is likely to be necessary to reference a specific, dated version of a specification, in an appropriate category andcolumn).</t>column.</t> <t>Each Registered Performance Metric in the Performance Metrics Registry has a revision number, starting at zero. Each change to a Registered Performance Metric following this process increments the revision number by one.</t> <t>When a revised Registered Performance Metric is accepted into the Performance Metrics Registry, the date of acceptance of the most recent revision is placed into therevisionRevision Date column of theregistryRegistry for that Registered Performance Metric.</t> <t>Where applicable, additions to Registered Performance Metrics in the form of text in the Comments or Remarks column should include the date, but such additions may not constitute a revision according to this process.</t> <t>Olderversion(s)versions of the updatedmetric entriesMetric Entries are kept in theregistryRegistry for archival purposes. The older entries are kept with all fields unmodified(version, revision date)(including Revision Date) except for thestatus field that SHALLStatus field, which <bcp14>SHALL</bcp14> be changed to"Deprecated".</t> </section> <section title="Deprecating'Deprecated'.</t> <t>This process should not in any way be construed as allowing the Performance Metrics Experts to overrule IETF consensus. Specifically, any Registered PerformanceMetrics"> <t>ChangesMetrics thatare not permissible bywere added to theabove criteriaPerformance Metrics Registry with IETF consensus require IETF consensus for revision or deprecation.</t> </section> <section anchor="deprecating-metrics" numbered="true" toc="default"> <name>Non-Backward-Compatible Deprecation of Registered PerformanceMetric's revision may only be handled by deprecation.Metrics</name> <t>This section describes how to make a non-backward-compatible update to a Registered Performance Metric. A Registered Performance MetricMAY<bcp14>MAY</bcp14> be deprecated and replacedwhen: <list style="numbers"> <t>thewhen:</t> <ol spacing="normal" type="1"> <li>the Registered Performance Metric definition has an error or shortcoming that cannot be permissibly changedas in Section 8.2 Revisingper <xref target="revise-reg-perf-metrics"/> ("Revising Registered PerformanceMetrics; or</t> <t>theMetrics"), or</li> <li>the deprecation harmonizes with an external reference that was itself deprecated through that reference's accepted deprecationmethod.</t> </list></t>method.</li> </ol> <t>A request for deprecation is sent to IANA, which passes it to the PerformanceMetricMetrics Experts for review. When deprecatingana Performance Metric, the Performance MetricdescriptionDescription in the Performance Metrics Registrymust<bcp14>MUST</bcp14> be updated to explain the deprecation, as well as to refer toanythe new PerformanceMetricsMetric created to replace the deprecated Performance Metric.</t><t>The<t>When a new, non-backward-compatible Performance Metric replaces a (now) deprecated metric, the revision number ofathe new Registered Performance Metric is incrementedupon deprecation,over the value in the deprecated version, and therevision Date updated,current date is entered aswith any revision.</t>the Revision Date of the new Registered Performance Metric.</t> <t>The intentional use of deprecated Registered Performance Metrics should result in a log entry or human-readable warning by the respective application.</t> <t>Names and Metric IDs of deprecated Registered Performance Metrics must not be reused.</t> <t>The deprecated entries are kept with allfieldsAdministrative columns unmodified, except theversion, revision date,Status field (which is changed to 'Deprecated').</t> </section> <section numbered="true" toc="default"> <name>Obsolete Registry Entries</name> <t>Existing Registry Entries may become obsolete over time due to:</t> <ol> <li>the Registered Performance Metric is found to contain considerable errors (and no one sees the value in the effort to fix it), or</li> <li>one or more critical References (or sections thereof) have been designated obsolete by the SDO, or</li> <li>other reasons brought to the attention of IANA and thestatusRegistry Experts.</li> </ol> <t>When a Performance Metric Registry Entry is declared obsolete, the Performance Metric Description in the Performance Metrics Registry is updated to explain the reasons the Entry is now obsolete and has not been replaced (Deprecation always involves replacement).</t> <t> Obsolete entries are kept with all Administrative columns unmodified, except the Status field(changed(which is changed to"Deprecated").</t> </section>'Obsolete'). </t> </section><!--<sectiontitle="Performance Metricsnumbered="true" toc="default"> <name>Registry Format Version and Future Changes/Extensions</name> <t>The Registry Format Version defined in this memo is 1.0, andother Registries"> <t>BENOIT: TBD.</t> <t>THE BASIC IDEA IS THAT PEOPLE COULD DIRECTLY DEFINE PERF. METRICS IN OTHER EXISTING REGISTRIES, FOR SPECIFIC PROTOCOL/ENCODING. EXAMPLE: IPFIX. IDEALLY, ALL PERF. METRICS SHOULD BE DEFINED IN THIS REGISTRY AND REFERS TO FROM OTHER REGISTRIES.</t>candidate Registry Entries complying with this memo <bcp14>MUST</bcp14> use 1.0.</t> <t>The Registry Format can only be updated by publishing a new RFC with the new format (Standards Action).</t> <t>When a Registered Performance Metric is created or revised, then it uses the most recent Registry Format Version.</t> <t>Only one form of Registry extension is envisaged:</t> <t indent="3">Adding columns, or both categories and columns, to accommodate unanticipated aspects of new measurements and metric categories.</t> <t>If the Performance Metrics Registry is extended in this way, the version number of future entries complying with the extension <bcp14>SHALL</bcp14> be incremented (in either the unit or the tenths digit, depending on the degree of extension).</t> </section> </section>--><sectiontitle="Security considerations">numbered="true" toc="default"> <name>Security Considerations</name> <t>Thisdraftdocument defines aregistry structure,Registry structure and does not itself introduce any new security considerations for the Internet. The definition of Performance Metrics for thisregistryRegistry may introduce some security concerns, but the mandatory references should have their own considerations for security, and such definitions should be reviewed with security in mind if the security considerations are not covered by one or more reference standards.</t> <t>The aggregated results of theperformance metricsPerformance Metrics described in thisregistryRegistry might reveal network topology information that may be considered sensitive. If such cases are found, then access control mechanisms should be applied.</t> </section> <sectiontitle="IANA Considerations">anchor="iana-cons" numbered="true" toc="default"> <name>IANA Considerations</name> <t>With the background and processes described in earlier sections,this document requests the followingIANAActions.</t> <t>Editor's Note: Mock-ups of the implementation of this set of requests have been prepared with IANA's help during development of this memo, and have been captured inhas taken theProceedings of IPPM working group sessions. IANA is currently preparing a mock-up. A recent version is available here: http://encrypted.net/IETFMetricsRegistry-106.html</t>actions described below.</t> <sectiontitle="Registry Group">numbered="true" toc="default"> <name>Registry Group</name> <t>The newregistryRegistry groupSHALL be named, "PERFORMANCE METRICS Group".</t>is named Performance Metrics. This document refers to it as the “Performance Metrics Group” or “Registry Group", meaning all registrations appearing on <eref target="https://www.iana.org/assignments/performance-metrics"><https://www.iana.org/assignments/performance-metrics></eref>.</t> <t>For clarity, note that this document and <xref target="RFC8912"/> use the following conventions to refer to the various IANA registries related to Performance Metrics.</t> <table anchor="IANAterms-map"> <thead> <tr> <th></th> <th>RFC 8911 and RFC 8912</th> <th>IANA Web page</th> </tr> </thead> <tbody> <tr> <td>Page Title</td> <td>Performance Metrics Group</td> <td>Performance Metrics</td> </tr> <tr> <td>Main Registry</td> <td>Performance Metrics Registry</td> <td>Performance Metrics Registry</td> </tr> <tr> <td>Registry Row</td> <td>Performance Metrics Registry Entry</td> <td>registration (also template)</td> </tr> </tbody> </table> <t>Registration Procedure: Specification Required</t> <t>Reference:<This RFC></t>RFC 8911</t> <t>Experts: Performance Metrics Experts</t> <!-- <t>Note: TBD</t> --> </section> <sectiontitle="Performance Metricnumbered="true" toc="default"> <name>Performance Metrics NameElements">Elements</name> <t>Thisdocumentmemo specifies and populates theprocedureRegistries for the PerformanceMetricsMetric NameElement Registry setup. IANA is requestedElements. The Name assigned tocreateanew set of registries forPerformance MetricNameRegistry Entry consists of multiple Elementscalled "Registered Performance Metric Name Elements". Each Registry, whose names are listed below:</t> <t><list style="empty"> <t>MetricType:</t> <t>Method:</t> <t>SubTypeMethod:</t> <t>Spec:</t> <t>Units:</t> <t>Output:</t> </list></t> <t>willseparated by an "_" (underscore), in the order defined in <xref target="name-sec7-1-2"/>. IANA has created the following registries, which contain the current set of possibilities for each Element in the PerformanceMetrics Registry Entry Names.</t> <t>To populateMetric Name.</t> <ul empty="true" spacing="normal"> <li>MetricType</li> <li>Method</li> <li>SubTypeMethod</li> <li>Spec</li> <li>Units</li> <li>Output</li> </ul> <t>At creation, IANA has populated the Registered PerformanceMetricMetrics Name Elementsat creation, the IANA is asked to useusing the lists of values for eachname elementName Element listed inSection 7.1.2.<xref target="name-sec7-1-2"/>. The Name Elements in eachregistryRegistry arecase-sensitive.</t>case sensitive.</t> <t>When preparing a MetricentryEntry forRegistration,registration, the developerSHOULD<bcp14>SHOULD</bcp14> choose NameelementsElements from among the registered elements. However, if the proposed metric is unique in a significant way, it may be necessary to propose a new NameelementElement to properly describe the metric, as described below.</t> <t>A candidate Metric EntryRFC or immutable documentproposes a set of values for its Name Elements. These are reviewed by IANA and an ExpertReview would proposeReviewer. It is possible that a candidate Metric Entry proposes a new value for a Name Element (that is, one that is not in the existing list of possibilities), ormoreeven that it proposes a newelement values required to describe the unique entry, and theName Element. Such newname element(s) would be reviewed along with the metric entry. Newassignmentsfor Registered Performance Metric Name Elements will beare administered by IANA through the Specification Required policy(which<xref target="RFC8126" format="default"/>, which includes ExpertReview) <xref target="RFC8126"/>, i.e.,Review (i.e., review by one of a group ofexperts, thePerformanceMetricMetrics Experts, who are appointed by the IESG upon recommendation of the Transport AreaDirectors.</t>Directors).</t> </section> <sectiontitle="Newnumbered="true" toc="default"> <name>New Performance MetricsRegistry">Registry</name> <t>This document specifies theprocedure forPerformance Metrics Registry. The Registrysetup. IANA is requested to create a new registry for Performance Metrics called "Performance Metrics Registry". This Registry will containcontains the following columns in the Summarycolumns:</t> <t><list style="empty"> <t>Identifier:</t> <t>Name:</t> <t>URI:</t> <t>Description:</t> <t>Reference:</t> <t>Change Controller:</t> <t>Version:</t> </list>Descriptionscategory:</t> <ul empty="true" spacing="normal"> <li>Identifier</li> <li>Name</li> <li>URI</li> <li>Description</li> <li>Reference</li> <li>Change Controller</li> <li>Version</li> </ul> <t>Descriptions of these columns and additional information found in the template forregistry entriesRegistry Entries (categories and columns) are further defined insection<xreftarget="columns"/>.</t>target="columns" format="default"/>.</t> <t>The Identifier 0 should be Reserved. The Registered Performance Metric uniqueidentifierIdentifier is an unbounded integer (range 0 to infinity). The Identifier values from 64512 to6553665535 are reserved for private or experimental use, and the user may encounter overlapping uses. When addingnewlynew Registered Performance Metrics to the Performance Metrics Registry, IANASHOULD<bcp14>SHOULD</bcp14> assign the lowest availableidentifierIdentifier to the new Registered Performance Metric. If a Performance Metrics Expert providing review determines that there is a reason to assign a specific numericidentifier,Identifier, possibly leaving a temporary gap in the numbering, then the Performance Metrics ExpertSHALL<bcp14>SHALL</bcp14> inform IANA of this decision.</t> <t>Names starting with the prefixPriv_"Priv_" are reserved for privateuse,use and are not considered for registration. The"Name"Name column entries are further defined insection<xreftarget="columns"/>.</t>target="columns" format="default"/>.</t> <t>The"URI"URI column will have a URL tothe full template ofeachregistry entry.completed Registry Entry. The Registry Entry textSHALL<bcp14>SHALL</bcp14> beHTML-izedHTMLized to aid thereader, with links to reference RFCsreader (similar to the way thatInternet DraftsInternet-Drafts areHTML-ized,HTMLized, the same tool can perform thefunction)function), with links to referenced section(s) of an RFC or another immutable document.</t> <t>The“Reference”Reference column will include an RFC number, an approved specification designator from another standards body, or some other immutable document.</t> <t>New assignments for the Performance Metrics Registry will be administered by IANA through the Specification Requiredpolictypolicy <xref target="RFC8126" format="default"/> (which includes ExpertReview) <xref target="RFC8126"/>,Review, i.e., review by one of a group ofexperts,experts -- in the case of this document, the PerformanceMetricMetrics Experts, who are appointed by the IESG upon recommendation of the Transport AreaDirectors,Directors) or by Standards Action. The experts can be initially drawn from the Working Group Chairs, document editors, and members of the Performance Metrics Directorate, among other sources of experts.</t> <t>Extensionsofto the Performance Metrics Registry require IETF Standards Action. Only one form ofregistryRegistry extension is envisaged:</t><t><list style="numbers"> <t>Adding<ul empty="false" spacing="normal"> <li>Adding columns, or both categories and columns, to accommodate unanticipated aspects of new measurements and metriccategories.</t> </list>Ifcategories.</li> </ul> <t>If the Performance Metrics Registry is extended in this way, theVersionversion number of future entries complying with the extensionSHALL<bcp14>SHALL</bcp14> be incremented(either in(in either the unit or the tenths digit, depending on the degree ofextension.</t>extension).</t> </section> </section> <sectiontitle="Blankanchor="blank-reg-template" numbered="true" toc="default"> <name>Blank RegistryTemplate">Template</name> <t>This section provides a blank template to help IANA andregistry entryRegistry Entry writers.</t> <sectiontitle="Summary">numbered="true" toc="default"> <name>Summary</name> <t>This category includes multiple indexes to theregistry entry:Registry Entry: the element ID andmetric name.</t>Metric Name.</t> <sectiontitle="ID (Identifier)">numbered="true" toc="default"> <name>ID (Identifier)</name> <t><insert a numericidentifier,Identifier, an integer, TBD></t> </section> <sectiontitle="Name">numbered="true" toc="default"> <name>Name</name> <t><insertnamethe Name, according to the metric naming convention></t> </section> <sectiontitle="URI">numbered="true" toc="default"> <name>URI</name> <t>URL:https://www.iana.org/<eref target="https://www.iana.org/performance-metrics/"/> ...<name></t><Name></t> </section> <sectiontitle="Description">numbered="true" toc="default"> <name>Description</name> <t><provide a description></t> </section> <sectiontitle="Change Controller"> <t/>numbered="true" toc="default"> <name>Reference</name> <t><provide the RFC or other specification that contains the approved candidate Registry Entry></t> </section> <section numbered="true" toc="default"> <name>Change Controller</name> <t><provide information regarding the entity responsible for approving revisions to the Registry Entry (including contact information for an individual, where appropriate)></t> </section> <sectiontitle="Versionnumbered="true" toc="default"> <name>Version (of RegistryFormat)"> <t/>Format)</name> </section> </section> <sectiontitle="Metric Definition">numbered="true" toc="default"> <name>Metric Definition</name> <t>This category includes columns to prompt the entry of all necessary details related to the metric definition, including the immutable document reference and values of input factors, calledfixed parameters.</t>"Fixed Parameters".</t> <sectiontitle="Reference Definition"> <t><Fullnumbered="true" toc="default"> <name>Reference Definition</name> <t><provide a full bibliographic reference to an immutabledoc.></t> <t><specificdocument></t> <t><provide a specific section reference and additional clarifications, if needed></t><t/></section> <sectiontitle="Fixed Parameters">numbered="true" toc="default"> <name>Fixed Parameters</name> <t><list and specify Fixed Parameters, input factors that must be determined and embedded in the measurement system for use when needed></t> </section> </section> <sectiontitle="Methodnumbered="true" toc="default"> <name>Method ofMeasurement">Measurement</name> <t>This category includes columns for references to relevant sections of the immutabledocuments(s)document&wj;(s) and any supplemental information needed to ensure an unambiguousmethodsmethod for implementations.</t> <sectiontitle="Reference Method">numbered="true" toc="default"> <name>Reference Method</name> <t><for the metric, insert relevant section references and supplemental info></t> </section> <sectiontitle="Packetnumbered="true" toc="default"> <name>Packet StreamGeneration"> <t><listGeneration</name> <t><provide a list of generationparametersParameters and section/spec references if needed></t> </section> <sectiontitle="Trafficnumbered="true" toc="default"> <name>Traffic Filtering(observation) Details"> <t>The measured results based on a filtered version of the packets observed, and this section(Observation) Details</name> <t>This category provides the filter details (whenpresent).</t> <t><section reference>.</t> <t/>present), which qualify the set of packets that contribute to the measured results from among all packets observed.</t> <t><provide a section reference></t> </section> <sectiontitle="Sampling Distribution">numbered="true" toc="default"> <name>Sampling Distribution</name> <t><insert time distribution details, or how this isdiffdifferent from the filter></t><t/></section> <sectiontitle="Run-timenumbered="true" toc="default"> <name>Runtime Parameters and DataFormat"> <t>Run-timeFormat</name> <t>Runtime Parameters are input factors that must be determined, configured into the measurement system, and reported with the results for the context to be complete.</t><t><list<t><provide a list ofrun-time parameters,Runtime Parameters and their data formats></t><t/></section> <sectiontitle="Roles"> <t><listsnumbered="true" toc="default"> <name>Roles</name> <t><list the names of the differentrolesRoles from themeasurement method></t> <t/>Measurement Method></t> </section> </section> <sectiontitle="Output">numbered="true" toc="default"> <name>Output</name> <t>This category specifies all details of theOutputoutput of measurements using the metric.</t> <sectiontitle="Type">numbered="true" toc="default"> <name>Type</name> <t><insert the name of the outputtype,type -- raw results or a selected summary statistic></t> </section> <sectiontitle="Reference Definition">numbered="true" toc="default"> <name>Reference Definition</name> <t><describe the reference data format for each type of result></t> </section> <sectiontitle="Metric Units">numbered="true" toc="default"> <name>Metric Units</name> <t><insert units for the measured results, and provide the referencespecification>.</t>specification></t> </section> <sectiontitle="Calibration">numbered="true" toc="default"> <name>Calibration</name> <t><insert information on calibration></t> </section> </section> <sectiontitle="Administrative items"> <t/>numbered="true" toc="default"> <name>Administrative Items</name> <t>This category provides administrative information.</t> <sectiontitle="Status"> <t><currentnumbered="true" toc="default"> <name>Status</name> <t><provide status: 'Current' ordeprecated></t>'Deprecated'></t> </section> <sectiontitle="Requester"> <t><name or RFC,numbered="true" toc="default"> <name>Requester</name> <t><provide a person's name, an RFC number, etc.></t> </section> <sectiontitle="Revision"> <t><1.0></t>numbered="true" toc="default"> <name>Revision</name> <t><provide the revision number: starts at 0></t> </section> <sectiontitle="Revision Date"> <t><format YYYY-MM-DD></t>numbered="true" toc="default"> <name>Revision Date</name> <t><provide the date, in YYYY-MM-DD format></t> </section> </section> <sectiontitle="Commentsnumbered="true" toc="default"> <name>Comments andRemarks"> <t><Additional (Informational)Remarks</name> <t><list any additional (informational) details for this entry></t> </section> </section> </middle> <back> <references> <name>References</name> <references> <name>Normative References</name> <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.2026.xml"/> <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.2119.xml"/> <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.2330.xml"/> <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.3986.xml"/> <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8126.xml"/> <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.6390.xml"/> <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.6576.xml"/> <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7799.xml"/> <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8174.xml"/> <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.5644.xml"/> </references> <references> <name>Informative References</name> <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7679.xml"/> <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.2681.xml"/> <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.3432.xml"/> <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.3550.xml"/> <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.3611.xml"/> <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.4148.xml"/> <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.5474.xml"/> <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.5475.xml"/> <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.5477.xml"/> <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.6035.xml"/> <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.6248.xml"/> <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7012.xml"/> <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7014.xml"/> <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7594.xml"/> <!-- draft-ietf-ippm-initial-registry (RFC 8912) --> <reference anchor="RFC8912" target="https://www.rfc-editor.org/info/rfc8912"> <front> <title>Initial Performance Metrics Registry Entries</title> <author initials="A" surname="Morton" fullname="Alfred Morton"> <organization/> </author> <author initials="M" surname="Bagnulo" fullname="Marcelo Bagnulo"> <organization/> </author> <author initials="P" surname="Eardley" fullname="Philip Eardley"> <organization/> </author> <author initials="K" surname="D'Souza" fullname="Kevin D'Souza"> <organization/> </author> <date month="November" year="2021"/> </front> <seriesInfo name="RFC" value="8912"/> <seriesInfo name="DOI" value="10.17487/RFC8912"/> </reference> <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.6991.xml"/> </references> </references> <sectiontitle="Acknowledgments">numbered="false" toc="default"> <name>Acknowledgments</name> <t>Thanks toBrian Trammell and Bill Cerveny,<contact fullname="Brian Trammell"/> and <contact fullname="Bill Cerveny"/>, IPPMchairs,co-chairs during the development of this memo, for leadingsomeseveral brainstorming sessions on this topic. Thanks toBarbara Stark and Juergen Schoenwaelder<contact fullname="Barbara Stark"/> and <contact fullname="Juergen Schoenwaelder"/> for the detailed feedback and suggestions. Thanks toAndrew McGregor<contact fullname="Andrew McGregor"/> for suggestions on metric naming. Thanks toMichelle Cotton<contact fullname="Michelle Cotton"/> for her early IANA review, and toAmanda Barber<contact fullname="Amanda Baber"/> for answering questions related to the presentation of theregistryRegistry and accessibility of the complete template via URL. Thanks toRoni Even<contact fullname="Roni Even"/> for his review and suggestions to generalize the procedures. Thanks to~allall of the Area Directors for their reviews.</t> </section><!-- <section title="Appendix: Examples"> <section title="Example IPPM Active Registry Entry"> <t>This section is Informational.</t> <t>This section gives an example registry entry for the active metric described in <xref target="RFC3393"/>, on Packet Delay Variation.</t> <section title="Registry Indexes"> <t>This category includes multiple indexes to the Registered Performance Metrics, the element ID and metric name.</t> <section title="Identifier"> <t>An integer having enough digits to uniquely identify each entry in the Registry.</t> </section> <section title="Name"> <t>A metric naming convention is TBD.</t> <t>One possibility based on IPPM's framework is:</t> <t>Act_IP_UDP_One-way-pdv_95th-percentile_Poisson</t> </section> <section title="URI"> <t>Prefix urn:ietf:params:performance:metric</t> </section> <section title="Status"> <t>current</t> </section> <section title="Requestor"> <t>Alcelip Mornuley</t> </section> <section title="Revision"> <t>1.0</t> </section> <section title="Revision Date"> <t>2014-07-04</t> </section> <section title="Description"> <t>An assessment of packet delay variation with respect to the minimum delay observed on the stream.</t> </section> <section title="Reference Specification(s)"> <t><xref target="RFC2330"/><xref target="RFC3393"/><xref target="RFC5481"/><xref target="RFC5905"/></t> </section> </section> <section title="Metric Definition"> <t>This category includes columns to prompt the entry of all necessary details related to the metric definition, including the RFC reference and values of input factors, called fixed parameters.</t> <section title="Reference Definition"> <t>See sections 2.4 and 3.4 of <xref target="RFC3393"/>. Singleton delay differences measured are referred to by the variable name "ddT".</t> </section> <section title="Fixed Parameters"> <t>Since the metric's reference supplies a list of Parameters as part of its descriptive template, a sub-set of the Parameters have been designated as designated as Fixed Parameters for this entry.</t> <t><list style="symbols"> <t>F, a selection function defining unambiguously the packets from the stream selected for the metric. See section 4.2 of <xref target="RFC5481"/> for the PDV form.</t> <t>L, a packet length in bits. L = 200 bits.</t> <t>Tmax, a maximum waiting time for packets to arrive at Dst, set sufficiently long to disambiguate packets with long delays from packets that are discarded (lost). Tmax = 3 seconds.</t> <t>Type-P, as defined in <xref target="RFC2330"/>, which includes any field that may affect a packet's treatment as it traverses the network. The packets are IP/UDP, with DSCP = 0 (BE).</t> </list></t> </section> </section> <section title="Method of Measurement"> <t>This category includes columns for references to relevant sections of the RFC(s) and any supplemental information needed to ensure an unambiguous methods for implementations.</t> <section title="Reference Method"> <t>See section 2.6 and 3.6 of <xref target="RFC3393"/> for singleton elements.</t> </section> <section title="Stream Type and Stream Parameters"> <t>Poisson distributed as described in <xref target="RFC2330"/>, with the following Parameters.</t> <t><list style="symbols"> <t>lambda, a rate in reciprocal seconds (for Poisson Streams). lambda = 1 packet per second</t> <t>Upper limit on Poisson distribution (values above this limit will be clipped and set to the limit value). Upper limit = 30 seconds.</t> </list></t> </section> <section title="Traffic Filter"> <t>NA</t> </section> <section title="Measurement Timing"> <t>NA</t> </section> <section title="Output Type and Data Format"> <t>See section 4.3 of <xref target="RFC3393"/> for details on the percentile statistic.</t> <t>The percentile = 95.</t> <t>Data format is a 32-bit unsigned floating point value.</t> <t>Individual results (singletons) should be represented by the following triple</t> <t><list style="symbols"> <t>T1 and T2, times as described below in the Run-time parameters section.</t> <t>ddT as defined in section 2.4 of <xref target="RFC3393"/></t> </list>if needed. The result format for ddT is *similar to* the short format in <xref target="RFC5905"/> (32 bits) and is as follows: the first 16 bits represent the *signed* integer number of seconds; the next 16 bits represent the fractional part of a second.</t> </section> <section title="Metric Units"> <t>See section 3.3 of <xref target="RFC3393"/> for singleton elements.</t> <t><xref target="RFC2330"/> recommends that when a time is given, it will be expressed in UTC.</t> <t>The timestamp format (for T, Tf, etc.) is the same as in <xref target="RFC5905"/> (64 bits) and is as follows: the first 32 bits represent the unsigned integer number of seconds elapsed since 0h on 1 January 1900; the next 32 bits represent the fractional part of a second that has elapsed since then.</t> </section> <section title="Run-time Parameters and Data Format"> <t>Since the metric's reference supplies a list of Parameters as part of its descriptive template, a sub-set of the Parameters have been designated as Run-Time Parameters for this entry. In related Registered Performance Metrics, some of the parameters below may be designated as Fixed Parameters instead.</t> <t><list style="symbols"> <t>Src, the IP address of a host (32-bit value for IPv4, 128-bit value for IPv6)</t> <t>Dst, the IP address of a host (32-bit value for IPv4, 128-bit value for IPv6)</t> <t>T, a time (start of test interval, 128-bit NTP Date Format, see section 6 of <xref target="RFC5905"/>)</t> <t>Tf, a time (end of test interval, 128-bit NTP Date Format, see section 6 of <xref target="RFC5905"/>)</t> <t>T1, the wire time of the first packet in a pair, measured at MP(Src) as it leaves for Dst (64-bit NTP Timestamp Format, see section 6 of <xref target="RFC5905"/>).</t> <t>T2, the wire time of the second packet in a pair, measured at MP(Src) as it leaves for Dst (64-bit NTP Timestamp Format, see section 6 of <xref target="RFC5905"/>).</t> <t>I(i),I(i+1), i >=0, pairs of times which mark the beginning and ending of the intervals in which the packet stream from which the measurement is taken occurs. Here, I(0) = T0 and assuming that n is the largest index, I(n) = Tf (pairs of 64-bit NTP Timestamp Format, see section 6 of <xref target="RFC5905"/>).</t> </list></t> </section> </section> <section title="Comments and Remarks"> <t>Lost packets represent a challenge for delay variation metrics. See section 4.1 of <xref target="RFC3393"/> and the delay variation applicability statement<xref target="RFC5481"/> for extensive analysis and comparison of PDV and an alternate metric, IPDV.</t> </section> </section> <section title="Example RTCP-XR Registry Entry"> <t>This section is Informational.</t> <t>This section gives an example registry entry for the end-point metric described in <xref target="RFC7003"/>, for RTCP-XR Burst/Gap Discard Metric reporting.</t> <section title="Registry Indexes"> <t>This category includes multiple indexes to the Registered Performance Metrics, the element ID and metric name.</t> <section title="Identifier"> <t>An integer having enough digits to uniquely identify each entry in the Registry.</t> </section> <section title="Name"> <t>A metric naming convention is TBD.</t> </section> <section title="URI"> <t>Prefix urn:ietf:params:performance:metric</t> </section> <section title="Status"> <t>current</t> </section> <section title="Requestor"> <t>Alcelip Mornuley</t> </section> <section title="Revision"> <t>1.0</t> </section> <section title="Revision Date"> <t>2014-07-04</t> </section> <section title="Description"> <t>TBD.</t> </section> <section title="Reference Specification(s)"> <t><xref target="RFC3611"/><xref target="RFC4566"/> <xref target="RFC6776"/><xref target="RFC6792"/> <xref target="RFC7003"/></t> </section> </section> <section title="Metric Definition"> <t>This category includes columns to prompt the entry of all necessary details related to the metric definition, including the RFC reference and values of input factors, called fixed parameters. Section 3.2 of <xref target="RFC7003"/> provides the reference information for this category.</t> <section title="Reference Definition"> <t>Packets Discarded in Bursts:</t> <t>The total number of packets discarded during discard bursts. The measured value is unsigned value. If the measured value exceeds 0xFFFFFD, the value 0xFFFFFE MUST be reported to indicate an over-range measurement. If the measurement is unavailable, the value 0xFFFFFF MUST be reported.</t> </section> <section title="Fixed Parameters"> <t>Fixed Parameters are input factors that must be determined and embedded in the measurement system for use when needed. The values of these parameters is specified in the Registry.</t> <t>Threshold: 8 bits, set to value = 3 packets.</t> <t>The Threshold is equivalent to Gmin in [RFC3611], i.e., the number of successive packets that must not be discarded prior to and following a discard packet in order for this discarded packet to be regarded as part of a gap. Note that the Threshold is set in accordance with the Gmin calculation defined in Section 4.7.2 of [RFC3611].</t> <t>Interval Metric flag: 2 bits, set to value 11=Cumulative Duration</t> <t>This field is used to indicate whether the burst/gap discard metrics are Sampled, Interval, or Cumulative metrics <xref target="RFC6792"/>:</t> <t>I=10: Interval Duration - the reported value applies to the most recent measurement interval duration between successive metrics reports.</t> <t>I=11: Cumulative Duration - the reported value applies to the accumulation period characteristic of cumulative measurements.</t> <t>Senders MUST NOT use the values I=00 or I=01.</t> </section> </section> <section title="Method of Measurement"> <t>This category includes columns for references to relevant sections of the RFC(s) and any supplemental information needed to ensure an unambiguous methods for implementations. For the Burst/Gap Discard Metric, it appears that the only guidance on methods of measurement is in Section 3.0 of <xref target="RFC7003"/> and its supporting references. Relevant information is repeated below, although there appears to be no section titled "Method of Measurement" in <xref target="RFC7003"/>.</t> <section title="Reference Method"> <t>Metrics in this block report on burst/gap discard in the stream arriving at the RTP system. Measurements of these metrics are made at the receiving end of the RTP stream. Instances of this metrics block use the synchronization source (SSRC) to refer to the separate auxiliary Measurement Information Block <xref target="RFC6776"/>, which describes measurement periods in use (see <xref target="RFC6776"/>, Section 4.2).</t> <t>This metrics block relies on the measurement period in the Measurement Information Block indicating the span of the report. Senders MUST send this block in the same compound RTCP packet as the Measurement Information Block. Receivers MUST verify that the measurement period is received in the same compound RTCP packet as this metrics block. If not, this metrics block MUST be discarded.</t> </section> <section title="Stream Type and Stream Parameters"> <t>Since RTCP-XR Measurements are conducted on live RTP traffic, the complete description of the stream is contained in SDP messages that proceed the establishment of a compatible stream between two or more communicating hosts. See Run-time Parameters, below.</t> </section> <section title="Traffic Filter"> <t>NA</t> </section> <section title="Measurement Timing"> <t>NA</t> </section> <section title="Output Type and Data Format"> <t>The output type defines the type of result that the metric produces.</t> <t><list style="symbols"> <t>Value: Packets Discarded in Bursts</t> <t>Data Format: 24 bits</t> <t>Reference: Section 3.2 of <xref target="RFC7003"/></t> </list></t> </section> <section title="Metric Units"> <t>The measured results are apparently expressed in packets, although there is no section of <xref target="RFC7003"/> titled "Metric Units".</t> </section> <section title="Run-time Parameters and Data Format"> <t>Run-Time Parameters are input factors that must be determined, configured into the measurement system, and reported with the results for the context to be complete. However, the values of these parameters is not specified in the Registry, rather these parameters are listed as an aid to the measurement system implementor or user (they must be left as variables, and supplied on execution).</t> <t>The Data Format of each Run-time Parameter SHALL be specified in this column, to simplify the control and implementation of measurement devices.</t> <t>SSRC of Source: 32 bits As defined in Section 4.1 of [RFC3611].</t> <t>SDP Parameters: As defined in <xref target="RFC4566"/></t> <t>Session description v= (protocol version number, currently only 0)</t> <t>o= (originator and session identifier : username, id, version number, network address)</t> <t>s= (session name : mandatory with at least one UTF-8-encoded character)</t> <t>i=* (session title or short information) u=* (URI of description)</t> <t>e=* (zero or more email address with optional name of contacts)</t> <t>p=* (zero or more phone number with optional name of contacts)</t> <t>c=* (connection information—not required if included in all media)</t> <t>b=* (zero or more bandwidth information lines) One or more Time descriptions ("t=" and "r=" lines; see below)</t> <t>z=* (time zone adjustments)</t> <t>k=* (encryption key)</t> <t>a=* (zero or more session attribute lines)</t> <t>Zero or more Media descriptions (each one starting by an "m=" line; see below)</t> <t>m= (media name and transport address)</t> <t>i=* (media title or information field)</t> <t>c=* (connection information — optional if included at session level)</t> <t>b=* (zero or more bandwidth information lines)</t> <t>k=* (encryption key)</t> <t>a=* (zero or more media attribute lines — overriding the Session attribute lines)</t> <t>An example Run-time SDP description follows:</t> <t>v=0</t> <t>o=jdoe 2890844526 2890842807 IN IP4 192.0.2.5</t> <t>s=SDP Seminar i=A Seminar on the session description protocol</t> <t>u=http://www.example.com/seminars/sdp.pdf e=j.doe@example.com (Jane Doe)</t> <t>c=IN IP4 233.252.0.12/127</t> <t>t=2873397496 2873404696</t> <t>a=recvonly</t> <t>m=audio 49170 RTP/AVP 0</t> <t>m=video 51372 RTP/AVP 99</t> <t>a=rtpmap:99 h263-1998/90000</t> </section> </section> <section title="Comments and Remarks"> <t>TBD.</t> </section> </section> <section title="Example Generalized Passive Octet Count Entry"> <t>This section gives an example registry entry for a generalized the passive metric octetDeltaCount described in the IPFIX registry"/>.</t> <section title="Registry Indexes"> <t>This category includes multiple indexes to the Registered Performance Metrics, the element ID and metric name.</t> <section title="Element Identifier"> <t>An integer having enough digits to uniquely identify each entry in the Registry.</t> <t>TBD by IANA.</t> </section> <section title="Metric Name"> <t>A metric naming convention is TBD.</t> <t>Pas_IP_Octet-Delta-General</t> </section> <section title="URI"> <t>urn:ietf:params:performance:metric-something</t> </section> <section title="Status"> <t>Current</t> </section> <section title="Requester"> <t>TBD</t> </section> <section title="Revision"> <t>0</t> </section> <section title="Revision Date"> <t>TBD</t> </section> <section title="Metric Description"> <t>A delta count of the number of octets observed. </t> </section> <section title="Reference Specification(s)"> <t>octetDeltaCount described in the IPFIX registry.</t> </section> </section> <section title="Metric Definition"> <t>This category includes columns to prompt the entry of all necessary details related to the metric definition, including the RFC reference and values of input factors, called fixed parameters.</t> <section title="Reference Definition"> <t>octetDeltaCount described in the IPFIX registry.</t> </section> <section title="Fixed Parameters"> <t> As this is the generalised version of the IP delta count metric, there are no fixed parameters.</t> </section> </section> <section title="Method of Measurement"> <section title="Reference Implementation"> <t>For <metric>.</t> <t><section reference></t> <t/> </section> <section title="Stream Type and Stream Parameters"> <t>NA</t> </section> <section title="Traffic Filter Criteria"> <t> This measurement only covers IP packets and the IP payload (including the IP header) of these packets. Non-IP packets (BPDUs, ISIS) will not be accounted. Layer 2 overhead (Ethernet headers, MPLS, QinQ, etc.) will also not be represented in the measurement. </t> </section> <section title="Measurement Timing"> <t> This is a continous measurement of the IP octets seen in the traffic selection scope (run-time parameter). </t> <t>The measurement interval is a run time parameter. </t> <t>There is no sampling.</t> </section> <section title="Output Type(s) and Data Format"> <t>It is possible that multiple observation intervals are reported in a single report. In such a case concatination of the interval reports (deltaOctetCount, start-time, end-time) is allowed. </t> <t>The delta octet count metric reports a observation start time and end time. </t> <t><list style="symbols"> <t>Value: observation-start-time and observation-end-time</t> <t>Data Format: 64-bit NTP Time-stamp Format</t> <t>Reference: section 6 of <xref target="RFC5905"/></t> </list></t> </section> <section title="Metric Units"> <t>The measured results are expressed in octets with a data format of unsigned64 as described in the IPFIX registry.</t> </section> <section title="Run-time Parameters and Data Format"> <t>Run-time Parameters are input factors that must be determined, configured into the measurement system, and reported with the results for the context to be complete.</t> <t><list style="symbols"> <t>samplingTimeInterval, length of time a single report covers. unsigned32 microseconds <xref target="RFC5477"/> </t> <t>observationInterface, ifindex of interface to monitor. -1 represents all interfaces. -2 representing WAN facing and -3 represents LAN facing. unsigned32.</t> <t>observation direction, unsigned8 where 0 represents incoming traffic on interface, 1 outgoing and 2 represents both incoming and outgoing. </t> </list></t> </section> </section> <section title="Comments and Remarks"> <t>Additional (Informational) details for this entry</t> </section> </section> <section title="Example 5min Passive Egress IP Destination Octet Count Entry on WAN Interface"> <t>tbd</t> <t>This section is Informational.</t> <t>This section gives an example registry entry for the passive accounting of byte counts and destination address on outgoing WAN IP. The byte count and IP address is based on octetDeltaCount and destinationIPv4Address, as described in the IPFIX registry.</t> <section title="Registry Indexes"> <t>This category includes multiple indexes to the Registered Performance Metrics, the element ID and metric name.</t> <section title="Element Identifier"> <t>An integer having enough digits to uniquely identify each entry in the Registry.</t> <t>TBD by IANA.</t> </section> <section title="Metric Name"> <t>A metric naming convention is TBD.</t> <t>Pas_IPDst-Octet-Delta-WAN-egress</t> </section> <section title="URI"> <t>urn:ietf:params:performance:Pas_IPDst-Octet-Delta-WAN-egress</t> </section> <section title="Status"> <t>Current</t> </section> <section title="Requester"> <t>The IPPM working group</t> </section> <section title="Revision"> <t>0</t> </section> <section title="Revision Date"> <t>Today</t> </section> <section title="Metric Description"> <t>This example passive measurement registry entry measures per-destination IP bytes sent. The byte count and IP address are based on octetDeltaCount and destinationIPv4Address, as described in IPFIX Registry. This metric can be used to understand outgoing top destinations per agent, saturation of link utilization towards a single destination and other bandwidth utilization uses.</t> </section> <section title="Reference Specification(s)"> <t>octetDeltaCount described in IPFIX registry</t> </section> <section title="Fixed Parameters"> <t>Measurement Interval = 300 sec</t> <t>IPFIX Template = KEY:destinationIPv4Address,egressInterface=WAN Value:octetDeltaCount </t> </section> <section title="Traffic Filter"> <t>PSAMP: "ipVersion == 4 AND egressInterface==WAN"</t> </section> <section title="Sampling Distribution"> <t>No sampling</t> </section> <section title="Run-time Parameters"> <t>None</t> </section> </section> <section title="Example 5min Passive Egress Octet Count Entry on WAN Interface"> <t>tbd</t> <t>This section is Informational.</t> <t>This section gives an example registry entry for accounting of outgoing WAN IP traffic the passive metric in terms of octetDeltaCount, as described in the IPFIX registry.</t> <section title="Registry Indexes"> <t>This category includes multiple indexes to the Registered Performance Metrics, the element ID and metric name.</t> <section title="Element Identifier"> <t>An integer having enough digits to uniquely identify each entry in the Registry.</t> <t>TBD by IANA.</t> </section> <section title="Metric Name"> <t>A metric naming convention is TBD.</t> <t>Pas_IP-Octet-Delta-WAN-egress</t> </section> <section title="URI"> <t>urn:ietf:params:performance:metric-something</t> </section> <section title="Status"> <t>Current</t> </section> <section title="Requester"> <t>TBD</t> </section> <section title="Revision"> <t>0</t> </section> <section title="Revision Date"> <t>TBD</t> </section> <section title="Metric Description"> <t>A delta count of the number of octets observed outgoing on WAN interface. </t> </section> <section title="Reference Specification(s)"> <t>octetDeltaCount described in the IPFIX registry</t> </section> </section> <section title="Metric Definition"> <t>This category includes columns to prompt the entry of all necessary details related to the metric definition, including the RFC reference and values of input factors, called fixed parameters.</t> <section title="Reference Definition"> <t>octetDeltaCount described in the IPFIX registry"/></t> </section> <section title="Fixed Parameters"> <t> As this is a specific version of Pas_IP-Octet-Delta-General that performs metering of all outgoing WAN traffic.</t> <t><list style="symbols"> <t>samplingTimeInterval= 300000000, length of time a single report covers. unsigned32 microseconds <xref target="RFC5477"/> </t> <t>observationInterface= -2, ifindex of interface to monitor. -1 represents all interfaces. -2 representings WAN facing and -3 represnets LAN facing. unsigned32.</t> <t>observation direction= 1, unsigned8 where 0 represents incoming traffic on interface, 1 outgoing and 2 represents both incoming and outgoing. </t> </list></t> </section> </section> <section title="Method of Measurement"> <section title="Reference Implementation"> <t>For <metric>.</t> <t><section reference></t> <t/> </section> <section title="Stream Type and Stream Parameters"> <t>NA</t> </section> <section title="Traffic Filter Criteria"> <t> This measurement only covers IP packets observed in the WAN outgoing direction. The bytes counted are the IP payload (including the IP header) of these packets. Non-IP packets (BPDUs, ISIS) will not be accounted. Layer 2 overhead (Ethernet headers, MPLS, QinQ, etc.) will also not be represented in the measurement. </t> </section> <section title="Measurement Timing"> <t> This is a continous measurement of the IP octets seen in the traffic selection scope (run-time parameter), each of a 5 minute duration. </t> <t>There is no sampling.</t> </section> <section title="Output Type(s) and Data Format"> <t>It is possible that multiple observation intervals are reported in a single report. In such a case concatination of the interval reports (deltaOctetCount, start-time, end-time) is allowed. </t> <t>The delta octet count metric reports a observation start time and end time. </t> <t><list style="symbols"> <t>Value: observation-start-time and observation-end-time</t> <t>Data Format: 64-bit NTP Time-stamp Format</t> <t>Reference: section 6 of <xref target="RFC5905"/></t> </list></t> </section> <section title="Metric Units"> <t>The measured results are expressed in octets with a data format of unsigned64 as described in the IPFIX registry</t> </section> <section title="Run-time Parameters and Data Format"> <t>There are no run-time parameters for this registry entry.</t> </section> </section> <section title="Comments and Remarks"> <t>Additional (Informational) details for this entry</t> </section> </section> <section title="Example Passive RTP Lost Packet Count"> <t>tbd</t> </section> <section title="Example BLANK Registry Entry"> <t>This section is Informational. (?)</t> <t>This section gives an example registry entry for the <type of metric and specification reference> .</t> <section title="Registry Indexes"> <t>This category includes multiple indexes to the Registered Performance Metrics, the element ID and metric name.</t> <section title="Identifier"> <t>An integer.</t> </section> <section title="Name"> <t>A metric naming convention is TBD.</t> </section> <section title="URI"> <t>Prefix urn:ietf:params:performance:metric</t> </section> <section title="Status"> <t>current</t> </section> <section title="Requestor"> <t>name or RFC, etc.</t> </section> <section title="Revision"> <t>1.0</t> </section> <section title="Revision Date"> <t>YYYY-MM-DD</t> </section> <section title="Description"> <t>TBD.</t> </section> <section title="Reference Specification(s)"> <t>RFC...</t> </section> </section> <section title="Metric Definition"> <t>This category includes columns to prompt the entry of all necessary details related to the metric definition, including the RFC reference and values of input factors, called fixed parameters.</t> <t><possible section reference>.</t> <section title="Reference Definition"> <t/> <t/> </section> <section title="Fixed Parameters"> <t>Fixed Parameters are input factors that must be determined and embedded in the measurement system for use when needed. The values of these parameters is specified in the Registry.</t> <t><list fixed parameters></t> </section> </section> <section title="Method of Measurement"> <t>This category includes columns for references to relevant sections of the RFC(s) and any supplemental information needed to ensure an unambiguous methods for implementations.</t> <section title="Reference Method"> <t>For <metric>.</t> <t><section reference></t> <t/> </section> <section title="Stream Type and Stream Parameters"> <t><list of stream parameters>.</t> <t><references></t> <t/> </section> <section title="Traffic Filter Criteria"> <t> <list filter criteria limitations and allowances > </t> </section> <section title="Measurement Timing"> <t> < list timing requirements and limitations > </t> </section> <section title="Output Type and Data Format"> <t>The output type defines the type of result that the metric produces.</t> <t><list style="symbols"> <t>Value:</t> <t>Data Format: (There may be some precedent to follow here, but otherwise use 64-bit NTP Timestamp Format, see section 6 of <xref target="RFC5905"/>).</t> <t>Reference: <section reference></t> </list></t> </section> <section title="Metric Units"> <t>The measured results are expressed in <units>,</t> <t><section reference>.</t> </section> <section title="Run-time Parameters and Data Format"> <t>Run-time Parameters are input factors that must be determined, configured into the measurement system, and reported with the results for the context to be complete.</t> <t><list of run-time parameters></t> <t><reference(s)>.</t> </section> </section> <section title="Comments and Remarks"> <t>Additional (Informational) details for this entry</t> </section> </section> </section> --> </middle> <back> <references title="Normative References"> <?rfc include="reference.RFC.2026"?> <?rfc include="reference.RFC.2119"?> <?rfc ?> <?rfc include='reference.RFC.2330'?> <?rfc include='reference.RFC.3986'?> <?rfc ?> <?rfc include="reference.RFC.8126"?> <?rfc ?> <?rfc include="reference.RFC.6390"?> <?rfc include='reference.RFC.6576'?> <?rfc include='reference.RFC.7799'?> <?rfc include='reference.RFC.8174'?> </references> <references title="Informative References"> <?rfc ?> <?rfc include='reference.RFC.7679'?> <?rfc include="reference.RFC.2681"?> <?rfc ?> <?rfc include="reference.RFC.3432"?> <?rfc include="reference.RFC.3550"?> <?rfc include="reference.RFC.3611"?> <?rfc include="reference.RFC.4148"?> <?rfc include="reference.RFC.5474"?> <?rfc include="reference.RFC.5475"?> <?rfc include="reference.RFC.5477"?> <?rfc ?> <?rfc ?> <?rfc include="reference.RFC.6035"?> <?rfc include="reference.RFC.6248"?> <?rfc ?> <?rfc ?> <?rfc include="reference.RFC.7012"?> <?rfc include="reference.RFC.7014"?> <?rfc include='reference.RFC.7594'?> <?rfc include='reference.RFC.8194'?> <?rfc ?> <?rfc include='reference.I-D.ietf-ippm-initial-registry'?> <?rfc include='reference.RFC.6991'?> </references></back> </rfc>