AVTCoreInternet Engineering Task Force (IETF) R. van BrandenburgInternet-DraftRequest for Comments: 7272 H. StokkingIntended status:Category: Standards Track O. van DeventerExpires: March 02, 2014ISSN: 2070-1721 TNO F. Boronat M. MontagudUniversitat Politecnica de ValenciaUPV K. Gross AVA NetworksAugust 29, 2013 Inter-destinationJune 2014 Inter-Destination Media Synchronizationusing(IDMS) Using the RTP Control Protocol (RTCP)draft-ietf-avtcore-idms-13Abstract This document defines a new RTP Control Protocol (RTCP) Packet Type and an RTCP Extended Report (XR) Block Type to be used for achieving Inter-Destination Media Synchronization (IDMS). IDMS is the process of synchronizing playout across multiplegeographically distributedmedia receivers. Using the RTCP XR IDMS Report Block defined in this document, media playout information from participants in a synchronization group can be collected. Based on the collected information, an RTCP IDMS Settings Packet can then be sent to distribute a common target playout point to which all the distributed receivers, sharing a media experience, can synchronize. Typical use cases in which IDMS is useful are social TV, shared service control(i.e.(i.e., applications where two or more geographically separated users are watching a media stream together), distance learning, networked video walls, networked loudspeakers, etc. Status of This Memo ThisInternet-Draftissubmitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documentsan Internet Standards Track document. This document is a product of the Internet Engineering Task Force (IETF).Note that other groups may also distribute working documents as Internet-Drafts. The listIt represents the consensus ofcurrent Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents validthe IETF community. It has received public review and has been approved fora maximumpublication by the Internet Engineering Steering Group (IESG). Further information on Internet Standards is available in Section 2 ofsix monthsRFC 5741. Information about the current status of this document, any errata, and how to provide feedback on it may beupdated, replaced, or obsoleted by other documentsobtained atany time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on March 02, 2014.http://www.rfc-editor.org/info/rfc7272. Copyright Notice Copyright (c)20132014 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 32. Rationale . . .1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 32.1. Applicability of RTCP to IDMS2. Rationale . . . . . . . . . . . . . .3 2.2. IDMS and ETSI. . . . . . . . . . . . 3 2.1. Applicability of RTCP to IDMS . . . . . . . . . . .4 3. Terminology. . . 3 2.2. IDMS and ETSI . . . . . . . . . . . . . . . . . . . . . . 44.3. Inter-Destination Media Synchronization (IDMS)use casesUse Cases . . 45.4. Overview of IDMSoperationOperation . . . . . . . . . . . . . . . . . 56.5. Architecture for Inter-Destination Media Synchronization . . 76.1.5.1. Media Synchronization Application Server (MSAS) . . . . . 76.2.5.2. Synchronization Client (SC) . . . . . . . . . . . . . . . 86.3.5.3. Communication between MSAS and SCs . . . . . . . . . . . 87.6. RTCP XRBlock forIDMS(IDMSReportBlock)Block . . . . . . . . . . . . . . . . . . 88.7. RTCP Packet Type for IDMS (IDMSSettings) . . . .Settings Packet) . . . . . . 119.8. Timing and NTP Considerations . . . . . . . . . . . . . . . . 1310.9. On theuseUse ofpresentation timestampsPresentation Timestamps . . . . . . . . . . . . 1411.10. SDPSignallingSignaling for RTCP IDMS Settings PacketType .. . . . . . . . . 1512.11. SDPrulesRules . . . . . . . . . . . . . . . . . . . . . . . . . .15 12.1.16 11.1. Offer/AnswerrulesRules . . . . . . . . . . . . . . . . . . . 1612.2.11.2. DeclarativecasesCases . . . . . . . . . . . . . . . . . . . 1713.12. Security Considerations . . . . . . . . . . . . . . . . . . . 1714.13. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 1814.1.13.1. RTCP IDMS Packet Type . . . . . . . . . . . . . . . . . 1814.2.13.2. RTCP XR IDMS Report Block . . . . . . . . . . . . . . . 1914.3.13.3. RTCP-IDMS SDP Attribute . . . . . . . . . . . . . . . . 1914.4.13.4. IDMS XR Block SPST Registry . . . . . . . . . . . . . . 1914.5.13.5. Contact Information for Registrations . . . . . . . . . 2015.14. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 2016.15. References . . . . . . . . . . . . . . . . . . . . . . . . . 2016.1.15.1. Normative References . . . . . . . . . . . . . . . . . . 2016.2.15.2. Informative References . . . . . . . . . . . . . . . . . 21Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 221. IntroductionInter-Destination Media Synchronization (IDMS)IDMS refers to the playout of media streams at two or more geographically distributed locations in atime synchronizedtime-synchronized manner. It can be applied to both unicast and multicast media streams and can be applied to any type and/or combination of streaming media, such as audio,videovideo, and text(subtitles).[Ishibashi2006](subtitles). [Ishibashi2006] and [Boronat2009] provide an overview of technologies and algorithms for IDMS.IDMSInter-Destination Media Synchronization (IDMS) requires the exchange of information on mediareceiptarrival andplayoutpresentation times among participants in an IDMS session. It may also require signaling for the initiation and maintenance of IDMS sessions and groups of receivers. The presented RTCP specification for IDMS is independent of theusedsynchronizationalgorithm,algorithm employed, which isout-of-scopeout of scope of this document. 1.1. Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. 2. Rationale 2.1. Applicability of RTCP to IDMS Currently, a large share of real-time applications make use of RTP and RTCP [RFC3550]. RTP provides end-to-end network transport functions suitable for applications requiring real-time data transport, such as audio,videovideo, or data, over multicast or unicast network services. The timestamps, sequence numbers, and payload (content) type identification mechanisms provided by RTP packets are very useful for reconstructing the original mediatiming,timing and the original order of packets and for detecting packet loss at theclient side.receiver. The data transport is augmented by a control protocol (RTCP) to allow monitoring of the data delivery in a manner that is scalable to largegroups,groups and to provide minimal control and identification functionality. RTP receivers and senders provide reception quality feedback by sending out RTCPReceiver Reportreceiver report (RR) andSender Reportsender report (SR) packets [RFC3550], respectively, which may be augmented byeXtended Reportsextended report (XR) blocks [RFC3611].Both RTP and RTCP are intended to be tailored to modifications in order to include profile-specific information required by particular applications, and the guidelines on doing so are specified in [RFC5868].IDMS involves the collection,summarizingsummarization, and distribution of RTP packet arrival andplayoutpresentation times. As information on RTP packet arrival times andplayoutpresentation times can be considered reception quality feedback information, RTCP is well suited for carrying outIDMS, which may facilitate the implementation and deployment in typical multimedia applications.IDMS. 2.2. IDMS and ETSI A first version of IDMS for use with RTP/RTCP was standardized by ETSITISPANTelecommunications and Internet converged Services and Protocols for Advanced Networking (TISPAN) in [TS183063], resulting in an IANA registration for an RTCPIDMSXRBlock. At some point in time, thisBlock Type. This work was then brought as input to the IETF AVTCORE WG for further standardization, leveraging the RTP/RTCP expertise present in the AVTCORE WG. This document is the result of that effort. Although the IDMS protocol described in this document has evolved significantly from the version that was originally specified by ETSI TISPAN, it is still backwards compatible with the ETSI version. As such, it had been decided in ETSI to update the TS 183 063 document to reference this document as the normative specification of IDMS. This update can be found in newer versions of TS 183 063(>3.5.2).(i.e., versions higher than 3.5.2). In accordance, this document proposes to update the IANA registration for the RTCPIDMSXR IDMS Report Block to point to this document. Finally, this document proposes an IANA registry forSPSTSynchronization Packet Sender Type (SPST) values, allowing the registration of extensions to this document. 3.Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. 4.Inter-Destination Media Synchronization (IDMS)use casesUse Cases Thereareis a large number of use cases in which IDMS might be useful. This section will highlight some of them. It should be noted that this section is in no way meant to be exhaustive. A first usage scenario for IDMS is social TV. Social TV is the combination of media content consumption by two or more users at different devices and locations combined withthereal-time communication between those users. An example of social TV is when two or more users are watching the same television broadcast at different devices and locations, while communicating with each other using text,audioaudio, and/or video. A skew in their media playout processes can have adverse effects on their experience. A well-known use case here is one friend experiencing a goal in a football match well before or afterotheranother friend(s). Another potential use case for IDMS is a networked video wall. A video wall consists of multiple computer monitors, video projectors, or television sets tiled together contiguously or overlapped in order to form one large screen. Each of the screens reproduces a portion of the larger picture. In some implementations, each screen may be individually connected to the network and receive its portion of the overall image from a network-connected video server or video scaler. Screens are refreshed at 60 hertz (every 16-2/3 milliseconds) or potentially faster. If the refresh is not synchronized, the effect of multiple screens acting as one is broken, with users noticing tearing effects and no longer perceiving a single image. A third usage scenario is that ofthenetworkedloudspeakers,loudspeakers in which two or more speakers are connected to the network individually. Such situationscancan, forexampleexample, be found in large conference rooms, legislative chambers, classrooms (especially those supporting distancelearning)learning), and other large-scale environments such as stadiums. Since humans are moresusceptiblesensitive to differences in audio delay compared to video delay, this use case needs even more accuracy than the video wall use case. Depending on the exact application, the need for accuracy can then be in the range of microseconds.5.4. Overview of IDMSoperationOperation This section provides a brief example of how the RTCP functionality is used for achieving IDMS. The section is tutorial in nature and does not contain any normative statements. Alice's . . . . . . .tv:abc.com . . . . . . . Bob's TV (Sync Client) (Sync Server) Laptop (Sync Client) | | | | Media Session | | |<=====================>| | | Invite(URL, SyncGroupId) | |------------------------------------------------->| | | Media SessionSet-upSetup | | |<========================>| | | | | Callset-upSetup | |<================================================>| | | | | RTP Packets | RTP Packets | |<----------------------|------------------------->| | RR + XR IDMS Report | | |---------------------->| RR + XR IDMS Report | | |<-------------------------| | RTCP IDMS Settings | RTCP IDMS Settings | |<----------------------|------------------------->| | | | Figure 1: Example of atypicalTypical IDMSsessionSession Alice is watching TV in her living room. At somepointpoint, she sees thata football game ofBob's favorite team ison.playing football. She sends him an invite to watch the program together. Embedded in the invitation is the link to the media server and a unique sync-group identifier. Bob, who is also at home, receives the invite on his laptop. He accepts Alice'sinvitationinvitation, and the RTP client on his laptop sets up a sessiontowith the media server. AVoIPVoice over IP (VoIP) connection to Alice's TV is also set up, so that Alice and Bob can talk while watching the game together. As is common with RTP, both the RTP client in Alice's TV as well as the one in Bob's laptop send periodic RTCPReceiver Reports (RR)RRs to the media server. However, in order to make sure Alice and Bob see the events in the football game at(approximately)the same time, their clients also periodically send an RTCP XR IDMS Report Block to the Sync Server function of the media server. Included in the RTCP XR IDMSblocksReport Blocks are timestamps on when both Alice and Bob received (and, optionally, when they played out) a particular RTP packet. The Sync Server function in the media server calculates a reference client from the received RTCP XR IDMS Report Blocks(e.g.(e.g., by selecting the most lagged client as the reference for IDMS). It then sends an RTCP IDMS SettingspacketPacket containing the playout information of this reference client to the sync clients of both Alice and Bob. In thiscasecase, Bob's connection has the longest delay and the referenceclient thereforeclient, therefore, includes a delay similar to the one experienced by Bob. Upon reception of this information, Alice's RTP client can choose what to do with this information. In thiscasecase, it decreases its playout rate temporarily until the playout time matches with the reference client playout(and thus(and, thus, matches Bob's playout). Another option for Alice's TV would be to simply pause playback until it catches up. The exact implementation of the synchronization algorithm is up to the client. Upon reception of the RTCP IDMS Settingspacket,Packet, Bob's client does not have to do anything since it is already synchronized to the reference client (since it is based on Bob's delay). Note that other synchronization algorithms may introduce even more delay than the one experienced by the most delayed client,e.g.e.g., to account for delay variations, for new clients joining an existing synchronization group, etc. For this functionality to work correctly, it is necessary that thewall clockswallclocks of the receivers are synchronized with each other. While Alice and Bob both report when they receive, and optionally when theyplay out,playout, certain RTPpackets. Inpackets, in order to correlate their reports to each other, it is necessary that their wallclocks are synchronized.6.5. Architecture for Inter-Destination Media Synchronization The architecture for IDMS, which is based on a sync-maestro architecture [Boronat2009], issketcheddiagrammed below. In this particular case, the Synchronization Client (SC) and Media Synchronization Application Server (MSAS) entities are shown as additional functionality for the RTP receiver andsendersender, respectively. +-----------------------+ +-----------------------+ | | SR + | | | RTP Receiver | RTCP | RTP Sender | | | IDMS | | | +-----------------+ | <----- | +-----------------+ | | | | | | | | | | | Synchronization | | | | Media | | | | Client | | | | Synchronization | | | | (SC) | | | | Application | | | | | | | | Server | | | | | | RR+XR | | (MSAS) | | | | | | -----> | | | | | +-----------------+ | | +-----------------+ | | | | | +-----------------------+ +-----------------------+ Figure 2: IDMS Architecture Diagram6.1.5.1. Media Synchronization Application Server (MSAS) An MSAS collects RTP packet arrival times andplayoutpresentation times from one or moreSC(s)SCs in a synchronization group by receiving RTCPIDMSXRReports.IDMS reports. The MSAS summarizes and distributes this information to the SCs in the synchronization group as synchronization settings via the RTCP IDMS Settings Packet messages,e.g.e.g., by determining the SC with the most lagged playout and using its reported RTP packet arrival time andplayoutpresentation time as a summary. It should be noted that while thefigurediagram above shows the MSAS as part of the RTPSender,sender, this is not necessary. For example, an MSAS might also be implemented as an independent function in the network or in a master/slave type of architecture where one of the SC devices also acts as an MSAS. Wherever the MSAS is implemented, it is important that the MSAS has access to the RTP stream to which the XRReportsreports apply, so that it is able to correlate theIDMSRTCP XRReportsIDMS reports coming from different SCs.6.2.5.2. Synchronization Client (SC) An SC reports on RTP packet arrival times andplayoutpresentation times of a media stream. It can receive IDMS Settings Packets containing summaries of suchinformation,information and use that to adjust its playout buffer. The SC sends RTCPIDMSXRReportsIDMS reports to the MSAS.6.3.5.3. Communication between MSAS and SCs Two different message types are used for the communication between MSAS and SCs. For the SC->MSAS message containing the arrival and playout information of a particular client, an RTCP XR IDMS Report Block is used (see Section7).6). For the MSAS->SC message containing the synchronization settings instructions, a new RTCP IDMS Settings PacketTypeis defined (see Section8). 7.7). 6. RTCP XRBlock forIDMS(IDMSReportBlock)Block This section specifies a new RTCP XR Block Type, the RTCP XR IDMS Report Block, for reporting IDMS information to an MSAS. Inparticularparticular, it is used to provide feedback information onreceiptarrival times and presentation times of RTP packets. Its definition is based on [RFC3550] and [RFC3611]. In most cases, a single RTP receiver will only be part of a single IDMS session,i.e.i.e., it will report onreceiptarrival and presentation times of RTP packets from a single RTP stream in a certain synchronization group. In some cases, however, an RTP receiver may be a member of multiple synchronization groups for the same RTP stream,e.g.e.g., watching a single television program simultaneously with different groups. In even further cases, a receiver may wish to synchronize different RTP streams at the same time, either as part of the same synchronization group or as part of multiple synchronization groups. These are all valid scenarios forIDMS,IDMS and will require multiple reports by an SC. This document does not define new rulesonfor when to send RTCP reports, but uses the existing rules specified in [RFC3550] for sending RTCP reports. When the RTCP reporting timer allows an SC to send an IDMS report, the SC SHOULD report on an RTP packet received during the period since the last RTCP XR IDMS Report Block was sent. Because of RTP timestamp rollover, there is ambiguity in mapping RTP timestamps to NTP timestamps. The recommendation to report on recent RTP packets serves to manage this ambiguity. For more details on which packet to report on, see below under'Packet"Packet Received RTPtimestamp'.timestamp". 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | BT=12 | SPST |Resrv|P| block length=7 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | PT | Resrv | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Media Stream Correlation Identifier | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SSRC of media source | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Packet Received NTP timestamp, most significant word | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Packet Received NTP timestamp, least significant word | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Packet Received RTP timestamp | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Packet Presented NTP timestamp | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ The RTCP XR IDMSreport blockReport Block consists of 8 32-bit words, with the following fields: Block Type (BT): 8 bits. It identifies the block format. Its value is set to 12. Synchronization Packet Sender Type (SPST): 4 bits. This field identifies the role of the packet sender for this specificeXtendedExtended Report. It can have the following values, as enumerated in a registry maintained by IANA (see Section14.4):13.4): SPST=0 Reserved for future use. SPST=1 The packet sender is an SC. It uses this XR to report synchronization status information. Timestamps relate to the SC input. SPST=2-4 Values defined by ETSI TISPAN (see [TS183063]). SPST=5-15 Unassigned. Reserved bits (Resrv): 3 bits. These bits are reserved for future definition. In the absence of such a definition, the bits in this field MUST be set to zero at transmission and MUST be ignored by the receiver. Packet Presented NTP timestamp flag (P): 1 bit. Bit set to 1 if the Packet Presented NTP timestamp field contains a value, 0 if it is empty. If this flag is set tozero,0, then the Packet Presented NTP timestamp SHALL be ignored by the receiver. Block Length: 16 bits. This field indicates the length of the block in32 bit32-bit words minus one and is set to 7, as this RTCP XR IDMS BlockTypeReport has a fixed length. Payload Type (PT): 7 bits. This field identifies the format of the media payload, according to [RFC3551]. This is the payload type of the RTP packet reported upon. The PT field is needed in the case where the MSAS is neither theMedia Servermedia server nor a receiver of the media stream,i.e.i.e., it is implemented as athird-partthird-party entity. In such cases, the MSAS needs the PT to determine the rate of advancement of the timestamps of the RTP media stream to be able to relate reports from different SCs on different RTP timestamp values. Reserved bits (Resrv): 25 bits. These bits are reserved for future use and SHALL be set to 0 at transmission and MUST be ignored by the receiver. Media Stream Correlation Identifier: 32 bits. This identifier is used to correlate synchronized media streams. The value 0 (all bits are set "0") indicates that this field is empty. The value 2^32-1 (all bits are set "1") is reserved for future use. If the RTCP Packet Sender is an SC (SPST=1), then the Media Stream Correlation Identifier field contains the Synchronization Group Identifier (SyncGroupId) to which the report applies.SSRC:Synchronization Source (SSRC): 32 bits. The SSRC of the media source is set to the value of the SSRC identifier carried in the RTP header [RFC3550] of the RTP packet to which the XR IDMS relates. Packet Received NTP timestamp: 64 bits. This timestamp reflectsthe wall clockthe wallclock time at the moment of arrival of the first octet of the RTP packet to which the XR IDMS relates. It is formatted based on the NTP timestamp format as specified in [RFC5905]. See Section98 for more information on how this field is used. Packet Received RTP timestamp: 32 bits. This timestamp has the value of the RTP timestamp carried in the RTP header [RFC3550] of the RTP packet to which the XR IDMS relates. Several consecutive RTP packets will have equal timestamps if they are (logically) generated at once, e.g., belong to the same video frame. It may well be the case that one receiver reports on the first RTP packethavingthat has a certain RTPtimestamptimestamp, and a second receiver reports on the last RTP packethavingthat has that same RTP timestamp. This would lead to an error in the synchronization algorithm due to the faulty interpretation of considering both reports to be on the same RTP packet. When reporting on an RTPpacketpacket, which is one of several consecutive RTP packets having equal timestamps, an SC SHOULD report on the RTP packet it received with the lowest sequence number. Note thatwith'lowest sequence number' here is meant to be the first in the sequence of RTP packets just received, not from an earlier time before the lastwrap-aroundwrap around of RTP timestamps (unless thiswrap-aroundwrap around occurs during the sequence with equal RTP timestamps). Packet Presented NTP timestamp: 32 bits. This timestamp reflects thewall clockwallclock time at the moment the rendered media unit(e.g.(e.g., video frame or audio sample) contained in the first byte of the associated RTP packet is presented to the user. It is based on the time format used by NTP and consists of the least significant 16 bits of the NTP seconds part and the most significant 16 bits of the NTP fractional second part. If no Packet Presented NTP timestamp is available, this field SHALL be set to 0 and be consideredemptyempty, and the Packet Presented NTP timestamp flag (P) SHALL be set to 0. With regards to NTP epoch and rollover, the value of the Packet Presented NTP timestamp is considered to always be greater than the Packet Received NTPTimestamptimestamp and to be within 2^16 seconds of it. Presented in this context means the moment the data is played out to the user of the system,i.e.i.e., sound played out through speakers, video images being displayed on some display, etc. The accuracy resulting from the synchronization algorithm will only be as good as the accuracy with which the SCs can determine the delay between receiving packets and presenting them to theend-user. 8.end user. If no presentation timestamps are reported by SCs, the ability to accurately synchronize playout may be limited. 7. RTCP Packet Type for IDMS (IDMSSettings)Settings Packet) This section specifies the RTCPPacket Typepacket type for indicating synchronization settings instructions to the receivers of the RTP media stream. Its definition is based on [RFC3550]. Synchronization settings take the form of a report referencing a real or hypothetical RTP packet selected or contrived by the MSAS. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |V=2|P| Resrv |PT=TBDPT=211 | length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SSRC of packet sender | +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ | SSRC of media source | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Media Stream Correlation Identifier | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Packet Received NTP timestamp, most significant word | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Packet Received NTP timestamp, least significant word | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Packet Received RTP timestamp | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Packet Presented NTP timestamp, most significant word | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Packet Presented NTP timestamp, least significant word | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ The first 64 bits form the header of the RTCPPacket Type,packet type, as defined in [RFC3550]. The SSRC of the packet sender identifies the sender of the specific RTCP packet. The RTCP IDMSpacketSettings Packet consists of 7 32-bit words, with the following fields: PT:To be determined upon registration with IANA, inserted211, as registered bythe RFC Editor upon registration withIANA. SSRC: 32 bits. The SSRC of the media source is set to the value of the SSRC identifier of the media source carried in the RTP header [RFC3550] of the RTP packet to which the RTCP IDMSpacketSettings Packet relates. Media Stream Correlation Identifier: 32 bits. This identifier is used to correlate synchronized media streams. The value 0 (all bits are set "0") indicates that this field is empty. The value 2^32-1 (all bits are set "1") is reserved for future use. The Media Stream Correlation Identifier contains the SyncGroupId of the group to which this packet is sent. Packet Received NTP timestamp: 64 bits. This timestamp reflects thewall clockwallclock time at the reference client at the moment it received the first octet of the RTP packet to which this packet relates. It can be used by the synchronization algorithm on the receiving SC to adjust its playout timing in order to achieve synchronization,e.g.e.g., to set the required playout delay. The timestamp is formatted based on the NTP timestamp format as specified in [RFC5905]. See Section98 for more information on how this field is used. Because RTP timestamps do wrap around, the sender of this packet MUST use recent values,i.e.i.e., choose NTP timestamps that reflect current time and not too far in the future or in the past so as to create ambiguity with regards to RTP timestamp wrap around. Packet Received RTP timestamp: 32 bits. This timestamp has the value of the RTP timestamp carried in the RTP header [RFC3550] of the RTP packet to which the XR IDMS relates. This SHOULD relate to the first arriving RTP packet containing this particular RTP timestamp, in case multiple consecutive RTP packets contain the same RTP timestamp. Packet Presented NTP timestamp: 64 bits. This timestamp reflects thewall clockwallclock time at the reference client at the moment it presented the rendered media unit(e.g.(e.g., video frame or audio sample) contained in the first octet of the associated RTP packet to the user. The timestamp is formatted based on the NTP timestamp format as specified in [RFC5905]. If no Packet Presented NTP timestamp is available, this field SHALL be set to 0 and be considered empty. This field MAY be left empty if none or only one of the receivers reported on presentation timestamps. Presented here means the moment the data is played out to the user of the system. In some use cases(e.g.(e.g., phased array transducers), the level of control an MSAS might need to have over the exact moment of playout is so precise that a32bit32-bit PresentedTimestamptimestamp will not suffice. For this reason, this RTCPPacket Typepacket type for IDMS includes a64bit64-bit Presented Timestamp field. Since an MSAS will in practice always add some extra delay to the delay reported by the most lagged receiver (to account for packet jitter), it suffices for theIDMSRTCP XR IDMS Report BlockTypewith which the SCs report on their playout to have a32bit32-bit Presented Timestamp field.9.8. Timing and NTP Considerations To achieve IDMS, the different receivers involved need synchronized(wall) clockswallclocks as a common timeline for synchronization. This synchronized clock is used for reporting the Packet Received NTPTimestampstimestamp and the Packet Presented NTPTimestamp,timestamp, and for interpretation of these fields in received IDMSreports.Settings Packets. Depending on the synchronization accuracy required, different clock synchronization methods can be used. For social TV, synchronization accuracy should be achieved on the order of hundreds of milliseconds. In that case, correct use of NTP on receivers will in most situations achieve the required accuracy. As a guideline, to deal with clock drift of receivers, receivers should synchronize their clocks at the beginning of a synchronized session. In case of high required accuracy, the synchronized clocks of different receivers should not drift beyond the accuracy required for the synchronization mechanism. In practice, this can mean that receivers need to synchronize their clocks repeatedly during a synchronization session. Because of the stringent synchronization requirements for achieving good audio quality in some use cases, a high accuracy will be needed. In this case, use of the global NTP system may not be sufficient. For improved accuracy, a local NTP server could be set up, or some other more accurate clock synchronization mechanism can be used, such as GPS time or the Precision Time Protocol[IEEE-1588]. [I-D.draft-ietf-avtcore-clksrc][IEEE1588-2008]. [RFC7273] defines a set ofSDPSession Description Protocol (SDP) parameters for signaling the clock synchronization source or sources available to and used by the individual receivers. SCs MAY usethis draft[RFC7273] to indicate their clock synchronization source or sources in use and available. Using these parameters, an SC can indicate which synchronization source is being used at themoment, the last time the SC synchronized with this source and the synchronization frequency.moment. An SC can also indicate any other synchronization sources available to it. This allows multiple SCs in an IDMS session to use the same or a similar clock source for their session. Applications performing IDMS may or may not be able to choose a synchronization method for the systemclock,clock because this may be a system-wide settingwhichthat the application cannot change. How applications deal with this is up to the implementation. The application might control the system clock, or it might use a separate application clock or even a separate IDMS session clock. It might also report on the system clock and the synchronization method used, without being able to change it.[I-D.draft-ietf-leap-seconds][RFC7164] presents some guidelines on how RTP senders and receivers should deal with leap seconds. When relying on NTP for clock synchronization, IDMS is particularly sensitive toleap second inducedleap-second-induced timing discrepancies. It is RECOMMENDED to take the guidelines specified in[I-D.draft-ietf-leap-seconds][RFC7164] into account when implementing IDMS.10.9. On theuseUse ofpresentation timestampsPresentation Timestamps A receiver can report on different timing events,i.e.i.e., on packet arrival times and on playout or presentation times. A receiver SHALL report on arrival times and a receiver MAY report on playout times. RTP packet arrival times are relatively easy to report on. Normally, the processing and playout of the same media stream by different receivers will take roughly the same amount of time. Synchronizing on packet arrivaltimes,times may lead to some accuracy loss, but it will be adequate for many applications, such as social TV. Also, if the receivers are in some way controlled,e.g.e.g., having the same buffersettings,settings and decoding and rendering times, high accuracy can be achieved. However, if all receivers in a synchronization session have the ability to reporton, and thuson and, thus, synchronizeon,on actualplayout times, or packetpresentation times, thismaywill be more accurate. It is up to the applications and implementations of this RTCP extension whether to implement and usethis. 11.presentation timestamps. 10. SDPSignallingSignaling for RTCP IDMS Settings PacketTypeThe SDP attribute rtcp-idms is used to signal the use of the RTCP IDMS Settings PacketType for IDMSand the associated RTCP XRBlock for IDMS.IDMS Report Block. It is also used to carry an identifier of the synchronization group to which clients belong or will belong. The SDP attribute is used as a media-level attribute during session setup. This means that in case of multiple related streams, IDMS is performed on one of them. The other streams will be synchronized to this reference or master stream using existing inter-stream synchronization(i.e.(such as lip-sync) solutions,i.e.i.e., usingSender Reportssender reports based on a common clock source. Basic guidelines for choosing the media stream for IDMS is to choose audio above video, as humans aremoremost sensitive to degradation in audioquality than in video quality.synchronization. When using multi-description or multi-view codecs, the IDMS control should be performed on the base layer. This SDP attribute is defined as follows, using Augmented Backus-Naur Form [RFC5234]. rtcp-idms = "a=" "rtcp-idms" ":" sync-grp CRLF sync-grp = "sync-group=" SyncGroupId SyncGroupId = 1*10DIGIT ; Decimal value from 0 through 4294967294 DIGIT = %x30-39 SyncGroupId is a 32-bit unsigned integer and represented in decimal. SyncGroupId identifies a group of SCs for IDMS. The value SyncGroupId=0 represents an empty SyncGroupId. The value 4294967295 (2^32-1) is reserved for future use. For a description on the value of SyncGroupId to include, see Section12.11. The following is an example of the SDP attribute for IDMS. a=rtcp-idms:sync-group=4212.11. SDPrules 12.1.Rules 11.1. Offer/AnswerrulesRules The SDP usage for IDMS follows the rules defined in [RFC4566] andsectionSection 5 of [RFC3611] on SDPsignalling,signaling with the exception of what is stated here. The IDMS usage of RTCP is a(loosely coupled)loosely coupled collaborative attribute, in the sense that receivers send their status information and, in response, the MSAS(asynchronously)asynchronously sends synchronization setting instructions. The rtcp-idmsattribute thusattribute, thus, indicates the ability to send and receive indicated RTCP messages. This section defines how this SDP attribute should be used with regard to an offer/answer context. It is expected that, in most cases, the rtcp-idms attribute will be used in an offer/answer context where receivers will havepre- determined,predetermined, through some means outside the scope of this document, a SyncGroupId before the media session issetup.set up. However,it is also supportedA sender thatthe MSASassignssuchaSyncGroupId,SyncGroupId is also supported forexample ifcases, for example, where the MSAS contains group managementfunctionality.functionality and is co- located with or otherwise communicates with the sender. Thus, boththe MSASsenders andthe SCsreceivers can insert the attribute and the SyncGroupId. Furthermore,itthe attribute is allowed toinsert the attributeto be inserted for more than one media stream, allowing an SC to become part of multiple synchronization groups simultaneously. This effectively couples two (or more) synchronization groups to each other. If the rtcp-idms attribute is inserted more than once for a particular media session, each SyncGroupId SHALL only be inserted once. In order to join an IDMS session, the receiver (the SC) inserts the rtcp-idms attribute as amedia levelmedia-level attribute in the SDP offer. This SDP offer can be an initialoffer,offer if the media session is starting as a synchronized session. The SDP offer can also be an update to an existing media session, converting the session to an IDMS session. If the receiver has apre-determinedpredetermined SyncGroupId value, it SHOULD use this value for setting the SyncGroupId parameter in the rtcp-idms attribute. If the receiver does not know the SyncGroupId to be used, it MAY leave the SyncGroupId parameter empty by setting its value to 0. TheMSASsender SHALL include the rtcp-idms attribute in its answer. If the value of the SyncGroupId parameter in the offerwasis not empty (not equal to 0), theMSASsender SHOULD NOT change the SyncGroupId in its answer. If the SyncGroupIdwasis empty, theMSASsender SHALL include the proper SyncGroupId in its answer. If theMSASsender receives an offer with the value of the SyncGroupId parameter set to 0, and cannot determine the proper SyncGroupId, it SHALL remove the attribute from its answer.An MSASA sender receiving an SDP offer without the rtcp-idms attribute can also decide that IDMS is applicable to that media session. In such a case, theMSASsender MAY insert the rtcp-idms attribute, including a non- empty SyncGroupId, as part of its answer. A receiver receiving an rtcp-idms attribute as part of the SDP answer froman MSAS,a sender SHALL start sendingIDMSRTCP XR IDMS reports (following all the normal RTCP rules for sending RTCP XRblocks)IDMS Report Blocks) and SHALL be ready to start receiving IDMS Settings. As usual, if a receiver does not support the attribute(e.g.(e.g., in case of anMSAS-insertedMSAS- inserted IDMS attribute), it SHALL ignore the attribute. Different updates are applicable to such an IDMS session. Updates can be sentommittingomitting the rtcp-idms attribute, thereby endingthe (involvement in)involvement in the synchronization session. Updates can also be sent including the rtcp-idms attribute, but with a different SyncGroupId. This indicates a switch in the synchronization group.Updates can also be sent including another rtcp-idms attribute, indicating a membership of another synchronization group, effectively merging the current group(s) with the new one. 12.2.11.2. DeclarativecasesCases In certain situations, there is no offer/answer context, but only a declarative modus. In this case, the MSAS just inserts the rtcp-idms attribute and a valid SyncGroupId. Any receiver receiving the rtcp- idms attribute in such a declarative case SHALL start sendingIDMSRTCP XR IDMS Report Blocks and SHALL be ready to start receiving RTCP IDMS Settingspackets. 13.Packets. 12. Security Considerations The security considerations described in [RFC3611] apply to this document as well. The RTCP XR IDMS Report Block defined in this document is used to collect,summarizesummarize, and distribute information on packetreception-reception andplayout-timesplayout times of streaming media. The information may be used to orchestrate the media playout at multiple devices. Errors in the information, either accidental or malicious, may lead to undesired behavior. For example, if one device erroneously or maliciously reports a two-hour delayed playout, then another device in the same synchronization group could decide to delay its playout by two hours as well, in order to keep its playout synchronized. A user would likely interpret thistwo hourtwo-hour delay as a malfunctioning service. Therefore, the application logic of both SCs and MSASs should check for out-of-bound information. Differences in playout time exceeding configured limits(e.g.(e.g., more than ten seconds) could be an indication of such out-of-bound information. Apart from checking for out-of-bound information in theend-points,endpoints, an IDMS implementation can reduce its vulnerability to attacks by including source authentication and message integrity measures, reducing the potential for man-in-the-middle attacks.[I-D.draft-ietf-avtcore-rtp-security-options][RFC7201] provides an overview of the security options in RTPenvironments,environments and includes a set of recommendations for message integrity and source authenticationwhichthat are applicable to IDMS. In addition to preventing man-in-the-middle attacks from inserting erroneous IDMSReports,reports, the message confidentiality mechanisms outlined in[I-D.draft-ietf-avtcore-rtp-security-options][RFC7201] also prevent third parties from determining that two or more end hosts are receiving the same stream by looking at the Media Stream Correlation Identifier. Apart from attacking an IDMS session directly by sending incorrect IDMSReports,reports, and with it introducing delays for all devices in a synchronization group, another potential vulnerability comes from theusedclock synchronizationmethod.method used. Should an attacker succeed in adjusting an SC's wallclock, that SC will report incorrect IDMSReports.reports. In order to prevent such clock synchronization attacks, it is recommended to use a secure time synchronization service.14.13. IANA Considerations This document defines a new RTCP packet type, the RTCP IDMS Packet (IDMS Settings), within the existing Internet Assigned Numbers Authority (IANA) registry of RTCP Control Packet Types. This document also defines a new RTCP XR Block Type, theIDMSRTCP XR IDMS Report Block, within the existing IANA registry of RTCP Extended Reports (RTCP XR) Block Types. Further, this document defines a new SDP attribute "rtcp-idms" within the existing IANA registry of SDP Parameters, which is part of the "att-field (media level only)". Finally, this document defines a new IANA registry subordinate to the IANA RTCP Extended Reports (RTCP XR) Block Type Registry: the IDMS XR Block SPST Registry.14.1.13.1. RTCP IDMS Packet Type This document assigns the packet type valueTBD211 in the IANA 'RTCP Control Packet types(PT) Registry'(PT)' registry to the RTCP IDMS PacketType. [Note to RFC Editor: please replace TBD with the IANA-provided RTCP Packet Type value for this packet type, both in this section as well as in the figure and text in Section 8] 14.2.(IDMS Settings). 13.2. RTCP XR IDMS Report Block This document updates the assignment of value 12 from the RTCP XR Block Type for reporting IDMS information as per [TS183063] to the RTCP XR IDMS Report Block defined in this document.[Note to RFC Editor: this block type value is currently assigned to [TS183063]. This document replaces [TS183063] as the normative specification of the RTCP XR IDMS Report Block. Upon publication of this document as RFC, [TS183063] will be changed to reflect this.The RTCP XR IDMS Report Block contains an extensible SPST valuefield and thereforefield; therefore, a new registry for this field is required. This new registry is defined in Section14.4. 14.3.13.4. 13.3. RTCP-IDMS SDP Attribute The SDP attribute "rtcp-idms" defined by this document is registered with the IANA registry of SDP Parameters as follows: SDP Attribute ("att-field"): Attribute name: rtcp-idms Long form: RTCP IDMS Parameters Type of name: att-field Type of attribute: media level Subject to charset: no Purpose: see Section1110 of this document Reference: this document Values: see this document14.4.13.4. IDMS XR Block SPST Registry This document defines a new IANA registry subordinate to the IANA RTCP Extended Reports (RTCP XR) Block Type Registry: the IDMS XR Block SPST Registry. Initial values for the IDMS XR Block SPST Registry are given below; future assignments are to be made through the Specification Required policy [RFC5226]. The registry is limited to 16 entries (numbered 0-15), with 0 being Reserved. Values 5-15 are available for assignment. In accordance with [RFC5226], a Designated Expert will review any applications made to IANA for the registry. Primary criteria for the Designated Expert to use when reviewing new applications are clarity of the specification and, due to therelativerelatively small value range of SPST values available, potential overlap in functionality with existing SPST registrations. Value Name Reference ----- ---- --------- 1 Synchronization ClientsectionThis document, Section 7 2 MSAS [TS183063] 3 SC Prime Input [TS183063] 4 SC Prime Output [TS183063]14.5.13.5. Contact Information for Registrations The contact information for the registrations is: Ray van Brandenburg (ray.vanbrandenburg@tno.nl) Brassersplein 2 2612CT, Delft, The Netherlands15.14. Contributors The following people haveparticipated as co-authors orprovided substantial contributions to this document: Omar Niamut, Fabian Walraven, IshanVaishnaviVaishnavi, and Rufael Mekuria. In addition, the authors would like to thank Aidan Williams, Colin Perkins, Magnus Westerlund, Roni Even, Peter Musgrave, Ali Begen, QinWuWu, and Rob Koenen for their review comments and contributions to the text.16.15. References16.1.15.1. Normative References[I-D.draft-ietf-avtcore-clksrc] Williams, A., Gross, K., van Brandenburg, R., and H. Stokking, "RTP Clock Source Signalling, draft-ietf- avtcore-clksrc-05", May 2013.[RFC2119] Bradner, S., "KeyWordswords for use in RFCs to Indicate RequirementLevels,Levels", BCP 14, RFC2119",2119, March 1997. [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, "RTP: A Transport Protocol for Real-TimeApplications, RFC3550",Applications", STD 64, RFC 3550, July 2003. [RFC3551] Schulzrinne, H. and S. Casner, "RTP Profile for Audio and VideoconferencesConferences with MinimalControl, RFC3551",Control", STD 65, RFC 3551, July 2003. [RFC3611] Friedman, T.,Ed.,Caceres, R.,Ed.,and A. Clark,Ed.,"RTP Control Protocol Extended Reports (RTCPXR), RFC3611",XR)", RFC 3611, November 2003. [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session DescriptionProtocol, RFC4566",Protocol", RFC 4566, July 2006. [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 5226, May 2008. [RFC5234] Crocker,D., Ed.D. and P. Overell, "Augmented BNF for SyntaxSpecifications, RFC5234",Specifications: ABNF", STD 68, RFC 5234, January 2008. [RFC5905] Mills, D., Martin, J.,Ed.,Burbank, J., and W. Kasch, "Network Time Protocol Version 4: Protocol and AlgorithmsSpecifications, RFC5905", FebruarySpecification", RFC 5905, June 2010.16.2.[RFC7273] Williams, A., Gross, K., van Brandenburg, R., and H. Stokking, "RTP Clock Source Signalling", RFC 7273, June 2014. 15.2. Informative References [Boronat2009] Boronat, F., Lloret, J., and M. Garcia, "Multimedia group and inter-stream synchronization techniques: a comparativestudy,study", Elsevier Information Systems34 (2009), pp. 108-131", 2009. [I-D.draft-ietf-avtcore-rtp-security-options] Westerlund, M. and C. Perkins, "Options for Securing RTP Sessions", July 2013. [I-D.draft-ietf-leap-seconds] Gross, K. and R. Brandenburg, van, "RTP and Leap Seconds, draft-ietf-avtcore-leap-second-02", October 2012. [IEEE-1588] ,34, Pages 108-131, March 2009, <http://www.sciencedirect.com/science/article/pii/ S0306437908000525>. [IEEE1588-2008] IEEE, "1588-2008 - IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems",2008.July 2008, <http://standards.ieee.org/findstds/ standard/1588-2008.html>. [Ishibashi2006] Ishibashi, Y., Nagasaka, M., and N. Fujiyoshi, "SubjectiveAssessmentassessment ofFairnessfairness among users in multipointcommunications,communications", Proceedings of the 2006 ACM SIGCHIinternationinternational conference on Advances in computer entertainment technology,2006", . [RFC3711] Baughner, M., McGrew, D., Naslund, M., Carrara, E., andArticle No. 69, June 2006, <http://dl.acm.org/citation.cfm?id=1178905>. [RFC7164] Gross, K.Normann, "The Secure Real-time Transport Protocol (SRTP)",and R. Brandenburg, "RTP and Leap Seconds", RFC 7164, March2004. [RFC5868] Ott, J.2014. [RFC7201] Westerlund, M. and C. Perkins,"Guidelines"Options forExtending theSecuring RTPControl Protocol (RTCP), RFC5968", September 2010.Sessions", RFC 7201, April 2014. [TS183063], "IMS-basedETSI, "Telecommunications and Internet converged Services and Protocols for Advanced Networking (TISPAN); IMS-based IPTV stage 3specification,specification", TS 183 063v3.5.2",v3.5.2, March 2011. Authors' Addresses Ray van Brandenburg TNO Brassersplein 2 Delft 2612CTtheThe Netherlands Phone: +31-88-866-7000Email:EMail: ray.vanbrandenburg@tno.nl Hans Stokking TNO Brassersplein 2 Delft 2612CTtheThe Netherlands Phone: +31-88-866-7000Email:EMail: hans.stokking@tno.nl M. Oskar van Deventer TNO Brassersplein 2 Delft 2612CTtheThe Netherlands Phone: +31-88-866-7000Email:EMail: oskar.vandeventer@tno.nl Fernando Boronat Universitat Politecnica de ValenciaUniversitat Politecnica de Valencia(UPV) Valencia 46730 Spain Phone: +34 962 849 341Email:EMail: fboronat@dcom.upv.es Mario Montagud Universitat Politecnica de ValenciaUniversitat Politecnica de Valencia(UPV) Valencia 46730 Spain Phone: +34 962 849 341Email:EMail: mamontor@posgrado.upv.es Kevin Gross AVA Networks Phone: +1-303-447-0517Email:EMail: Kevin.Gross@AVAnw.com