3.1. Configuration-Time Errors

The Transport Services system should have a list of supported protocols available, each of which have transport features reflecting the capabilities of the protocol. Once an application specifies its Transport Properties, the Transport Services system matches the required and prohibited properties against the transport features of the available protocols (see Section 6.2 of [RFC9622] for the definition of property preferences).¶

In the following cases, failure should be detected during preestablishment:¶

• A request by an application for properties that cannot be satisfied by any of the available protocols. For example, if an application requires perMsgReliability, but no such feature is available in any protocol on the host running the Transport Services system, this should result in an error.¶

• A request by an application for properties that are in conflict with each other, such as specifying required and prohibited properties that cannot be satisfied by any protocol. For example, if an application prohibits reliability but then requires perMsgReliability, this mismatch should result in an error.¶

To avoid allocating resources that are not finally needed, it is important that configuration-time errors fail as early as possible.¶

3.2. Role of System Policy

The properties specified during preestablishment have a close relationship to system policy. The implementation is responsible for combining and reconciling several different sources of preferences when establishing Connections. These include, but are not limited to:¶

1. Application preferences, i.e., preferences specified during preestablishment via Selection Properties.¶

2. Dynamic system policy, i.e., policy compiled from internally and externally acquired information about available network interfaces, supported transport protocols, and current/previous Connections. Examples of ways to externally retrieve policy-support information are through OS-specific statistics/measurement tools and tools that reside on middleboxes and routers.¶

3. Default implementation policy, i.e., predefined policy by the OS or application.¶

In general, any protocol or path used for a Connection must conform to all three sources of constraints. A violation that occurs at any of the policy layers should cause a protocol or path to be considered ineligible for use. If such a violation prevents a Connection from being established, this should be communicated to the application, e.g., via the EstablishmentError event. For an example of application preferences leading to constraints, an application may prohibit the use of metered network interfaces for a given Connection to avoid user cost. Similarly, the system policy at a given time may prohibit the use of such a metered network interface from the application's process. Lastly, the implementation itself may default to disallowing certain network interfaces unless explicitly requested by the application.¶

It is expected that the database of system policies and the method of looking up these policies will vary across various platforms. An implementation should attempt to look up the relevant policies for the system in a dynamic way to make sure it reflects an accurate version of the system policy, since the system's policy regarding the application's traffic may change over time due to user or administrative changes.¶

4.1. Structuring Candidates as a Tree

As noted above, the consideration of multiple candidates in a gathering and racing process can be conceptually structured as a tree; this terminological convention is used throughout this document.¶

Each leaf node of the tree represents a single coherent connection attempt with an endpoint, a network path, and a set of protocols that can directly negotiate and send data on the network. Each node in the tree that is not a leaf represents a connection attempt that is either underspecified or includes multiple distinct options. For example, when connecting on an IP network, a connection attempt to a hostname and port is underspecified because the connection attempt requires a resolved IP address as its Remote Endpoint Identifier. In this case, the node represented by the connection attempt to the hostname is a parent node with child nodes for each IP address. Similarly, an implementation that is allowed to connect using multiple interfaces will have a parent node of the tree for the decision between the network paths with a branch for each interface.¶

The example aggregate connection attempt above can be drawn as a tree by grouping the addresses resolved on the same interface into branches:¶

                             ||
              +==============================+
              | www.example.com:443/any path |
              +==============================+
                 //                     \\
+===========================+      +===========================+
| www.example.com:443/Wi-Fi |      |  www.example.com:443/LTE  |
+===========================+      +===========================+
             ||                            //               \\
+============================+  +=====================+  +==========================+
| [2001:db8:23::1]:443/Wi-Fi |  |  192.0.2.1:443/LTE  |  | [2001:db8:42::1]:443/LTE |
+============================+  +=====================+  +==========================+

The rest of this section will use a notation scheme to represent this tree. The root node (or parent node) of the tree will be represented by a single integer, such as "1". ("1" is used assuming that this is the first connection made by the system; future connections created by the application would allocate numbers in an increasing manner.) Each child of that node will have an integer that identifies it, from 1 to the number of children. That child node will be uniquely identified by concatenating its integer to its parent's identifier with a dot character (".") in between, such as "1.1" and "1.2". Each node will be summarized by a tuple of three elements: endpoint, path (labeled here by interface), and protocol. In Protocol Stacks, the layers are separated by a slash character ("/") and ordered with the protocol closest to the application first. The above example can now be written more succinctly as:¶

1 [www.example.com:443, any path, TCP]
  1.1 [www.example.com:443, Wi-Fi, TCP]
    1.1.1 [[2001:db8:23::1]:443, Wi-Fi, TCP]
  1.2 [www.example.com:443, LTE, TCP]
    1.2.1 [192.0.2.1:443, LTE, TCP]
    1.2.2 [[2001:db8.42::1]:443, LTE, TCP]

When an implementation is asked to establish a single connection, only one of the leaf nodes in the candidate set is needed to transfer data. Thus, once a single leaf node becomes ready to use, the connection establishment tree is considered ready. One way to implement this is by having every leaf node update the state of its parent node when it becomes ready until the root node of the tree is ready, which then notifies the application that the Connection as a whole is ready to use.¶

A connection establishment tree may consist of only a single node, such as a connection attempt to an IP address over a single interface with a single protocol.¶

1 [[2001:db8:23::1]:443, Wi-Fi, TCP]

A root node may also only have one child (or leaf) node, such as a when a hostname resolves to only a single IP address.¶

1 [www.example.com:443, Wi-Fi, TCP]
  1.1 [[2001:db8:23::1]:443, Wi-Fi, TCP]

1 [www.example.com:443, Wi-Fi, TCP]
  1.1 [[2001:db8::1]:443, Wi-Fi, TCP]
  1.2 [192.0.2.1:443, Wi-Fi, TCP]
  1.3 [[2001:db8::2]:443, Wi-Fi, TCP]
  1.4 [[2001:db8::3]:443, Wi-Fi, TCP]

1 [term-printer._ipp._tcp.meeting.example.com, Wi-Fi, TCP]
  1.1 [term-printer.meeting.example.com:631, Wi-Fi, TCP]
    1.1.1 [31.133.160.18:631, Wi-Fi, TCP]

1 [192.0.2.1:443, any path, TCP]
  1.1 [192.0.2.1:443, Wi-Fi, TCP]
  1.2 [192.0.2.1:443, LTE, TCP]

1 [www.example.com:443, any path, HTTP/TCP]
  1.1 [192.0.2.8:443, any path, HTTP/HTTP Proxy/TCP]
  1.2 [192.0.2.7:10234, any path, HTTP/SOCKS/TCP]
  1.3 [www.example.com:443, any path, HTTP/TCP]
    1.3.1 [192.0.2.1:443, any path, HTTP/TCP]

1 [www.example.com:443, any path, HTTP]
  1.1 [www.example.com:443, any path, HTTP3/QUIC/UDP]
    1.1.1 [192.0.2.1:443, any path, HTTP3/QUIC/UDP]
  1.2 [www.example.com:443, any path, HTTP2/TLS/TCP]
    1.2.1 [192.0.2.1:443, any path, HTTP2/TLS/TCP]

1 [www.example.com:4740, any path, reliable-inorder-stream]
  1.1 [www.example.com:4740, any path, SCTP]
    1.1.1 [192.0.2.1:4740, any path, SCTP]
  1.2 [www.example.com:4740, any path, TCP]
    1.2.1 [192.0.2.1:4740, any path, TCP]

1. [www.example.com:80, any path, reliable-inorder-stream]
1.1 [192.0.2.1:443, Wi-Fi, reliable-inorder-stream]
1.1.1 [192.0.2.1:443, Wi-Fi, TCP]
1.2 [192.0.3.1:443, LTE, reliable-inorder-stream]
1.2.1 [192.0.3.1:443, LTE, SCTP]
1.2.2 [192.0.3.1:443, LTE, TCP]

4.2. Candidate Gathering

The step of gathering candidates involves identifying which paths, protocols, and endpoints may be used for a given Connection. This list is determined by the requirements, prohibitions, and preferences of the application as specified in the Selection Properties.¶

4.3. Candidate Racing

The primary goal of the Candidate Racing process is to successfully negotiate a Protocol Stack to an endpoint over an interface to connect a single leaf node of the tree with as little delay and as few unnecessary connections attempts as possible. Optimizing these two factors improves the user experience, while minimizing network load.¶

This section covers the dynamic aspect of connection establishment. The tree described above is a useful conceptual and architectural model. However, an implementation is unable to know all of the nodes that will be used until steps like name resolution have occurred; many of the possible branches ultimately might not be attempted.¶

There are three different approaches to racing the attempts for different nodes of the connection establishment tree:¶

1. Simultaneous¶

2. Staggered¶

3. Failover¶

Each approach is appropriate in different use cases and branch types. However, to avoid consuming unnecessary network resources, implementations should not use simultaneous racing as a default approach.¶

The timing algorithms for racing should remain independent across branches of the tree. Any timer or racing logic is isolated to a given parent node and is not ordered precisely with regard to children of other nodes.¶

4.4. Completing Establishment

The process of connection establishment completes when one leaf node of the tree has successfully completed negotiation with the Remote Endpoint, or else all nodes of the tree have failed to connect. The first leaf node to complete its connection is then used by the application to send and receive data. This is signaled to the application using the Ready event in the API (Section 7.1 of [RFC9622]).¶

Successes and failures of a given attempt should be reported up to parent nodes (toward the root of the tree). For example, in the following case, if 1.1.1 fails to connect, it reports the failure to 1.1. Since 1.1 has no other child nodes, it also has failed and reports that failure to 1. Because 1.2 has not yet failed, 1 is not considered to have failed. Since 1.2 has not yet started, it is started and the process continues. Similarly, if 1.1.1 successfully connects, then it marks 1.1 as connected, which propagates to the root node 1. At this point, the Connection as a whole is considered to be successfully connected and ready to process application data.¶

1 [www.example.com:443, Any, TCP]
  1.1 [www.example.com:443, Wi-Fi, TCP]
    1.1.1 [192.0.2.1:443, Wi-Fi, TCP]
  1.2 [www.example.com:443, LTE, TCP]
...

If a leaf node has successfully completed its connection, all other attempts should be made ineligible for use by the application for the original request. New connection attempts that involve transmitting data on the network ought not to be started after another leaf node has already successfully completed because the Connection as a whole has now been established. An implementation could choose to let certain handshakes and negotiations complete to gather metrics that influence future connections. Keeping additional connections is generally not recommended because those attempts were slower to connect and may exhibit less desirable properties.¶

4.5. Establishing Multiplexed Connections

Multiplexing several Connections over a single underlying transport connection requires that the Connections to be multiplexed belong to the same Connection Group (as is indicated by the application using the Clone action). When the underlying transport connection supports multi-streaming, the Transport Services System can map each Connection in the Connection Group to a different stream of this connection.¶

For such streams, there is often no explicit connection establishment procedure for the new stream prior to sending data on it (e.g., with SCTP). In this case, the same considerations apply to determining stream establishment as apply to establishing a UDP connection, as discussed in Section 4.4.1. This means that there might not be any "establishment" message (like a TCP SYN).¶

4.6. Handling Connectionless Protocols

While protocols that use an explicit handshake to validate a connection to a peer can be used for racing multiple establishment attempts in parallel, connectionless protocols such as raw UDP do not offer a way to validate the presence of a peer or the usability of a Connection without application feedback. An implementation should consider such a Protocol Stack to be established as soon as the Transport Services system has selected a path on which to send data.¶

However, this can cause a problem if a specific peer is not reachable over the network using the connectionless protocol or data cannot be exchanged with the peer for any other reason. To handle the lack of an explicit handshake in the underlying protocol, an application can use a Message Framer (Section 6) on top of a connectionless protocol to only mark a specific connection attempt as ready when some data has been received or after some application-level handshake has been performed by the Message Framer.¶

4.7. Implementing Listeners

When an implementation is asked to Listen, it registers with the system to wait for incoming traffic to the Local Endpoint. If no Local Endpoint Identifier is specified, the implementation should use an ephemeral port.¶

If the Selection Properties do not require a single network interface or path but allow the use of multiple paths, the Listener object should register for incoming traffic on all of the network interfaces or paths that conform to the Properties. The set of available paths can change over time, so the implementation should monitor network path changes and change the registration of the Listener across all usable paths as appropriate. When using multiple paths, the Listener is generally expected to use the same port for listening on each.¶

If the Selection Properties allow multiple protocols to be used for listening and the implementation supports it, the Listener object should support receiving inbound connections for each eligible protocol on each eligible path.¶

5.1. Sending Messages

The effect of the application sending a Message is determined by the top-level protocol in the established Protocol Stack. That is, if the top-level protocol provides an abstraction of framed Messages over a connection, the receiving application will be able to obtain multiple Messages on that connection, even if the framing protocol is built on a byte-stream protocol like TCP.¶

5.2. Receiving Messages

Similar to sending, receiving a Message is determined by the top-level protocol in the established Protocol Stack. The main difference with receiving is that the size and boundaries of the Message are not known beforehand. The application can communicate the parameters for the Message in its Receive action, which can help the Transport Services Implementation know how much data to deliver and when. For example, if the application only wants to receive a complete Message, the implementation should wait until an entire Message (datagram, stream, or frame) is read before delivering any Message content to the application. This requires the implementation to understand where Messages end, either via a supplied Message Framer or because the top-level protocol in the established Protocol Stack preserves message boundaries. The application can also control the flow of received data by specifying the minimum and maximum number of bytes of Message content it wants to receive at one time.¶

If a Connection finishes before a requested Receive action can be satisfied, the Transport Services system should deliver any partial Message content outstanding, or if none is available, an indication that there will be no more received Messages.¶

5.3. Handling of Data for Fast-Open Protocols

Several protocols allow sending higher-level protocol or application data during their protocol establishment, such as TFO [RFC7413] and TLS 1.3 [RFC8446]. This approach is referred to as sending Zero-RTT (0-RTT) data. This is a desirable feature, but it poses challenges to an implementation that uses racing during connection establishment.¶

The application can express its preference for sending messages as 0-RTT data by using the zeroRttMsg Selection Property on the Preconnection. Then, the application can provide the message to send as 0-RTT data via the InitiateWithSend action. In order to be sent as 0-RTT data, the message needs to be marked with the safelyReplayable send parameter. In general, 0-RTT data may be replayed (for example, if a TCP SYN contains data, and the SYN is retransmitted, the data will be retransmitted as well but may be considered a new connection instead of a retransmission). When racing connections, different leaf nodes have the opportunity to send the same data independently. If data is truly safely replayable, this is permissible.¶

Once the application has provided its 0-RTT data, a Transport Services Implementation should keep a copy of this data and provide it to each new leaf node that is started and for which a protocol instance supporting 0-RTT is being used. Note that the amount of data that can actually be sent as 0-RTT data varies by protocol, so any given Protocol Stack might only consume part of the saved data prior to becoming established. The implementation needs to keep track of how much data a particular Protocol Stack has consumed and ensure that any pending 0-RTT-eligible data from the application is handled before subsequent Messages.¶

It is also possible for Protocol Stacks within a particular leaf node to use a 0-RTT handshake in a lower-level protocol without any safely replayable application data if a higher-level protocol in the stack has idempotent handshake data to send. For example, TFO could use a Client Hello from TLS as its 0-RTT data without any data being provided by the application.¶

0-RTT handshakes often rely on previous state, such as TFO cookies, previously established TLS tickets, or out-of-band distributed pre-shared keys (PSKs). Implementations should be aware of security concerns around using these tokens across multiple addresses or paths when racing. In the case of TLS, any given ticket or PSK should only be used on one leaf node, since servers will likely reject duplicate tickets in order to prevent replays (see Section 8.1 of [RFC8446]). If implementations have multiple tickets available from a previous connection, each leaf node attempt can use a different ticket. In effect, each leaf node will send the same early application data, yet encoded (encrypted) differently on the wire.¶

6.1. Defining Message Framers

A Message Framer is primarily defined by the code that handles events for a framer implementation, specifically how it handles inbound and outbound data parsing. The function that implements custom framing logic will be referred to as the "framer implementation", which may be provided by a Transport Services implementation or the application itself. The Message Framer refers to the object or function within the main Connection implementation that delivers events to the custom framer implementation whenever data is ready to be parsed or framed.¶

The API examples in this section use the notation conventions for the Transport Services API defined in Section 1.1 of [RFC9622].¶

The Transport Services Implementation needs to ensure that all of the events and actions taken on a Message Framer are synchronized to ensure consistent behavior. For example, some of the actions defined below (such as PrependFramer and StartPassthrough) modify how data flows in a Protocol Stack and require synchronization with sending and parsing data in the Message Framer.¶

When a Connection establishment attempt begins, an event can be delivered to notify the framer implementation that a new Connection is being created. Similarly, a stop event can be delivered when a Connection is being torn down. The framer implementation can use the Connection object to look up specific properties of the Connection or the network being used that may influence how to frame Messages.¶

MessageFramer -> Start<connection>
MessageFramer -> Stop<connection>

When a Message Framer generates a Start event, the framer implementation has the opportunity to start writing some data prior to the Connection delivering its Ready event. This allows the implementation to communicate control data to the Remote Endpoint that can be used to parse Messages.¶

Once the framer implementation has completed its setup or handshake, it can indicate to the application that it is ready for handling data with this call.¶

MessageFramer.MakeConnectionReady(connection)

Similarly, when a Message Framer generates a Stop event, the framer implementation has the opportunity to write some final data or clear up its local state before the Closed event is delivered to the application. The framer implementation can indicate that it has finished with this call.¶

MessageFramer.MakeConnectionClosed(connection)

If the implementation encounters a fatal error at any time, it can also cause the Connection to fail and provide an error.¶

MessageFramer.FailConnection(connection, error)

Should the framer implementation deem the candidate selected during racing unsuitable, it can signal this to the Transport Services API by failing the Connection prior to marking it as ready. If there are no other candidates available, the Connection will fail. Otherwise, the Connection will select a different candidate and the Message Framer will generate a new Start event.¶

Before an implementation marks a Message Framer as ready, it can also dynamically add a protocol or framer above it in the stack. This allows protocols that need to add TLS conditionally, like STARTTLS [RFC3207], to modify the Protocol Stack based on a handshake result.¶

otherFramer := NewMessageFramer()
MessageFramer.PrependFramer(connection, otherFramer)

A Message Framer might also choose to go into a passthrough mode once an initial exchange or handshake has been completed, such as the STARTTLS case mentioned above. This can also be useful for proxy protocols like SOCKS [RFC1928] or HTTP CONNECT [RFC7230]. In such cases, a Message Framer implementation can intercept sending and receiving of Messages at first, but then indicate that no more processing is needed.¶

MessageFramer.StartPassthrough()

6.2. Sender-Side Message Framing

Message Framers generate an event whenever a Connection sends a new Message. The parameters to the event align with the Send action in the API (Section 9.2 of [RFC9622]).¶

                        MessageFramer
                              |
                              V
NewSentMessage<connection, messageData, messageContext, endOfMessage>

Upon receiving this event, a framer implementation is responsible for performing any necessary transformations and sending the resulting data back to the Message Framer, which, in turn, will send it to the next protocol. To improve performance, implementations should ensure that there is a way to pass the original data through without copying.¶

MessageFramer.Send(connection, messageData)

To provide an example, a simple protocol that adds the length of the Message data as a header would receive the NewSentMessage event, create a data representation of the length of the Message data, and then send a block of data that is the concatenation of the length header and the original Message data.¶

6.3. Receiver-Side Message Framing

In order to parse a received flow of data into Messages, the Message Framer notifies the framer implementation whenever new data is available to parse.¶

The parameters to the events and calls for receiving data with a framer align with the Receive action in the API (Section 9.3 of [RFC9622]).¶

MessageFramer -> HandleReceivedData<connection>

Upon receiving this event, the framer implementation can inspect the inbound data. The data is parsed from a particular cursor representing the unprocessed data. The application requests a specific amount of data it needs to have available in order to parse. If the data is not available, the parse fails.¶

MessageFramer.Parse(connection, minimumIncompleteLength, maximumLength)
                                 |
                                 V
             (messageData, messageContext, endOfMessage)

The framer implementation can directly advance the receive cursor once it has parsed data to effectively discard data (for example, discard a header once the content has been parsed).¶

To deliver a Message to the application, the framer implementation can either directly deliver data that it has allocated or deliver a range of data directly from the underlying transport and simultaneously advance the receive cursor.¶

MessageFramer.AdvanceReceiveCursor(connection, length)
MessageFramer.DeliverAndAdvanceReceiveCursor(connection, messageContext, length, endOfMessage)
MessageFramer.Deliver(connection, messageContext, messageData, endOfMessage)

Note that MessageFramer.DeliverAndAdvanceReceiveCursor allows the framer implementation to earmark bytes as part of a Message even before they are received by the transport. This allows the delivery of very large Messages without requiring the implementation to directly inspect all of the bytes.¶

To provide an example, a simple protocol that parses the length of the Message data as a header value would receive the HandleReceivedData event and call Parse with a minimum and maximum set to the length of the header field. Once the parse succeeded, it would call AdvanceReceiveCursor with the length of the header field and then call DeliverAndAdvanceReceiveCursor with the length of the body that was parsed from the header, marking the new Message as complete.¶

7.1. Pooled Connection

For applications that do not need in-order delivery of Messages, the Transport Services Implementation may distribute Messages of a single Connection across several underlying transport connections or multiple streams of multi-streaming connections between endpoints, as long as all of these satisfy the Selection Properties. The Transport Services Implementation will then hide this connection management and only expose a single Connection object, which we call a "Pooled Connection". This is in contrast to Connection Groups, which explicitly expose combined treatment of Connections, giving the application control over multiplexing, for example.¶

Pooled Connections can be useful when the application using the Transport Services system implements a protocol such as HTTP, which employs request/response pairs and does not require in-order delivery of responses. This enables implementations of Transport Services systems to realize transparent connection coalescing and connection migration and to perform per-message endpoint and path selection by choosing among multiple underlying connections.¶

7.2. Handling Path Changes

When a path change occurs, e.g., when the IP address of an interface changes or a new interface becomes available, the Transport Services Implementation is responsible for notifying the Protocol Instance of the change. The path change may interrupt connectivity on a path for an active Connection or provide an opportunity for a transport that supports multipath or migration to adapt to the new paths. Note that, in the model of the Transport Services API, migration is considered a part of multipath connectivity; it is just a limiting policy on multipath usage. If the multipath Selection Property is set to Disabled, migration is disallowed.¶

For protocols that do not support multipath or migration, the Protocol Instances should be informed of the path change but should not be forcibly disconnected if the previously used path becomes unavailable. There are many common usage scenarios that can lead to a path becoming temporarily unavailable and then recovering before the transport protocol reaches a timeout error. These are particularly common using mobile devices. Examples include:¶

• an Ethernet cable becoming unplugged and then plugged back in;¶
• a device losing a Wi-Fi signal while a user is in an elevator and reattaching when the user leaves the elevator; and¶
• a user losing the radio signal while riding a train through a tunnel.¶

If the device is able to rejoin a network with the same IP address, a stateful transport connection can generally resume. Thus, while it is useful for a Protocol Instance to be aware of a temporary loss of connectivity, the Transport Services Implementation should not aggressively close Connections in these scenarios.¶

If the Protocol Stack includes a transport protocol that supports multipath connectivity, the Transport Services Implementation should also inform the Protocol Instance about potentially new paths that become permissible based on the multipath Selection Property and the multipathPolicy Connection Property choices made by the application. A protocol can then establish new subflows over new paths while an active path is still available or, if migration is supported, also after a break has been detected, and should attempt to tear down subflows over paths that are no longer used. The Connection Property multipathPolicy of the Transport Services API allows an application to indicate when and how different paths should be used. However, detailed handling of these policies is implementation specific. For example, if the multipath Selection Property is set to active, the decision about when to create a new path or to announce a new path or set of paths to the Remote Endpoint, e.g., in the form of additional IP addresses, is implementation specific. If the Protocol Stack includes a transport protocol that does not support multipath but does support migrating between paths, the update to the set of available paths can trigger the connection to be migrated.¶

In the case of a Pooled Connection (Section 7.1), the Transport Services Implementation may add connections over new paths to the pool if permissible based on the multipath policy and Selection Properties. If a previously used path becomes unavailable, the Transport Services system may disconnect all connections that require this path, but it should not disconnect the pooled Connection object exposed to the application. The strategy to do so is implementation specific, but it should be consistent with the behavior of multipath transports.¶

9.1. Protocol State Caches

Some protocols will have long-term state to be cached in association with endpoints. This state often has some time after which it is expired, so the implementation should allow each protocol to specify an expiration for cached content.¶

Examples of cached protocol state include:¶

• The DNS protocol can cache resolved addresses (such as those retrieved from A and AAAA queries) associated with a Time To Live (TTL) to be used for future hostname resolutions without requiring asking the DNS resolver again.¶

• TLS caches session state and tickets based on a hostname, which can be used for resuming sessions with a server.¶

• TCP can cache cookies for use in TFO¶

Cached protocol state is primarily used during Connection establishment for a single Protocol Stack, but it may be used to influence an implementation's preference between several candidate Protocol Stacks. For example, if two IP address Endpoint Identifiers are otherwise equally preferred, an implementation may choose to attempt a connection to an address for which it has a TFO cookie.¶

Applications can use the Transport Services API to request that a Connection Group maintain a separate cache for protocol state. Connections in the group will not use cached state from Connections outside the group, and Connections outside the group will not use state cached from Connections inside the group. This may be necessary, for example, if application-layer identifiers rotate and clients wish to avoid linkability via trackable TLS tickets or TFO cookies.¶

9.2. Performance Caches

In addition to protocol state, Protocol Instances should provide data into a performance-oriented cache to help guide future protocol and path selection. Some performance information can be gathered generically across several protocols to allow predictive comparisons between protocols on given paths:¶

• Observed RTT¶

• Connection establishment latency¶

• Connection establishment success rate¶

These items can be cached on a per-address and per-subnet granularity and averaged between different values. The information should be cached on a per-network basis since it is expected that different network attachments will have different performance characteristics. Besides Protocol Instances, other system entities may also provide data into performance-oriented caches. This could for instance be signal strength information reported by radio modems like Wi-Fi and mobile broadband or information about the battery-level of the device. Furthermore, the system may cache the observed maximum throughput on a path as an estimate of the available bandwidth.¶

An implementation should use this information, when possible, to influence preference between candidate paths, endpoints, and protocol options. Eligible options that historically had significantly better performance than others should be selected first when gathering candidates (see Section 4.2) to ensure better performance for the application.¶

The reasonable lifetime for cached performance values will vary depending on the nature of the value. Certain information, like the connection establishment success rate to a Remote Endpoint using a given Protocol Stack, can be stored for a long period of time (hours or longer) since it is expected that the capabilities of the Remote Endpoint are not changing very quickly. On the other hand, the RTT observed by TCP over a particular network path may vary over a relatively short time interval. For such values, the implementation should remove them from the cache more quickly or treat older values with less confidence/weight.¶

[RFC9040] provides guidance about sharing of TCP Control Block information between connections on initialization.¶

10.1. TCP

Connectedness:

Connected¶

Data Unit:

Byte-stream¶

Connection Object:

TCP connections between two hosts map directly to Connection objects.¶

Initiate:

CONNECT.TCP. Calling Initiate on a TCP Connection causes it to reserve a local port and send a SYN to the Remote Endpoint.¶

InitiateWithSend:

CONNECT.TCP with parameter user message. Early safely replayable data is sent on a TCP Connection in the SYN, as TFO data.¶

Ready:

A TCP Connection is ready once the three-way handshake is complete.¶

EstablishmentError:

Failure of CONNECT.TCP. TCP can throw various errors during connection setup. Specifically, it is important to handle a RST being sent by the peer during the handshake.¶

ConnectionError:

Once established, TCP throws errors whenever the connection is disconnected, such as due to receiving a RST from the peer.¶

Listen:

LISTEN.TCP. Calling Listen for TCP binds a local port and prepares it to receive inbound SYN packets from peers.¶

ConnectionReceived:

TCP Listeners will deliver new connections once they have replied to an inbound SYN with a SYN-ACK.¶

Clone:

Calling Clone on a TCP Connection creates a new Connection with equivalent parameters. These Connections, and Connections generated via later calls to Clone on an Established Connection, form a Connection Group. To realize entanglement for these Connections, with the exception of connPriority, changing a Connection Property on one of them must affect the Connection Properties of the others too. No guarantees of honoring the Connection Property connPriority are given; thus, it is safe for an implementation of a Transport Services system to ignore this property. When it is reasonable to assume that Connections traverse the same path (e.g., when they share the same encapsulation), support for it can also experimentally be implemented using a congestion control coupling mechanism (for example, see [TCP-COUPLING] or [RFC3124]).¶

Send:

SEND.TCP. On its own, TCP does not preserve message boundaries. Calling Send on a TCP connection lays out the bytes on the TCP send stream without any other delineation. Any Message marked as Final will cause TCP to send a FIN once the Message has been completely written, by calling CLOSE.TCP immediately upon successful termination of SEND.TCP. Note that transmitting a Message marked as Final should not cause the Closed event to be delivered to the application as it will still be possible to receive data until the peer closes or aborts the TCP connection.¶

Receive:

With RECEIVE.TCP, TCP delivers a stream of bytes without any Message delineation. All data delivered in the Received or ReceivedPartial event will be part of a single stream-wide Message that is marked Final (unless a Message Framer is used). EndOfMessage will be delivered when the TCP Connection has received a FIN (CLOSE-EVENT.TCP) from the peer. Note that reception of a FIN should not cause the Closed event to be delivered to the application, as it will still be possible for the application to send data.¶

Close:

Calling Close on a TCP Connection indicates that the Connection should be gracefully closed (CLOSE.TCP) by sending a FIN to the peer. It will then still be possible to receive data until the peer closes or aborts the TCP connection. The Closed event will be issued upon reception of a FIN.¶

Abort:

Calling Abort on a TCP Connection indicates that the Connection should be immediately closed by sending a RST to the peer (ABORT.TCP).¶

CloseGroup:

Calling CloseGroup on a TCP Connection (CLOSE.TCP) is identical to calling Close on this Connection and on all Connections in the same ConnectionGroup.¶

AbortGroup:

Calling AbortGroup on a TCP Connection (ABORT.TCP) is identical to calling Abort on this Connection and on all Connections in the same ConnectionGroup.¶

10.2. MPTCP

Connectedness:

Connected¶

Data Unit:

Byte-stream¶

The Transport Services API mappings for MPTCP are identical to TCP. MPTCP adds support for multipath properties, such as multipath and multipathPolicy, and actions for managing paths, such as AddRemote and RemoveRemote.¶

10.3. UDP

Connectedness:

Connectionless¶

Data Unit:

Datagram¶

Connection Object:

UDP Connections represent a pair of specific IP addresses and ports on two hosts.¶

Initiate:

CONNECT.UDP. Calling Initiate on a UDP Connection causes it to reserve a local port but does not generate any traffic.¶

InitiateWithSend:

Early data on a UDP Connection does not have any special meaning. The data is sent whenever the Connection is Ready.¶

Ready:

A UDP Connection is ready once the system has reserved a local port and has a path to send to the Remote Endpoint.¶

EstablishmentError:

UDP Connections can only generate errors on initiation due to port conflicts on the local system.¶

ConnectionError:

UDP Connections can only generate Connection errors in response to Abort calls. (Once in use, UDP Connections can also generate SoftError events (ERROR.UDP) upon receiving ICMP notifications indicating failures in the network.)¶

Listen:

LISTEN.UDP. Calling Listen for UDP binds a local port and prepares it to receive inbound UDP datagrams from peers.¶

ConnectionReceived:

UDP Listeners will deliver new connections once they have received traffic from a new Remote Endpoint.¶

Clone:

Calling Clone on a UDP Connection creates a new Connection with equivalent parameters. The two Connections are otherwise independent.¶

Send:

SEND.UDP. Calling Send on a UDP connection sends the data as the payload of a complete UDP datagram. Marking Messages as Final does not change anything in the datagram's contents. Upon sending a UDP datagram, some relevant fields and flags in the IP header can be controlled: DSCP (SET_DSCP.UDP), DF in IPv4 (SET_DF.UDP), and ECN flag (SET_ECN.UDP).¶

Receive:

RECEIVE.UDP. UDP only delivers complete Messages to Received, each of which represents a single datagram received in a UDP packet. Upon receiving a UDP datagram, the ECN flag from the IP header can be obtained (GET_ECN.UDP).¶

Close:

Calling Close on a UDP Connection (ABORT.UDP) releases the local port reservation. The Connection then issues a Closed event.¶

Abort:

Calling Abort on a UDP Connection (ABORT.UDP) is identical to calling Close except that the Connection will send a ConnectionError event rather than a Closed event.¶

CloseGroup:

Calling CloseGroup on a UDP Connection (ABORT.UDP) is identical to calling Close on this Connection and on all Connections in the same ConnectionGroup.¶

AbortGroup:

Calling AbortGroup on a UDP Connection (ABORT.UDP) is identical to calling Close on this Connection and on all Connections in the same ConnectionGroup.¶

10.4. UDP-Lite

Connectedness:

Connectionless¶

Data Unit:

Datagram¶

The Transport Services API mappings for UDP-Lite are identical to UDP. In addition, UDP-Lite supports the msgChecksumLen and recvChecksumLen Properties that allow an application to specify the minimum number of bytes in a Message that need to be covered by a checksum.¶

This includes: CONNECT.UDP-Lite; LISTEN.UDP-Lite; SEND.UDP-Lite; RECEIVE.UDP-Lite; ABORT.UDP-Lite; ERROR.UDP-Lite; SET_DSCP.UDP-Lite; SET_DF.UDP-Lite; SET_ECN.UDP-Lite; GET_ECN.UDP-Lite.¶

10.5. UDP Multicast Receive

Connectedness:

Connectionless¶

Data Unit:

Datagram¶

Connection Object:

Established UDP Multicast Receive connections represent a pair of specific IP addresses and ports. The direction Selection Property must be set to unidirectional receive, and the Local Endpoint must be configured with a group IP address and a port.¶

Initiate:

Calling Initiate on a UDP Multicast Receive Connection causes an immediate EstablishmentError. This is an unsupported operation.¶

InitiateWithSend:

Calling InitiateWithSend on a UDP Multicast Receive Connection causes an immediate EstablishmentError. This is an unsupported operation.¶

Ready:

A UDP Multicast Receive Connection is ready once the system has received traffic for the appropriate group and port.¶

EstablishmentError:

UDP Multicast Receive Connections generate an EstablishmentError indicating that joining a multicast group failed if Initiate is called.¶

ConnectionError:

The only ConnectionError generated by a UDP Multicast Receive Connection is in response to an Abort call.¶

Listen:

LISTEN.UDP. Calling Listen for UDP Multicast Receive binds a local port, prepares it to receive inbound UDP datagrams from peers, and issues a multicast host join. If a Remote Endpoint Identifier with an address is supplied, the join is Source-Specific Multicast, and the path selection is based on the route to the Remote Endpoint. If a Remote Endpoint Identifier is not supplied, the join is Any-Source Multicast, and the path selection is based on the outbound route to the group supplied in the Local Endpoint.¶

There are cases where it is required to open multiple connections for the same address(es). For example, one Connection might be opened for a multicast group to for a multicast control bus, and another application later opens a separate Connection to the same group to send signals to and/or receive signals from the common bus. In such cases, the Transport Services system needs to explicitly enable reuse of the same set of addresses (equivalent to setting SO_REUSEADDR in the Socket API).¶

ConnectionReceived:

UDP Multicast Receive Listeners will deliver new Connections once they have received traffic from a new Remote Endpoint.¶

Clone:

Calling Clone on a UDP Multicast Receive Connection creates a new Connection with equivalent parameters. The two Connections are otherwise independent.¶

Send:

SEND.UDP. Calling Send on a UDP Multicast Receive connection causes an immediate SendError. This is an unsupported operation.¶

Receive:

RECEIVE.UDP. The Receive operation in a UDP Multicast Receive connection only delivers complete Messages to Received, each of which represents a single datagram received in a UDP packet. Upon receiving a UDP datagram, the ECN flag from the IP header can be obtained (GET_ECN.UDP).¶

Close:

Calling Close on a UDP Multicast Receive Connection (ABORT.UDP) releases the local port reservation and leaves the group. The Connection then issues a Closed event.¶

Abort:

Calling Abort on a UDP Multicast Receive Connection (ABORT.UDP) is identical to calling Close except that the Connection will send a ConnectionError event rather than a Closed event.¶

CloseGroup:

Calling CloseGroup on a UDP Multicast Receive Connection (ABORT.UDP) is identical to calling Close on this Connection and on all Connections in the same ConnectionGroup.¶

AbortGroup:

Calling AbortGroup on a UDP Multicast Receive Connection (ABORT.UDP) is identical to calling Close on this Connection and on all Connections in the same ConnectionGroup.¶

10.6. SCTP

Connectedness:

Connected¶

Data Unit:

Message¶

Connection Object:

Connection objects can be mapped to an SCTP association or a stream in an SCTP association. Mapping Connection objects to SCTP streams is called "stream mapping" and has additional requirements as follows. The following explanation assumes a client-server communication model.¶

Stream mapping requires an association to already be in place between the client and the server, and it requires the server to understand that a new incoming stream should be represented as a new Connection object by the Transport Services system. A new SCTP stream is created by sending an SCTP message with a new stream id. Thus, to implement stream mapping, the Transport Services API must provide a newly created Connection object to the application upon the reception of such a message. The necessary semantics to implement a Transport Services system's Close and Abort primitives are provided by the stream reconfiguration (reset) procedure described in [RFC6525]. This also allows a stream id to be reused after resetting ("closing") the stream. To implement this functionality, SCTP stream reconfiguration [RFC6525] must be supported by both the client and the server side.¶

To avoid head-of-line blocking, stream mapping should only be implemented when both sides support message interleaving [RFC8260]. This allows a sender to schedule transmissions between multiple streams without risking that transmission of a large message on one stream might block transmissions on other streams for a long time.¶

To avoid conflicts between stream ids, the following procedure is recommended: the first Connection, for which the SCTP association has been created, must always use stream id zero. All additional Connections are assigned to unused stream ids in growing order. To avoid a conflict when both endpoints map new Connections simultaneously, the peer that initiated association must use even stream ids whereas the remote side must map its Connections to odd stream ids. Both sides maintain a status map of the assigned stream ids. Generally, new streams should consume the lowest available (even or odd, depending on the side) stream id; this rule is relevant when lower ids become available because Connection objects associated with the streams are closed.¶

SCTP stream mapping as described here has been implemented in a research prototype; a description of this implementation is given in [NEAT-flow-mapping].¶

Initiate:

If this is the only Connection object that is assigned to the SCTP Association or stream mapping is not used, CONNECT.SCTP is called. Else, unless the Selection Property activeReadBeforeSend is Preferred or Required, a new stream is used: if there are enough streams available, Initiate is a local operation that assigns a new stream id to the Connection object. The number of streams is negotiated as a parameter of the prior CONNECT.SCTP call, and it represents a trade-off between local resource usage and the number of Connection objects that can be mapped without requiring a reconfiguration signal. When running out of streams, ADD_STREAM.SCTP must be called.¶

InitiateWithSend:

If this is the only Connection object that is assigned to the SCTP association or stream mapping is not used, CONNECT.SCTP is called with the "user message" parameter. Else, a new stream is used (see Initiate for how to handle running out of streams), and this just sends the first message on a new stream.¶

Ready:

Initiate or InitiateWithSend returns without an error, i.e., SCTP's four-way handshake has completed. If an association with the peer already exists, stream mapping is used, and enough streams are available, a Connection object instantly becomes Ready after calling Initiate or InitiateWithSend.¶

EstablishmentError:

Failure of CONNECT.SCTP.¶

ConnectionError:

TIMEOUT.SCTP or ABORT-EVENT.SCTP.¶

Listen:

LISTEN.SCTP. If an association with the peer already exists and stream mapping is used, Listen just expects to receive a new message with a new stream id (chosen in accordance with the stream id assignment procedure described above).¶

ConnectionReceived:

LISTEN.SCTP returns without an error (a result of successful CONNECT.SCTP from the peer) or, in the case of stream mapping, the first message has arrived on a new stream (in this case, Receive is also invoked).¶

Clone:

Calling Clone on an SCTP association creates a new Connection object and assigns it a new stream id in accordance with the stream id assignment procedure described above. If there are not enough streams available, ADD_STREAM.SCTP must be called.¶

Send:

SEND.SCTP. Message Properties such as msgLifetime and msgOrdered map to parameters of this primitive.¶

Receive:

RECEIVE.SCTP. The "partial flag" of RECEIVE.SCTP invokes a ReceivedPartial event.¶

Close:

If this is the only Connection object that is assigned to the SCTP association, CLOSE.SCTP is called and the Closed event will be delivered to the application upon the ensuing CLOSE-EVENT.SCTP. Else, the Connection object is one out of several Connection objects that are assigned to the same SCTP association, and RESET_STREAM.SCTP must be called, which informs the peer that the stream will no longer be used for mapping and can be used by future Initiate, InitiateWithSend, or Listen calls. At the peer, the event RESET_STREAM-EVENT.SCTP will fire, which the peer must answer by issuing RESET_STREAM.SCTP too. The resulting local RESET_STREAM-EVENT.SCTP informs the Transport Services system that the stream id can now be reused by the next Initiate, InitiateWithSend, or Listen calls, and invokes a Closed event toward the application.¶

Abort:

If this is the only Connection object that is assigned to the SCTP association, ABORT.SCTP is called. Else, the Connection object is one out of several Connection objects that are assigned to the same SCTP association, and shutdown proceeds as described under Close.¶

CloseGroup:

Calling CloseGroup calls CLOSE.SCTP, which closes all Connections in the SCTP association.¶

AbortGroup:

Calling AbortGroup calls ABORT.SCTP, which immediately closes all Connections in the SCTP association.¶

In addition to the API mappings described above, when there are multiple Connection objects assigned to the same SCTP association, SCTP can support Connection properties such as connPriority and connScheduler where CONFIGURE_STREAM_SCHEDULER.SCTP can be called to adjust the priorities of streams in the SCTP association.¶

12.1. Considerations for Candidate Gathering

The Security Considerations of the Transport Services architecture [RFC9621] forbids gathering and racing with Protocol Stacks that do not have equivalent security properties. Therefore, implementations need to avoid downgrade attacks that allow network interference to cause the implementation to select less secure, or entirely insecure, combinations of paths and protocols.¶

12.2. Considerations for Candidate Racing

See Section 5.3 for security considerations around racing with 0-RTT data.¶

An attacker that knows a particular device is racing several options during connection establishment may be able to block packets for the first connection attempt, thus inducing the device to fall back to a secondary attempt. This is a problem if the secondary attempts have worse security properties that enable further attacks. Implementations should ensure that all options have equivalent security properties to avoid incentivizing attacks.¶

Since results from the network can determine how a connection attempt tree is built, such as when DNS returns a list of resolved endpoints, it is possible for the network to cause an implementation to consume significant on-device resources. Implementations should limit the maximum amount of state allowed for any given node, including the number of child nodes, especially when the state is based on results from the network.¶