<?xmlversion="1.0" encoding="UTF-8"?> <?rfc sortrefs="yes"?> <?rfc subcompact="no"?> <?rfc symrefs="yes"?> <?rfc toc="yes"?> <?rfc tocdepth="3"?> <?rfc compact="yes"?> <?rfc subcompact="no"?>version='1.0' encoding='UTF-8'?> <!DOCTYPE rfc [ <!ENTITY nbsp " "> <!ENTITY zwsp "​"> <!ENTITY nbhy "‑"> <!ENTITY wj "⁠"> ]> <rfc xmlns:xi="http://www.w3.org/2001/XInclude" category="std" docName="draft-ietf-idr-bgp-sendholdtimer-15" number="9687" updates="4271" obsoletes="" ipr="trust200902" submissionType="IETF"consensus="true">consensus="true" sortRefs="true" symRefs="true" tocInclude="true" tocDepth="3" version="3" xml:lang="en"> <front> <title abbrev="BGP SendHoldTimer">Border Gateway Protocol 4 (BGP-4) Send Hold Timer</title> <seriesInfo name="RFC" value="9687"/> <author fullname="Job Snijders" initials="J." surname="Snijders"> <organization>Fastly</organization> <address> <postal><street /><city>Amsterdam</city><code /><country>Netherlands</country> </postal> <email>job@fastly.com</email> </address> </author> <author fullname="Ben Cartwright-Cox" initials="B." surname="Cartwright-Cox"> <organization abbrev="Port 179">Port 179 Ltd</organization> <address> <postal><street /><city>London</city><code /><country>United Kingdom</country> </postal> <email>ben@benjojo.co.uk</email> </address> </author> <author fullname="Yingzhen Qu" initials="Y" surname="Qu"> <organization abbrev="Futurewei">Futurewei Technologies</organization> <address> <postal><street /> <city>Santa Clara</city> <code /><city>San Jose</city> <region>CA</region> <code>95131</code> <country>UnitedStates</country>States of America</country> </postal> <email>yingzhen.ietf@gmail.com</email> </address> </author> <date/> <area>Routing</area> <workgroup>IDR</workgroup>month="November" year="2024"/> <area>RTG</area> <workgroup>idr</workgroup> <keyword>BGP</keyword> <keyword>TCP</keyword> <abstract> <t> This document defines theSendHoldtimer,<tt>SendHoldTimer</tt>, along with theSendHoldTimer_Expires<tt>SendHoldTimer_Expires</tt> event, for the Border Gateway Protocol (BGP) Finite State Machine (FSM). Implementation of theSendHoldTimer<tt>SendHoldTimer</tt> helps overcome situations where a BGP connection is not terminated after the local system detects that the remote system is not processing BGP messages. This document specifies that the local system should close the BGP connection and not solely rely on the remote system for connection closure when theSendHoldTimer<tt>SendHoldTimer</tt> expires. This document updatesRFC4271.RFC 4271. </t> </abstract><note title="Requirements Language"> <t> The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 <xref target="RFC2119"/> <xref target="RFC8174"/> when, and only when, they appear in all capitals, as shown here. </t> </note></front> <middle><section title="Introduction"><section> <name>Introduction</name> <t> This document defines the<tt>SendHoldtimer</tt>,<tt>SendHoldTimer</tt>, along with the <tt>SendHoldTimer_Expires</tt> event, for the Border Gateway Protocol (BGP)<xref target="RFC4271" />Finite State Machine (FSM) defined insection 8.<xref target="RFC4271" section="8"/>. </t> <t> Failure to terminate a blocked BGP connection can result in network reachability issues, and the subsequent failure to generate and deliver BGP UPDATE messages to another BGP speaker of the local system is detrimental to all participants of the inter-domain routing system. This phenomena is thought to have contributed to IP trafficblackholingpacket loss events in the global Internet routing system <xref target="bgpzombies"/>. </t> <t> This specification intends to improve this situation by requiring that BGP connections be terminated if the local system has detected that the remote system cannot possibly have processed any BGP messages for the duration of the <tt>SendHoldTime</tt>. Through standardization of the aforementioned requirement, operators will benefit from consistent behavior across different BGP implementations. </t> <t> BGP speakers following this specification do not rely exclusively on remote systems closing blockedconnections, but willconnections; they also locally close blocked connections. </t> </section><section title="Example<section> <name>Requirements Language</name> <t> The key words "<bcp14>MUST</bcp14>", "<bcp14>MUST NOT</bcp14>", "<bcp14>REQUIRED</bcp14>", "<bcp14>SHALL</bcp14>", "<bcp14>SHALL NOT</bcp14>", "<bcp14>SHOULD</bcp14>", "<bcp14>SHOULD NOT</bcp14>", "<bcp14>RECOMMENDED</bcp14>", "<bcp14>NOT RECOMMENDED</bcp14>", "<bcp14>MAY</bcp14>", and "<bcp14>OPTIONAL</bcp14>" in this document are to be interpreted as described in BCP 14 <xref target="RFC2119"/> <xref target="RFC8174"/> when, and only when, they appear in all capitals, as shown here. </t> </section> <section> <name>Example of aproblematic scenario">Problematic Scenario</name> <t> In implementations lacking the concept of a <tt>SendHoldTimer</tt>, a malfunctioning or overwhelmed remote speaker may cause data on the BGP socket in the local system to accumulate ad infinitum. This could result in forwarding failure and traffic loss, as the overwhelmed speaker continues to utilize stale routes. </t> <t> An example fault state: as BGP runs over TCP <xreftarget="RFC9293" />,target="RFC9293"/>, it is possible for a BGP speaker in the Established state to encounter a BGP speaker that is advertising a TCP Receive Window (RCV.WND) of size zero.This 0The size zero of this window prevents the local system from sending KEEPALIVE, UPDATE, or any other critical BGP messages across the network socket to the remote speaker. </t> <t> Generally BGP implementations have no visibility into lower-layer subsystems such as TCP or the speaker's current Receive Window size, and there is no existing BGP mechanism for such a blocked connection to be recognized. Hence BGP implementations are not able to handle this situation in a consistent fashion. </t> <t> Themajorprimary issuearising fromthat arises when a BGP speakerbeingis unable to send a BGP message to agivenremote speaker is thatas a result thatthe affected speakersubsequently ismay end up operatingbased on stalewith outdated routing information. Failure of the BGP speaker to send (and thus the remote speaker to receive) BGP messages on a single BGP session can negatively impact the ability of an entire autonomous system (or even a group of autonomous systems) to converge. </t> </section><section title="Changes<section> <name>Changes to RFC 4271 -SendHoldTimer"><tt>SendHoldTimer</tt></name> <t> BGP speakers are implemented following a conceptual model "BGP Finite State Machine" (FSM), which is outlined insection 8 of<xreftarget="RFC4271"/>.target="RFC4271" section="8" sectionFormat="of"/>. This specification adds a BGP timer, <tt>SendHoldTimer</tt>, and updates the BGP FSM asfollows:indicated in the following subsections. </t><section title="Session Attributes"><section> <name>Session Attributes</name> <t> The following optional session attributes for each connection are added toSection 8, beforethe list in <xref target="RFC4271" section="8" sectionFormat="of"/> appearing just prior to "The optional session attributes support different features of the BGP functionality that have implications for the BGP FSM statetransitions": </t>transitions":</t> <t>NEW</t> <blockquote><list style="empty"><ul empty="true" spacing="normal"> <li> <t> 14)SendHoldTimer<tt>SendHoldTimer</tt> </t> </li> <li> <t> 15)SendHoldTime<tt>SendHoldTime</tt> </t></list></li> </ul> </blockquote> <t>The<tt>SendHoldTime</tt> determines how long a BGP speaker will stay in the Established state before the TCP connection is dropped because no BGP messages can be transmitted to its peer. A BGP speaker can configure the value of the <tt>SendHoldTime</tt> for each peer independently. </t> </section><section title="Timer<section> <name>Timer Event:SendHoldTimer_Expires"><tt>SendHoldTimer_Expires</tt></name> <t> Another timer event is added toSection 8.1.3 of<xreftarget="RFC4271"/>target="RFC4271" section="8.1.3" sectionFormat="of"/> asfollowing:follows: </t> <t>NEW</t> <blockquote><dl newline="true"> <dt><ul empty="true"> <li><t> Event 29:SendHoldTimer_Expires </dt> <dd> <dl> <dt>Definition:</dt> <dd>An<tt>SendHoldTimer_Expires</tt></t> <ul empty="true"> <li>Definition: An event generated when theSendHoldTimer expires.</dd> <dt>Status:</dt> <dd>Optional</dd> </dl> </dd> </dl><tt>SendHoldTimer</tt> expires.</li> <li>Status: Optional</li> </ul></li> </ul> </blockquote> </section><section title="Changes<section> <name>Changes to theFSM">FSM</name> <t>The following changes are made tosection 8.2.2 in<xreftarget="RFC4271"/>.</t>target="RFC4271" section="8.2.2" sectionFormat="of"/>.</t> <t>In "OpenConfirm State", the handling of Event 26 is revised as follows:</t> <t>OLD</t> <blockquote> <t> If the local system receives a KEEPALIVE message (KeepAliveMsg (Event 26)), the local system:<list style="hanging"> <t hangText="-">restarts the HoldTimer and</t> <t hangText="-">changes</t> <ul empty="true"><li><ul><li>restarts the <tt>HoldTimer</tt> and</li> <li>changes its state toEstablished.</t> </list> </t>Established.</li> </ul></li></ul> </blockquote> <t>NEW</t> <blockquote> <t> If the local system receives a KEEPALIVE message (KeepAliveMsg (Event 26)), the local system:<list style="hanging"> <t hangText="-">restarts the HoldTimer,</t> <t hangText="-">starts</t> <ul empty="true"><li><ul> <li>restarts the <tt>HoldTimer</tt>,</li> <li>starts theSendHoldTimer<tt>SendHoldTimer</tt> if theSendHoldTime<tt>SendHoldTime</tt> is non-zero,and</t> <t hangText="-">changesand</li> <li>changes its state toEstablished.</t> </list> </t>Established.</li> </ul></li></ul> </blockquote> <t> The following paragraph is added tosection 8.2.2<xref target="RFC4271" section="8.2.2" sectionFormat="of"/> in "Established State", after the paragraphwhichthat ends "unless the negotiatedHoldTime<tt>HoldTime</tt> value iszero.":zero": </t> <t>NEW</t> <blockquote><list style="empty"><t>If theSendHoldTimer_Expires<tt>SendHoldTimer_Expires</tt> (Event 29) occurs, the local system:<list style="hanging"> <t hangText="-">(optionally)</t> <ul empty="true"><li><ul> <li>(optionally) sends a NOTIFICATION message with the BGP Error Code "Send Hold Timer Expired" if the local system can determine that doing so will not delay the following actions in thisparagraph,</t> <t hangText="-">logsparagraph,</li> <li>logs an error message in the local system with the BGP Error Code "Send Hold TimerExpired",</t> <t hangText="-">releasesExpired",</li> <li>releases all BGPresources,</t> <t hangText="-">setsresources,</li> <li>sets the ConnectRetryTimer tozero,</t> <t hangText="-">dropszero,</li> <li>drops the TCPconnection,</t> <t hangText="-">incrementsconnection,</li> <li>increments the ConnectRetryCounter by1,</t> <t hangText="-">(optionally)1,</li> <li>(optionally) performs peer oscillation damping if the DampPeerOscillations attribute is set to TRUE,and</t> <t hangText="-">changesand</li> <li>changes its state toIdle.</t> </list> </t>Idle.</li> </ul> </li> <li> <t> Each time the local system sends a BGP message, it restarts theSendHoldTimer<tt>SendHoldTimer</tt> unless theSendHoldTime<tt>SendHoldTime</tt> value is zero or the negotiatedHoldTime<tt>HoldTime</tt> value is zero, in whichcasescase theSendHoldTimer<tt>SendHoldTimer</tt> is stopped. </t> </li> <li> <t> TheSendHoldTimer<tt>SendHoldTimer</tt> is stopped following any transition out of the Established state as part of the "release all BGP resources" action. </t></list></li> </ul> </blockquote> </section> <sectiontitle="Changesanchor="timers"> <name>Changes to BGPTimers" anchor="timers">Timers</name> <t> <xref target="RFC4271" section="10"/> summarizes BGPTimers.timers. This document adds another optional BGP timer: <tt>SendHoldTimer</tt>. </t> <t>NEW</t> <blockquote> <t>SendHoldTime<tt>SendHoldTime</tt> is an FSM attribute that stores the initial value for theSendHoldTimer.<tt>SendHoldTimer</tt>. IfSendHoldTime<tt>SendHoldTime</tt> isnon-zeronon-zero, then itMUST<bcp14>MUST</bcp14> be greater than the value ofHoldTime,<tt>HoldTime</tt>; see <xrefsection="5" target="I-D.ietf-idr-bgp-sendholdtimer"/>target="implcons" /> of [RFC9687] for suggested default values. </t> </blockquote> </section> </section><section title="Send<section> <name>Send Hold Timer Expired ErrorHandling">Handling</name> <t> If the local system does not send any BGP messages within the period specified in <tt>SendHoldTime</tt>, then a NOTIFICATION message with the "Send Hold Timer Expired" Error CodeMAY<bcp14>MAY</bcp14> be sent and the BGP connectionMUST<bcp14>MUST</bcp14> be closed. Additionally, an errorMUST<bcp14>MUST</bcp14> be logged in the local system, indicating theSend"Send Hold TimerExpiredExpired" Error Code. </t> </section> <sectiontitle="Implementation Considerations"anchor="implcons"> <name>Implementation Considerations</name> <t> Due to the relative rarity of the failure mode that this specification is designed to address, and also the fact that network operators may be unfamiliar with the formal specification of BGP fault detection mechanisms such as <tt>HoldTimer</tt>, it is likely that a large number of operatorsarewill be unaware of thenecessity ofneed for an additional mechanism such as<tt>SendHoldtimer</tt>.<tt>SendHoldTimer</tt>. </t> <t> Accordingly, it isRECOMMENDED<bcp14>RECOMMENDED</bcp14> that implementations of this specification enable<tt>SendHoldtimer</tt><tt>SendHoldTimer</tt> by default, without requiring additional configuration of theBGP speakingBGP-speaking device. </t> <t> The default value of <tt>SendHoldTime</tt> for a BGP connectionSHOULD<bcp14>SHOULD</bcp14> be the greater of: </t> <ul> <li>8minutes;minutes or</li> <li>2 times the negotiatedHoldTime</li><tt>HoldTime</tt></li> </ul></t><t> ImplementationsMAY<bcp14>MAY</bcp14> make the value of <tt>SendHoldTime</tt> configurable, either globally or on a per-peer basis, within the constraints set out in <xreftarget="timers"/> above.target="timers"/>. </t> <t> The subcode for NOTIFICATION message "Send Hold Timer Expired" is set to 0 and is notused,used; no additional data is to be appended to the end of a "Send Hold Timer Expired" NOTIFICATION message. </t> </section><section title="Operational Considerations"><section> <name>Operational Considerations</name> <t> When the local system recognizes that a remote speakerishas notprocessingprocessed any BGP messages for the duration of the <tt>SendHoldTime</tt>, it is likely that the local system will not be able to inform the remote peer through a NOTIFICATION message as to why the connection is being closed. Thisdocumentsdocument suggests that an attempt to send a NOTIFICATION message with the "Send Hold Timer Expired"error code isError Code still be made, if doing so will not delay closing the BGP connection.MeanwhileMeanwhile, an error message is loggedintoin the local system. </t> <t> Other mechanisms can be used as well, forexampleexample, BGP speakersSHOULD<bcp14>SHOULD</bcp14> provide this reason ("Send Hold Timer Expired") as part of their operationalstate; e.g.state (for example, bgpPeerLastError in the BGP MIB <xreftarget="RFC4273">BGP MIB</xref>.target="RFC4273"/>). </t> </section><section title="Security Considerations"><section> <name>Security Considerations</name> <t> This specification does not change BGP's security characteristics. Implementing the BGP <tt>SendHoldTimer</tt> as specified in this document will enhance network resilience by terminating connections with malfunctioning or overwhelmed remote peers. </t> </section><section title="IANA Considerations"><section> <name>IANA Considerations</name> <t> IANA has registeredcodevalue 8 for "Send Hold Timer Expired" in the "BGP Error (Notification) Codes" registryinwithin the "Border Gateway Protocol (BGP) Parameters" registry group. </t> </section><section title="Acknowledgements"><section> <name>Acknowledgements</name> <t> The authors would like to thankWilliam McCall, Theo<contact fullname="William McCall"/>, <contact fullname="Theo deRaadt, John Heasley, Nick Hilliard, Jeffrey Haas, Tom Petch, Susan Hares, Keyur Patel, Ben Maddison, Claudio Jeker,Raadt"/>, <contact fullname="John Heasley"/>, <contact fullname="Nick Hilliard"/>, <contact fullname="Jeffrey Haas"/>, <contact fullname="Tom Petch"/>, <contact fullname="Susan Hares"/>, <contact fullname="Keyur Patel"/>, <contact fullname="Ben Maddison"/>, <contact fullname="Claudio Jeker"/>, andJohn Scudder<contact fullname="John Scudder"/> for their helpful review of this document. </t> </section> </middle> <back><references title="Normative References"> <?rfc include="reference.RFC.2119.xml"?> <?rfc include="reference.RFC.4271.xml"?> <?rfc include="reference.RFC.8174.xml"?> <reference anchor="RFC9293" target="https://www.rfc-editor.org/info/rfc9293"> <front> <title>Transmission Control Protocol (TCP)</title> <author fullname="Wesley M. Eddy" initials="W." surname="Eddy" role="editor"/> <date month="August" year="2022"/> <abstract> <t>This document specifies the Transmission Control Protocol (TCP). TCP is an important transport-layer protocol in the Internet protocol stack, and it has continuously evolved over decades of use and growth of the Internet. Over this time, a number of changes have been made to TCP as it was specified in RFC 793, though these have only been documented in a piecemeal fashion. This document collects and brings those changes together with the protocol specification from RFC 793. This document obsoletes RFC 793, as well as RFCs 879, 2873, 6093, 6429, 6528, and 6691 that updated parts of RFC 793. It updates RFCs 1011 and 1122, and it should be considered as a replacement for the portions of those documents dealing with TCP requirements. It also updates RFC 5961 by adding a small clarification in reset handling while in the SYN-RECEIVED state. The TCP header control bits from RFC 793 have also been updated based on RFC 3168. </t> </abstract> </front> <seriesInfo name="RFC" value="9293"/> <seriesInfo name="DOI" value="10.17487/RFC9293"/> </reference> <!-- NOTE TO RFC-EDITOR: The below reference is intended as a 'self-reference', but unsure how to do that properly in xml2rfc See https://mailarchive.ietf.org/arch/msg/idr/sY-om-d3kZC1ypoBXAS17NCXtXk/ and https://mailarchive.ietf.org/arch/msg/idr/L6MinGVjHn94gRhCi2lGAQ9GoHw/ for discussion on the concern of carrying over a literal non-reference into the patched target document content --> <reference anchor="I-D.ietf-idr-bgp-sendholdtimer" target="https://datatracker.ietf.org/doc/html/draft-ietf-idr-bgp-sendholdtimer"> <front> <title>Border Gateway Protocol 4 (BGP-4) Send Hold Timer</title> <author fullname="Job Snijders" initials="J." surname="Snijders"> <organization>Fastly</organization> </author> <author fullname="Ben Cartwright-Cox" initials="B." surname="Cartwright-Cox"> <organization>Port 179 Ltd</organization> </author> <author fullname="Yingzhen Qu" initials="Y." surname="Qu"> <organization>Futurewei Technologies</organization> </author> <date /> <abstract> <t>This document defines the SendHoldtimer, along with the SendHoldTimer_Expires event, for the Border Gateway Protocol (BGP) Finite State Machine (FSM). Implementation of the SendHoldTimer helps overcome situations where a BGP connection is not terminated after the local system detects that the remote system is not processing BGP messages. This document specifies that the local system should close the BGP connection and not solely rely on the remote system for connection closure when the SendHoldTimer expires. This document updates RFC4271.</t> </abstract> </front> <seriesInfo name="Internet-Draft" value="draft-ietf-idr-bgp-sendholdtimer"/> </reference><references> <name>References</name> <references> <name>Normative References</name> <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.2119.xml"/> <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.4271.xml"/> <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8174.xml"/> <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.9293.xml"/> </references><references title="Informative References"> <reference anchor="openbgpd" target="https://marc.info/?l=openbsd-tech&m=160820754925261&w=2"> <front> <title>bgpd send side hold timer</title> <author fullname="Claudio Jeker"><organization>OpenBSD</organization></author> <date month="December" year="2020" /> </front> </reference> <reference anchor="frr" target="https://github.com/FRRouting/frr/pull/11225"> <front> <title>bgpd: implement SendHoldTimer</title> <author fullname="David Lamparter"><organization>NetDEF</organization></author> <date month="May" year="2022" /> </front> </reference> <reference anchor="neo-bgp" target="https://bgp.tools/kb/bgp-support"> <front> <title>What does bgp.tools support</title> <author fullname="Ben Cartwright-Cox"><organization>Port 179 Ltd</organization></author> <date month="Aug" year="2022" /> </front> </reference> <reference anchor="BIRD" target="https://gitlab.nic.cz/labs/bird/-/commit/bcf2327425d4dd96f381b87501cccf943bed606e"> <front> <title>BIRD Internet Routing Daemon</title> <author fullname=" Katerina Kubecova"><organization>CZ.NIC</organization></author> <date month="Oct" year="2023" /> </front> </reference><references> <name>Informative References</name> <reference anchor="bgpzombies" target="https://labs.ripe.net/author/romain_fontugne/bgp-zombies/"> <front> <title>BGP Zombies</title> <author fullname="RomainFontugne"><organization>IIJFontugne"> <organization>IIJ ResearchLab</organization></author>Lab</organization> </author> <datemonth="april" year="2019" />month="April" year="2019"/> </front> </reference><?rfc include="reference.RFC.4273.xml"?><xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.4273.xml"/> </references> </references><section title="Implementation status - RFC EDITOR: REMOVE BEFORE PUBLICATION"> <t> This section records the status of known implementations of the protocol defined by this specification at the time of posting of this Internet-Draft, and is based on a proposal described in RFC 7942. The description of implementations in this section is intended to assist the IETF in its decision processes in progressing drafts to RFCs. Please note that the listing of any individual implementation here does not imply endorsement by the IETF. Furthermore, no effort has been spent to verify the information presented here that was supplied by IETF contributors. This is not intended as, and must not be construed to be, a catalog of available implementations or their features. Readers are advised to note that other implementations may exist. </t> <t> According to RFC 7942, "this will allow reviewers and working groups to assign due consideration to documents that have the benefit of running code, which may serve as evidence of valuable experimentation and feedback that have made the implemented protocols more mature. It is up to the individual working groups to use this information as they see fit". </t> <t> <list style="symbols"> <t> OpenBGPD <xref target="openbgpd"/> </t> </list> <list style="symbols"> <t> FRRouting <xref target="frr"/> </t> </list> <list style="symbols"> <t> neo-bgp (bgp.tools) <xref target="neo-bgp"/> </t> </list> <list style="symbols"> <t> BIRD <xref target="BIRD"/> </t> </list> </t> <t> Patches to recognize error code 8 were merged into OpenBSD's and the-tcpdump-group's tcpdump implementations. </t> </section></back> </rfc>