Internet Engineering Task Force (IETF) M. Kühlewind
Request for Comments: 9400 Ericsson
Category: Informational M. Duke
ISSN: 2070-1721 Google
April
June 2023
Guidelines for the Organization of Fully Online Meetings
Abstract
This document provides guidelines for the planning and organization
of fully online meetings, regarding the number, length, and
composition of sessions on the meeting agenda. These guidelines are
based on the experience gained by holding online meetings during the
COVID-19 pandemic in 2020 and 2021.
Status of This Memo
This document is not an Internet Standards Track specification; it is
published for informational purposes.
This document is a product of the Internet Engineering Task Force
(IETF). It represents the consensus of the IETF community. It has
received public review and has been approved for publication by the
Internet Engineering Steering Group (IESG). Not all documents
approved by the IESG are candidates for any level of Internet
Standard; see Section 2 of RFC 7841.
Information about the current status of this document, any errata,
and how to provide feedback on it may be obtained at
https://www.rfc-editor.org/info/rfc9400.
Copyright Notice
Copyright (c) 2023 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Revised BSD License text as described in Section 4.e of the
Trust Legal Provisions and are provided without warranty as described
in the Revised BSD License.
Table of Contents
1. Introduction
1.1. Requirements Language
2. Some History
3. Guidelines for Online Meeting Planning
3.1. Time Zone Selection
3.1.1. Guidelines for Selection
3.2. Number of Days and Total Hours per Day
3.3. Session/Break Length
3.4. Number of Parallel Tracks
4. Additional Considerations and Recommendations
4.1. Full vs. Limited Agenda (and Interim Meetings)
4.2. Flexibility of Time Usage
4.3. Inclusivity and Socializing
4.4. Experiments
4.5. IANA Considerations
4.6. Security Considerations
5. References
5.1. Normative References
5.2. Informative References
Acknowledgments
Authors' Addresses
1. Introduction
In 2020, the COVID-19 pandemic forced the IETF to convert all its
plenary meetings to online-only events. This document records the
experience gained by holding plenary meetings fully online and
proposes guidelines based on this experience. In general,
participant surveys indicated satisfaction with the organization of
these meetings.
Although these guidelines reflect lessons learned in 2020 and 2021,
the IETF is encouraged to continue to experiment with the format and
agenda of fully online meetings, using this document as a baseline.
Hybrid meetings (meaning meetings that have large remote
participation but also onsite participation) are out of scope.
However, some of the experience gained from fully online meetings
might also provide input for decisions regarding the organization of
hybrid meetings.
1.1. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in
BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all
capitals, as shown here.
This document uses the term "plenary meeting" for the whole IETF
meeting that covers the IETF meeting week; this term is used to
distinguish the plenary meeting from other IETF meetings like
"interim meetings". The term "administrative plenary" is used for
the respective session during the IETF meeting week that is usually
hosted on Wednesday.
2. Some History
When the World Health Organization (WHO) declared a worldwide
pandemic in March 2020, the IETF canceled its plenary meeting and
organized an online replacement in less than 2 weeks. For this first
online-only meeting, the agenda was reduced to a set of sessions that
benefited most from cross-area participation, like BoFs, first-time
meetings of new working groups, and dispatch sessions. It also
included the administrative plenary to preserve the official handover
procedures that occur at March IETF meetings, as described in
[RFC8713].
With a reduced agenda, the meeting format was two sessions (about 4
hours) per day with a maximum of 2 two parallel tracks. Other working
group meetings were scheduled as interims over the following 6 weeks.
The IESG published a purely advisory recommended schedule
[INTERIM-SCHEDULE] to reduce conflicts among those interims.
While satisfaction was high right after the meeting
[IETF107-FEEDBACK], some participants later indicated in mailing list
discussions that the period of intensive interims had a greater
impact on their calendar than a single plenary meeting week, and in
some meetings participation was reduced. Those interims tended to
occur at times convenient for the bulk of participants, which was
convenient for most but could exclude those in less common time
zones.
For the remainder of 2020 and 2021, the online schedule was switched
back to be similar to an in-person meeting (1- to 2-hour slots and
8-9
eight or nine parallel tracks). However, each day was limited to 5-6
hours in recognition that remote participation is more tiring.
All fully online meetings followed the time zone of the planned in-
person meeting location. As a 6-hour agenda has some flexibility
regarding the start time while still fitting within a previously used
8-hour in-person agenda, the start time was approximately noon, with
adjustments of an hour or so to mitigate the impact of early morning
hours in time zones with many participants. As selection of in-
person meeting sites was consistent with the 1-1-1 guideline as
documented in [RFC8719], this approach was intended to share the
burden across all common geographies roughly equally.
3. Guidelines for Online Meeting Planning
3.1. Time Zone Selection
The following algorithm was not used in 2020 or 2021, but it enables
most participants to avoid late-night sessions in two out of every
three fully online IETF plenary meetings. Basically, every fully
online meeting is for two regions of the three regions described in
[RFC8719], with one being roughly after sunrise and the other around
sundown. This has the trade-off that the third region is in the
middle of night.
The times are also seasonally adjusted to leverage differentials in
Daylight Saving Time. These time slots are as follows, in UTC, based
on the Daylight Saving Practices at the time of publication:
+===============+=========================+=========================+
| Name | Times (Northern Summer) | Times (Northern |
| | | Winter) |
+===============+=========================+=========================+
| North America | 0500-1100 UTC | 0600-1200 UTC |
| Night | | |
+---------------+-------------------------+-------------------------+
| Asia Night | 1300-1900 UTC | 1400-2000 UTC |
+---------------+-------------------------+-------------------------+
| Europe Night | 2200-0400 UTC | 2200-0400 UTC |
+---------------+-------------------------+-------------------------+
Table 1
Note that the "Europe Night" slot covers the "early morning" slot for
Asia where most countries do not have Daylight Saving Time.
If Daylight Saving Practices change -- this change is under
consideration in multiple countries at the time of publication --
this table may need adjustment.
The intent of rotating between these three slots is to scatter
meetings throughout the course of the global day, to maximize the
ease of participants so that no attendee has to be consistently
inconvenienced, regardless of their location and what time of day is
optimal for their schedule. However, as participation is distributed
globally, it needs to be acknowledged that restricting the scheme to
three regions observes the intent of [RFC8719] but does not achieve
the goal of two non-late-night sessions for all participants equally.
3.1.1. Guidelines for Selection
The IETF SHOULD select a start time from these three choices based on
the prior three meetings. The following table covers all
permutations of previous meetings held in person in Region A, B, or C
or remotely in the nights of one of those regions.
+====================+==================+==============+===========+
| Three Meetings Ago | Two Meetings Ago | Last Meeting | Online |
| | | | Selection |
+====================+==================+==============+===========+
| Any | Any | In-Person A | A Night |
+--------------------+------------------+--------------+-----------+
| Any | Online A Night | Online B | C Night |
| | | Night | |
+--------------------+------------------+--------------+-----------+
| Online A Night | In-Person B | Online B | C Night |
| | | Night | |
+--------------------+------------------+--------------+-----------+
| In-Person A | In-Person B | Online B | A Night |
| | | Night | |
+--------------------+------------------+--------------+-----------+
| In-Person A | In-Person A | Online A | See below |
| | | Night | |
+--------------------+------------------+--------------+-----------+
| Online A Night | Online B Night | Online C | A Night |
| | | Night | |
+--------------------+------------------+--------------+-----------+
Table 2
This table follows two basic guidelines:
1) Whenever a fully online meeting follows an in-person meeting, the
online meeting time is used that most disadvantages most the
participants in the time zone where the in-person meeting was
held.
2) If multiple fully online meetings follow each other, the time
zone selection should be rotated based on the most recent time
zones in which the in-person meetings were held.
The final case occurs in the rare event that back-to-back in-person
plenary meetings occur in the same region. In this case, find the
most recent meeting that was in neither 'A' (if in person) nor 'A
Night' (if fully online). If this meeting was in person in region
'B', then the next meeting should be in 'B Night'. If it was remote
in 'B Night', the next meeting should be in 'C Night'.
3.2. Number of Days and Total Hours per Day
By 2021, fully online meetings were consistently held over 5 days
with roughly 6-hour meeting days. The day with the administrative
plenary, which concludes with multiple open mic sessions, sometimes
exceeded this limit.
Six hours of online meetings, with two 30-minute breaks, was a
compromise between the physical limits of attending an online meeting
in an inconvenient time zone and the demand for many sessions with a
manageable number of conflicts. The IETF 109 feedback
[IETF109-SURVEY] indicated broad satisfaction with a 5-day meeting
but only medium satisfaction with the overall length of each day.
The IETF did not seriously consider extending sessions into the
weekend before or after the main meeting week, although at IETF 108
and subsequent meetings the Hackathon occupied the entire week before
(see [RFC9311]).
3.3. Session/Break Length
For fully online meetings, there are typically fewer sessions per day
than for in-person meetings, to keep the overall meeting day to
roughly 6 hours. With fewer sessions, chairs were offered only two
options for session length (instead of three).
IETF 108, based on an indicated preference of the community,
scheduled 50- and 100-minute slots, with 10-minute breaks, in order
to keep the overall day length at 5 hours. This resulted in many
sessions going over time, which indicated that 10 minutes for breaks
is not practical.
The survey after IETF 109 [IETF109-SURVEY] showed high satisfaction
with 60/120-minute session lengths and 30-minute breaks, and a
significant improvement in satisfaction over IETF 108.
The longer breaks, while extending the day, provided adequate time
for meals, exercise, and "hallway" conversations using online tools.
3.4. Number of Parallel Tracks
In-person meetings are limited in the number of parallel tracks by
the number of meeting rooms, but online meetings are not. However,
more parallel tracks would increase the number of possible agenda
conflicts.
If the total number of requested sessions exceeds the capacity of the
usual 8 eight parallel tracks, it is possible for a fully online
meeting to simply use more tracks. If the number and length of
meeting days are seen as fixed, this decision is implicitly made by
the working group chairs requesting a certain number of sessions and
length.
IETF 111 used 9 nine parallel tracks for some of the sessions and
experienced slightly more conflicts in the agenda-scheduling process,
though there was no statistically significant increase in
dissatisfaction about conflicts in the survey [IETF111-SURVEY].
The IESG encouraged working group chairs to limit their session
requests and use interim meetings aggressively for focused work.
4. Additional Considerations and Recommendations
4.1. Full vs. Limited Agenda (and Interim Meetings)
The IETF 108 meeting survey [IETF108-SURVEY] asked about the
structure of that meeting (full meeting) compared to that of IETF
107, which hosted only a limited set of sessions followed by interims
in the weeks after. The structure of IETF 108 was preferred by 82%.
Respondents valued cross-participation and an intensive meeting week
for maintaining project momentum.
Furthermore, a well-defined meeting time, rather than spreading many
interims over the whole year, can make deconflicting with other non-
IETF meetings easier.
However, interim meetings can also help to reduce scheduling
conflicts during an IETF week and allow for a more optimal time slot
for the key participants. While interim meetings are less likely to
attract people with casual interest, they provide a good opportunity
for the most active participants of a group to have detailed
technical discussions and solve recorded issues efficiently.
4.2. Flexibility of Time Usage
This document recommends further experiments with reducing conflicts
by leveraging the increased flexibility of the online format.
An in-person meeting must fit all sessions into an acceptable length
for international travel (usually roughly a week), but online
meetings do not have that constraint.
Therefore, it would be possible to keep most regular working group
sessions within the usual 5 main meeting days but have some of the
more conflicted sessions in other dedicated time slots. As the
Hackathon for fully online meetings is usually held in the week
before the online plenary meeting [RFC9311], that week is already a
highly active week for many IETF participants and might provide an
opportunity to schedule a few selected sessions.
This might work especially well for sessions that are of high
interest to a large part of the community, such as BoFs and dispatch
meetings, and therefore hard to schedule during the main IETF week.
At IETF 112, the IESG ran an experiment where the administrative
plenary was scheduled on the Wednesday before the official session
week. The experiment report [IETF112-EXPERIMENT] found that it led
to a reduction in scheduling conflicts but also a slight drop in
attendance of the administrative plenary, partly due to insufficient
awareness.
4.3. Inclusivity and Socializing
Participation in the fully online meetings in 2021 was high and had a
stable per-country distribution, even though time zones were rotated.
This indicates that online meetings support a more consistent
geographic distribution of participants than in-person meetings,
where participation often fluctuates based on the location.
However, online meetings do not provide an equivalent opportunity to
socialize. Despite significant investment in tools to foster hallway
conversations, many did not use those tools, whether due to ignorance
of them, dislike of the tools, or a preference for other activities
at home (including sleep and food) over hallway interactions.
There was a decrease in submissions of new (-00) Internet-Drafts
during 2020 and 2021, although the overall number of draft
submissions remained stable, which stable; this decrease in new submissions might result
have resulted from the loss of these interactions. Informal
conversations might be important to inspire new work.
4.4. Experiments
This document recommends further experiments with the meeting
structure. Often, only practical experience can answer open
questions. A given meeting SHOULD only experiment with one major
change at a time in order to be able to assess the outcome correctly.
Furthermore, the IESG SHOULD announce any such experiment well in
advance, so people can adjust to changes and potentially provide
feedback.
4.5. IANA Considerations
This document has no IANA actions.
4.6. Security Considerations
This document has no security considerations.
5. References
5.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>.
[RFC8719] Krishnan, S., "High-Level Guidance for the Meeting Policy
of the IETF", BCP 226, RFC 8719, DOI 10.17487/RFC8719,
February 2020, <https://www.rfc-editor.org/info/rfc8719>.
5.2. Informative References
[IETF107-FEEDBACK]
Daley, J., "IETF 107 Virtual Meeting Survey", 17 April
2020, <https://www.ietf.org/media/documents/ietf-107-
survey-results.pdf>.
[IETF108-SURVEY]
Daley, J., "IETF 108 Meeting Survey", 13 August 2020,
<https://www.ietf.org/blog/ietf-108-meeting-survey/>.
[IETF109-SURVEY]
Daley, J., "IETF 109 Post-Meeting Survey", 7 December
2020,
<https://www.ietf.org/blog/ietf-109-post-meeting-survey/>.
[IETF111-SURVEY]
Daley, J., "IETF 111 post-meeting survey", 22 23 August 2021,
<https://www.ietf.org/blog/ietf-111-post-meeting-survey/>.
[IETF112-EXPERIMENT]
IESG, "IETF 112 Plenary Experiment Evaluation", 4 February
2022, <https://www.ietf.org/blog/ietf112-plenary-
experiment-evaluation/>.
[INTERIM-SCHEDULE]
Cooper, A., "Subject: Post-IETF-107 Recommended Virtual
Interim Schedule", message to the Working Group Chairs
mailing list, 13 March 2020,
<https://mailarchive.ietf.org/arch/msg/wgchairs/
l382SqKVVHoTzFw9kIYl2boM6_c/>.
[RFC8713] Kucherawy, M., Ed., Hinden, R., Ed., and J. Livingood,
Ed., "IAB, IESG, IETF Trust, and IETF LLC Selection,
Confirmation, and Recall Process: Operation of the IETF
Nominating and Recall Committees", BCP 10, RFC 8713,
DOI 10.17487/RFC8713, February 2020,
<https://www.rfc-editor.org/info/rfc8713>.
[RFC9311] Eckel, C., "Running an IETF Hackathon", RFC 9311,
DOI 10.17487/RFC9311, September 2022,
<https://www.rfc-editor.org/info/rfc9311>.
Acknowledgments
Thanks to Brian Carpenter, Lars Eggert, Toerless Eckert, Charles
Eckel, Jason Livingood, Sanjeev Gupta, Dale Worley, and Mark
Nottingham for their reviews reviews, and thanks to the many from more for their other people who
provided input and suggestions on the time zone discussion!
Authors' Addresses
Mirja Kühlewind
Ericsson
Email: mirja.kuehlewind@ericsson.com
Martin Duke
Google
Email: martin.h.duke@gmail.com