Writing I-Ds using HTML Comodo Group Inc.philliph@comodo.com
General
RFCRequest For CommentsThis memo presents a technique for using as subset HTML as a source format for documents in the Internet-Drafts (I-Ds) and Request for Comments (RFC) series. The HTML subset provides equivalent functionality to the traditional XML2RFC tool but is designed for ease of use with readily available editing tools.Extensions to the HTML markup allow the use of external bibliography files to manage citations and references in the manner of BibTeX. Planned extensions include building concordances of defined terms and normative language.One of the more difficult aspects of working in the IETF is producing Internet Drafts that comply with the IETF formating criteria originally developed in the age of valve tube machines.HTML2RFC is a HTML microformat for Internet Drafts (and RFCs). Documents written in the HTML2RFC microformat may be converted to a variety of target formats that currently include Internet Draft Plaintext [RFC2223], XML2RFC [RFC2629] and HTML in the IETF presentation stylesheet.While existing tools enable the generation of documents in Internet Draft format, none provide the features necessary to allow the document editor to focus on the task of editing text rather than dealling with the distraction of producing text in the obligated format.One area in which all recent document formats fall short is in the management of references in citations. While the online citation libraries introduced in XML2RFC are exceptionally useful, accessing them though XML entities or cut-and-paste of the citation sources is tedious and error prone. Patashnik and Lamport demonstrated a much superior system in 1985 in BibTeX.[BibTeX].[RFC2629] which describes the XML2RFC tool was published in 1999, a time when XML was the document format of the future. Although advanced XML editing tools were expensive with poor usuability, this deficiency was expected to be resolved in the very near future. [RFC2629] did not make editing Internet Drafts easy but it was better than the available alternatives and made it possible to produce documents in both the plaintext format required for Internet Draft subissions and readable formats such as HTML.Almost fifteen years later, XML document authoring tools are still expensive and none of the many tools the author has tried offers more than a modest improvement over editing XML source by hand. The tools merely make it a little easier to enter XML tags. The author is still required to remember the many idiosyncracies of the XML2RFC DTD which uses a mixture of case conventions (hangText vs texttable) and oddly chosen tag and attribute names. The only consistency in the design of the DTD is that it follows none of the conventions established in HTML. Even the paragraph tag is inexplicably labelled <t>.Another area in which use of [RFC2629] is unsatisfactory is in the handling of citatitions and references. While the online library of references is certainly very useful, making use of it in a document is fiddly and time consuming. Citations must be managed in three separate parts of the document. Entity declarations linking to the external reference declarations are inserted at the top of the file while the citation is added to the end of the file. To create a reference to the citation in the body of the text we must create a crosslink to the anchor defined inside the external reference which may or may not be the same as the entity name defining the reference.The method of handling references makes it difficult to prepare documents in offline environments (e.g. a plane) and can cause editing tools that attempt DTD validation to behave slowly or unpredictably as attempts are made to retrieve the external content while the file is being edited.One of the principal reasons that HTML has become the most widely used document markup format is that the basic structure markup is very simple and can be learned in a few minutes. Virtually every modern text editor offers at least the ability to generate HTML as an output markup. A wide range of HTML editing tools is available for every modern computing platform. Many of the best tools are free.While entering HTML tags is easy, editing or reviewing text that is cluttered with markup tags is unnecessarily tedious and error prone. Editing a document describing a XML protocol in raw XML source form is particularly error prone as every occurrence of an XML markup character must be escaped, often rendering the text almost unreadable. Many HTML editors allow the editor to view source and formatted views of their document side by side or quickly switch from one to the other. This is much more efficient than trying to edit XML that is converted into readable text by a stylesheet.The original HTML 2.0 markup described in [RFC1866] was intended to provide a means of describing the document structure. The most recent edition, HTML 5 [W3C-CR-html5-20121217] is considerably more complicated, offering a melange of structure, format, style and scripting features. Fortunately almost all the requirements of HTML2RFC are met by features of the original HTML 2.0 markup that have not been updated or obsoleted since.One of the key innovations introduced in [BibTeX] was the ability to separate the tasks of document editing and bibliography management. Once compiled, a bibliography could be shared between multiple documents or even multiple users. It is not uncommon for a good bibliography file to be shared by whole university departments.The reference management feature of xml2rfc also provides considerable advantages over previous approaches. The online library of reference descriptions is maintained by automatic tools eliminating the need to enter most entries manually. While the xml.resoure.org repository hardly exhaustive it is rare that an Internet Draft contains more than one or two citations that are outside its scope.The major drawback to the xml2rfc approach is that the online tool fails completely when editing documents in offline mode. Authors can maintain a local cache of references they use in their own documents but they must develop the necessary tools to do so themselves.HTML2RFC combines the advantages of the BibTeX and xml2rfc approaches. The string [!RFC1149] within the text declares a normative reference to RFC1149 while [~RFC1149] declares an informative reference. The HTML2RFC tool attempts to resolve references by consulting a local bibliography file. Unresolved references are then resolved through a set of (extensible) rules and (optionally) appended to one or more of the user's personal bibliography files. For example, citation labels that begin with the prefix 'RFC' may be resolved by a HTTP query to the xml.resource.org repository.* Basics - elements are defined by tags, * Elements may have attributes but these are used sparingly. In particular HTML markup conventions require that text which is normally displayed to the user be entered as element content rather than as attributes.* character entities are used to escape the markup characters* Except in the <pre> element, multiple spaces are collapsed to single spaces. To prevent whitespace folding, use the tag.* Every HTML document must contain the following elements:Although the HTML markup requires that a <title> element be present, the term title is understood to be text to be shown in 'title bar' of the window frame in which the document is displayed. HTML2RFC uses the first <h1> tag in the document to display the title instead as this is guaranteed to be displayed prominently in a HTML text editor while many do not bother to show the <title> element contents at all.Following the convention established by xml2rfc, HTML2RFC divides the RFC content into front, middle and back sections. These are delimited by <h1> heading elements as follows:The front matter begins with the first <h1> element and ends at the second <h1> element.The middle matter begins with the second <h1> element and ends at the first <h1> element with a heading that begins with the string 'Appendix:'.The back matter begins with he first <h1> element with a heading that begins with the string 'Appendix:' and runs to the end of the document.It is not necessary to create sectionsfor References or Authors' Addresses, these will be included and given appropriate titles as necessary. The outline of an HTML2RFC document is therefore:Document structure in HTML is infered from the placement of heading elements. This approach is considerably easier to manage and less error prone than the xml2rfc approach which requires the author to correctly mark the beginning and the end of each section.HTML2RFC uses only the core HTML markup designed for declaring document structure: <h1> ... <h6>, <p>, <pre>, <ul>, <ol>, <dl>, <li>, <dt>, <dd>, <table>, <th>, <tr>, <th>, <td>.Only the id attribute is currently recognized, all other attributes are discarded or overwritten.MarkupStatusAttributesDescription<html>, <head>, <title>, <body>Necessary for well formed HTML <h1>, <h2, <h3>, <h4>, <h5>, <h6>idTitle / Section Headings<p>idParagraph<pre>idPreformatted text (e.g. code, examples)<ol>, <ul>, <li>idBulleted and numbered lists<dl>, <dt>, <dd>idDefinition lists<table>, <tr>, <th>, <td>, <caption>*id, width*, align*Tables<a>*Internal / External document links<span>*Mark boundaries of normative text<dfn>*Declare a defined term<em>*Reference a defined term<?include?>file, formatInclude text from external file<?bibliography?>file, format, cacheUse the specified bibliography<?bibsource?>*prefix, patternUse the specified bibliography resolution source<?stylesheet?>*Specify a style sheet to include[label]Normative reference[label]Informative reference[Left square braceAlthough HTML provides most of the prinitives required to markup an RFC, it does not provide the necessary tags to markup the text that appears at the front of an Internet Draft, the author names, document type and other information that is presented in the masthead and opening section. This information is declared in HTML2RFC using a 'microformat' a stylized method of using HTML tags. This section describes the use of standard HTML markup tags to define the document structure. The microformat is described in [Section ?] following. The use of the id attribute to create references to document sections is described in [Section ?]The <h1>, <h2, <h3>, <h4>, <h5>, <h6> elements specify section headings. The element content is the heading text. Document section and appendix numbers are added automatically.The first and second <h1> element in the document are special. The first <h1> element specifies the title of the document and begins the document properties microformat section described below. The second <h1> element marks the end of the document properties microformat and the beginning of the document proper.A <h1> element with heading text that begins with the string 'Appendix:' is an appendix heading. Once one appendix heading is encountered in a document, all the following <h1> elements are interpreted as appendix headings whether they begin with the string 'Appendix:' or not.The <p> and <pre> elements specify paragraphs. <p> elements specify ordinary paragraphs that will be formatted by the processing tool while the <pre> element is used for preformatted text such as code examples where spaces must be preserved.The <ul> element specifies an unordered list (aka bullet point). Each list item is specified by a list item <li> element:To create an ordered list (aka numbered), the <ol> element is used instead:Definition list <dl> elements contain lists of defined term <dt> and defined data <dd> tags. Although the formatting of <dt> and <dd> elements strongly sugests that they are pairs, HTML permits them to appear in any order.Lists may be nested in lists to create a hierachy. The inner list type may be the same as the type of the enclosing list or a different type.Due to the need to support conversion to IETF Plaintext and xml2rfc formats, the html2rfc table model is considerably simpler than the html model. The colspan and rowspan attributes are ignored and the width attribute is currently unimplemented.The <table> element specifies a table. A table element may contain only <tr> and <caption> elements. Each <tr> element specfies a row of the table. The optional <caption> element specifies a title for the table. The <caption> element is only permitted as the first and/or last element in the list:Table cells are specified using the <th> or <td> elements. <th> elements specify heading cells, <td> elements specify cells in the body of the table. If the first row of a table contains at least one <th> cell, the header will be repeated at the head of each page if the table is split across pages. Note that while the formatting tool may split a table so that different rows are presented on different pages, a page boundary will not be placed in the middle of a row unless the cell is too long to present any other way.The document properties are specified in the front matter which begins with the first <h1> heading and ends with the second <h1> heading. The only HTML elements permitted within the front matter are <p>, <dl>, <dt> and <dd>. All other tags are ignored but may raise warnings.The document abstract is specified using <p> elements. <p> elements that are empty or only contain whitespace are ignored. All other <p> elements specify a paragraph of the abstract. Since Internet Drafts are required to provide an abstract, at least one <p> element that contains text is required.Document properties are tag value pairs specified using a definition list <dl> element. Each <dt> element specifies a tag and each <dd> element a value. Some document properties permit a list of values to be specified. A document can only have one file name but it can have multiple keywords. When there are multiple values for the same property, it is only necessary to specify the tag once as successive <dd> elements specify values for the tag specified in the last <dt> element encountered.The document properties for this document are specified as follows:The document properties have been formatted in tabular form to help comprehension but this is not necessary.Each document property is specified by means of a case insensitive tag. In most cases the document property tag is the same as the corresponding xml2rfc element or attribute. Although most authors are likely to need to consult the documentation when specifying document properties, this is considered to be acceptable as the need to add properties to an existing document is relatively infrequent.TagDescriptionValuesabbrevAbreviated document title for page headersdocnameDocument filename (excluding version)versionDocument version number00..99yearPublication yearmonthPublication monthdayPublication dayiprIntellectual Property RightsSee belowareaIETF areaworkgroupIETF working groupnumberSepcifies an RFC numbercategorye.g. Standards trackSee belowupdatesNumber of an RFC updated by this RFCobsoletesNumber of an RFC made obsolete by this RFCseriesnoe.g. BCP 42, STD 69The IPR property specifies a label which will eventually identify a set of boilerplate text that can be imported from an external file. This will avoid the need for users to update their document authoring tools in response to changes in IETF policy.[TBS]Authors are specified in the same way as the document properties. The author properties are specified as follows:TagDescriptionauthorStart new author and specify full namesurnameAuthor surnameinitialsAuthor initialsorganizationOrganizationstreetPostal address streetcityPostal address citycodePostal address post codecountryPostal address countryphoneTelephone numberemailEmail addressuriWeb siteIn practice the Author, surname, initials, organization and email are usually sufficient. In addition to making existing document editing tasks easier, HTML2RFC adds several new features to make maintaining a standards document easier:Documents may be split into multiple files facilitating use of automatically generated content.Citations specified in the body of the text are detected automatically and resolved to generate the necessary references.Concodances of defined terms and normative text may be generated.The <?include?> Processing directive is used to include text from an external source. The directive takes two attributes:The file nameThe file format, xml or txtThe text from the included document is inserted at the current position in the fileIn the following example, the text for Sections four and five are inserted from an external file.The include file processing directive is useful for combining sections of text that are either generated or verified by automatic tools.The format parameter is intended to allow the processing tool to perform appropriate format conversion as the text is included. At present only XML fragments are supported.Since the semantics of the processing directive are only understood by the document processing tools and are disregarded by text editors, a HTML2RFC document that includes <?include?> processing directives MUST be a well formed HTML document both before and after the external document is inserted.One of the main advantages of moving from the XML2RFC model of specifying sections as nested XML elements to the HTML model of infering section boundaries from headings is that it makes the use of text from included files considerably more flexible.The following format types are reserved for future use:FormatUsexmlXML document or fragment. The initial XML document declaration is ignored.htmlHTML document of fragment If the toplevel element is <html>, only the contents of the inner <body> element are incorporated.codeCode in an unspecified programming languageebnfExtended Bacus Naur FormjsonJSON text exampleasn1ASN1 SchemaxsdXML SchemagdlGoedel structure filecC programming languagecsC#javaJavajsJavascriptThe <?stylesheet?> processing directive takes the same file= attribute as the <?include?> directive except that the directive is only processed when generating HTML output in which case the included file MUST contain a CSS stylesheet and the text is copied to the output file directly to a <style> element in the HTML <head> element.Citations in the body of the text are specified by means of a citation marker. The citation marker tells the document processorThat a citation label is about to followThat the citation is normative or informativeThe label to be used to retrieve the corresponding reference from the bibliographic sources(optional) The preferred citation tag to use in the final textEach reference is identified by means of a text label. For convenience the documents commonly referenced in IETF specifications have compact predfined labels. For example, [RFC822] has the label RFC822.Citation markers have the following forms:Normative reference to the reference 'label' to be presented as [label] in the final text.Normative reference to the reference 'label' to be presented as [tag] in the final text.Informative reference to the reference 'label' to be presented as [label] in the final text.Informative reference to the reference 'label' to be presented as [tag] in the final text.The document processing tool automatically resolves the citation labels to references and presents them as lists of normative and informative references.If a document contains a normative and an informative reference to the same document, the reference is listed in the normative references section.Citation labels are resolved by consulting a bibliography file or bibliographic source. HTML2RFC documents may declare bibliography files and bibliographic sources by means of processing instructions. Document processing tools May consult additional built in bibliographic sources. The order of precedence for resolving bibliographic sources is:Bibliography files in the order they are declared in the source documentBibliographic sources in the order they are declared in the source documentBibliographic sources built in to the processing toolBibliography files are XML fragments that contain a list of xml2rfc <reference> elements, each of which defines a single reference. The <?bibliography?> processing instruction declares a bibliography file to be searched to resolve references. The filename of the bibliography to be searchedThe format of the bibliography to be searched If present, the document processor will append any references returned from one of the bibliographic sources to the end of the file at the end of document processing.Use of the cached attribute allows references fetched from bibliographic souces to be saved for future use. Rather than risk inserting new references into the middle of a file, the tool appends new reference entries to the end. Since the XML specification requires that an XML document only have one toplevel document, this means that the bibliography files are actually XML fragments rather than XML documents and some XML editing tools may present the user with tedious and unnecessary warnings.The html2rfc tool has the following bibliographic sources built in:PrefixDocument SeriesExample citationRFCRequest for CommentsRFC822RFC-Request for Comments (alternate style)RFC-822DRAFT-Internet Draftdraft-hallambaker-html2rfcW3C-W3C recommendationsW3C.xkms3GPP.3GGPANSI.ANSI standardsCCITT.CCITTFIPS.Federal Information Processing StandardsIEEE.Selected IEEE standardsISO.Selected ISO standardsITU.Selected ITU documentsNIST.Selected NIST documentsOASIS.Selected OASIS documents PKCS.RSA Labs Public Key Cryptography StandardsPKCS.7.1993X.OSI document series X.509.2000One of the main objectives in designing a tool that can be used to produce documents for multiple standards development organizations is to encourage other organizations to maintain bibliographic source servers providing access to authoritative versions of the documents.The <?bibsource?> processing directive allows additional bibliographic sources to be defined. A bibliographic source is simply a Web server that responds to a GET request to fetch the content corresponding to a URI formed from the citation label by performing a set of predefined processing rules.Users are not expected to need to use the <?bibsource?> processing directive as the html2rfc tool contains built in rules to access all the known bibliographic sources the tool is currently capable of reading.dA URI pattern to be expanded to form the URI to be used to attempt resolution of the reference.A bibliographic source is only queried if the citation label matches the prefis specified. For example, a bibliographic source with the prefix 'RFC' will be queried to attempt resolution of the citation label RFC822 but not the label draft-hallambaker-htnl2rfc.The uri attribute actually specifies a URI matching pattern. The URI matching pattern may contain one or two hash '#' characters.If one hash character is specified, it is replaced by the portion of the citation label that follows the prefix. If the citation label is RFC822 and the prefix is 'RFC', a single hash character will be replaced by the string '822'.Several of the bibliographic sources at xml.resource.org have filenames in which the document number of numbered documents are padded with preceeding zeros. While this maintains the correct collation order for directory listings and the like, the simple character susbstitution mechanism would break unless we remembered to use RFC0822 as the citation label.More complicated substitution capabilities may be attempted by specifying a C# string formatting directive in between two hash marks. For example the matching specifier #d4# tells the processor to convert the portion of citation label following the prefix to an integer and then format the integer with preceeding zeros so that it is at least 4 characters in length.Cross references to other parts of a document are specified using the HTML id attribute and anchor <a> element.The id attribute declares an anchor label within the current document. Anchor labels MUST NOT be declared more than once.The processing tool automatically creates anchor labels for section headings that do not have an id attribute specified as follows:idUseSection_<n>Section heading Section_<n>_<n>Subsection heading (etc.)Appendix_<a>Appendix headingAppendix_<a>_<d> ... Appendix subheading (etc.)ReferencesReferences_NormativeReferences_InformativeAbstractTitleAuthor_<n>Author #nThe anchor element <a> creates a cross reference to the section identified by an anchor. The href attribute specifies the link target, in this case a URI document fragment identifier which is denoted by a hash character '#' followed by the anchor label.If the content of the anchor element begins with the prefix [?, the entire contents of the anchor element will be replaced by a suitable cross reference text.For example the following cross reference would be rendered as (see Section 4):Tracking the use of normative language inside the text can take considerable effort in the later stages of standards development. The keywords MUST, SHOULD and MAY defined in [RFC2119] require particular actions to be taken when determining if a specification is ready to progress from RECOMMENDED to STANDARD status.HTML2RFC processing tools can easily identify the occurrence of normative keywords but merely counting the number of MUST keywords and the pages on which they occur adds little value since the context of the requirement is lost. Heuristic approaches such as extracting the sentence in which the normative language occurs offers only a modest improvement in clarity.The HTML <span> element MAY be used to specify the portion of text in which the normative language is used that is sufficient to identify the term in a concordance. If the <span> element specifies an id attribute, it specifies the label to be used to identify the normative requirement in the concordance.For example, the normative language in the preceeding paragraph is marked as follows:The concordance of normative language would contain the following entry as a result:The HTML <span> element MAY be used to specify the portion of text in which the normative language is used that is sufficient to identify the term in a concordance.The concordance of defined terms is generated in the same manner as the concordance of normative language except that the html <dfn> tag is used to specify the definition in the text.[Reference Not Found!][Reference Not Found!]Instructions to RFC AuthorsUSC/Information Sciences InstituteUSC/Information Sciences InstituteRFC authorsKey words for use in RFCs to Indicate Requirement LevelsHarvard Universitykeyword[Reference Not Found!]Hypertext Markup Language - 2.0MIT Laboratory for Computer ScienceMIT Laboratory for Computer Science, W3 ConsortiumWriting I-Ds and RFCs using XMLInvisible Worlds, Inc.RFCRequest for CommentsI-DInternet-DraftXMLExtensible Markup LanguageBibTeXingrfctool is a command line tool that converts from the HTML2RFC subset of HTML to other document formats. At the time of writing these are:HTML output formatted using a stylesheet that produces an IETF style masthead.IETF Plaintext format at 58 lines per page and 72 characters per line and a formfeed between lines.[~RFC2629] format to allow HTML2RFC documents to be shared with other authors.The following commands are supported:Describe the rfctool toolRead the file <input> as HTML2RFC source and produce the specified output format(s).Read the file <input>, perform processing of embedded <?include?> processing directives and write the result to the file <output>.The default command is /html.The /lazy option tells the tool to only process the input file if either the inpout file or the tool itself is newer than all the specified outputs. This option allows html2rfc to be used in build scripts without requiring unnecessary actions.The /html command accepts the following options but these are not currently implemented:Specify a bibliography file to be includedSpecify a file to be incorporated into the HTML output as a stylesheetGenerate HTML output compliant with W3C formatting rulesExtract boilerplate text definitions from the specified fileThe tool requires that the input file be strictly compliant with XML requirements but does not perform schema validation and simply ignores markup that it does not understand. The document outputs are (or should be) fully compliant with the HTML and/or XML2RFC DTDs.The following xml2rfc features are not yet implemented due to lack of time:Table column widths. Centered and right formatted table columns.References to specific document sections.Citation of external URL resourcesError handling does not identify source line where problem occurred.Number ordered lists in the txt output.Headers and footers in HTML outputUse of <caption> tag to generate captions for tables.The following features were considered but not implemented due to lack of timeAbility to create lists of normative language sorted into MUST / SHOULD / RECOMMENDED / MAYAbility to specify definitions and defined terms and raise errors when defined terms have no definitionAbility to import code files and automatically translate into examples / schemas / etc.Ability to read XML2RFC formatAbility to read back generated HTML (round trip)Write out the list of errors into the HTML output as XML comments.Add anchors into HTML output to show page transitions.Lists of tables, figuresThe following xml2rfc features are not implemented on account of diminishing returns:Ability to specify symbols other than * in lists Use of letters rather than numbers in lists Use of complicated formatting in lists. The following features are intentionally omitted:Ability to chose whether to generate a ToC, depth etc. Every document needs a ToC.Generating output in PDF: Use a tool like wkhtmltopdf instead.The following are really just bugs:The tool does not support P tags nested inside LI, DT or DD elements as separate paragraph markersNo handling of <BR> tag