===========
BSON Corpus
===========

:Title: BSON Corpus
:Author: Luke Lovett, David Golden
:Lead: Jeff Yemin
:Advisors: Craig Wilson
:Status: Approved
:Type: Standards
:Minimum Server Version: N/A
:Last Modified: July 20, 2017
:Version: 2.0

.. contents::

Abstract
========

The official BSON specification does not include test data, so this
pseudo-specification describes tests for BSON encoding and decoding.  It also
includes tests for MongoDB's "Extended JSON" specification (hereafter
abbreviated as ``extjson``).

Meta
====

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD",
"SHOULD NOT", "RECOMMENDED",  "MAY", and "OPTIONAL" in this document are to be
interpreted as described in `RFC 2119`_.

.. _RFC 2119: https://www.ietf.org/rfc/rfc2119.txt

Motivation for Change
=====================

To ensure correct operation, we want drivers to implement identical tests
for important features.  BSON (and ``extjson``) are critical for correct
operation and data exchange, but historically had no common test corpus.
This pseudo-specification provides such tests.

Goals
-----

* Provide machine-readable test data files for BSON and ``extjson`` encoding
  and decoding.

* Cover all current and historical BSON types.

* Define test data patterns for three cases: (a) conversion/roundtrip, (b)
  decode errors, and (c) parse errors.

Non-Goals
---------

* Replace or extend the offical BSON spec at http://bsonspec.org.

* Provide a formal specification for ``extjson``.

Specification
=============

The specification for BSON lives at http://bsonspec.org.  The ``extjson``
format specification lives at
https://github.com/mongodb/specifications/blob/master/source/extended-json.rst.

Test Plan
=========

This test plan describes a general approach for BSON testing.  Future BSON
specifications (such as for new types like Decimal128) may specialize or
alter the approach described below.

Description of the BSON Corpus
------------------------------

This BSON test data corpus consists of a JSON file for each BSON type, plus
a ``top.json`` file for testing the overall, enclosing document and a
``multi-type.json`` file for testing a document with all BSON types.
There is also a ``multi-type-deprecated.json`` that includes deprecated keys.

Top level keys
~~~~~~~~~~~~~~

* ``description``: human-readable description of what is in the file

* ``bson_type``: hex string of the first byte of a BSON element (e.g. "0x01"
  for type "double"); this will be the synthetic value "0x00" for "whole
  document" tests like ``top.json``.

* ``test_key``: (optional) name of a field in a single-BSON-type ``valid`` test
  case that contains the data type being tested.

* ``valid`` (optional): an array of validity test cases (see below).

* ``decodeErrors`` (optional): an array of decode error cases (see below).

* ``parseErrors`` (optional): an array of type-specific parse error case (see
  below).

* ``deprecated`` (optional): this field will be present (and true) if the
  BSON type has been deprecated (i.e. Symbol, Undefined and DBPointer)

Validity test case keys
~~~~~~~~~~~~~~~~~~~~~~~

Validity test cases include 'canonical' forms of BSON and Extended JSON that
are deemed equivalent and may provide additional cases or metadata for
additional assertions.  For each case, keys include:

* ``description``: human-readable test case label.

* ``canonical_bson``: an (uppercase) big-endian hex representation of a BSON
  byte string.  Be sure to mangle the case as appropriate in any roundtrip
  tests.

* ``canonical_extjson``: a string containing a Canonical Extended JSON document.
  Because this is itself embedded as a *string* inside a JSON document,
  characters like quote and backslash are escaped.

* ``relaxed_extjson``: (optional) a string containing a Relaxed Extended JSON
  document.  Because this is itself embedded as a *string* inside a JSON
  document, characters like quote and backslash are escaped.

* ``degenerate_bson``: (optional) an (uppercase) big-endian hex representation
  of a BSON byte string that is technically parseable, but not in compliance
  with the BSON spec.  Be sure to mangle the case as appropriate in any
  roundtrip tests.

* ``degenerate_extjson``: (optional) a string containing an invalid form of
  Canonical Extended JSON that is still parseable according to type-specific
  rules.  (For example, "1e100" instead of "1E+100".)

* ``converted_bson``: (optional) an (uppercase) big-endian hex representation
  of a BSON byte string.  It may be present for deprecated types. It represents
  a possible conversion of the deprecated type to a non-deprecated type, e.g.
  symbol to string.

* ``converted_extjson``: (optional) a string containing a Canonical Extended
  JSON document.  Because this is itself embedded as a *string* inside a JSON
  document, characters like quote and backslash are escaped.  It may be
  present for deprecated types and is the Canonical Extended JSON
  representation of ``converted_bson`.

* ``lossy`` (optional) -- boolean; present (and true) iff ``canonical_bson``
  can't be represented exactly with extended JSON (e.g. NaN with a payload).

Decode error case keys
~~~~~~~~~~~~~~~~~~~~~~

Decode error cases provide an invalid BSON document or field that
should result in an error. For each case, keys include:

* ``description``: human-readable test case label.

* ``bson``: an (uppercase) big-endian hex representation of an invalid
  BSON string that should fail to decode correctly.

Parse error case keys
~~~~~~~~~~~~~~~~~~~~~

Parse error cases are type-specific and represent some input that can not
be encoded to the ``bson_type`` under test.  For each case, keys include:

* ``description``: human-readable test case label.

* ``string``: a text or numeric representation of an input that can't be
  parsed to a valid value of the given type.

Extended JSON encoding, escaping and ordering
---------------------------------------------

Because the ``canonical_extjson`` and other Extended JSON fields are embedded
in a JSON document, all their JSON metacharacters are escaped.  Control
characters and non-ASCII codepoints are represented with ``\uXXXX``.  Note that
this means that the corpus JSON will appear to have double-escaped characters
``\\uXXXX``.  This is by design to ensure that the Extended JSON fields remain
printable ASCII without embedded null characters to ensure maximum portability
to different language JSON or extended JSON decoders.

There are legal differences in JSON representation that may complicate
testing for particular codecs.  The JSON in the corpus may not resemble
the JSON generated by a codec, even though they represent the same data.
Some known differences include:

* JSON only requires certain characters to be escaped but allows any character
  to be escaped.

* The JSON format is *unordered* and whitespace (outside of strings) is not
  significant.

Implementations using these tests MUST normalize JSON comparisons however
necessary for effective comparison.

Language-specific differences
-----------------------------

Some programming languages may not be able to represent or transmit all
types accurately.  In such cases, implementations SHOULD ignore (or modify)
any tests which are not supported on that platform.

Testing validity
----------------

To test validity of a case in the ``valid`` array, we consider up to five
possible representations:

* Canonical BSON (denoted herein as "cB") -- fully valid, spec-compliant BSON

* Degenerate BSON (denoted herein as "dB") -- invalid but still parseable BSON
  (bad array keys, regex options out of order)

* Canonical Extended JSON (denoted herein as "cEJ") -- A string format based on
  the JSON standard that emphasizes type preservation at the expense of
  readability and interoperability.

* Degenerate Extended JSON (denoted herin as "dEJ") -- An invalid form of
  Canonical Extended JSON that is still parseable.  (For example, "1e100"
  instead of "1E+100".)

* Relaxed Extended JSON (denoted herein as "rEJ") -- A string format based on
  the JSON standard that emphasizes readability and interoperability at the
  expense of type preservation.

Not all input types will exist for a given test case.

There are two forms of BSON/Extended JSON codecs: ones that have a language-native
"intermediate" representation and ones that do not.

For a codec *without* an intermediate representation (i.e. one that translates
directly from BSON to JSON or back), the following assertions MUST hold
(function names are for clarity of illustration only):

* for cB input:

  * bson_to_canonical_extended_json(cB) = cEJ

  * bson_to_relaxed_extended_json(cB) = rEJ (if rEJ exists)

* for cEJ input:

  * json_to_bson(cEJ) = cB (unless lossy)

* for dB input (if it exists):

  * bson_to_canonical_extended_json(dB) = cEJ

  * bson_to_relaxed_extended_json(dB) = rEJ (if rEJ exists)

* for dEJ input (if it exists):

  * json_to_bson(dEJ) = cB (unless lossy)

* for rEJ input (if it exists):

  *  bson_to_relaxed_extended_json( json_to_bson(rEJ) ) = rEJ

For a codec that has a language-native representation, we want to test both
conversion and round-tripping.  For these codecs, the following assertions MUST
hold (function names are for clarity of illustration only):

* for cB input:

  * native_to_bson( bson_to_native(cB) ) = cB

  * native_to_canonical_extended_json( bson_to_native(cB) ) = cEJ

  * native_to_relaxed_extended_json( bson_to_native(cB) ) = rEJ (if rEJ exists)

* for cEJ input:

  * native_to_canonical_extended_json( json_to_native(cEJ) ) = cEJ

  * native_to_bson( json_to_native(cEJ) ) = cB (unless lossy)

* for dB input (if it exists):

  * native_to_bson( bson_to_native(dB) ) = cB

* for dEJ input (if it exists):

  * native_to_canonical_extended_json( json_to_native(dEJ) ) = cEJ

  * native_to_bson( json_to_native(dEJ) ) = cB (unless lossy)

* for rEJ input (if it exists):

  * native_to_relaxed_extended_json( json_to_native(rEJ) ) = rEJ

Implementations MAY test assertions in an implementation-specific
manner.

Testing decode errors
---------------------

The ``decodeErrors`` cases represent BSON documents that are sufficiently
incorrect that they can't be parsed even with liberal interpretation of
the BSON schema (e.g. reading arrays with invalid keys is possible, even
though technically invalid, so they are *not* ``decodeErrors``).

Drivers SHOULD test that each case results in a decoding error.
Implementations MAY test assertions in an implementation-specific
manner.

Testing parsing errors
----------------------

The interpretation of ``parseErrors`` is type-specific.  For example,
helpers for creating Decimal128 values may parse strings to convert them
to binary Decimal128 values.  The ``parseErrors`` cases are strings that
will *not* convert correctly.

The documentation for a type (if any) will specify how to use these
cases for testing.

For type "0x00" (i.e. top-level documents), the ``parseErrors`` entries have a
``description`` field and an ``string`` field.  Parsing the ``string`` field
as Extended JSON MUST result in an error.

Drivers SHOULD test that each case results in a parse error.
Implementations MAY test assertions in an implementation-specific
manner.

Deprecated types
----------------

The corpus files for deprecated types are provided for informational purposes.
Implementations MAY ignore or modify them to match legacy treatment of
deprecated types.  The ``converted_bson`` and ``converted_extjson`` fields MAY
be used to test conversion to a standard type or MAY be ignored.

Implementation Notes
====================

A tool for visualizing BSON
---------------------------

The test directory includes a Perl script ``bsonview``, which will
decompose and highlight elements of a BSON document.  It may be used like
this::

    echo "0900000010610005000000" | perl bsonview -x

Notes for certain types
-----------------------

Array
~~~~~

Arrays can have degenerate BSON if the array indexes are not set as
"0", "1", etc.

Boolean
~~~~~~~

The only valid values are 0 and 1.  Other non-zero numbers MUST be
interpreted as errors rather than "true" values.

Binary
~~~~~~

The Base64 encoded text in the extended JSON representation MUST be padded.

Code
~~~~

There are multiple ways to encode Unicode characters as a JSON document.
Individual implementers may need to normalize provided and generated
extended JSON before comparison.

Decimal
~~~~~~~

NaN with payload can't be represented in extended JSON, so such conversions are
lossy.

Double
~~~~~~

There is not yet a way to represent Inf, -Inf or NaN in extended JSON.  Even if
a $numberDouble is added, it is unlikely to support special values with
payloads, so such doubles would be lossy when converted to extended JSON.

String representation of doubles is fairly unportable so it's hard to provide
a single string that all platforms/languages will generate.  Testers may
need to normalize/modify the test cases.

String
~~~~~~

There are multiple ways to encode Unicode characters as a JSON document.
Individual implementers may need to normalize provided and generated
extended JSON before comparison.

DBPointer
~~~~~~~~~

This type is deprecated.  The provided converted form (``converted_bson``)
represents them as DBRef documents, but such conversion is outside the scope of
this spec.

Symbol
~~~~~~

This type is deprecated.  The provided converted form converts these to
strings, but such conversion is outside the scope of this spec.

Undefined
~~~~~~~~~

This type is deprecated.  The provided converted form converts these to Null,
but such conversion is outside the scope of this spec.

Reference Implementation
========================

The Java, C# and Perl drivers.

Design Rationale
================

Use of extjson
--------------

Testing conversion requires an "input" and an "output".  With a BSON string
as both input and output, we can only test that it roundtrips correctly --
we can't test that the decoded value visible to the language is correct.

For example, a pathological encoder/decoder could invert Boolean true and
false during decoding and encoding.  The BSON would roundtrip but the
program would see the wrong values.

Therefore, we need a separate, semantic description of the contents of a BSON
string in a machine readable format.  Fortunately, we already have extjson as a
means of doing so.  The extended JSON strings contained within the tests adhere
to the Extended JSON Specification.

Repetition across cases
-----------------------

Some validity cases may result in duplicate assertions across cases,
particularly if the ``degenerate_bson`` field is different in different cases,
but the ``canonical_bson`` field is the same.  This is by design so that each
case stands alone and can be confirmed to be internally consistent via the
assertions.  This makes for easier and safer test case development.

Changes
=======

Version 2.0 - May 26, 2017

* Revised to be consistent with Extended JSON spec 2.0: valid case fields
  have changed, as have the test assertions.

Version 1.3 - January 23, 2017

* Added ``multi-type.json`` to test encoding and decoding all BSON types within
  the same document.

* Amended all extended JSON strings to adhere to the Extended JSON
  Specification.

* Modified the "Use of extjson" section of this specification to note that
  canonical extended JSON is now used.

Version 1.2 - November 14, 2016

* Removed "invalid flags" BSON Regexp case.

Version 1.1 – October 25, 2016

* Added a "non-alphabetized flags" case to the BSON Regexp corpus file;
  decoders must be able to read non-alphabetized flags, but encoders must
  emit alphabetized flags.

* Added an "invalid flags" case to the BSON Regexp corpus file.
