JSON’s problem is that its specification is too small. It tells you 9007199254740993 is a well-formed number and then declines to say which number.

YAML went the other way. The 1.2.2 specification is a book. It has a formal grammar, a chapter on recommended schemas, and an answer for nearly everything. And it will still read your config file and decide, without asking, that Norway is false.

The Norway Problem

Here is a config file. Every value in it is a string that a human would read as a string.

country: NO
duration: 1:20
mode: 0755

Three parsers, on those exact bytes:

PyYAML 6.0.3   {'country': False, 'duration': 80,     'mode': 493}
ruamel.yaml    {'country': 'NO',  'duration': '1:20', 'mode': 755}
js-yaml        {"country":"NO",   "duration":"1:20",  "mode":755}

NO is the ISO 3166 code for Norway. PyYAML returns the boolean false, because YAML 1.1 recognized twenty-two spellings of true and false, and NO is one of them. The spec lists them as a single regular expression:

y|Y|yes|Yes|YES|n|N|no|No|NO
|true|True|TRUE|false|False|FALSE
|on|On|ON|off|Off|OFF

Count them. Twenty-two. Six of those are country codes, single letters, or ordinary English words that appear in real data. Nothing in the file said “this is a boolean.” The parser inferred it from the shape of the text, and the shape of the text was two letters.

1:20 became 80 because YAML 1.1 supported sexagesimal integers, so a duration is read as base 60. One times sixty, plus twenty.

0755 became 493 because a leading zero meant octal. That is a file mode that no longer means what it says.

It Was Fixed in 2009

This is the part that makes YAML different from the other formats in this series.

CSV never had a standard. Markdown had too many. YAML had exactly one problem, everybody agreed it was a problem, and the working group fixed it. YAML 1.2 arrived in 2009 and threw all of it out. Base 60 is gone. Implicit octal is gone. The Core schema recognizes true and false and their case variants, and nothing else.

Seventeen years later, the two YAML 1.2 parsers above return strings, and PyYAML returns False.

PyYAML implements YAML 1.1. It is the default YAML library for Python, it is what pip install pyyaml gives you, and the specification it implements was superseded when the iPhone 3GS was current. The fix exists. It shipped. Most of the ecosystem simply stayed where it was, because changing the type of NO in a minor release breaks every config file that relied on it.

A format can be fixed and still be broken, if the fix arrives after the implementations do.

Everything Else That Isn’t a String

The country-code case is famous. It is not the only one, and the rest are quieter:

version: 1.10   ->  1.1        (float, and .10 became .1)
build: 010      ->  8          (octal)
port: 8080      ->  8080       (int, fine, until you concatenate it)
answers: [y, n] ->  ['y', 'n'] (strings)
answers: [yes, no] -> [True, False]

The first one is the one that should bother you. A semantic version of 1.10 parses as the float 1.1, which is a different version, and it does it silently in a file whose entire job is to record which version you meant.

And note the last two lines. y and n stay strings in PyYAML while yes and no become booleans, because PyYAML’s resolver implements a narrower set than the 1.1 spec’s regexp advertises. So the answer to “does this parser coerce single letters” is neither yes nor no. It is “some of them, and you have to test.”

Two Ways to Weaponize the Convenience

YAML has anchors. You define a node once with &name and reference it with *name. It is a useful feature for config files with repeated blocks, and it composes.

That is the problem. It composes exponentially.

a: &a ["lol","lol","lol","lol","lol","lol","lol","lol","lol"]
b: &b [*a,*a,*a,*a,*a,*a,*a,*a,*a]
c: &c [*b,*b,*b,*b,*b,*b,*b,*b,*b]
d: &d [*c,*c,*c,*c,*c,*c,*c,*c,*c]
e: &e [*d,*d,*d,*d,*d,*d,*d,*d,*d]

That file is 202 bytes. Expanding it produces 74,732 nodes, of which 59,049 are copies of the string lol. Add one more line and multiply by nine. This is the billion laughs attack, and the important detail is that safe_load does not stop it. Aliases are not a dangerous tag, they are a core language feature working as designed.

The second way is tags. YAML can annotate a node with a type, and PyYAML historically honored tags that construct arbitrary Python objects:

!!python/object/apply:os.system
args: ['id']

yaml.load() on untrusted input would run that. It became CVE-2017-18342, CVSS 9.8, published June 2018, with a description that is unusually blunt for the genre: “In PyYAML before 5.1, the yaml.load() API could execute arbitrary code if used with untrusted data.”

The fix took two releases and three years. PyYAML 5.1 deprecated the unsafe default in March 2019. PyYAML 6.0 finally made the Loader argument mandatory in October 2021, so the dangerous call stopped being the short one:

>>> yaml.load('a: 1')
TypeError: load() missing 1 required positional argument: 'Loader'

The vulnerability was published in 2018. Making the unsafe call harder to type than the safe one landed in 2021.

What To Do About It

• Quote anything that isn’t obviously a number. Country codes, versions, file modes, git SHAs, anything a human would call an identifier. Quoting is never wrong.
• Know which YAML version your parser speaks. If it is Python, assume 1.1 and the Norway problem unless you chose otherwise. ruamel.yaml gives you 1.2.
• Never call yaml.load on input you did not write. safe_load, always. On PyYAML 6 the language makes you say which you meant, which is the correct design.
• Bound the input. safe_load is not a defense against alias expansion. If you parse YAML you did not author, cap the document size before it reaches the parser.
• Use a schema. The value of a schema here is not validation, it is that it declares the type instead of letting the parser guess it from the characters.

YAML’s failure is the opposite of JSON’s, and it produces the same result. JSON declined to say what values mean, so implementations disagreed. YAML said what values mean in enormous detail, got it wrong in 2005, corrected it in 2009, and the correction never fully landed.

Next in this series is XML, which is the one format here that did specify everything. It has a schema language, a query language, a transformation language, and a namespace system. It is worth asking what all of that bought.

Sources

• YAML 1.2.2 Specification — October 2021; the schemas chapter and the rule that tabs “must not be used in indentation, since different systems treat tabs differently”
• YAML 1.1 Boolean type — the twenty-two-form regexp, working draft dated 2005
• CVE-2017-18342 — the yaml.load() RCE, CVSS 9.8
• PyYAML CHANGES — 5.1 (2019) deprecated the unsafe default, 6.0 (2021) made Loader required

I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].