{
  "version": "https://jsonfeed.org/version/1",
  "title": "Interoperability on LLBBL Blog",
  "icon": "https://avatars.micro.blog/avatars/2023/40/125738.jpg",
  "home_page_url": "https://llbbl.blog/",
  "feed_url": "https://llbbl.blog/feed.json",
  "items": [
      {
        "id": "http://llbbl.micro.blog/2026/08/20/your-json-parser-disagrees-with.html",
        "title": "Your JSON Parser Disagrees With Mine",
        "content_html": "<p>Markdown has too many specs. CSV has one that nobody agreed to follow. JSON is supposed to be the happy ending: a grammar small enough to print on a business card, standardized twice over, and elevated to a full Internet Standard.</p>\n<p>It is, and the parsers still disagree with each other about what your file says.</p>\n<p>Not about whether it&rsquo;s valid. About what the values are.</p>\n<hr>\n<h2 id=\"the-same-number-three-answers\">The Same Number, Three Answers</h2>\n<p>Here is a JSON document. It is unambiguously valid by every specification.</p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;\"><code class=\"language-json\" data-lang=\"json\"><span style=\"display:flex;\"><span>{<span style=\"color:#f92672\">&#34;id&#34;</span>: <span style=\"color:#ae81ff\">9007199254740993</span>}\n</span></span></code></pre></div><p>Three parsers, on the same machine (Node 24.13.0, Python 3.13.12, jq 1.8.2), on those exact bytes:</p>\n<pre tabindex=\"0\"><code>node   : {&#34;id&#34;:9007199254740992}\npython : {&#34;id&#34;: 9007199254740993}\njq     : {&#34;id&#34;:9007199254740993}\n</code></pre><p>Node gave back a different number than the one in the file. It didn&rsquo;t error, didn&rsquo;t warn, didn&rsquo;t round-trip. The last digit changed from 3 to 2.</p>\n<p>The reason is that JSON&rsquo;s grammar allows a number to be any sequence of digits, but JavaScript represents every number as an IEEE 754 double. Above 2⁵³ the integers stop being exactly representable, and <code>Number.MAX_SAFE_INTEGER</code> is 9007199254740991. Our value is two past it.</p>\n<p>This is not a JavaScript bug. Node is behaving exactly as specified. The specification simply declines to say how many digits a number may have, and every implementation answers that question with whatever its host language does.</p>\n<p>Those version numbers matter, which is its own version of the problem. <code>jq</code> only began preserving decimal literals in 1.7, whose release notes list &ldquo;use decimal number literals to preserve precision.&rdquo; Run that same file through jq 1.6 and it goes through a double and hands you Node&rsquo;s answer. The tool doesn&rsquo;t just disagree with other parsers. It disagrees with its own past self.</p>\n<p>If you have ever wondered why APIs send 64-bit IDs as strings, this is why. Twitter&rsquo;s snowflake IDs, database primary keys, anything above 2⁵³ has to be quoted or it silently degrades in half the ecosystem.</p>\n<hr>\n<h2 id=\"duplicate-keys-are-legal\">Duplicate Keys Are Legal</h2>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;\"><code class=\"language-json\" data-lang=\"json\"><span style=\"display:flex;\"><span>{<span style=\"color:#f92672\">&#34;role&#34;</span>: <span style=\"color:#e6db74\">&#34;user&#34;</span>, <span style=\"color:#f92672\">&#34;role&#34;</span>: <span style=\"color:#e6db74\">&#34;admin&#34;</span>}\n</span></span></code></pre></div><p>RFC 8259 says names within an object <em>should</em> be unique. Should, not must. And it goes on to describe what happens otherwise as varying between implementations.</p>\n<p>In practice:</p>\n<pre tabindex=\"0\"><code>node   : {&#34;role&#34;:&#34;admin&#34;}\npython : {&#34;role&#34;: &#34;admin&#34;}\njq     : {&#34;role&#34;:&#34;admin&#34;}\n</code></pre><p>All three take the last one. That&rsquo;s the common behavior, and it is not required.</p>\n<p>The RFC itself spells out all three possibilities:</p>\n<blockquote>\n<p>When the names within an object are not unique, the behavior of software that receives such an object is unpredictable. Many implementations report the last name/value pair only. Other implementations report an error or fail to parse the object, and some implementations report all of the name/value pairs, including duplicates.</p>\n</blockquote>\n<p>That third option, keeping both, isn&rsquo;t even representable in most languages&rsquo; object types. Nicolas Seriot tested parsers across a dozen languages against cases like this one and concluded there are &ldquo;no two parsers that agree on what is wrong and what is right.&rdquo;</p>\n<p>Python will show you both if you ask:</p>\n<pre tabindex=\"0\"><code>raw pairs: [(&#39;role&#39;, &#39;user&#39;), (&#39;role&#39;, &#39;admin&#39;)]\n</code></pre><p>The pairs are all there in the document. Choosing one is an interpretation layered on top of parsing.</p>\n<p>Now put two parsers in one system. Apache CouchDB did, and it became CVE-2017-12635.</p>\n<p>CouchDB used an Erlang parser for authentication and a JavaScript engine for the validation that runs when a document is written. The Erlang parser resolved duplicate keys to the <strong>first</strong> value. The JavaScript engine resolved them to the <strong>last</strong>. So a request like this:</p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;\"><code class=\"language-json\" data-lang=\"json\"><span style=\"display:flex;\"><span>{<span style=\"color:#f92672\">&#34;roles&#34;</span>: [<span style=\"color:#e6db74\">&#34;_admin&#34;</span>], <span style=\"color:#960050;background-color:#1e0010\">...,</span> <span style=\"color:#f92672\">&#34;roles&#34;</span>: []}\n</span></span></code></pre></div><p>was read by the write-time validation as an ordinary unprivileged user, because it saw the last <code>roles</code> key and found it empty. It was then read by the authentication layer as an administrator, because that saw the first one. Non-admin users could grant themselves admin.</p>\n<p>CouchDB&rsquo;s fix was to change the Erlang parser to take the last key, matching JavaScript. Not because last-wins is correct, but because <em>agreeing</em> is correct.</p>\n<p>This is the JSON version of the ZIP two-index problem from a few posts back. When a format permits two answers to the same question, the gap between two components is where the vulnerability lives.</p>\n<hr>\n<h2 id=\"nan-is-not-json-and-python-emits-it-anyway\">NaN Is Not JSON, and Python Emits It Anyway</h2>\n<p>JSON has no way to express not-a-number or infinity. The grammar has no room for them.</p>\n<p>Python&rsquo;s standard library writes them regardless:</p>\n<pre tabindex=\"0\"><code>&gt;&gt;&gt; json.dumps({&#34;a&#34;: float(&#34;nan&#34;), &#34;b&#34;: float(&#34;inf&#34;)})\n&#39;{&#34;a&#34;: NaN, &#34;b&#34;: Infinity}&#39;\n</code></pre><p>That output is not JSON. It&rsquo;s Python&rsquo;s default behavior, and it produces a file that other parsers reject or mangle. Feeding those exact bytes onward:</p>\n<pre tabindex=\"0\"><code>node : SyntaxError - Unexpected token &#39;N&#39;, &#34;{&#34;a&#34;: NaN, &#34;b&#34;: &#34;... is not valid JSON\njq   : {&#34;a&#34;:null,&#34;b&#34;:1.7976931348623157e+308}\n</code></pre><p>Node&rsquo;s response is correct and useful: this is not JSON, here&rsquo;s where it broke.</p>\n<p><code>jq</code>&rsquo;s response is the one that should worry you. It accepted the invalid document and made up values. <code>NaN</code> became <code>null</code>. <code>Infinity</code> became <code>1.7976931348623157e+308</code>, the largest finite double. No error, no warning. If that ran in the middle of a data pipeline you would get numbers out the other end, and they would be wrong in a way no downstream check is likely to catch.</p>\n<p>The same divergence shows up with a merely-enormous exponent, which <em>is</em> valid JSON:</p>\n<pre tabindex=\"0\"><code>input: {&#34;v&#34;: 1e999}\n\nnode   : {&#34;v&#34;:null}\npython : {&#39;v&#39;: inf}\njq     : {&#34;v&#34;:1E+999}\n</code></pre><p>Three parsers, one valid input, three different values. Node converts to infinity then serializes it as <code>null</code> because it can&rsquo;t represent infinity on the way out. Python gives you a float infinity object. <code>jq</code> preserves the literal.</p>\n<hr>\n<h2 id=\"why-a-small-spec-doesnt-save-you\">Why a Small Spec Doesn&rsquo;t Save You</h2>\n<p>JSON&rsquo;s specifications are good, and they are small. The problem is that they specify <strong>syntax</strong>, and almost every failure above is about <strong>semantics</strong>.</p>\n<p>The grammar tells you <code>9007199254740993</code> is a well-formed number. It does not tell you what number it is, because that would require committing to a numeric model, and committing to a numeric model would have meant excluding some language from implementing JSON natively. The looseness is why JSON is everywhere. It is also why the same bytes mean different things in different places.</p>\n<p>The standards process eventually acknowledged this. RFC 7493 defines <strong>I-JSON</strong>, a restricted profile that closes these holes: no duplicate names at all, numbers that should not exceed what an IEEE 754 double holds exactly, high-precision values recommended to travel as strings, and mandatory UTF-8. Only the duplicate-name rule is a hard <code>MUST NOT</code>, which tells you something about how much of this was still negotiable in 2015.</p>\n<p>I-JSON is what most people think JSON already is. It exists as a separate document precisely because JSON isn&rsquo;t that.</p>\n<hr>\n<h2 id=\"what-to-do-about-it\">What To Do About It</h2>\n<ul>\n<li><strong>Send large integers as strings.</strong> Anything that could exceed 2⁵³: IDs, timestamps in nanoseconds, financial values in minor units.</li>\n<li><strong>Reject duplicate keys</strong> at your trust boundary rather than letting your parser pick. Most libraries offer a hook.</li>\n<li><strong>Don&rsquo;t let a language&rsquo;s default serializer decide</strong> whether it emits valid JSON. Python needs <code>allow_nan=False</code> to be honest.</li>\n<li><strong>Validate before you transform.</strong> A parser that repairs invalid input is more dangerous than one that rejects it, because the repair is silent.</li>\n<li><strong>Target I-JSON</strong> for anything crossing a system boundary. It costs nothing and removes the whole category.</li>\n</ul>\n<p>JSON did not fail. It succeeded so completely that it got implemented hundreds of times by people reading a short document, and a short document leaves a lot of decisions to the reader. The format that&rsquo;s easy to implement is the format that gets implemented differently everywhere.</p>\n<p>That&rsquo;s the same sentence I could have written about Markdown, and about CSV. The pattern across this whole series is that a format&rsquo;s ambiguities don&rsquo;t stay theoretical. They become somebody&rsquo;s incident.</p>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li><a href=\"https://datatracker.ietf.org/doc/html/rfc8259\">RFC 8259</a> — the current JSON standard, and STD 90</li>\n<li><a href=\"https://datatracker.ietf.org/doc/html/rfc7493\">RFC 7493</a> — I-JSON, the profile that closes the interoperability holes</li>\n<li><a href=\"https://ecma-international.org/publications-and-standards/standards/ecma-404/\">ECMA-404</a> — the parallel Ecma grammar standard</li>\n<li><a href=\"https://seriot.ch/security/parsing_json.html\">Nicolas Seriot, &ldquo;Parsing JSON is a Minefield&rdquo;</a> — the systematic survey of parser disagreement</li>\n<li><a href=\"https://github.com/nst/JSONTestSuite\">JSONTestSuite</a> — the executable test corpus behind that research, over 300 cases</li>\n<li><a href=\"https://docs.couchdb.org/en/stable/cve/2017-12635.html\">CouchDB&rsquo;s writeup of CVE-2017-12635</a> — the duplicate-key privilege escalation, in the vendor&rsquo;s own words</li>\n</ul>\n<blockquote>\n<p>I&rsquo;d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at <a href=\"https://micro.blog/llbbl?remote_follow=1\">@logan@llbbl.blog</a>.</p>\n</blockquote>\n",
        "date_published": "2026-08-20T10:00:00-05:00",
        "url": "https://llbbl.blog/2026/08/20/your-json-parser-disagrees-with.html",
        "tags": ["Programming","Json","File-formats","Parsing","Interoperability"]
      }
  ]
}
