A PDF does not have a current version. It has a stack of them, and the one your reader shows you is whichever one is on top.
\nThat is not a bug, but a clause in the specification, it is the reason PDFs can be signed and annotated without invalidating anything that came before, and it is also why a court filing with black rectangles over the sensitive parts keeps handing those parts to anyone who runs pdftotext.
I built a 619-byte PDF by hand to show the mechanism, because the mechanism is small enough to fit in a blog post.
\nHere is the entire tail of that file:
\nxref\n0 6\n0000000000 65535 f \n0000000015 00000 n \n0000000064 00000 n \n0000000121 00000 n \n0000000247 00000 n \n0000000366 00000 n \ntrailer\n<< /Size 6 /Root 1 0 R >>\nstartxref\n436\n%%EOF\nA parser opens this file by seeking to the end, finding %%EOF, reading the number above it, and jumping to byte 436. That is the cross-reference table. Every line in it is a ten-digit zero-padded byte offset for one object. Object 4 lives at byte 247. Object 5 lives at byte 366. The trailer then says the document catalog is object 1, and the parser follows the graph from there.
Nothing is read sequentially. There is no “parse the header, then stream forward.” The header is four bytes of version number and a comment containing four deliberately non-ASCII bytes, present only so that any transport handling the file is forced to treat it as binary.
\nIf you read the ZIP post earlier this month, this should feel familiar. Both formats put their index at the end so that appending is cheap. Both formats therefore have the same structural property: the index is authoritative, and the bytes it doesn’t point at are still sitting right there in the file.
\nStart with the failure, the one that has been in the news.
\nMy page has a single content stream with a text-drawing operator in it:
\nBT /F1 12 Tf 20 60 Td (CONFIDENTIAL: Q3 layoffs begin March 14) Tj ET\nTo redact it, I append a filled black rectangle covering the same coordinates:
\n0 0 0 rg 15 52 270 24 re f\nOn screen that is a black bar. The text is underneath it and completely invisible. Then:
\n$ pdftotext overlay.pdf -\nCONFIDENTIAL: Q3 layoffs begin March 14\nThe rectangle is a drawing instruction. Tj is a different drawing instruction. Text extraction never rasterizes anything, so it never learns that one shape happens to sit on top of another. It walks the content stream, collects the text operators, and hands them over.
This is what Paul Manafort’s lawyers filed in January 2019. The blacked-out passages came straight back out with copy and paste, and what came out was that Manafort had shared 2016 campaign polling data with Konstantin Kilimnik. A corrected version went up quickly.
\nIt is also the 93-page TSA screening manual posted to a federal contracting site in December 2009, with black boxes over the tolerances used by airport metal and explosive detectors and over the special handling rules for CIA officers, diplomats, and law enforcement. Same mistake, ten years earlier, and the second time nobody could claim the technique was novel.
\nNow the interesting case. This time I actually replace the content stream. New object 4, no text operator, only the rectangle. The old object stays where it is, because PDF’s editing model appends:
\n$ pdftotext redacted.pdf -\n$\nEmpty. The text is gone from the document as the parser understands it. This is an edit, not an overlay, and every viewer will agree the page contains a black bar and nothing else.
\n$ strings redacted.pdf | grep -i confidential\nBT /F1 12 Tf 20 60 Td (CONFIDENTIAL: Q3 layoffs begin March 14) Tj ET\nThe file is 812 bytes. The first 616 of them are the original document, untouched, including its own xref, its own trailer, and its own %%EOF. The new revision was appended after that:
$ grep -abo '%%EOF' redacted.pdf\n613:%%EOF\n806:%%EOF\nTwo end-of-file markers in one file. The second trailer carries a /Prev key pointing back at the first cross-reference table:
trailer\n<< /Size 6 /Root 1 0 R /Prev 436 >>\nstartxref\n695\n%%EOF\nSo recovering the unredacted document is not forensics. It is head:
$ head -c 616 redacted.pdf > recovered.pdf\n$ pdftotext recovered.pdf -\nCONFIDENTIAL: Q3 layoffs begin March 14\nThat is a valid, complete, openable PDF. Not a fragment, not a carved string. The original revision was always a self-contained file; the redaction just parked another file behind it.
\nYou can see why the format works this way. Digital signatures need the signed byte range to stay byte-identical, so an annotation or a form fill has to append rather than rewrite. Incremental save is also why a 400-page PDF takes a moment to annotate instead of a minute. The design is coherent. It just means that “save” and “erase” are unrelated operations, and most software offers you the first one while you are thinking about the second.
\nIn 2021 Supriya Adhatarao and Cédric Lauradoux collected 39,664 PDF files published by 75 security agencies across 47 countries and looked at what was still in them. Seven of the 75 agencies had made any attempt at sanitization at all. Of the files those seven had sanitized, 65% still contained sensitive information.
\nThese are security agencies. Sanitizing documents is a thing they have written policies about. The success rate among the small minority that tried was roughly one in three.
\nThe reason is not incompetence. It is that the safe operation and the obvious operation are different operations, and the file gives you no feedback about which one you performed. Both produce a document with a black bar on it.
\nqpdf --linearize in.pdf out.pdf rebuilds the document as a single revision and drops what nothing points at. If the output still has two %%EOF markers, the collapse did not happen.strings and pdftotext. Both take ten seconds. Between them they would have caught every failure in this post.%%EOF markers on any PDF you receive, not just the ones you send. Use grep -ac '%%EOF' file.pdf; without -a, grep decides the file is binary and reports nothing at all./Info dictionary and the XMP packet, and none of that is touched by redacting page content.The formats in the first half of this series failed by being unclear about what their bytes meant. PDF is not unclear about anything. It says precisely where every object is and precisely which ones are current, and it says it in a structure that keeps the non-current ones exactly where they were. The file is honest. It is the word “redacted” that is doing the lying.
\n\n\n", "date_published": "2026-08-26T10:00:00-05:00", "url": "https://llbbl.blog/2026/08/26/your-pdf-remembers-what-you.html", "tags": ["Programming","security","File-formats","Pdf","Forensics"] } ] }I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @logan@llbbl.blog.
\n