Compression
-
Looping Is Not in the GIF Spec
The GIF89a specification defines the signature, the color tables, the LZW encoder, the four interlacing passes, transparency, and the per-frame delay. It does not define animation looping. Download it and search: the word “loop” does not appear in the document once.
Every GIF you have ever watched repeat did so because of a 19-byte block that Netscape made up in 1995 and nobody ever standardized.
I saved the same eight frames twice, once with a loop count and once without:
loop.gif 1969 bytes noloop.gif 1950 bytesNineteen bytes. That is the entire difference between an animation and a slideshow that stops.
The Nineteen Bytes
$ grep -abo NETSCAPE loop.gif 28:NETSCAPE $ grep -abo NETSCAPE noloop.gif $At byte 25, three bytes before that match, the block starts:
0x21 extension introducer 0xFF application extension label 0x0B block size: 11 "NETSCAPE" 8 bytes "2.0" 3 bytes 0x03 sub-block size: 3 0x01 sub-block ID: loop count 0x0000 loop count, little-endian; zero means forever 0x00 block terminatorThe GIF89a spec does define a generic Application Extension, in section 26: a container where a vendor can stash eleven bytes of identifier and whatever payload they like, which decoders that don’t recognize the identifier must skip. That was the extension point Netscape used, and the feature won so completely that it is now the only reason the format still exists.
Section 14 of the same document, well before it gets there, tells you not to do this:
This approach is recommended in favor of using Application Extensions, which become overhead for all other applications that do not process them.
The spec authors saw the vendor-extension move coming, wrote down that it was a bad idea, and then shipped the mechanism anyway. Six years later a browser vendor used it to define the format’s most famous feature.
There is no RFC for this. There is no W3C note. There is no erratum to GIF89a. The WHATWG’s own wiki page for GIF does not document the block either; it links out to a third-party page that reverse-engineered it. Every GIF encoder in the world implements a format feature whose only description is somebody else’s notes.
The Header, and the Ghost of 1995
The first sixteen bytes tell you most of what the file is:
$ xxd -g 1 -l 32 loop.gif 00000000: 47 49 46 38 39 61 c8 00 64 00 81 00 00 ff 5a 28 GIF89a..d.....Z( 00000010: 14 14 1e 00 00 00 00 00 00 21 ff 0b 4e 45 54 53 .........!..NETSGIF89a, thenc8 00and64 00: 200 by 100, little-endian, which is the opposite of what JPEG, PNG, and MP4 all chose. Then a packed byte,0x81, whose low three bits say the global color table holds 2^(1+1) colors. Four colors, twelve bytes, and then at byte 25 the Netscape block starts immediately.The per-frame timing lives in a different block, the Graphic Control Extension, and it has a problem:
GCE at byte 44: delay field = 8 (hundredths of a second) GCE at byte 386: delay field = 8 (hundredths of a second) 8 Graphic Control Extensions totalHundredths of a second. Two bytes, unsigned, little-endian. The finest interval the format can express is 10 milliseconds, so the theoretical ceiling is 100fps and there is no way to say 60fps evenly.
It gets worse in practice. Browsers treat a delay of 0 or 1 as 10, so asking for 100fps gets you 10fps. Firefox has done this since at least 2004, and the reason in the source comments is compatibility with how Netscape behaved. A delay of 2 is the practical floor, which puts the real maximum at 50fps.
So the animation feature was defined by Netscape, and the animation timing is still bounded by an emulation of Netscape’s bugs, thirty years after Netscape.
LZW Is Just Worse
Here is the part that makes GIF’s survival strange.
I took a 512x512 image with 151,112 distinct colors and reduced it to a 256-color adaptive palette. Then I saved those exact same quantized pixels two ways:
pal.gif 59198 bytes (LZW) pal.png 29154 bytes (DEFLATE) true.png 64106 bytes (DEFLATE, all 151,112 colors, lossless)Identical pixels, identical palette. PNG is less than half the size. And full-color lossless PNG, with every one of those 151,112 colors intact, is only 8% bigger than the GIF that threw 150,856 of them away.
LZW builds a dictionary of repeated index sequences with codes that grow from 2 bits to a hard ceiling of 12. DEFLATE does LZ77 plus Huffman coding and has no such ceiling. On anything that isn’t flat-color line art, it is not close.
To be fair to the format, GIF is not always the loser. My eight-frame animation of a flat orange circle on a flat dark background came out:
anim.gif 1969 bytes anim.png 2170 bytes (APNG) anim.webp 4392 bytesOn a handful of frames of solid color, GIF’s per-frame overhead is small and LZW does fine. That is the shape of content GIF was designed for in 1987, when it was moving graphics over a 2400-baud modem, and it is still competitive there. It is just that nobody uses GIF for that anymore. They use it for compressed video of a person reacting to something, which is close to the worst possible input for a 256-color palette and a 12-bit dictionary.
The Format PNG Was Built To Replace
At the end of December 1994, CompuServe and Unisys jointly announced that software reading or writing GIF would need a license for US Patent 4,558,302, Terry Welch’s LZW patent, granted in 1985.
The response was immediate. PNG went from nothing to a frozen ninth draft on 7 March 1995, roughly ten weeks later. It is a better format on essentially every axis: better compression, 24-bit color, an alpha channel instead of one transparent index, and no patent. The League for Programming Freedom ran a “Burn All GIFs” campaign. Everyone agreed GIF should die.
The patent expired on 20 June 2003, and by then the argument had been over for years. PNG had won the still-image half completely.
It lost the other half because it did not have animation, and by the time APNG existed the web had already decided that a looping image was a GIF. The format that PNG was created to replace survives entirely on the one capability PNG shipped without, which the GIF specification also does not have, and which exists only because a browser vendor stuffed it into a generic extension slot and shipped it.
What To Do About It
- Stop using GIF for video. A short MP4 or a WebM is smaller by an order of magnitude, plays with hardware decoding, and does not have a 256-color palette. Every major platform silently transcodes your GIF uploads to video already.
- Use PNG for anything still. There is no case in 2026 where a static GIF is the right answer.
- If you must ship GIF, quantize deliberately. Pick the palette yourself rather than letting the encoder guess, and dither on purpose. The default adaptive palette is rarely the best 256 colors for your image.
- Never set a frame delay below 2. You will get 10fps instead of the 50 or 100 you asked for, and it will look like a bug in your code.
- Check for the loop block if animations mysteriously play once.
grep -abo NETSCAPE file.gif. Plenty of encoders omit it by default, including Pillow unless you passloop=0.
Every format in this series has had a gap between what the specification says and what implementations do. GIF is the case where the gap swallowed the format. The spec describes a still-image format with optional frame timing. What the world actually uses is that spec plus one undocumented vendor block, and the vendor has been gone since 2003.
Sources
- GIF89a specification — CompuServe, 31 July 1989; the Application Extension block is section 26
- WHATWG Wiki: GIF — where the web platform’s own wiki sends you for the looping extension, which is off-site
- US Patent 4,558,302 — Welch, granted 1985, expired 20 June 2003
- PNG specification history — the ninth draft froze on 7 March 1995
- Mozilla bug 232822 — animated GIF frame delays of 10ms are slowed to 100ms
- What’s In A GIF — the clearest walkthrough of GIF’s LZW bit packing anywhere
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
-
JPEG Quality 80 Is Not a Setting
MP4’s boxes are exact about everything: every byte accounted for, every offset written down. JPEG is exact about its structure too, and then hands you one control that has no defined meaning at all.
It destroys things. That is the entire point, and it will not tell you how much.
Here is one 512x512 PNG, encoded to JPEG four times, on the same machine, with every encoder set to quality 80.
q80-cjpeg.jpg 42595 bytes (libjpeg-turbo 3.2.0) q80-pillow.jpg 42595 bytes (Pillow 12.3.0) q80-sips.jpg 67865 bytes (macOS sips) q80-ffmpeg.jpg 13847 bytes (ffmpeg -q:v 80)Same input. Same number typed into the same-named parameter. A 5x spread in output size.
The Number Is an Index, Not a Measurement
There is no quality field in a JPEG file. Nothing in ISO/IEC 10918-1 defines a scale from 0 to 100. What the file actually carries is a quantization table: 64 integers that every DCT coefficient in an 8x8 block gets divided by before rounding. Bigger divisors, more coefficients rounded to zero, smaller file, more damage.
“Quality 80” is just a name your encoder gives to one particular table. Here is the first row of the luminance table each of those four files chose:
cjpeg 6 4 4 6 10 16 20 24 pillow 6 4 4 6 10 16 20 24 sips 2 2 2 3 4 5 7 8 ffmpeg 8 62 73 85 100 104 112 131cjpeg and Pillow are identical because Pillow links libjpeg and inherits its table. Both are the standard Annex K example table scaled by a linear formula: at quality 80 the scale factor is 40%, and 16 x 0.40 rounds to 6, 11 x 0.40 rounds to 4, and so on down the row.
macOS
sipsdivides by 2 where libjpeg divides by 6. Its quality 80 lands somewhere around libjpeg’s 93, and it is not the standard table scaled differently, it is a different table. Apple picked their own numbers.ffmpeg is the funny one. Its
-q:vfor MJPEG runs 2 to 31, where lower is better. So-q:v 80gets clamped to 31, the worst setting it has:$ ffmpeg -i source.png -q:v 31 f31.jpg $ cmp f31.jpg q80-ffmpeg.jpg $Byte-identical. Asking ffmpeg for 80 asks it for the ugliest image it knows how to make, and it does not warn you, because 80 is a perfectly valid thing to say to a parameter that happens to top out at 31.
None of these encoders is wrong. The specification never told them what 80 means.
The Damage Is Mostly Not Where You Think
The DCT-and-quantize step gets all the attention. It is not usually the thing wrecking your image.
Before any of that happens, the encoder converts RGB to YCbCr and then, by default, throws away three quarters of the color information. 4:2:0 subsampling averages the two chroma channels over 2x2 pixel blocks. Human vision is much less sensitive to color detail than to brightness detail, so most of the time you cannot see it.
Most of the time. Here is a 400x120 image, pure red on pure blue, encoded at quality 95 both ways:
size max channel error mean error 4:4:4 (no subsampling) 19688 bytes 20 0.59 4:2:0 (default) 10782 bytes 232 14.70Quality 95 is a setting people reach for when they want the image to be basically untouched. At 4:2:0 a single channel is off by 232 out of 255. The red and blue have the same luminance, so the entire edge between them lives in chroma, and chroma is the part that got averaged away.
This is why red text on a colored background looks like it was scanned by a fax machine, why UI screenshots with colored syntax highlighting come out muddy, and why a logo saved as JPEG at “high quality” still has a smeared halo. Turning subsampling off costs about 80% more bytes here and takes the error from 232 to 20.
cjpeg -sample 1x1does it. In Pillow it issubsampling=0. Almost no tool exposes it in a GUI, and almost every default is 4:2:0.
Generation Loss Converges
The folk wisdom is that re-saving a JPEG degrades it a little more each time, forever, until it turns to soup. I re-encoded the same image 50 times at quality 85, decoding and re-encoding each round, and measured the drift from the original:
gen size(bytes) mean err max err 1 48911 3.51 194 2 49532 4.11 188 5 49351 5.21 197 10 49299 6.17 199 25 49249 7.20 190 50 49189 7.54 190Most of the loss happens on save one. Generation two adds about half a point. Generations 25 through 50 add a third of a point between them, and the maximum error never moves at all.
It converges because quantization is idempotent once you land on the grid. Decode a coefficient that was rounded to 6 times its divisor, transform it back, and it quantizes to the same bucket. The image reaches a fixed point that survives re-encoding. What breaks this is changing anything: a different quality, a different subsampling mode, a crop that shifts the 8x8 block boundaries, or a rotation that resamples. Then you land on a new grid and pay the first-generation cost again.
Which is the actual reason
jpegtranexists:$ jpegtran -rotate 180 -outfile r1.jpg original.jpg $ jpegtran -rotate 180 -outfile r2.jpg r1.jpg $ cmp original.jpg r2.jpg $Two 180-degree rotations, byte-identical to the input.
jpegtranpermutes the already-quantized coefficient blocks without ever decoding to pixels, so there is nothing to re-quantize. Rotating, flipping, and cropping on 8-pixel boundaries are all lossless if you use the right tool. Almost nobody does.
What the File Looks Like
Worth thirty seconds, because the structure is unusually clean. Every marker is
0xFFfollowed by a type byte, and every marker except the two bare ones carries a 2-byte big-endian length:$ xxd -g 1 -l 32 q80-cjpeg.jpg 00000000: ff d8 ff e0 00 10 4a 46 49 46 00 01 01 00 00 01 ......JFIF...... 00000010: 00 01 00 00 ff db 00 43 00 06 04 05 06 05 04 06 .......C........ffd8is Start of Image.ffe0is APP0, length0x0010, containing the literal stringJFIF\0. Then at offset 20,ffdbis Define Quantization Table, length 67, table 0, and the bytes after it are the 64 divisors in zigzag order.06 04 05 06 05 04is that first row I printed above, read diagonally.Walking the whole file:
0 FFD8 SOI 2 FFE0 APP0 length 16 20 FFDB DQT length 67 89 FFDB DQT length 67 158 FFC0 SOF0 length 17 177 FFC4 DHT length 31 210 FFC4 DHT length 181 393 FFC4 DHT length 31 426 FFC4 DHT length 181 609 FFDA SOS length 12 42593 FFD9 EOIEverything structural fits in the first 623 bytes. The remaining 42,000 are entropy-coded scan data with no framing at all, which creates one last problem: if a Huffman code happens to emit the byte
0xFF, a decoder scanning for markers would misread it. So the encoder stuffs a0x00after every literal0xFF, and the decoder throws it away. This file contains 504 of those.
What To Do About It
- Never move a quality number between tools. Quality 80 in Photoshop,
cjpeg,sips, and ffmpeg are four unrelated things. If you are porting a pipeline, re-tune by measuring output size or error, not by copying the integer. - Turn off chroma subsampling for anything with saturated color edges. Logos, screenshots, charts, text.
-sample 1x1in cjpeg,subsampling=0in Pillow. Leave 4:2:0 on for photographs, where it is nearly free. - Use
jpegtranfor rotations and crops.-rotate,-flip,-crop, and-perfectoperate on coefficients and cost nothing. - Stop worrying about generation loss and start worrying about generation one. The first save is where the damage is. Keep the original.
- Don’t put a JPEG in the middle of a pipeline. Every intermediate step should be PNG or the raw source. JPEG is an output format.
- Strip the metadata deliberately. APP1 holds Exif, which holds GPS coordinates, camera serial numbers, and an embedded thumbnail. Some editors update the pixels and leave the old thumbnail in place, so the crop you made survives only in the big version.
The interesting thing about JPEG is not that it is lossy. Everyone signed up for lossy. It is that the one control the format exposes to users, the quality number, is not part of the format, has no defined meaning, and is quietly reinterpreted by every tool that offers it. You are not setting the quality. You are picking a preset out of a list you cannot see.
Sources
- ITU-T T.81 / ISO/IEC 10918-1 — the core JPEG specification; Annex K holds the example quantization tables everybody scales
- ITU-T T.871 — JFIF, standardized in 2011, nineteen years after the industry started shipping it
- W3C copy of the JFIF 1.02 specification — the original C-Cube document from September 1992
- Independent JPEG Group — libjpeg, whose quality-to-table formula became the de facto meaning of the number
- libjpeg-turbo — what almost everything actually links against today
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
- Never move a quality number between tools. Quality 80 in Photoshop,
-
ZIP Files Are Read Backwards
Every format in this series so far reads front to back. PNG starts with a signature and you walk chunks in order. A text file is bytes from the beginning. Markdown parsers scan line by line, top to bottom.
ZIP reads backwards. The index is at the end of the file, and a reader is expected to seek to the end first and work its way back.
That one decision explains almost everything strange about ZIP, including a few things that look like bugs and one thing that is definitely a bug.
The Index Lives at the End
Here’s a real ZIP containing two small text files. 241 bytes total. Scanning it for the four-byte record signatures gives the whole layout:
offset bytes ascii record 0 50 4b 03 04 P K . . Local File Header <- hello.txt 53 50 4b 03 04 P K . . Local File Header <- second.txt 108 50 4b 01 02 P K . . Central Directory Header 163 50 4b 01 02 P K . . Central Directory Header 219 50 4b 05 06 P K . . End of Central DirectoryEvery one of those starts with the same two bytes.
0x50is decimal 80, which isPin ASCII.0x4bis decimal 75, which isK.PK. Phil Katz, who wrote PKZIP in 1989, put his initials in the first two bytes of every structure in the format, and they are still there in every.docx,.jar, and.epubon your machine.The two bytes after
PKare the record type:03 04for a local file header,01 02for a central directory entry,05 06for the end-of-central-directory record. Those aren’t printable characters, which is deliberate. A four-byte constant made of two readable letters and two control bytes is unlikely to appear by accident in text, and easy to spot by eye in a hex dump.The last 22 bytes are the End of Central Directory record, and it’s the entry point:
50 4b 05 06 00 00 00 00 02 00 02 00 6f 00 00 00 6c 00 00 00 00 00 signature 0x06054b50 total CD records 2 central dir size 111 bytes central dir offset 108 comment length 0A reader opens the file, jumps to the end, finds that record, reads “the index is at offset 108,” seeks there, and reads the catalog. Listing the contents of a 4 GB archive touches a few hundred bytes.
Note that each file appears twice: once as a Local File Header immediately before its compressed data, and once as an entry in the Central Directory at the end. Hold that thought.
Why Would You Do This?
Because in 1989 you were writing to a floppy disk, and often to a floppy disk that wasn’t big enough.
If the index goes at the front, you have to know everything about every file before you write the first byte: how many files, how big each one compresses to, where each one lands. That means compressing everything to a temporary location, then writing the header, then copying it all back. On a machine with 640K of RAM and two floppy drives, that’s brutal.
Put the index at the end and you can stream. Compress a file, write it, remember where it went. Compress the next one. When you run out of files, write down everything you remembered. One pass, no temporary copy, and you never needed to know the total size in advance.
TAR solved the same problem by having no index at all, which is why
tarhas to read an entire archive to find one file, and why you cannot randomly access a.tar.gz. ZIP got both streaming writes and random-access reads. That’s the trade that made it win.
The Backwards Scan Is Fuzzier Than It Sounds
The EOCD record is 22 bytes, so you’d think a reader could just read the last 22 bytes and be done.
It can’t, because the record ends with a variable-length archive comment of up to 65,535 bytes. The signature isn’t at a fixed offset from the end of the file. So a reader has to seek near the end and scan backwards looking for the four-byte signature, potentially across 65,557 bytes.
Searching for a magic number is not the same as knowing where a structure is. If those four bytes happen to appear inside the comment, or inside compressed data near the end of the file, a naive parser can lock onto the wrong one. Different implementations pick different candidates when there’s more than one. This is a recurring source of “this archive opens in one tool and not another.”
You Can Put Anything in Front of a ZIP
If a reader finds the archive by scanning backwards from the end, then whatever sits at the front of the file is not the reader’s problem.
Take a valid 69-byte PNG, take the 241-byte ZIP, and concatenate them with
cat. No special tooling:$ file polyglot.png polyglot.png: PNG image data, 1 x 1, 8-bit/color RGB, non-interlaced $ unzip -l polyglot.png Length Date Time Name --------- ---------- ----- ---- 12 08-10-2026 16:44 hello.txt 13 08-10-2026 16:44 second.txt --------- ------- 25 2 filesOne 310-byte file. An image viewer reads the PNG signature at byte 0 and renders an image. An archive tool scans backwards, finds the EOCD, and extracts two files. Both are correct. Neither is being fooled by a trick; they’re each doing exactly what their format says to do.
This is the mechanism behind self-extracting archives, where the front of the file is a real executable and the back is a real ZIP. The same property is why “GIFAR” attacks worked: a file that a server accepted as a harmless image was loaded by Java as an archive of classes.
It also means the offsets inside the Central Directory are relative to the start of the archive, not the start of the file, and readers have to work out that difference. Prepending data shifts everything, and well-behaved parsers cope by computing the delta between where the EOCD says the directory should be and where it found it.
Two Indexes, One File
Back to that detail from earlier: every file’s name and metadata are stored twice, in the Local File Header and again in the Central Directory.
Nothing enforces that they agree.
Here’s the same archive with only the Central Directory copy of the first filename patched from
hello.txttoBOGUS.txt. The local header is untouched:$ unzip -l mismatch.zip Length Date Time Name --------- ---------- ----- ---- 12 08-10-2026 16:44 BOGUS.txt 13 08-10-2026 16:44 second.txt local file header at offset 0 still says: hello.txtThe archive is not corrupt.
unziplists it happily. It just contains two different answers to “what is this file called,” and which one you get depends on which structure your parser decided to trust.Now imagine two programs reading the same archive, one checking a signature and the other extracting files. That’s the Android “Master Key” bug from 2013, and the detail is better than the summary.
An APK is a ZIP. The attacker puts two entries in it, both named
classes.dex. Android’s Java verifier loaded entries into a map keyed by filename, so a duplicate name overwrote the earlier one and the last entry was the one whose signature got checked. The native installer used a hash table with linear probing that didn’t replace on collision, so the first entry was the one that got loaded and run. Plant malicious code first, legitimately signed code second, and the device verifies one file and executes the other.A second bug the same year came from the same “two readings, one file” family, via a signed integer. The extra-field length is a 16-bit value, and the Java code read it signed. A length of 65,533 (
0xFFFD) sign-extends to −3. Since the offset of the compressed data is computed by adding that length, a negative value moves the read pointer backward into the header region instead of forward past it.The lesson generalizes past ZIP. Any format that stores the same fact twice has to decide what happens when the copies disagree, and “the spec doesn’t say” is the same answer as “attackers decide.”
Offsets Are Just Numbers
The Central Directory locates each file by offset. Nothing in the format says two entries can’t point at the same bytes.
The classic zip bomb didn’t need that.
42.zipis 42 kilobytes of archives nested five layers deep, sixteen at each layer, unpacking to roughly 4.5 petabytes. The defense is obvious once you’ve seen it: cap recursion depth, don’t auto-extract nested archives.David Fifield’s 2019 construction doesn’t recurse at all. It expands in a single pass, so depth limits are irrelevant. The trick is overlap: many Central Directory entries reference one shared kernel of compressed data, and each entry’s compressed stream uses DEFLATE’s stored-block mode to quote the next entry’s local file header as literal bytes. Entries nest inside each other, and output grows quadratically against input.
He published several, and they aren’t interchangeable:
File Compressed Uncompressed Ratio Needs Zip64 zbsm.zip42 KB 5.5 GB ~130,000:1 No zblg.zip10 MB 281.4 TB ~28,000,000:1 No zbxl.zip46 MB 4.5 PB ~98,000,000:1 Yes The Zip64 requirement on the largest one matters, because not every reader supports Zip64, which makes the merely-enormous version the more portable weapon.
None of these are malformed files. Every one is a valid archive that a conforming parser is supposed to accept. The format allows two entries to describe the same bytes, and no rule anywhere says the total uncompressed size has to bear any relationship to the file you’re holding.
Everything Is Secretly a ZIP
Once you know the structure, you start recognizing it:
.docx,.xlsx,.pptxare ZIP archives of XML.jar,.war,.apkare ZIP archives of class files and resources.epubis a ZIP of XHTML.odt,.odsare ZIP of XML again
That’s not a coincidence or a hack. ISO/IEC 21320-1, “Document Container File,” defines a constrained ZIP profile for exactly this use. It narrows the format so a
.docxreader doesn’t have to implement all of ZIP’s accumulated history: compression must be stored or deflated and nothing else, and the various encryption and digital-signature mechanisms in the original spec are all forbidden.It’s a narrowing, not a rewrite. Zip64 version 1 is still permitted, for instance; only version 2 is ruled out. The profile is best understood as a list of the parts of ZIP that turned out to be a bad idea.
Which means the next post in this series is mostly about a ZIP file with XML inside it. You already know half of how a Word document works.
Sources
- PKWARE APPNOTE.TXT — the original and still-authoritative ZIP specification, currently version 6.3.10; §4.3.16 defines the end of central directory record
- ISO/IEC 21320-1:2015 — the constrained ZIP profile used by document formats. Fair warning, this one is a paid ISO standard; the catalog page tells you what it covers but you cannot read the text without buying it
- Library of Congress format description for ZIP — history and preservation notes
- David Fifield: A Better Zip Bomb — the overlapping-stream construction
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
-
How PNG Actually Stores Your Pixels
I’m starting a series on file formats. Not “here are the ten image formats you should know,” but the actual bytes: what’s in the file, in what order, and why someone decided it should be that way.
Starting with PNG, because it’s the format most developers touch every day and almost nobody has looked inside.
I am likely to cover a few things that other explainer documents have covered, such as chunk structure and chunk types. However, I’d like to dig into some details that are not often mentioned, such as where your pixels went.
A Format Born From a Patent Fight
PNG exists because of a licensing ambush. On 28 December 1994, right in the middle of the holidays, Unisys announced an agreement to start collecting royalties from authors of GIF-supporting software, on the strength of its patent on the LZW compression algorithm that GIF used.
The response was fast. A draft for a replacement format was posted to
comp.graphicson 4 January 1995, one week later. It was originally called PBF, for Portable Bitmap Format, and got renamed to PNG two days after that. The format shipped as a W3C Recommendation in October 1996.Two things about that origin still show in the bytes. The format is aggressively defensive, because it was designed by people who expected files to be mangled in transit. And it is aggressively extensible, because they had just watched a format become unusable for reasons that had nothing to do with its technical design.
Eight Bytes of Paranoia
Every PNG starts with the same eight bytes:
Hexadecimal: 89 50 4E 47 0D 0A 1A 0A ASCII/Ctrl: \x89 P N G \r \n \x1A \nP N Gin the middle is obvious. The other five bytes are a booby trap for 1995-era file transfer, and each one catches a specific failure:0x89has the high bit set. Some 7-bit transfer paths stripped bit 7 from every byte. If that happened, this byte arrives as0x09and the file is detectably wrong on byte one.0x0D 0x0Ais a DOS line ending. A text-mode FTP transfer that “helpfully” converts CRLF to LF mangles it.0x1Ais Ctrl-Z, the MS-DOS end-of-file marker. If youTYPEa PNG at a DOS prompt, output stops here instead of spraying binary at your terminal and leaving it in a weird state.0x0Ais a bare LF, catching the opposite conversion: LF silently expanded to CRLF.
Who would have thought that so many bits were used just to account for line endings in different operating systems? I suppose it’s good to plan ahead when designing a file format.
Everything Is a Chunk
After the signature, a PNG is a flat sequence of chunks. No central directory, no offset table. You read them in order.
Every chunk has the same four-field shape:
Field Size Notes Length 4 bytes Big-endian, counts only the data field Chunk Type 4 bytes Four ASCII letters Chunk Data Length bytes Can be zero-length CRC-32 4 bytes Computed over type and data, not over length Two details worth keeping. The length field is 32 bits but the spec caps values at 2³¹−1, so the high bit is always clear. And the CRC covers the type plus the data but skips the length, which means a corrupted length field is not detected by the chunk’s own checksum.
The chunk type is where PNG does something clever. Those four letters are ASCII, and bit 5 of an ASCII letter is what distinguishes uppercase from lowercase (
Ais0x41,ais0x61). PNG uses that bit in each of the four positions as a flag:Position Uppercase means Lowercase means 1st Critical: decoder must understand it Ancillary: safe to ignore 2nd Public, registered in the spec Private, vendor-specific 3rd Reserved, must be uppercase today (reserved for future use) 4th Unsafe to copy if pixels changed Safe to copy blindly So a decoder that has never heard of
tEXtcan tell from the lowercasetthat skipping it is fine. A decoder hittingIDATsees the uppercaseIand knows it cannot skip it. The capability negotiation is encoded in the name itself, which means you can add chunk types decades later without breaking old readers. This is why APNG could bolt animation onto PNG without a version bump.Four chunk types are critical:
IHDR(header, always first),PLTE(palette),IDAT(the pixels), andIEND(a zero-length terminator).
IHDR Is the Decode Key
IHDRis exactly 13 bytes and it comes first because nothing else can be interpreted without it:- Width (4 bytes) and Height (4 bytes), big-endian
- Bit depth (1 byte): bits per sample, one of 1, 2, 4, 8, 16
- Color type (1 byte): what a pixel is made of
- Compression method (1 byte): always 0
- Filter method (1 byte): always 0
- Interlace method (1 byte): 0 for none, 1 for Adam7
Bit depth and color type together determine everything about the pixel layout, and only certain combinations are legal:
Color type Name Samples per pixel Legal bit depths 0 Greyscale 1 1, 2, 4, 8, 16 2 Truecolor (RGB) 3 8, 16 3 Indexed 1 (a palette index) 1, 2, 4, 8 4 Greyscale + alpha 2 8, 16 6 Truecolor + alpha (RGBA) 4 8, 16 Note the gaps. You cannot have 16-bit indexed color, because a palette holds at most 256 entries and 8 bits already addresses all of them. You cannot have 1-bit RGB, because a “1-bit red sample” isn’t a useful thing. The table isn’t arbitrary; each missing cell is a combination that would be incoherent.
Also note that bit depth is per sample, not per pixel. A bit depth of 16 with color type 6 means 16 bits each for R, G, B, and A: 64 bits per pixel. That’s the “64-bit RGBA” you see in PNG marketing.
Where the Pixels Actually Live
Uncompress all the
IDATdata and concatenate it, and you get a byte stream. That stream is not a grid. It’s a sequence of scanlines, one per image row, top to bottom. And each scanline is:[1 filter type byte][packed sample data for the whole row]That leading byte is not pixel data. It’s a number from 0 to 4 saying which filter was applied to this row.
The sample data is packed with no padding between pixels and no separators. Samples appear in a fixed order within each pixel:
- Greyscale:
grey - Truecolor:
red, green, blue - Indexed:
palette index - Greyscale + alpha:
grey, alpha - Truecolor + alpha:
red, green, blue, alpha
Here is an example PNG, filter type 0 (no filtering) on both rows:
scanline 0: 00 ff 00 00 00 ff 00 00 00 ff ff ff 00 ^^ filter byte ^^^^^^^^ red pixel (ff,00,00) ^^^^^^^^ green pixel (00,ff,00) scanline 1: 00 00 00 00 80 80 80 ff ff ff ff 00 ff ^^ filter byte ^^^^^^^^ black ^^^^^^^^ whiteTwelve bytes of pixel data per row (4 pixels × 3 samples), each prefixed by one filter byte, for 26 bytes of raw stream. The complete file, signature and all four chunks included, is 83 bytes.
The whole model at 8-bit depth: walk the row, emit samples in order, move on. No alignment, no padding, no per-pixel headers.
Below 8 Bits, Pixels Share Bytes
Bit depths of 1, 2, and 4 only apply to greyscale and indexed images. Multiple pixels get packed into a single byte.
These samples are packed into bytes with the leftmost sample in the high-order bits of a byte followed by the other samples for the scanline.
Leftmost pixel goes in the high bits. So for a 12-pixel-wide 1-bit greyscale image:
pixels : 1 1 0 1 0 0 0 1 1 0 1 1 packed bytes : 0xd1 0xb0 11010001 10110000 ^^^^ unusedTwelve pixels need 12 bits, which rounds up to 2 bytes, leaving 4 bits spare at the end. The spec’s language on those leftover bits:
When there are multiple pixels per byte, some low-order bits of the last byte of a scanline may go unused. The contents of these unused bits are not specified.
Scanlines always start on a byte boundary. Row 2 never continues in the leftover bits of row 1’s last byte.
At bit depth 16, each sample is two bytes, most significant byte first. The spec calls it network byte order. On x86 and ARM, which are little-endian, that means every 16-bit sample needs a byte swap on read and on write.
The Filter Byte Is the Whole Trick
Now back to that leading byte on every scanline.
PNG uses DEFLATE, the same algorithm as gzip and zip. If you just DEFLATE’d raw pixels, PNG would compress about as well as gzipping a bitmap, which is to say barely at all. Photographs and gradients don’t repeat exact byte sequences, and LZ77 needs exact repeats.
So before compressing, PNG transforms each scanline into differences from its neighbors. Five filters are available, chosen per scanline:
Type Name Transform 0 None store the byte as-is 1 Sub subtract the byte from the pixel to the left 2 Up subtract the byte from the pixel above 3 Average subtract the average of left and above 4 Paeth subtract whichever of left/above/upper-left is the best predictor All arithmetic is mod 256, which is what makes it reversible without storing a sign. And “the pixel to the left” means the byte at the same position in the previous pixel, so for RGB the red sample is compared against the previous red sample, not against the previous blue.
Take a 16-pixel greyscale gradient stepping by 10:
raw scanline : 00 0a 14 1e 28 32 3c 46 50 5a 64 6e 78 82 8c 96 after Sub filter : 00 0a 0a 0a 0a 0a 0a 0a 0a 0a 0a 0a 0a 0a 0a 0aSixteen distinct byte values become two. The image is unchanged and the transform is exactly reversible, but LZ77 now sees a run it can encode in almost nothing. On this toy row, DEFLATE produces 28 bytes for the raw version and 15 for the filtered one.
Sixteen bytes is far too small for DEFLATE to stretch its legs, so don’t read that ratio as typical. The point is the entropy collapse: filtering doesn’t compress anything, it rearranges the data so the compressor has something to find.
That per-scanline choice is also why two encoders produce different-sized files from identical pixels. libpng, ImageMagick,
oxipng, andzopflipngall ship different filter-selection heuristics. Same spec, same decoded output, different bytes on disk. Most PNG optimizers are search algorithms over filter choices, not better compressors.
The Compression Pipeline, End to End
Putting it together:
raw pixels -> pack into scanlines (samples in order, sub-byte packing if needed) -> prepend a filter byte per scanline, apply the filter -> DEFLATE the whole concatenated stream (LZ77 + Huffman) -> wrap in a zlib container (RFC 1950) -> split across one or more IDAT chunksA few consequences fall out of that ordering:
The zlib stream spans chunks.
IDATboundaries are arbitrary. A decoder must concatenate everyIDATpayload and then decompress; decompressing them individually fails. Encoders split them for streaming, not for structure.The Adler-32 checksum in the zlib wrapper covers filtered bytes, not your original pixels. It validates decompression, not image fidelity. The per-chunk CRC-32 is what protects against transmission corruption.
Compression is global across the image. LZ77’s 32KB sliding window means row 400 can match against row 380 if they’re similar. This is why a 64×64 solid color block compresses to 136 bytes while a 64×64 gradient of the same dimensions takes 10,362 bytes, against 12,288 bytes raw. Uniformity compresses; novelty doesn’t.
And a practical one: for that solid-color block, encoding as indexed color with a one-entry palette produces a 99-byte file instead of 136, because each pixel is one index byte instead of three samples. If your image has few colors, color type 3 usually beats truecolor even after DEFLATE gets its turn.
Interlacing, Briefly
If the interlace byte in
IHDRis 1, the image uses Adam7: the pixels are transmitted in seven passes over an 8×8 grid, coarse to fine, so a partially-downloaded image renders as a low-resolution preview that sharpens.Two things to know. Each pass is filtered and encoded as an independent sub-image with its own scanlines and filter bytes, so a decoder can’t treat the stream as one grid. And Adam7 typically makes files larger, because breaking the image into seven sparse sub-images destroys exactly the local coherence that filtering and LZ77 depend on. It was a good trade on a 28.8k modem. On any modern connection it costs size and complexity for a progressive render nobody waits around to see.
What This Buys You
The design decisions hold up well for a 1996 format:
- Unknown chunks are safe by construction, so the format extended to EXIF metadata, ICC profiles, and animation without ever breaking old decoders.
- Every chunk is individually checksummed, so corruption is localized and detectable rather than silently rendering garbage.
- Filtering is a preprocessing step, not a compression format, which means encoders can get better forever without touching the spec. A file written by
zopflipngtoday decodes fine in a 1997 reader.
That extensibility is not just historical. PNG got a Third Edition as a W3C Recommendation on 24 June 2025, which finally standardized APNG, added an
eXIfchunk for camera metadata, and brought in HDR through three new chunks (cICP,mDCV,cLLI). Thirty years on, the container still had room.Where it shows its age is DEFLATE, which is a 1990s compressor. Lossless WebP does beat it: Google’s own study puts WebP lossless at 23% smaller than PNGs already optimized with ZopfliPNG, and 42% smaller than default libpng output. Worth noting the baseline matters enormously there, and Google’s WebP FAQ quotes a different figure (26%) than the study it links to.
Lossless AVIF is a murkier story than the marketing suggests. AOMedia publishes no general lossless-AVIF-versus-PNG number at all; its quantified claims (50% versus JPEG, 30% versus WebP) are all about lossy encoding. The only primary figure available is 10% versus a 16-bit PNG for a single demo image using a new v1.2.0 feature. Independent testing regularly finds lossless AVIF producing larger files than PNG for flat synthetic images like icons, UI, and charts. If you’re picking a format for screenshots and diagrams, test on your own images rather than trusting a general ranking.
Next in the series: the opposite of all this. A text file, which announces nothing about itself at all.
Sources
- W3C PNG Specification, Third Edition — the current standard; §7.2 covers scanlines and sample packing, §9 covers filtering
- RFC 2083 — the original 1997 IETF PNG specification
- RFC 1950 (zlib) and RFC 1951 (DEFLATE) — the compression layer
- libpng PNG history — the Unisys announcement, the PBF name, and the January 1995 timeline
- WebP Lossless and Alpha Study — Google’s 23%/42% lossless figures and their baselines
- AOMedia on AVIF v1.2.0 — the 10% lossless figure, and its narrow scope
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].