30daysoftime
-
Day 18: DST and The Related Software Bugs
This is one of two or three DST posts in the 30 Days of Time series. Today’s angle: the software bugs.
Twice a year, in most of the developed world, the clocks jump forward in spring and back in fall. The hour that doesn’t exist in spring materializes and then disappears in the fall. This is the source of more shipped bugs than any other single phenomenon in software.
The Two Impossible Hours
The mechanics, if you’ve never thought hard about them.
Spring forward. On the transition Sunday in March (in the US), the clock reads
01:59:59and then immediately reads03:00:00. The hour from 2:00 to 2:59 AM does not exist. 2:30 AM on that day is not a time. If you tell a computer to do something at 2:30 AM on that day, you have asked it to do something at a time that doesn’t exist.What it does is up to the library:
- It might silently skip.
- It might silently run at 3:30 instead.
- It might throw an exception.
- It might run at the wrong time and silently throw your reports off.
The classic landmine is a daily task scheduled in local time, for example a job set to run at
1:30 AM. If you’re using the standard Linuxcrondaemon, it has battle-tested, built-in logic to detect DST transitions and prevent duplicates.The problems are usually at the application layer. If you are using an application-level scheduler or Cron library that hasn’t been configured properly and blindly trusts the system clock, you can get into a situation where that 1:30 a.m. doesn’t exist or runs twice.
A Short Tour of Named Disasters
March 2007, United States. Congress passed the Energy Policy Act of 2005, which moved DST to begin three weeks earlier and end one week later. The change took effect in March 2007. Every system in the country running on a tz database older than mid-2006 spent three weeks in March, and one week in November, off by an hour. Banks ran payroll at the wrong time. BlackBerry calendars showed every meeting an hour off. Federal agencies had to issue advisories. The DOE later estimated the extension saved about 0.5% of electricity per day of extended DST, or roughly 1.3 TWh annually. The remediation cost across every affected piece of software in the country dwarfed that figure. (More on that later.)
New Year 2011, iOS. Non-recurring alarms set for January 1 or 2, 2011 did not fire, in any time zone. People slept through work. Apple’s official advice was to set one-time alarms as recurring until January 3. This was on top of an iOS DST bug from a few months earlier, when the fall 2010 transition shifted alarms by an hour in countries that had already changed clocks. Two months after the New Year’s bug, iOS again mis-handled the US spring DST transition. Apple released an apologetic fix and quietly rewrote the alarm subsystem.
Brazil, April 2019. Brazil canceled DST after decades of observing it, via Decree 9,764. This is fine for clocks going forward, but the cancellation was announced only a few months in advance, and the IANA tz database had to ship updates fast. Every Brazilian server running on a stale cache spent the next year an hour off, in particular for any future-scheduled event saved as “local time.”
Palestine. For about a decade running, Google Calendar shipped wrong DST data for Palestine, because the Palestinian Authority changes DST rules with short notice and the IANA volunteers don’t always learn in time. Meetings between Israeli and Palestinian colleagues would silently shift by an hour twice a year.
The Shape of the Failure
The DST bugs usually go like this:
- A piece of software was written with the assumption that local time is well-defined and monotonic.
- Local time is neither.
- The author never hit edge cases. It only happens twice a year, in certain regions, under certain settings.
- The bug ships. It runs fine for six months. Then it doesn’t.
The mitigations are well-known and this is why we do what we do.
- Store UTC. Always. The IANA zone ID goes in a separate column. Never, ever store a naked local timestamp.
- Recompute the local display every time. Treat local-time as a view, not data.
- Never schedule anything between 2 and 3 AM local. That hour does not exist in your country half the time.
- Use libraries that surface the ambiguity. The older Python
pytzlibrary would throw when you constructed an impossible local time. The modernzoneinfohandles it silently via afoldattribute, meaning you have to manually check for ambiguity. JavaScript’sDateproduces inconsistent results across engines. TheTemporalAPI, which reached Stage 4 in March 2026 and ships in Chrome 144, Firefox 139, and Node 26, lets you explicitly reject ambiguous times. Use it the soonest you can. - Keep tzdata current. This is a system-package problem and most teams forget about it until something breaks.
The tricky part of software has always been that we think that the wall clock or the wall time, the number you see on a daily basis is the same as the actual physical passage of time when in reality they are not. Daylight savings time is a really great example of the absurdity of our timekeeping.
Week 3 Recap
If this is the first time you are reading this series I figured a recap is order. Week 3 has been about the infrastructure of practical timekeeping, the layer where computers, calendars, and humans actually have to agree on what time it is.
- Day 13: Unix Time, 1,780,620,532 — The 10-digit integer counting up from 1970 that runs every computer on Earth, and the weird properties hiding behind the name.
- Day 14: The Bug That Didn’t End the World, and the One That Still Might — Y2K was a save, not a hoax, and Y2038 is the one nobody is preparing for.
- Day 15: The Man Who Synchronized the World — David Mills, NTP, and the forty-year project that keeps every networked clock on Earth within a few milliseconds of UTC.
- Day 16: How the World Agreed on a Date Format (Except the US) — The century-long campaign that produced
2026-06-08T14:30:00Z, and why a bare05/06/26is still an act of faith. - Day 17: Time Zones Are a Nightmare — 38 named offsets in active use, half-hour zones, and why “what time is it there?” is the wrong question.
- Day 18 (today): DST, and the bugs that ride along with it twice a year.
The picture I want to leave you with is that every problem we’ve covered in Week 3 is a downstream consequence of a deeper one. The system isn’t fragile because of bad programmers. It’s fragile because the underlying thing, “what time is it, here, right now,” was never a single answer, and we’ve been pretending it was.
What’s Coming
Week 4 is about the cracks. What if a minute had 61 seconds? What if October had only 21 days? What if every meeting on every calendar landed on the same weekday, forever? Each of those has actually happened, or is being voted on, or was almost adopted. Week 4 covers leap seconds and their abolition, the DST fight nobody can win, and the calendars we use, almost used, and may yet use.
Sources
- Energy Policy Act of 2005 (Wikipedia). Details the US DST schedule change that took effect in 2007.
- Impact of Extended Daylight Saving Time on National Energy Consumption (US DOE). The 0.5%-per-day savings estimate from the post-2007 study.
- Apple confirms New Year’s alarm bug (AppleInsider). Coverage of the iOS 2011 non-recurring alarm bug and Apple’s workaround.
- Daylight saving time in Brazil (Wikipedia). History of DST in Brazil, including the 2019 abolition via Decree 9,764.
- Zune 30GB leap year bug (Wikipedia). The firmware loop that bricked Zunes on New Year’s Eve 2008.
- 2012 Reddit leap second outage (Wired). Write-up on the Linux
hrtimerbug that took down Reddit, LinkedIn, and Qantas. - TC39 Advances Temporal to Stage 4 (Socket). Current status of the JavaScript
TemporalAPI.
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
-
Day 17 — Time Zones Are a Nightmare
Yesterday I wrote about how
2026-05-24T14:30:00Zwon the format wars. ThatZat the end, it turns out, is quite important. It says Zulu, it says UTC, it says: I am refusing to participate in the nightmare that is Timezones.However, today we participate.
The lie you were told in school
There are 24 time zones, one for every hour, neat 15° slices of the globe.
There are not. There are at least 38 named offsets in active use right now, and the list changes a few times a year.
Some of them are not on the hour:
- India is at UTC+5:30. The whole country, one zone, half-hour offset.
- Nepal is at UTC+5:45. Forty-five minutes. Because Nepal decided in 1986 that it wanted its civil time anchored to the meridian passing through Gauri Shankar, not Delhi.
- Newfoundland is at UTC−3:30.
- The Chatham Islands are at UTC+12:45.
The range isn’t 24 hours either. It runs from UTC−12 to UTC+14, a 26-hour spread, because Kiribati got tired of being split by the international date line in 1995 and just moved the line. One day the Line Islands woke up and it was tomorrow.
Then there’s China, which geographically spans five time zones and politically uses one (UTC+8), so in the far west of Xinjiang the sun rises at what the clock insists is 10 AM. North Korea changed its offset twice in the 2010s, from UTC+9 to UTC+8:30 in 2015 to mark liberation from Japan, then back to UTC+9 in 2018 to align with Seoul during a diplomatic thaw.
Time zones are not geography. Time zones are politics with a clock face glued to the front.
How we got here
Before about 1850, every town in the world ran on its own clock. Noon was when the sun was overhead here, which meant noon in Boston was several minutes off from noon in New York, which was off from Philadelphia, which was off from everywhere.
Nobody cared, because nobody was traveling fast enough for it to matter.
Then the railroads showed up.
When your train leaves at “noon” and arrives at “3 PM” and every station defines noon differently, you can’t print a schedule. Britain rolled out Railway Time (GMT everywhere) in the 1840s. The American railroads, bless them, didn’t wait for permission. On November 18, 1883, they unilaterally divided the United States into four zones. Newspapers called it “the day of two noons” because clocks across the country jumped, sometimes forward, sometimes back, to land on the new shared time.
The following year, in October 1884, twenty-five countries met in Washington for the International Meridian Conference and made it official. Greenwich is 0°, the universal day starts at midnight in Greenwich, every other place is some offset from that.
France abstained. France wanted Paris. France held out until 1911.
The “database” holding the world together
When your phone shows you the right time after you land in Tokyo, when your calendar correctly reschedules a meeting because Mexico canceled DST a few years ago, when your server logs all line up across a deploy in three continents, that all happens because of a single, voluntarily maintained text file.
It’s called the IANA Time Zone Database, also known as the Olson Database, after Arthur David Olson, an NIH employee who started maintaining it in 1986 as a side project. Today it lives under IANA stewardship and is primarily maintained by Paul Eggert, a UCLA computer scientist who has been doing this, mostly alone, for decades.
Every Unix system, every Linux distro, every Mac, every iPhone, every Android phone, Java, Python, Go, Rust, Postgres, browsers, every piece of software that knows what time it is, gets its time zone rules from this database. The release cadence is multiple updates per year, almost always triggered by some country’s parliament deciding to change DST rules with three months’ notice.
The format is something like
America/New_York,Europe/Berlin,Asia/Kolkata,Pacific/Kiritimati. Area, slash, location. NotEST, notGMT+5, because those are offsets and offsets aren’t enough. The rules are what you need, because the rules change with politics.The whole arrangement is held together by a small group of volunteers, a mailing list, and Paul Eggert’s continued willingness to keep doing this. If he ever stops, somebody else will have to start.
Why this is one of the hardest problems in working programmer software
A few categories of pain, none of them solvable, all of them shipped to production daily:
1. Ambiguous local times. When the clocks fall back in November, the hour from 1:00 to 2:00 AM happens twice. If a user schedules a meeting at “01:30 local time” on the wrong day, which 01:30 do they mean? There is no correct answer. Your software picks one and someone shows up an hour off.
2. Nonexistent local times. When the clocks spring forward in March, 2:30 AM doesn’t exist. If somebody’s medication-reminder app is set for 02:30, what does it do that morning? Skip? Run at 03:30? Run at 01:30 the previous hour? There is no correct answer.
3. Future timestamps are mutable. If you store a meeting as “October 15, 2027 at 3 PM in Mexico City,” and Mexico cancels DST between now and then, which it did, in 2022, the meeting moves. The number of hours from now until that meeting changes after you saved it. The cardinal rule, the only thing that saves you, is this: store UTC and the IANA zone name separately, never store a local timestamp alone, and recompute on display.
4. JavaScript’s
Dateobject. It’s not great but I worte more about it here, There is a fix called the JavaScript Temporal API. It’s technically here now, though browser support is still rolling out, and we will all be happier when it’s fully supported everywhere.
The list of lies (about time)
There’s a famous post called Falsehoods Programmers Believe About Time, and a partial sample from the time-zone section gives you the texture:
- “There are 24 time zones.” (38+, give or take, depending on the week.)
- “A day is 24 hours.” (DST transitions make some days 23 or 25.)
- “Time zones don’t change.” (They change several times a year.)
- “If I store the UTC offset I don’t need the zone ID.” (You do, for any future date.)
- “UTC is a time zone.” (UTC is a time scale. Zones are offsets from it. This distinction is going to matter more than it sounds like it should.)
All are gotchas that programmers encounter when working with time zones.
The ultimate example of technical debt
The time zone system isn’t broken, per say… it’s working exactly as designed.
It was designed by railroad executives in 1883, ratified by diplomats in 1884, and then handed off to every country on Earth to amend at will. Every president who has ever moved a DST date for political reasons, every dictator who has ever changed the national offset to flatter a neighbor, every parliament that has voted to abolish daylight saving without specifying when, has added their fingerprint to the IANA database.
It is a working international system. It is also a Rube Goldberg machine running on a tar pit, held aloft by Paul Eggert and a mailing list.
If you ever thought Timezones were bad, tomorrow it gets worse. We’re going to talk about Daylight Saving Time, and the specific, named software disasters it has caused.
Sources
- Nepal Standard Time — Wikipedia. Details Nepal’s 1986 decision to anchor civil time to the Gauri Shankar meridian (
UTC+5:45). - Time in Kiribati — Wikipedia. Covers the 1995 shift of the International Date Line, creating the 26-hour global spread.
- Time in North Korea — Wikipedia. Documents the geopolitical shifts between
UTC+8:30andUTC+9. - Day of Two Noons — Wikipedia. History of American railroads unilaterally standardizing time on November 18, 1883.
- International Meridian Conference — Wikipedia. The 1884 agreement that established Greenwich as 0° (and France’s holdout until 1911).
- IANA Time Zone Database — Wikipedia. History of the Olson database, its maintenance by Arthur David Olson and Paul Eggert, and its fundamental role in modern computing.
- Daylight saving time in Mexico — Wikipedia. Details the national abolishment of DST in October 2022.
- JavaScript Temporal API — TC39 Documentation. The modern fix for JavaScript’s notoriously broken
Dateobject (currently rolling out to browsers). - Falsehoods Programmers Believe About Time — Noah Sussman’s canonical post detailing the myriad ways developers misunderstand time.
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
-
Day 16: How the World Agreed on a Date Format (Except the US)
There is a war that has been quietly raging for about a century, and it is fought over six little characters.
05/06/26In the United States, that is May 6, 2026. In most of Europe, it is June 5, 2026. In Japan, which uses a year-month-day order and frequently uses the imperial calendar, the
05might be read as year 5 of the Reiwa era (2023). In Iran, the entire premise is wrong, because Iran’s official calendar is the Solar Hijri and the current year is 1405.So here we are. A date written by one person and parsed by another is, in the general case, an act of faith.
The most consequential standards effort of the late 20th century was an attempt to end this. It succeeded, sort of. The format it produced,
2026-06-08T14:30:00Z, looks unremarkable now, but it represents a multi-decade campaign to drag the world’s date conventions into a single, unambiguous, machine-parseable shape.That standard is ISO 8601, and the story of how it won is the story of why your API logs look the way they do.
What’s actually wrong with
05/06/26Let me give you the tick of the tock (lay of the land). Every culture has a different intuition about which number comes first in a date, and none of them is wrong.
In the United States, the convention is month-day-year. This descends from spoken American English, “May sixth, twenty-twenty-six,” where the month comes first in speech.
In most of Europe, Latin America, and much of Asia, the convention is day-month-year. “The fifth of June, twenty-twenty-six.” The day is the first specific element.
In East Asia, the convention is year-month-day, written largest-unit-first. This reflects a linguistic preference for going general-to-specific that runs the opposite direction of the English phrasing.
Who is to say which is more correct than another? The problem is that a single string of digits separated by slashes can mean three different things depending on who wrote it, and there is no way other way to tell.
When the ambiguity bites
It’s not just an annoyance.
International travel figured this out the hard way. Across global passport documentation, the convention settled on a three-letter month abbreviation:
08 JUN 2026. It’s unambiguous because no month is named06. The international passport standard (ICAO Doc 9303) mandatesJANthroughDECfor exactly this purpose.In healthcare, patient safety organizations have flagged date ambiguity as a documented source of medication error: a chart that says
7/8/09can be read as July 8 by one clinician and August 7 by another.The shape of the problem is the same across medicine, aviation, logistics, contracts, and customs declarations. Different conventions lead to confusion and errors.
The standard
In 1988, ISO published
ISO 8601:1988. Pick one format, make it unambiguous, make it sort lexicographically, make it machine-parseable, and standardize the world on it.The format they picked:
2026-06-08T14:30:00ZThe choice of
YYYY-MM-DDwas deliberate. Year-month-day is the East Asian convention, but it has a technical property that the other two don’t: it sorts correctly as a string.2025-12-31comes before2026-01-01whether you sort by character or by number.12/31/25and01/01/26do not. For the emerging computing industry of the late 1980s, databases, log files, file systems, this was a decisive advantage.The capital
Tseparates the date from the time. Not pretty, but unambiguous. The trailingZ(informally pronounced “Zulu”) means UTC. This timestamp has no timezone offset, it is anchored directly to UTC.What actually use: RFC 3339
ISO 8601 is too permissive for engineering use.
It allows fractional seconds. It allows omitting components. It allows the basic form (
20260608T143000Z) without separators. It allows week dates and ordinal dates. It allows24:00:00as midnight (this was removed in 2019, then reinstated by amendment, in one of those standards-committee compromises that satisfies no one).So in 2002, the IETF published
RFC 3339. RFC 3339 is a profile of ISO 8601, a strict subset that picks one form and forbids the rest. The basic form is disallowed. Week dates are disallowed. The time component is mandatory. The timezone designator is mandatory.This is what every modern internet API actually uses. GitHub, AWS, Stripe, Cloudflare, OpenAI. They accept RFC 3339, not full ISO 8601. They reject
20260608T143000Zeven though it’s legal ISO 8601.What everyone calls “ISO 8601” in casual conversation is, almost always, RFC 3339.
What ISO 8601 isn’t
A few things worth being clear about:
- ISO 8601 is not UTC. UTC is a timescale. ISO 8601 is a format.
- ISO 8601 is not Unix time. Unix time is the integer
1781055000. ISO 8601 is the string2026-06-08T14:30:00Z. They can represent the same instant. They are not the same thing. - ISO 8601 does not solve leap seconds. The format permits
:60in the seconds field, but what to do with such a value is implementation-defined. - ISO 8601 does not include the calendar system. It assumes the Gregorian calendar. No provision for Islamic, Hebrew, or Buddhist calendars.
The civilizational payoff
There is a sense in which
2026-06-08T14:30:00Zis the most consequential string format in modern computing.While legacy systems still cling to their own formats—HTTP headers use RFC 1123, Git and JWTs use integer Unix timestamps, and X.509 certificates use ASN.1—RFC 3339 has conquered the modern web. It is the default serialization for datetime objects in modern programming languages. It appears in the JSON payloads of almost every modern API (GitHub, Stripe, AWS, OpenAI). It is the standard format for XML’s
xs:dateTime. It is written into millions of cloud infrastructure log lines every second.It is the closest thing modern technical infrastructure has to a universal vocabulary for the question “when did this happen?”
It won because it was unambiguous and sortable, and a single committee was willing to pick one of three equally valid cultural conventions and tell the other two cultures to deal with it. Most international standards die in the negotiation. ISO 8601 survived because the technical advantages of
YYYY-MM-DDwere strong enough to overwhelm the political cost.Us Americans haven’t adopted it (yet). We still write
06/08/2026on bank checks, forms and filings, but the machines we all use are on 8601 and they are doing most of the talking.
Sources
- Japanese era name — Wikipedia — Reiwa began 1 May 2019; Reiwa 5 = 2023.
- Solar Hijri calendar — Wikipedia — year 1405 began 21 March 2026, ends 21 March 2027.
- ISO 8601 — Wikipedia — first published 1988; ISO 8601-1:2019 removed
24:00; the 2022 amendment reinstated it. - ISO 8601-1:2019/Amd 1:2022 — the amendment that put
24:00:00back. - RFC 3339 — Date and Time on the Internet: Timestamps — IETF, July 2002. Profile of ISO 8601 used by most modern APIs.
- RFC 3339 vs ISO 8601 — visual map of which forms each standard accepts; basic form (
20260608T143000Z) is valid ISO 8601 but not RFC 3339. - Machine-readable passport (ICAO Doc 9303) — Wikipedia — ICAO standard requiring three-letter month abbreviations (
DD MMM YYYY) in the visual inspection zone of all passports. - ISMP List of Error-Prone Abbreviations — highlights the risk of ambiguous documentation and dates in medical records.
- RFC 1123 — Requirements for Internet Hosts — specifies the required date format for HTTP Date headers.
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
Tomorrow: Unix time, the second-counting system that runs under every timestamp you’ve ever seen, and the rollover problem that hits in 2038.
Programming Software development 30daysoftime Standards Iso8601
-
Day 15: The Man Who Synchronized the World
David Mills, the father of internet time, wrote the protocol that synchronizes every computer on Earth. He did it as a professor at the University of Delaware, on a project he started in the early 1980s and never stopped working on.
The code lives in your laptop, your phone, your router, every cloud server you’ve ever touched, and every satellite in low Earth orbit. The protocol it implements is called NTP. The reason your computer’s clock is correct, right now, within a few milliseconds of UTC, is that Mills spent forty years of his life making sure it would be.
He once described the early ARPANET days as a “sandbox” where researchers were simply told to “do good deeds.” Part of the allure of the time-synchronization work, he told The New Yorker in 2022, was that he was just about the only one doing it. He had his own “little fief.”
For forty years, that is exactly what it was.
The problem
The early internet had a clock problem. As soon as there were enough machines on the network that “what time is it?” didn’t have a single answer, somebody was going to have to write a protocol. Each computer had its own oscillator. Each oscillator drifted at its own rate. Two machines that agreed at noon could be tens of seconds apart by midnight.
Why did this matter? For most things, it didn’t. For some things, it mattered a lot. A file saved on one machine and copied to another could look older than the version it overwrote, confusing every backup tool that assumed time moves forward. Cryptographic handshakes that expire after a few seconds could fail because the two ends disagreed on what “a few seconds ago” meant. Database replicas could apply writes in the wrong order and corrupt their own state. Email between two servers could arrive timestamped before it was sent. Debugging a multi-machine bug meant correlating log entries across clocks that didn’t agree about which event came first.
Mills decided the actual problem was that there was no protocol for negotiating the truth (in this case, time) across multiple systems. The clock on his desk was wrong. Every other clock was also wrong. The question wasn’t “who has the right time?”, it was “given that nobody has the right time and the network adds an unknown delay to every measurement, how does the system converge on a consensus that is closer to UTC than any individual node could achieve alone?”
His first NTP RFC,
RFC 958, was published in September 1985. We now call that protocol NTPv0, or the prototype. In it, Mills nailed down the four-timestamp packet format and the offset/delay math that has been in every revision since. The packet format and the core algorithm haven’t meaningfully changed in forty years. That kind of staying power is rare in any field. In internet infrastructure, where the half-life of a protocol can be measured in single-digit years, it is quite commendable.The four timestamps
NTP’s core insight is that the network delay between client and server can be measured, not just guessed, as long as both sides record their own timestamps for both legs of the conversation. Four timestamps are exchanged in a single round trip:
Client Server ────── ────── T₁ ──── request ───────────────► T₂ T₃ T₄ ◄────────────── response ─────- T₁ — the client sends the request (client clock)
- T₂ — the server receives it (server clock)
- T₃ — the server sends the response (server clock)
- T₄ — the client receives it (client clock)
Now the client has four numbers. T₁ and T₄ are in the client’s reference frame, T₂ and T₃ are in the server’s. From those four numbers, two things fall out: the round-trip delay (how long the conversation took, minus the time the server spent thinking) and the clock offset (how far the client’s clock is from the server’s). The client now knows how wrong it is, and by how much.
The math depends on one critical assumption: the network is symmetric. The packet takes the same time to travel in both directions.
If you’ve been following along in the series, you know there are a lot of ways to measure time. Atomic clocks. GPS receivers. The quartz crystal in your laptop. Radio signals broadcast from government antennas. They don’t all tick at the same rate, and they don’t all agree on what the current time is. How does NTP reconcile across that much varity in time sources?
The stratum hierarchy
NTP organizes the world’s clocks into a tree, with depth measured in strata.
Stratum 0 is the reference. Cesium atomic clocks. Hydrogen masers. GPS receivers. Radio receivers tuned to WWV, DCF77, or MSF. These are not on the network, they’re physical devices wired directly to a small number of computers via PPS pulses on serial ports.
Stratum 1 is the small group of servers wired directly to Stratum 0. There are perhaps a few thousand of these globally. NIST runs some. Major universities run some. The big internet exchanges run some.
Stratum 2 servers sync with Stratum 1, Stratum 3 with Stratum 2, and so on down to Stratum 15. Stratum 16 means “unsynchronized, do not trust.”
A typical Linux laptop syncs against Stratum 2 or 3 servers. A typical cloud VM syncs against its provider’s internal Stratum 1 fleet. Your phone syncs against whatever its carrier provides. The whole tree is held together by NTP itself, recursively.
The genius of the design is that there is no central authority. Mills did not own the protocol. There is no “official NTP server.” Anyone can run a Stratum 1 with the right hardware, and anyone can run a Stratum 2+ by syncing with a few Stratum 1s of their choice. The largest public pool,
pool.ntp.org, is a volunteer effort started in 2003 by Adrian von Bidder. It currently aggregates a few thousand donated stratum-2 servers worldwide and serves several billion requests per day. Nobody is in charge of it. It just works.The slew, not the step
There are three different times to keep track of on every synced computer. The reference time is what UTC says, the truth NTP is chasing. The tick rate is how fast the computer’s oscillator pulses. It’s supposed to produce one second of clock time per real second, but always drifts a little. The system clock is what gets reported when an application asks for the current time. Synchronizing means closing the gap between the system clock and the reference time without breaking anything that depends on the system clock being well-behaved.
NTP’s primary tool for that is the slew: it adjusts the tick rate, making each tick slightly longer or shorter than nominal, so the system clock drifts into alignment on its own. The alternative would be to jump the clock forward or backward by the full offset (a step), which is fast but can produce duplicate keys in a database, expire valid TLS sessions, or cause a logging system to mis-order events.
Mills designed
ntpdto slew conservatively. A 200ms gap might take several minutes to close, and corrections larger than about 128ms would get stepped because slewing them gradually was prohibitively slow. That trade-off worked for the always-on Unix workstations of the 1980s and 90s. It works less well for the modern reality of laptops that suspend for hours and resume with a clock that hasn’t been touched since last Tuesday, or cloud VMs that get migrated between hosts. Modern variants likechronyslew more aggressively for exactly that reason. When you open your laptop lid, you want the clock right now, not after fifteen minutes of imperceptible easing.The legacy
In a sense, NTP is the thing that made the modern internet possible.
Without well-synchronized clocks, you cannot have SSL certs. The browser needs to know when the cert expires, and if its clock is off by more than a few minutes, the encryption breaks. The same goes for databases. No matter the type, NoSQL or otherwise, they all depend on a clock to record when an operation took place.
Without NTP, cell towers wouldn’t agree on when to hand off a call. Financial transactions wouldn’t be enforceable. And all those log files you’ll totally read one day wouldn’t make any sense. NTP is foundational to all of it. It runs as a daemon on every machine, the ones you stare at all day, the ones you don’t see, and the ones you don’t care about.
We remember Mills as the internet’s “Father Time” and the man who synchronized the world. Neither is a metaphor.
Sources
- In Memoriam: David Mills (UDaily, March 2024) — University of Delaware’s obituary; biographical detail, career timeline.
- David L. Mills — Wikipedia — congenital glaucoma from birth, vision worsening from ~2012, fully blind by 2022; UDel professor 1986–2008.
- David Mills, the internet’s Father Time, dies at 85 — The Register — death date (Jan 17, 2024), age 85.
- RFC 958 — Network Time Protocol (September 1985) — the original NTPv0 specification.
- Network Time Protocol — Wikipedia — version lineage: RFC 958 (v0, 1985), RFC 1059 (v1, 1988), RFC 1119 (v2, 1989), RFC 1305 (v3, 1992), RFC 5905 (v4, 2010), RFC 8915 (NTS, 2020).
- NTP pool — Wikipedia — Adrian von Bidder started the pool in January 2003; Ask Bjørn Hansen has run it since 2005.
- MiFID II RTS 25 clock synchronization (Meinberg) — 100µs requirement for high-frequency trading at sub-1ms gateway latency.
- A Brief History of NTP Time: Confessions of an Internet Timekeeper (Mills, PDF) — Mills’ own history of NTP.
- The Thorny Problem of Keeping the Internet’s Time (The New Yorker, September 2022) — Nate Hopper’s profile of David Mills and the fragile state of NTP maintenance.
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
Tomorrow: ISO 8601, the format wars, the carnage of MM/DD vs DD/MM, and why
2026-06-07T14:30:00Zwon. -
Day 14: The Bug That Didn't End the World, and the One That Still Might
On December 31, 1999, a measurable percentage of the developed world stockpiled bottled water, withdrew cash from ATMs, and stayed up to see if the lights would go out at midnight.
They didn’t. Planes did not fall from the sky. Power grids did not collapse. Bank balances did not reset. The new millennium arrived, the champagne was opened, and by January 3rd everyone agreed it had been a hoax.
It was not a hoax. It was a save.
Y2K was a global $300+ billion engineering effort spread across roughly five years and almost every government and Fortune 500 IT department on Earth. The reason nothing happened on January 1, 2000 is that for half a decade, an enormous number of people worked very hard so that nothing would happen. The bug was real. The fix worked. Most people forgot it was ever a problem.
Twelve years from now, a structurally identical bug detonates again, but first let’s understand what happened in ‘99.
Y2K: the bug
The Y2K bug is almost embarrassingly simple. From the 1960s through the 1980s, computer storage was expensive enough that programmers had a habit of representing years with two digits,
99instead of1999,73instead of1973. It saved two bytes per date. Across a payroll system tracking millions of employees, that mattered.The assumption baked into that decision was: we’ll have rewritten this system long before the century rolls over.
This is the most consistently wrong assumption in software engineering. Code outlives its authors’ confidence. By the late 1990s, vast amounts of critical infrastructure, bank ledgers, airline reservation systems, hospital records, utility billing, social security disbursement, military logistics, nuclear plant monitoring, were running on COBOL programs from the 60s and 70s that had been patched but never rewritten. The language is unfamiliar to many but the fix it later approach is relable to everyone. The developers at the time all quietly assumed that the year
99was less than the year00.When the rollover hit,
99-12-31 + 1 day = 00-01-01looked, mathematically, like jumping back to 1900. Interest calculations would compute negative ages. Pensioners would suddenly be billed for a century of debt. Reservation systems would mark every upcoming flight as having departed in the past. Insurance policies would expire en masse.The reason planes did not fall is that, starting roughly in 1995, every major airline, manufacturer, FAA system, and air traffic controller began an exhaustive audit-and-fix campaign.
The reason the power grid did not collapse is that every utility company in North America and Europe ran the same campaign on their SCADA systems.
The reason your bank balance was still correct on January 1, 2000 is that someone, somewhere, spent late nights in 1997 reading printouts of code written before they were born.
The estimated total global cost: $300 to $600 billion. The amount of measurable damage on January 1, 2000: small enough that people argued for the next decade about whether the spend had been justified.
It was. The bug was real. The fix worked. The result of a successful preventive engineering campaign is that it looks, in retrospect, like the problem was never there.
Y2038: the same bug, different number
Twelve years from now, specifically, January 19, 2038, at 03:14:07 UTC, a structurally identical bug fires for a different reason.
Unix time is stored, on a huge amount of legacy infrastructure, as a signed 32-bit integer. That gives you about 2.1 billion seconds of positive range from the 1970 epoch. 2.1 billion seconds is 68 years. 1970 + 68 = 2038.
At
03:14:07 UTCon that date, the counter hits its maximum value,2,147,483,647. The next tick overflows. In two’s-complement signed integer arithmetic, the value rolls over to its most negative possible value:-2,147,483,648. Interpreted as a Unix timestamp, that’s December 13, 1901.Every 32-bit Unix-derived system that hasn’t been patched will, in the span of one tick, conclude that it is now the early 20th century. The effects are the same family of effects as Y2K, but applied to a much wider deployment surface.
File modification times become nonsensical. SSL certificates appear expired, or worse, not-yet-valid. NTP synchronization fails. Filesystems with 32-bit inode timestamps lose ordering. Embedded device firmware that schedules tasks based on wall-clock time begins executing at random intervals. Industrial control systems that latch state machines on “time since last event” calculations latch on negative durations and either freeze or behave unpredictably.
Modern desktop and server operating systems are mostly fine. Linux finished migrating to 64-bit
time_ton all architectures by kernel 5.6 (2020) and glibc 2.32. macOS and Windows have been 64-bit-clean for over a decade. AWS, GCP, and Azure all run 64-bit kernels.The problem is not where you are reading this. The problem is in the physical world that keeps everything running.
The long tail is enormous
Estimates of the number of currently deployed 32-bit embedded devices that interact with
time_tin some way range from a few hundred million to several billion.Industrial controllers, automotive ECUs, network routers, smart-meter firmware, point-of-sale terminals, medical imaging devices, GPS units, cable boxes, elevator controllers, traffic light systems, ATM internals, payment terminals, building HVAC, water-treatment SCADA, satellite firmware, oil rig control systems, and the embedded computer in your refrigerator.
Each one, depending on vintage and vendor, may or may not have been patched.
Many of these devices are not internet-connected and cannot be patched remotely. Many are running firmware whose source code has been lost. Many are running firmware whose vendor no longer exists. Many are in places where physical access is hard, a deep-sea oil platform, a satellite in geostationary orbit, a controller welded inside an industrial machine.
The Y2K fix worked because the affected systems were largely centralized: mainframes in data centers, software at named companies, code with active maintainers. You could audit it. You could rewrite it. You could ship a patch.
Y2038 is decentralized. The affected systems are everywhere.
The Buff Must Flow
In 2022, Microsoft Exchange Server stopped delivering email worldwide. The cause was a 32-bit signed integer in Exchange’s anti-malware scanner that stored the date as a long-form number. On New Year’s Day, the value tipped over the limit and the scanner refused to load. Mail queues backed up everywhere. Microsoft shipped an emergency script the next day. They called it Y2K22.
On April 6, 2019, the GPS week number counter rolled over. The failure mode was familiar, an integer designed when the engineers thought it was going to be big enough turned out, decades later, not to be. NYC’s municipal wireless network went down. KLM grounded a flight. Older car and marine GPS units showed dates in 1999.
Two examples of overflows hitting production and breaking real things. Y2038 will be every one of those at once, in places nobody is thinking about.
Y2038 is foreseeable. We know about it. We know what needs to be done. We have twelve years. We should get started sooner rather than later. A lot of important systems need to be replaced, and the fewer that fall through the cracks, the better.
There’s no checklist for the devices we’ve already forgotten about, but maybe there should be.
Tomorrow: The Smear, how Google, Amazon, and Meta quietly decided to stop telling the truth about leap seconds, and why everyone else followed.
Sources
- Year 2000 problem — Wikipedia
- Year 2038 problem — Wikipedia
- Microsoft Exchange year 2022 bug in FIP-FS breaks email delivery — BleepingComputer
- Microsoft Exchange Fixes Disruptive ‘Y2K22’ Bug — BankInfoSecurity
- GPS week number rollover — Wikipedia
- GPS Week Number Rollover — GPS.gov
- The impact and resolution of the GPS week number rollover of April 2019 — Geoscientific Instrumentation (Copernicus)
- Linux kernel 5.6 — 64-bit time_t support for 32-bit architectures (KernelNewbies)
- The Open Group Base Specifications: time.h
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
-
Day 13: Unix Time, 1,780,620,532
That’s roughly what time it is, right now, as I type this.
Not 8:48 PM. Not “Thursday.” Not “June 4th, 2026.” None of those are what your computer thinks “now” is. To your laptop, your phone, your car’s infotainment system, the streaming server pushing this page to your browser, and the ATM in the corner store, now is a number. A 10-digit integer. Counting up, one tick per second, since a fixed moment in 1970.
That number runs the world. It’s the closest thing the global computing infrastructure has to a heartbeat. And it has some weird properties, almost none of which are explained by the name it goes by.
Unix time.
The clock under every clock
Open a terminal. Type
date +%s. You’ll see something like1780620532come back. That’s Unix time. Seconds since the Unix epoch,1970-01-01T00:00:00 UTC.Every modern operating system tracks time this way internally, even if it dresses up the output for you. The pretty “8:48 PM” on your menu bar is a calculation: take the current Unix timestamp, apply your timezone offset, run it through the calendar rules, format it for display. The underlying number is just
1,780,620,532-and-change, counting up.JavaScript’s
Date.now()? Unix time in milliseconds. Java’sSystem.currentTimeMillis()? Unix time in milliseconds. Python’stime.time()? Unix time as a float. Go’stime.Now().Unix()? Unix time. PostgreSQL’sEXTRACT(epoch FROM ...)? Unix time. SQLite’sstrftime('%s', 'now')? Unix time.It’s the lingua franca of computing. Two systems written in different languages, on different continents, with different calendars in their UIs, agree about what now means because they both agree about this one number.
Why 1970?
The honest answer is: convenience.
In the early 1970s, Ken Thompson and Dennis Ritchie were building Unix at Bell Labs. They needed a way to represent time on a 32-bit machine. Their first attempt counted 1/60 of a second per tick in a 32-bit integer, and overflowed in about two and a half years. So they switched to 1 tick per second, which gave them roughly 136 years of range in a signed 32-bit integer.
Then they needed a zero. They picked
1970-01-01because:- It was recent enough that the historical calendar mess (Julian vs. Gregorian, the dropped days in 1582, the year that started in March) was someone else’s problem.
- It was round.
- It predated every Unix system anyone cared to represent.
- It was conveniently close to UTC’s formalization a couple of years later.
That’s it. There’s no cosmological significance to 1970-01-01. It’s not aligned with any astronomical event. It’s the timestamp equivalent of
git init. We’ll start counting from here, and we’ll figure the rest out later.The “later” turned out to mean everywhere.
The thing that isn’t there: leap seconds
The computer’s time problem mostly comes from UTC.
Unix time is defined as the number of seconds since the Unix epoch. You might reasonably assume that if I have two timestamps, the difference between them is the actual number of physical seconds that elapsed between those two moments.
It is not.
Unix time does not count leap seconds. Since 1972, the IERS has inserted 27 leap seconds into UTC, extra seconds added to keep civil time aligned with Earth’s slowing rotation. Unix time pretends they never happened. The Unix clock has, over its 56-year lifetime, “lost” almost half a minute relative to reality.
Even weirder: during the actual leap second, when UTC ticks
23:59:59 → 23:59:60 → 00:00:00, Unix time has to do something. POSIX doesn’t specify what. So implementations have invented three different answers:- Repeat the second. The clock shows
23:59:59for two real seconds and then jumps to00:00:00. Two distinct physical moments share the same timestamp. File mtimes can collide, log entries can appear out of order. - Insert the second. The clock briefly shows
23:59:60, which is a valid UTC string but breaks every parser that assumes seconds run 00–59. Linux kernels do this. Hilarity ensues at midnight. - Smear it. Don’t insert the second at all. Slow every clock down by a tiny fraction over a 24-hour window so it absorbs the missing second smoothly. Google does this. Amazon does it. Facebook does it.
So “Unix time” in 2026 means three subtly different things depending on whether your server is running stock Linux, smeared Google time, or one of the dozens of variants in between. Two timestamps from two providers may disagree by a second, and both are correct under their own definitions.
That’s what the spec authors call “implementation-defined behavior” and what the rest of us call “why distributed-system logs don’t line up.”
The number is also a string
Integers are easy for computers but humans expect a string. Unix time is the easiest timestamp format to compare, sort, and store because it’s an integer, but as soon as we convert to human-readable format, all that changes.
To find out which one is earlier, subtract. To sort a million events, sort the integers. To store one efficiently, write 8 bytes. To send one over the network, send 8 bytes.
Compare this to a full ISO 8601 timestamp like
2026-06-04T16:47:23.512847+00:00. That’s a 32-character string that needs to be parsed, validated, normalized for timezone, and converted to a comparable representation before you can do anything with it. Every comparison is a parsing pass. Every storage is 4× the bytes. Every sort is a string sort with calendar rules.Unix time is fast. It’s so fast that even formats designed to replace it (Google’s Spanner, AWS’s KSUIDs, Twitter’s Snowflake) embed Unix-like millisecond counts at their core and just append entropy bytes around them.
The ubiquity isn’t an accident. It’s the natural result of picking the representation that’s cheapest at every step.
The Untimes
Unix time is a convention that has eaten the world.
It’s anchored to UTC, which means it inherits UTC’s quirks. It’s embedded controllers in cars, industrial equipment, network gear, satellite firmware, gas pumps, so pretty much every piece of modern infrastructure.
1,780,620,532is just a number, a timestamp. It’s used by your bank for transactions, used by your file system for its files, but also it’s a hack. A 56-year-old dart in the board of of time, that ignores leap seconds, depends on UTC, has three different definitions during the same physical second, and we built the entire internet on top of it.Tomorrow will be on what happens when the bill comes due. Y2K and Y2038, the bug that didn’t end the world, and the bug that still might.
Sources
- Unix time — Wikipedia
- Leap second — Wikipedia
- Coordinated Universal Time — Wikipedia
- International Earth Rotation and Reference Systems Service — Wikipedia
- Leap Smear — Google Developers
- Look Before You Leap — The Coming Leap Second and AWS
- It’s time to leave the leap second in the past — Engineering at Meta
- How Precision Time Protocol handles leap seconds — Engineering at Meta
- Leap second bug cripples Linux servers at airlines, Reddit, LinkedIn — The Register
- Resolve Leap Second Issues in Red Hat Enterprise Linux
- History of Unix — Wikipedia
- Snowflake ID — Wikipedia
- ksuid — segmentio (GitHub)
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
-
Day 12: Earth to Mars, Come In
NASA’s Mars mission control runs on Mars time, not Earth time.
For the first few months of every landed mission (Curiosity, Perseverance, Insight), the scientists shift their work schedules by 39 minutes every day.
They go to sleep 39 minutes later, wake up 39 minutes later, do all their meetings 39 minutes later than yesterday.
After a few weeks they’re working at 3 AM Earth time. After a month they’re working at noon. After two months they’re back to where they started.
This is what living on Mars time looks like to humans. It is exhausting and demonstrably bad for sleep. They do it anyway because the rovers don’t care about Earth’s schedule.
Let’s talk about why time on Mars is hard.
The sol
A Martian solar day is called a sol, and it’s about 24 hours, 39 minutes, 35 seconds.
That’s almost an Earth day. Close enough that NASA’s first instinct in the 1970s was to use Earth time for Mars missions anyway.
Bad idea. Within a single Martian month, your Earth-time schedule drifts nearly a full day out of phase with the local Martian one.
The rover’s morning camera shots happen during your meeting. Its solar panels charge while you’re trying to sleep.
So NASA started using Mars time for the human side of mission ops. They built Mars-time wristwatches in the 1990s, actual mechanical watches modified to run 2.7% slower, and gave them to mission controllers.
They built Mars-time-aware mission scheduling software. They renamed “day” to “sol” so nobody got confused.
It worked. It was also brutal on the humans, because human circadian rhythms evolved for Earth’s 24-hour day, not Mars’s 24-hour-39-minute one.
Mars time shifts your sleep 39 minutes later every day, which is roughly equivalent to flying west across two time zones, every day, forever.
After a few months I can see everyone getting their schedules wrecked.
Each rover has its own time zone
Mars doesn’t have just one time. Each rover gets its own time zone, called Local Mean Solar Time (LMST), based on its specific longitude on Mars.
Curiosity is in Gale Crater. Perseverance is in Jezero Crater. They are about 3,700 kilometers apart on Mars, and they keep local solar times that differ by about four hours.
Curiosity is roughly four hours later in its day than Perseverance, because Gale Crater is 60 degrees east of Jezero.
If you’re a rover and you want to know when the sun will rise tomorrow, you use your LMST, not the other rover’s.
There’s also a coordinated reference: Coordinated Mars Time (MTC), anchored to Airy Crater, which is roughly the Mars equivalent of Greenwich.
Airy is where Mars’s prime meridian sits by IAU convention, and MTC is the mean solar time at that location. Most planetary scientists default to MTC when reporting Mars events without specifying a longitude.
So Mars has its own day (the sol), its own time zones (LMST per location), its own Greenwich (Airy Crater), and its own UTC-analog (MTC).
The whole planet has built up a parallel set of timekeeping conventions, derived from the same basic problem Earth solved: a rotating body needs a way to talk about when things happen.
The Moon is being figured out right now
In 2024, the White House Office of Science and Technology Policy directed NASA to establish Coordinated Lunar Time (LTC) by 2026.
The Artemis program needs it. So does every commercial lunar lander launching this decade: SpaceX, Blue Origin, ispace, Astrobotic.
All of them need to coordinate communications, navigation, and surface operations on the Moon, and they need a shared time standard to do it.
The Moon’s timekeeping problem is harder than Mars’s, for two reasons.
First, the Moon’s day is 29.5 Earth days long (synodic). A lunar “noon” lasts 14 Earth days, and so does the night.
The whole concept of “day” as a unit of human activity falls apart. Lunar mission ops will likely use Earth-anchored time for everything and ignore the local sun.
Second, relativity matters more than you’d think. Clocks on the Moon run about 58 microseconds per day faster than clocks on Earth’s surface, due to the Moon’s weaker gravity well.
That’s bigger than GPS’s 38 µs/day. If you want a lunar communication network to synchronize with Earth-side networks, you have to bake in the correction the same way GPS did, but more aggressively.
LTC is being designed right now. The current proposal is an atomic timescale traceable back to TAI, with relativistic corrections applied at the lunar surface.
It will probably be ready before Artemis 3 lands humans?
Deep space and the light-delay problem
Beyond the Moon and Mars, time becomes a different problem entirely: light delay.
- Round-trip to Mars: 6 to 44 minutes, depending on orbital geometry
- Round-trip to Jupiter or Saturn: hours
- Round-trip to Voyager 1, currently 24 billion kilometers from Earth: about 46 hours
You can’t run NTP to a spacecraft beyond the Moon. The sync protocol assumes round trips of milliseconds, and the universe doesn’t oblige.
The Deep Space Network (NASA’s array of giant antennas at Goldstone, Madrid, and Canberra) sends time-tagged commands to spacecraft, and spacecraft tag their telemetry with their own onboard atomic clocks.
The clocks have to be reliable for years or decades without correction, because by the time a round trip resolves, you’ve moved on to the next problem.
For interplanetary work, physicists use three relativistic coordinate timescales:
- TCB (Barycentric Coordinate Time): a clock that lives at the center of mass of the solar system. The natural frame for tracking planets, asteroids, comets.
- TCG (Geocentric Coordinate Time): a clock at Earth’s center. The frame for tracking Earth satellites and orbital mechanics close to Earth.
- TT (Terrestrial Time): a clock on Earth’s geoid. What UTC is derived from. What humans live in.
TCG drifts about 22 milliseconds per year from TT. TCB drifts nearly half a second per year from both.
Most humans never encounter this.
Anyone doing calculating planetary calculations does.
The deeper point
“The day” is parochial. It works only on the body where it’s defined.
Earth time scales fine on Earth. GPS scales fine in Earth orbit. Mars time scales fine on Mars. None of them scale to each other.
Every body in the solar system has its own “now,” and there’s no single instant that applies everywhere at once.
It’s a consequence of relativity. Time literally runs at different rates at different gravitational potentials and different velocities.
A clock at the solar system barycenter ticks differently than a clock on Earth. There is no “true” rate. There are only frames.
So how do we coordinate? The standard answer, for spacecraft and astronomers, is to pick a coordinate frame, anchor everything to it, and convert as needed at the destination. TCB for solar-system work. UTC for Earth civilians. GPS for navigation. LTC (coming) for the Moon. MTC for Mars.
The clocks themselves are “easy” but the coordination between them is the not.
The civilian question, what time is it if I want to call my friend on Mars, has no clean answer.
You pick a coordinate frame, you both agree to use it, and you live with the conversion. There’s no “Mars time on your phone” because there’s no Mars infrastructure to sync your phone with.
And even if there were, you’d still have to handle the light delay.
Tomorrow we come back to Earth, and to the most ubiquitous time format in human history: the number of seconds since midnight, January 1, 1970.
Sources
- Timekeeping on Mars — Wikipedia
- Coordinated Lunar Time — Wikipedia
- Barycentric Coordinate Time — Wikipedia
- Geocentric Coordinate Time — Wikipedia
- Terrestrial Time — Wikipedia
- Curiosity rover — Wikipedia
- Perseverance rover — Wikipedia
- InSight — Wikipedia
- NASA Deep Space Network — Wikipedia
- Voyager 1 — Wikipedia
- Clockmaker Helps Mars Rover Keep Mars Time — NPR
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
-
Day 11: What If We Put Clocks in Space?
In 1977, three years before GPS launched, the engineers building the satellites had to make a decision.
The clocks they were about to put in orbit were going to run faster than the clocks on the ground. By about 38 microseconds per day.
That sounds like nothing, but over 24 hours of GPS operation, an uncorrected clock would put you 11 kilometers off your actual position.
They had two options:
- Adjust the time signal on the ground, applying the correction as the data came back down.
- Pre-tune the clocks on the satellites to run slow by exactly the right amount, so that by the time relativity sped them back up, they’d tick at the right rate.
GPS chose option two.
They built the clocks to run at 10.22999999543 MHz instead of the nominal 10.23 MHz, so that orbital relativity speeds them up to ~10.23 MHz by the time the signal hits your phone.
The correction is baked in.
That’s what putting clocks in space looks like. One decision, and now everyone on Earth gets both navigation and time from the same signal.
This post is about the impact of that decision.
Why Clocks in Orbit Run Faster
Two relativistic effects act on a GPS satellite clock, and they push in opposite directions.
Special relativity slows the satellite clock down because it’s moving fast. General relativity speeds it up because it sits in weaker gravity than the ground. Gravity wins. Net result: the satellite clock gains about 38 microseconds per day.
Sounds like nothing. But uncorrected, that 38 microseconds drifts your GPS position by 11 km in 24 hours. Within a day of launch, GPS would be useless for anything more precise than “are you in the right country.”
This was known before launch. It was tested. It works.
What GPS Time Actually Is
GPS time is its own scale, started at midnight on January 6, 1980, and ticking continuously since. No leap seconds. No time zones.
The relationship to the other scales is fixed and simple:
GPS = TAI − 19 seconds (constant since launch) GPS = UTC + 18 seconds (today)GPS−TAI never changes. GPS−UTC grows every time UTC gets a leap second, and freezes after the 2035 leap-second abolition.
It is, in every meaningful sense, the most accurate clock in your daily life. And you’ve never seen it.
What It’s Used For
GPS time runs almost everything that needs precise timing in modern civilization, but it’s invisible because nobody consumes it directly.
- Finance. US and EU regulators (MiFID II, SEC) require trading firms to timestamp orders to microsecond precision. GPS-disciplined oscillators are how.
- Telecom. Cellular base stations need their carrier frequencies aligned across the network. GPS clocks them. Without GPS, your phone would struggle to hand off between towers.
- Power grid. Phasor Measurement Units monitor the AC waveform across the entire grid, synchronized to GPS. This is how grid operators detect instabilities before they cascade into blackouts.
- Datacenters. Stratum-1 NTP servers are typically GPS-disciplined. Every clock you’ve ever checked on a computer ultimately traces back, through several layers of network sync, to a GPS receiver in someone’s rack.
- Aviation, surveying, autonomous vehicles, drones, scientific instruments, particle physics. Anything built since 1995 that needs accurate timing or positioning, which is essentially everything.
The civilian world runs on GPS time. It just doesn’t admit it.
What It Didn’t Solve
Putting clocks in space solved navigation.
It did not solve timekeeping.
Your watch is still on local time. Your calendar uses civic dates with leap seconds buried in the UTC. You’re reading a clock face anchored to a Roman calendar, a Babylonian 24-hour day, and an Earth rotation that nobody can predict.
GPS time is great if you are a satellite, a financial trader, a power-grid engineer, a fighter jet, or a cell tower.
It is not great if you are trying to know what time to pick up your kid from school.
For that, you still need wall time, which still needs UTC, which still needs leap seconds, which still needs Earth’s wobbling rotation.
We built absurdly precise atomic clocks. We launched them into orbit. We baked relativity corrections into the silicon. We covered the planet in time signals accurate to nanoseconds.
And your meeting is still at 3 PM on Tuesday.
GPS quietly handles the part it needs to handle. But all of this assumes you’re on Earth.
Where This Goes
Earth orbit needs relativity corrections. The Moon needs more. Mars needs different ones still.
The further you get from Earth, the more “GPS-style time” stops being a solution.
Tomorrow: if an hour is an Earth measurement, so how do you tell time on a planet that doesn’t have them?
Sources
- Error analysis for the Global Positioning System — Wikipedia
- GPS time — Wikipedia
- Schriever Space Force Base — Wikipedia
- Phasor measurement unit — Wikipedia
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
Science Infrastructure 30daysoftime Timekeeping Gps Relativity
-
Day 10: The Zero Point
Three epochs quietly run the world:
- The Unix epoch. Midnight, January 1, 1970. Almost every computer measures time as seconds since this instant.
- The GPS epoch. Midnight, January 6, 1980. Every GPS satellite, every navigation chip in every phone, measures time as seconds since this instant.
- The astronomical epoch (J2000.0). Noon, January 1, 2000, Terrestrial Time. Almost every star catalog, planetary orbit calculation, and space mission uses this instant.
Three different zeros. Three different conventions. None of them line up with anything you’d find on a calendar. Here is why.
You Can’t Have a Clock Without a Zero
A clock counts intervals. To tell you what time it is right now, it needs to know how many intervals have passed since something. The “since something” is the epoch: a fixed, agreed-upon instant from which all measurement runs.
Most timekeeping systems hide their epoch behind a calendar facade. “April 14, 2026” is meaningful to humans, but underneath, the computer is doing arithmetic on a single integer counted from a particular zero.
The calendar is the friendly mask.
The epoch is the actual machinery.
The Three Big Epochs
Unix Epoch: January 1, 1970, 00:00:00 UTC
Picked in the early 1970s by the engineers building Unix. They needed a zero point for the system’s internal
time_tinteger. 1970 was recent enough to feel current, far enough away to leave room for negative numbers (events before 1970), and round enough to remember.I think that they probably thought, like, well, if time is all relative, then let’s just pick some arbitrary time and it doesn’t matter.
It was an choice, not an astronomical one, just relative to some arbitrary point they decided.
So let me say that again.
The Unix epoch has no relationship to any natural event. It is a convention that, through the pervasive nature of Unix, became the default for all modern computing.
GPS Epoch: January 6, 1980, 00:00:00 UTC
The GPS satellite constellation started broadcasting on January 6, 1980. The epoch was just the moment the system turned on.
Why January 6? Because that’s a Sunday, and the GPS week-counting system uses weeks, and weeks start on Sunday.
The first GPS week is week zero.
GPS time has run continuously from that instant and has never had a leap second adjustment, so it is currently 18 seconds ahead of UTC, a gap that keeps growing.
But more on that in a tomorrow’s post.
J2000.0: January 1, 2000, 12:00:00 Terrestrial Time
This is the astronomers’ epoch, and it’s the most carefully chosen of the three. Notice two things:
- It’s noon, not midnight.
- It’s in Terrestrial Time, not UTC.
Both choices have reasons.
Why noon? Astronomers observe at night. A “day” for an astronomer historically started at noon and ran through the following noon, so a single night’s observation session never straddled a date boundary.
If you started a date at midnight, half the stars you saw last night would log on one date and half on the next.
Annoying for astronomers, so they decided to reduce their suffering by redefining the epoch.
The Julian Date system, introduced by Joseph Scaliger in 1583, runs from noon to noon for this reason.
Noon TT on January 1, 2000 was Julian Date 2,451,545.0 exactly, a perfectly round Julian-Date integer.
Why such a huge number?
Because Julian Dates count days from noon on January 1, 4713 BC, the start of Scaliger’s count.
He picked that year because three big calendar cycles (solar, lunar, and the Roman indiction) all aligned there, and because it sat well before any recorded astronomical observation, so every date in history would be a positive integer.
By noon on January 1, 2000, exactly 2,451,545 days had elapsed.
The “0” at the end of “J2000.0” is a flag for that round number, a clean integer in a counting system older than telescopes.
Why Terrestrial Time and not UTC? Because UTC has leap seconds and Terrestrial Time doesn’t.
TT is the smooth atomic timescale we built two days ago (TAI + 32.184 seconds). Anchor your epoch to UTC and every leap second shifts your historical observations sideways. Anchor it to TT and it stays put. That’s why the canonical zero is in TT.
TAI: 2000-01-01 11:59:27.816 UTC: 2000-01-01 11:58:55.816 TT: 2000-01-01 12:00:00.000 ← this is J2000.0Other Epochs Worth Knowing
A few more that show up in working systems:
- Modified Julian Date (MJD): November 17, 1858, midnight. Used in space-mission control because it drops the leading digits of a full Julian Date, saving bytes in old memory-constrained systems.
- TAI origin: January 1, 1958, midnight UT2. The instant the cesium-coordinated TAI scale started running.
- Year zero of the Gregorian calendar: there isn’t one. The calendar jumps from 1 BC to 1 AD with no year zero in between, breaking date arithmetic across the boundary and serving as a low-grade gotcha in historical software.
The Deep-Time Temptation
Some people, looking at this collection of arbitrary-feeling start points, ask why we don’t just pick something physically meaningful. The formation of the Earth, the formation of the solar system, the Big Bang.
The answer is precision.
We don’t know any of those instants to better than millions of years. Earth formed roughly 4.54 billion years ago, plus or minus 50 million. The solar system, 4.567 billion years ago, plus or minus 1 million. The Big Bang, 13.8 billion years ago, plus or minus 20 million.
A reference epoch that is uncertain to a million years isn’t a reference…
The astronomical zero needs to be knowable to the nanosecond, recoverable in the future from preserved records, and verifiable against real observations.
Of every candidate, J2000.0 is the best at all three.
Modern atomic clocks were running in 2000. Star positions on that day are catalogued.
The exact instant is recorded across thousands of observatories.
If civilization collapses and is rebuilt, J2000 is recoverable from physical artifacts. The formation of the Earth is not.
What the Epoch Is Doing
Pick your epoch and you pick what your system can and can’t represent.
- Unix time can’t go before 1970 without negative numbers, and there is the whole integer-overflow issues after a few centuries.
- GPS time started in 1980 and counts strictly forward. Nothing before is representable.
- J2000.0 sits at the present, so calculations naturally span backwards and forwards by tens of thousands of years with full precision.
The choice of epoch is often the most invisible design decision in a timekeeping system, but it shapes everything downstream.
Some of the strangest bugs in software history, Y2K, the 2038 problem, GPS week rollovers, trace back to picking a zero without thinking about the consequences.
Tomorrow we’ll see what happens when one of those choices has to deal with relativity, gravity, and the curvature of spacetime.
The Gee-Pee-Ess time, and the clocks that ship from the factory wrong on purpose.
Sources
- Unix time — Wikipedia
- GPS time — Wikipedia
- Epoch (astronomy) — Wikipedia
- Julian day — Wikipedia
- Terrestrial Time — Wikipedia
- Year 2038 problem — Wikipedia
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
Programming 30daysoftime Astronomy Timekeeping Computing-history
-
Day 9: A Clock You Can Use to Measure a Mountain
Yesterday I said cesium was about to be deposed. Today’s the deposition.
The successor is the optical lattice clock. It is so precise that, it can detect the altitude difference between two desks in the same office.
That last sentence sounds crazy but Its what optical clocks do.
Here’s how.
More ticks per second
The trick of the optical lattice clock is simple in concept: use a finer ruler.
A cesium atom oscillates at about 9 × 10⁹ Hz (~9 GHz, microwave). That’s how many “ticks” per second the clock has to count to keep time.
A strontium-87 atom has a clock transition at about 4 × 10¹⁴ Hz (~430 THz, visible light). That’s 50,000 times more ticks per second.
Same second. More tick marks on the ruler.
Faster clock makes better time.
Faster clocks are better than slower ones.
Faster clocks make for better timekeeping.
See the difference and why we need to upgrade the second?
Today’s best optical clocks drift about one second over the age of the universe. 13.8 billion years. Our best cesium clocks are nowhere near that, drifting the same second in roughly 300 million years instead. About a hundred times worse.
The magic wavelength trick
So if optical is obviously better, why didn’t we do this first?
Because it’s hard. Two reasons, really.
First, you can’t just point an optical clock at a free-floating atom. Atoms in flight have all sorts of velocity, Doppler-shifting the frequency you measure. You have to hold them still. But the only way to hold an atom still is to grip it with something, and gripping it with a laser changes the energy levels you’re trying to measure. Catch-22.
Second, even if you could pin the atom, you couldn’t count that fast. Microwave at 9 GHz fits inside what electronics can divide down and tick off. Visible light at 400+ THz does not. We had no way to count optical-frequency oscillations until the optical frequency comb showed up in 1999, work that won Hänsch and Hall the 2005 Nobel.
In 1967 when the CGPM picked cesium, none of this existed. The laser was seven years old. Laser cooling wouldn’t be demonstrated until the late 1970s. Frequency combs were 30+ years away. Cesium at 9 GHz wasn’t the best clock you could imagine. It was the best clock you could build.
The fix came in 2003 from a Japanese physicist named Hidetoshi Katori. He proposed trapping the atoms in a standing-wave laser pattern, a 3D optical lattice, like an egg crate made of light, and tuning the lattice laser to a specific frequency called the magic wavelength.
At the magic wavelength, the lattice light affects both energy levels of the clock transition by exactly the same amount. The trap is invisible to the clock. The atoms are held still, but the energy levels they emit at are unperturbed. You get to measure the atomic transition cleanly while the atom sits frozen in midair.
Optical lattice clocks are a beautiful piece of physics. Do not ask me about the math behind them.
Which atom?
Two main candidates have been thinking about for the title of “next SI second”:
- Strontium-87. Run in optical lattices by labs around the world: NIST/JILA in Boulder, NPL in the UK, RIKEN in Japan, SYRTE in Paris. About 1 part in 10¹⁸ uncertainty. Currently the favorite.
- Ytterbium-171. Comparable accuracy, different sensitivity profile (different ways the clock can go wrong, which is good for cross-checking against strontium).
There are also trapped-ion optical clocks, which hold a single ion in an electric field instead of a lattice of neutral atoms. The aluminum-ion logic clock at NIST has hit about 1 part in 10¹⁹. That’s one second of drift in 300 billion years. The amount of drift is older than the universe itself. That sounds pretty good to me.
What this gets you
Optical clocks are sensitive enough to see general relativity at human scale.
What does that mean? It means you can put two clocks on different shelves and watch one tick slower than the other.
Einstein said clocks deeper in a gravitational field run slower than clocks higher up. Near Earth, the effect is about 1 part in 10¹⁶ per meter of altitude. A clock on the floor runs slower than a clock on the table. For most of human history this was a theoretical curiosity. Now it’s a measurement.
In 2022, a JILA strontium clock measured a 1 millimeter height difference as a frequency shift. One millimeter. The thickness of a credit card. The clock could tell which side of the card it was sitting on, from gravitational time dilation alone.
This opened a new field: relativistic geodesy. Put an optical clock at two locations, compare frequencies, and you’ve directly measured the gravitational potential difference between them. Detect underground oil, magma movement, ice sheet melt, anything that shifts mass around the planet.
Cesium can’t do this.
The next “second”
The BIPM is planning to redefine the SI second around an optical transition by 2030. Same dance as 1955: measure the new number against the current cesium standard, freeze it at that precision, declare it the new second.
Four layers of backwards-compatibility instead of three. The fingerprints accumulate.
Where this goes
We’ve now spent three days on how a second is built. The next question is the one the calendar dodges: which second?
You need a reference point. A zero. An origin.
Tomorrow we’ll talk about where you start counting from, and why astronomers picked noon on January 1, 2000.
Sources
- Optical clock — Wikipedia
- Hidetoshi Katori — Wikipedia
- Quantum logic clock — Wikipedia
- Relativistic geodesy — Wikipedia
- Bothwell et al., Nature 2022 — millimeter-scale gravitational redshift
- Gravitational time dilation — Wikipedia
- Jun Ye — Wikipedia
- Epoch (astronomy) — Wikipedia
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].