The thorny problem of maintaining internet time | Catch My Job


In 1977, David Mills, an eccentric engineer and computer scientist, got a job at COMSAT, a satellite corporation based in Washington, D.C. Mills was an inveterate tinkerer: He once built a hearing aid for a girl’s uncle and consulted with Ford on how paper-tape computers could be put into cars. Now, in COMSATMills became involved in ARPANET, a computer network that would become the forerunner of the Internet. Several researchers have already used the network to connect their remote computers and trade information. But the fidelity of that shared data was compromised by a distinct flaw: the machines did not share a single, reliably synchronized time.

Over the decades, Mills has acquired a broad expertise in mathematics, engineering and computing. In the early 1970s, as a lecturer at the University of Edinburgh, he wrote programs that decoded shortwave radio and telegraph signals. Later, mostly for fun, he studied how clocks in an electrical network could wander for a few seconds on a hot summer’s day. (The extent of their drifts depended not only on temperature, but also on whether the grid used coal or hydropower.) Now he concentrated on the problem of keeping time over a remote computer network. Clock time, Mills learns, is the result of an endless search for consensus. Even the times told by the world’s most accurate government-maintained “master clocks” are integral parts of the readings of several atomic clocks. The master clocks, in turn, are averaged to help create the International Civil Time, known as Coordinated Universal Time and initialized as UTC

To solve the problem of time synchronization on ARPANET, Mills created what developers call a protocol — a collection of rules and procedures that creates a lingua franca for different devices. The ARPANET was experimental and capricious: electronics failed regularly and technological misbehavior was common. His protocol sought to discover and correct these misdeeds, creating a consensus about time through an ingenious system of doubt. Mills prided himself on humorous nomenclature, so his system of clock synchronization distinguished reliable “truechimers” from deceptive “fakers.” An operating system called Fuzzball, which he designed, facilitated early work. Mills called his creation the Network Time Protocol, and NTP soon became a key component of the nascent Internet. Programmers followed his instructions when writing timing code for their computers. By 1988, Mills had perfected NTP to the point where he could synchronize the clocks of connected computers that showed vastly different times to within tens of milliseconds—a fraction of the blink of an eye. “I always thought it was some kind of black magic,” Vint Cerf, an internet infrastructure pioneer, told me.

Today, we take global time synchronization for granted. It is crucial to the Internet, and thus to civilization. Vital systems — power grids, financial markets, telecommunications networks — rely on them to keep records and sort cause from effect. NTP works in partnership with satellite systems, such as the Global Positioning System (GPS) and other technologies to synchronize the time on our many online devices. Time kept by precise and closely aligned atomic clocks, for example, can be broadcast via GPS to numerous receivers, including those in cell towers; these receivers can be connected to NTP servers that then distribute the time to Internet-connected devices, almost all of which run NTP (Atomic clocks can also directly submit time to NTP servers.) The protocol works on billions of devices, coordinating the time on every continent. Society has never been more synchronized.

For decades, Mills was the person who decided how the NTP should operate (although he disputes the suggestion that he operated with complete sovereignty). Quirky, prickly, authoritative, and sometimes opaque — “He doesn’t suffer fools gladly,” said one longtime associate — he served as the father of the Internet. But his term is coming to an end. Mills was born with glaucoma. When he was a child, the surgeon managed to save part of the vision in his left eye, and he has always worked using very large computer displays. About a decade ago, his eyesight began to fail, and he is now completely blind. Examining computer code and writing explanations and corrections became maddeningly boring. Drawing diagrams or composing complex mathematical equations is almost impossible.

A few years ago I visited Mills at his modest home in suburban Delaware. He and his wife, Beverly, have lived there since 1986, when Mills became a professor at the University of Delaware, a position he held for twenty-two years until his retirement. As we sat in his kitchen, our conversation was regularly interrupted by an automated voice announcing the time from the next room. The oven and microwave oven clocks were out of sync. Mills, who sports a snow-white beard and wears a charcoal fisherman’s sweater, keeps track of the time for himself using a talking wristwatch that connects via radio signal to the main clock in Colorado.

He led me upstairs to his office, slowly making his way through the house feeling for a series of memorized “navigation points.” At his desk, where the cat lay on top of some crackling radio equipment, Mills sat down at his computer. He used the keyboard to pull out a research paper he was working on, with suggestions for improvements to NTP (He asks his wife and daughter to proofread what he’s typing.) As he used the arrow keys to scroll, the computer spoke aloud. “This memo explores new security and protocol improvements,” the voice said. “Empty.” Content overview. Empty. One. Two. Two points. . . . Three. Three. Four. Four point one. . . .” He soon got lost. “I’m doing what I can using the voice you hear,” Mills said. “But I observe myself and comment on this: man is made to perform an English composition with the eyeball.”

Technology does not stand still. The Internet continues to grow in both scope and complexity; even as its infrastructure ages, our world depends on its functioning to an ever-increasing degree. The continued evolution of the Internet time synchronization system is essential. Yet Mills’s inability to contribute quickly to the NTP weakened his authority over him. In his absence, it seems that only a few people are able and willing to oversee the critical but neglected software. A competition has begun to influence how clocks are synchronized across the internet.

Mills was born in 1938 in Oakland, California, eleven years after the development of the first quartz watch and nine years before the construction of the first transistor. He took a steam train to a school for the visually impaired in San Mateo and admired the engineers who ran it. In his teens, he became a model railroad and ham radio enthusiast, communicating with friends and patching Navy Seabees at the South Pole to their wives. His father, an engineer and salesman, co-founded National Oil Seal, a company that made equipment to prevent leaks inside machinery. (“You may not know what it is, but there are at least two in your car engine,” his father told him, about the stamps.) His mother trained as a pianist at the Toronto Conservatory of Music before staying home to raise him and his two younger brothers.

The family moved around, and Mills’ teachers were not always responsive to his vision impairment. Mills remembers a teacher telling him in eleventh grade, “You’ll never make it to college” — a remark that was “like waving a flag in front of a bull,” he said. In 1971, Mills earned his Ph.D. in Computer and Communication Sciences from the University of Michigan; after a two-year stay teaching in Edinburgh, he moved with his wife and two children to the University of Maryland, which denied him tenure after five years. “It was the best thing that ever happened to me,” Mills said. He started working on COMSATwhere he had access to funds from the Ministry of Defense, some of which were intended for ARPANET. “It was a sandbox,” he later told an interviewer. “They just told us, ‘Do good deeds.’ But the good works were things like the development of electronic mail and protocols.” Part of the appeal of working on time synchronization, he told me, was that he was the only one doing it. He had his “little fief”.

At NTP, Mills built a system that allowed endless tinkering, and he found joy in optimization. “The actual use of weather information was not of central interest,” he recalled. The new internet had several clocks for synchronization. But during the 1980s the network grew rapidly, and by the 1990s the widespread adoption of personal computers required the Internet to incorporate millions more devices than its original designers had envisioned. Coders created versions of NTP that ran on Unix and Windows machines. Others wrote “reference implementations” of NTP—open source codebases that illustrated how the protocol should run and were freely available for users to adapt. Government agencies, including the National Institute of Standards and Technology (NIST) and the US Naval Observatory, began distributing the time kept by their master clocks using NTP

A loose community of people around the world have set up their own servers to provide time through the protocol. In 2000, NTP servers sent eighteen billion requests for time synchronization from several million computers—and over the next few years, as broadband access expanded, requests to the busiest NTP servers increased tenfold. Time servers used to be “well-lit in the US and Europe, but dark elsewhere in South America, Africa and the Pacific Rim,” Mills wrote in a 2003 article.“Today, the sun never sets or even approaches the horizon on NTP. Developers began to treat the protocol as an assumption—it seemed natural to them that synchronized time was reliable and readily available. Mills’ little fiefdom was everywhere.

NTP works by telling computers to send small time-stamped messages to time-checking devices that are superior to them in the hierarchy. The top layer of the hierarchy consists of servers that are intimately connected to highly precise clocks that are kept in tight sync with Coordinated Universal Time. Time then trickles down, layer by layer, to machines at the bottom of the hierarchy, such as ordinary laptops. The protocol keeps track of the moments that pass as a time-checked message is sent, received, returned, and received again by its original sender. All the while, a collection of algorithms—a “popcorn muffler,” a “blow filter”—sorts through the data, sorting out false and true chimeras and instructs the clocks to adjust their times based on the timestamp message they’re being told.


Source link