Introduction

The modern web was not built using a single, overarching blueprint. It evolved as a patchwork of contributions from people working at companies or in their own capacity as hobbyists, facing specific constraints, trying to solve specific problems. As with most other large human endeavors, it was often impossible to tell if an idea would continue to solve a single problem or if it would scale beyond anything their inventors imagined. In retrospect, many of the paradigm shifts that enabled the modern web seemed obvious, but at the time, they might have been anything but.

Why These Stories Matter

As practicing engineers, reading the history of major innovations and paradigm shifts will help us appreciate the system design principles that lasted. It’s often the case that major innovative ideas can also be retooled towards new classes of problems. Learning about the constraints faced and approaches taken by inventors of some of the modern webs most foundational ideas can help us make more informed decisions when designing systems. It’s also fun to know how we got here.

The Pattern

Each chapter of Building the Web will follow a similar pattern:

  1. The constraint. Something broke, or was about to. Traffic was growing, hardware was failing, users were waiting too long.
  2. The solution. Someone had an idea - sometimes elegant, sometimes a hack held together with duct tape. But it worked.
  3. The evolution. Other people faced similar problems and adapted the solution. It got formalized, standardized, commercialized.
  4. The modern form. Today’s version of the idea, shaped by decades of iteration. Often unrecognizable from the original, but built on the same foundation.

Who This Is For

This series is for engineers who want to understand the infrastructure they use every day. If you’re senior enough to have opinions about system design, you’ll find the historical context illuminating. If you’re earlier in your career, you’ll get a foundation that most people only acquire through years of accumulated folklore.

You don’t need to know the history to use a CDN. But knowing it makes you better at deciding when you need one - and what to do when it’s not enough. Understanding the process of innovation also helps you understand when you can repurpose fundamental ideas to solve entirely new problems.