This is an exciting time to be working on the Internet. Today hundreds of millions of users are connected to this global network, using it as part of their daily workflow. Many of the most innovative applications reach viral popularity literally overnight, and multi-millionaires are being made every day.
As many of these emerging services grow, they come to require something better than commodity Internet services purchased at the edge of the Internet. To continue the wave of mass adoption, these services require flawless performance at massive scale, achievable only by connecting directly to the core of the Internet. Massive volumes of Internet traffic are exchanged at the core of the Internet in a sort of open Internet transit marketplace. This area is where the largest Internet Service Providers (ISPs) in the world interconnect, where the largest content providers interconnect with the largest regional ISPs, where the content delivery networks offload their traffic directly onto the broadband networks. The information and strategies described in this book will enable the next generation of Internet services companies to connect to and leverage the performance benefits at the core of the Internet as well.
This book is the assimilation of thousands of discussions with hundreds of the smartest peering coordinators in the world. I spent over a decade on the road, traveling to almost every continent, racking up over 500,000 air miles and spending over $500,000 in travel budget. I attended every Internet Operations conference so I could understand how Internet interconnection worked. I documented what I learned, citing the sources when allowed, and then walked others through the resulting white paper to see if I got it right. After about 100 walkthroughs, I would have a white paper that documented a particular aspect of Internet interconnection. I went through this process to produce 12 peering white papers over the course of 10 years. This base research is the raw source material for this book.
The information contained in this book is valuable. As a consultant, I help executive teams build effective peering and transit strategies. I also advise investors on the core Internet industries (ISPs, carriers, colocation centers, IXPs, CDNs, etc.). About 70% of the information I provide during these consulting engagements is information contained within this book. The rest of the information is provided on-site as the teams develop their strategies and investment theses. On-site consulting costs a lot of money, but to these teams, collectively, their time is worth a lot more. They prefer a two-day on-site consulting engagement where they learn together and develop a strategy based on the information I collected during the last couple of decades. Internet interconnection and colocation is a highly specialized area of expertise, and there are only a few of us that can deliver this type of on-site consulting. This book provides this proven valuable information in a much more cost-effective package.
My hope and expectation is that this material will help the next generation of peering coordinators, network engineers, network architects, and business leaders understand and leverage the power of peering at the core of the Internet.
This book is not about technology. There are many great books about the protocols, hardware, and algorithms. One can look at the protocols, the hardware, and the algorithms, but that doesn’t tell the story about how the Internet works at the core.
This book is about the core of the Internet, the relationships between the companies that operate the central infrastructure, the “interconnection machinery” that makes the Internet work. It describes the business relationships that simultaneously demonstrate symbiosis and grudging interdependence. This book models the Internet as a Global Internet Peering Ecosystem, and describes the players, their power positions, the corresponding motivations and, indeed, predictable behavior.
This book includes everything you need to know about the Global Internet Peering Ecosystem to make the critical decision: when and why does your company need to connect to the core of the Internet? Some parts of this book will be of interest to you immediately, and some parts will be of interest to you later on. All of it is essential for business people, executives, and technical staff focused on the Internet.
Together, we will take an outward-in view of the Internet.
|Section I||Connecting to the Edge of the Internet|
|Section II||Connecting to the Core of the Internet|
|Section III||The Global Internet Peering Ecosystem|
|Section IV||The Tricks of the Trade: The Playbooks|
We will start by looking at how most companies attach to the edge of the Internet, and learn about the industry terms. We then dive into the core of the Internet, where the largest ISPs and content companies in the world peer their networks. Then we take a 30,000-foot view on the Global Internet Peering Ecosystem, and recognize the commonalities across the globe. With this holistic view of the Global Internet Peering Ecosystem, we will finish by examining the “Tricks of the Trade” – the playbooks collected from some of the smartest peering coordinators and Internet Exchange Point Operators in the world. These playbooks document how ISPs obtain peering where they otherwise would not have been able to. The IXP playbook documents how the most sophisticated IXPs build, grow, maintain, and attack or defend their critical mass of peering participants.
This book takes the reader beyond “The Internet is a network of networks” and documents the internal workings of Internet interconnection and the strategic nature of these interconnections. This information is essential for any Internet services company building out at scale.
This book is for anyone who wants to learn how the core of the Internet works.
Internet Service Providers will benefit from understanding the methods and clever maneuvers surrounding interconnection used around the globe. This book highlights some of the issues and clever tactics that seasoned peering professionals see and develop after years in the field.
Internet Exchange Point Operators (IXPs) will find this book helpful for building a peering ecosystem in their facilities. They will also better understand their customers’ mindset and motivations. This book presents the financial value proposition that Internet Exchange Point Operators provide to customers. The business case for peering at an Internet Exchange Point is clearly explained. Most important to IXPs, this book documents the tactics that the smartest IXPs in the world have used to build and grow, attack and defend a critical mass or peers. This material has been used as training material for IXP staff as well as for their customers and investor base.
Hardware vendors have found this material of practical value for better understanding their customers’ deployment environment, their applications, and their motivations. While this book was not written explicitly for this audience, the larger network equipment manufacturers are now becoming keenly interested in this material.
Students have used the previously released white papers in their classes for years. This book, rewritten from this earlier material, provides a good starting point for understanding the Internet core. The book provides a pragmatic industry view on interconnections that are essential for identifying research projects that are relevant.
Investors in colocation, data centers, IXPs, or large-scale Internet services will find this book to be an invaluable resource in understanding how the Internet actually works at the core.
This book is an assimilation of thousands of discussions with some of the brightest peering coordinators in the world between 1994 and 2011. A subset of these people is listed in the acknowledgements. Many more of them provided valuable data and insights but asked to remain anonymous. What you hold in your hands is based on over fifteen years of primary research.
Notes from the field.
Stories and Anecdotes to Make a Point
Throughout this book I include stories and anecdotes that illustrate a particular point. I will format them as I do here to set them apart from the generalizations made in the book. When I consult with clients and when I present peering workshops, I have found that these stories help bring to life the points presented in the material.
In The Grand Design, Stephen Hawking says that a model is a good model if it is “1) elegant, 2) contains few arbitrary or adjustable elements, 3) agrees with and explains all existing observations, and 4) makes detailed predictions about future observations that can prove or disprove the model if they are not borne out.”
While the modeling of the Internet core as an ecosystem is not of the scale or importance of his work, I believe the Global Internet Peering Ecosystem model is a good model. The model is 1) simple, 2) contains few variables, 3) agrees with the observations made around the world, and 4) provides detailed predictions on the behavior of the parties involved at the Internet core.
The Global Internet Peering Ecosystem has many characteristics of a living ecosystem. There are individual identifiable species that hold power positions within the ecosystem, complete with motivations and predictable behaviors consistent with their position. The ecosystem itself has morphed over time to adjust to the stimulus placed upon it by the customers and their applications at the edge. The scope of this Internet ecosystem is global, so the stakes are very high, and information has been hard to gather. This book uncovers perhaps the most important topics on the Internet today – the evolution of the Global Internet Peering Ecosystem, and are we going in the right direction?
Palo Alto, California