Peering into the Soft Underbelly of Net Neutrality
The net neutrality fight is moving in new directions, and quickly. Today FCC Chair Tom Wheeler announced that the FCC would press forward with new “Open Internet” rules, undeterred by last month’s court decision striking down most of the old ones. Last week, Comcast and Time Warner Cable announced plans to merge. The merger would create the largest Internet Service Provider (ISP) in the nation, with five times the subscribers of its closest competitor. With only one or two broadband providers available in most parts of the country, prices may soar while the quality of services plummets. A lack of competition raises serious concerns that huge ISPs will be able to favor particular sites and services.
Wheeler’s announcement today included a strong commitment to promoting the piece of the Open Internet rules that did survive judicial review: transparency. Unfortunately, even “transparency” is tougher to enforce than many might think. That’s because so much of our connectivity depends on essentially secret “peering” agreements between providers.
To understand peering, let’s back up to last week: news of the Comcast-Time Warner Cable merger broke while EFF was investigating a claim that Verizon was discriminating against traffic from Netflix and Amazon Web Services. A blogger in Texas named David Raphael reported that his home Internet speed to Netflix was dramatically slowed down. He tested his connection, and it turned out that indeed his Verizon FiOS Internet connection was operating thousands of kilobytes per second slower than normal—when connecting to certain sites, but not others.
Dave’s problems made some nervous that Verizon had already begun to slow down connections to disfavored websites. In light of last month’s court decision, many feared ISPs would act like the door was wide open for them to discriminate against traffic on their networks. As broadband markets appear to consolidate, those who care about an open, innovative Internet are rightly worried.
Verizon denied that it was discriminating against Netflix or other types of traffic, but we’ve seen ISPs dissemble on this point before. In 2007, when Comcast was caught intermittently blocking users' ability to exchange data using the BitTorrent peer-to-peer file-sharing protocol, the company made numerous false claims about its network interference before finally admitting its bad behavior and halting the disruptions.
What makes the issue even more complicated is that we don’t have a common understanding of what it means to discriminate, or provide unequal treatment to some traffic. For example, traffic could be slowed in practice because of infrastructural problems that are different from “intentional” throttling. Or ISPs could argue that systematically slowing traffic to certain destinations wasn't actually discrimination because they hadn't programmed their equipment to make it slow.
To try to unpack Dave’s complaints, EFF put out a call to our tech-volunteer mailing list and asked other Verizon FiOS users around the country to test their connection speeds. The results were somewhat inconsistent. Most showed a relatively quick connection to Amazon Web Services, even in the evenings, but a few slow speeds were reported. Was Verizon only slowing down connection speeds in some parts of the country? Was Netflix targeted? Was it a technical malfunction?
These inconsistencies seem to match other people's experience: the slowdown, if any, affected only certain FiOS users, only some of the time. This teaches one key lesson: we need a regular, widespread, and methodical system for testing infrastructure to help understand the size and shape of connectivity problems and slowdowns when they develop on the Internet, so that accusations of discrimination or blocking don't have to be based on anecdote alone.
Of course, slowdowns that affect only certain regions or that are noticeable only some of the time are frustrating, and could be genuine indications of a larger problem. Even if ISPs like Verizon aren't explicitly blocking or slowing traffic, there could be deeper, more systematic issues that affect connections to certain destinations but not others.
This points to a conceptually subtle but important part of the problem: the Internet is a “network of networks,” and data traveling from point to point may pass through the equipment of Comcast, Level 3, AT&T, Verizon, Cogent, and many other companies. The interconnections between these systems are almost unnoticeable to end users when the Internet is working smoothly, but they can be brought to the foreground when particular links become overloaded with congestion caused by rising traffic levels.
And that brings us to the issue of “peering.” Connections between web service providers, websites, and ISPs depend on agreements to exchange Internet traffic with each other, or “peering” links. Operators of backbone and web services make peering agreements with ISPs about how to exchange Internet traffic so that data can be carried efficiently from one part of the Internet to another.
Peering relationships are crucially important to the fabric of the Internet, but today they're often hidden behind a veil of trade secrecy and business strategy. And providers often butt heads in ways that interfere with and shape Internet availability in discriminatory, non-neutral ways. Simply put, peering disputes are a serious threat to the future of our open Internet.
Here’s a case in point: in 2010 Comcast and Level 3, one of the largest content delivery networks (CDN) and web service providers in the world, had a major disagreement over peering. Netflix, a company whose Internet video streaming accounts for nearly one third of the America’s download traffic, had just signed a contract with Level 3 to be their transit provider. Comcast said that the new traffic coming across their infrastructure from Level 3 was not within the scope of the two companies’ original peering agreement. Comcast refused to build new ports to handle the increase in traffic from Level 3, claiming that they weren’t responsible for building new network infrastructure needed to deliver a major influx of streaming video to users.
Comcast’s solution: charge Level 3 to deliver content to Comcast subscribers. That meant Comcast was trying to charge users to connect to the Internet and charge data centers to connect to users—a doubly profitable solution. The companies have since struck a cost-sharing deal, the finer details of which are spelled out in secretive peering agreements.
More recently, in 2013, there was a peering dispute between Verizon and Cogent. Cogent is one of the nation’s largest Internet providers, particularly providing infrastructure to operators of web sites and services. Cogent and Verizon peer with each other in locations around the country, and when the ports that they use to connect to each other become full of traffic, Verizon will typically have to add more ports to keep the data flowing as smoothly as before.
But with video streaming on the rise, Verizon claimed that Cogent had to share the cost in building network infrastructure to handle the tremendous uptick in traffic. Cogent refused to pay, and Verizon refused to provide Cogent with any more ports. As a result, many Verizon customers had serious trouble connecting to websites that rely on Cogent for Internet connectivity.
And back in 2008, a peering dispute between Cogent and Sprint became so bad that the two companies stopped exchanging traffic entirely. At the time, Sprint customers could not access data from companies that rely on Cogent. The Internet was partitioned—divided into separate parts that couldn't directly communicate with one another. Even though neither company may have wanted this outcome, the results are as bad for Internet users as if traffic to particular destinations had been blocked deliberately.
Peering disputes often seem to boil down to which company should have to pay for the upgrades in network architecture required to connect users to innovative new services. Should it be the content providers, the backbone companies that own data links, or residential ISPs? Increasingly, big residential ISPs like Verizon are asking for money from other players who want to send more data to end users—data that must go through the ISPs to reach their customers.
Discrimination can happen over any connection
Some journalists and network engineers have claimed that network operators may intentionally delay or avoid upgrades to garner more leverage in peering negotiations. That is, company A may know that the capacity of their connection to company B is inadequate and that the connection is frequently congested—but not fix the problem because it wants operator B to pay up in exchange for better connectivity.
Simply put, non-neutral behavior isn’t only a matter of the relationship between ISPs, websites, and users. And it isn't only a matter of blocking traffic or forbidding users from reaching a certain site or from using certain software. It could be a matter of infrastructure fights that make some parts of the Internet dramatically faster and more reliable to reach than others. The details of the relationships and connections between ISPs and all other web services are extremely important to any long-term vision of network neutrality. The problem is that peering contracts are negotiated behind closed doors and often contain non-disclosure agreements, making it difficult to locate the source of a slowed down connection unless one of the companies involved chooses to take its case to the public.
This is why it was remarkably difficult to find the cause of Dave’s slow Internet. It’s hard to tell whether his bad Internet connectivity is an infrastructure problem or if ISPs are engaging in deliberate discriminatory, anti-competitive behavior. Companies are likely to disagree about what 'nondiscriminatory' peering would mean, since no two Internet networks are exactly alike, but these inevitable debates should not prevent us from articulating the goal that Internet interconnections need to be offered fairly to all.
And even when the reason for bad connectivity is identified, companies sometimes try to deny or shift the responsibility to build the infrastructure needed to meet the public’s ever-increasing data demands. ISPs may argue that refusing to maintain network architecture isn’t technically the same as interfering with network traffic. But refusing to improve infrastructure can be a form of discrimination. Keep in mind that if the Comcast and Time Warner Cable’s merger comes to fruition, Comcast will have a tremendous amount of leverage in closed-door peering disputes.
So let’s shine some light on these secret contracts. Peering agreements need to be brought to public view. And peering arrangements should be offered on nondiscriminatory terms, so that particular networks—and with them, particular Internet sites and services—aren't favored with access to fast connectivity on terms that aren't available to their competitors.
Bringing transparency to the network of networks will be complicated, but it’s not impossible. Unfettered, ubiquitous, and high-speed access to the Internet is far too important to be negotiated away in obscurity.