This morning the Associated Press reported that Comcast is interfering with users' ability to run file sharing applications over its network.

Since we spoke to Comcast last month and understood them to deny that they are doing this, we've been running our own tests.

The results of our tests have agreed with AP's. Comcast is forging TCP RST packets which cause connections to drop (a technique also used by Internet censorship systems in China). These packets cause software at both ends to believe, mistakenly, that the software on the other side doesn't want to continue communicating.

The TCP RST packet forging seems to be protocol-specific: as AP reported, it at least sometimes happens directly in response to specific BitTorrent protocol events. This contradicts Comcast's statement to us that their network management does not target or discriminate against particular protocols. The timing of the injected packets suggests that something on Comcast's network understands the BitTorrent protocol and treats it differently from other protocols.

We confirmed this by trying to download files from Comcast subscribers using BitTorrent. We disabled any firewall or NAT software and connected the machines at both ends directly to the Internet, and ran Wireshark, a packet capture tool. This allows us to see exactly what packets each end sent and exactly what packets each end receives. If ISPs between both points were not forging packets, no packets should have been received by one end that bear the other end's IP address but were not sent by it(*). (This is comparable to recording a telephone conversation at both ends and then comparing the recordings to see whether the phone company sent the conversation through faithfully and unmodified.)

Unfortunately, the resulting packet traces look drastically different from one another: each user routinely receives huge numbers of TCP RST packets that appear to have been sent by the other user. But the packet trace at the other end confirms that these packets were never transmitted; they must have been generated and injected by an ISP along the way.

How do we know that Comcast is responsible? Apart from the large number of reports accusing Comcast, increasingly accompanied by packet traces showing suspicious RST packets, we repeated the experiment with two different Comcast connections (one in San Francisco, and one in Oregon) and saw the RST packets appear in both cases. When the Comcast user in Oregon -- Robb Topolski, a source for the AP story and one of the first to carefully document the RST forging phenomenon -- switched on a VPN connection that caused his communications to be encrypted and routed through a different ISP, the RST packets completely disappeared.

Comcast keeps telling its users that the problems they're seeing are not its fault. It's time for Comcast to come clean about what it's doing and take its users' reports seriously.

(*) Note to network experts: IP fragmentation does allow for the possibility that a single packet sent by one end could arrive in multiple fragments, none of which was originally sent in fragmented form. However, we did not observe any fragmentation at all during our experiments and, in particular, the forged RST packets are clearly not fragments of packets sent by the other end.

Related Issues