Software for Keeping ISPs Honest
Yesterday's announcement of a détente between Comcast and BitTorrent was great news. Unfortunately, the general problem of ISPs doing strange things to Internet traffic without telling their customers is likely to continue in the future. EFF and many other organizations are working on software to test ISPs for unusual (mis)behavior. In this detailed post, we have a round-up of the tools that are out there right now, and others that are in development...
When you sign up for an Internet connection, you expect it to actually be an Internet connection. You expect that you can run whatever applications and protocols you choose over the link, or indeed that you can write your own software and run that.
There is a disturbing trend, however, of ISPs stepping in to meddle with your communications, deciding that some applications and protocols are more suitable than others. Or deciding that they can inject advertisements into your queries for domain names, or your browser's exchanges with web sites. Or deciding that encrypted traffic should be throttled across the board.
Whatever you may think about the merits of these practices, we think it's obvious that consumers have a right to know what they're paying for. Only then can they exert pressure on an ISP to change its ways, or vote with their wallets and take their business elsewhere. As we argued in a recent submission to the FCC, ISPs should (at a minimum) disclose the nature of their "network management" practices.
But disclosure will never be enough. Internet users need to be able to test networks themselves to make sure that packets and web pages arrive as they were sent, to make sure that DNS queries are correctly answered, and that ISPs comply with the Internet's standards. That needs to happen around the whole planet.
There are lots of approaches to ISP testing
Before we start talking about all the tools that are popping up for ISP testing, it's worth noting that there are a lot of different ways to test a network, with many different pros and cons. For instance, the software may:
- Actively send "synthetic", pre-determined test traffic, or passively observe the way the network treats natural traffic;
- If the traffic is synthetic, the testing software may try to cope with the complex variation in operating systems and network environments, or try to simplify things by creating or insisting on a known test environment;
- Passive testing systems may focus on one or a small number of protocols, or they may try to test for interference in any protocol that is present;
- The software may (1) be unilateral, just trying to detect interference or delay by examining what's happening on a single computer's network connection, or (2) be multi-party, synchronizing and comparing records from computers that are talking to each other, or (3) be in between, only having authoritative records from one end but possessing special knowledge about how the other end will behave;
- Non-unilateral testing systems may rely on a central server, or they may just try to coordinate records in a peer-to-peer fashion;
- Software may operate at the packet level, measuring integrity, latency and reliability on a per-packet basis, or it may operate at a higher level, confirming (for example) that web pages arrive intact or that a link is running at a certain speed, without worrying about any of the individual packets.
It's a good idea for the Internet community to be pursuing most of these different possibilities, because they're all useful in different situations, and we don't yet know which techniques will prove to be the most important.
Existing and soon-to-be-released tools and data
Last year, EFF released a simple utility called pcapdiff (many thanks to the people who've sent us patches and bug reports; we'll be releasing version 0.2 shortly). EFF is also working on a much more elaborate tool for testing ISPs, which will be called Switzerland. More on that below.
An Italian group has developed an ISO CD image that can be used to test an Internet connection (the CD uses pcapdiff, too!). It's called The Gemini Project. Deploying a whole temporary operating system on a CD is a great example of the "simplify the test environment" approach we described above.
Vuze, the company formed around the Azureus BitTorrent client, has released a plugin that counts the number of RST packets sent to your BT client. These statistics are interesting, but remember that there are legitimate RST packets, and the presence of TCP RSTs isn't evidence that they were spoofed by an intermediary.
Lauren Weinstein has set up a very informative mailing list called NNSquad to discuss "network neutrality" and to find ways to test it. Lauren and some other members of that group have been putting together a piece of software, called NNMA, that will be released soon.
A group of researchers at the Max Planck Institute for Software Systems has published a Java applet that generates BitTorrent traffic from within your browser and looks for unusual treatment of it (they haven't posted any source code, so we aren't sure how reliable their tests are — but if you've seen interesting results from this applet, please let us know).
Several other academic research groups and individual researchers are working in this space too. The Networks Group at Northwestern University has a large project, and several network testing apps in the pipeline. Robb Topolski, Juan Carlos de Martin at the Politecnico di Torino, Parminder Chhabra at Boston University and collaborators, and the networking group at Berkeley's ICSI, and Dan Kaminsky have all told us about interesting tools and results that are in the pipeline. We'll update our Test Your ISP page as these are published.
As we mentioned above, EFF and a team of excellent volunteers has been working on its own more sophisticated network testing tool, which will be called Switzerland. We're aiming to publish a draft specification in the next week or two, and to have an alpha release for network engineers and protocol developers to try out by the end of April. In the mean time, we'll say that (unlike most of the above projects), our first architectural objective is to do full multi-party passive monitoring, looking for forged packets in any protocol as well as unusual dropped packet and latency statistics. We'll do that by having each client report secure summaries of the traffic that's being tested to a server running Switzerland code. Full details and discussion will be in the spec!
Recent DeepLinks Posts
Aug 29, 2016
Aug 29, 2016
Aug 26, 2016
Aug 25, 2016
Aug 24, 2016
- Abortion Reporting
- Analog Hole
- Anti-Counterfeiting Trade Agreement
- Artificial Intelligence & Machine Learning
- Bloggers' Rights
- Border Searches
- Broadcast Flag
- Broadcasting Treaty
- Cell Tracking
- Coders' Rights Project
- Computer Fraud And Abuse Act Reform
- Content Blocking
- Copyright Trolls
- Council of Europe
- Cyber Security Legislation
- Defend Your Right to Repair!
- Development Agenda
- Digital Books
- Digital Radio
- Digital Video
- DMCA Rulemaking
- Do Not Track
- E-Voting Rights
- EFF Europe
- Electronic Frontier Alliance
- Encrypting the Web
- Export Controls
- Fair Use and Intellectual Property: Defending the Balance
- FAQs for Lodsys Targets
- File Sharing
- Fixing Copyright? The 2013-2016 Copyright Review Process
- Free Speech
- Genetic Information Privacy
- Government Hacking and Subversion of Digital Security
- Hollywood v. DVD
- How Patents Hinder Innovation (Graphic)
- International Privacy Standards
- Internet Governance Forum
- Know Your Rights
- Law Enforcement Access
- Legislative Solutions for Patent Reform
- Locational Privacy
- Mandatory Data Retention
- Mandatory National IDs and Biometric Databases
- Mass Surveillance Technologies
- Medical Privacy
- Mobile devices
- National Security and Medical Information
- National Security Letters
- Net Neutrality
- No Downtime for Free Speech
- NSA Spying
- Offline : Imprisoned Bloggers and Technologists
- Online Behavioral Tracking
- Open Access
- Open Wireless
- Patent Busting Project
- Patent Trolls
- PATRIOT Act
- Pen Trap
- Policy Analysis
- Public Health Reporting and Hospital Discharge Data
- Reading Accessibility
- Real ID
- Reclaim Invention
- Search Engines
- Search Incident to Arrest
- Section 230 of the Communications Decency Act
- Social Networks
- SOPA/PIPA: Internet Blacklist Legislation
- State-Sponsored Malware
- Student Privacy
- Stupid Patent of the Month
- Surveillance and Human Rights
- Surveillance Drones
- Terms Of (Ab)Use
- Test Your ISP
- The "Six Strikes" Copyright Surveillance Machine
- The Global Network Initiative
- The Law and Medical Privacy
- TPP's Copyright Trap
- Trade Agreements and Digital Rights
- Trans-Pacific Partnership Agreement
- Travel Screening
- Trusted Computing
- UK Investigatory Powers Bill
- Video Games