Over the weekend, there was an odd story about people using AT&T's wireless network trying to log in to Facebook, and suddenly finding themselves logged in to somebody else's Facebook account. What could have caused such a strange phenomenon to occur? What does it tell us about the innards of the mobile web, and what lessons might it convey for network and application design?
Ars Technica had a good post documenting some of the possibilities, and AT&T has now made some public statements containing a few key clues about the problem. We have a few things to add.
[Warning - this post gets fairly technical]
1. Facebook. Facebook needs to start using HTTPS for everything! Without HTTPS and secure cookies, the private and sensitive information in their users' accounts is vulnerable to being mixed up by ISPs' proxy servers, logged, eavesdropped or pilfered by hackers.1 Google now uses HTTPS by default for every interaction with Gmail, and there's no excuse for Facebook not to do the same.
2. AT&T. Here, the story is more complicated, but the short summary is that AT&T (and all other ISPs) really need to migrate away from using proxy and gateway servers to perform complicated software tasks.
The problem at the ISP's end appears to have been a manifestation of an engineering hangover from WAP 1.0, which was the first attempt to bring the Web to mobile phones. WAP made a number of design decisions intended to work around the limitations of 1990s-era cell phones, including tiny storage space, limited bandwidth, and small keypads. In retrospect, some of those design decisions appear to have been unwise. A relevant example was the decision to involve the wireless carrier in website authentication. Where the normal HTTP Web stores authentication cookies on users' computers, early versions of WAP specified that cookies should be stored on proxy servers called WAP gateways, operated by wireless carriers.2 Another practice was to try to avoid ever having to make the user type a username and password with only a numeric keypad, by circulating URLs that contained automatic authentication parameters.
It was this WAP tradition of getting ISPs intimately involved in authentication that led to a situation today where a malfunction on AT&T's proxies could let one user log in to another's Facebook account. This situation is bad for the privacy and security of mobile web users, and it carries some important lessons about the division of responsibility between ISPs and web and application providers.
Wherever possible, ISPs should try to avoid solving complicated problems — like web authentication — by using proxy and gateway servers on their network. Inevitably, having an extra machine in the loop raises the complexity of the solution and increases the number of possible points of failure. If this had been a problem with a website smaller than Facebook, the chances are that it would have remained undiagnosed and unfixed for much longer.
There is a lot of engineering controversy about whether it's ever appropriate for complex application functions to be performed by proxies, gateways or transcoders operated by ISPs. One key argument is that if the ISPs pick a poor solution, or don't all implement exactly the same thing, then developers and users will be worse off than if the ISP had done nothing at all.
Whether or not this is true in all cases, it's clear, at the very least, that ISPs need to be extremely cautious in this space. They need to only deploy a proxy-type solution when it is certain that clients and servers can't solve the problem for themselves. They need to be transparent: follow well-established standards, clearly document their practices, and answer technical questions promptly. Lastly, they should offer users and application providers a standardised way to opt-out of the proxies if they might cause technical or security problems.
Even as mobile phones and mobile browsers are approaching the sophistication of desktop PCs, many mobile carriers are continuing to play strange and undocumented tricks with subscribers' data communications.
And AT&T in particular still has a way to go with respect to transparency. Their public statements indicated that they had deployed some new security measures in the wake of the Facebook affair. When we asked them what those measures were, their spokesperson's response was:
In terms of the new security measures AT&T has put into place, due to security sensitivity, we aren't providing specifics.
AT&T's disappointing response is to retreat to security through obscurity. But long experience teaches that security through obscurity is usually no security at all.
- 2. In practice, this made cookie authentication unusable in WAP, because the way that WAP gateways were implemented and configured was insufficiently standardized, and because many developers realised that it was unacceptable to trust carriers' gateway servers with so much of their authentication housekeeping. This meant that websites had to fall back to a practice known as "URL rewriting" or "URL decoration", which meant adding an authentication token to every URL. In practice, this is frequently equivalent to putting the user's password in the URL.