2015-05-29

Courageous journalism at the BBC

I kid, obviously. When describing the current controversy over the Washington D.C. Metro refusing to take any "issue-oriented" adverts until next year just so that they can avoid showing the prize-winning "Draw Mohammed" cartoon, the BBC resorts to words rather than a picture to describe the salient image.

The advert calls for Americans to support free speech and features a bearded, turban-wearing Muhammad waving a sword and shouting: "You can't draw me!"
In reply, a cartoon bubble portrays an artist grasping a pencil and saying: "That's why I draw you."
How odd, you would have thought that they would have included an image of the cartoon rather than laboriously describe its contents.

Just to make the point, here's the image in question:

The spineless BBC writer isn't shy of displaying their orientation towards issues:

Ms Geller insists the cartoon is a "political opinion" which does not contain any violence.
Ms Geller is of course correct. There's no violence in that picture: the gentleman depicted is holding a sword, but that's as far as it goes. Yet the writer takes particular care to use reported speech and quotes, presumably to demonstrate that he or she is emphatically not in sympathy with Ms Geller or (mysteriously unnamed in the article) artist Bosch Fawstin.

Deary me. Truely, the BBC has resigned from actual journalism in order to be at the back of the line when crocodile feeding time comes around.

I'm really not keen on Pamela Gellar, but the rest of the world seems to be bending over backwards to make her admittedly extreme opinions seem really quite rational and reasonable. And we are surprised when Muslim extremism is emboldened by this obvious cowardice?

2015-05-19

Delays are good for you - the MTA proves it

No, really, they do. New York's Metropolitan Transit Authority (something like Transport for London) has produced an outstanding video that shows why making some subway trains late makes others less late:

Yes, the idea is that sometimes delaying a train can prevent further delays by not compounding the gap between trains. Anyone who has waited impatiently on a hot subway platform might find this concept counterintuitive, but transportation experts generally agree that that the evenness of service is as crucial as avoiding individual delays.
The MTA video makes a compelling case. The key insight is that once a platform gets crowded enough, due to constant feed of new passengers and a delayed train, it becomes slower for the next train to debark and embark passengers. So an already delayed train gets more delayed as it progresses down the line. The solution? Spot a train that's getting near the critical delay time and give it priority to progress through the network even if this involves delaying other (less delayed trains).

It's a great example that, even in what we regard as relatively simple systems, there can be a complex interplay between entities that produce highly unintuitive results. Deliberately delaying trains can actually be good for the system as a whole (if not for the passengers sitting in the delayed train with their faces pressed into a fellow passenger's unwashed armpit).

2015-05-13

You should care about moving to HTTPS

Eric Mill's "We're Deprecating HTTP and it's going to be okay" is a must-read call-to-arms for everyone with a site on the Internet, explaining why the transition from unencrypted web traffic (HTTP) to encrypted (HTTPS) is actually fundamental to the future existence of the democratic web-as-we-know it.

For the 90% of my reading audience who are already saying "Bored now!" here's why it matters to you. Sir Tim Berners-Lee invented HTTP (the language of communication between web browser and web server) in CERN, a European haven of free thought, trust and international co-operation. The 1930s idea that "Gentlemen do not read each other's mail" was - surprisingly, given the history of cryptographic war in WW2 - fundamental to HTTP; messages might have transited systems owned by several different groups, but none of them would have thought to copy the messages passing through their system, let alone amend them.

This worked fine as long as no-one was interested in the communication of harmless nerds about their hobbies, much as the government-owned Royal Mail doesn't bother to copy the contents of postcards passing through their sorting offices because they only contain inane drivel about sun, sea and sand. However, once people realized that they could communicate freely about their occasionally subversive ideas across borders and continents, and financial institutions woke to the possibility of providing services without paying for expensive un-scalable fallible human cashiers, many governments and other less-legal entities wanted to read (and sometimes alter) Internet traffic.

Mills gives two great examples of where HTTPS prevented - and could have prevented further - nation-state abuse of Internet content:

- The nation of India tried and failed to ban all of GitHub. HTTPS meant they couldn't censor individual pages, and GitHub is too important to India's tech sector for them to ban the whole thing.
- The nation of China weaponized the browsers of users all over the world to attack GitHub for hosting anti-censorship materials (since like India, they can't block only individual pages) by rewriting Baidu's unencrypted JavaScript files in flight.
And closer to home, Cameron's plan to make all online communication subject to monitoring is so stupidly illiberal and expensively pointless that it deserves to be made impractical by general adoption of HTTPS. GCHQ and friends can tap all the Internet traffic they like: if it's protected by HTTPS, the traffic is just taking up disk space to no practical purpose. Brute-forcing, even with nation-state resources, is so expensive that it's reserved for really high-value targets. GCHQ would have to go after something fundamental like a Certificate Authority, which would leave big and obvious fingerprints, or compromise a particular user's machine directly, which doesn't scale.

As long as users are still relaxed about the absence of a padlock in their browser bar, HTTP will continue to provide a route for governments to snoop on their citizens' traffic. So let's give up on HTTP - it has had its day - and move to a world where strongly encrypted traffic is the default.

2015-04-30

You can't be too careful - car crashes

The class of systems with high distributed costs and focused but inadequate benefits is going to have another member: auto-calling police in the event of a car crash:

In the event of a crash, the device calls the E.U.'s 911 equivalent (112) and transmits to authorities important information including location, time, and number of passengers in the vehicle. An in-car button will also be installed in all vehicles. The eCall requirement will add an estimated $100 to the price of a car.
$100 on each (new) car sold: so how many new cars are sold in the EU each year? About 14 million in 2012. So this measure will cost $1.4 billion, and maybe $150 million in the UK. What's the benefit?
Each year nearly 26,000 people are killed in the E.U. by car crashes. This new device is estimated to reduce that number by 10 percent, saving 2,600 lives annually, by cutting down emergency response time by as much as 60 percent.
The cost of a life for purposes of safety varies by country and mode of transport, but let's take $1 million as the average. Given the quoted statistics, $2.6 billion saving (though optimistic, probably lower) comprehensively dwarfs $1.4 billion cost (though also optimistic, probably higher). Why isn't this a slam-dunk decision?

The problem is twofold: a) zeroing cost for lives saved, and b) the assumption of 10% saving. Let's consider each in turn.

If an injury is potentially fatal but not actually fatal due to timely intervention, it's almost always due to either early suppression of severe blood loss, or timely (within 1-2 mins) clearing of obstructed airway. The latter isn't relevant due to emergency service response times, so we only consider the former. This injured person will still need emergency treatment followed by several days of hospital care, and quite possibly follow-on care of injuries, rehab, and in some cases reduced lifetime tax payments due to reduced earnings and disability payments, so you're looking at order of $100K average costs. That's still not really significant.

However, consider a typical case where a life is saved: a car driver has an accident in the countryside when no-one is around. His car calls 112 and so the police (not the ambulance service initially, because they are too stretched to respond to wild goose chases) respond to his location. Seeing the crash they call for an ambulance which arrives 10-30 minutes before it would have otherwise arrived due to a passer-by report - people tend to notice a crashed car with no emergency services around it. He would have died due to shock (depletion of oxygen to the critical organs due to blood loss / asphyxiation / traumatic damage to heart and lungs) but the ambulance got there in time to oxygenate him and transport to hospital. Just how common is this?

Fatal road accidents rarely happen on remote roads - unsurprisingly, they happen where there are many more cars and roadside obstructions to run into. If an accident happens where passers-by are prevalent, this system doesn't help at all since nearly all passers-by have mobile phones. So we're only looking at a small fraction - 5% is optimistic - of accidents. The press release assumed 10%, so the benefit has already halved and is perilously close to the cost.

But bleeding to death is not a common cause of death from road accidents for drivers/passengers. Much more likely is traumatic head injury, which tends to kill them right there in the car. Unsecured drivers/passengers fly through the windscreen, or secured drivers/passengers bang their head against the car frame. This kills instantly, or in a few minutes. Another mechanism is the "third collision" where the car bangs into a tree (collision 1), the driver bangs into their seatbelt (collision 2) and then the free-hanging organs like lungs, heart bang into the drivers chest, or their blood vessels bang into ligaments that cheesewire them (collision 3). If you're in this situation and your aorta (the major blood vessel coming out of the heart) is damaged you can expect a 60%-80% chance of death no matter how quickly you get to the hospital.

Therefore, before we stick the European population with an extra $1 billion of annual costs, why don't we conduct a limited experiment introducing this requirement into a single country which is similar to another country in road crash death rates to see what effect, if measurable, this measure has? Or is the notion of trade-offs too alien to the EU?

2015-04-23

Journos writing about trading and high-speed computing

I have to admit, this amused me - the Daily Mail trying to write about high-frequency trading:

Suspected rogue trader Navinder Sarao lived in his parents' modest home because it gave him a split-second advantage worth millions of pounds, it was claimed yesterday.
His family's semi-detached house in suburban West London is closer to an internet server used by one of the major financial exchanges, giving him a nanosecond advantage over rivals in the City.
[...]
Sarao, 36, was dubbed the 'Hound of Hounslow' after it emerged he lived at home with his parents, despite allegedly making £26.7million in just four years of dealing from their home.
And yet you'd think that renting a small flat in Slough and paying for Internet access there would have improved his speed advantage; at a cost of about £50K for four years, that would have been a bargain. Why, it's almost as if the Daily Mail journalists had no idea what they were talking about....

2015-04-02

Active attack on an American website by China Unicom

I wondered what the next step in the ongoing war between Western content and Chinese censorship might be. Now we have our answer.

"Git" is a source code repository system which allows programmers around the world to collaborate on writing code: you can get a copy of a software project's source code onto your machine, play around with it to make changes, then send those changes back to Git for others to pick up. Github is a public website (for want of a more pedantic term) which provides a repository for all sorts of software and similar projects. The projects don't actually have to be source code: anything which looks like plain text would be fine. You could use Github to collaborate on writing a book, for instance, as long as you used mostly text for the chapters and not e.g. Microsoft Word's binary format that makes it hard for changes to be applied in sequence.

Two projects on Git are "greatfire" and "cn-nytimes" which are, respectively, a mirror for the Greatfire.org website focused on the Great Firewall of China, and a Chinese translation of the New York Times stories. These are, obviously, not something to which the Chinese government wants its citizenry to have unfettered access. However, Github has many other non-controversial software projects on it, and is actually very useful to many software developers in China. What to do?

Last week a massive Distributed Denial of Service (DDoS) attack hit Github:

The attack began around 2AM UTC on Thursday, March 26, and involves a wide combination of attack vectors. These include every vector we've seen in previous attacks as well as some sophisticated new techniques that use the web browsers of unsuspecting, uninvolved people to flood github.com with high levels of traffic. Based on reports we've received, we believe the intent of this attack is to convince us to remove a specific class of content. [my italics]
Blocking Github at the Great Firewall - which is very easy to do - was presumably regarded as undesirable because of its impact on Chinese software businesses. So an attractive alternative was to present the Github team with a clear message that until they discontinued hosting these projects they would continue to be overwhelmed with traffic.

If this attack were just a regular DDoS by compromised PCs around the world it would be relatively trivial to stop: just block the Internet addresses (IPs) of the compromised PCs until traffic returns to normal levels. But this attack is much more clever. It intercepts legitimate requests from worldwide web browsers for a particular file hosted on China's Baidu search engine, and modifies the request to include code that commands repeated requests for pages from the two controversial projects on Github. There's a good analysis from NetreseC:

In short, this is how this Man-on-the-Side attack is carried out:
1. An innocent user is browsing the internet from outside China.
2. One website the user visits loads a JavaScript from a server in China, for example the Badiu Analytics script that often is used by web admins to track visitor statistics (much like Google Analytics).
3. The web browser's request for the Baidu JavaScript is detected by the Chinese passive infrastructure as it enters China.
4. A fake response is sent out from within China instead of the actual Baidu Analytics script. This fake response is a malicious JavaScript that tells the user's browser to continuously reload two specific pages on GitHub.com.

The interesting question is: where is this fake response happening? We're fairly sure that it's not at Baidu themselves, for reasons you can read in the above links. Now Errata Security has done a nice bit of analysis that points the finger at the Great Firewall implementation in ISP China Unicom:

By looking at the IP addresses in the traceroute, we can conclusive prove that the man-in-the-middle device is located on the backbone of China Unicom, a major service provider in China.
That existing Great Firewall implementors have added this new attack functionality fits with Occam's Razor. It's technically possible for China Unicom infrastructure to have been compromised by patriotically-minded independent hackers in China, but given the alternative that China Unicom have been leant on by the Chinese government to make this change, I know what I'd bet my money on.

This is also a major shift in Great Firewall operations: this is the first major case I'm aware of that has them focused on inbound traffic from non-Chinese citizens.

Github look like they've effectively blocked the attack, after a mad few days of scrambling, and kudos to them. Now we have to decide what the appropriate response is. It seems that any non-encrypted query to a China-hosted website would be potential fair game for this kind of attack. Even encrypted (https) requests could be compromised, but that would be a huge red arrow showing that the company owning the original destination (Baidu in this case) had been compromised by the attacker: this would make it 90%+ probable that the attacker had State-level influence.

If this kind of attack persists, any USA- or Europe-focused marketing effort by Chinese-hosted companies is going to be thoroughly torpedoed by the reasonable expectation that web traffic is going to be hijacked for government purposes. I wonder whether the Chinese government has just cut off its economic nose to spite its political face.

2015-03-04

What does "running your own email server" mean?

There's lots of breathless hyperbolae today about Hillary Clinton's use of a non-government email address during her tenure as Secretary of State. The Associated Press article is reasonably representative of the focus of the current debate:

The email practices of Hillary Rodham Clinton, who used a private account exclusively for official business when she was secretary of state, grew more intriguing with the disclosure Wednesday that the computer server she used traced back to her family's New York home, according to Internet records reviewed by The Associated Press.
[...]
It was not immediately clear exactly where Clinton's computer server was run, but a business record for the Internet connection it used was registered under the home address for her residence in Chappaqua, New York, as early as August 2010. The customer was listed as Eric Hoteham.
Let's apply a little Internet forensics to the domain in question: clintonemail.com. First, who owns the domain?
$ whois clintonemail.com
[snip]
Domain Name: CLINTONEMAIL.COM
Registry Domain ID: 1537310173_DOMAIN_COM-VRSN
Registrar WHOIS Server: whois.networksolutions.com
Registrar URL: http://networksolutions.com
Updated Date: 2015-01-29T00:44:01Z
Creation Date: 2009-01-13T20:37:32Z
Registrar Registration Expiration Date: 2017-01-13T05:00:00Z
Registrar: NETWORK SOLUTIONS, LLC.
Registrar IANA ID: 2
Registrar Abuse Contact Email: abuse@web.com
Registrar Abuse Contact Phone: +1.8003337680
Reseller:
Domain Status:
Registry Registrant ID:
Registrant Name: PERFECT PRIVACY, LLC
Registrant Organization:
Registrant Street: 12808 Gran Bay Parkway West
Registrant City: Jacksonville
Registrant State/Province: FL
Registrant Postal Code: 32258
Registrant Country: US
Registrant Phone: +1.5707088780
Registrant Phone Ext:
Registrant Fax:
Registrant Fax Ext:
Registrant Email: kr5a95v468n@networksolutionsprivateregistration.com
So back in January this year the record was updated, and we don't necessarily know what it contained before that, but currently Perfect Privacy, LLC are the owners of the domain. They register domains on behalf of people who don't want to be explicitly tied to that domain. That's actually reasonably standard practice: any big company launching a major marketing initiative wants to register domains for their marketing content, but doesn't want the launch to leak. If Intel are launching a new microbe-powered chip, they might want to register microbeinside.com without their competitors noticing that Intel are tied to that domain. That's where the third party registration companies come in.

The domain record itself was created on the 13th of January 2009, which is a pretty strong indicator of when it started to be used. What's interesting, though, is who operates the mail server which receives email to this address. To determine this, you look up the "MX" (mail exchange) records for the domain in question, which is what any email server wanting to send email to hillary@clintonemail.com would do:

$ dig +short clintonemail.com MX
10 clintonemail.com.inbound10.mxlogic.net.
10 clintonemail.com.inbound10.mxlogicmx.net.
mxlogic.net were an Internet hosting company, bought by McAfee in 2009. So they are the ones running the actual email servers that receive email for clintonemail.com and which Hillary's email client (e.g. MS Outlook) connected to in order to retrieve her new mail.

We do need to take into account though that all we can see now is what the Internet records point to today. Is there any way to know where clintonemail.com's MX records pointed to last year, before the current controversy? Basically, no. Unless someone has a hdr22@clintonemail.com mail from her home account which will have headers showing the route that emails took to reach her, or has detailed logs from their own email server which dispatched an email to hdr22@clintonemail.com, it's probably not going to be feasible to determine definitively where she was receiving her email. However, CBS News claims that the switch to mxlogic happened in July 2013 - that sounds fairly specific, so I'll take their word for it for now. I'm very curious to know how they determined that.

All of this obscures the main point, of course, which is that a US federal government representative using a non-.gov email address at all for anything related to government business is really, really bad. Possibly going-to-jail bad, though I understand that the specific regulation requiring a government employee to use a .gov address occurred after Hillary left the role of SecState (Feb 2013). Still, if I were the Russian or Chinese foreign intelligence service, I'd definitely fancy my chances in a complete compromise of either a home-run server, or of a relatively small-scale commercial email service (mxlogic, for instance).

Desperately attempting to spin this whole situation is Heidi Przybyla from Bloomberg:

OK, let's apply our forensics to jeb.org:
$ dig +short jeb.org MX
5 mx1.emailsrvr.com.
10 mx2.emailsrvr.com.
emailsrvr.com is, like mxlogic.net, a 3rd party email hosting service, apparently specialising in blocking spam. I'm not surprised that someone like Jeb Bush uses it. And, like Hillary, he isn't "running his own email server", he's using an existing commercial email server. It's not Gmail/Outlook.com/Yahoo, but there's not reason to think it's not perfectly serviceable, and it's not controlled by Bush so if they log or archive incoming or outgoing email his correspondence is legally discoverable.

The difference between Jeb Bush and Hillary Clinton of course, as many others note, is that Jeb is not part of the US federal government and hence not subject to federal rules on government email...

2015-02-28

No cash for CASH

For those following along with our previous adventures with the prodnoses of Consensus Action on Salt and Health (CASH) their 2014 accounts make an entertaining read, with not a little schadenfreude.

Deprived of the £100K that our friends at the Marcela Trust sent in their direction in 2013, via OMC Investments, their fairly steady expenditure rate of £150K per year is maintained this year, but since their income was £30K rather than £140K they ended up with a £120K deficit in spending, eroding their capital down to £766K. At this rate, in 6-7 more years they will be out of funds and out of luck. It seems that no-one really likes CASH or wants to give them money in any quantity - at least, not while the world is watching.

The note in the "Movement in funds" section on p.33 is amusing:

The designated fund will provide working capital to the charity to enable it to continue its unique activities whilst the trustees implement their fundraising strategy.
Yes, I'd be interested in what that strategy is going to be. Are they going to try to tap government funds in the classic fakecharity game - lobby the government to give them money to lobby the government? I'll be watching the CASH website and their subsidiary organisation Action on Sugar to see what they're up to.

2015-02-26

Net neutrality - be careful what you wish for

I'm driving my forehead into an ever-deepening dent on my desk in despair at the news that the US Federal Communications Commission has approved new rules governing net neutrality in the USA. This may seem like the sort of news that a progressive geek like your humble bloghost would welcome, but it turns out to involve some inconvenient wrinkles.

The EFF, guardians of liberty, were originally cheering on behalf of net neutrality. Then, 2 days ago, they started to get a little concerned with some of the details being proposed by the FCC:

Unfortunately, if a recent report from Reuters is correct, the general conduct rule will be anything but clear. The FCC will evaluate "harm" based on consideration of seven factors: impact on competition; impact on innovation; impact on free expression; impact on broadband deployment and investments; whether the actions in question are specific to some applications and not others; whether they comply with industry best standards and practices; and whether they take place without the awareness of the end-user, the Internet subscriber.
In essence, the proposed rules for Net Neutrality gave the FCC - a US government agency, headed by a former lobbyist for the cable and wireless industry - an awfully wide scope for deciding whether innovations in Internet delivery were "harmful" or not. There's no way that this could go horribly wrong, surely?

Broadband in the USA

Now, let's start with the assertion that there is an awful lot wrong with broadband provision in the USA currently. It's a lot more expensive than in the UK, it's almost always supplied by the local cable TV provider, and in general there is very little if any choice in most regions. See the broadband provider guide and choose min, max of 1 - there's an awful lot of the USA with monopoly provision of wired high-speed internet.

The dominant ISPs with high-speed provision are Comcast, AT+T, Time Warner, CenturyLink and Verizon. It would be fair to say that they are not particularly beloved. Comcast in particular is the target of a massive amount of oppprobium: type "Comcast are " in your favourite search engine, and you get autocompletion suggestions including "liars", "crooks", "criminals". American broadband is approximately twice the price of British, and you generally get lower speeds and higher contention ratios (you share a pipe of fixed size with a lot of people, so if your neighbours are watching streaming video then you're out of luck). As effective monopolies, ISPs were in a very powerful position to charge Internet services for streaming data to their customers, as last year's Comcast-Netflix struggle showed - and it ended with Netflix effectively forced to pay Comcast to ship the bytes that Netflix customers in Comcast regions were demanding.

Google's upstart "Google Fiber" offering of 1 Gbps (125 MB per second) fiberoptic service tells a story in itself. It's targeting a relatively short list of cities but has been very popular whenever it opened signups. It has spurred other broadband providers to respond, but in a very focused way: AT+T is planning to offer 1Gbps service, but only in Google Fiber's inaugural area of Kansas City which is impressive in its brazenness. Other community-based efforts are starting to bear fruit, e.g. NAP is proposing their Avalon gigabit offering in part of Atlanta, Georgia. However, most of the USA is still stuck with practical speeds that have not changed noticeably in half a decade. Entrenched cable ISPs have spent plenty of money on lobbyists to ensure that states and cities make it expensive and difficult for newcomers to compete with them, requiring extensive studies and limiting rights to dig or string fiber-optic cable to residential addresses.

So there's clearly a problem; why won't Net Neutrality solve it?

The ISP problem

Net neutrality essentially says that you (an ISP) can't discriminate between bytes from one service and bytes from a different service. Suppose you have two providers of streaming Internet movies: Netflix and Apple iTunes. Suppose Comcast subscribers in rural Arkansas pay Comcast for a 20Mbps service, easily sufficient for HD streaming video. Comcast controls the network which ends at their customers' home routers, and when it receives a TCP or UDP packet (small chunk of data) from their customers they will look at its destination address and forward it either to its destination - e.g. a server in the Comcast network - or to one of the other Internet services they "peer" to. Peering is a boundary across which Internet entities exchange Internet data. When data comes back across that boundary with the address of one of their customers, Comcast routes the data to the customer in question. So far, so good.

Now the customer is paying Comcast for their connection, so it's not really reasonable for Comcast to force them to pay more for more data above and beyond the plan they've agreed. If you've got a 20 Mbps connection, you expect to be able to send / receive 20Mbps more or less forever. Comcast might have a monthly bandwidth cap beyond which you pay more or get a lower speed, but that should be expressed in your plan. Comcast might weight certain kinds of traffic lower than others, so that when 20 people are contending for use of a 100 Mbps pipe traffic which is less sensitive to being dropped (e.g. streaming video) is dropped more often than more sensitive traffic (web page fetches), but that's all reasonable as long as you know how many people you're contending with and what the rules are.

Streaming video is one kind of traffic that's problematic for ISPs: it requires very little bandwidth from the paying customer. They send an initial message "I want to see this video" and then a low volume of following messages to control the video stream and assure the video streaming service that someone really is still watching it. From Comcast's point of view, though, they have a large amount of latency-sensitive traffic coming into their network from a peering point, so they need to route it through to the destination user and use up a large chunk of their network capacity in the process. If lots of people want to watch videos at once, they'll have to widen the incoming pipe from their peer; that will involve buying extra hardware and paying for its associated management overhead so that they can handle the traffic, as long as they are the limiting factor. (Their peer might also be the limiting factor, but that's less likely).

So the more data users stream concurrently, the more it costs Comcast. This can be mitigated to some extent by caching - storing frequently used data within the Comcast network so that it doesn't have to be fetched from a peer each time - and indeed this is a common strategy used by content delivery networks like Akamai and video streaming firms like YouTube. They provide a bunch of their own PCs and hard disks which Comcast stores inside its datacenters, and when a user requests a resource (video, image, music file, new operating system image) which might be available in that cache they will be directed to the cache computers. The cache will send the data directly if it's available; if not, it will download it and send it on, but store it locally so if someone else requests it then it's ready to send to them directly. This has the effect of massively reducing the bandwidth for popular data (large ad campaigns, "Gangnam Style" videos, streaming video releases), and also increases reliability and reduces latency of the service from the user's perspective, but costs the provider a substantial overhead (and operational expertise) to buy, emplace and maintain the hardware and enable the software to use it.

The non-neutral solution

If Netflix aren't willing or able to pay for this, Comcast is stuck with widening their pipe to their peers. One might argue that that's what they're supposed to do, and that their customers are paying them to be able to access the Greater Internet at 20Mbps, not just Comcast's local services. But Comcast might not see it this way. They know what destination and source addresses belong to Netflix, so they might decide "we have 100 Gbps of inbound connectivity on this link, and 50 Gbps of that is Netflix video streaming source addresses at peak. Let's reduce Netflix to a maximum of 20 Gbps - at peak, any packet from Netflix video streaming sources has a 60% chance of being dropped - and see what happens."

You see where the "neutrality" aspect comes in? Comcast is dropping inbound traffic based solely on its source address - what company it comes from. Only internal Comcast configuration needs to be changed. From the customer's point of view, Netflix traffic is suddenly very choppy or even nonfunctional at peak times - but YouTube, Facebook, Twitter etc. all work fine. So Netflix must be the problem. Why am I paying them money for this crap service? (Cue angry mail to Netflix customer support).

Net Neutrality says that Comcast can't do this - it can't discriminate based on source or destination address. Of course, it's not really neutral because ISPs might still blacklist traffic from illegal providers e.g. the Pirate Bay, but since that's normally done at the request of law enforcement it's regarded as OK by most.

The problem

The USA has handed the Federal Communications Commission, via the "general conduct" rules, a massive amount of control of and discretion in the way in which ISPs handle Internet traffic. It presumes that the FCC has the actual best interests of American consumers at heart, and is intelligent and foresighted enough to apply the rules to that effect. Given the past history of government agencies in customer service and in being effectively captured by the industries they are supposed to regulate, this seems... unwise.

2015-02-15

Failing to listen to the sounds of Chinese silence

I was moved by an interesting yet flawed piece by John Naughton in the Grauniad, analysing the kinds of censorship applied by the Chinese government:

So they [researchers] clicked on the URLs associated with a sample of posts and found that some – but not all – had vanished: the pages had disappeared from cyberspace.
The question then was: what was it about the "disappeared" posts that had led to them being censored? And at that point the experiment became very interesting indeed. First of all, it confirmed what other researchers had found, namely that, contrary to neoliberal fantasy, speech on the Chinese internet is remarkably free, vibrant and raucous. But this unruly discourse is watched by a veritable army (maybe as many as 250,000-strong) of censors. And what they are looking for is only certain kinds of free speech, specifically, speech that has the potential for engendering collective action – mobilising folks to do something together in the offline world.

The study quoted is indeed interesting, and highlights one particular and significant aspect of Chinese censorship. Where Naughton fails, though, is in failing to note the unseen, and this is picked up by CiF commentator steviematt:

The Harvard research and Gary King's opinion are both flawed beyond belief.
It only factors the number of posts that were originally published and then disappeared over the course of weeks and months. It ignores the fact that most posts that are critical never have a chance of passing through the filters in the first place.
Indeed, Naughton fails to notice that many of the websites that the West takes for granted in being able to express their opinions are completely blocked in China. Within China, sites like Twitter and Facebook are essentially completely unavailable. YouTube: no chance. You can get to a limited set of Google sites (search and maps are on-and-off accessible in my experience), but it's very iffy. Blogger seems completely blocked. Bing search seems to work fine though. Why is that?

It's because if you are a western firm who wants to provide an Internet site within China, you have to partner with a Chinese company and accept the conditions of serving users within China - key in this is agreeing to provide identity information of your users (source IP addresses , times logged on etc.) at the "request" of the government. The case of Yahoo and the Chinese dissident Shi Tao is illuminating:

According to a letter Amnesty International received from Yahoo! (YHOO), and Yahoo!'s own later public admissions, Yahoo! China provided account-holder information, in compliance with a government request, that led to Shi Tao's sentencing.
Jerry Yang, then-CEO of Yahoo, got roasted by Congress for providing this information when this story came out. Truth be told, though, he really didn't have much choice - Yahoo had presumably agreed to these conditions when it started serving China-based users. If you don't want to play ball with those conditions, and it seems that Google, Twitter and Facebook don't, you're going to be serving outside China and prone to getting blocked by the Great Firewall.

So when Naughton comments "only some kinds of activities are blocked" it's actually in the context of "only some users are willing to discuss these kinds of activities on sites where they know the government has the right to waltz in and demand their details at any time" (before presumably visiting them at home and offering them an extended stay at a pleasant little camp out in the country, for a year or ten.)

Rumours suggest that Facebook might announce something aimed at Chinese users but it's not obvious how they're going to deal with the existing restrictions. Still, Zuckerberg's a smart guy and doesn't seem to be an obvious patsy for the Chinese regime, so it's possible he's got something clever up his sleeve. Stay tuned.