I'm starting to believe that May is trolling the Guardianistas

I thought that the chorus of butthurt from the why-didn't-the-plebs-listen-to-ME part of the Remain camp was finally starting to die down, but then May appointed Johnson as Foreign Secretary, and oh my goodness. My Twitter feed and Farcebook timeline have erupted in caterwauling once again.

Note that this has the effect of focusing the limited Guardianista attention on Johnson and his various alleged[1] faux pas, and there's been very little comment on the appointment of the sharp and strongly pro-Brexit David Davis as "Minister for Brexit". I rather suspect Davis is going to be the source of most of the actual heartache for the Remainers in the next couple of years.

[1] Most of which I suspect they're overselling. Johnson has his flaws, Heaven knows, but he's a smart cookie, extremely well travelled, with a highly multinational family. And I'd endorse him as Foreign Secretary solely on the basis of his trolling of the Chinese about ping pong at the Beijing Olympics.


Denatonium Benzoate loses its crown

Also known as Bitrex, Denatonium Benzoate held the record for the most bitter substance on earth until 24th June 2016. A teaspoon of the substance added to an Olympic size swimming pool (volume 2.5M litres) makes the water noticeably bitter. Bitrex has been a very successful additive to poisonous substances to prevent accidental ingestion, such as car antifreeze.

Sadly for manufacturer Macfarlan Smith, since 24th June Bitrex's record has been overtaken by the UK Guardian opinion page. One opening paragraph has the same bitterness impact as approximately 300ml of Denatonium Benzoate. Rumours suggest that Macfarlan Smith has opened negotiations with Jonathan Freedland, Nick Cohen and Polly Toynbee for purchase of their spleens as a manufacturing source of the Bitrex successor.

It is serendipitous that the name "Bitrex" is an anagram of the new product: "Brexit".


Toys firmly out of prams

I predicted a certain amount of tantrums, but really didn't think it would get this bad this quickly. Scotland and London wanting to split off and rejoin Europe, Labour Party stalwarts gunning for Corbyn (who, up until a couple of hours ago, must have thought he'd played a blinder) and Twitter and Facebook in meltdown with Remainers calling Leavers "racist idiots" and worse.

Heavens sake, you're all adults, bloody act like it. This was a full national referendum with a turnout of 74% which is way above recent elections. If your side lost, sit down and put up with it. Don't whine like a three year old deprived of an ice cream. Leave seem to have been a heck of a lot more restrained in their unexpected win than you'd have been in their place.

Not entirely surprised by Cameron chucking the towel in. He seems to be one of the few people today (and maybe the only Remainer) acting with dignity.


Referendum predictions

I have no idea on the actual result. I don't think I could place a bet if I was offered 50:50 odds on each choice. That said, the breakdown by region is going to be very interesting, and I wonder if the rain/floods will hit turnout in the SE, and whether that will make a material difference.

If "Remain" wins: The Guardian (and, less obviously, BBC) will be insufferable. Juncker et al will keep true to their promise not to give any concessions to the UK, even if the result is knife-edge. UKIP effectively dissolves in a frenzied pit of backbiting. Who knows what the UKIP voters will do at the next election?

If "Leave" wins: Immediate witch-hunt from Guardian, BBC. Cameron resigns. Panic in Europe. Stock markets burning. Sweden and maybe Denmark start feeling popular pressure to exit or form referendum. Juncker et al refuse any trade deals with the UK. Boris's hair a fixture on the international news.

I've observed my Facebook stream becoming increasingly stridently pro-Remain over the past 2 weeks. The Leavers are keeping very quiet, presumably because they're swamped by insufferable Remainers if they post anything. Remain posts seem to be relatively free of Leaver comments. So is this due to Remain having an insurmountable majority, due to me having a supermajority of Remain friends, or because the Leavers don't care what the Remainers think or do?

Going by their selection of stories and interviewees, the BBC have steadily abandoned impartiality over the past couple of weeks. The only really studiously neutral Beebite I've seen has been the indefatigueable Kuenssberg.


Weasel will find a way

After the furore last year when it turned out that UK airport shops were demanding boarding passes to save themselves VAT but not save you any money I assumed that this was the effective end of the weasel. From my recent experience at Birmingham International (motto: "We put the 'slack jaw' in 'security'") it seems not.

First stop: the bookshop, to buy some doorstop-sized illiterate literature. No shortage of supply. I present the volume to the lady at the till who demands: "Boarding pass?" with no hint of shame. I enquire whether it's actually mandatory, at which point she rings up the transaction with no further questions. 1-0.

Next stop: W H Smith, for a magazine. Avoiding the single human-manned till I opt for the self-service till. I scan the magazine for a grand total of £2.50 - and it asks for a boarding pass, and won't proceed until I scan one. I hit the "my boarding pass won't scan" button, wait a minute for the roaming attendant to punch the override and proceed on my way. But hell, I remember the huge fuss in August 2015 about this. It seems that the airport shops were content to let the hubbub die down, then go back to their old ways.

Don't let them do this! Make them pay a cost in salaried worker time for each time they demand a boarding pass. Once the average worker salary rate times delay is more than the expected VAT, they will shut up about the boarding passes and let us buy our dubious literature un-monitored and without delay. (Until 1-2 years later when some bright MBA spark spots an opportunity to re-introduce the practice, at which point we hang them from the Heathrow radar pillar as a warning to others.)


The implications of the "Out" threats

With the UK In/Out referendum less than three weeks away, and the BetFair odds on "Leave" starting to come down - albeit still very far from 50-50 - it has been instructive to listen to the veiled, and not so veiled, threats about what will happen if the populace vote for "Leave".

A good example was the comment in late May from Jean-Claude "Piss Artist" Juncker, European Commission President:

"The United Kingdom will have to accept being regarded as a third country, which won't be handled with kid gloves.
"If the British leave Europe, people will have to face the consequences -- we will have to, just as they will. It's not a threat but our relations will no longer be what they are today."
Apparently EU officials don't want to have lengthy negotiations[1] with a Brexited UK, which makes sense. But of course, the easiest course for both sides would be to retain status quo ante: continue trade under the same conditions and tariff schedule as before. Why wouldn't this be the starting point? In general, trade tariffs hurt the populace of the country / countries that impose them: they make imported goods more expensive for their populace. The main function of trade tariffs is to protect local industry from "abuse" from "dumping" by foreign manufacturers: selling goods below the cost of local produce. This may not be good for local industry, but it's certainly good for anyone who wants to buy those goods, at least in the short term.

It seems fairly clear that, whatever the merits of the "Remain" and "Leave" positions, the EU establishment is happy to cut off its population's noses to spite the UK's faces. One has to ask: if national membership of the EU is supposed to be of benefit to the population, why would the EU take action to screw over all their population in order to punish a member nation that wanted to leave?

[1] Note that the EU can apparently spare the manpower to negotiate a mostly free trade agreeement with Canada, which has half the population and a bit more than half the GDP of the UK.


I'm starting to think that Trump might just pull this off...

Trump's political opponents seem hell-bent on getting him elected. Dixit Linus Torvalds, father of Linux and otherwise political moderate:

It used to be that the only thing that made Donald Trump look good was comparing him with the other Republican candidates. Because even a whiny five-year old megalomaniac looks positively stellar when compared to a religious nut who loves the death penalty.
Now, those other Republican candidates are gone. That should make for a saner baseline, no?
These days, it's the anti-Trump protesters that make "the Donald" look good in comparison.
Christ, people. You're doing it wrong.

One can only assume that this is in reference to the sustained violence at the Trump rally in San Jose, CA last night which seemed to be perpetrated by a motley crew of students, Mexican nationalists and union-backed thugs and involved Trump supporters being pelted with eggs, sucker-punched, and clubbed on the side of the head. I watched the videos and it was indisputably appalling. The American Constitution has the First Amendment which guarantees the right to free speech; as P. J. O'Rourke remarked, it also implies the responsibility to live with the consequences. If you vocally support Trump because you hate people with brown skin, you're an asshat and the concomitant public opprobrium is your problem. But if you are physically attacked for supporting the Republican party candidate for President, then there are other laws which should come into play and they should be squarely aimed at - and enforced on - your attacker.

The Bay Area news organisations - with the commendable exception of KRON 4 were carefully keeping the lid on reports of the violence last night. Even CNN sat on it until reporting on the violence was unavoidable; even then, there were strenuous efforts to deflect the blame towards Trump. San Jose mayor Sam Liccardo's comments were particularly awful:

"Our police officers have done an extremely courageous and professional job so far," Liccardo told The Associated Press Thursday night. "At some point Donald Trump needs to take responsibility for the irresponsible behavior of his campaign."
Yes, heaven forfend that a Presidential candidate actually speak clearly about his intentions to enforce the law of the land and secure a nation's borders. There are very reasonable arguments to be had about whether this is a good idea or not, but the implicit blaming of Trump for the actions of the protestors was disgraceful. Liccardo has the luxury of an electorate who would vote him in based on party affiliation even if it came out that he framed OJ, spied for China, and buggered raccoons on his free weekends, so the concept of trying to win an election based on popular policy is doubtless alien to him. His blatant repudiation of the First Amendment might well be related to metropolitan California's sustained attack on the Second Amendment, but neither does him any credit.

Faced with a Twitter firestorm, he tried to walk this back later on:

but it's clear where his sympathies lie.

If I were Donald Trump, I'd be campaigning from now until November in Democrat stronghold cities around the USA. It won't win me those states, but the widely-reported predictable riots and abuse from the opposition will steadily win me marginal voters in every marginal state around the country. Even if those marginal voters can't stand me (or my hair), they'd rather be with me than the scumballs throwing eggs and beating up women.


Hillary Clinton to give counter-terrorism speech at Stanford University

"Do as I say, not as I do!" Hillary Clinton will urge listeners to stand in solidarity with Europe in order to defeat the Islamic State group:

"Today's attacks will only strengthen our resolve to stand together as allies and defeat terrorism and radical jihadism around the world."
Heaven forfend that we strenuously deny any connection between organized terrorism and this event and instead speciously blame random YouTube videos for incitement.

Hillary certainly has balls. She probably took Bill's in exchange for keeping quiet about his dalliances...


Trump's Republican problem

Long-time readers (both of them) will know of my affection for "my favorite wonk", Megan McArdle. She has been canvassing on Twitter for information about where Donald Trump support is coming from and where it won't ever come from, and has just published a great list of anonymous quotes from lifelong Republicans who won't vote Trump even if he's the Republican candidate:

  • I've always voted Republican [...] I have generally avoided voting third-party for fear of helping the Democratic candidate win. However, if Trump wins the nomination, I will vote for the winner of the Libertarian party nomination. I will not support Trump under any circumstances.
  • [I will] stay home or not vote for President if Trump is the Republican nominee. After voting basically a straight ticket Republican since I have been eligible to vote, this is truly amazing.
  • I have never voted for a Democrat before, but I care too much about the future of this country to let a blithering imbecile become the President.
  • I hate Hillary Clinton, but at the very least I know she will do what [she thinks] is best for this country. I cannot say the same about Trump.
I'd say "read the whole thing", but be aware that you'll be there a while.

I really can't imagine many Democrats voting for (say) Republican Ted Cruz if the Democrat candidate was either Sanders or Clinton, no matter what they though of the Democrat. I wonder if this will turn out to be the most compelling reason for the Republicans to band together and stop Trump - not so much to stop him being President, but to stop him being such a horrible Republican candidate that he would keep Republican supporters at home and let in Clinton or Sanders.


Analysing the blue-red hat problem in the face of user error

Everyone knows computers are getting smarter - unless they're being programmed by a major corporation for a government contract - but there has recently been another leap in the level of smart. DeepMind (now part of Google) has built an AI that has successfully deduced the optimal solution to the hat problem:

100 prisoners stand in line, one in front of the other. Each wears either a red hat or a blue hat. Every prisoner can see the hats of the people in front – but not their own hat, or the hats worn by anyone behind. Starting at the back of the line, a prison guard asks each prisoner the colour of their hat. If they answer correctly, they will be pardoned [and if not, executed]. Before lining up, the prisoners confer on a strategy to help them. What should they do?
Tricky, n'est ce pas?

The obvious part first: the first prisoner to answer, whom we'll designate number 1, has no information about his hat colour. Assuming blue and red hats are assigned with equal probability, he can answer either "red" or "blue" with a 50% chance of success and 50% chance of getting executed; he has no better strategy for self-survival. What about the other prisoners?

Applying information theory, our system has 100 binary bits of state - 100 people, each with 1 bit of state relating to whether their hat is blue or not. We generate 99 bits of knowledge about that state as the hat-wearers give answers. So the maximum we can expect to discover reliably is 99/100 hat values. How can we get close to this?

If everyone just guesses their own hat colour randomly, or everyone says "blue", or everyone says "red", then on average 50% of people survive. How to do better? We need to communicate information to people further down their line about their hat colour.

Let's get the first 50 people in line to tell the next 50 people in line about their hat colour. Person 1 announces the hat colour of person 51, person 2 of person 52 and so on. So the last 50 people are guaranteed to survive because they have been told their hat colour. The first 50 people each have a 50-50 chance of survival because the colour they "guess" has no necessary relation to the colour of their hat. On average 25 of them survive, giving an average survival of 75% of people.

The DeepMind algorithm relies on an insight based on the concept of parity: an 0/1 value encapsulating critical state, in this case the number of blue hats seen and guessed, modulo 2. The first user counts the number of blue hats seen and says "blue" if that number is even, and "red" if odd. He still has a 50-50 chance of survival because he has no information about his hat. The second user counts the number of blue hats. If even, and the first person said "blue", then he and the first person both saw the same number of blue hats - so his hat must be red. If even, and the first person said "red", his hat must be blue because it changed the number of blue hats seen between the first person and him. Similar reasoning on the odd case means that he can announce his hat colour with full confidence.

What about person 3? He has to listen to person 1 and person 2, and observe the hat colours in front of him, to deduce whether his hat is blue; his strategy, which works for all others after him too, is to add the parity values (0 for blue, 1 for red) for heard and seen hats modulo 2, and if 0 then announce "blue", if 1 then announce "red". Follow this down the line, and persons 2 through 100 are guaranteed survival while person 1 has a 50-50 chance, for an average 99.5% survival rate.

Of course, this is a fairly complicated algorithm. What if someone mis-counts - what effect does it have? We don't want a fragile algorithm where one person's error can mess up everyone else's calculations, such as with "Chinese whispers." Luckily, a bit of thought (confirmed by experiment) shows us that both the future-casting and parity approaches are resilient to individual error. For future-casting, if one of the first 50 people makes an error then it makes no difference to their chance of survival but their correspondent in the second half of the line is doomed. If one of the second 50 people makes an error then they are doomed unless their correspondent also makes a mistake - generally unlikely, a 10% chance. So if 10% of users make errors then the approximate number of survivors is (75 - 10) + 1, i.e. 66%.

Surprisingly, the parity approach is also robust. It turns out that if user N makes a mistake then they doom themselves, and also doom user N+1 who relies on user N's calculation. But because both user N and N+1 make erroneous guesses, this brings the parity value back in line for user N+2, whose guess will be correct (absent any other errors). So the approximate number of survivors given a 10% error rate is 99.5 - 10*2 = 79.5%

Here's Python code to test the various algorithms: save it as "hats.py" and run it (e.g. "chmod 0755 hats.py ; ./hats.py" on OS X or Linux). It runs 10 trials of 100 people and reports the average number of survivors, based on a 10% error rate in hat wearers following their strategy. Default strategy is the parity algorithm.


import random

person_count = 100
half_person_count = person_count / 2
# Hat choices
hat_choices = ['r','b']
hat_opposite = {'b':'r', 'r':'b'}
# 10% error rate in guesses
error_rate = 0.1

def guess_constant(heard_guesses, seen_hats):
    return 'b'

def guess_random(heard_guesses, seen_hats):
    return random.choice(hat_choices)

def guess_future(heard_guesses, seen_hats):
    """ First half of list calls out hat of correspondent in second half of list """
    full_list = heard_guesses + ['x'] + seen_hats
    my_index = len(heard_guesses)
    if my_index < half_person_count:
        # Call out the hat of the person in the second half of the list, hope same as mine
        return full_list[my_index+half_person_count]
        # Remember what was called out by my corresponding person in first half of list
        return heard_guesses[my_index - half_person_count]

def guess_parity(heard_guesses, seen_hats):
    """ Measure heard and seen parity of blue hats, call out blue for even, red for odd."""
    heard_blue_count = len([g for g in heard_guesses if g == 'b'])
    seen_blue_count = len([s for s in seen_hats if s == 'b'])
    if (heard_blue_count + seen_blue_count) % 2 == 0:
        return 'b'
        return 'r'

def run_test(guess_fun):
    hat_list = [ random.choice(hat_choices) for i in range(0, person_count) ]
    print "Actual: " + "".join(hat_list)
    answer_list = []
    score_list = []
    error_list = []
    correct = 0
    for i in range(0, person_count):
        guess = guess_fun(answer_list, hat_list[i+1:])
        if random.random() < error_rate:
            guess = hat_opposite[guess]
        if guess == hat_list[i]:
            correct += 1
    print "Called: " + "".join(answer_list)
    print "Score:  " + "".join(score_list)
    print "Errors: " + "".join(error_list)
    print "%d correct" % correct
    return correct

if __name__ == "__main__":
    trial_count = 10
    correct_total = 0
    for i in range(0, trial_count):
        print "\nTrial %d" % (i+1)
        correct_total += run_test(guess_parity)
    print "\nAverage correct: %d" % (correct_total / trial_count)
You can change the "guess_parity" value in the run_test() invocation on the penultimate line to "guess_future" for the "warn the second half" strategy, or "guess_random" for the random choice.

This is a lousy problem for use in software engineering job interviews, by the way. It's a famous problem, so candidates who have heard it are at a major advantage to those who haven't. It relies on a key and non-obvious insight. A candidate who hasn't encountered the problem before and solves it gives a very strong "hire" signal, but a candidate who fails to find the optimal solution could still be a valid hire. The least worst way to assess candidates based on this problem is whether they can write code to evaluate these algorithms, once the algorithms are described to them.