Reading the runes
Jun. 8th, 2016 09:07 am![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
peterbirks I have long been fascinated by opinion polls -- indeed, by statistics in general. That I am hopeless at the mathematical side of statistics just adds to my fascination.
Opinion polls seek to guess how people in their millions will act on the basis of relatively small samples. It was immediately obvious that just asking 1,000 people at random in the street would be at risk of generating an erroneous response (although the degree of that likely erroneousness is possibly less than many would think).
Pollsters realized that a good way to increase the accuracy would be to ensure that the sample of 1,000 people reflected as much as possible the population as a whole - age distribution, sex distribution, and so on.
This, however, leads to another problem. Over the years it was discovered that, shock, horror, what people said was not always the same as what they did. Even more concerning, what people really believed was often different from what they did (the famous female claimed belief in what attracts them to a sexual partner/life partner differs drastically from empirical evidence of whom women actually choose). The ways in which questions were phrased also had a significant impact on the response.
Clearly, opinion polling was something of a nightmare. And, given the misperformance of the pollsters leading up to the last general election in the UK, the pollsters still haven't got it right.
So, what is it that they are getting wrong?
The two major problems are the aforementioned "tendency to deceive" (people respond with what they think they ought to say, rather than what they really feel) -- a factor that has been a curse for the intellectual left-wing for decades. These days they flood Twitter and Facebook, demonstrate to their own satisfaction that the argument has been won, and wake up the day after voting to have been told "fuck off". The secret ballot allows visceral emotions to come into play. A person might not vote for a candidate because he or she doesn't like the fact that the candidate is fat. But no respondent to an opinion poll is likely to say that, and no online social media campaign is going to mention "the elephant in the room" if a candidate is 25 stone-plus and female.
The second problem is more complex -- one that is only just coming to be fully appreciated. That is, how do you decide what is a "representative" sample?
In the early days of polling, the techniques were primitive - mainly age and sex. This came most unstuck in 1948 in the US, when a telephone poll predicted that Harry Truman would lose. As seems obvious now, the key was in the phrase "telephone poll". With a market penetration still under 50%, people with a telephone were markedly more likely to be better off, and, therefore, Republican voters.
So, clearly we have to add "income" to our representative mix. In fact, what pollsters need to do is to add any variation in the make-up of the general population that is positively correlated with the way that people are likely to vote.
You can see the problem here. This in itself is something of a judgment call. As it is a sample, the pollsters must by definition filter out "irrelevancies". The problem appears to be that in a dynamic society, some things that used to be relevant have ceased to be so, while other things which did not use to be important, now are.
With the referendum, where "all bets are off" when it comes to traditional party politics, the problem is multiplied. What on earth is "relevant" when it comes to picking a true representative sample, when the split is not along traditional party lines? Also, there appear to be significantly more "elephants in the room" -- things which neither side are prepared to mention, but which could be significant factors when it comes to voting. That in turn feeds back to a higher likelihood of a "propensity to deceive" and a greater danger that the phrasing of the poll question will distort the result from reality.
I'd quite like to see some sample results from randomly asking 200 people each in, say, five streets in England. I suspect that the numbers obtained would not be a long way different from the carefully calculated "representative samples".
In poker I have long argued that you can learn more from small samples than you think. The conventional wisdom in poker is that you can't learn anything from, say, a player's actions over five hands. I argued, way back in the early 2000s, that if this was all that you had to work with, ignoring it was stupid, just because there was a higher probability that the answer you obtained would be wrong. Sure, with five hands the standard deviation is many times higher than it would be on a sample of 50, 500 and 5,000. But it is not TEN times higher than the sample of 50 - it's closer to three. It is not a thousand times higher than a sample of 5,000 -- it's closer to 80.
Sure, the conclusion you reach if the player raises four times and folds once in his or her first five hands might be erroneous. But the probability that this player is loose-aggressive is still significantly higher than it was when you had a sample size of zero.
In other words, completely random samples (and I mean virtually completely -- no self-selection on the basis of sex and age and only a minor one on grounds of geography) might have their place. And they have one plus -- they are much easier, quicker and cheaper to compile.
Peter Kellner, in his blog, referred to an interesting statistc -- that being the percentage of people who see Brexit as a "risk" compared with Remain as "safe". The rough percentage appears to be that 10pp more people see Brexit as the "risk option".
This offers an interesting left-field take on the referendum. It means that 10pp of the "Remain is safer" believers, or 5% of voters, would need to think that Brexit was "a risk worth taking", to make Brexit the likely winner. The remaining voters would be committed to Brexit or Remain either way. That 1-in-20 number strikes me as uncomfortable reading for Brexiters. Look at the general population's attitude to risk-taking on a major level. Nearly all of it is about risk-avoidance. Indeed, the huge risks that they do take are usually ones that they take unwillingly and, not infrequently, without the knowledge that they are taking the risk (see 40-year mortgages, Equitable Life, negative equity in the early 1990s). When a risk is known and perceived, and conceived to be significant, people usually plump for safety.
From that point of view, the Remainers' best argument could well be the one that they are uncomfortable to make -- that, even if being in the EU is shit, the equivalent of an abusive relationship -- even if this is the case -- we are now so inextricably tied into the EU that the risk of leaving is too great. That, no matter how bad it is, leaving would be too big a risk.
This is what I mean by "the elephant in the room". It's probably Remainers' strongest argument, but it is one that no Remainer is willing to accept exists (or, if they are, willing to campaign on it).
no subject
Date: 2016-06-08 10:55 am (UTC)no subject
Date: 2016-06-08 01:06 pm (UTC)Part of the issue is that we have a binary choice but not a binary problem.
The first part is that people do not agree what the problem is. The likes of you and I do not like the structure of the EU at the moment, and we do not think it sustainable.
However, even amongst all of these people who agree on that problem, some think it would be better to try to stay and attempt to fix it, while others think that it would be best to leave and watch the smash-up from the outside.
A possible historical corollary is the euro. People say "wow, that was a bullet we dodged", but it's possible (I'd say, 30% to 40%) that if we HAD joined the euro, it would no longer exist. The presence of the UK in the single currency would have hastened its demise. Perhaps if we vote to stay in the EU, the inevitable change will be accelerated, and that will be for the long-term good. Perhaps, on the other hand, it would be better for us to watch it as an outsider.
Cameron has finally begun to address this point, referring to "quitters" and accepting that the real battle would begin the day after we vote to stay in.
The second binary choice is the headline one. A general pro or anti-Europe feeling and, more particularly, anti-immigration. That's a more simplistic argument whereby what you think the problem is (or isn't) necessarily dictates the way you will vote.
The major influence on my voting intention so far has been the Remainers' initial determination to dismiss all Brexiters as Farage-type little Englanders. The Remainers refusal to engage in economic debate and their "we have the high ground" attitude has royally pissed me off. You will not persuade me to vote one way because I disagree with the majority of the people who are voting the other. For me the major factor is -- do I want to be on the inside or the outside when the inevitable future reshaping of the EU (and Europe) takes place. I think that the doom-mongering of the Remainers is either bluff (particularly from the City and big business -- they just don't fancy the extra work and uncertainty) or innumeracy (trust me, the economy would not collapse by even a quarter of many of the Remainers' predictions, no more than celebrities ever follow through with their threat to leave the country if a tax is increased by a relatively small amount.)
Stats
Date: 2016-06-08 06:06 pm (UTC)But they'll still "know" that, in some way, "it will be bad."
Now add a further mind-bender into the equation. What are the main factors driving net immigration?
a) The open frontiers of the EU
b) The pull factors of "a better life than Iraq/Yemen/Afghanistan/Nigeria."
The man in the street will assume -- I think; I've argued this down the pub -- that the massively dominant factor is (1).
But what if it's (2)?
All you end up doing is to swap EU immigrants with West African immigrants. Now, given that Nigerians (for example) are one of the more successful immigrant groups in terms of basically doing the hard yakka ... that might actually be a good thing.
But second order effects are not really what this referendum is about, are they?
Machine Learning
Date: 2016-06-08 06:15 pm (UTC)Don't let humans normalize for population bias. Let machines do it.
This is, after all, essentially the sole intellectual principle behind Google and Bing and other search engines. (Billions of dollars of processing power helps, but we're talking a far more limited domain here.)
Humans will inch forwards according to their own biases. Gender this decade. Race the next. And so on.
Machines will exercise a purely neutral ranking, given (in the representative cases of Bing and Google on yer average query) something like SEVENTY possible signals.
I don't believe anybody has applied ML to polling techniques yet, but I suggest that the technology is just about mature to use.
no subject
Date: 2016-06-24 02:49 pm (UTC)Too late mush. We're leaving anyway.
No thanks to self-loathing wet liberals like yourself.
no subject
Date: 2016-06-24 11:04 pm (UTC)But, hey, the EU is run by people who know better and don't need to listen because they are not elected. Niall L.
no subject
Date: 2016-06-25 02:45 pm (UTC)"It appears that the majority of Sunderland residents who voted "leave" thought it meant they would be getting out of Sunderland."
Oh, it's one of those delightful middle-class lefty jokes aimed at working class labour party voters who don't do what is expected of them.
Get out the horse whip!
You ignorant tosser.
no subject
Date: 2016-06-27 09:16 am (UTC)Hilarious watching them do a U-turn on Corbyn. A God one moment, a sod the next.
There they all were camping outside St Pauls, protesting against the 1%. The next moment they are in bed with the 1% and the TTIP supporters over Brexit.
They grudingly accept a Tory government with 37% of the vote but want to over-turn a 51% majority vote.
No World War 3, no financial market melt-down.
Time for lefties to stop taking hallucinogens and face reality.
When bankers and corporate leaders tell you what to do then surely you must smell a rat??? All they care about is themselves. You mean nothing to them.
no subject
Date: 2016-06-29 08:36 am (UTC)no subject
Date: 2016-07-02 12:02 pm (UTC)"I am a free-trader"
Surely Birks is not stupid enough to think that Brexit voters hate Europe. The EU and Europe and not the same. Brexit voters know that they are European but that they want a better Europe than the EU can provide.
Does Birks think that free-trade will cease? For EU nations to stop trading with the UK will be case of shooting themselves in the foot.
When you live life on Twitter or Facebook you highten your isolation by associating only with like-minded people.
Birks, you have created a bubble of unreality around you. A Jobsian Unreality Distortion Field.
Maybe you are going through a mid-life crisis and want to "identify" with young people. A life gone to waste. No wife. No children. You want to feel that you have achieved something but you never will.