Quantum Conundrum, Puzzling People, and Survey Statistics
Order matters, we all know this when it comes to math, but did you know the order of questions asked can affect how you answer them? It’s true and it isn’t new news, the question-order effect is why survey organizations normally change the order of questions between different respondents, hoping to cancel out this bias. But that isn’t the interesting part, not by a long shot.
It turns out that quantum theory is a much better predictor of the survey results than conventional methods of predictions.
“Human behavior is very sensitive to context. It may be as context sensitive as the actions of some of the particles that quantum physicists study,” said Zheng Wang, lead author of the study and associate professor of communication at The Ohio State University.
“By using quantum theory, we were able to predict a surprising regularity in human behavior with unusual accuracy for the social sciences in a large set of different surveys.”
Question-order effect can be seen, for example, in one of the surveys used in the study. This was a Gallup poll that asked Americans– among other things– whether Bill Clinton was honest and trustworthy and whether Al Gore was honest and trustworthy.
The survey changed the order in which these questions were asked between the respondents and –as was to be expected– there were question-order effects found. In other words, when respondents were asked about Clinton first, 49 percent said that both Clinton and Gore were trustworthy. But when respondents were asked about Gore first, 56 percent said that both were trustworthy.
The pattern that quantum theory predicted — and that the researchers found — was that the number of people who switch from “yes-yes” to “no-no” when the question order is reversed must be offset by the number of people who switch in the opposite direction. Crazy idea, I know, but stick with me here.
In this case, the number of people who said “no-no” – that both Clinton and Gore were not trustworthy – went from 28 percent when the Clinton question was asked first to 21 percent when Gore was asked about first.
That 7 percent decline essentially cancels out the 7 percent increase in the number of people who said “yes-yes” when the question order was reversed.
Likewise, the number of people who switched from “yes-no” to “no-yes” was offset by the number of people who switched in the opposite direction.
The researchers called this phenomenon “quantum question equality.” And crazy enough, they found it in every one of the surveys studied.
“When you think about it from our normal social science perspective, the finding is very bizarre,” Wang said. “There’s no reason to expect that people would always change their responses in such a systematical way, from survey to survey to create this pattern.”
But from a quantum perspective, the finding makes perfect sense, Wang said. “It is exactly what we would have predicted from quantum theory. We mathematically derived this precise prediction of quantum question equality from quantum theory before we looked at any data. This had to be true if our theory is right.”
Hopefully I can sum up the next interesting bit in a way that makes sense as this is getting a little long. This relationship only holds true in, at least in quantum physics, when new information isn’t added into the system. In other words, the questions had to be asked back to back without giving any other data.
This too ended up being true, the researchers looked at another survey [not included in the 70 surveys they used], in which participants were asked whether disgraced former baseball players Pete Rose and “Shoeless” Joe Jackson should be admitted to the Baseball Hall of Fame. The order in which people were asked about the two players was varied to deal with the question-order effects. But the results from this survey didn’t show, as predicted by the researchers, the pattern found in the 70 surveys in the study.
That’s because in between asking each question, the surveyors introduced new information by explaining to participants who these baseball players were and why there was a controversy about whether they should be admitted to the Hall of Fame.
“Usually, in the social sciences we’re talking about parameters: If we can predict that one factor is always larger or smaller than another, we consider that a strong finding,” she said. “But here we found a quite precise answer that is always nearly zero – the number of people who switch an answer one way are always offset by the number of people who switch in the opposite direction. That number never changed. In other words, their difference is always nearly zero. And that level of exactness is almost never found in social science research.”
What is stranger than the fact this quantum physics model works for survey data like that is the simple question, why? Why would quantum mechanical equations predict with remarkable accuracy how people are going to respond to survey questions? Why are the number of people who switch from “yes-yes” to “no-no” when question order is reversed be offset by the number of people who switch in the opposite direction?
For now at least, there is no explanation. Proving once again, sometimes truth is stranger than fiction.
Want the full study? How about another question?
How about another question? Want the full study?
Well you can find the full study —here! [see what I did there?]
Zheng Wang, Tyler Solloway, Richard M. Shiffrin, & Jerome R. Busemeyerb (2014). Context effects produced by question orders reveal quantum nature of human judgments Proceedings of the National Academy of Sciences : 10.1073/pnas.1407756111