In physics, researchers use the scientific method to test specific predictions. If the evidence gathered in a well-designed study does not match the exact prediction of the theory, researchers know that the theory is at least partially incorrect and should be revised or discarded for a better one.
Psychology doesn’t work the same way, according to a recent article by Perspectives on the Psychological Sciences, written by Stijn Debrouwere and Yves Rosseel at Ghent University in Belgium. Psychological theories are rarely rejected, regardless of the results obtained by researchers. Instead, researchers go to great lengths to explain inconvenient findings and design their studies to be vague enough to “support” any theory.
Debrouwere and Rosseel write that psychology is “an experimental science in which interventions lack ecological validity, a science that theorizes freely but misses some of the basic facts, and a science that uses statistics to whitewash uncertainty.”
One of the sharpest criticisms of psychological research is that strikingly positive studies—which become accepted “common knowledge”—almost always fail to replicate when other researchers retest the same effect. For example, a study in which researchers attempted to replicate results from the psychological sciences found that only 36% of results were confirmed after another test.
And according to the authors, the most well-known psychological experiments, such as Zimbardo’s Stanford prison experiment, run the gamut from unethical to poorly documented conduct to outright lies. Little can be generalized from studies that are misleading at best and fraudulent at worst. In a survey of 2,000 psychologists, more than half admitted to misrepresenting their results (such as changing outcome measures or “spinning” negative results to appear positive), so this problem is pervasive. .
Some researchers have suggested that improving the quality of psychological research can solve this problem. Solutions have been proposed, such as pre-registration of trials, open access to data, documentation of effect sizes (instead of a simple binary measure of statistical significance) and alerting readers to biases, for example, financial conflicts of interest. And these methods would likely help avoid fraud, “turn” negative conclusions into positive conclusions, and other forms of bias.
However, Debrouwere and Rosseel argue that these solutions do not solve the underlying problem: the scientific method cannot really be applied to psychological questions.
They write: “We believe that psychology is fundamentally incompatible with theoretical science based on assumptions.
According to Debrouwere and Rosseel, predictions in physics are extremely specific and testable. For example, Newtonian mechanics predicted that light would bend slightly around a mass like a sun, and Einstein agreed, but according to Einstein’s theory of relativity, the specific value amount that he would bend was different. When astronomers measured this during an eclipse, they found that light distorted to the exact degree specified by Einstein, not Newton. This demonstrated conclusively that something was missing from Newton’s theory and that Einstein’s theory of relativity was correct.
When it comes to psychological science, however, Debrouwere and Rosseel write that the researchers make vague predictions and, even when they are contradicted, the researchers explain the fact that their study failed and in effect claim that the study was still a success.
They give this example: “A psychologist can posit that people with high social status will tend to do what they can to maintain that status (our theory) and therefore might be inclined to boycott social competitors or denying access to important resources (our theory). hypothesis). However, if it turns out that instead those with high social status tend to be magnanimous, that too is easily explained: generosity is a great way to show one’s higher status. Our theory aligns equally well with both hypotheses, even though they are diametrically opposed to each other.
In the physics example, measuring the degree to which light bends around a large celestial body would inevitably prove that at least one of the two theories was wrong. Either it would bend to the exact degree predicted by Newton, predicted by Einstein, or neither. Whatever the result, it would conclusively demonstrate that at least one theory (and possibly both) was wrong.
But in the psychology example, whatever the researchers found didn’t prove anything. Any result was vague enough – and the theory itself vague enough – to be explained.
So Debrouwere and Rosseel ask, what is the point of tests like these?
“What if competing theories lead to similar causal models? What if we simply cannot predict the magnitude of an effect even with the most sophisticated theories? What if it is unclear when a phenomenon will or will not manifest? »
However, the authors present an answer. They write that the emphasis on the science of hypothesis testing is doomed to failure in psychology. But psychology can focus on the descriptive and taxonomic science that forms the basis of disciplines like zoology, botany, mycology, and even meteorology.
“In order to know more about the world,” they write, “we can explore and document the wide variety of phenomena, organize them, see if there are obvious regularities.”
This type of science forces researchers to set aside their biases about generalizable laws of human behavior and instead observe with curiosity facets of human experience in as many different contexts as there are people:
“This leads to a very different type of research in which we are not proving or disproving theories, but rather trying to find the conditions under which a particular phenomenon or mechanism will or will not appear, which strengthens and weakens it. Through slow, careful mapping of territory, we will begin to see if a behavioral or cognitive phenomenon is widespread, robust, or ephemeral, if it strongly affects our actions or life outcomes, or if it is just a a curiosity with limited impact.
Debrouwere, S., & Rosseel, Y. (2021). The conceptual, cunning and conclusive experiment in psychology. Perspectives on the Psychological Sciences. https://doi.org/10.1177/17456916211026947 (link)