New studies have intensified the debate over psychology’s “reproducibility" problems.
Psychology’s current reproducibility crisis would not have surprised William James, who once fretted that psychology might never transcend its “confused and imperfect state.” Photo of William James via Wikimedia Commons.
It’s a tough time to be a young psychologist. This thought keeps occurring to me as we search for a new psychology professor at my school, Stevens Institute of Technology. When I meet candidates, I have to ask about their field’s replication—and credibility—crisis.
I feel as though I’m pressing them on some sordid personal matter, like whether alcoholism runs in their families, but the topic is unavoidable. Last summer, a group called the “Open Science Collaboration” reported inScience that it had replicated fewer than half of 100 studies published in major psychology journals.
The New York Times declared in a front-page story that the report “confirmed the worst fears of scientists who have long worried that [psychology] needed a strong correction. The vetted studies were considered part of the core knowledge by which scientists understand the dynamics of personality, relationships, learning and memory. Therapists and educators rely on such findings to help guide decisions, and the fact that so many of the studies were called into question could sow doubt in the scientific underpinnings of their work.”
The crisis keeps generating headlines. On Friday, a group of four prominent psychologists led by Daniel Gilbert of Harvard claimed in Science that last year’s Open Collaboration study was statistically flawed and did not prove its claim that “the reproducibility of psychological science is low.” “Indeed,” Gilbert and his co-authors state, “the data are consistent with the opposite conclusion, namely, that the reproducibility of psychological science is quite high.”
In a rebuttal, 44 authors involved in the Open Science Collaboration countered that the “very optimistic assessment” of Gilbert’s group “is limited by statistical misconceptions and by causal inferences from selectively interpreted, correlational data.”
The exchange, Benedict Carey notes in the Times, “is likely to feed an already lively debate about how best to conduct and evaluate so-called replication projects of studies.” That’s too cheery an assessment. The exchange reveals that psychologists cannot even agree on basic methods for arriving at “truth,” whatever that is.
It gets worse. Over at Slate, Daniel Engber reports that the influential theory of “ego depletion”—which holds that willpower is a finite resource that diminishes with use--might have been “debunked.”
Roy Baumeister and three other psychologists presented experimental evidence for the theory in a 1998 paper that has been cited more than 3,000 times. The theory has supposedly been corroborated by hundreds of other studies, and it underpins the 2011 bestseller Willpower: Rediscovering the Greatest Human Strength, by Baumeister and journalist John Tierney.
But a study of ego depletion involving “more than 2,000 subjects tested at two-dozen different labs on several continents,” Engber reports, found “exactly nothing. A zero-effect for ego depletion: No sign that the human will works as it’s been described, or that these hundreds of studies amount to very much at all.” The new study is scheduled for publication next month in Perspectives on Psychological Science.
Spelling out the disturbing implications, Engber notes that the ego-depletion effect “has been recreated in hundreds of different ways, and the underlying concept has been verified via meta-analysis. It’s not some crazy new idea, wobbling on a pile of flimsy data; it’s a sturdy edifice of knowledge, built over many years from solid bricks. And yet, it now appears that ego depletion could be completely bogus, that its foundation might be made of rotted-out materials. That means an entire field of study—and significant portions of certain scientists’ careers—could be resting on a false premise. If something this well-established could fall apart, then what’s next?”
Good question, over which all young psychologists are no doubt agonizing. To cheer themselves, they might consider the following four points:
First, there’s nothing new about psychology’s credibility crisis. More than a century ago, William James worried that the field he helped create might never transcend its “confused and imperfect state.”
Second, all scientific fields struggle with replication issues. Behavioral genetics and psychiatry are arguably much less credible than psychology, and string and multiverse theorists don’t even have empirical results to replicate!
Third, psychologists are still doing important, empirically sound work. Two who recently spoke at my school are Sheldon Solomon, co-creator of terror-management theory, which predicts how fear of death affects us; and Philip Tetlock, leader of a study on “superforecasters,” ordinary people who do a better job than many so-called experts at predicting social phenomena.
Fourth, psychology is arguably healthier than many other fields precisely because psychologists are energetically exposing its weaknesses and seeking ways to overcome them.
I look forward to discussing these issues with the young psychologists visiting my school.
Further Reading:
Meta-Post: Horgan Posts on Brain and Mind Science.
Meta-Post: Horgan Posts on Antidepressants, Brain Implants, Psychedelics, Meditation and Other Therapies for Mental Illness.
A Dig Through Old Files Reminds Me Why I’m So Critical of Science.
Thank You Mr Horgan and Scientific American.
" the report “confirmed the worst fears of scientists who have long worried that [psychology] needed a strong correction."
Semi private rooms at Club Fed would be the Perfect place to start correcting it.
Psychiatrists And Psychologists Not Reliable Expert Witnesses
Us Title 18 Part 1 Chapter 19 Sec. 371 Conspiracy To Defraud The United States: 5 years
Us Title 18 Part 1 Chapter 47 Sec. 1031 Major Fraud Against The United States: 10 years
Us Title 18 Part 1 Chapter 47 Sec. 1035 False Statements: 5 years
Us Title 18 Part 1 Chapter 63 Sec. 1341 Mail Fraud: Frauds and Swindles: 20 years
Us Title 18 Part 1 Chapter 63 Sec. 1347 Health Care Fraud: 10 years
Us Title 18 Part 1 Chapter 63 Sec. 1349 Attempt and Conspiracy
Us Title 18 Part 1 Chapter 95 Sec. 1957 Engaging In Monetary Transactions In Property Derived From Specified Unlawful Activity: over $10,000, 10 years
Just the Tip of the Iceberg.
- The Law (13)
No comments:
Post a Comment