Authors Retract, Replace Highly Cited JAMA Psych Paper For "Pervasive Errors"

retraction watch

Authors have retracted a highly cited JAMA Psychiatry study about depression after failing to account for some patient recoveries, among other mistakes.

It’s a somewhat unusual notice — it explains that the paper has been retracted and replaced with a new, corrected version.

The study, which included 452 adults with major depressive disorder, concluded that cognitive therapy plus medication works better to treat depression than pills alone. But after it was published, a reader pointed out that some of the numbers in a table were incorrect. The authors reviewed the data and redid their analysis, and discovered “a number of pervasive errors.”

The notice (termed “notice of retraction and replacement”) explains the consequences of those errors:

These errors, once corrected, have not changed the final conclusion of this study—that cognitive therapy combined with antidepressant medication treatment enhanced rates of recovery relative to treatment with medication alone. However, the corrections do result in changes to numerous data in the Abstract, text, Table, and Figures. Three of the findings that were previously reported as statistically significantly different are no longer significantly different: the interaction between recovery rate and severity, the number of patients who dropped out of each group, and the number of serious adverse events in each group. Because of these errors, we have reconducted our analyses with the correct data, have corrected all findings and interpretations, and have requested that JAMA Psychiatry retract and replace the original article.

They explain that the biggest issue was that they miscounted the patients who had recovered:

The major problem with our original reported data was that the automated algorithm that we used to track patient progress sometimes failed to recognize remissions or recoveries that occurred or it recognized only later instances. This resulted in missing recoveries in 13 patients. There were 5 additional patients who recovered in the combined treatment group (the correct number is 170 not 165), and 8 additional patients who recovered in the medications only group (the correct number is 148 not 140). In addition, 282 patients recovered earlier and 16 patients recovered later than was initially recognized in our analyses. Differences between the treatment conditions were largely unaffected but the median time to recovery was reduced. In the process of hand-checking our data, we also found 2 patients who were credited with remissions that they did not achieve (1 in each group). Correcting these errors affected the cell sizes reported in Figures 2 and 3 and the exact values of the tests reported and also resulted in a change in the interaction between severity and treatment condition, which is no longer a statistically significant interaction.

The correct version of the article now appears in JAMA Psychiatry, with the retracted version attached as a supplement. “Effect of cognitive therapy with antidepressant medications vs antidepressants alone on the rate of recovery in major depressive disorder” has been cited 25 times since it was published in 2014, according to Thomson Reuters Web of Science (which has labeled it a “highly cited paper,” based on the expected rate of citations in that particular field.)

The notice describes more errors in the paper, including a patient who was classified with the wrong diagnosis at intake, and a miscounting of patients who met the criteria for chronic depression and recurrent depression, were unemployed, and had previously used antidepressant medication.

The retraction notice explains how the errors came to light:

We apologize to the journal and its readers for the errors that were present in our original publication, and we appreciate the action taken by the reader who alerted us to the undercounts in the Table just described that prompted our complete review of the data, analyses, and conclusions. The relevant data and findings have now been corrected, and the Abstract, text, Table, and Figures 1, 2, and 3 in our article have been corrected and replaced online with a new supplement that includes a version of the original retracted article showing the original errors and a version of the replacement article showing what was corrected.

It’s not often we see such a transparent explanation of what went wrong with a paper — an example of “doing the right thing.”

The findings may have had an impact on clinical practice — a few months after the study appeared, Evidence-Based Mental Health published a commentary about it by Sharon C Sung at Duke-NUS, which discusses how it may have affected her treatment choices for patients:

These results suggest that high-quality medication management may be sufficient to obtain recovery for patients with less severe and/or chronic MDD. The more time- and cost-intensive ADM plus CT strategy may be best suited for patients with severe, non-chronic MDD. Pending replication of these findings, I would be more likely to refer patients with severe depression of less than 2 years duration to receive combined treatment as a first-line intervention. For those with less severe depression of longer duration, I would wait to see if they respond to ADM prior to augmenting with psychotherapy.

We’ve found three other corrections for last author Robert Gallop, who works at West Chester University. A correction note on “Preventing Depression among Early Adolescents in the Primary Care Setting,” published in Journal of Abnormal Child Psychology, explains “a paragraph describing intervention effects on explanatory style for negative events was accidentally left out.” The paper — on which Gallop is the last author — has been cited 89 times. 

Gallop is the third of six authors on “Combined Medication and CBT for Generalized Anxiety Disorder With African American Participants: Reliability and Validity of Assessments and Preliminary Outcomes” in Behavior Therapy; according to the correction, there was an error in the statistical analysis, and in a few of the statistics provided on the sample. That paper has been cited four times. Gallop is the fourth of five authors on “A Tutorial on Count Regression and Zero-Altered Count Models for Longitudinal Substance Use Data,” a highly cited paper in Psychology of Addictive Behaviors with a relatively minor correction: an error in a URL in the paper. The paper has been cited 47 times.

We’ve reached out to first author Steven Hollon, who works at Vanderbilt University, to Gallop, and to the journal for more information on the retraction. We’ll update this post with anything else we learn.

Update, April 22 11:50 am:

Annette Flanagin, the Executive Managing Editor for The JAMA Network, told us more about why the article was both retracted and corrected:

As we stated when we announced our policy on use of retraction and replacement in an Editorial 2015, “Retractions are typically reserved for articles that have resulted from scientific misconduct, such as fabrication, falsification, or plagiarism, or from pervasive error for which the results cannot be substantiated.” (See reference below.) In this case, inadvertent errors resulted in changes to some of the findings, although the general conclusions of the study are unchanged. As we also noted in that Editorial, errors do occur, and if the errors are pervasive and result in a major change in the direction or significance of the findings, interpretations, and conclusions, and the science is considered reliable, we will consider retraction and replacement as an effective approach to ensuring transparency and an accurate scientific record.

Here’s the citation and link to the Editorial noted above:Heckers S, Bauchner H, Flanagin A. Retracting, Replacing, and Correcting the Literature for Pervasive Error in Which the Results Change but the Underlying Science Is Still Reliable. JAMA Psychiatry. 2015;72(12):1170-1171. doi:10.1001/jamapsychiatry.2015.2278.http://archpsyc.jamanetwork.com/article.aspx?articleid=2466828

She reminded us that JAMA has invoked their retraction and replacement policy before — for a small study on using gamma rays to treat OCD that we reported on last yearWe spoke to Hollon by phone, who provided more thoughts on the retraction. He told us that the researchers had a data management service construct a system to track patients, which included the automated algorithm mentioned in the retraction notice. Hollon called it,

a beautiful machine that wasn’t handled with sufficient care. All of this is on me…I should have checked every algorithm.

He added:

It was kind of like turning a Maserati over to a teenager, and I drove it over the edge.

He said that he plans to be more careful in the future:

I’m personally embarrassed that this happen, but on the other hand, I’m enthusiastic and energized…I learned something in the process.

Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our new daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.

Thank You retraction watch

HT to MIA