April 16, 2012
A Sharp Rise in Retractions Prompts Calls for Reform
By CARL ZIMMER
In the fall of 2010, Dr. Ferric C. Fang made an unsettling discovery. Dr.
Fang, who is editor in chief of the journal Infection and Immunity, found
that one of his authors had doctored several papers.
It was a new experience for him. “Prior to that time,” he said in an
interview, “Infection and Immunity had only retracted nine articles over a
40-year period.”
The journal wound up retracting six of the papers from the author, Naoki
Mori of the University of the Ryukyus in Japan. And it soon became clear
that Infection and Immunity was hardly the only victim of Dr. Mori’s
misconduct. Since then, other scientific journals have retracted two dozen
of his papers, according to the watchdog blog Retraction Watch.
“Nobody had noticed the whole thing was rotten,” said Dr. Fang, who is a
professor at the University of Washington School of Medicine.
Dr. Fang became curious how far the rot extended. To find out, he teamed up
with a fellow editor at the journal, Dr. Arturo Casadevall of the Albert
Einstein College of Medicine in New York. And before long they reached a
troubling conclusion: not only that retractions were rising at an alarming
rate, but that retractions were just a manifestation of a much more profound
problem — “a symptom of a dysfunctional scientific climate,” as Dr. Fang
put it.
Dr. Casadevall, now editor in chief of the journal mBio, said he feared that
science had turned into a winner-take-all game with perverse incentives
that lead scientists to cut corners and, in some cases, commit acts of
misconduct.
“This is a tremendous threat,” he said.
Last month, in a pair of editorials in Infection and Immunity, the two
editors issued a plea for fundamental reforms. They also presented their
concerns at the March 27 meeting of the National Academies of Sciences
committee on science, technology and the law.
Members of the committee agreed with their assessment. “I think this is
really coming to a head,” said Dr. Roberta B. Ness, dean of the University
of Texas School of Public Health. And Dr. David Korn of Harvard Medical
School agreed that “there are problems all through the system.”
No one claims that science was ever free of misconduct or bad research.
Indeed, the scientific method itself is intended to overcome mistakes and
misdeeds. When scientists make a new discovery, others review the research
skeptically before it is published. And once it is, the scientific community
can try to replicate the results to see if they hold up.
But critics like Dr. Fang and Dr. Casadevall argue that science has changed
in some worrying ways in recent decades — especially biomedical research,
which consumes a larger and larger share of government science spending.
In October 2011, for example, the journal Nature reported that published
retractions had increased tenfold over the past decade, while the number of
published papers had increased by just 44 percent. In 2010 The Journal of
Medical Ethics published a study finding the new raft of recent retractions
was a mix of misconduct and honest scientific mistakes.
Several factors are at play here, scientists say. One may be that because
journals are now online, bad papers are simply reaching a wider audience,
making it more likely that errors will be spotted. “You can sit at your
laptop and pull a lot of different papers together,” Dr. Fang said.
But other forces are more pernicious. To survive professionally, scientists
feel the need to publish as many papers as possible, and to get them into
high-profile journals. And sometimes they cut corners or even commit
misconduct to get there.
To measure this claim, Dr. Fang and Dr. Casadevall looked at the rate of
retractions in 17 journals from 2001 to 2010 and compared it with the
journals’ “impact factor,” a score based on how often their papers are
cited by scientists. The higher a journal’s impact factor, the two editors
found, the higher its retraction rate.
The highest “retraction index” in the study went to one of the world’s
leading medical journals, The New England Journal of Medicine. In a
statement for this article, it questioned the study’s methodology, noting
that it considered only papers with abstracts, which are included in a small
fraction of studies published in each issue. “Because our denominator was
low, the index was high,” the statement said.
Monica M. Bradford, executive editor of the journal Science, suggested that
the extra attention high-impact journals get might be part of the reason for
their higher rate of retraction. “Papers making the most dramatic advances
will be subject to the most scrutiny,” she said.
Dr. Fang says that may well be true, but adds that it cuts both ways — that
the scramble to publish in high-impact journals may be leading to more and
more errors. Each year, every laboratory produces a new crop of Ph.D.’s,
who must compete for a small number of jobs, and the competition is getting
fiercer. In 1973, more than half of biologists had a tenure-track job within
six years of getting a Ph.D. By 2006 the figure was down to 15 percent.
Yet labs continue to have an incentive to take on lots of graduate students
to produce more research. “I refer to it as a pyramid scheme,” said Paula
Stephan, a Georgia State University economist and author of “How Economics
Shapes Science,” published in January by Harvard University Press.
In such an environment, a high-profile paper can mean the difference between
a career in science or leaving the field. “It’s becoming the price of
admission,” Dr. Fang said.
The scramble isn’t over once young scientists get a job. “Everyone feels
nervous even when they’re successful,” he continued. “They ask, ‘Will
this be the beginning of the decline?’ ”
University laboratories count on a steady stream of grants from the
government and other sources. The National Institutes of Health accepts a
much lower percentage of grant applications today than in earlier decades.
At the same time, many universities expect scientists to draw an increasing
part of their salaries from grants, and these pressures have influenced how
scientists are promoted.
“What people do is they count papers, and they look at the prestige of the
journal in which the research is published, and they see how many grant
dollars scientists have, and if they don’t have funding, they don’t get
promoted,” Dr. Fang said. “It’s not about the quality of the research.”
Dr. Ness likens scientists today to small-business owners, rather than
people trying to satisfy their curiosity about how the world works. “You’
re marketing and selling to other scientists,” she said. “To the degree
you can market and sell your products better, you’re creating the revenue
stream to fund your enterprise.”
Universities want to attract successful scientists, and so they have erected
a glut of science buildings, Dr. Stephan said. Some universities have gone
into debt, betting that the flow of grant money will eventually pay off the
loans. “It’s really going to bite them,” she said.
With all this pressure on scientists, they may lack the extra time to check
their own research — to figure out why some of their data doesn’t fit
their hypothesis, for example. Instead, they have to be concerned about
publishing papers before someone else publishes the same results.
“You can’t afford to fail, to have your hypothesis disproven,” Dr. Fang
said. “It’s a small minority of scientists who engage in frank misconduct.
It’s a much more insidious thing that you feel compelled to put the best
face on everything.”
Adding to the pressure, thousands of new Ph.D. scientists are coming out of
countries like China and India. Writing in the April 5 issue of Nature, Dr.
Stephan points out that a number of countries — including China, South
Korea and Turkey — now offer cash rewards to scientists who get papers into
high-profile journals. She has found these incentives set off a flood of
extra papers submitted to those journals, with few actually being published
in them. “It clearly burdens the system,” she said.
To change the system, Dr. Fang and Dr. Casadevall say, start by giving
graduate students a better understanding of science’s ground rules — what
Dr. Casadevall calls “the science of how you know what you know.”
They would also move away from the winner-take-all system, in which grants
are concentrated among a small fraction of scientists. One way to do that
may be to put a cap on the grants any one lab can receive.
Such a shift would require scientists to surrender some of their most
cherished practices — the priority rule, for example, which gives all the
credit for a scientific discovery to whoever publishes results first. (Three
centuries ago, Isaac Newton and Gottfried Leibniz were bickering about who
invented calculus.) Dr. Casadevall thinks it leads to rival research teams’
obsessing over secrecy, and rushing out their papers to beat their
competitors. “And that can’t be good,” he said.
To ease such cutthroat competition, the two editors would also change the
rules for scientific prizes and would have universities take collaboration
into account when they decide on promotions.
Ms. Bradford, of Science magazine, agreed. “I would agree that a scientist
’s career advancement should not depend solely on the publications listed
on his or her C.V.,” she said, “and that there is much room for
improvement in how scientific talent in all its diversity can be nurtured.”
Even scientists who are sympathetic to the idea of fundamental change are
skeptical that it will happen any time soon. “I don’t think they have much
chance of changing what they’re talking about,” said Dr. Korn, of Harvard.
But Dr. Fang worries that the situation could be become much more dire if
nothing happens soon. “When our generation goes away, where is the new
generation going to be?” he asked. “All the scientists I know are so
anxious about their funding that they don’t make inspiring role models. I
heard it from my own kids, who went into art and music respectively. They
said, ‘You know, we see you, and you don’t look very happy.’ ”
