Common Research Problems
Growing Pressure for Positive Results
Science is a competitive field. Scientists have intense pressure to produce meaningful results. As a result, fewer and fewer papers are being published that show “negative results”—i.e. that their hypothesis was false. In 1990, “negative results” accounted for 30% of published papers—that number has fallen to a mere 14%.
Another problem is that scientists are under pressure to publish new, groundbreaking research, rather than performing studies to replicate results from previous research. Journals are exclusive and want to publish striking results that present a “major advance.” Replicating studies, however, is extremely important. It’s a key part of confirming findings and eliminating scientific fraud.
Sloppy Lab Work
Labs can often be messy and chaotic. In far too many cases, samples and chemicals are mislabeled and even forgotten. The Wall Street Journal took a hard look at this issue after a cancer researcher had his work on head-and-neck cancer retracted from the journal Oral Oncology due to the fact that the cells he was studying were actually cervical cancer cells. The WSJ highlights the extent of the problem: “Cancer experts seeking to solve the problem have found that a fifth to a third or more of cancer cell lines tested were mistakenly identified—with researchers unwittingly studying the wrong cancers, slowing progress toward new treatments and wasting precious time and money.”
The problem is incredibly widespread: “Cell repositories in the U.S., U.K., Germany and Japan have estimated that 18% to 36% of cancer cell lines are incorrectly identified.”
While the National Institutes of Health and the scientific community are slowly trying to weed out these problems by increasing scrutiny on papers submitted using cell lines and setting up a central repository of cell lines, cell contamination remains a major problem in scientific research.
Sloppy lab conditions can also lead to another major problem: mycoplasma infestations. Mycoplasma is a bacteria that can spread rapidly throughout lab cultures, compromising scientists’ potential findings. The problem is also widespread. A recent article in Nature covered the problem and interviewed researchers who “found that more than one-tenth of gene-expression studies, many published in leading journals, show evidence of Mycoplasma contamination.”
Alarmingly, the pressure to produce prestigious research has led a number of scientists to simply fake results or plagiarize from other researchers. In the last year, articles have been retracted from prestigious journals in which authors:
- “Knowingly and intentionally falsifying” results in a study of cancerous tumors,
- Duplication or “self-plagiarism” in a study of liver cancer,
- “Large sections of text duplicated from previously published articles” in a study of gastrointestinal cancer.
Unfortunately, this is just a small sample of the many instances of fraud every year. A recent study found that fraud is the reason for 43% of all journal retractions.
Scientific fraud can have huge implications. Remember the study that linked vaccinations and autism? Even though it was retracted after researchers said it was based on doctored information about children’s medical records, the myth of the vaccine/autism link is pervasive and continues to be repeated.
Reliance on Self-Reported Data
A number of frequently cited studies, particularly studies of nutrition, rely on the information that study participants self-report. This makes it difficult to fully trust a study’s findings–self-reported data is notoriously unreliable.
Just how reliable is self-reported data? Consider that consumers consistently give the food on Southwest Airlines high marks…despite the fact that the airline doesn’t serve meals.
During a recent session hosted by the American Society for Nutrition, Dr. David Allison took a highly critical look at self-reported data, highlighting a recent paper “that looked at energy intake of respondents in NHANES from 1971-2012, finding that 67.3% of women and 58.7% of men were not physiologically plausible – i.e. the number of calories is ‘incompatible with life.'”
That certainly doesn’t stop researchers from using this method or stop the media from reporting on these studies. Here are just a few examples of major media outlets reporting on studies that rely on self-reported data without explaining the limitations of such research:
- BBC: “Sleep quality ‘improves with age’“
- Science World Report: “Eating Baked or Broiled Fish Helps Improve Brain Health“
- Washington Post: “There’s a gender gap in bullying — watch it widen as kids grow up“