Brain-imaging studies hampered by small data sets, study finds
For two decades, researchers have used brain-imaging technology to try to identify how a person’s brain structure and function are linked to a range of mental-health illnesses, from anxiety and depression to suicide. up to the trend.
But a new paper published Wednesday in Nature answers the question of whether much of this research is actually yielding valid conclusions. Many such studies, the paper’s authors found, involve fewer than two dozen participants, which is just shy of the number needed to produce reliable results.
“You need thousands of individuals,” said Scott Marek, a psychiatric researcher at Washington University School of Medicine in St. Louis and author of the paper. He described the finding as a “gut punch” for specific studies that use imaging to try to better understand mental health.
Studies that use magnetic-resonance imaging technology usually arm their findings with a cautionary statement noting the small sample size. But enlisting participants can be time-consuming and costly, ranging from $600 to $2,000 an hour, said Dr., a neurologist at Washington University School of Medicine and another author on the paper. Nico Dosenbach said. The average number of subjects who use brain imaging in mental-health-related studies is about 23, he said.
But the Nature paper demonstrates that data taken from just two dozen subjects is generally insufficient to be reliable and may in fact draw “largely inflated” conclusions,” Dr. Dosenbach said.
For their analysis, the researchers examined three of the largest studies using brain-imaging technology to reach conclusions about brain structure and mental health. All three studies are underway: the Human Connectome Project, with 1,200 participants; The Adolescent Brain Cognitive Development, or ABCD, study, with 12,000 participants; and the UK Biobank study, with 35,700 participants.
The authors of the Nature paper looked at subsets of data within those three studies to determine whether the smaller slices were misleading or “reproducible,” meaning the findings could be considered scientifically valid.
For example, the ABCD study looked at, among other things, whether the thickness of the brain’s gray matter could be related to mental health and problem-solving ability. The authors of the Nature paper looked at smaller subsets within the larger study and found that the subset produced results that were unreliable compared to results obtained by the full data set.
On the other hand, the authors found, when results were generated from sample sizes that included several thousand subjects, the findings were similar to those of the full data set.
The authors ran millions of calculations using different sample sizes and traced hundreds of brain regions across various major studies. Over and over again, researchers found that subsets of data from fewer than several thousand people did not produce results consistent with the full data set.
Dr Marek said the paper’s findings “absolutely” apply beyond mental health. Other fields, such as genomics and cancer research, have their own calculations with the limitation of small sample sizes and have tried to correct course, he said.
“My hunch is it’s about population science in one of those areas,” he said.
#Brainimaging #studies #hampered #small #data #sets #study #finds