Opinion

How to read - and understand - a scientific study

Washington Post|Published

Keeping the brain active is highly recommended to stave off age-related dementia, and a recent study suggests your coffee habit might also affect your brain health. However, a researcher cautions people to engage with the nuances of studies and advises how to decode scientific research.

Image: Cottonbro Studio/Pexels

Richard M. Ransohoff

 

Many people like me - scientists and coffee enthusiasts - were intrigued by a long-term study that found that those who drank multiple cups of coffee a day were less likely to develop dementia.

I’m a neuroscientist who drinks a lot of strong coffee, so that got my attention. And I would love for it to be true. But this was a correlational study, meaning it didn’t (and couldn’t) look at whether caffeine intake actually decreases dementia risk. It reinforced for me the importance of reading the actual study, rather than stories about the study, because scientific researchers are usually pretty candid about the limitations of their work.

Encountering this important caveat, which is not unusual, also showed me that laypeople can and should be reading studies like this one. And what a time to be reading them! I’m on the board of directors at the American Brain Foundation and can attest that we are in a golden age of scientific progress. I can also offer some advice to people without the training and experience of a neuroscientist on how to make sense of work that is often dense and tough to interpret. It can be done, and it’s well worth the trouble - partly so people can vet information about compelling health-related topics and partly because there’s such joy in observing progress in action.

I am sensitive to how difficult scientific studies are to read. I was a literature major long ago, before I became a doctor and researcher. Both my parents were writers, and they schooled me first to understand and then to communicate.

Johns Hopkins University’s Bloomberg School of Public Health recently put out a guide for non-scientists who want to read research publications. I’m glad the school did that, if only because laypeople who say they are “doing their own research” often are actually relying on other laypeople trying to interpret the research. If you truly want to make your own judgments, the best way is to read the research yourself. That requires investing some time.

My recommendations for reading and understanding the studies differ somewhat from the Hopkins recipe. Here’s my advice:

Look at when and where the study was published. Check for signs that it appeared in a “predatory” journal, which is a journal that charges academics to publish and fails to provide peer review and other hallmarks of scientific legitimacy. Sometimes, unfortunately, you have to do research on the research.

Take some time to really read. Scientific studies are not things you can skim or scroll through on your phone. In time, you will be able to get the gist of a study by reading the abstract - but not at first.

Read the introduction. This tells you what knowledge gaps the researchers were trying to fill.

Skip the methods and statistical analyses. These are highly technical and less useful for laypeople.

Go to the discussion section. That’s where the authors will describe what they found and what they think it means.

Read the limitations section. It’s important, because scientists and their academic editors care deeply about their work not being misunderstood or exaggerated.

The limitations section is where researchers examining a new drug, for example, will explain whether it was tested on mice or on cell cultures. These are the beginnings of drug treatment, not the end. Many treatments work on mice but not on people. The research you’re reading about might be just the first step in a journey of a thousand miles (or a dozen years), so keep that in mind. Most papers in biomedical science suggest that the work will lead to new treatment strategies, but that’s to attract grant funding and attention.

It’s worth noting that studies about experiments that fail are important, too. They tell us why a path should be abandoned, or how it might work if modified. Science is self-correcting. Finding mistakes and describing them is a virtue, not a vice.

The limitations section is also where researchers will say whether a study is correlational. Scientists love to say that correlation is not causation, which simply means that two things can often go together without one causing the other. Muscle cars have higher death rates than other cars. Does that mean they’re unsafe? Or does it mean that the people who buy them are more likely to drive dangerously? Ice cream sales and shark attacks both rise in the summer, but not because one causes the other.

With coffee and dementia, it’s possible that drinking a lot of caffeine could stave off dementia. It’s also possible that the kind of people who drink a lot of coffee have active minds and want to stay engaged in their work, which could stave off dementia. Which might it be? Could it be both? Or neither? There could be totally unrelated factors that underlie increased coffee intake and lower risk of dementia.

Those are questions for further studies, which I hope to read someday. I hope you do as well.

 

*Richard M. Ransohoff is venture partner at the health care investment firm Third Rock Ventures and serves on the board of directors and research advisory committee of the American Brain Foundation.