How to Evaluate Medical Research

Every day medical research is reported on by the popular press. By its very nature, medical research can be confusing to read and to understand. Most researchers write their studies in the language of science and, lacking a good working knowledge of that particular medical field, you may find the concepts and terms to be very confusing at best. Additionally, you must keep in mind that having a study reported on in the popular press doesn’t make if a good or “valid” study. Digging down into the study and asking some very critical questions about it can help you evaluate the study and better understand it.

Who paid for the study?
A lot can be garnered just by knowing who paid to have a study done. For example, a 1998 publication of a research paper in the medical journal The Lancet by Andrew Wakefield lent support to a subsequently discredited theory that colitis and autism spectrum disorders “could” be caused by the combined Measles, Mumps and Rubella (MMR) vaccine. The popular press ran with the story, immunization rates for MMR dropped, the incidence of the three diseases that the combined vaccine prevents increased and at least one death from measles occurred. Yet no one looked at who paid Wakefield to do his research study until February 2004 when a reporter, Brian Deer, wrote in The Sunday Times of London that, prior to submitting his research paper to the Lancet Wakefield received a payment of £55,000 from Legal Aid Board solicitors who were seeking evidence to use against vaccine manufacturers. Deer further revealed that several of the parents that Wakefield quoted in his study as saying that MMR had damaged their children were part of the group of litigants in the case. The Lancet eventually retracted the paper and Wakefield eventually lost his medical license.

Did the researcher study animals or people?
Not all animals study directly correlates to what will happen in humans. Differences in physiology, anatomy and other factors mean that the best that you can do from an animal study is suggest that a human trial would or would not be successful. However, until a human study is done you cannot be sure that the same results will be achieved.

Who are the people that were observed in the study?
Sometimes a very specific group or “class” of individuals is being studied and you need to know that. When a researcher narrows the focus to a specific class of individuals it limits how broadly the conclusions of the study can be applied to the general population. While the research may be important, it may be very limited in its scope.

Was the study a randomized controlled clinical trial.
By using a randomized controlled trial approach the researcher ends up with study subjects who are randomly allocated to receive either one or the other of the alternative treatments within the study. All study subjects are followed in exactly the same way with the exceptions of the variable being studied. This type of study minimizes bias in the study because the assignment of the variable being study is done essentially like the flip of a coin.

Where was the research done?
This may seem like a matter of little importance but there is a growing body of evidence that says that the physical setting can greatly influence study outcomes.

Were there side effects?
It is important to note how the researchers monitored for side effects and what those side effects were. All potential side effects should be noted during a study, but then it is important for the researchers to dig down into the data and determine whether a potential side effect becomes a recognized side effect or not. Just because a researcher observes a potential side effect after a medical intervention does not necessarily mean that the medical intervention caused that side effect. A + B does not always equal C, in other words, a correlation between the two phenomena does not equal cause-and-effect..

Who is reporting the results and through what publication(s)?
Both your researcher and the venue through which they publish should be trusted. Is the researcher known for doing quality research? Has his or her research ever been discredited or identified as a “weak” study? Is the publisher known for publishing peer reviewed studies? A peer reviewed study is one where “peers” of the researcher who are qualified members of the profession within the relevant field and of similar competence to the author review the work so that standards of quality are maintained and credibility of the research is provided.