I. Critical Question
- - Does this article address an important question in the field?
- - If it is a clinical trial, does the study examine an area where it is non-consensus on the appropriate treatment?
- - If it is a clinical trial, is it a "pivotal" trial, i.e. fulfill FDA requirements for safety and efficacy?
- - If it is a case report or case series, is there new information provided to the standard of care or to an area of unmet need?
- - If it is a basic science study, does it provide key information that can be applied to clinical medicine?
II. Study Design
- - Are there enough data points (eg. patients or measurements) to make a meaningful conclusion? For example, if a blood pressure study only measures blood pressure three times after the study drug is administered, can you really say that it is useful clinically?
- - Does the study population approximate the real world? For example, if a study of stroke prevention excludes anyone with cardiac disease, can you conclude that the treatment works for the majority of persons with stroke.
- - Are the analytical techniques appropriate? This is probably the hardest thing to evaluate, but you should look at the disease itself and determine if the results are truly convincing or not for a similar scenario.
III. Strength of the data
- - What is the likelihood that inaccuracies in measurements or normal variability in the study population account for the results? For example, if you take a series of math tests and score 97% on test 1, 96% on test 2, 94% on test 3, 93% on test 4, and 92% on test 5, does this mean that you are losing knowledge? What if the mean for your classmates stayed the same at 85% with a standard deviation of 5%?
- - When looking at confidence intervals, check to see if the 95% level crosses unity (normally "1" in an odds ratio). If the odds ratio consistently stays on the same side of unity, it means that there is at least a 95% chance that the direction of the risk (up or down) is in the same direction.
IV. Generalizability
- - This is directly impacted by the study design. Sometimes, the study population is so small, or the results are so specific to the restricted population, that it is impossible to say that the treatment will be effective for the typical patient. However, preliminary studies are often done to help design a larger study.