Composition and size of the sample: Was this the right group to ask the question of? Are there too few subjects for the research question? Did the researcher tell you which subjects were excluded and why?
Distribution of answers: Were the answers too widely varied to consider the responses representative of the group? Were there multiple responses and perspectives? Were the answers inconclusive?
Research question and result: Was the overarching question guided and formulated by clinical experience? Does the research question or hypothesis match the result? Is the correlation real or an accident? Did the presenter confuse the concepts of correlation and causation?
Qualitative data and interpretation: Did the researcher provide additional qualitative information from comments (especially with a small sample size)? Did the researcher show data-blindness by ignoring clinical signs and understanding?
Transparency of the narrative: Did the author explain the experimental methodology and design as well as indicate what was not chosen? What were the inherent challenges and were they explained? Do the conclusion and results make sense from what was learned or was the researcher leading you? Did the researcher indicate what was inconclusive as well?
Read the May article, “Five Ways to Lie With Statistics—Or at Least Tell a Better Story,”on our website.
