How to Critically Appraise a Research Paper

Being asked to critically appraise a research paper can feel intimidating, especially the first time. You are not being asked to decide whether the paper is “good” or “bad” in some vague sense. You are being asked to judge, step by step, whether the study was done well enough, whether its results can be trusted, and whether those results matter for the question you care about.

Table of Contents

What Does It Mean to Critically Appraise a Research Paper?

Critical appraisal is the structured assessment of a research paper’s validity, results, and relevance. It is not the same as summarising a paper, and it is not the same as criticising it for the sake of it. A good appraisal is fair: it credits what the authors did well and names precisely where the study is weak, and why that weakness matters.

Most appraisal frameworks — including the widely used CASP (Critical Appraisal Skills Programme) checklists — organise the task around three questions:

Are the results of the study valid? Was the study designed and carried out in a way that minimises bias?
What are the results? How large was the effect, and how precise is the estimate?
Will the results help locally? Do the findings apply to your population, setting, or question?

Keep those three questions in front of you throughout the appraisal. They stop you from getting lost in the details.

Step 1: Read the Paper Twice — Once for the Story, Once for the Scrutiny

On your first read, do not take notes. Just follow the story: what did the researchers want to find out, what did they do, what did they find, and what do they think it means? This gives you the shape of the paper.

On your second read, slow down and interrogate each section. The abstract is a good place to start, but never appraise a paper from the abstract alone. Abstracts can overstate findings and omit limitations. Always work from the full text.

Before you begin, write down the research question in PICO form (Population, Intervention, Comparison, Outcome) if it is a quantitative study, or as a clear statement of the phenomenon of interest if it is qualitative. This one-sentence question is your anchor: every judgement you make should come back to whether the study answers it.

Step 2: Check Whether the Question Was Clear and Worth Asking

A study can be perfectly executed and still be unimportant. Ask:

Is the research question stated clearly, and does it match the title and aims?
Does the introduction explain why this question matters — a gap in the literature, a clinical or practical problem, a contradiction in earlier findings?
Is the study design appropriate for the question? A randomised trial suits a treatment question; a qualitative interview study suits a question about experience. A mismatch here is a serious flaw, not a minor detail.

Step 3: Judge the Validity — How Was the Study Actually Done?

This is the heart of critical appraisal, and it is where most of the marks sit in a coursework appraisal. Work through the methods section and ask:

Who was recruited, and how? Was the sample defined clearly? Was recruitment consecutive, random, or convenience-based — and does that invite selection bias?
Was the sample big enough? Look for a sample-size or power calculation. An underpowered study can miss a real effect, while a study with no justification for its sample size should make you cautious.
Was there a comparison group, and was it fair? In trials, was allocation random, and was it concealed from the people enrolling participants? Were the groups similar at baseline?
Were participants and researchers blinded where possible? Blinding is not always feasible — nobody can be blinded to whether they received talking therapy — but the authors should explain what was done and why.
How were outcomes measured? Were the measures valid and reliable for this population? Were they measured the same way in every group?
How much data was lost? High dropout, especially if it differs between groups, can distort results. Check whether the authors analysed everyone they randomised (intention-to-treat analysis) or only those who finished.
Was the analysis appropriate? Do the statistical tests match the type of data? Were confounders adjusted for? Was the analysis plan decided before the data were seen, or does it look as though the authors fished for significant results?

For qualitative studies, the validity questions change. Ask whether the approach (interviews, focus groups, observation) fits the question, whether sampling was purposeful and explained, whether data saturation was reached, whether analysis was systematic and transparent, and whether the researchers reflected on their own influence on the findings (reflexivity). A good qualitative paper lets you follow its reasoning from raw data to themes.

Step 4: Look at the Results — What Was Found, and How Precisely?

Start with the numbers, not the authors’ interpretation of them. A statistically significant result can be trivially small, so look for the actual difference between groups, not just the p-value. Confidence intervals tell you how precise the estimate is: a wide interval means the true effect could be much bigger or much smaller than the headline number.

In qualitative papers, the “results” are themes, supported by quotes. Ask whether the themes are supported by the data shown, whether contradictory cases are discussed, and whether the quotes genuinely illustrate the claim attached to them.

Step 5: Ask Whether the Results Apply to Your Question

A valid study with impressive results can still be useless to you. Ask:

Are the participants like the people you care about? A study of university students may not generalise to older adults, or to a different country or health system.
Is the intervention or setting realistic? Could the treatment be delivered in a normal clinic, school, or workplace — or only in a highly resourced research unit?
Do the benefits outweigh the harms and costs? The paper should report adverse effects and, ideally, enough detail for you to judge feasibility.

If the answer to these questions is no, that does not make the study bad — it makes it less relevant to your purpose. Say so explicitly in your appraisal.

Step 6: Write Up Your Appraisal Clearly

A strong critical appraisal has a clear structure. Use this skeleton:

Summary: one short paragraph — the question, design, sample, key result.
Strengths: what the study did well (design, sample, analysis, reporting).
Limitations: the main weaknesses, ordered by importance, with a sentence on why each one matters.
Overall judgement: how much confidence you place in the findings, and whether the study answers your question.
Implications: what follows from the study — for practice, policy, or further research.

Be specific. “The study has limitations” says nothing. “The study lost 35% of participants by follow-up, and dropouts were more anxious at baseline, so the treatment effect is probably overestimated” is an appraisal.

Worked Example: Appraising a Trial of a Mindfulness App for Student Anxiety

To see the process in action, here is a short worked example. Imagine a paper reporting a randomised trial in which 120 undergraduate students with mild anxiety were assigned to either a six-week mindfulness app or a waitlist control. Anxiety was measured with the GAD-7 before and after the six weeks. The authors report that the app group improved by 4 points on the GAD-7, compared with 1 point in the control group (p = 0.01), and conclude the app is effective.

Validity: Randomisation is a strength, and the GAD-7 is a well-established measure. But the paper says nothing about allocation concealment, no one was blinded (unavoidable here, but unacknowledged), and 28% of the app group stopped using the app by week three. The analysis included only students who completed the study, not everyone randomised — so the results describe the keenest users, not the average student.

Results: A 3-point difference on the GAD-7 is modest; a change of around 4 points is often treated as clinically meaningful, so the between-group difference sits below that bar. No confidence interval is reported, so the precision of the estimate is unclear. The waitlist design also matters: waitlist controls tend to inflate apparent effects, because simply expecting help can change how people report symptoms.

Overall judgement: The study suggests the app may help motivated students a little, but the missing intention-to-treat analysis, the unblinded self-reported outcome, and the waitlist comparison mean the effect is probably overstated. More evidence is needed before recommending the app widely.

Notice what this appraisal does: it credits the randomisation and the validated measure, names each weakness with its consequence, and ends with a proportionate judgement — not a verdict of “good” or “bad”.

Common Mistakes to Avoid

Appraising the topic instead of the study. “Anxiety in students is an important issue” is not an appraisal. Judge how this study was done.
Listing flaws without explaining why they matter. Every limitation you name needs a consequence: what does it do to the results?
Treating a p-value as a measure of importance. A p-value tells you about compatibility with chance, not about the size or value of an effect.
Trusting the abstract. Abstracts are marketing for the paper. Appraise the full text.
Being only negative. A balanced appraisal credits strengths. If you cannot find any, you are probably missing something.
Using the wrong checklist. CASP has a different checklist for trials, cohort studies, case-control studies, qualitative studies, and systematic reviews. Match the tool to the design.

Quick Critical Appraisal Checklist

Keep this beside you while you work through a paper:

Is the research question clear and focused?
Is the design right for the question?
Was recruitment fair, and the sample adequately described?
Was allocation random and concealed (trials)? Was there a suitable comparison?
Were outcomes measured validly and consistently?
Was follow-up complete, and were dropouts handled properly?
Was the analysis appropriate, and were confounders addressed?
How big was the effect, and how precise is it?
Are all pre-specified outcomes reported?
Do the results apply to my population and setting?

FAQ

How long should a critical appraisal be?
For coursework, 800–1,500 words is typical: a short summary, then strengths, limitations, and a judgement. Longer appraisals are fine when the paper is complex, but length is not quality — one precise limitation beats three vague ones.

Which appraisal tool should I use?
Use the CASP checklist that matches the study design — there are separate checklists for randomised trials, cohort studies, case-control studies, qualitative research, and systematic reviews. Other respected tools include the Joanna Briggs Institute (JBI) checklists and, for trials, the Cochrane Risk of Bias tool. Pick one tool and apply it consistently.

Can I appraise a paper if I am not an expert in its topic?
Yes. Critical appraisal is about method, not subject expertise. You need to understand the design, the measures, and the analysis — not to be a world expert on the condition or intervention. When you genuinely cannot judge something (an unusual statistical method, for instance), say so rather than guessing.

What if I cannot find any weaknesses?
Every study has limitations, including well-conducted ones — the authors themselves usually list several in the discussion. Read the limitations section, then look for what they did not mention: generalisability, funding sources, and unmeasured confounders are common omissions.

Should I appraise the writing quality?
Only where it affects understanding. Clear writing is pleasant, but your job is to judge the science. A badly written paper can report a sound study, and a beautifully written one can hide serious flaws.

Final Thoughts

Learning how to critically appraise a research paper is a skill that improves fast with practice. Appraise two or three papers with the checklist above, and the process stops feeling like guesswork. Start with the three big questions — are the results valid, what are they, and do they apply to me — and let the detail follow from there.

How to Write a Critical Appraisal of a Research Paper