12 Questions to Help You Make Sense of Cohort Study Checklist – Critical Appraisal Skills Programme (CASP)

(A) Are the results of the study valid?

Screening Questions

1. Did the study address a focused issue?

HINT: A question can be ‘focused’ In terms of

• The population studied
• The risk factors studied
• The outcomes considered
• Is it clear whether the study tried to detect a beneficial or harmful effect?

Yes, can’t tell no

2. Was the cohort recruited acceptably?

HINT: Look for selection bias, which might compromise the generalisability of the findings:

• Was the cohort representative of a defined population?
• Was there something special about the cohort?
• Was everybody included who should have been included?

Yes, can’t tell No

Is it Worth Continuing?

Detailed Questions

3. Was the exposure accurately measured to minimise bias?

HINT: Look for measurement or classification bias:

• Did they use subjective or objective measurements?
• Do the measurements truly reflect what you want them to (have they been validated)
• Were all the subjects classified into exposure groups using the same procedure

Yes, can’t tell  no

4. Was the outcome accurately measured to minimise bias?

HINT: Look for measurement or classification bias:

• Did they use subjective or objective measurements?
• Do the measures truly reflect what you want them to (have they been validated)?
• Has a reliable system been established for detecting all the cases (for measuring disease occurrence)?
• Were the measurement methods similar in the different groups?
• Were the subjects and/or the outcome assessor blinded to exposure (does this matter)?

Yes, can’t tell  no

5. (a) Have the authors identified all important confounding factors?

• List the ones you think might be
• important that the author missed.

Yes, can’t tell  No

(b) Have they taken account of the confounding factors in the design and/or analysis?

HINT: Look for restrictions in design, and techniques, e.g. modelling, stratified-, regression-, or sensitivity analysis, to correct, control or adjust for confounding factors

Yes, can’t tell  no

6. (a) Was the follow-up of subjects complete enough?

Yes, can’t tell  no

(b) Was the follow-up of subjects long enough?

HINT: Consider

• The good or bad effects should have had long enough to reveal themselves
• The persons who are lost to follow-up may have different outcomes than those available for assessment
• In an open or dynamic cohort, was there anything special about the outcome of the people leaving, or the exposure of the people entering the cohort?

Yes, can’t tell  no

(B) What are the results?

7. What are the results of this study?

HINT: Consider

• What are the bottom-line results?
• Have they reported the rate or the proportion between the exposed/unexposed, the ratio/the rate difference?
• How strong is the association between exposure and outcome (RR)?
• What is the absolute risk reduction (ARR)?

Yes, can’t tell no

8. How precise are the results?

HINT: Look for the range of the confidence intervals, if given.

Yes, can’t tell no

9. Do you believe the results?

HINT: Consider

• Big effect is hard to ignore!
• Can it be due to bias, chance or confounding?
• Are the design and methods of this study sufficiently flawed to make the results unreliable?
• Bradford Hills criteria (e.g. time sequence, dose-response gradient, biological plausibility, consistency)

Yes, can’t tell no

(C) Will the results help locally?

10. Can the results be applied to the local population?

HINT: Consider whether

• A cohort study was the appropriate method to answer this question
• The subjects covered in this study could be sufficiently different from your population to cause concern
• Your local setting is likely to differ much from that of the study
• You can quantify the local benefits and harms

Yes, can’t tell no

11. Do the results of this study fit with other available evidence?

Yes, can’t tell no

12. What are the implications of this study for practice?

HINT: Considero

• One observational study rarely provides sufficiently robust evidence to recommend changes to clinical practice or
• Within health policy decision-making
• For certain questions, observational studies provide the only evidence
• Recommendations from observational studies are always stronger when supported by other evidence

Yes, can’t tell No