The GRACE Initiative

The GRACE Initiative set out to develop a useable set of key principles that could be used to assess the quality of observational research for studying treatment effectiveness. We began by developing a core set of good practice principles to address the design, conduct, analysis, and reporting of observational studies of comparative effectiveness, working in collaboration with the International Society of Pharmacoepidemiology and with some seed funding from the National Pharmaceutical Council. A checklist was then created to facilitate GRACE reviews.

The GRACE checklist is distinguished by its extensive validation. Checklist items underwent initial testing by more than a hundred volunteers representing diverse perspectives from 5 continents. These validation efforts demonstrate that the strong sensitivity and specificity of GRACE and its utility for decision-support set it apart from other checklists.

Some key points from our validation studies included:

• The best predictor of quality is the use of a sensitivity analysis to provide quantitative or semi-quantitative estimates of how much a study's results are depending on key assumptions.
• Concordance of expert opinion on the quality of the rated articles was 52% and most checklist items performed better.