Clinical Practice Guidelines: What’s Next?

Steps Forward

The Institute of Medicine Report

The release of the 2011 Institute of Medicine (IOM, now the National Academy of Medicine) report Clinical Practice Guidelines We Can Trust was an important step forward.¹ With this report, for the first time, an authoritative body proposed methods for guideline development that could no longer be ignored. According to the IOM report, clinical practice guidelines are defined as “statements that include recommendations, intended to optimize patient care, that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options.”¹ Since 2011, a set of practice recommendations not explicitly informed by a systematic review should no longer be considered a clinical practice guideline. This change in definition resulted in a reduction of nearly 50% in the number of guidelines listed on the National Guideline Clearinghouse (NGC), from 2619 in 2014 to 1440 in 2018 (as older guidelines without a systematic review are removed from the site).

However, some operational issues still need to be clarified. For example, does a new systematic review need to be conducted for guideline development, or can existing systematic reviews be used? If the latter, how old can the review be and still be considered up to date? Moreover, the change in definition, plus the explication of the criteria that make a guideline trustworthy (use of a multidisciplinary panel of experts, managing conflicts of interest, consideration of patient subgroups and patient preferences, using an explicit and transparent process for development, providing ratings of both the quality of the evidence and strength of the recommendations) brought a much-needed “raising of the bar” to guideline development. The more internationally representative Guidelines International Network standards² released in 2012 were in general similar.

The GRADE Framework

Although the first major article from the Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group was published in 2004, the last few years have seen a substantial increase in the number of guideline groups using GRADE and the number of options available within GRADE.³ The step forward with GRADE is the separate assessments of the certainty (or quality or strength) of the evidence and the strength of the guideline recommendation. This decoupling makes it more transparent in determining the evidence base on which a recommendation is founded. The GRADE framework for assessing the certainty of the evidence works particularly well for interventions for chronic diseases.

The areas for which the GRADE framework has not worked as well, such as patient safety and quality improvement interventions, are the subject of active GRADE working groups. When the evidence available consists of pooled results of randomized trials, GRADE has demonstrated good reliability (κ = 0.6-0.7).⁴ However, since it requires subjective interpretation of the evidence, GRADE is likely less reproducible but probably in general no less so than other systems of assessing evidence. Overall, the wider adoption of GRADE and its Evidence to Decision framework, accompanied by concomitant research into those parts of GRADE that work well and those parts that may need improving, generally represents an important advance for guideline development.

Steps Backward

Too Many Guidelines

In 2003, 1402 guidelines were indexed on the NGC, whereas by 2013, that number had increased to 2619. Many of these guidelines are about the same or similar topics but were produced by different organizations. Guidelines from different groups can either agree or mostly agree with each other (making them redundant and a potential waste of scarce guideline development resources) or they can disagree, leaving clinicians and patients uncertain about which recommendation to follow. Breast cancer screening among women aged 40 to 50 years, treatment of type 2 diabetes, and the definition and treatment of hypertension are particularly noteworthy recent examples of where guidelines disagree.⁵ Having guidelines from different groups with major differences in recommendations does not serve patients or clinicians, as these different recommendations inevitably get viewed through the lens of vested interests. In addition, when there are substantial differences in major recommendations of guidelines, patients and clinicians may not trust the process and then may simply dismiss the recommendations. Fewer and more trustworthy guidelines are needed. In terms of what organizations should be responsible for guideline development, the type of organization matters less than adherence to a transparent and rigorous approach to managing conflict of interest and guideline development.

The Demise of the National Guideline Clearinghouse

On July 16, 2018, free access to the National Guideline Clearinghouse (NGC) website was inactivated because of lack of funding for it to continue. This means the 2.6 million annual visitors to the site will no longer have free access to summaries of 1385 guidelines, many of which have also been assessed using the NGC Extent Adherence to Trustworthy Standards assessment tool. This shuttering of free access to the NGC undermines one of the key recommendations of the IOM Trustworthy Guidelines report, namely that the NGC provide users with information to help in distinguishing trustworthy guidelines from the rest: “The NGC is a highly useful guideline dissemination tool… the NGC should eliminate clinical practice guidelines for which trustworthiness cannot be determined, and identify the trustworthiness of those retained.”¹ The loss of free access to the NGC is a major step backward for users of clinical practice guidelines.

Forward Progress Needed

Many issues still need to progress in practice guideline development, but 2 are of particular importance.

Conflict of Interest

Financial conflicts of interest among organizations and experts creating practice guidelines are highly prevalent, can in some instances be quite large, are often unreported or reported inaccurately, are subject to inadequate or nonexistent policies on disclosure, and in some cases have been associated with differences in recommendations.⁶ The concern about financial conflicts of interest extends beyond whether panel members receive money from vested industry interests. A guideline that recommends more visits and procedures may be regarded as being potentially self-serving when the guideline committee consisted solely or predominantly of members of the clinical group likely to benefit from the increased demand. The IOM report recommends that committee members with conflicts of interest should be in the minority,¹ and because certain recommendations provide financial benefits to certain clinical specialties, organizations should require that such specialists be a minority on most guideline committees. Organizations that produce practice guidelines must do a better job at reporting and managing conflicts of interest. The current status quo has not been sufficient to improve this issue. Refusing publication on the part of journals for guidelines that do not meet the IOM standards on conflict of interest might be one effective way to stimulate improvement.

Updating

The need for regular and more frequent updating has been an important but unaddressed aspect of clinical practice guidelines for some time, and there is general agreement about the need to address this issue, The Guidelines International Network Updating Guidelines Working Group⁷ has been making progress with some of the methodologic issues, and some organizations are trying to take steps forward.⁸ The main barrier does not involve knowing how to accomplish updating but rather determining who should pay for it. Machine learning may be helpful in the future to update guidelines more efficiently, but so far its proven ability to substitute for human effort is limited to just a few examples. However, the urgency of updating clinical practice guidelines cannot wait for a technological advance that for a decade has seemed to be just around the corner. It is not clear that a new transfusion of resources into the system is needed to achieve effective surveillance and updating of practice guidelines. If different organizations that develop competing guidelines would stop producing multiple guidelines on the same topic and would pool their resources to develop joint guidelines, the result would be fewer and more regularly updated guidelines without need for a net increase in resources. That would also serve to make clinical practice guidelines even more useful for clinicians and patients.

Clinical practice guidelines will remain an important part of medicine. Trustworthy guidelines not only contain an important review and assessment of the medical literature but establish norms of practice. Ensuring that guidelines are up-to-date and that the development process minimizes the risk of bias are critical to their validity. Reconciling the differences in major guidelines is an important unresolved challenge.