Harvard Library of EvidenceHarvard Library of EvidenceHarvard Library of Evidence
Harvard Medical School Library of Evidence

Harvard’s Public Repository of Evidence for Clinical Decision Support
Transparently Graded
Continuously Updated
Tell Me More


The effective use of a clinical decision support system means patients get the right tests, the right medications, and the right treatment, particularly for chronic conditions.


The Library is a project of the HMS Countway Library of Medicine and representatives of other HMS hospitals and Harvard institutions.

Read more

Legislative Context

The Library is intended to support the national imperative to encourage the practice of evidence-based medicine, including the efforts of the ...

Read more

Literature Search

Identification of the sources was accomplished through a literature search of MEDLINE (1980-2016) using the text search terms, “decision rule”, ...

Read more

Grading Methodology

The Library’s evidence grading methodology is based on the 2009 version of the Oxford Centre for Evidence-based Medicine (CEBM) ...

Read more

Roles and Responsibilities

Project Manager
Validating Clinician

Read more


Appropriate use criteria for advanced diagnostic imaging services, Section 218 (b) of the Protecting Access to Medicare Act of 2014.

Our Remarks

Overall, we would like to congratulate CMS on articulating the vision behind effective implementation of AUC and Clinical Decision Support (CDS) so well in the Proposed Rules. Given our experience with the grading and evaluation of medical evidence, we especially applaud the Proposed Rules’ emphasis that “the goal of this statutory AUC program is to promote the evidence-based use of advanced diagnostic imaging to improve quality of care and reduce inappropriate imaging.” In addition, we thank CMS and Intermountain Healthcare for organizing the multidisciplinary forum we attended, which allowed allowing for active engagement by the diverse stakeholders affected by this program – it was immensely valuable.

As the only curated freely-available library of evidence, our comments will focus on the aspects of the Proposed Rules that pertain to the Priority Clinical Areas. However, it is important to note that we strongly support CMS’ intent to support both focused and broad implementations of CDS. In our experience scoring evidence meant to guide imaging across multiple clinical indications, we have found its strength to be highly variable. While QPLEs should be allowed to implement CDS broadly if they so desire, more focused implementations utilizing specific high-quality evidence will be more effective based, on our experience.

Clinical Scope of Priority Clinical Areas

We applaud the Proposed Rule’s intention of bridging the aforementioned gap between broad and focused implementations of CDS. However, we are concerned about the statement requiring that Clinical Decision Support Mechanisms (CDSM) include AUC that “reasonably address the entire clinical scope of priority clinical areas” as a floor in order to qualify. While we support the concept of a floor upon which subsequent expansion can be built (and envision a similar expansion for the Library concurrent with that process), we are concerned because – at least for some of the PCAs proposed – covering the “entire clinical scope” with high quality evidence is simply impossible. We will discuss this in detail below, but the chart below may be helpful for overall context. Currently, the Library of Evidence contains over 700 pieces of evidence, 424 of which relate to the PCAs listed in the Proposed Rule.

However, the strength of the evidence for each of these PCAs varies widely. We generally classify evidence graded using the Oxford Centre for Evidence-Based Medicine, Levels of Evidence (2009) as high quality if it is a Level 1-4 – in contrast to Level 5 evidence, defined as “Expert opinion without explicit critical appraisal.” For the PCAs listed in the Proposed Rule, the proportion of Level 5 low quality evidence ranges from 8% (for Low Back Pain) to 82% (for Cancer of the Lung.) Attempting to implement AUC coverage for each PCA that addresses the “entire clinical scope” would necessarily require CDSM to include this low quality evidence, which is (we believe) antithetical to CMS’ intent.

Instead, we suggest that CMS delegate to QPLEs the authority to rigorously evaluate evidence and formulate AUCs based on this evaluation. If the QPLE determines that high quality evidence exists to inform the development of AUCs related to specific PCAs, then it should be required to implement them. Conversely, if a QPLE determines that there is not high quality evidence to support AUC targeting a specific PCA, it should be required to note this publicly, stating which evidence was evaluated and deemed to be of insufficient strength. Given that the HMS Library of Evidence has already conducted these evidence reviews and allows for free public access to our database, we would also support QPLEs referencing our review to meet these requirements.

Specific Priority Clinical Areas

As a multidisciplinary and interprofessional group we agree with the broad concepts behind the PCAs outlined by the proposed rules. These conditions are seen in all patient populations and in healthcare systems both large and small. Most of them have clear opportunities for improved care for CMS beneficiaries through evidence-based recommendations. We do, however, suggest five modifications to the list, for the reasons noted below:

A. Given the clinical significance, varied clinical presentations (often other than chest pain), and prevalence of pulmonary embolism, we support extracting this condition from the Chest Pain category. We suggest creating a separate PCA to govern the imaging of patients suspected of having pulmonary embolism. There is a wealth of validated evidence-based literature surrounding the appropriate imaging for patients with suspected pulmonary embolism, and we believe that this supports its separate and distinct emphasis in the list of PCAs.

B. For the PCA of Abdominal Pain, we suggest the notation of the specific qualifying diagnoses of pancreatitis, appendicitis, and renal colic. While the number of potential causes for abdominal pain is broad, these three diagnoses have strong evidence-bases for imaging. Conversely, there is little high-quality evidence for the imaging of generalized and undifferentiated abdominal pain.

C. For the Cancer of the Lung PCA, we advocate the addition of lung cancer screening to the already present qualifiers of primary or metastatic, suspected or diagnosed. The inclusion of lung cancer screening would be beneficial as there are well defined and evidenced based criteria outlining the population that benefits from screening examinations.

D. We suggest wholly removing Altered Mental Status from the list of PCAs. The wide spectrum of potential clinical causes for Altered Mental Status – including but not limited to toxic, hypoxemic, oncologic, metabolic, environmental, infectious, cardiovascular, neurologic, and psychiatric – make this PCA exceptionally broad without clear evidence-based recommendations for imaging for many of the potential causes.

E. We would support adding the “traumatic” modifier to Cervical or Neck Pain, similar to its inclusion as a modifier for the Headache symptom. There are a number of evidence-based guidelines for imaging of patients with traumatic cervical pain, but a distinct lack of evidence guiding the use of imaging in atraumatic pain.

The Revisions to the Proposed Rule present a true step forward and provide appropriate clarifications for PAMA. We are truly appreciative of the efforts by CMS to implement PAMA in a thoughtful manner and to solicit input regarding this implementation. If we can be of any additional assistance, please contact either of us.

Read the document for details.


In response to the AHRQ National Guideline Clearinghouse (NGC, guideline.gov) Web site’s unavailability after July 16, 2018, the HMS Library of Evidence will provide The National Guideline Clearinghouse Extent Adherence to Trustworthy Standards (NEATS) Assessments for guidelines in which NEATS Assessments have been produced thus far by the AHRQ. The NEATS Instrument is available on our site here. This does not imply any endorsement by the AHRQ NGC.

Summer 2016 Updates

Statute and Regulation Highlights

  • Protecting Access to Medicare Act of 2014 (PAMA)
    • PAMA Section 218(b), “Promoting Evidence-Based Care”
    • Requires ordering professionals to consult Appropriate Use Criteria (AUC) through a Clinical Decision Support (CDS) mechanism when ordering specified advanced imaging services
    • Consequence for non-compliance: no reimbursement for the technical or professional component of imaging
  • CMS Rules for specifying applicable AUCs:
    • Proposed rule published July 15, 2015
    • Final rule released October 30, 2015, published November 16, 2015

Statute and Regulation Detailed Analysis

The Statute changes the content of claims

"With respect to an applicable imaging service furnished in an applicable setting and paid for under an applicable payment system..., payment for such service may only be made if the claim for the service includes the following:

  • (i) Information about which qualified clinical decision support mechanism was consulted by the ordering professional for the service.
  • (ii) Information regarding
    • (I) whether the service ordered would adhere to the applicable appropriate use criteria specified under paragraph (2);
    • (II) whether the service ordered would not adhere to such criteria; or
    • (II) whether such criteria was not applicable to the service ordered.
  • (iii) The national provider identifier of the ordering professional (if different from the furnishing professional)."

Appropriate Use Criteria (AUC), as defined by the final rule

  • "criteria only developed or endorsed by national professional medical specialty societies or other provider-led entities to assist ordering professionals and furnishing professionals in making the most appropriate treatment decision for a specific clinical condition for an individual. To the extent feasible, such criteria must be evidence-based."
  • "An AUC set is a collection of individual appropriate use criteria. An individual criterion is information presented in a manner that links:
    • a specific clinical condition or presentation;
    • one or more services; and,
    • an assessment of the appropriateness of the service(s)."

(Qualified) Provider Led Entities (QPLEs and PLEs), as defined by the final rule

  • "Provider-led entity (PLE) means a national professional medical specialty society or other organization that is comprised primarily of providers or practitioners who, either within the organization or outside of the organization, predominantly provide direct patient care."
  • "(c) Qualified provider-led entity. To be qualified by CMS, a PLE must adhere to the evidence-based processes described when developing or modifying AUC. A qualified PLE may develop AUC, modify AUC developed by another qualified PLE, or endorse AUC developed by other qualified PLEs."

Required evidence-based processes for QPLE Designation

A QPLE must perform all of the following when developing or modifying AUC:

  • (1)* Utilize an evidentiary review process that includes
    • Systematic literature review of the clinical topic and relevant imaging studies;
    • Assessment of the evidence using a formal, published and widely recognized methodology for grading evidence
  • (2) Utilize at least one multidisciplinary team with autonomous governance, decision-making and accountability for developing or modifying AUC:
    • At least 7 members including at least one physician with expertise in the clinical topic of the AUC, one physician with imaging expertise relevant to the AUC, one PCP, one expert in statistics, one expert in clinical trial design
  • (3) Utilize a publicly transparent COI process
  • (4)** Publish each criterion on the PLE website, identifying title, authors (all members of team), and key references for evidence
  • (5)** Identify on the PLE website each AUC relevant to a priority clinical area
  • (6)** Identify key points in an individual criterion as evidence-based or consensus-based, and grade such key points in terms of strength of evidence using a formal, published and widely recognized methodology
  • (7)* Utilize a transparent process for updating all AUC
  • (8)** Post the process for developing or modifying AUC on the PLE website
  • (9) Disclose parties external to the PLE when such parties have involvement in the AUC development process

     * = Requirements met by Library.
     ** = Published by Library for QPLEs to display on their own websites.

Process to become a QPLE

  • Submit application to CMS that documents adherence to each of the 9 requirements
  • Meet the definition of PLE
  • Apply by January 1 of any year (CMS 6 month review)
  • Current List of QPLEs
See QPLEs and the Library

Local Adaptation of AUC

  • On the substance of local best practices, CMS added:
    • "We believe that allowing for variations in AUC based on local circumstances is important to ensure that AUC consultation can be incorporated into clinical practice throughout the country. We agree that local variations should still meet the program requirements to ensure that the evidence to support modification is evaluated and graded and only performed by qualified PLEs.”


  • CMS clarified the schedule for proposing and finalizing the rules for CDS mechanisms, and stated that ordering professionals will not be required to consult applicable AUC on January 1, 2017.
    • CMS noted that the rule has massive impact across almost every medical specialty and “could have a particular impact on primary care physicians”
  • "If we were to follow a similar process for CDS as we have for specifying AUC, the initial list of CDS mechanisms would be available in the summer of 2017.”
  • "We anticipate including further discussion and adopting policies regarding claims-based reporting requirements in the CY 2017 and CY 2018 rulemaking cycles. Therefore, we do not intend to require that ordering professionals meet this requirement by January 1, 2017.”

Proposed Priority Clinical Areas

Updated 11/3/2016

CMS has developed and are proposing eight priority clinical areas, reflecting both the significance and the high prevalence of some of the most disruptive diseases in the Medicare population. The Library of Evidence covers 100% of these priority clinical areas.

  • Coronary artery disease (suspected or diagnosed)
  • Suspected pulmonary embolism
  • Headache (traumatic and non-traumatic)
  • Hip pain
  • Low back pain
  • Shoulder pain (to include suspected rotator cuff injury)
  • Cancer of the lung (primary or metastatic, suspected or diagnosed)
  • Cervical or neck pain