Reassessing existing funded health services and products to support appropriate care

Nov 29, 2013

The Challenge

In Ontario, new healthcare services and technologies coming into the system must undergo a rigorous evidence-based assessment. But what about existing services and technologies that are already in use? Which of those should be subjected to new evidence-based analyses to determine if they add value and are appropriate? Without a framework to identify priorities for reassessment from the thousands of existing funded services and products, the prospect of determining which services to scale back, focus or discontinue would be problematic at best.

The Improvement Project

These questions led Dr. Alison Paprica, Director, Health System Planning & Research Branch, Ministry of Health and Long-Term Care to CFHI’s EXTRA program to pursue development of a reassessment framework as her improvement project. Dr. Paprica consulted with many stakeholders to get the project moving, including her mentors, renowned health policy experts Dr. Adam Elshaug and Dr. Tony Culyer. Based on advice from ministry stakeholders and external experts, the framework proposed by Elshaug was adapted for Ontario. The end result is a framework that flags practices which are likely to be low value - or even harmful - to patients. According to the framework, candidates are flagged for reassessment if they meet two or more of the following triggers: an evidence based recommendation against use by an external body; nominated by a local clinical expert; safety concerns as noted in the literature; regional and/or temporal variation that suggest inappropriate use; the change is likely to provide benefit to significant number of people in Ontario (i.e. change would have a positive impact for at least 1000 individuals per year); and the change would provide a cost saving (i.e. result in cost savings of at least $1M per year).

The Key Results and Impact 

Dr. Paprica’s project has resulted in several quick wins for her province, and beyond. Her work has led to the development of the following definition of appropriateness which has been adopted by the Council of the Federation Health Care Innovation Working Group (HCIWG) comprising all provincial and territorial health ministers: 

In the context of healthcare, appropriateness is the proper or correct use of health services, products and resources. Inappropriate care, in contrast, can involve overuse, underuse and/or misuse of health services, products and resources. Appropriateness is primarily determined by analyses of the evidence of clinical effectiveness, safety, economic implications, and other health system impacts. The practical application of appropriateness is made when these analyses are qualified by (a) clinician judgment, particularly in atypical circumstances and (b) societal and ethical principles and values, including patient preferences.

Also in support of the HCIWG, a survey was developed and implemented that identified three Canada-wide priorities for diagnostic imaging appropriateness: imaging for low back pain, imaging for minor head trauma and imaging for uncomplicated headache. These priorities were accepted by Premiers at the July 2013 Council of the Federation meeting, with a recommendation to increase appropriate use for them through guidelines.

In addition to these impacts, in April 2013, a ministry Evidence-Based Analysis Priorities Committee (EBAPC) was established with the mandate to function as the ministry focal point for evidence-based analyses performed by external groups. The role of EBAPC is to synthesize information to produce recommendations that inform decisions made through existing decision making processes at the Ontario Ministry of Health and Long- Term Care.

The project has had impacts at the institutional/ organizational, budgetary, human resources and policy levels. The Ontario reassessment framework identified candidates that have since proceeded to selective disinvestment, resulting in an estimated $59 million per year (CDN$) freed up for investment elsewhere. For example, partial disinvestment has been implemented in Ontario for routine vitamin B12 testing, routine ferritin testing, and daily use of diabetes test strips by people with diabetes who do not take insulin. Each of these practices was a priority for reassessment based on triggers 1 and 6 of the reassessment framework and, after further involvement of local stakeholders (without additional external context-free scientific evidence), selective disinvestment was implemented.

Having the framework and an objective process for identifying reassessment priorities has bolstered confidence among staff in making and defending decisions, enhanced interest in evidence-based disinvestment and contributed to evidence-based operational policy across the ministry. The Committee has initiated an annual process to gather suggestions of potential products and services that are deemed to be of low value and it will continue to help provincial/territorial policy work on appropriateness. It is expected that the reassessment framework, and the processes and templates of the Committee may be useful for other bodies and jurisdictions. Efforts are currently underway to disseminate the project findings and share the templates and processes in order spread the learnings.


Dr. Alison Paprica
System Planning and Research Branch
Ministry of Health and Long-Term Care Ontario

To learn more about this project or the EXTRA program, visit or email us at

1 Elshaug, A.G., Moss, J.R., Littlejohns, P., Karnon, J., Merlin, T. L., Hiller, J. E. (2009b). Identifying existing health care services that do not provide value for money. Med J Aust, 190(5), 269-7