Development and Testing of a Decision Support Tool for Healthcare Performance Measurement

by Geoffrey Anderson | Mar 01, 2001

Key Implications for Decision Makers

  • The research literature has defined measures of health care performance, and applying those measures to routinely collected data could provide important information for healthcare management and reform.
  • The feasibility of using routinely collected data for measuring performance is determined by the validity of the existing data elements, the availability of data on key aspects of performance, and issues of privacy and confidentiality.
  • Decision makers use performance measures for evaluation and planning. For performance measurement data to be more useful to them it needs to be easy to access and designed to answer a range of different questions in a timely fashion.
  • A decision support tool using the expertise of researchers to define the key dimensions of performance and a sophisticated interface system to provide easy real time access to information can be useful to decision makers.
  • Future work on performance measurement will require efforts to maintain and improve the quality of data and initiatives to design systems that make relevant information easily available to decision makers.
  • Development of these systems will require leadership, commitment, and collaboration of researchers, data collection agencies, and decision makers.

Executive Summary

Measuring the performance of the healthcare system is a key element in healthcare management, reform, and renewal. The purpose of this study was to systematically examine research application and knowledge translation issues in the context of this increasing interest in healthcare system performance measurement. More specifically, the goals of the study were to 1) identify some of the key issues faced in applying measures of effectiveness, access, and quality of care described in the research literature to hospital discharge data routinely collected by the Canadian Institute for Health Information (CIHI); and 2) to develop and evaluate a decision support tool that would provide relevant information derived from those measures to decision makers. The funding partners for the project were the Ontario Hospital Association, the department of health policy, management, and evaluation at the University of Toronto, and CIHI. The study was done with the active involvement of a range of organizations involved in healthcare planning, management, and evaluation.

Applying techniques for measuring healthcare system performance to the CIHI data provided a useful reminder of the issues consistently faced when administrative data are used for evaluation: concerns over the validity of existing data elements, and the lack of data on key aspects of performance. Maintaining and improving the validity of data is complex and will require ongoing assessment and initiatives to improve coding. The problem is compounded by the incentive for institutions to change their coding practices in response to the use of the data for performance assessment or funding. Real commitment to meaningful performance measurement will mean adding new data elements to administrative data specifically for the purpose of performance measurement and making better use of linkages among data sources to provide a more comprehensive assessment. Along with these data issues, statistical and ethical issues will need to be addressed in the assessment of the performance of low-population regions, small hospitals, or individual providers.

For performance measurement information to have a positive effect on the system, it must be made available to and used by decision makers. This study led to the development of a decision support tool that makes researcher expertise in producing information from administrative data available to decision makers through an interface that can be easily accessed in real time by decision makers. Decision makers can use the information to address a wide range of questions regarding healthcare system performance. The decision support tool was evaluated by a range of decision makers who routinely made use of administrative data for evaluation, planning, and management. They found the decision support tool to be useful and, although they identified some constraints and limitations, they felt this type of tool could play an important role in decision-making. A challenge for the future will be finding the support and leadership to develop such tools for those who could use them.

This project brought researchers, data collectors, and decision makers together to work through a process for taking measures described in the research literature, applying them to an existing data source, and making them relevant and accessible. Future work on performance measurement will require this type of collaboration and co-operation.