Continuity of Mental Health Services Study of Alberta: A Research Program on Continuity of Mental Health Care

by Carol E. Adair | Apr 01, 2004

Key Implications for Decision Makers

  • Continuity of care is a complex phenomenon, comprised of events in the process of mental healthcare that occur both among diverse services at one point in time and within services over a longer period of time.
  • Better continuity was associated with higher community costs (averaging $1,511 higher for the highest continuity category compared to the lowest) and lower hospital costs (averaging $4,790 lower for the highest continuity category compared to the lowest). This relationship held even after adjusting for age, household income, suicidal thoughts and behaviour, and duration of illness. This finding is new evidence that health service systems with good continuity of care can reduce hospital costs.
  • The Alberta Continuity of Services Scale for Mental Health is an instrument to measure continuity from both the "subjective" client/patient perspective and the "objective" observer perspective. Initial indications are that the instrument has reasonable construct validity and reliability; however, it needs to be refined before it can be used widely or routinely to measure the performance of mental health services.
  • Clients in different local programs or in different demographic subgroups had different levels of continuity. These differences should be analysed to inform local service improvements.
  • Patients at risk of poor continuity of care are those with a diagnosis of mood disorders, current or a history of suicidal thoughts or behaviour, and those with addictions.
  • There were no differences for patients already in treatment according to their gender, ethnic status, education, or employment. This indicates that care is relatively equitable.
  • Continuity of care is associated with improved quality of life, community functioning, and satisfaction with services.
  • The strong correlation between satisfaction with services and patient/client reports of continuity may be partly explained by a tendency to respond favourably or unfavourably to questions about services in general, rather than a real assessment of the very broad and abstract experience of continuity.
  • An intervention team that helps patients receive continuous services in the community was associated with patients having lower symptom levels, better community functioning, better quality of life, and better self-reported health, compared with those who received usual care.

Executive Summary

Continuity of care is defined as "a process involving the orderly, uninterrupted movement of patients among the diverse elements of the service delivery system."1 In mental health services, continuity of care has long been considered to be essential for good patient/client outcomes for those with severe and persistent mental illness. However, there is little evidence to support either the basic premise that continuity of care leads to improved outcomes or the contention that service changes designed to improve continuity actually achieve either continuity itself or the intended clinical outcomes in a cost-effective manner. The lack of evidence is felt to be due to inadequate measures of continuity, as well as the lack of well-designed studies to test the set of related research questions at hand. Reasons for the lack of progress in measuring continuity are clear - care continuity is, by its nature, a complex, multi-dimensional process that occurs over time and among multiple settings during an individual's care.

The Continuity of Mental Health Services Study of Alberta was a three-phase multi-year research program, which operated between October 1999 and August 2003 in three health regions in Alberta. Its purposes by phase were:

  • to develop and test a reliable and valid standard measure of continuity of care for mental health services by operationalizing components of continuity of care as expressed in the mental health literature and through interviews with patients/clients, families, and services providers in the field;
  • to describe continuity of care in three parallel cohorts of patients/clients with severe and persistent mental illness and its relationship with clinical and economic outcomes; and
  • to pilot an economic evaluation methodology which will examine the effectiveness of new initiatives designed to achieve continuity of care, and their efficiency in terms of costs and benefits.

In Phase I, a systematic literature review and interviews with patient/clients and their families provided conceptual material to develop draft items for two components of an instrument to measure continuity - a patient/client self-report component and a component containing items to be rated by a trained observer on health event information collected prospectively. Draft items for the patient/client report component were pre-tested with patients/clients and mental healthcare professionals and then piloted in a sample of 317 patients/clients in three regions. Draft items for the observer component were validity-tested by seven raters on five case scenarios developed from health records. Both components were found to have reasonable validity and reliability, although there was much variability among items in performance.

The resulting instrument - the Alberta Continuity of Services Scale for Mental Health - was then used in a 17-month cohort study of adult service recipients with severe and persistent mental illness in three health regions (Phase II), along with measures of service utilization, outcomes, and costs. Follow-up rates for cohort participants were very high at 85 percent. The study samples were comparable by region on many variables, but significant differences were found on recruitment location, education, income, and duration of illness. These differences were adjusted for in multivariate analysis to make regional comparisons on continuity, costs, and outcomes appropriate for reporting to local stakeholders.

In initial analyses differences among regions on the mean overall score for the patient/client scale were not significant, but the overall score masked interesting variations among regions on the 43 individual items. In contrast with the patient/client scale, mean total score differences were found to be statistically significant by region on the observer-rated scale. Ratings on nine of the 17 items were also significantly different by region, with regions alternating rank position according to the specific item. Correlation between the two scales was only moderate in magnitude, suggesting that they are only partially measuring the same concept or that the method of reporting introduces some bias. In the total sample, continuity of care as measured by either scale had statistically significant associations with some of the demographic variables and clinical variables. Higher levels of continuity were associated with older age, lower annual household income, psychotic diagnoses, and no suicidality or substance use. Continuity of care was also associated with health outcomes as measured at the end of the study period, including patterns in the expected direction across disease-specific quality of life and symptom severity and statistically significant associations with generic quality of life, self-reported health status, community functioning, service satisfaction, hospital costs, and community costs.

In Phase III, we piloted a method to study the effectiveness of a new service designed to improve continuity of care (the community extension team) in the Calgary Health Region. This phase also used the study's measurement tool, the outcome measures, and the methods for costing that were used in Phase II, and experimented with methods for randomized study designs in natural practice settings. Initial findings show that the patients who received the intervention had lower symptom levels and better functioning, quality of life, and self-reported health than those who received usual care. Suicide attempts, hospital admission, and inpatient days were also lower among the intervention group, but ER and mobile crisis team events were more prevalent for as-yet-unknown reasons. Given that the intent was to pilot methods, the sample size precluded the establishment of statistical significance for these findings. Data verification via chart review and cost comparisons are yet to be made. A larger, confirmatory study will be necessary to fully test the community extension team model.

Recommendations

The study's measurement tool should undergo further refinement, including a detailed review of which items were most strongly associated with outcomes and reduction to a smaller item set before it is ready for use in performance measurement.

Scores on individual continuity items at the regional and program levels can be used to identify specific areas to target for general services improvement.

The rich data set from the Phase II cohort study can now be used for in-depth studies of a set of mental health service issues that were identified by families and participants, including basic supports (housing and income); patterns of medication use and co-morbid physical illness; crisis and emergency services; work/vocational supports; psychiatrist services; use of complementary and alternative therapies and private providers; and addictions co-morbidities.

New and existing services intended to improve continuity of care can be evaluated using the methods and measures that were piloted with the community extension team - use of a standard approach will permit comparison of the performance of service innovations across settings and patient populations.

The community extension team model should be further evaluated with a larger study sample once the intervention achieves more stable funding and greater consistency in delivery.

1 Bachrach, L., Continuity of care for chronic mental patients: a conceptual analysis. American Journal of Psychiatry, 1981. 138(11): p. 1449-56.