HHS Quality of Care Measurement and
Monitoring Studies

since the introduction of Medicare's prospective payment system (PPS) for inpatient hospital services in 1983. They tend to focus on process and outcome variables that primarily deal with aspects of health service utilization and health status.

The measurement studies are mainly concerned with the development or testing of quality assessment methods. They often explore possible refinements in the measurement of quality of care, address a number of issues regarding the appropriateness of existing data for research in this area, and may incorporate a monitoring component within their framework.

The studies listed in table VI.1 are characterized according to their current status, funding, research designs, the focus of quality of care variables (structure, process, or outcome of care), and the types of data analyzed. In the discussion below, we review some of the particularly important HHS studies currently underway or recently completed, beginning with the monitoring studies, which are categorized according to whether they focus on measurement of health status or outcomes, assessment of the process of care, or some combination of the two. The measurement studies are discussed last.

Direct Assessments of
Health Status and
Outcomes of Care

Concerns about the appropriateness of hospital discharge decisions under PPS and the availability and quality of posthospital care services are reflected in several HHS-supported studies, which directly assess health status and health care outcomes.

The Aftercare study, developed jointly by ASPE and HCFA and contracted
to Mathematica, investigates the types of posthospital care received by
Medicare beneficiaries and the adequacy of that care. In the design
phase, researchers are developing approaches for measuring the ade-
quacy of posthospital care and its relationship to the care outcomes of
individual patients. Specific attention was devoted to the development
of instruments for abstracting relevant information from hospital
records, refining questions to be administered to posthospital patients
(at 2 and 6 weeks after discharge), and refining clinical guidelines for
determining the adequacy of aftercare. The instruments and sampling
procedures are to be tested in a pilot study conducted at eight hospitals
in two states (scheduled to be completed in 1988). A national survey
designed to produce estimates of the incidence, nature, and conse-
quences of inadequate care for Medicare patients discharged from hospi-
tals is planned as the third stage of the study.

HHS Quality of Care Measurement and
Monitoring Studies

A recently completed study, the Health Status at Discharge Project funded by HCFA, has developed measures of patient status at discharge, which can be constructed from data contained in patient medical records. In this study, the Northwest Oregon Health Systems developed a summary health status instrument that measures physical and mental status at time of hospital discharge in order to characterize patient dependency levels. The dependency scale was originally constructed with six items: activity, bathing, medications, procedures, symptoms, and age. Scores on these items are computed from information contained in the patient's medical records. In subsequent testing of the instrument, the age and medications items were dropped from the scale. The final version demonstrated high reliability among items in the instrument (alpha-r = .86) and among those administering the instrument (r=.92). This instrument was field tested at four hospitals in the Portland, Oregon, area in 1986 on a sample of 2,622 randomly selected medical abstracts drawn from three medical DRGS and two surgical DRGS. Results obtained with this instrument showed the level of patient dependency to be significantly higher in the post-PPS sample.

The National Center for Health Services Research and Health Care Technology Assessment (NCHSR&HCTA) is also supporting studies focusing on the assessment of patient care in subacute care settings. One study, Outcomes of Nursing Home Discharges, is examining the types of care and care outcomes of patients discharged from nursing homes. The study followed patients for 2 years after they were discharged, tracking their care requirements, including subsequent nursing home and hospital care. Data on patients' care needs and social support characteristics were collected. Researchers will attempt to use the data to define and refine a measure of a “continuing episode of long-term care." Study data also constitute a baseline against which the effects of PPS or other program or policy changes can be assessed. Preliminary analyses showed no changes in pre-PPS and post-PPS assessments of nursing home discharge outcomes, including the proportion dying in skilled nursing facilities, or in returns to the hospital. Comparisons among samples of admissions to nursing homes following hospitalization selected in 1980, 1982-1983, and 1984 showed a marked increase in Medicare admissions in the later samples, as well as modest increases in case-mix (a measure of the overall average diagnostic complexity of the patient populations). However, the changes were observed in the 1980 to 1982-1983 comparisons, as well as in the pre-PPS and post-PPS comparisons.

A second NCHSR&HCTA study is examining the impact of hospital discharge planning on patient outcomes by assessing systematic random

HHS Quality of Care Measurement and
Monitoring Studies

Assessing the Process of Care

samples of patients aged 60 or older after discharge from five acute care hospitals. Information concerning the provider and type of discharge planning is being analyzed to determine how it relates to meeting patient needs in terms of (1) medication and treatment, (2) nursing services and education, (3) rehabilitative care, (4) support services, and (5) continuity of physician care.

NCHSR&HCTA has been particularly involved in examining the process of
medical care and the appropriateness of medical decisionmaking. One
recent study, for example, assessed the effects of using a computerized
system for organizing medical records data in ambulatory care settings
(COSTAR; see chapter 6) on the quality of care provided in a general medi-
cal clinic. A randomized controlled clinical trial was conducted, in which
the control group used only the conventional paper record, while the
experimental group had access only to the COSTAR record. Six process
measures of quality were examined: patient satisfaction, record acquisi-
tion speed, record economy, staff acceptance, clinic flows, and compli-
ance with health screening. In addition, the study compared some
indicators of patient health, social service interventions, and feedback
of information to physicians. The study report has not yet been
completed.

An ongoing study, Appropriate and Inappropriate Use of Ancillary Ser-
vices, is testing criteria for the proper use of five diagnostic tests. The
study will attempt to determine whether these criteria sets are effective
in modifying physicians' test-ordering behaviors and in encouraging
more appropriate and economical use of these services. Another
NCHSR&HCTA study, Quality Differences Among Primary Care Practition-
ers, will use data from over 14,000 episodes of care provided to patients
in the Boston area over a 5-year period to examine how staff and practi-
tioner characteristics relate to how care is actually provided. Quality
will be measured by comparing the process of care to detailed diagnosis
and symptom-specific criteria sets developed in previous research. Spe-
cifically, the effect of practitioner gender and role (resident, staff physi-
cian, nurse practitioner) will be analyzed. The researchers plan to
examine issues such as whether a strong commitment to quality assur-
ance on the part of physicians in leadership roles actually results in
higher quality care or whether female practitioners give better care to
female patients. They will also attempt to identify staffing patterns
likely to provide higher quality care to certain types of patient
populations.

HHS Quality of Care Measurement and
Monitoring Studies

Studies Combining
Process and Outcomes
Data

Another NCHSR&HCTA-funded study, Evaluating Outcomes of Hospital Care Using Claims Data, will combine analysis of computerized billing data with examination of medical records. This study, like the Rand Nonintrusive Outcomes Study discussed below, is designed to evaluate the utility of claims data for identifying possible problems in patient outcomes. Two data sets, Medicare claims data and data collected by the Manitoba Health Commission, will be analyzed. In the first phase of the study, researchers will test hypotheses about the relationship between therapy and outcomes for a subset of conditions and procedures. In the second phase, researchers will validate outcomes for alternative approaches to prostatectomy.

Two studies conducted by the Rand Corporation are attempting to address both process and outcomes in assessing quality of care in the Medicare population. The Clinical Analysis of PPS Impacts on the Quality of Inpatient Medical Care study will produce nationally representative information on two outcome measures (mortality and morbidity) before and after the introduction of PPS. This study involves abstracting extensive information from medical records to determine whether there have been changes in the process of care that have affected the morbidity and mortality of Medicare patients. Expert physician panels are developing treatment standards for six disease categories: hip fracture, myocardial infarction, congestive heart failure, pneumonia, cerebrovascular accident, and depression. These consensus-based explicit standards will identify appropriate appraisal and treatment strategies for each of the six conditions. PPS impacts on quality of care can be assessed by comparing the actual treatment identified in the patient's medical records with the treatment standards established by the physician panels. This strategy is being currently applied to a sample of about 17,000 medical records drawn from hospitals in five states. The results of this study should provide information on the link between treatment performance and quality of care outcomes for the conditions being examined.

Research strategies that link process and outcome variables represent a
rigorous approach to reducing measurement error. However, this strat-
egy can rarely be implemented since there are no national-level data sys-
tems that incorporate information needed to identify process and
outcome indicators of quality. With this issue in mind, Rand has been
funded by HCFA to determine the adequacy of existing Medicare data
files for providing data on patient outcomes that are sensitive to varia-
tions in the quality of health care.