Peer Review Organizations medical records by trained health professionals. There is, however, practically no information available to document how all these review efforts are working. In particular, it is not known whether PROS are effectively identifying and correcting quality of care problems and changing physician and hospital behavior in ways that raise overall levels of quality of care. The wide variation in PRO quality objectives and review criteria and the systems PROS have developed for addressing suspected problems provide an opportunity to learn more about which methods are most effective. To do so, comparative analyses of the systems in place are essential. The need for such assessments is underscored by the wide divergence in the incidence of quality problems identified by PROS. The PROMPTS-2 and SuperPRO systems for reviewing these quality assessment activities do little to generate information on how well these methods work. HCFA has not encouraged the SuperPRO to analyze the comparative effectiveness of individual PRO methods for identifying quality problems and has not designed data reporting systems that facilitate this type of analysis. (See chapter 5.) Both the SuperPRO and PROMPTS-2 reviews focus on whether, once a case is selected for review, it is reviewed correctly, rather than on whether potential quality problems are being selected for review. In part, this reflects the dual role of the PROS, who are responsible for both controlling unnecessary or inappropriate utilization of Medicare services and assessing quality of care. As the system is currently designed, the majority of cases selected for review are those targeted because of utilization concerns; for these cases, quality review is performed as a collateral activity. If PROS are to be effective in their role as quality of care reviewers, however, HCFA needs to know how to structure the selection and review of cases to maximize the efficiency of PRO quality reviews. The advantages and disadvantages of methods to target quality reviews also need to be assessed. As is discussed in chapter 3, carriers and intermediaries use computer screens and profiling techniques to identify coverage and utilization problems, but not to target specific instances of substandard care. PROS build their own profiles from case review data. Intramurally, HCFA has devoted substantial resources to develop methods to identify aberrant patterns of patient mortality following hospitalization. Yet, despite these efforts, little is being done to systematically explore whether there are ways to integrate these varied approaches to better target cases with suspected quality problems. The expansion of PRO review to Medicare HMOs and CMPS reinforces the need for systematic evaluation. Because these reviews may involve Peer Review Organizations Recommendations extensive examination of ambulatory care, the current PRO review methods cannot be simply transferred to review care provided in prepaid settings. These reviews are just beginning, and HCFA has the opportunity to assess how well the new ambulatory review systems actually work. An adequate evaluation plan must allow for comparisons among the systems, including comparisons of their ability to identify quality problems and to improve the processes and outcomes of patient care over time. We recommend that the Secretary of HHS direct the Administrator of HCFA to fund additional studies to analyze the comparative effectiveness of particular PRO review methods, and the utility of current methods for establishing PRO quality objectives. These analyses should include assessments of whether different written review criteria or protocols generate significantly different rates of problems identified, and whether the identification of problems using these methods leads to significant changes in the incidence of quality problems over time. We recommend that the Secretary of HHS direct the Administrator of HCFA to initiate studies to assess the strengths and weaknesses of the current assignment of responsibilities among carriers, intermediaries and PROS with respect to processing and screening Medicare claims data and performing medical reviews to identify quality of care problems and substandard providers and suppliers. These studies should specifically examine whether a realignment of responsibilities could improve the efficiency and effectiveness of Medicare quality review activities. We recommend that the Secretary of HHS direct the Administrator of HCFA to develop comparative information on the effectiveness of the quality review methods used by the peer review organizations reviewing quality of care in Medicare HMOS and CMPS. These studies should also produce comparative information on the overall levels of quality of care provided in the participating HMOS and CMPS. This would require the collection of standard information on the use of services and health care outcomes across plans. Chapter 5 Existing Data Resources for Assessing Quality of Care Contractor-Generated The Sources of the Data As described earlier, Medicare review activities have not been designed The usefulness of Medicare administrative data for measuring quality of The basic sources of patient-level information for monitoring Medicare services are billing data and Medicare enrollment and eligibility data that carriers and intermediaries can access from central HCFA files. Medicare billing data, however, have gaps that limit their utility for measuring the structure, process, and outcomes of care. Most of the information maintained by carriers and intermediaries is limited to what is submitted on claim forms and information received in response to queries to HCFA to check on beneficiary eligibility or deductible status. Only in the rare instances of full-scale reviews (described in chapter 3) do carriers obtain medical records and other supporting information. Intermediaries limit their data collection to information from claims and queries, except for medical reviews of skilled nursing facilities, hospices, and home health agency claims, which may require review of medical records. The initial data source for PRO Medicare reviews is the Unibill file of Medicare bills generated by intermediaries. After verifying that interim bills have been excluded from the data files, and inaccurate and incomplete records have been corrected, PROS base their reviews on actual dis Existing Data Resources for Assessing Data Relevant to Quality of Care Assessment charge bills for hospitals that are covered by the Medicare prospective payment system. PROS also review a sample of bills from PPSexempt hospitals. PROS may obtain other relevant data from intermediaries or other sources as needed, but they are prohibited from collecting or having others collect for them any information that duplicates that which HCFA requires intermediaries to collect. PROS may, for example, obtain data from HCFA or Social Security beneficiary files to verify posthospital mortality data. Inpatient bills submitted to intermediaries contain some information on patient outcomes, primarily patient discharge status indicating, for example, whether the patient died in the hospital or was discharged to a subacute care facility (see appendix V), and date of death. Billing files accumulated over time also provide information about previous use of Medicare inpatient services, which allows the computation of eligibility, copayment, and deductible requirements over a Medicare benefit period. Thus, outcome measures for inpatient services that can be derived from billing data are mortality, and to a limited extent, morbidity (as indicated by readmissions and some inpatient diagnoses) and disability (as indicated by admission to rehabilitative subacute care). These measures are discussed in greater detail in chapter 6. Obtaining accurate data for these relatively crude outcome measures may require linking information from several sources. Diagnostic information, which can indicate negative outcomes as well as provide the information required for many analyses of the process of care, is reported on part A billing forms. Although part B billing forms include space to indicate the patient diagnosis for which the services were given, HCFA does not require claims processors to enter the diagnostic information on manually submitted forms onto computerized billing files, and no diagnostic data are included in billing files submitted to HCFA by carriers.' The only relevant outcome information available on part B billing forms are procedure codes, some of which may indicate possible negative patient outcomes, such as procedures used in treating specific postoperative problems.2 'This is apparently because for cases where patients submit bills directly to the carrier (that is, when physicians do not accept assignment), HCFA believes the diagnostic information may be unreliable. This issue is discussed below. 2For example, codes 52606 and 52650 are used to indicate procedures used to treat postoperative complications (bleeding and infection) of transurethral prostate surgery. Existing Data Resources for Assessing Accuracy and However, as we discuss in appendix III, HCFA does require that claims submitted directly from providers to carriers through the electronic media claims system include ICD-9-CM codes. Further, some carriers have already developed procedures for coding diagnostic information on manually submitted paper claims as part of their self-initiated utilization review systems. Some carriers using diagnostic data report that these screening activities are cost-effective. Diagnoses can, for example, be linked with procedures data to detect inappropriate use of diagnostic tests. Conversely, diagnoses without the appropriate tests could indicate inadequate care. Thus, there is the potential for expanding the collection of diagnostic data on part B bills and strengthening the availability of information to monitor both processes and outcomes of care. There is more information available on process than outcome indicators of quality. Inpatient part A billing forms contain information on surgical procedures and services provided to the beneficiary; outpatient part B bills contain information on types of visits and procedures performed. This information is reported using standardized ICD-9-CM diagnosis and procedure codes for part A inpatient admissions and HCFA common procedures codes for part B services. Bills contain very little information on the structural characteristics of Medicare providers. However, bills always contain provider numbers, which would allow billing information to be linked with provider information such as that contained in the provider of services file maintained by HCFA (see below). Statistical profiles of providers (such as utilization patterns and charges) can be linked to individual beneficiary records. Contractors use three methods to check the accuracy of data submitted to them: edits, screens, and reviews. Extensive data edits check primarily for completeness and consistency; they are discussed below. Data accuracy may also be checked as a byproduct of the various screening and review activities discussed in chapter 3. Carriers and intermediaries apply consistency edits to ensure that all required fields on the Medicare billing forms contain numbers or letters in the appropriate ranges. This step ensures that the required information is complete and appears to be appropriate. In addition, all part A 3A computer edit program looks for invalid data or inconsistencies in each bill. For example, a month greater than 12 or more than 60 lifetime reserve days used are invalid data items. However, consistency edits do not compare data on the bill with information in other HCFA records but only check for consistency of the data in each bill. |