This audit was performed as part of the Treasury Inspector General for Tax Administration’s strategy to assess the relevance and reliability of the customer satisfaction performance measures as they relate to the Government Performance and Results Act of 1993 (GPRA). Our overall objective was to assess the validity of the information used to measure customer satisfaction in the Internal Revenue Service (IRS) Collection Field function (Collection). The scope of our audit consisted of reviewing how the information is obtained and used.

The GPRA requires executive agencies to establish standards for measuring the effectiveness of their performance and to prepare multi-year strategic plans, annual performance plans, and performance reports on prior year accomplishments. The first annual performance reports were to have been provided to the Congress by March 31, 2000. The GPRA allows the Congress to use the results of these reports to help determine the budget appropriations for the agencies. Therefore, it is essential that the IRS ensures all of its performance data are valid.

The current IRS strategic plan establishes three strategic goals: service to each taxpayer, service to all taxpayers, and productivity through a quality work environment. Service to each taxpayer is measured by customer satisfaction, which is determined through surveys conducted by an outside vendor.

The Collection survey uses a sample of only unpaid accounts (taxpayer delinquent accounts) taken for the Collection Quality Measurement System (CQMS). The IRS planned to begin submitting data from the Integrated Collection System (ICS) to the vendor for sample selection in February 2000.

IRS management did not ensure that a case selection process was put in place that would provide an accurate measurement of customer satisfaction for Collection customers. As a result, the IRS needs to qualify any of the data from the Collection Customer Satisfaction Surveys that will be presented in the Fiscal Year (FY) 1999 Annual Performance Report. This information is not reliable for use in establishing baselines for future year goals. The data are not a valid measurement of customer satisfaction because major categories of Collection taxpayer contacts were not in the universe sampled for the survey, non-respondents were not surveyed, and the district samples were not correctly taken. In addition, the data do not readily correlate to the IRS’ future business units.

Collection decided to use the sample selected for quality for the CQMS as the sample for the Customer Satisfaction Surveys. However, the CQMS sample methodology excludes up to 40 percent of the universe of taxpayers contacted by the Collection function from consideration for the Customer Satisfaction Survey. The CQMS review is designed to evaluate how effectively closed balance due cases are worked. The IRS FY 2000 Performance Plan reports that Collection is to collect unpaid tax accounts, secure delinquent returns, and assist taxpayers in resolving tax account problems. The current Customer Satisfaction Survey includes only taxpayers contacted for unpaid tax accounts and excludes those who had a delinquent return or resolved an account problem through other means, such as an Offer in Compromise. Collection plans to have the vendor begin selecting the sample from the ICS database beginning in Calendar Year 2000. The ICS is an automated system used to document and process all aspects of Collection cases.

Only 26 percent of the taxpayers receiving Customer Satisfaction Survey questionnaires responded. The Office of Management and Budget and the IRS require at least a 70 percent response rate to surveys to better ensure the accuracy of the results. With such a low response rate, the IRS must be careful when presenting the survey results, as the opinions of non-respondents often do not match those of the respondents.

The districts were not using a random sample to select cases for the CQMS review. In some districts, employees either selected the first nine cases from each branch or used their personal judgment to select the sample. We also found that some districts are not always submitting the required nine cases for the weekly sample. We obtained the number of cases each district was required to provide and the actual number of cases each submitted for the period March 1, 1999, to September 30, 1999, from the Centralized CQMS Site. None of the districts submitted the required sample size over this period and, nationally, the number submitted was approximately 42 percent less than required.

The IRS cannot take existing Customer Satisfaction Survey data for Collection and relate it to its future four business units. Although the vendor has been capturing information relating to market segmentation, the categories, as reported, do not all correspond to the IRS’ definition of its future four business units.

We recommend that the Assistant Commissioner (Collection) and the Director, Office of Program Evaluation and Risk Analysis (OPERA), implement the following recommendations and, working in conjunction with the Director, Strategic Planning and Budgeting, ensure disclosure of survey limitations for purposes of the GPRA. The Annual Performance Report should clearly state the limitations in the sampling procedures and the resultant effect on Collection’s ability to report customer satisfaction. The Annual Performance Report also should inform the Congress about the change from the CQMS database to the ICS database as the source for sample selection and the time needed for this action to result in valid measurement of overall customer satisfaction.

The Assistant Commissioner and the Director, OPERA, should explore other avenues to increase the response rate to the mailed-out questionnaires from the current 26 percent to the required 70 percent. One possible approach would be to contract with a vendor to conduct a telephone survey of a random statistical sample of taxpayers not responding to the mail survey.

The Assistant Commissioner also should put procedures in place that will allow the measurement of customer satisfaction for each future business unit.

Management’s Response: Management’s response was due on June 14, 2000. As of June 27, 2000, management had not responded to the draft report.