This audit was performed as part of the Treasury Inspector General for Tax Administration’s overall strategy to assess the reliability of the Internal Revenue Service’s (IRS) customer (taxpayer) satisfaction performance measures as they relate to the Government Performance and Results Act of 1993 (GPRA). The GPRA is intended to improve agency performance and provide objective information to congressional and executive branch decision-makers to assist them in appropriating and allocating federal funds. The law requires executive agencies to prepare multi-year strategic plans, annual performance plans, and performance reports on prior year accomplishments.

The IRS established balanced performance measures to support achievement of its strategic goals: provide quality service to each taxpayer, serve all taxpayers, and be productive through a quality work environment. Achievement of these goals is measured through customer satisfaction, employee satisfaction, and business results. Customer satisfaction surveys are used to develop performance measures that balance customer needs with IRS operational needs. Taxpayers that receive specific kinds of services from the IRS might be asked to rate the service. These survey results are being summarized by a vendor and used by the IRS to evaluate the overall satisfaction with IRS services.

The IRS originally designed the surveys as part of its balanced performance management system and did not anticipate using the surveys for GPRA reporting purposes. Subsequently, the IRS used the survey results to fulfill the GPRA reporting requirements. The intent of the GPRA is that the Congress will use the performance measurement results to help evaluate IRS budget appropriations. Therefore, it is essential that the IRS accurately measure its success in meeting the performance goals.

This review evaluated the accuracy, validity, and reliability of the information used by the IRS to measure customer satisfaction with the Automated Collection System (ACS). The ACS is a computerized inventory of customers who owe taxes or have not filed returns, and represents a significant part of IRS activities for providing customer service. Customer representatives make telephone calls to customers on the ACS inventory, as well as answer telephone calls from these customers. Each day, Customer Service employees ask a number of ACS customers to take the survey, which measures customer satisfaction with the service provided by ACS employees.

The present results of the ACS Customer Satisfaction Survey should not be used to satisfy all requirements of the GPRA, unless appropriately qualified. The original purpose of the ACS Customer Satisfaction Survey was not for GPRA reporting, but to measure the overall trend of taxpayer satisfaction with ACS service. While the survey may be an effective marketing tool to gauge taxpayers’ satisfaction with the services provided by the ACS offices, a higher standard is required when these results are used for GPRA reporting purposes. Internal controls should be in place to ensure the survey results are reliable and that the data used for GPRA reporting purposes has been verified and validated.

IRS executives have not established a management process to ensure the survey is conducted appropriately to measure the level of satisfaction all customers receive from interactions with ACS employees. There is little accountability over the survey, and few management controls are in place to ensure the survey results are reliable and can be verified and validated. Without reliable information, the IRS cannot provide a basis for comparing program results with the established performance goals and will not achieve the benefits of the GPRA intended by the Congress.

We conducted testing in the National Office and three ACS call sites. We discussed GPRA measurements and requirements with National Office managers. We also interviewed Customer Service managers and employees, reviewed telephone system documentation to determine how ACS personnel select customers for the survey, and observed Customer Service employee selection and monitoring of 29 telephone calls for the survey.

In addition to our testing, we consulted with an expert statistician to solicit an independent opinion regarding the results of the survey. We asked the statistician to focus on the procedures and practices used to select survey participants and determine if they produced a random sample suitable for projecting an overall customer satisfaction rate for GPRA purposes.

The current management control process is not adequate to ensure the ACS Customer Satisfaction Survey is administered properly. Specifically:

• More accountability is needed for the ACS Customer Satisfaction Survey results. Three different IRS offices currently provide oversight for the surveys, but none of them appear accountable for the survey results.
• More management controls are needed to ensure the integrity and reliability of the survey results. The ACS Customer Satisfaction Survey sample selection methodology excludes specific customers who interact with the ACS Program: customers telephoned by customer representatives, customers who telephone ACS call sites after surveyors’ regular work hours (generally after 5 p.m.), and some customers who do not speak English.
• More controls are needed to provide all ACS customers with the opportunity to be included in the survey. The ACS Customer Satisfaction Survey sample selection techniques are a source of potential bias. The instructions for sample selection are not clear and are not always followed when selecting customers for the survey.

We believe that without reliable survey results, the IRS cannot meet the GPRA requirements of establishing measurable performance goals and reporting of accurate results.

To ensure that the IRS accurately measures the level of satisfaction customers receive from interactions with ACS services, additional actions are needed. We recommend that the Director, Strategic Planning and Budget, consider establishing a process to oversee the ACS Customer Satisfaction Survey. This will ensure compliance with survey selection criteria and procedures. We also recommend that the survey selection criteria be revised to ensure that all ACS customers are given an equal chance of being selected for the survey.

Management’s Response: Management’s response was due on May 8, 2000. As of May 11, 2000, management had not responded to this draft report.