The Government Performance and Results Act of 1993 (GPRA) requires all federal agencies to develop annual performance plans that must include indicators to measure progress towards their annual goals. These indicators are intended to inform the President, the Congress, and other interested parties of the expected level of achievement for a program or activity. The Internal Revenue Service (IRS) developed GPRA indicators to report on four categories related to its toll-free telephone service: quality of calls, quantity of calls, customer satisfaction, and employee satisfaction. This review concentrates only on those indicators related to the quantity of calls.

In 1998, the IRS placed a renewed emphasis on customer service by revising its mission statement to include providing top quality service by helping America’s taxpayers to understand and meet their tax responsibilities. The IRS Commissioner has defined top quality service to include improving access to the IRS’ toll-free telephone system, which is a cornerstone of its customer service operations. In support of its mission, the IRS set two goals for toll-free telephone service in Fiscal Year (FY) 2001:

• "Increase ability to meet customer account service expectations."
• "Meet taxpayer demands for better traditional assistance services."

The FY 2001 budget for providing toll-free telephone services is $385 million for salaries and benefits alone. This represents an increase of $39 million over the FY 2000 budget.

Our audit objective was to determine whether the IRS’ annual performance plans for the quantity measurement of toll-free telephone service provide an effective framework to communicate the progress towards meeting strategic goals and objectives in compliance with the GPRA requirements.

The IRS has designated two GPRA quantity indicators for toll-free telephone service. They are Level of Service (LOS), which gives the percentage of total calls attempted that are answered, and Toll-Free Telephone Service Workload, which tells the total calls answered broken down by automated calls and those answered by a Customer Service Representative (CSR). While these indicators provide some useful information, they do not fully comply with guidance for implementing the GPRA provided by the Office of Management and Budget (OMB). The GPRA indicators should facilitate the assessment of whether performance goals, in this case meeting taxpayer expectations and demands for toll-free telephone service, have been achieved. However, the Workload indicator does not address the goals and the LOS indicator addresses only a portion of the goals – access to the system. Furthermore, neither of these indicators ties cost to programs as the GPRA requires agencies to do. Without better information on the actual taxpayer experience and the associated costs, the usefulness of the current GPRA quantity indicators to the Congress and IRS management is diminished.

Neither of the two GPRA quantity indicators being used by the IRS for toll-free telephone service addresses the true taxpayer experience (i.e., how long taxpayers wait to receive assistance), nor do they measure the cost of providing toll-free telephone service. The OMB guidelines for implementing the GPRA state that plans should not "…skimp on what is measured, resulting in a narrowly-drawn or fragmented picture of performance." In addition, they emphasize that "Agencies should strive to include goals or indicators for unit cost, even if only approximate costs can be estimated." Cost data is essential for both the Congress and IRS management to make effective cost decisions about the toll-free telephone program.

Wage and Investment (W&I) Division management recently recommended to the IRS Commissioner the use of an indicator that describes the percentage of calls answered within a prescribed time period (i.e., 30 seconds). This is similar to an indicator that is commonly used by call centers in the private sector. While competitive industries tend to have goals for answering calls within a 15 to 30 second range, a more common goal for government organizations is 2 to 5 minutes. According to one guide on call center management, this measure ties the resources needed to get the desired results as well as gives the clearest indication of what callers experience when they attempt to call an organization. While this indicator would greatly improve the information provided about the IRS’ toll-free service, this replacement may still not provide GPRA quantity indicators that reflect:

• Separate measures of service provided to taxpayers via automated service versus service provided by CSRs.
• The cost of providing each of these two services.

The length of time that callers wait until assistance is received is less for automated calls than for CSR calls. Including both of these types of calls into one measure would present an inaccurate picture of the length of time that taxpayers are required to wait to speak with a CSR.

To provide both IRS management and the Congress with useful information to assess the progress of the toll-free telephone system in meeting the IRS’ mission and to assist in budget decisions, the GPRA quantity indicators should be either revised or expanded to include the above information. Additionally, the proposed replacement indicator should include a time standard that more realistically reflects the expectations of government agencies.

The GPRA indicators used by the IRS to report on the quantity of toll-free telephone services do not include data from the same toll-free telephone lines. The LOS indicator is based on five of the IRS’ toll-free lines, while the workload indicators for "total calls answered" include several of the other toll-free lines as well as administrative lines. However, the Customer Account Services (CAS) functions from the W&I and Small Business/Self Employed (SB/SE) Divisions have managerial and budgetary responsibility for only the three main toll-free telephone lines which they refer to as the customer service lines.

Including the additional toll-free lines in the GPRA indicators distorts the true picture of the taxpayer experience in attempting to receive detailed information regarding customer account services. For example, as of March 24, 2001, including the two additional toll-free lines in the LOS calculation had increased that indicator by three percentage points. Furthermore, using additional toll-free lines in these GPRA quantity indicators erodes the connection between budget and performance that is an important aspect of the GPRA process.

The CAS Directors in the W&I and SB/SE Divisions need to develop GPRA quantity indicators for their toll-free telephone service that provide an accurate representation of the taxpayer experience and include the cost of providing that experience. In addition, these indicators should be consistently reported to include only the three main customer service toll-free telephone lines and should provide separate data for automated services and services which involve interaction with IRS CSRs.

Management’s Response: The IRS agreed to some changes to its GPRA quantity indicators for the toll-free telephone system, but disagreed with some aspects of the report. It will revise its indicators to include only the three main toll-free telephone lines. In addition, it is developing a service-level indicator that includes a time component. The IRS has also developed a new formula for its LOS measure which it has renamed Assistor LOS. However, this measure still includes some categories of automated calls.

The IRS did not specifically address whether it would implement our recommendation to have a separate service-level indicator for automated calls, or our recommendation on providing a cost-per-call indicator, although it did state that program costs can be readily related to quantity measures.

Office of Audit Comment: The Office of Audit will continue to work with the IRS to identify the actions and schedule for improving the GPRA quantity indicators. The IRS’ response provided some indication of the planned actions; however, further clarity is needed for specific recommendations.