Better GPRA Quantity Indicators Are Needed for Toll-Free Telephone Service

August 2001

Reference Number: 2001-30-131

 

This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.

August 14, 2001

MEMORANDUM FOR COMMISSIONER, WAGE AND INVESTMENT DIVISION

COMMISSIONER, SMALL BUSINESS/SELF-EMPLOYED

DIVISION

FROM: Pamela J. Gardiner /s/ Pamela J. Gardiner

Deputy Inspector General for Audit

SUBJECT: Final Report - Better GPRA Quantity Indicators Are Needed for Toll-Free Telephone Service

This report presents our review of the Government Performance and Results Act of 1993 (GPRA) quantity indicators that the Internal Revenue Service (IRS) uses to report on its toll-free telephone service. Our objective was to evaluate the effectiveness of the IRS’ performance plans in communicating progress towards meeting strategic goals and objectives.

In summary, the two GPRA indicators used for reporting on the quantity aspect of toll-free telephone service do not provide sufficient information to IRS management and the Congress for effective decision-making. In addition, these indicators do not address the true taxpayer experience in attempting to call the IRS’ toll-free lines, nor do they address the cost of providing toll-free telephone service. To better comply with the GPRA, the IRS should develop GPRA quantity indicators for the toll-free telephone system that better reflect the taxpayer experience by including: a more realistic time standard for a government organization during which calls should be answered, separate statistics for automated calls and those answered by Customer Service Representatives, and a cost-per-call measure. In addition, these indicators should be confined to the three main toll-free telephone lines which taxpayers use for account information and tax law questions.

The IRS agreed to make some changes to its GPRA quantity indicators for toll-free telephone service. However, the IRS neither designated a responsible management official for carrying out these actions, nor established target completion dates in its response to the draft report. While the IRS agreed with our recommendation to establish a time standard for answering calls, it disagreed with the portion of that recommendation relating to making the standard more reflective of a government organization than private industry. The IRS agreed to revise its indicators to include only the three main toll-free telephone lines. However, the IRS did not address whether it would develop a GPRA service-level indicator for automated calls or a cost-per-call indicator. In addition, we are concerned that the new service-level indicator for assistor calls continues to include some automated calls. We have included both the IRS’ response and our comments regarding the response in the main body of this report. The full text of the IRS’ comments is included as an appendix.

Copies of this report are being sent to the IRS managers who are affected by the report recommendations. Please contact me at (202) 622-6510 if you have questions or Gordon C. Milbourn III, Assistant Inspector General for Audit (Small Business and Corporate Programs), at (202) 622-3837.

Table of Contents

Executive Summary

Objective and Scope

Background

Results

Improved GPRA Quantity Indicators Are Needed for Effective Planning and Budget Allocation for the Toll-Free Telephone System

GPRA Quantity Indicators Should Be Revised to Include Only the Three Main Toll-Free Telephone Lines

Conclusion

Appendix I – Detailed Objective, Scope, and Methodology

Appendix II – Major Contributors to This Report

Appendix III – Report Distribution List

Appendix IV – Management’s Response to the Draft Report

Executive Summary

The Government Performance and Results Act of 1993 (GPRA) requires all federal agencies to develop annual performance plans that must include indicators to measure progress towards their annual goals. These indicators are intended to inform the President, the Congress, and other interested parties of the expected level of achievement for a program or activity. The Internal Revenue Service (IRS) developed GPRA indicators to report on four categories related to its toll-free telephone service: quality of calls, quantity of calls, customer satisfaction, and employee satisfaction. This review concentrates only on those indicators related to the quantity of calls.

In 1998, the IRS placed a renewed emphasis on customer service by revising its mission statement to include providing top quality service by helping America’s taxpayers to understand and meet their tax responsibilities. The IRS Commissioner has defined top quality service to include improving access to the IRS’ toll-free telephone system, which is a cornerstone of its customer service operations. In support of its mission, the IRS set two goals for toll-free telephone service in Fiscal Year (FY) 2001:

The FY 2001 budget for providing toll-free telephone services is $385 million for salaries and benefits alone. This represents an increase of $39 million over the FY 2000 budget.

Our audit objective was to determine whether the IRS’ annual performance plans for the quantity measurement of toll-free telephone service provide an effective framework to communicate the progress towards meeting strategic goals and objectives in compliance with the GPRA requirements.

Results

The IRS has designated two GPRA quantity indicators for toll-free telephone service. They are Level of Service (LOS), which gives the percentage of total calls attempted that are answered, and Toll-Free Telephone Service Workload, which tells the total calls answered broken down by automated calls and those answered by a Customer Service Representative (CSR). While these indicators provide some useful information, they do not fully comply with guidance for implementing the GPRA provided by the Office of Management and Budget (OMB). The GPRA indicators should facilitate the assessment of whether performance goals, in this case meeting taxpayer expectations and demands for toll-free telephone service, have been achieved. However, the Workload indicator does not address the goals and the LOS indicator addresses only a portion of the goals – access to the system. Furthermore, neither of these indicators ties cost to programs as the GPRA requires agencies to do. Without better information on the actual taxpayer experience and the associated costs, the usefulness of the current GPRA quantity indicators to the Congress and IRS management is diminished.

Improved GPRA Quantity Indicators Are Needed for Effective Planning and Budget Allocation for the Toll-Free Telephone System

Neither of the two GPRA quantity indicators being used by the IRS for toll-free telephone service addresses the true taxpayer experience (i.e., how long taxpayers wait to receive assistance), nor do they measure the cost of providing toll-free telephone service. The OMB guidelines for implementing the GPRA state that plans should not "…skimp on what is measured, resulting in a narrowly-drawn or fragmented picture of performance." In addition, they emphasize that "Agencies should strive to include goals or indicators for unit cost, even if only approximate costs can be estimated." Cost data is essential for both the Congress and IRS management to make effective cost decisions about the toll-free telephone program.

Wage and Investment (W&I) Division management recently recommended to the IRS Commissioner the use of an indicator that describes the percentage of calls answered within a prescribed time period (i.e., 30 seconds). This is similar to an indicator that is commonly used by call centers in the private sector. While competitive industries tend to have goals for answering calls within a 15 to 30 second range, a more common goal for government organizations is 2 to 5 minutes. According to one guide on call center management, this measure ties the resources needed to get the desired results as well as gives the clearest indication of what callers experience when they attempt to call an organization. While this indicator would greatly improve the information provided about the IRS’ toll-free service, this replacement may still not provide GPRA quantity indicators that reflect:

The length of time that callers wait until assistance is received is less for automated calls than for CSR calls. Including both of these types of calls into one measure would present an inaccurate picture of the length of time that taxpayers are required to wait to speak with a CSR.

To provide both IRS management and the Congress with useful information to assess the progress of the toll-free telephone system in meeting the IRS’ mission and to assist in budget decisions, the GPRA quantity indicators should be either revised or expanded to include the above information. Additionally, the proposed replacement indicator should include a time standard that more realistically reflects the expectations of government agencies.

GPRA Quantity Indicators Should Be Revised to Include Only the Three Main Toll-Free Telephone Lines

The GPRA indicators used by the IRS to report on the quantity of toll-free telephone services do not include data from the same toll-free telephone lines. The LOS indicator is based on five of the IRS’ toll-free lines, while the workload indicators for "total calls answered" include several of the other toll-free lines as well as administrative lines. However, the Customer Account Services (CAS) functions from the W&I and Small Business/Self Employed (SB/SE) Divisions have managerial and budgetary responsibility for only the three main toll-free telephone lines which they refer to as the customer service lines.

Including the additional toll-free lines in the GPRA indicators distorts the true picture of the taxpayer experience in attempting to receive detailed information regarding customer account services. For example, as of March 24, 2001, including the two additional toll-free lines in the LOS calculation had increased that indicator by three percentage points. Furthermore, using additional toll-free lines in these GPRA quantity indicators erodes the connection between budget and performance that is an important aspect of the GPRA process.

Summary of Recommendations

The CAS Directors in the W&I and SB/SE Divisions need to develop GPRA quantity indicators for their toll-free telephone service that provide an accurate representation of the taxpayer experience and include the cost of providing that experience. In addition, these indicators should be consistently reported to include only the three main customer service toll-free telephone lines and should provide separate data for automated services and services which involve interaction with IRS CSRs.

Management’s Response: The IRS agreed to some changes to its GPRA quantity indicators for the toll-free telephone system, but disagreed with some aspects of the report. It will revise its indicators to include only the three main toll-free telephone lines. In addition, it is developing a service-level indicator that includes a time component. The IRS has also developed a new formula for its LOS measure which it has renamed Assistor LOS. However, this measure still includes some categories of automated calls.

The IRS did not specifically address whether it would implement our recommendation to have a separate service-level indicator for automated calls, or our recommendation on providing a cost-per-call indicator, although it did state that program costs can be readily related to quantity measures.

Office of Audit Comment: The Office of Audit will continue to work with the IRS to identify the actions and schedule for improving the GPRA quantity indicators. The IRS’ response provided some indication of the planned actions; however, further clarity is needed for specific recommendations.

Objective and Scope

This review was performed as part of our Fiscal Year (FY) 2001 emphasis area focusing on customer service activities. Our overall objective was to determine whether the Internal Revenue Service’s (IRS) annual performance plans for measuring the quantity of toll-free telephone service provide an effective framework to communicate its progress towards meeting strategic goals and objectives as required by the Government Performance and Results Act of 1993 (GPRA).

To assess the quantity indicators for the IRS’ toll-free telephone service, we reviewed documentation of its strategic planning efforts, reviewed documentation of industry best practices, and interviewed IRS management officials responsible for the toll-free telephone operations. We did not review the accuracy of the IRS’ reported data nor did we review any procedures that the IRS may have in place to verify its data.

We performed work at the Wage and Investment (W&I) Division’s Joint Operations Center in Atlanta, Georgia, and at the W&I Division’s and Small Business/Self-Employed (SB/SE) Division’s offices in New Carrollton, Maryland. Our review was conducted between December 2000 and April 2001 and was performed in accordance with Government Auditing Standards.

Details of our objective, scope, and methodology are presented in Appendix I. Major contributors to this report are listed in Appendix II.

Background

The Congress enacted the GPRA to assist in ensuring that the federal government delivers better results to its taxpayers. Information developed under the GPRA is also intended to help the Congress in decision making.

To assist federal agencies in meeting their missions, the GPRA requires them to prepare multi-year strategic plans for how they will deliver high-quality products and services to the American public. These plans must include a mission statement and be supported by annual performance plans that set goals for the fiscal year that will help to achieve long-term strategic goals. The annual plans must also include indicators to measure progress towards the annual goals. According to the GPRA, a major purpose of these indicators is to inform the President, the Congress, and other interested parties of the expected level of achievement for the program or activity.

The IRS’ mission was revised in 1998 to place a greater emphasis on customer service. This new mission is to "Provide America’s taxpayers top quality service by helping them understand and meet their tax responsibilities and by applying the tax law with integrity and fairness to all." The IRS’ toll-free telephone system is a cornerstone of its customer service operations, and the IRS Commissioner has defined top quality service to include improving access to this system.

To support its mission, the IRS established the following two FY 2001 goals for the toll-free telephone system:

The IRS has developed GPRA indicators to report on four aspects of its toll-free telephone service: quality of calls, quantity of calls, customer satisfaction, and employee satisfaction. Our review concentrated only on those indicators that address the quantity of calls.

The IRS uses two GPRA indicators for reporting on the quantity measure of its toll-free telephone services. The first of these is level of service (LOS), which is the percentage of total calls attempted that are answered. The second is Toll-Free Telephone Service Workload which is comprised of the following two components:

Despite the IRS’ mission and the goals of the divisions providing the toll-free telephone service, customers are not receiving good toll-free telephone service. In the IRS Commissioner’s April 3, 2001, testimony before the Subcommittee on Oversight of the House Committee on Ways and Means, he stated that about a third of the calls do not get through and the IRS’ current level of telephone service is "unacceptable."

The IRS recognizes the need to further improve toll-free telephone service, and some progress has occurred in the past year. A change in the telephone system technology now allows more calls to be initially sent to automated services through TeleTax. This enhancement resulted in over six million more calls being answered in FY 2001 through March 24, 2001, when compared to the same time period for FY 2000. The following table provides a comparison of the service provided on the TeleTax line and the three customer account service toll-free telephone lines. The comparison covers the first 6 months of FY 2000 and FY 2001.

Toll-Free Telephone Service on TeleTax and the Three Customer Account Service Lines through March 24 of Each Fiscal Year

 

FY 2000

FY 2001

1. Call Attempts to the 3 Customer Account Service Lines

41,246,681

29,313,894

2. Less: Busy Signals

8,993,451

5,899,744

3. Calls Available to Answer

32,253,230

23,414,150

4. Less: Abandoned Calls

7,351,420

6,506,189

5. Total Calls Answered by
the 3 Customer Account
Service Lines

24,901,810

16,907,961

6. Less: Automated Calls
Answered

8,880,084

2,149,511

7. Customer Service
Representative Calls
Answered

16,021,726

14,758,450

8. LOS on the 3 Customer
Account Service Lines
(line 5 divided by line 1)

60.37%

57.68%

9. TeleTax Calls Answered

21,559,691

35,843,177

10. Total Calls Answered
(line 5 plus line 9)

46,461,501

52,751,138

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Source: IRS Weekly Snapshot Report, March 24, 2001

This table shows that Customer Service Representatives (CSRs) answered almost 1.3 million fewer calls in FY 2001 than during the same time period in FY 2000. This may have occurred as a result of the less complex calls previously handled by CSRs being diverted to automated services.

Results

The GPRA quantity indicators used by the IRS for its toll-free telephone service do not, by themselves, provide sufficient information to determine whether the IRS is accomplishing its mission to provide taxpayers with top quality service. They also do not fully comply with the guidance for implementing the GPRA provided by the Office of Management and Budget (OMB). The most critical areas where information is lacking are:

The current IRS GPRA quantity indicators for toll-free telephone service do provide some useful information. For example, the LOS compares the number of customers who received an answer from a CSR, or through automation, to the total number of customers who tried to call the IRS. The Toll-Free Telephone Service Workload measure can be used as a basis for comparison with the number of calls answered in prior fiscal years.

However, the GPRA indicators should facilitate an assessment of whether performance goals, in this case meeting taxpayer expectations and demands for toll-free telephone service, have been achieved. Toll-Free Telephone Service Workload does not address these goals, and LOS only addresses a portion (i.e., access to the system) of them. Furthermore, neither of these indicators implements OMB’s guidance to tie cost to programs. Without better information on the actual taxpayer experience and associated cost data, the usefulness of these GPRA indicators to the Congress, as well as IRS management, is diminished.

The IRS has already recognized some of the weaknesses with its current GPRA quantity indicators. In a briefing for the IRS Commissioner on January 17, 2001, W&I Division management recommended eliminating LOS as an indicator in FY 2002 and establishing new indicators, which align with their strategic direction and goals. Seven new quantity indicators, some containing as many as three separate calculations, were proposed. However, none of the proposed indicators includes a cost component.

Improved GPRA Quantity Indicators Are Needed for Effective Planning and Budget Allocation for the Toll-Free Telephone System

The LOS indicator traditionally used by the IRS to assess the quantity of service provided to taxpayers on its toll-free telephone system does not provide a useful measure of the actual service received by taxpayers. As shown in the table on page 4 of this report, the LOS reported by the IRS declined in FY 2001 although the IRS answered significantly more taxpayer calls when both automated and CSR calls are counted.

Additional concerns about the LOS measure were raised in a July 14, 2000, briefing to the IRS Commissioner.
W&I Division management stated that the LOS does not measure the following factors, thereby not giving a full picture of the customer’s experience:

The above factors would be important for both the W&I and SB/SE Divisions in their internal management of the IRS’ toll-free telephone system. Of these, however, we believe that only a measure of the length of the taxpayer’s wait before speaking with a CSR is necessary for the GPRA quantity indicators. Such an indicator ties in closely with the IRS’ mission of providing top quality service to taxpayers. It also provides the IRS with a baseline against which they can measure efforts to improve service to taxpayers.

However, to improve the GPRA quantity indicators, several enhancements are needed.

The indicator proposed by W&I Division management needs to more realistically reflect the standards for government organizations

A significant new toll-free quantity indicator recommended to the IRS Commissioner by W&I Division management is the Assistor Response Level (ARL). The ARL is comparable to an indicator used in the private sector called Service Level. According to a guide on call center management, this type of measure ties together the resources needed to get the desired results as well as gives the clearest indication of what callers experience when they attempt to call an organization.

The IRS named the new indicator ARL, rather than Service Level, to prevent confusion with the current LOS indicator. We concur with using an appropriately defined form of ARL (as described in this report) as a GPRA quantity indicator for toll-free telephone service.

In an effort to set a "best in class" goal, W&I Division management has also recommended that the ultimate goals for ARL should be a 90 percent service level within 30 seconds for calls involving taxpayer accounts, and an 85 percent service level within 30 seconds for calls involving tax law questions. However, it is anticipated that it will take the IRS several years to reach these levels. The baseline for FY 2000 was only 31 percent in 30 seconds.

While competitive industries tend to set service level goals for answering calls within a 15 to 30 second range, a more common service level goal for government organizations is based on answering a set percentage of calls within 2 to 5 minutes. A guide on call center management stresses the need to focus on a service level objective that can be realistically achieved and comments that "…some of these organizations do a…good job of hitting these objectives consistently, a lesson many with more lofty goals could use."

Better information is needed for management analysis

Neither the IRS’ current two GPRA quantity indicators for toll-free telephones nor the proposed ARL indicator provide additional significant information. The OMB guidelines for implementing GPRA state that plans should not "…skimp on what is measured, resulting in a narrowly-drawn or fragmented picture of performance." In addition, they emphasize that "Agencies should strive to include goals or indicators for unit cost, even if only approximate costs can be estimated."

We consider the following information as vital to both the IRS and the Congress for determining the adequacy of service to taxpayers and for basing budget decisions:

Separate reporting for automated-answered calls and CSR-answered calls is necessary to accurately reflect actual taxpayer experience

The service level the IRS provides to taxpayers selecting automated telephone services varies from that provided to taxpayers calling to speak with a CSR. When calling the IRS’ toll-free "1040 line," for example, the first menu option is automated refund information. If the taxpayers desire another type of assistance, they must go through several more menu options. Once they pick a topic, the first option offered is also recorded information on that topic. Thus, taxpayers calling to speak with a CSR will spend more time in the menu system.

According to the guide on call center management previously mentioned in this report, a service level indicator provides a "stable target for planning and budgeting" and provides a basis for determining such things as how many staff are needed. Without separate indicators for automated and CSR calls, the main function of the service level indicator would be negated.

Furthermore, the new ARL quantity indicator includes a time period for taxpayer assistance. If automated calls were not separately measured, however, the ARL would present an inaccurate picture of the length of time that taxpayers are required to wait to speak with a CSR.

Actual costs of automated and CSR calls are necessary for planning purposes

Currently, the IRS has no GPRA indicators for toll-free telephone services that tie its costs to the services that are delivered. One important aspect of the GPRA is to have agencies tie their budgets to their program goals. In fact, one requirement of annual plans is that budget resources must be aligned with performance goals. However, a January 2001 GAO report on the IRS’ toll-free telephone service stated that the "IRS does not establish a long-term telephone customer service goal reflecting the needs of taxpayers and the costs and benefits of meeting that goal…." Furthermore, the Chair of the Senate Governmental Affairs Committee recently stated that the Committee will be holding agencies accountable for results and that the best way to achieve accountability is by tying the agencies’ budgets to performance objectives.

The IRS’ FY 2000 budget for providing toll-free telephone services exceeded $346 million in salaries and benefits alone and included 8,561 FTE. For FY 2001, that budget increased to $385 million, which included 8,729 FTE. While government standards require reporting full costs, not all costs are available for toll-free telephone service. However, OMB guidance states that "Agencies should strive to include goals or indicators for unit cost, even if only approximate costs can be estimated."

A cost-per-call analysis would provide a linkage between program goals, as well as assist IRS management and the Congress in making decisions about future investments in the toll-free telephone system.

There is a reason why LOS is used, but there are serious effects of not using better indicators

We questioned the reasons for using LOS as a GPRA indicator even though it provides insufficient information. W&I Division management stated that it was a measure that had been historically used by the IRS and, thus, provided some basis of comparison from year to year.

The current GPRA quantity indicators communicate nothing about the effect on individual customers or the cost of either automated or CSR service. For example, these indicators do not show how long customers wait to speak with a CSR or how long they wait to receive automated assistance. Awareness of significantly shorter wait times for automated service has the potential to move more calls to the less expensive alternative of automation. Furthermore, the absence of cost-per-call data does not provide either IRS management or the Congress with information on whether actions being taken to improve toll-free telephone service are affecting the cost per CSR-answered call or automated calls.

Recommendations

To assist the IRS in complying with the GPRA and to better reflect the taxpayer experience, the CAS Directors for the W&I and SB/SE Divisions need to:

  1. Develop a Service Level quantity indicator for CSR calls that reflects a more realistic level of anticipated service for a government organization.
  2. Provide separate Service Level indicators for automated, versus CSR-answered, calls.
  3. Develop a Cost-per-Call indicator. A methodology for calculating this indicator would be to divide the FTE and telecommunications costs by the total calls answered.

Management’s Response: For Recommendation 1, the IRS agreed that a measure that quantifies the percentage of calls answered within a target time could be a useful measure. To this end, it is currently gathering data for this measure and will develop a set of targets for FY 2003. The response did not specify whether the IRS will continue to use the 30-second target time that it is currently measuring; however, the response did state that the IRS disagreed with using a benchmark of up to 5 minutes.

In responding to Recommendation 2, the IRS stated that automated and assistor service each requires an individual measure. Further, it stated that it has already changed its LOS measure to an Assistor LOS, which provides a measure of the taxpayer’s ability to obtain assistor services when the taxpayer wants to obtain them. The IRS’ response did not address the portion of our recommendation related to providing an automated LOS GPRA indicator.

The IRS’ response to Recommendation 3 stated that program costs can be readily related to quantity measures. The IRS provided a table of costs including a cost-per-call measure in its response.

Office of Audit Comment: The Office of Audit will continue to work with the IRS to identify the actions and schedule for improving the GPRA quantity indicators.

The IRS’ response provided some indication of the planned actions; however, further clarity is needed for specific recommendations. Specifically, the IRS’ response to Recommendation 1 did not clarify whether the anticipated new measure will be used as an externally-reported GPRA measure. As stated in this report, we believe that it is important to use this as a GPRA measure to give a clear indication of what taxpayers experience when they attempt to call the IRS. In regard to setting the time standard at a level more commensurate with a government organization, in part, our reasoning is that setting the standard on par with for-profit industries can, for the near term, lead to an unrealistic performance gap to overcome. Statistics from the IRS through July 21, 2001, of FY 2001 show that only 27.73 percent of account calls and only 53.05 percent of tax law calls were answered within the 30 seconds selected by IRS management. Clearly, the IRS has a long way to go to meet its goals of answering 90 percent and 85 percent, respectively, of these calls within 30 seconds.

As with Recommendation 1, the IRS’ response to Recommendation 2 was unclear as to whether the new Assistor LOS is intended to be a GPRA measure. Consequently, we contacted IRS personnel to determine if the Assistor LOS was fully implemented or if there was a future planned implementation date. As part of the IRS’ reply to our inquiry, we were provided a copy of the formula for this indicator. Included in the formula are calls that are forwarded to an automated system when the IRS telephone system is too overloaded to accept more calls being routed to a CSR. In those cases, the taxpayer is given a choice of calling back or using an automated system. We do not believe that the inclusion of these calls is an appropriate separation of CSR and automated call reporting. For example, IRS reports for FY 2001 (through March 2, 2001), using the prior LOS formula, show an LOS of 57.94 percent, down about 7.5 percent from the prior year. Following the change to the new Assistor LOS, the March 9, 2001, reports show an Assistor LOS of 65.5 percent, up about 6 percent from the prior year.

The IRS’ response to Recommendation 3 does not address providing a cost-per-call indicator as a GPRA measure. Even though cost information is available after inquiry, we strongly believe that the GPRA cost and performance measure combination is essential to improving service and making decisions.

GPRA Quantity Indicators Should Be Revised to Include Only the Three Main Toll-Free Telephone Lines

In addition to the three CAS lines, generally referred to by the IRS as its "Customer Service Toll-Free," the GPRA LOS indicator includes two additional lines that are provided for ordering tax forms and handling overdue tax payments. The management and budget responsibility for the three main toll-free telephone lines rests with the CAS functions in the W&I and SB/SE Divisions. However, neither of the other two toll-free lines reported in the GPRA LOS calculation is a CAS responsibility.

The IRS’ FY 2001 data, through March 24, 2001, shows that the LOS that would be reported under GPRA was 61 percent. During the same period, however, the LOS for the 3 CAS toll-free telephone lines that taxpayers use to call for account, tax law, and refund information was only 58 percent. Therefore, LOS on the CAS toll-free lines was actually less than the GPRA indicator.

LOS calculations are derived by dividing the total number of calls answered by the total number of calls attempted. However, the GPRA workload indicator of calls answered also includes many toll-free lines, such as the criminal investigation line and various administrative lines, which are not included in the GPRA LOS calculation. As with the GPRA LOS calculation, these lines are not within the management or budgetary responsibility of the CAS functions.

OMB guidance states that:

"The annual plan should be directly linked to the agency’s budget. The performance goals, particularly the performance target levels, are set based on the funding expected to be available to achieve the goals."

Including toll-free telephone lines that are budgeted by other functional areas as part of the GPRA quantity indicators erodes the connection between budget and performance that is an important aspect of the GPRA process. Furthermore, it inaccurately presents results to both the Congress and senior IRS management.

IRS management informed us that they continue using the LOS indicator because it has been historically used. While we believe that it is likely that the structure of the indicators comes from a similar historical usage pattern, the inclusion of other toll-free lines distorts the actual taxpayer experience in attempting to receive detailed information regarding customer account services.

Recommendation

To add value to the GRPA quantity indicators, the CAS Directors for the W&I and SB/SE Divisions should:

  1. Ensure that the GPRA quantity indicators for toll-free telephone service report only on the three main customer service toll-free telephone lines.

Management’s Response: The IRS agreed to revise the GPRA quantity indicators in FY 2002 to include only the three primary customer service toll-free telephone lines.

Conclusion

The W&I and SB/SE Divisions of the IRS have developed numerous indicators to evaluate the service provided by the toll-free telephone system. However, those selected as GPRA quantity indicators do not provide a clear picture of the service received by taxpayers. In addition, they do not provide any connection with the cost of providing toll-free services to taxpayers.

Improved GPRA indicators are needed to focus on actual taxpayer experience and to clarify the actual costs of providing the toll-free telephone service to taxpayers.

Appendix I

Detailed Objective, Scope, and Methodology

The overall audit objective was to determine whether the Internal Revenue Service’s (IRS) annual performance plans for the quantity measurement of toll-free telephone service provide a framework to communicate the progress towards meeting strategic objectives and goals in compliance with Government Performance and Results Act of 1993 (GPRA) requirements.

To accomplish our objective, we:

I. Identified the specific GPRA measures that IRS management had designated for assessing the quantity aspect of its toll-free telephone system.

  1. Reviewed the annual performance plans for the Wage and Investment and Small Business/Self-Employed Divisions to determine to what extent the annual performance goals and measures described the performance for subsequent comparison with actual performance.
    1. Determined whether the performance measures adequately indicated progress towards performance goals.
    2. Evaluated whether annual performance goals were objective, measurable, and quantifiable.
    3. Identified whether outcome goals were included when possible.
  1. Determined whether the annual performance goals were linked to the strategic goals for the quantity of toll-free telephone service.
  2. Determined whether the GPRA quantity measures for toll-free telephone service provided IRS management and the Congress with sufficient information on which to base budget decisions and to determine the adequacy of service to taxpayers.

Appendix II

Major Contributors to This Report

Gordon C. Milbourn III, Assistant Inspector General for Audit (Small Business and Corporate Programs)

Philip Shropshire, Director

William E. Stewart, Audit Manager

Karen J. Stafford, Senior Auditor

Gwendolyn S. Gilboy, Auditor

Denise M. Gladson, Auditor

Appendix III

Report Distribution List

Commissioner N:C

Deputy Commissioner N:DC

Deputy Commissioner, Small Business/Self-Employed Division S

Deputy Commissioner, Wage and Investment Division W

Director, Customer Account Services, Small Business/Self-Employed Division S:CAS

Director, Customer Account Services, Wage and Investment Division W:CAS

Director, Accounts Management, Wage and Investment Division W:CAS:AM

Acting Director, Joint Operations Center W:CAS:JOC

Director, Organizational Performance Division N:CFO:OP

Director, Legislative Affairs CL:LA

Director, Office of Program Evaluation and Risk Analysis N:ADC:R:O

Chief Counsel CC

National Taxpayer Advocate TA

Office of Management Controls N:CFO:F:M

Audit Liaisons:

Customer Account Services, Small Business/Self-Employed Division S:CAS

Customer Account Services, Wage and Investment Division W:CAS

Appendix IV

Management’s Response to the Draft Report

The response was removed due to its size. To see the complete response, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.