Significant Efforts Have Been Made to Provide Taxpayers Better Access to the
Toll-Free Telephone System, but Additional Improvements Are Needed

 

October 2002

 

Reference Number:  2003-30-001

 

This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.

 

October 11, 2002

 

 

 

MEMORANDUM FOR COMMISSIONER, SMALL BUSINESS/SELF-EMPLOYED DIVISION

                                    COMMISSIONER, WAGE AND INVESTMENT DIVISION

 

FROM:     Pamela J. Gardiner /s/ Pamela J. Gardiner

                 Acting Inspector General

 

SUBJECT:     Final Audit Report - Significant Efforts Have Been Made to Provide Taxpayers Better Access to the Toll-Free Telephone System, but Additional Improvements Are Needed (Audit # 200130042)

 

This report presents the results of our review of the effectiveness of the Internal Revenue Service’s (IRS) efforts to improve taxpayer access to Customer Service Representatives (CSR) assigned to the toll-free telephone system.  During the 2002 Filing Season, taxpayers called the IRS almost 76.5 million times.  The IRS’ toll-free telephone system is the contact method most taxpayers choose when seeking answers to tax law questions or trying to resolve account issues.  We performed this review in reaction to the continuing concerns that the Congress and other key stakeholders have raised about the IRS’ ability to meet the significant taxpayer demand for access to the toll-free system.

In summary, while the IRS answered 1.6 million more total calls during the 2002 Filing Season than in the prior year, the additional calls answered were through automated services.  During the 2002 Filing Season, the CSRs actually handled 1.73 million fewer calls than in the comparable 2001 period, and the number of calls answered by CSRs was 2.6 million less than the IRS planned for the 2002 Filing Season.  This occurred even though the IRS received 9 million more calls than in 2001.  The IRS cited its reallocation of resources to work paper inventory, an increase in transferred calls, and high CSR availability rates as the reasons why the “CSR calls answered" goal was not met.  The effects are that the average cost per call answered, in salary alone, increased by $0.57 per call, and taxpayer burden increased as a result of difficulties in attempting to reach a CSR.

The IRS’ toll-free system is designed to provide callers with automated assistance or access to a CSR when that service is desired.  The IRS’ critical measure for taxpayer access to a CSR is based on the percentage of callers who reach a CSR after selecting that option.  The measure omits calls where the taxpayer abandoned (i.e., disconnected the call) before choosing to enter the call queue to speak with a CSR or before completing an automated service.  The IRS experienced significant problems with abandoned calls early in the 2002 Filing Season.  While corrective actions were quickly taken, the percentage of taxpayer-abandoned calls still significantly increased over 2001 Filing Season levels.  Because the CSR Level of Service (LOS) measure omitted the 9.7 million abandoned calls that occurred during the 2002 Filing Season, key stakeholders and decision makers may not have a full understanding of taxpayer access problems.  For example, the IRS reported a CSR LOS of almost 72 percent for the 2001 Filing Season and a CSR LOS of more than 70 percent for the 2002 Filing Season.  However, if the primary abandon rate that occurred in 2001 had repeated in 2002, the CSR LOS would have been 56 percent.

We recommended that the IRS take action to identify methods to reduce the number of transfers by determining whether there are problems in the routing scripts and whether the CSR call specialization approach needs some modification, identify ways to further refine the forecast accuracy by consulting call center industry experts, and achieve greater flexibility to move CSRs from applications with low demand to applications with higher demand.  We also recommended that, for the current CSR service measure, the IRS develop criteria to distinguish between those taxpayers that call and abandon because they clearly did not want a service and those that disconnect for other reasons.

Management’s Response:  The Commissioner, Wage and Investment (W&I) Division, has planned or taken corrective actions for two of our four recommendations.  To alleviate scripting problems, the IRS is adding new toll-free numbers for specific services (including telephone numbers on notice correspondence for service specific to the type of notice), initiating skill-based routing to route calls with greater precision to CSRs, and using screeners for all tax law calls rather than the current practice of having the callers navigate through a complex menu.  The IRS has developed a training strategy for 2003 to address the need for greater flexibility to move CSRs among applications.  The training curriculum for newly-hired CSRs was changed to enable them to handle account and balance due calls.  In addition, all CSRs will be trained to have a primary specialty and a back-up application assignment.

The Commissioner, W&I Division, disagreed with our recommendation to identify ways to further refine the forecasting of call demand.  He stated that the two factors that affected the IRS’ forecast (a decline in tax law inquiries and telephone calls related to errors on the Rate Reduction Credit) could not have been foreseen or forecast and that, for the 2002 Filing Season, the variation in the percentages of forecast and actual demand by application was within 3 percent in 18 of 20 major applications.  The IRS believes that its methodology is an accepted industry practice that it has used successfully to forecast demand.

The Commissioner, W&I Division, also disagreed with our recommendation to adjust the CSR LOS formula.  He stated that it measures exactly what is intended and that a definition of the measure is published in the IRS’ Strategic Plan, Data Dictionary, Measures Matrix, and numerous other sources.  He further stated that this measure is one of many the IRS uses to gauge its customer service because no single measure can adequately address all aspects of customer experience.

Management’s complete response to the draft report is included as Appendix IV.

Office of Audit Comment:  The IRS’ summarization of its forecasting accuracy for the entire 2002 Filing Season does not necessarily portray the effectiveness of the forecast.  The forecasting of call demand must be accurate at each CSR skill level and at each half-hour call arrival interval.  Because unforeseen factors affected the IRS’ forecast in 2002, it does not follow that the forecasting process is incapable of being improved.

We concur that no one measure can adequately capture the taxpayer’s experience in calling the IRS.  However, the CSR LOS is the primary measure used by the IRS Commissioner when reporting to the Congress on the ability of taxpayers to contact a CSR.  In this context, the Commissioner also uses the measure to compare the service provided from one year to another.  As designed, we believe that the CSR LOS measure masks the difficulties that taxpayers encountered in obtaining live assistance.  The IRS’ data for the period January 1 through February 9, 2002, illustrates why this measure does not clearly reflect the customer’s experience and cannot be effectively used for comparison purposes.  For this time period, as computed by the IRS, the CSR LOS increased from almost 66 percent to more than 71 percent when compared with the same time period in 2001, even though 602,000 fewer calls had been handled by CSRs, and call demand had increased by more than 1.8 million.

While we still believe our recommendations are worthwhile, we do not intend to elevate our disagreement concerning these matters to the Department of Treasury for resolution.

Copies of this report are also being sent to the IRS managers who are affected by the report recommendations.  Please contact me at (202) 622-6510 if you have questions or Gordon C. Milbourn III, Assistant Inspector General for Audit (Small Business and Corporate Programs), at (202) 622-3837.

 

Table of Contents

Background

The Internal Revenue Service Took Several Actions to Improve Service to Taxpayers on Its Toll-Free System in Fiscal Year 2002

The Goal for Calls Answered by Customer Service Representatives Was Not Met

Recommendations 1 and 2:

Recommendation 3:

Performance Measures Do Not Accurately Reflect Taxpayer Experience in Receiving Service

Recommendation 4:

Appendix I – Detailed Objective, Scope, and Methodology

Appendix II – Major Contributors to This Report

Appendix III – Report Distribution List

Appendix IV – Management’s Response to the Draft Report

 

Background

The Internal Revenue Service’s (IRS) mission is to “Provide America’s taxpayers top quality service by helping them understand and meet their tax responsibilities and by applying the tax law with integrity and fairness to all.”  All efforts to help taxpayers understand and meet their tax responsibilities are critical to this mission. 

The toll-free telephone system is the contact method most taxpayers choose when seeking answers to tax law questions or trying to resolve account issues.  During the 2002 Filing Season, taxpayers called the IRS almost 76.5 million times. Over the past several years, the United States General Accounting Office (GAO), the National Taxpayer Advocate, the Congress, and other stakeholders have expressed concerns about access to the toll-free system and quality of service.

The IRS has committed extensive efforts and resources to provide taxpayers with better access to the toll-free system.  These efforts include expending hundreds of millions of dollars over the past decade to better equip the toll-free system to meet the demand for services.  At the same time, increased staffing during the Filing Season has been the norm.  In the 1999 and 2000 Filing Seasons, for example, hundreds of compliance employees were detailed to answer taxpayer calls.

The IRS’ large investments in technology were required to upgrade its toll-free system to a modern virtual call center, from one with inflexible routing and proprietary hardware and software with limited capacity and change capability.  These changes to technology also require a corresponding change to management systems and human capital strategies.  Stabilization periods are required before full integration of the technology, management, and human capital components can be realized.  While the IRS has made concerted efforts to meet taxpayer demand for toll-free services, the National Taxpayer Advocate reported in December 2001 that access to the toll-free system was still the number one problem taxpayers had with the IRS.

To perform the audit, we analyzed toll-free system performance data covering the 2002 Filing Season provided by the IRS’ Joint Operations Center in Atlanta, Georgia, and interviewed both Small Business/Self-Employed (SB/SE) and Wage and Investment (W&I) Division personnel located in Dallas, Texas; Oakland, California; Ogden, Utah; Atlanta, Georgia; New Carrollton, Maryland; and Memphis, Tennessee.  We also conferred with a call center management consultant to analyze and interpret the IRS’ call center performance data.  The audit was performed from December 2001 through June 2002 in accordance with Government Auditing Standards.  Detailed information on our audit objective, scope, and methodology is presented in Appendix I.  Major contributors to the report are listed in Appendix II.

The Internal Revenue Service Took Several Actions to Improve Service to Taxpayers on Its Toll-Free System in Fiscal Year 2002

The IRS has recognized that changes need to be made to improve the toll-free system.  The changes made for Fiscal Year (FY) 2002 included both technology enhancements and human capital-related actions.  A Customer Service Representative (CSR) productivity study was also performed.

A $65 million technology enhancement was implemented in August 2001.  The new communications system was designed to improve the efficiency and effectiveness of receiving, routing, and responding to taxpayer calls.  The system features included voice-activated programs that were expected to reduce the need for a large percentage of the staff assigned to screen and appropriately route calls.  These resources could then be reassigned to handle taxpayer calls.  An additional feature of the enhanced system was to more accurately route taxpayer calls to the most appropriate and available CSR.

Three human capital-related actions were taken to improve the availability of CSRs to respond to taxpayer questions: 

Although this audit did not review the quality of responses to toll-free calls, IRS management informed us that they had expended significant resources this year to improve quality.  And according to data provided by the IRS, quality did improve for the 2002 Filing Season.  

The IRS measures quality separately for Tax Law calls and Account calls.  In addition, the IRS measures the quality of answers to both types of calls in two ways:  Weighted Quality Rate (WQR) and Weighted Correct Response Rate (WCRR).  As shown in Table 1, the IRS’ quality measurement system showed improvement in all four measures in 2002.

Table 1 was removed due to its size.  To see Table 1, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

The IRS also performed a productivity study to determine the causes of high Average Handle Time (AHT).  This call site study included questionnaires and focus group sessions with both managers and CSRs.  As a result, some immediate improvement actions were taken, while management is still considering other actions.

Overall, the AHT decreased this year in comparison to the prior year and was also below the IRS’ planned AHT for the year.  Due to significant changes in other factors (e.g., call transfer rates) that affected the AHT but are not related to productivity increases/decreases, we were unable to determine the actual effect of the AHT improvement actions during the 2002 Filing Season.  One study conducted by an IRS research group attempted to eliminate the effect of the other factors.  This study showed that the AHT was still slightly improved over the plan and over last year’s actual AHT.

The IRS provided us with data showing improvements in two of its performance measures related to enhancing the taxpayer experience in calling the IRS’ toll-free telephone system.  These measures are the Average Speed of Answer (ASA) and the Assistor Response Level (ARL).

The ASA decreased from 323 seconds in FY 2001 to 238 seconds in FY 2002.  A report issued as part of the National Performance Review stated that, “Customer queue time approaching one minute is universally considered unacceptable, and is strongly associated with high levels of customer dissatisfaction….”  Correspondingly, the IRS has identified that reducing the ASA is an effective way to improve certain customer service satisfaction measures.  So, while there was a 26 percent decrease in the ASA in 2002,  additional improvement is still needed to increase taxpayer satisfaction.

The ARL increased from 37 percent in FY 2001 to 54 percent in FY 2002.  The IRS attributed much of this improvement to the implementation of the CC2001.  While this measure shows significant improvement, it was affected by transferred calls that nearly doubled in 2002.  Each time a call is transferred, the caller is placed back into a queue.  Thus, the total queue time for callers may have increased due to the very large number of transfers.  The call transfer problem is discussed in detail beginning on Page 9 of this report.

The Goal for Calls Answered by Customer Service Representatives Was Not Met

While the IRS answered 1.6 million more calls during the 2002 Filing Season when compared to the same time period in 2001, the additional calls answered were through automated services.  As shown in Figure 1, the number of CSR-answered calls fell short of the IRS’ goal by over 2.6 million calls in 2002, and the CSRs actually answered 1.73 million fewer calls than in 2001.  This occurred even though the IRS received 9 million more calls on the Toll-Free Combined lines in 2002.

Figure 1 was removed due to its size.  To see Figure 1, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

The IRS’ Customer Satisfaction Survey Toll-Free Site Report for April through June 2001 showed that Ease of Getting Through by Phone was the number one improvement opportunity identified by taxpayers.  Access to the Customer Service Toll-Free telephone service was also the number one problem that the National Taxpayer Advocate reported to the Congress for FY 2001.  In analyzing the problem, the National Taxpayer Advocate reported:

Many people complain that they are unable to get through to employees, must navigate a complicated menu system and spend excessively long times on hold.  These problems sometimes lead taxpayers to hang up in frustration and search for answers elsewhere.

The IRS’ toll-free system is designed to provide callers with access to automated services or access to a CSR when that service is desired.  Over 20 years ago, the then IRS Commissioner articulated the potential effect of taxpayers not having access to a CSR.  In an October 1981 memorandum to the Deputy Secretary of the Treasury, the IRS Commissioner discussed the probable effects of eliminating telephone assistance for taxpayers.  Among other things, the Commissioner noted:

Even with the availability of automated services, these potential effects are still present.  As shown in Figure 2, 28.8 million (about 38 percent) of the almost 76.5 million callers to the IRS during the 2002 Filing Season did not speak with a CSR or complete the use of automated services.

Figure 2 was removed due to its size.  To see Figure 2, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

Instead, these 28.8 million calls were not serviced because the caller had one of the following experiences:

·        Caller received a busy signal (0.8 percent of total calls).

·        Caller received a recorded message that no services were available or received a recorded message that only automated services were available (3.4 percent of total calls).

·        Caller disconnected prior to being placed in the CSR queue (12.6 percent of total calls).

·        Caller disconnected after being placed in the CSR queue (3.6 percent of total calls).

·        Caller disconnected after connecting with an automated service (17.2 percent of total calls).

There were three primary reasons cited by IRS management for the decrease in CSR calls answered:

·        Resources were reallocated to paper programs.

·        Transferred calls increased.

·        The CSR availability rates were high.

Resources were reallocated to paper programs

During FY 2002, 233 FTEs were reallocated to work on paper programs.  This represents the difference between the originally planned FTEs (8,754) and the current estimate of the actual FTEs (8,521) that will be used for the toll-free program.   Given the rate at which calls were answered per FTE, 233 FTEs equate to over 870,000 CSR calls answered.

The IRS stated that the FTE reallocation was needed because of two factors.  Early in the 2002 Filing Season, CSRs were not receiving the expected calls and correspondence inventories were increasing.

Transferred calls increased

Once a taxpayer reached a CSR, many times he or she was not likely to receive resolution from the first CSR who answered the call.  Although data for first-stop resolution versus transferred calls were not available, the available data imply a significant call transfer problem.

For example, 2.4 million (63 percent) of the 3.8 million calls for tax law assistance received during the 2002 Filing Season were transferred.  For account and refund calls, the call transfer rate was 14 percent.  Combined, the transfer rate for all calls received was 29 percent.  The 2002 transfer rate compared unfavorably with the 17 percent transfer rate that was experienced in the 2001 Filing Season through March 23, 2001.

The IRS has conducted analyses of its call transfer problem.  One analysis stated:

Many of the transfers are made within a few minutes of the CSR initially responds (sic) to the taxpayer.  However, in many instances, the conversation lasts a significant amount of time before the CSR recognizes the need for the taxpayer to be transferred to another application. 

Another IRS analysis determined that, in 93 percent of the instances, CSRs were transferring calls when appropriate.  The analysis concluded that, for the remaining 7 percent of the calls, the CSRs should have handled the calls without transferring them.

Identifying this smaller percentage of incorrectly transferred calls, the reviewers from the IRS’ Centralized Quality Review System posted the following information message on the IRS Intranet in February 2002:

The message was removed due to its size.  To see the message, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

The IRS had scheduled resources for the 2002 Filing Season based on a planned 11 percent transfer rate.  The call center consultant with whom we conferred stated that a reasonable transfer rate for the IRS would be 5 percent or less.  However, the IRS has provided us with a document that shows a 15 percent transfer rate as meeting world-class standards; the IRS believes that this is a more appropriate target.

We identified two probable causes of the high transfer rate that are linked.  First, problems with the scripts used to route calls during the 2002 Filing Season likely caused a problem in getting the calls to the appropriate CSRs.  When a call is misrouted, the first CSR must listen to the taxpayer’s question long enough to determine whether to answer or transfer the call.  The effect is that the total time to handle the call increases.  Thus, fewer calls can be answered.

Connected to the scripting problem is the IRS decision to increase the specialization of its CSRs in an effort to provide expertise to callers.  IRS management advised us they expected that CSR specialization would result in higher transfer rates, although they did not provide us with specific information on what effect specialization actually had on transfer rates.  IRS management agreed that reducing transfers was necessary.

The CSR availability rates were high

The measure used to calculate how long CSRs are available to take calls when none are coming in for their specific application is called Assistor Availability.  The IRS defines Assistor Availability as “the percent of actual workload hours spent in an available state.”  While availability is expressed in percentages, we converted the percentages to minutes for better understanding.  Figure 3 shows the wide range of CSR availability among the IRS call sites during the 2002 Filing Season.

Figure 3 was removed due to its size.  To see Figure 3, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

Various factors contribute to assistor availability rates.  These include accurately forecasting the demand by application by half-hour intervals and scheduling the CSRs by application by half-hour intervals.  When call centers specialize, these factors are especially critical because specialization narrows the allowance for error.

Forecasting the demand for different services

Forecasting involves using historical data and knowledge of current conditions to predict call type and call arrival by the half-hour interval.  Even small forecasting inaccuracies can cause significant problems with a call center’s ability to handle call demand.  Because of the vast span of tax law topics and the complexity of the calls, the IRS decided to train CSRs to handle only certain call categories.  Less experienced CSRs handle the easiest call categories, and the demand in those applications was lower than forecasted.  Consequently, an insufficient number of CSRs were available in applications where demand was higher, while other CSRs were waiting for calls where demand was less than forecasted.

In our discussion with IRS management on this report, they disagreed with our conclusions about forecast accuracy.  IRS management advised us that they hire and train based on historical data.  However, the mix of calls was very different this year due to a large number of calls resulting from the tax Rate Reduction Credit and the September 11, 2001, disaster issues.

The IRS also stated that its forecast was accurate within plus or minus 3 percent for 18 of the 20 major application groups.  However, the range in CSR availability rates shown in Figure 3 suggests that the actual call demand patterns created some situations where an insufficient number of CSRs were available where demand was higher (i.e., low minutes available), while other CSRs were waiting for calls where demand was lower (i.e., high minutes available).

Scheduling the CSRs

The scheduling of CSRs follows the forecast.  If the forecast proves to be inaccurate, it is difficult to quickly make significant adjustments to the schedule to meet the demand because, before moving CSRs from one application to another, the IRS must train them.  Since the time period to make the adjustment is small compared to the time needed to train, adjustments to the schedule are difficult.

These combined factors contributed to fewer calls being answered

The reallocation of resources, the increase in call transfers, and high CSR availability rates all contributed in some regard to the decline in CSR calls answered by the IRS in the 2002 Filing Season.  Not only did CSRs answer 1.73 million fewer calls than in the 2001 Filing Season, they also answered fewer calls per hour, decreasing slightly from 2.5 to 2.3.  Due to the magnitude of calls into the IRS toll-free system, however, approximately 1,122,000 more callers would have been served in the 2002 Filing Season if calls had been answered by CSRs at the same rate as they were answered in 2001.

The two major effects of the decline in CSR calls answered are higher IRS costs per call answered and additional taxpayer burden.  The average cost per call answered during the 2001 Filing Season for salary alone was $8.53.  If the 2002 federal government salary increase is disregarded, the 2002 cost per call would have been $9.10, or an increase of $0.57.

The taxpayer burden issue affects taxpayer satisfaction with the toll-free system.  Taxpayers are burdened by having to wait relatively long periods of time before speaking with a CSR.  Then, the taxpayers must repeat their questions after being transferred to a second or third CSR.  Some taxpayers are also burdened by the complexity of the scripts before even making a selection to speak to a CSR.  Therefore, increases in CSR calls answered should provide some improvement in taxpayer satisfaction with the toll-free system.

Recommendations

The Directors, Customer Account Services (CAS), in the W&I and SB/SE Divisions:

1.      Should strive to reduce the number of transferred calls.  These efforts should include identifying and correcting the problems with their current routing scripts that are causing taxpayers to reach CSRs who cannot help them, thus requiring calls to be transferred.  Additionally, the Directors should review the decision to specialize CSRs to determine whether some applications would benefit by being combined into a more pooled environment that would address the transfers and, possibly, the high availability rates in some sites.

2.      Need to explore possible ways to make forecasting more accurate.  Therefore, we recommend that they consider consulting with industry experts to determine whether additional precision is possible.

3.      Should explore ways to achieve greater flexibility in the scheduling process.  The CSR training program needs to be designed to accelerate training to allow faster resource shifts to those applications where call demand is high and agent availability is low or to allow expanding the scope of applications.

Management’s Response:  The Commissioner, W&I Division, advised that the IRS made major changes to the scripts after recognizing in early January that the primary abandon rate and the transfer rate exceeded their projections.  He also stated they recognized that additional longer-term solutions were needed.  The IRS has initiated a strategy to clearly define the services available by aligning specific services to specific toll-free telephone numbers, thus reducing taxpayer burden by reducing the number of self-selection script options and script complexity.  Additional toll-free numbers are being established and marketed.  Notice correspondence will have telephone numbers for service specific to the type of notice issued.  Skill-based routing will be initiated next year and will route calls with greater precision to CSRs based on matching the caller inquiry to pre-determined CSR skills.  The IRS will also use screeners for all tax law calls rather than the current practice of having the callers navigate through a complex menu.

The Commissioner, W&I Division, disagreed with our recommendation to explore ways to make the forecast more accurate.  He stated that the two factors that affected their forecast (a decline in tax law inquiries and the errors on the Rate Reduction Credit) could not have been foreseen or forecast.  The Commissioner, W&I Division, stated that, for the 2002 Filing Season, the variation in the percentages of forecast and actual demand by application was within + or – 3 percent in 18 of 20 major application groups.  The IRS said that its methodology, which applies projected changes to historic weekly call volumes, is an accepted industry practice that it has used successfully to forecast demand.

The Commissioner, W&I Division, responded to Recommendation 3 that the IRS has developed a training strategy for 2003 that will increase its flexibility to move employees.  The training curriculum for newly-hired CSRs was changed to enable them to handle account and balance due calls.  All CSRs will be trained to have a primary specialty and a back-up application assignment.

Office of Audit Comment:  The IRS’ summarization of its forecasting accuracy for an entire Filing Season, provided in response to Recommendation 2, does not necessarily portray the effectiveness of the forecast.  The forecasting of call demand must be accurate for each CSR-skill level and for each half-hour call arrival interval.  Because unforeseen factors affected the IRS’ forecast in 2002, it does not follow that the forecasting process is incapable of being improved.

Performance Measures Do Not Accurately Reflect Taxpayer Experience in Receiving Service

The IRS’ measure for providing taxpayers with access to a live assistor is called the CSR Level of Service (LOS).  The IRS’ measure for access to a CSR is based on the percentage of callers who reach a CSR after selecting that option.  The measure omits those calls where the taxpayer abandons (i.e., disconnects) before completing an automated service or choosing to enter the call queue to speak with a CSR.  The CSR LOS may not clearly communicate to stakeholders and decision makers some of the problems taxpayers had with obtaining service.

There are two reasons why the CSR LOS measure may not clearly communicate the ease with which taxpayers gained access to a CSR.  The first was cited in GAO testimony before the Congress on April 9, 2002:

 

Contrary to what its name implies, the CSR level of service measure does not reflect only those calls handled by assistors.  Some calls handled through automation are counted as having been answered in computing this measure.  Because it includes calls answered through automation, the CSR level of service measure may be overestimating the rate at which assistors are responding to taxpayers.

The GAO did not recommend an immediate change to this measure and other measures with which they identified concerns.  Instead, they will be conducting a review of the IRS’ Filing Season performance measures, including its telephone measures, with the expectation of issuing a report later this year.

The second reason why the CSR LOS formula may not clearly communicate is that it excludes all calls where taxpayers disconnected prior to completing the calls (these calls are identified as primary abandons) for any of the following reasons:

The IRS reported a CSR LOS of almost 72 percent for the 2001 Filing Season and a CSR LOS of more than 70 percent for the 2002 Filing Season.  To illustrate the effect that primary abandons have on the CSR LOS measure, we recomputed the 2002 figure.  If the 14 percent primary abandon rate that occurred in 2001 had repeated in 2002, the CSR LOS would have been 56 percent.

Call center industry literature suggests that primary abandons should be no more than approximately 3 to 5 percent of the total calls received.  However, during the 2002 Filing Season, 9.7 million (27 percent) of the 35.9 million calls received on the IRS’ Toll-Free Combined lines resulted in primary abandons.  This was an increase of 5.8 million (154 percent) primary abandons from 2001 when the primary abandons were 3.8 million (14 percent) of the calls received.

Figure 4 was removed due to its size.  To see Figure 4, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

The likelihood is that the primary abandons significantly increased between 2001 and 2002 because of problems in the routing scripts.  The IRS’ Customer Satisfaction Surveys support this conclusion by showing that the two areas with the lowest satisfaction ratings are “Ease of Getting Through” and “Automated Answering System.”  The IRS identified some potential problems with the routing scripts and realized a reduction in the primary abandon rate after changes were made to the scripts in February 2002.  To further address the complexity of the scripts, the IRS plans to implement several new toll-free lines for FY 2003.  These will include separate lines for individuals and businesses as well as a refund line.  That aside, from the CSR LOS measure used, the effect of primary abandons on taxpayer access to CSRs is still lost.

The call center industry differentiates between abandoned calls that should be included in service level calculations and those that should not.  Essentially, those calls that abandon within the first 15 to 20 seconds are considered callers who did not seriously intend to speak with a live assistor.  The IRS includes these abandoned calls in its primary abandon numbers so there is no differentiation between callers in the same manner as there is in the call center industry.

In our discussions with the IRS management about this report, they disagreed that any changes were required to the CSR LOS measure but did agree that another measure could be developed.  However, the IRS cannot commit to whether any new measure would be among the “critical” reported measures.  We believe that the recommended changes to the CSR LOS measure would more clearly communicate to stakeholders and decision makers the taxpayers’ experience, and would better link performance measures to budget. 

Recommendation

4.      The Director, CAS, W&I Division, should adjust the CSR LOS formula so that it more clearly communicates to stakeholders and decision makers the taxpayers’ experience in accessing a CSR.  Specifically, criteria should be established to distinguish between those taxpayers that clearly did not want to speak to a CSR and those that disconnected for other reasons.  Only those taxpayers that clearly did not want to speak with a CSR should be considered a primary abandon and not included in the CSR LOS formula.  For a suggested starting point, the call center industry uses a 20-second or less time period to define primary abandons.  All other calls should be included in the service level computation.

Management’s Response:  The Commissioner, W&I Division, disagreed with this recommendation.  He stated that the CSR LOS measures exactly what is intended and that a definition of the measure is published in the IRS’ Strategic Plan, Data Dictionary, Measures Matrix, and numerous other sources.  He also stated that the CSR LOS is one of many measures they use to gauge their customer service because no single measure can adequately address all aspects of customer experience.

Office of Audit Comment:  We concur that no one measure can adequately capture the taxpayer experience in calling the IRS.  However, the CSR LOS is the primary measure used by the IRS Commissioner when reporting to the Congress on the ability of taxpayers to contact a CSR.  In this context, the Commissioner also uses the measure to compare service from one year to another.

As designed, we believe that the CSR LOS measure masks the difficulties that taxpayers encountered in obtaining live assistance.  A clear example of why this measure does not clearly reflect the customer experience and cannot be effectively used for comparison purposes is illustrated by the IRS data for the period January 1 through February 9, 2002.  For this time period, as computed by the IRS, the CSR LOS increased from almost 66 percent to more than 71 percent when compared with the same time period in 2001, even though nearly 602,000 fewer calls had been handled by CSRs and call demand had increased by more than 1.8 million.

 

Appendix I

 

Detailed Objective, Scope, and Methodology

 

The overall objective of this review was to evaluate the effectiveness of the Internal Revenue Service’s (IRS) efforts to improve taxpayer access to Customer Service Representatives (CSR) assigned to the toll-free telephone system.

To accomplish this objective, we:

I.                    Reviewed industry practices to determine what procedures, processes, incentives, and measures are commonly used to ensure agent productivity.

A.                 Reviewed literature on leading call center best practices.

B.                 Reviewed the Incoming Calls Management Institute on-line literature.

C.                 Contacted call center management in one other government agency.

II.                 Determined whether the IRS is improving productivity and, if so, to what extent.

A.                 Compared IRS productivity over a 2-year period, Fiscal Year (FY) 2001 through FY 2002 (year-to-date), using standard industry measures.

B.                 Compared these IRS productivity measures to those of other government agencies and/or private industry to determine if they are reasonable.

C.                 Evaluated the IRS’ efforts to address attrition at its call centers.

D.                 Consulted with a private industry call center management expert to provide insight on the IRS’ productivity statistics versus those of private industry.

III.               Identified and evaluated the actions the IRS is taking to improve productivity as a result of its internal study of Average Handle Time (AHT) and any other recent studies.

A.                 Interviewed IRS management to determine what studies on productivity have been conducted in the past two years.

B.                 Interviewed the leaders of each of the three teams assigned to work on various aspects of productivity as a result of the IRS’ January 2001 AHT study.

C.                 Reviewed the reports of each of the three AHT teams and compared the recommendations with concerns expressed by employees/managers during the study.

D.                 Reviewed other studies and determined what actions have been recommended as a result of these studies.

E.                  Determined which recommendations proposed by the AHT teams and other studies have been implemented, which are scheduled for implementation, and which are being deferred.

 

Appendix II

 

Major Contributors to This Report

 

Gordon C. Milbourn III, Assistant Inspector General for Audit (Small Business and Corporate Programs)

Philip Shropshire, Director

William E. Stewart, Audit Manager

Karen J. Stafford, Senior Auditor

Robert A. Nicely, Senior Auditor

Michael R. Van Nevel, Senior Auditor

 

Appendix III

 

Report Distribution List

 

Commissioner  N:C

Deputy Commissioner  N:DC

Deputy Commissioner, Small Business/Self-Employed Division  S

Deputy Commissioner, Wage and Investment Division  W

Director, Customer Account Services, Small Business/Self-Employed Division  S:CAS

Director, Customer Account Services, Wage and Investment Division  W:CAS

Director, Joint Operations Center  W:CAS:JOC

Chief Counsel  CC

National Taxpayer Advocate  TA

Director, Legislative Affairs  CL:LA

Director, Office of Program Evaluation and Risk Analysis  N:ADC:R:O

Office of Management Controls  N:CFO:F:M

Audit Liaisons:

Commissioner, Small Business/Self-Employed Division  S

Commissioner, Wage and Investment Division  W

Director, Customer Account Services, Small Business/Self-Employed Division  S:CAS

Director, Customer Account Services, Wage and Investment Division  W:CAS

 

Appendix IV

 

Management’s Response to the Draft Report

 

The response was removed due to its size.  To see the complete response, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.