Additional Effort Answering Tax Law Questions Would Improve Customer Service

 

August 2004

 

Reference Number:  2004-40-150

 

 

This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.

 

August 24, 2004

 

 

MEMORANDUM FOR COMMISSIONER, WAGE AND INVESTMENT DIVISION

 

FROM:     (for) Gordon C. Milbourn III /s/ Daniel R. Devlin

                        Acting Deputy Inspector General for Audit

 

SUBJECT:     Final Audit Report - Additional Effort Answering Tax Law Questions Would Improve Customer Service (Audit # 200440038)

 

This report presents the results of our review of the Internal Revenue Service’s (IRS) toll-free telephone assistance program.  Our objective was to determine whether taxpayers received quality service when calling the IRS Toll-Free Customer Service telephone number (1‑800‑829‑1040) to ask tax law questions during the 2004 Filing Season.  We performed this audit as part of the Treasury Inspector General for Tax Administration’s (TIGTA) continuing annual reviews of the performance of key customer service programs.

In summary, IRS Customer Service Representatives (CSR) exceeded goals in each of the key measures of Professionalism and Timeliness but were less than satisfactory when providing accurate tax law answers during the 2004 Filing Season.  Each year, millions of taxpayers call the IRS for assistance in understanding the tax law and meeting their tax obligations.  During the 2004 Filing Season, IRS CSRs answered over 15.8 million Customer Account Services toll-free telephone calls.  Over 7.3 million of these calls were answered on the toll-free telephone number, 1-800-829-1040.  The IRS estimates that, in 4 million of these 7.3 million calls, taxpayers had questions about the tax law and spoke with a CSR.

With continued demands from taxpayers for assistance and the increased complexity of the Federal tax code, the quality of service to taxpayers remains among the major management challenges the IRS faces in Fiscal Year (FY) 2004.  One of the IRS’ major strategies for FY 2004 is to reduce taxpayer burden by improving the quality and efficiency of service delivery.  The aim is to provide prompt and courteous responses to all requests for assistance.  The IRS’ goal is to make its telephone operation a “world‑class customer service organization” that provides taxpayers with accessible and accurate tax assistance.

Using a judgmental sample of 129 screener and 354 CSR monitored calls during the period January 20 through February 18, 2004, we determined employees generally provided professional and timely service.  However, CSRs provided taxpayers with accurate answers to their tax law questions for only 62 percent (200 of 322) of the calls monitored.  This is a decrease from the 73 percent accuracy rate we reported last year for the period January 27 through March 13, 2003.  This happened, for the most part, because CSRs are not using or completely using the required Probe and Response (P&R) Guide to obtain sufficient information from taxpayers or are not correctly interpreting the tax law.  IRS results from a sample of 1,527 tax law calls monitored for customer accuracy during the same time period and for the same tax law applications indicated a customer accuracy rate of 74 percent (1,132 of 1,527 opportunities).  The IRS’ tax law accuracy goal for CSRs is 85 percent for FY 2004. 

Of the 122 toll-free tax law question calls in which taxpayers received inaccurate (incorrect or incomplete) answers, it is probable that 97 (80 percent) of the taxpayers would have received a correct and complete answer if the CSR had thoroughly used the P&R Guide and obtained sufficient information from the taxpayers.  The IRS is also concerned that, without sufficient use of the P&R Guide, CSRs are unable to fully understand a taxpayer’s situation and may provide guidance and information that is incorrect or incomplete.  Therefore, the Director, Accounts Management, Wage and Investment Division, included in the FY 2004 Program Letter that failure to use the Guide would be addressed as both a conduct and performance issue.  We are making no recommendations this year since the need to ensure better use of the P&R Guide was reported in our 2 prior year reviews of Toll-Free Tax Law Assistance.

Management’s Response:  IRS management noted that providing customers with accurate tax law answers remained a challenge during the 2004 Filing Season and acknowledged less than satisfactory performance early in the 2004 Filing Season.

They agreed that most errors could be avoided if the P&R Guide was used properly and stressed to the CSRs and managers the importance of completely using the P&R Guide.  IRS management also noted that the decline in the tax law accuracy rate was caused by a formatting revision to the P&R Guide in FY 2003 that did not enhance its usability as anticipated and that changes made to improve customer access had a negative impact on accuracy.  IRS management agreed with our outcome measures.  Management’s complete response to the draft report is included as Appendix VIII.

Please contact me at (202) 622-6510 if you have questions or Michael R. Phillips, Assistant Inspector General for Audit (Wage and Investment Income Programs), at (202) 927‑0597.

 

Table of Contents

Background

Customer Service on the Toll-Free Telephone Lines Has Not Reached “World-Class Service” Level

Appendix I – Detailed Objective, Scope, and Methodology

Appendix II – Major Contributors to This Report

Appendix III – Report Distribution List

Appendix IV – Outcome Measures

Appendix V – Embedded Quality

Appendix VI – Comparison of Treasury Inspector General for Tax Administration and Internal Revenue Service Calls Monitored by Question Category

Appendix VII – Quality Measures Used to Evaluate Taxpayers’ Experience

Appendix VIII – Management’s Response to the Draft Report

 

Background

Each year, millions of taxpayers contact the Internal Revenue Service (IRS) seeking assistance in understanding the tax law and meeting their tax obligations by either calling the various toll-free telephone assistance lines, accessing the IRS Internet web site, or visiting an IRS assistance center.  Although accessing the IRS web site is the single most frequent means of contact with the IRS for taxpayers, the second most frequent means is calling the toll-free telephone assistance lines.

According to the April 2003 IRS Oversight Board Annual Report, 90 percent of Americans surveyed viewed the IRS service of providing a toll-free telephone number to answer their questions as either very or somewhat important to them.  This same Report also reflected the fact that, although taxpayer demands for assistance continue to grow, so too does the complexity of the Federal tax code.

The quality of service provided to taxpayers remains among the major management challenges the IRS faces in Fiscal Year (FY) 2004.  One of the IRS’ major strategies for FY 2004 is to reduce taxpayer burden by improving the quality and efficiency of service delivery.  The aim is to provide prompt and courteous responses to all requests for assistance.  The IRS’ goal is to make its toll-free telephone operation a “world-class customer service organization” that provides taxpayers with accessible and accurate tax assistance.  To ensure a successful filing season, the Director, Accounts Management, Wage and Investment (W&I) Division, annually certifies that all elements of the basic filing season certification document are incorporated into the overall planning process and that all items will be monitored for progress and timely delivery.

During the 2004 Filing Season, IRS Customer Service Representatives (CSR) answered over 15.8 million Customer Account Services toll-free telephone calls.  Over 7.3 million of these calls were answered on the toll-free telephone number, 1-800-829-1040.  The IRS estimates that, in 4 million of these 7.3 million calls, taxpayers had questions about the tax law and spoke with a CSR.

When calling the toll-free telephone number with a tax law question(s), the taxpayer uses an automated menu.  The taxpayer is provided, in English, four touchtone main menu options with secondary options:

1.      Preparing or filing individual income tax returns.  This option provides the taxpayer with a second automated menu from which to choose one of the following options:  (1) ordering tax forms or publications, (2) finding information about a missing Form W-2 or Form 1099, (3) finding the amount of his or her Advanced Child Tax Credit, (4) finding addresses to mail payments or a tax return to the IRS, or (5) getting help with a tax law question.

2.      Requesting information on a tax refund or personal tax account.  This option provides the taxpayer with a second automated menu from which to choose one of the following options:  (1) requesting refund information or (2) getting answers to personal account questions.

3.      Using the business and specialty tax line or obtaining the address for the IRS Internet web site.

4.      Repeating the above systems.

If the taxpayer does not select an option or is calling from a rotary phone and is unable to select a touchtone option, the same script is repeated providing voice response options.  If the taxpayer selects an invalid option, he or she is transferred to an IRS employee (screener), who screens and transfers the call to the appropriate CSR to answer the taxpayer’s question.

Because of the complexity of the tax law and the need to serve millions of taxpayers with tax law questions, the IRS has divided the types of tax law questions commonly received into categories called “applications.”  These applications are designed to group calls with similar tax law subjects.  For example, calls about the various types of tax credits are all routed to one application, and calls about Individual Retirement Arrangements are routed to another.  The IRS believes training CSRs to answer questions in specific applications is a more efficient use of its resources and will improve the quality and accuracy of the answers provided to taxpayers because the CSRs have specialized knowledge.

Each application is staffed with CSRs trained to answer questions in a particular application(s).  Because the IRS has trained each CSR in a specific application(s), a taxpayer with multiple questions that cross more than one tax topic or application will probably speak with multiple CSRs to have his or her questions answered. 

CSRs are provided with numerous tools to help them answer tax law questions.  One tool CSRs use is the Probe and Response (P&R) Guide.  The P&R Guide is designed to give CSRs scripts to follow to ensure they address all issues related to a taxpayer’s particular question.  For example, there are scripts for questions on Who Must File, Dependents, Charitable Contributions, Individual Retirement Arrangements, and the Child Tax Credit. 

During the 2004 Filing Season, there were 88 individual P&R Guide scripts available.  The Internal Revenue Manual requires CSRs to use the available P&R Guide scripts, when applicable, to help ensure taxpayers receive correct and complete answers to their tax law-related questions.  The IRS is also concerned that, without sufficient use of the P&R Guide, CSRs are unable to fully understand a taxpayer’s situation and may provide guidance and information that is incorrect or incomplete.  Therefore, the Director, Accounts Management, W&I Division, included in the FY 2004 Program Letter that failure to use the Guide would be addressed as both a conduct and performance issue. 

If a taxpayer with a tax law question calls 1-800-829-1040 and chooses menu item # 1 and then option # 5, getting help with a tax law question, the taxpayer first speaks to a screener.  The screener greets the taxpayer, provides identifying information, and asks the taxpayer how he or she may direct the call.  If necessary, the screener asks the taxpayer questions to target the purpose of the call.  If needed, the screener paraphrases or asks the taxpayer more questions before deciding where to transfer the call.  Once the screener determines the topic, the screener uses the Telephone Transfer Guide to transfer the taxpayer to the correct procedural or tax law application. 

For example, a taxpayer having questions about whether he or she can claim his or her 19-year-old child as a dependent and whether he or she can claim the Child Tax Credit speaks:

First.                     With a screener who determines the taxpayer has a question about dependents and transfers the taxpayer to the first CSR to answer the question on dependents.

Second.           With a CSR trained on the dependency topic.  The CSR asks several questions to determine whether the taxpayer can claim the dependent exemption.  The CSR first asks if the child has a valid Taxpayer Identification Number (TIN) and for the amount of the taxpayer’s Adjusted Gross Income (AGI) because exemptions are phased out if the AGI exceeds filing status amounts. 

If the child has a valid TIN and the AGI is within appropriate limits, five more tests must be met for the taxpayer to claim the child as a dependent.  These tests are:

·        Member of Household or Relationship Test.  The CSR asks for information concerning the nature of the relationship (son, daughter, niece, etc.) or, if the child is not a qualifying relative, how long the child has lived in the home with the taxpayer.

·        Citizenship Test.  The CSR asks if the child is a United States citizen or resident alien.

·        Joint Return Test.  The CSR asks if the child is single or married.

·        Gross Income Test.  The CSR asks for the amount of the child’s gross income.

·        Support Test.  The CSR asks for the amount of support the taxpayer provided the child during the tax year.

If all five tests are met, the CSR advises the taxpayer that he or she may claim the child as a dependent.  The CSR asks if the taxpayer has further questions.  Since the taxpayer in this example has a question about whether he or she can claim the Child Tax Credit, the CSR transfers the taxpayer to another CSR trained in the Credits application.

Third.                  With a CSR trained in the Credits application.  The CSR asks the taxpayer a series of questions to determine whether the taxpayer qualifies for the Child Tax Credit.  The CSR asks the taxpayer if the child’s age is greater than 17.  Since the child is 19 years of age, the P&R Guide script instructs the CSR to advise the taxpayer that he or she is not eligible for the Child Tax Credit because the child has exceeded the maximum age limit.

The IRS uses the Embedded Quality (EQ) system to measure the quality of taxpayer assistance in the areas of professionalism, timeliness, and customer accuracy.  The EQ system links employee performance to organizational results related to the quality of customer service.  This method was designed to distinguish between “wrong” answers and procedural errors that do not affect the accuracy of the answer.  See Appendix V for details on the EQ system.

The IRS toll-free telephone systems have the ability to trace one call from the time the IRS’ telephone service provider delivers the call to the IRS to the time the call is terminated, but the IRS does not do so because of the large volume of calls it receives.  Therefore, we could not determine how long it took the taxpayers we monitored to get answers to all their questions.  The IRS does, however, monitor key segments in the “life” of a call (e.g., the segment from the time a screener answered the call to when it was transferred to a CSR, or from the time a CSR connected to the taxpayer to the time the taxpayer was transferred or the call ended).  The IRS believes this provides indications of the type of service the average caller receives.  In addition, the IRS’ remote monitoring system only allows monitoring of segments.  Because we used this same system to monitor our judgmental sample of calls, we were unable to track the taxpayers we monitored from the time they connected to the IRS through completion of the calls.  Instead, we could only monitor segments of the calls.

The audit was conducted in the IRS W&I Division Headquarters in Atlanta, Georgia, during the period January through April 2004.  Calls were monitored at 16 IRS toll-free call sites assigned to the default screener application and 8 IRS toll-free call sites assigned to tax law applications that answered tax law questions from English-speaking taxpayers in January and February 2004.  The audit was conducted in accordance with Government Auditing Standards.  Detailed information on our audit objective, scope, and methodology is presented in Appendix I.  Major contributors to the report are listed in Appendix II.

Customer Service on the Toll-Free Telephone Lines Has Not Reached “World-Class Service” Level

IRS employees are courteous and respectful and spend an appropriate amount of time with taxpayers when answering their tax law questions.  Using a judgmental sample of 129 screener and 354 CSR monitored calls and the IRS’ EQ Attributes to rate them, we determined employees provided professional and timely service.  However, CSRs provided taxpayers with accurate answers to their tax law questions for only 62 percent (200 of 322) of the calls monitored.  This happened, for the most part, because CSRs are not using or completely using the P&R Guide scripts or are not correctly interpreting the tax law.  Of the 122 toll-free tax law calls in which taxpayers received inaccurate (incorrect or incomplete) answers, it is probable that 97 (80 percent) taxpayers would have received a correct and complete answer if the CSR had thoroughly used the P&R Guide and obtained sufficient information from the taxpayers.  See Appendix VI for a comparison of the calls monitored by the Treasury Inspector General for Tax Administration (TIGTA) and IRS by topic for the same time period.

The accuracy rate decreased from the 73 percent we reported last year for the period January 27 through March 13, 2003.  The need for CSRs to fully probe taxpayers for information to ensure they provide accurate answers to tax law questions is an ongoing issue that both the IRS and we have previously identified. 

The IRS’ mission is to provide top-quality service to taxpayers.  Its W&I Division supports this mission by offering products and services tailored to particular taxpayer needs.  CSRs must assist taxpayers in a manner that warrants the highest degree of public confidence.  Taxpayers rely on the IRS to be the expert in explaining the tax law.  While CSRs are consistently courteous and limit their conversations with taxpayers so as many taxpayers as possible can speak with a CSR, additional effort is necessary for the IRS to achieve its goal of being a “world-class customer service organization.” 

Screeners and CSRs were professional when responding to taxpayers 

Using IRS EQ Attributes to rate the monitored screener and CSR calls on professionalism, we determined the screeners and CSRs, respectively, provided professional service for 99 percent (513 of 520 opportunities) and 98 percent (1,867 of 1,904 opportunities).  The IRS defines professional service as promoting a positive image of the IRS by using effective communication techniques. 

For the calls monitored, the employees provided a professional greeting and communicated with clear and appropriate language, used courteous and respectful tones, listened to the taxpayers in an effective manner to maximize understanding, and apologized for IRS errors when appropriate.  In addition, when appropriate, they provided taxpayers with an explanation of why a series of questions is required.  For the same time period, the IRS reported that 99 percent (1,608 of 1,621 opportunities) of its screeners and 100 percent (9,276 of 9,306 opportunities) of its CSRs followed procedures for providing professional service.  The IRS’ professionalism goal for screeners and CSRs is 99 percent for FY 2004. 

Screeners and CSRs were timely when responding to taxpayers

Using IRS EQ Attributes to rate the monitored screener and CSR calls on timeliness, we determined the screeners and CSRs were timely for 100 percent (135 of 135) and 98 percent (469 of 477), respectively, of the opportunities.  The IRS defines timeliness as resolving an issue in the most efficient manner through the use of proper workload management and time use techniques.  Timeliness is measured by factors or actions that are controllable by the screeners and CSRs, including whether the employee controlled unrelated or unnecessary dialogue initiated by the taxpayer and restrained from initiating extraneous dialogue with the taxpayer.  An additional timeliness attribute measures whether the employee placed the taxpayer on hold for research purposes for an appropriate length of time based on the complexity of the issue. 

For the same time period, the IRS reported that nearly 100 percent (421 of 423 opportunities) of its screeners and 99 percent (2,176 of 2,193 opportunities) of its CSRs followed procedures for providing timely service.  The IRS’ timeliness goal for screeners and CSRs is 99 percent for FY 2004.  

Although we could determine whether screeners and CSRs provided timely service to each taxpayer on the segments of the calls we monitored, we could not determine how long it took the taxpayers to get answers to all their questions.  The IRS estimates taxpayers spent between 10 minutes 49 seconds and 22 minutes 51 seconds on the telephone from the time a taxpayer dialed 1-800-829-1040 until the taxpayer obtained an answer from 1 CSR.

In addition, in our sample, taxpayers spoke to more than 1 CSR to obtain answers to their questions in 25 percent (87 of 354) of CSR call segments monitored.  The IRS stated the actual 2004 Filing Season ratio of tax law calls to services is 1 to 1.3, which means about one-third of all taxpayers calling with a tax law question spoke with more than 1 CSR. 

CSRs provided accurate answers to 62 percent of tax law questions 

Using IRS EQ Attributes to rate the monitored screener and CSR calls on accuracy, we determined the CSRs provided complete and correct answers to only 62 percent (200 of 322 opportunities) of the calls monitored.  The IRS considers a response “complete and correct” when the CSR obtains sufficient information to answer the question and gives a correct and complete answer.  The IRS’ definition of accuracy does not take into consideration any additional issues or procedures that do not directly affect the taxpayer.  IRS results from a sample of 1,527 tax law question calls monitored for customer accuracy during the same time period and for the same tax law applications indicated a customer accuracy rate of 74 percent (1,132 of 1,527 opportunities).  The IRS’ tax law accuracy goal for CSRs is 85 percent for FY 2004.  Table 1 summarizes the results of the 322 calls we monitored.

 

Table 1:  Results of 322 Monitored Calls

 

Total

Correct

Incorrect

Required P&R Guide

214

110

104

Did Not Require P&R Guide

78

65

13

Procedural Topics

25

25

0

Issues Unidentified

5

0

5

Total

322

200

122

Source:  Calls monitored by the TIGTA for the period January 20 through February 18, 2004.

To be rated as providing a complete and correct answer for tax law questions, CSRs must use all applicable elements of the P&R Guide to respond to the questions or must interpret the tax law correctly for those questions without an applicable P&R Guide script.  The CSRs were required to use the P&R Guide for 66 percent (214 of 322) of the calls we monitored.  However, the CSRs did not use the P&R Guide accurately or completely for 49 percent (104 of 214) of those calls.  In these instances, the CSRs either did not ask or obtain all the appropriate information required by the P&R Guide script, did not use the P&R Guide, used the P&R Guide script but did not interpret the tax law correctly, or did not give the required response. 

The CSRs were not required to use the P&R Guide for 24 percent (78 of 322) of the calls we monitored.  In these instances, the CSR must obtain facts from the taxpayer and interpret the tax law to answer the question.  However, the CSRs interpreted the tax law incorrectly or provided an incorrect response for 17 percent (13 of 78) of the calls not requiring the use of the P&R Guide. 

For 2 percent (5 of 322) of the monitored calls, the CSRs did not ask questions to identify and address the taxpayers’ issues to provide a correct and complete response.  All 5 (100 percent) of these taxpayers received incorrect answers.  The remaining 8 percent (25 of 322) of the monitored calls were not tax law questions; they related to procedural topics.  All of these 25 taxpayers (100 percent) received a correct answer.

The first step toward achieving taxpayer compliance is providing quality customer service.  Although IRS employees are professional and timely, the accuracy of the answers is essential to meeting the goal of providing quality customer service.  However, we are making no recommendations since the need to ensure better use of the P&R Guide was reported in our 2 prior year reviews of the toll-free tax law assistance program.

Management’s Response:  IRS management noted that providing customers with accurate tax law answers remained a challenge during the 2004 Filing Season and acknowledged less than satisfactory performance early in the 2004 Filing Season.  They agreed that most errors could be avoided if the P&R Guide was used properly and stressed to the CSRs and managers the importance of completely using the P&R Guide.  IRS management also noted that the decline in the tax law accuracy rate was caused by a formatting revision to the P&R Guide in FY 2003 that did not enhance its usability as anticipated and that changes made to improve customer access had a negative impact on accuracy.

 

Appendix I

 

Detailed Objective, Scope, and Methodology

 

The overall objective of this review was to determine whether taxpayers received quality service when calling the Internal Revenue Service (IRS) Toll-Free Customer Service telephone number (1-800-829-1040) to ask a tax law question during the 2004 Filing Season.  Specifically, we determined the process the taxpayer experienced and the quality of the response received when calling to get an answer to a tax law question. 

To accomplish our objective, we:

I.                    Identified the program’s goals and measures for providing quality service for Fiscal Year 2004 by discussing them with the IRS and researching IRS strategies, plans, reviews, and budgets. 

II.                 Determined how the IRS monitors, measures, and improves the quality of service provided by toll-free telephone operations tax law screeners and Customer Service Representatives (CSR).

III.               Obtained and reviewed the 2004 Filing Season Readiness Certifications from the Director, Accounts Management, Wage and Investment Division, and the five Field Directors, Accounts Management.

IV.              Prepared a judgmental sampling plan to monitor 112 screener and 336 tax law calls Monday through Friday, excluding holidays, during the period January 20 through February 27, 2004.  We selected a judgmental sample due to limited staff resources while attempting to monitor calls that were representative of the business workday hours of operation for the eight IRS call sites that answered toll-free tax law questions from English-speaking taxpayers during this time.  To determine the calls to be monitored by tax law application, site, and hour of day, we performed the following analyses:

A.     Determined the volumes and percentages of projected calls for the screener and 9 of 11 tax law applications to be monitored and the volumes and percentages of projected calls by call site for IRS Planning Period 2 (PP2) (PP2 is from January–June).  We did not monitor tax law calls from the Business Issues and the Complex Individual Issues Applications.

B.     Determined the expected time of day with the greatest call volumes to be 8:30 a.m. to 5:30 p.m. Central Standard Time by reviewing planned staffing by half-hour increments for the three tax law applications with the greatest projected call volumes for PP2.

C.     Calculated the percentage of the planned 112 screener calls to be monitored at the 16 screener call sites and the planned 336 CSR calls to be monitored by 9 tax law applications at the 8 tax law call sites.

D.     Remotely monitored 129 screener calls and 354 tax law CSR calls by tax law application and documented the conversations from January 20 through February 18, 2004.

V.                 Using the Embedded Quality (EQ) Measurement System used by the IRS to rate the screeners and CSRs, rated the quality of responses for 129 screener and 354 tax law CSR calls monitored.

VI.              Obtained and analyzed toll-free telephone documentation regarding the following:

·        Calls Received, Assistor Calls Answered, and Services Provided.

·        An estimated volume of Tax Law Assistor Calls Answered.

·        An average amount of time (by application) a taxpayer would be on the telephone from the time he or she connected to the 1-800-829-1040 menu script, to when he or she completed the call after receiving service from one CSR.

VII.            Provided copies of exceptions weekly to the IRS for agreement to the quality rating of each monitored call.

VIII.         Determined the results of the IRS’ EQ reviews achieved during our testing period. 

IX.               Discussed results with appropriate IRS personnel

 

Appendix II

 

Major Contributors to This Report

 

Michael R. Phillips, Assistant Inspector General for Audit (Wage and Investment Income Programs)

Augusta R. Cook, Director

Paula W. Johnson, Audit Manager

Lynn Faulkner, Lead Auditor

Jack Forbus, Senior Auditor

Sharon Shepherd, Senior Auditor

Robert Baker, Auditor

Jerry Douglas, Auditor

Andrea McDuffie, Auditor

Geraldine Vaughn, Auditor

 

Appendix III

 

Report Distribution List

 

Commissioner  C

Office of the Commissioner – Attn:  Chief of Staff  C

Deputy Commissioner for Services and Enforcement  SE

Deputy Commissioner, Wage and Investment Division  SE:W

Director, Customer Account Services, Wage and Investment Division  SE:W:CAS

Director, Strategy and Resources Management, Wage and Investment Division  SE:W:S

Acting Chief, Performance Improvement, Wage and Investment Division  SE:W:S:PI

Director, Accounts Management, Wage and Investment Division  SE:W:CAS:AM

Director, Joint Operations Center, Wage and Investment Division  SE:W:CAS:JOC

Chief Counsel  CC                   

National Taxpayer Advocate  TA

Director, Office of Legislative Affairs  CL:LA

Director, Office of Program Evaluation and Risk Analysis  RAS:O

Office of Management Controls  OS:CFO:AR:M

Audit Liaison:  Acting Senior Operations Advisor, Wage and Investment Division  SE:W:S

 

Appendix IV

 

Outcome Measures

 

This appendix presents detailed information on the measurable impact our results will have on tax administration.  This benefit will be incorporated into our Semiannual Report to the Congress.

Type and Value of Outcome Measure:

·        Taxpayer Burden – Actual; 122 taxpayers (see page 6).

Methodology Used to Measure the Reported Benefit:

We monitored a judgmental sample of 354 live taxpayer toll-free tax law calls received by the Internal Revenue Service (IRS) between January 20 and February 18, 2004.  Although our judgmental sample was not random (i.e., it could not produce statistically valid results), we nevertheless attempted to ensure it was representative of the population by creating a sampling plan in which the percentage of calls monitored by type of tax law question was reflective of the IRS’ planned 2004 Filing Season volumes of calls per type of tax law question (referred to as applications).  We included calls serviced by eight IRS call sites that answered toll-free tax law questions from English-speaking taxpayers during this time.  The Customer Service Representatives (CSR) transferred 32 of the 354 calls to another toll-free area without the taxpayer receiving assistance from the first CSRs; therefore, we could not evaluate these 32 calls for customer accuracy.  Of the 322 calls monitored for customer accuracy, 122 (38 percent) received inaccurate (incorrect and/or incomplete) answers to the related tax law questions.

We measured the customer accuracy of each call response by using the IRS’ seven attributes to determine whether the CSR asked questions to obtain sufficient information to identify the tax law issue and correctly applied the information to lead to a correct and complete response.  The seven attributes were divided into four categories (Probe and Response (P&R) Guide, Non-P&R Guide, Procedural Action/Answer, and Identification of Issue).  An error in any one of the associated attributes resulted in an error in customer accuracy.  These seven attributes are Apparent Use of the P&R Guide, Complete Use of the P&R Guide, Complete Answer per the P&R Guide, Obtain/Determine Tax Law Facts, Interpret/Apply Tax Law, Appropriate Procedural Action/Answer, and Issue Identification (see Appendix VII). 

Of the 122 toll-free tax law calls that received an inaccurate answer, 104 (85 percent) required the use of the P&R Guide.  In these instances, the CSRs did not ask or obtain all the appropriate information required by the P&R Guide script, did not use the P&R Guide, used the P&R Guide script but did not interpret the tax law correctly, or did not give the required response.  For 13 (11 percent) of the 122 inaccurate responses, the P&R Guide was not required; however, the CSRs did not interpret the tax law correctly, did not obtain all the necessary facts, or did not provide a correct response.  For the remaining 5 (4 percent) inaccurate responses, the CSRs did not ask questions to identify and address the taxpayer’s issue to provide a correct and complete response.

Increased and more complete use of the P&R Guide and additional training on correctly interpreting the tax law should have reduced taxpayer burden for 122 taxpayers during the 2004 Filing Season.

 

Appendix V

 

Embedded Quality

 

On October 1, 2002, the Internal Revenue Service (IRS) implemented a revised system for measuring the quality of taxpayer assistance which links employee performance to organizational results related to the quality of customer service.  The Embedded Quality (EQ) system replaced the previous “pass/fail” method and uses instead a “defect-per-opportunity” method.  This method was designed to distinguish between wrong answers and procedural defects that do not affect the accuracy of the answer. 

The comprehensive measures include timeliness, professionalism, and accuracy (comprised of customer, regulatory, and procedural accuracy).  Customer Accuracy reflects whether the Customer Service Representative (CSR) gave a correct/complete response/resolution to the taxpayer’s issue.  This measurement system was used on the toll-free telephone operations beginning with the 2003 Filing Season. 

The IRS Centralized Quality Review Staff measures the service provided by toll-free telephone operations CSRs by listening to a statistically valid sample of live taxpayer calls from among the various tax law applications and rating the calls using the IRS EQ system. 

In the EQ measurement and calculation process, the measures are calculated using the percentage correct based on the number of opportunities for defect within each of five “buckets.”  The buckets are divided into Accuracy, Professionalism, and Timeliness; the Accuracy bucket is further subdivided into Customer Accuracy, Regulatory Accuracy, and Procedural Accuracy buckets.  They are defined as follows:

·               Customer Accuracy:  giving the correct answer with the correct resolution. “Correct” is measured based upon the taxpayer receiving a correct response or resolution to his or her case or issue and, if appropriate, the CSR taking the necessary case actions or case disposition to provide this response or resolution.  For the purpose of coding, reviewers do not take into consideration any additional IRS issues or procedures that do not directly affect the taxpayer’s issue or case.  This measurement system was baselined on the toll-free telephone operations during the 2003 Filing Season.  For Fiscal Year 2004, the IRS’ goal for Toll-Free Tax Law Customer Accuracy is 85 percent. 

·               Regulatory Accuracy:  adhering to statutory/regulatory process requirements when making determinations on taxpayer accounts.

·               Procedural Accuracy:  adhering to nonstatutory/nonregulatory internal process requirements.

·               Professionalism:  promoting a positive image of the IRS by using effective communication techniques.

·               Timeliness:  resolving an issue in the most efficient manner through the use of proper workload management and time use techniques.

We also used the EQ system to perform our review.  Although we used a judgmental sample and the IRS used a statistically valid sample to measure the performance of the toll-free telephone operations CSRs, the results of the two independent reviews were similar in determining the areas in which improvements are needed.

 

Appendix VI

 

Comparison of Treasury Inspector General for Tax Administration and Internal Revenue Service Calls Monitored by Question Category

 

QUESTION CATEGORY

TIGTA

IRS

Filing Status/Dependents

26%

29%

Credits

20%

18%

Advanced Individual Issues

13%

6%

Small Business

10%

13%

Income

9%

9%

Pension/Social Security Benefits

8%

10%

Individual Retirement Arrangements

8%

4%

Deductions

3%

7%

Capital Gains and Losses (Schedule D)/ Sale of Home

3%

2%

Source:  Results of the TIGTA and IRS independent reviews of 9 of 10 toll-free tax law category/application calls monitored between January 20 and February 18, 2004, during the hours of 8:30 a.m. to 5:30 p.m. Central Standard Time, excluding weekends and holidays.

 

Appendix VII

 

Quality Measures Used to Evaluate Taxpayers’ Experience

 

In the Embedded Quality (EQ) measurement and calculation process, the measures are calculated using the percentage correct based on the number of opportunities for defect within each of five “buckets.”  The buckets are divided into Professionalism, Timeliness, and Accuracy; the Accuracy bucket is further subdivided into Customer Accuracy, Regulatory Accuracy, and Procedural Accuracy buckets. 

The EQ system uses a “defect-per-opportunity” method to distinguish between wrong answers and procedural defects.  Each call is rated for various attributes, known as opportunities.  Every attribute is not applicable to every call.  The following attributes were used to rate the monitored screener and Customer Service Representative (CSR) calls.

Professionalism

1.      Greeting – Did the CSR open the call in a professional manner and offer to help the taxpayer?

2.      Purpose Statement – When appropriate, did the CSR explain the process or actions to be taken to the taxpayer?

3.      Closing – Did the CSR appropriately close the contact with the taxpayer?

4.      Communication – Did the CSR use clear and appropriate language with no jargon to ensure communication is complete with the taxpayer?

5.      Courteous – Did the CSR provide courteous service by speaking in a respectful tone and checking back with the taxpayer at appropriate intervals when placed on hold?

6.      Effective Listening – Did the CSR listen to the taxpayer in an effective manner to maximize understanding without repeating facts that were already provided?

7.      Apologized – When appropriate, did the CSR apologize to the taxpayer for Internal Revenue Service (IRS) errors, including a long hold time?

Timeliness

1.      Appropriate Use of Talk Time – Did the CSR control the conversation and avoid extraneous dialogue with the taxpayer?

2.      Appropriate Hold Time – Did the CSR use the appropriate length of hold time for the complexity of the issue?

3.      Efficient Response/Resolution – Did the CSR take actions in an efficient manner to prevent any unnecessary delay to resolve the taxpayer’s issue?

Customer Accuracy

1.      Correct/Compete Response/Resolution – Did the CSR provide a correct/complete response/resolution to the taxpayer’s issue?

Regulatory Accuracy

1.      Assistor Identification (ID) – Did the CSR identify himself or herself by name and ID number?

2.      Interpret/Apply Tax Law – Did the CSR interpret/apply the tax law correctly?

Procedural Accuracy

1.      Issue Identification – Did the CSR ask questions, listen, and paraphrase the taxpayer’s question to identify and address the correct tax law issue to lead to a correct and complete response?

2.      Appropriate Transfer – Did the CSR correctly identify and transfer the call without resolving the taxpayer’s issues when needed?

3.      Appropriate Referral – Did the CSR answer a tax law question that is on the application the CSR is assigned to and transfer the taxpayer to another application/trained CSR to obtain an answer to a second question?

4.      Apparent Use of Probe and Response (P&R) Guide – Was there apparent use of the P&R Guide by the CSR?

5.      Complete Use of P&R Guide – Did the CSR address all appropriate probes according to the P&R Guide?

6.      Complete Response per P&R Guide – Did the CSR provide a complete response per the P&R Guide?

  1. Obtain/Determine Tax Law Facts – Did the CSR obtain/determine the appropriate facts on tax law issues not covered by the P&R Guide?

8.      Appropriate Procedural Action/Answer – Did the CSR take/provide the appropriate action/response?

9.      Input Form Order – Did the CSR input or prepare a form order for the taxpayer?

  1. Time Frames – Did the CSR explain required IRS time frames or deadlines to the taxpayer?

 

Appendix VIII

 

Management’s Response to the Draft Report

 

The response was removed due to its size.  To see the response, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.