The Tax Exempt and Government Entities Division’s Toll-Free Call Site Accuracy Rate and the Effectiveness and Efficiency of Call Site Operations Can Be Improved

 

September 2005

 

Reference Number: 2005-10-166

 

This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.

 

 

September 27, 2005

 

 

MEMORANDUM FOR COMMISSIONER, TAX EXEMPT AND GOVERNMENT ENTITIES DIVISION

 

FROM:                            Pamela J. Gardiner /s/ Pamela J. Gardiner

                                         Deputy Inspector General for Audit

 

SUBJECT:                     Final Audit Report - The Tax Exempt and Government Entities Division’s Toll-Free Call Site Accuracy Rate and the Effectiveness and Efficiency of Call Site Operations Can Be Improved  (Audit # 200510005)

 

This report presents the results of our review which evaluated whether the Tax Exempt and Government Entities (TE/GE) Division Customer Account Services (CAS) Quality Assurance (QA) function provides reliable customer accuracy rates and whether the TE/GE Division CAS function effectively uses the QA process as a basis for measuring and improving program effectiveness.  In addition, we evaluated whether the TE/GE Division CAS function Customer Service Representatives (CSR) complied with the statutory requirements to properly identify themselves and to prevent unauthorized disclosures when responding to customers’ telephone inquiries.   

In summary, the CAS QA function correctly reported all instances of improper disclosure and most instances (99.32 percent) of improper employee identification.  However, the customer accuracy rate for this period was overstated by approximately 6.1 percent (78.52 percent reported by the TE/GE Division CAS QA function versus 72.39 percent determined by our review).  For the additional five inaccurate responses we identified, the TE/GE Division CSRs did not follow established procedures or made statements that conflicted with written guidance.  One significant difference between the TE/GE Division CAS QA function’s review and our review was the method of listening to the sample of telephone calls.  The TE/GE Division CAS QA function reviewed live calls; we reviewed the same calls using the contact recording system, which is an automated recording system used widely in private industry to monitor customer contacts.  This technology enabled us to review the telephone calls several times to ensure we considered all of the questions and responses and to provide us the opportunity to research TE/GE Division CAS function procedures and Internal Revenue Service (IRS) publications for correct responses.

We also determined that CSRs could have provided enhanced customer service to the callers for seven additional telephone calls.  These seven telephone calls do not affect the accuracy rates but present an opportunity for the TE/GE Division CAS function to improve service to customers. 

Additional actions are also needed to increase the effectiveness and efficiency of the TE/GE Division CAS toll-free call site operations. Specifically, we determined the TE/GE Division CAS function should:

·         Compare the actual sample results to the sampling objectives shortly after the sampling period has ended to determine if adjustments to the sampling plans are needed.

·         Ensure issue codes for customers’ telephone inquiries are entered on the Aspect[1] system.

·         Develop plans to implement program improvements resulting from customer satisfaction surveys and monitor whether the changes improved customer satisfaction.

We recommended the Director, CAS, TE/GE Division, establish procedures for using contact recordings in the QA process; ensure the draft probe and response job aid for responding to customers’ inquiries about how to apply for tax-exempt status under Internal Revenue Code (I.R.C.) Section (§) 501(c)(3)[2] addresses the types of questions we identified for which enhanced customer service can be provided; and implement a process for developing probe and response guides that address the common types of customers’ questions.  We also recommended the Director, CAS, TE/GE Division, establish a coordinated process with the IRS Statistics of Income Division to compare the actual sample results to the sampling objectives shortly after the sampling period has ended, ensure issue codes are entered, evaluate the customer satisfaction survey results, develop action plans to improve service to customers, and monitor whether the changes resulted in improved customer satisfaction.

Management’s Response:  The Commissioner, TE/GE Division, agreed with the findings and recommendations in the report.  The Director, CAS, TE/GE Division, worked with the Statistics of Income Division contacts to develop a method to use contact recording beginning October 1, 2005.  The Director, CAS, TE/GE Division, has also developed and shared with employees a job aid for responding to customer’s inquiries about how to obtain recognition of tax-exempt status under I.R.C. § 501(c)(3), and in April 2005, an analyst was detailed to the QA function to analyze data and target topics for providing job aids.  In addition, the Director, CAS, TE/GE Division, will coordinate monthly with the Statistics of Income Division to compare actual sample results with sampling objectives to determine if adjustments are needed to the sample size.  TE/GE Division CAS function managers will reemphasize the input of issue codes.  The Director, CAS, has revised the scheduling of employee direct time and implemented new menu options on the toll-free line to respond to customer satisfaction results.  As an ongoing initiative, the CAS function forwards suggestions for topics or additional information to be put on the web (irs.gov) for enhancing web usage, and uses customer satisfaction results during workshops and in the development of the annual training plan.  Management’s complete response to the draft report is included as Appendix V.

Copies of this report are also being sent to the IRS managers who are affected by the report recommendations.  Please contact me at (202) 622-6510 if you have questions or Daniel R. Devlin, Assistant Inspector General for Audit (Headquarters Operations and Exempt Organizations Programs), at (202) 622-8500.

 

Table of Contents

Background

The Reliability of the Telephone Accuracy Rate Can Be Improved, and Opportunities Exist to Enhance Customer Service

Recommendations 1 and 2:

Recommendation 3:

Additional Actions Are Needed to Increase the Effectiveness and Efficiency of the Customer Account Services Toll-Free Call Site Operations

Recommendations 4 through 6:

Appendix I – Detailed Objectives, Scope, and Methodology

Appendix II – Major Contributors to This Report

Appendix III – Report Distribution List

Appendix IV – Outcome Measures

Appendix V – Management’s Response to the Draft Report

 

Background

The Tax Exempt and Government Entities (TE/GE) Division’s mission is to provide customers top-quality services by helping them understand and comply with applicable tax laws and to protect the public interest by applying the tax law with integrity and fairness to all. The TE/GE Division Customer Account Services (CAS) function assists customers of the Employee Plans (EP), Exempt Organizations (EO), and Government Entities (GE) functions and oversees the processing of TE/GE Division customer returns.  Toll-free telephone assistance is provided to TE/GE Division customers nationwide through the TE/GE Division CAS toll-free call site in Cincinnati, Ohio.

The TE/GE Division CAS toll-free call site assists customers who need:

·         Confirmation of the status of an application, including an Application for Recognition of Exemption Under Section 501(c)(3) of the Internal Revenue Code (Form 1023), an Application for Recognition of Exemption Under Section 501(a) (Form 1024), or an Application for Determination for Employee Benefit Plan (Form 5300).

·         A determination letter issued by the Internal Revenue Service (IRS) that an exempt organization’s application and supporting documents establish that it qualifies for tax-exempt status or that an employee plan’s terms conform to the requirements of Internal Revenue Code (I.R.C.) Section (§) 401(a).[3]

·         Answers to miscellaneous EO, EP, and GE function questions.  These questions commonly include issues such as requests to confirm tax exempt status, requests for guidance on the filing of EP and EO returns, questions resulting from notices and letters received by customers on issues related to an EP or EO return, and requests for guidance on entity changes.  

·         To speak to a TE/GE Division agent in another office regarding a pending determination application.

The TE/GE Division CAS Quality Assurance (QA) function measures the quality of service provided to customers through the TE/GE Division CAS toll-free call site by monitoring telephone calls.  During Fiscal Year (FY) 2004, the TE/GE Division CAS toll-free call site answered 506,572 telephone calls and the TE/GE Division CAS QA function reported a quality rating[4] of 90.4 percent for the service provided by the TE/GE Division CAS function assistors.

The TE/GE Division CAS QA function reviews calls to:

·         Identify errors that occurred in responding to customers’ telephone calls.

·         Analyze the reasons the errors occurred.

·         Identify and evaluate any error trends.

In addition, the TE/GE Division CAS QA function initiates corrective action to eliminate the cause of the error and follows up to ensure the corrective action was effective.

At the time of our review, the TE/GE Division CAS QA function conducted quality reviews by manually selecting sample telephone calls and evaluating the calls while they occurred.  The telephone calls were selected to achieve a statistically valid sample based on a schedule developed by the IRS Statistics of Income (SOI) Division.  The sample is designed with a 90 percent confidence level, a precision of +5 percent, and an expected error rate not to exceed 10 percent.

At the start of FY 2005, the CAS QA function implemented the Embedded Quality (EQ) effort to evaluate the quality of services provided to customers. The EQ measures are calculated based on the percentage of items addressed correctly within each of five elements:  Customer Accuracy, Regulatory Accuracy, Procedural Accuracy, Professionalism, and Timeliness.  The elements are defined as follows:

  • Customer Accuracy is giving the correct answer with the correct resolution. “Correct” is measured based upon the taxpayer receiving a correct response or resolution to his or her case or issue, and if appropriate, whether the assistor took the necessary case actions or case disposition to provide this response or resolution.
  • Regulatory Accuracy is adhering to statutory or regulatory process requirements when making decisions on taxpayer accounts or cases.
  • Procedural Accuracy is adhering to internal process requirements that are not required by statute or regulation.
  • Professionalism is promoting a positive image of the IRS by using effective communication techniques.
  • Timeliness is resolving an issue most efficiently using proper workload management and time use techniques.

The results of the TE/GE Division CAS QA function case analysis are documented on electronic review sheets called Data Collection Instruments and input to the IRS National Quality Review System.  Our review focused on two of the five elements, Customer Accuracy and Regulatory Accuracy (for the two aspects Proper Disclosure and Employee Identification).

This review was performed at the TE/GE Division CAS toll-free call site in Cincinnati, Ohio, in the CAS QA function during the period December 2004 through June 2005.  The audit was conducted in accordance with Government Auditing Standards.  Detailed information on our audit objectives, scope, and methodology is presented in Appendix I.  Major contributors to the report are listed in Appendix II.

The Reliability of the Telephone Accuracy Rate Can Be Improved, and Opportunities Exist to Enhance Customer Service

Based on analysis of the telephone calls reviewed by the TE/GE Division CAS QA function during the period November 2004 through January 2005, we determined the TE/GE Division CAS QA function correctly reported all instances of improper disclosure, most instances (99.32 percent) of improper employee identification, and that Customer Service Representatives (CSR) used clear and appropriate language when responding to customers’ telephone inquiries.  However, the customer accuracy rate reported for this period was overstated by approximately 6.1 percent (78.52 percent reported by the TE/GE Division CAS QA function versus 72.39 percent determined by our review; see Figure 1). The customer accuracy rate is a key method used to determine whether the TE/GE Division CAS toll-free call site is providing quality service to customers.  For seven additional telephone calls, we determined CSRs could have provided enhanced customer service to callers.  These seven calls do not affect the accuracy rates but present an opportunity for the TE/GE Division CAS function to improve service to customers.

In addition to the accuracy rates, we determined improvements are needed to increase the effectiveness and efficiency of the TE/GE Division CAS toll-free call site operations.  Specifically, the TE/GE Division CAS function should:

·         Compare the actual sample results to the sampling objectives shortly after the sampling period has ended to determine if adjustments to the sampling plans are needed.

·         Ensure issue codes for customers’ telephone inquiries are entered on the Aspect system.[5]

·         Develop plans to implement program improvements resulting from customer satisfaction surveys and monitor whether the changes improved customer satisfaction.

The actions listed above are presented in more detail later in the report.

Customer accuracy rates were overstated by 6.1 percent

During the period November 2004 through January 2005, the TE/GE Division CAS QA function reviewed a statistical sample of 149[6] customer telephone inquiries for customer accuracy.

 

Figure 1:  Comparison of TE/GE Division CAS QA Function and Treasury Inspector General for Tax Administration (TIGTA) Assessments of Incorrect or Incomplete Responses

 

Incorrect or Incomplete Response  per Sample

Percentage Incorrect[7]

Estimated Incorrect or Incomplete Response for Population[8]

CAS QA Function

32

21.48%

29,587[9]

TIGTA

37

27.61%[10]

38,040[11]

Difference

5

  6.13%

8,453

Source:  TE/GE Division CAS QA function records and TIGTA review of telephone calls.

 

The five additional calls involved three EO function customers inquiring about their tax-exempt status, one EO function customer inquiring about what activities an organization can engage in prior to receiving a formal determination of tax-exempt status, and one EP function customer inquiring about abatement of a proposed penalty.  In each of the five instances, the TE/GE Division CAS function CSRs did not follow established procedures or made statements that conflicted with written guidance.  We could not determine why the TE/GE Division CAS QA function did not identify these additional five calls; however, we believe if it was conducting its reviews using recorded calls instead of live calls, it would have had the opportunity to review each call several times and to stop a call if it needed to research a technical issue.  These 5 calls averaged 5.2 minutes in length, and 1 of the 5 contained more than 1 question by the caller.  In addition, the TE/GE Division CAS QA function reviews many other attributes to determine if the telephone call is answered in compliance with regulatory statutes and internal procedures.  These factors contributed to the TE/GE Division CAS QA function’s difficulty in reviewing live calls.

One significant difference between the TE/GE Division CAS QA function’s review and our review was the method of listening to the sample of telephone calls.  The TE/GE Division CAS QA function reviewed live calls; we reviewed 134 of the 149 calls review by the QA function using the contact recording system, which is an automated recording system used widely in private industry to monitor customer contacts.  In October 2004, the TE/GE Division CAS function started to use this system to record all customer calls answered.  The contact recording software records the actual call, including the caller’s questions, the CAS assistor’s responses, and any time the caller is placed on hold. This technology enabled us to review the telephone calls several times to ensure we considered all of the questions and responses and allowed us to research TE/GE Division CAS function procedures and IRS publications for correct responses.  TE/GE Division CAS function management advised us they are planning to use contact recording for the QA function beginning in the summer of 2005; they had not prepared any procedures at the time of our review.

CSRs can provide enhanced customer service to callers

For an additional 7 of the 148 (4.7 percent) telephone calls we reviewed, the CSRs provided accurate responses to customers but could have provided enhanced service and reduced taxpayer burden.  These seven calls included the following: 

  • Five telephone calls involved questions about how to become a tax-exempt organization and when tax-deductible contributions could be received. The CSRs could have provided enhanced service by more closely following published guidance when explaining the process for applying for recognition of tax-exempt status. 
  • One telephone call involved a question about whether an affiliated organization was included in the tax exemption of the national organization.  The CSR could have provided enhanced service by researching IRS records to determine if the national organization had a group exemption instead of instructing the caller to check with the national organization about a group exemption.
  • One telephone call involved a question about commercial software used to file some EO function returns.  The CSR could have provided enhanced service if clearer guidance existed about the software referred to on the IRS web site, IRS.gov.

The five customers calling for tax-exempt status did not appear to know the tax law regarding I.R.C. § 501(c)(3).[12]  IRS publications advise that most organizations wanting to be recognized by the IRS as an I.R.C. § 501(c)(3) public charity or private foundation must file a Form 1023 within 15 months of their creation.[13]  The TE/GE Division CAS function CSRs’ responses were technically correct in that a tax-exempt organization must file an application to receive a ruling or determination letter from the IRS recognizing its tax-exempt status. However, a customer may misunderstand that all tax-exempt organizations are required to immediately file the application and that the organization cannot conduct any activities until a determination is received, which may increase the burden on the organization. The TE/GE Division believes it is important for customers to realize that, until the IRS determines an organization is exempt from Federal income tax under I.R.C. § 501(c)(3), potential contributors do not have assurance that contributions to that organization will be tax deductible.  We believe explaining when to file the Form 1023 and which organizations are not required to file may help to reduce taxpayer burden for between 1,290 and 8,017[14] TE/GE Division EO function customers. 

One tool that would benefit TE/GE Division CAS function CSRs in responding to customers consistently and timely is a probe and response guide, which would allow assistors to research and respond to common questions from customers.  Similar guides were developed and used by other IRS toll-free telephone call sites starting in 1993.  The guides were created to be a simple, easy-to-use tool to guide CSRs through the IRS’ complex maze of tax laws, rules, and procedures.  They were designed to enable CSRs to perform their duties more precisely, correctly, and efficiently.  The guides were also designed to help eliminate errors made by CSRs, as identified by managerial and quality reviewers. 

In June 2000 (shortly after the TE/GE Division was created), we reported[15] that the TE/GE Division did not have sufficient guidelines to assist CSRs in responding to customer inquiries.  We recommended the Commissioner, TE/GE Division, improve the quality of customer service by completing guidelines for CSRs to use when responding to customers. In response to that report the Director, CAS, TE/GE Division, responded that the QA function staff for the TE/GE Division’s CAS call site will improve the desk guide and develop guidelines and other job aids similar to those used at other IRS division call sites.  During our current audit, TE/GE Division CAS function management advised us they started developing probe and response job aids at the beginning of FY 2003 and had developed job aids covering seven topics, which have been provided to the TE/GE CSRs to answer customer inquiries.  The TE/GE Division CAS function management recognized the need to develop other job aids for their CSRs.  In addition, the TE/GE CAS QA function started developing a draft guide to address a trend we identified early in our review regarding customers’ inquiries about applying for 501(c)(3) exempt organization status.  At the time of our review, the draft guide had not been released to the assistors.

Completing the implementation of contact recording processes and developing the necessary probe and response guides for TE/GE Division CAS function assistors will enable the TE/GE Division CAS toll-free call site to improve the accuracy of responses to customers’ telephone inquiries. 

Recommendations

1.      The Director, CAS, TE/GE Division, should establish procedures for using contact recordings in the quality assurance process.

Management’s Response:  The Director, CAS, TE/GE Division, worked directly with the established SOI Division contacts to develop a method to use contact recording and apply both a random start and random sampling for TE/GE CAS reviews of telephone contacts for the beginning of FY 2006.

2.      The Director, CAS, TE/GE Division, should ensure the draft probe and response job aid, for responding to customers’ inquiries about how to apply for tax-exempt status under I.R.C. § 501(c)(3), addresses the types of questions we identified for which enhanced customer service can be provided.

Management’s Response:  The Director, CAS, TE/GE Division, developed and shared with employees in August 2005 the probe and response guide (job aid) for tax‑exempt application inquiries.  The job aid includes information on how to obtain recognition of tax-exempt status, and which forms and publications are used by different organizations.

 

3.      The Director, CAS, TE/GE Division, should implement a process for developing probe and response guides that address the common types of questions customers have.

Management’s Response:  The Director, CAS, TE/GE Division, has developed several probe and response guides to address the common types of questions customers have.  The development of these job aids has been and will continue to be an ongoing process. In April 2005, a former systems analyst was detailed indefinitely to the Quality staff to analyze all available data, target topics for providing job aids, and improve overall quality of work and work processes.

Additional Actions Are Needed to Increase the Effectiveness and Efficiency of the Customer Account Services Toll-Free Call Site Operations

We determined additional actions are needed to increase the effectiveness and efficiency of the TE/GE Division CAS toll-free call site operations.  Specifically, the TE/GE Division CAS function should reevaluate the sample size needed for QA function review to determine if a smaller sample is justified.  This could result in more efficient use of resources as TE/GE Division CAS QA function reviewers may be able to review fewer telephone calls. In addition, issue codes indicating the type of customer calling and the topic of the call should be input to the system for all closed telephone calls to enable TE/GE Division CAS function management to identify trends in customers’ telephone inquiries. Finally, the TE/GE Division CAS QA function should develop action plans to implement program improvements indicated by customer satisfaction surveys and monitor whether the changes resulted in improved customer satisfaction.

Actual sample results should be periodically compared to the sampling objectives

The TE/GE Division CAS QA function does not compare the actual sample results to the sampling objectives shortly after the sampling period has ended.  Thus, TE/GE Division CAS function management is unable to determine if the sample size should be reevaluated.  When conducting a statistical sample, it is necessary to make an assumption as to the rate of occurrence of errors in the population.  Since the actual results of a sample will normally differ from the initial assumptions, it is essential that the sample results are appraised after the sampling is completed to determine if the sample results can be projected to the population with the same estimated sampling precision (e.g., +5 percent).  The TE/GE Division CAS function management relies on the IRS SOI Division for statistical sampling advice and was not aware that a sample needs to be reevaluated after the results are determined. 

For example, the TE/GE Division CAS function samples for FYs 2004 and 2005 are based on an expected occurrence rate (error rate) of 10 percent, a 90 percent confidence level, and a sampling precision of +5 percent.  During FY 2004, the actual error rates reported by the TE/GE Division CAS QA function ranged from 5.9 to 13.0 percent.  Using the reported TE/GE Division telephone customer quality ratings to appraise the sample results, we determined the actual sample precision obtained ranged from 3.52 to 4.91 percent, with an average of 4.12 percent for FY 2004.  This is well below the precision rate of +5 percent the TE/GE Division CAS function had established.  With a smaller error rate, fewer cases would have to be sampled to still maintain the precision rate of +5 percent.  We estimate that, if the TE/GE Division CAS QA function had reevaluated the sample size in FY 2004 in light of the prior month’s error rate, it could have reviewed 360 (20 percent) fewer cases, which could have provided resources for other TE/GE Division CAS function programs.

Conversely, for the 3-month period November 1, 2004, through January 31, 2005, the actual error rate was greater than the expected error rate, resulting in a sample precision of +5.85 percent.  Because the sample precision exceeded the established 5 percent by almost an entire percentage point, the usefulness of the information to TE/GE Division management may be diminished.  In this instance, a larger sample would have to be taken to maintain the desired precision level.  Appraising the sample results provides essential information about whether the sampling plan is achieving its objectives and whether resources are being effectively used.  Our discussions with IRS SOI Division personnel indicated they agree actual precision rates should be computed after the end of a sampling period.

The TE/GE Division CAS function should ensure issue codes are entered for customers’ telephone inquiries

The TE/GE Division CAS function established issue codes to be able to trend the different types of customer inquiries.  This assists TE/GE Division CAS function management in identifying new or frequent customer inquiries (which could be an indicator of the need for additional training) as well as other types of problems, such as erroneous notices generated by TE/GE Division customer returns processing.  Examples of the different issue codes include one code that is used when an EO function customer calls about the status of its Form 1023 and a different code that is used when an EO function customer calls about a penalty notice.  Although TE/GE Division CAS function procedures instruct the assistor to input to the Aspect system the issue code most applicable to the call when closing a telephone call, issue codes are frequently not entered since the entering of issue codes is not part of the quality review evaluation process for CSRs.  In FY 2004, 102,665 (21.27 percent) of the TE/GE Division customer telephone inquiries were not coded for an issue.  This problem continued in the first quarter of FY 2005, with 39,461 (31.35 percent) of the TE/GE Division customer telephone inquiries not having an issue code.

The correct and consistent use of issue codes is important to gauge the impact of events on the TE/GE Division CAS toll-free call site operations.  For example, on Friday, October 22, 2004, numerous notices were sent out to various tax-exempt organizations.  By Monday, October 25, 2004, the TE/GE Division CAS function had added two new issue codes to assess the impact of these notices on TE/GE Division CAS toll-free call site operations.  In FY 2004, all EO function customer telephone inquiries regarding a notice inquiring about filing a return were coded under a single issue code.  In October 2004, EO function customers calling about the same notice were separated into two issue codes–one for customers receiving a first notice about filing a return and one for customers receiving a fourth notice about filing a return.  If some of the uncoded calls in FY 2004 and first quarter of FY 2005 related to these notices, TE/GE Division CAS function management would not have a complete picture of the impact of these telephone inquiries on TE/GE Division CAS toll-free call site operations.  Calls that are not coded for an issue could negatively affect TE/GE Division CAS function management’s ability to identify trends, training needs, and systemic problems.

Plans are needed to implement changes recommended by customer satisfaction surveys

During FY 2004, the Pacific Consulting Group determined that the TE/GE Division CAS function should work on two primary areas:

·         Reduce the amount of time it takes to complete a customer’s call once the customer gets through to a TE/GE Division CAS function assistor.

·         Improve issue resolution. 

We determined that the time to complete a call is increasing and that customers and TE/GE Division employees who administer the customer satisfaction survey have different opinions about whether a customer’s issue was resolved by the end of the telephone call.

The TE/GE Division contracts with the Pacific Consulting Group to conduct customer satisfaction surveys. This research is part of the IRS agency-wide initiative to monitor and improve taxpayer satisfaction with the service provided. Some of the objectives of this study are to identify areas in which TE/GE Division CAS toll-free call site staff and managers can improve customer service and to track progress in improving customer satisfaction with TE/GE Division CAS toll-free call site service.

In each quarter of FY 2004, the consultant recommended the TE/GE Division CAS function focus improvement on the time it takes to complete a customer’s call once the customer gets through to an assistor.  From National Quality Review System data for TE/GE Division customer telephone inquiries reviewed during FY 2004, we determined the average time for an assistor to gather information and respond to the customer was 6.4 minutes.  The results from the first 4 months of FY 2005 showed telephone calls were averaging much longer, with the average being 10.3 minutes compared to an average of 7.1 minutes for the same period in FY 2004.

In addition, the consultant advised that TE/GE Division CAS function management should strive to improve issue resolution.  The customer and the TE/GE Division CAS function quality analyst who monitors the call for quality control and sampling purposes often hold different opinions about whether the issue is resolved.  These differing perspectives can include instances where the customer thinks the issue is resolved, but the monitor thinks it is not resolved, and instances where the customer thinks the issue is not resolved, but the monitor thinks the issue is resolved.  While data showed fewer customers disagreed about issue resolution in the first quarter of FY 2005 when compared to any quarter in FY 2004, the consultant continued to advise that TE/GE Division CAS function management focus improvement on issue resolution.

 

Figure 2:  Percentage of Calls in Which Customer and Monitor Disagreed on Issue Resolution

Figure 2 was removed due to its size.  To see Figure 2, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

 

The TE/GE Division CAS QA function reviews the customer satisfaction survey and develops tips or ideas intended to improve customer satisfaction.  These tips are distributed to TE/GE Division CAS function managers, and it is suggested by the QA function that the tips be shared with CSRs. However, because the TE/GE Division CAS function does not have established procedures for the use of customer satisfaction surveys and for monitoring the survey results for any program improvements over time, the managers are not required to take any actions regarding the suggested tips.  While the current process may provide more awareness of service for which TE/GE Division customers are less than satisfied, it may not result in improved customer satisfaction.  We believe a more formal process, which includes developing action plans to address issues raised in the customer satisfaction surveys and monitoring the survey results over time, would provide better assurance of increasing customer satisfaction.

The IRS Communications and Liaison Division developed a guide entitled Using Customer Satisfaction Survey Data to Develop Improvement Plans.  This guide describes a series of steps for this process, including facilitating a group meeting to understand the data, determine priorities, consider alternative solutions, and develop implementation plans to make better use of customer satisfaction data.  Another step in this process is developing an action plan that is specific, measurable, achievable, realistic, and timely.  The action plan should identify the steps to be taken to improve the customer service, the person(s) responsible for implementing those steps, a timeline for completing each action, and a method for monitoring whether the changes resulted in improved customer satisfaction.  A process that includes a specific and measurable plan may enable TE/GE Division CAS function management to focus on specific improvements to customer satisfaction survey needs. 

Recommendations

4.      The Director, CAS, TE/GE Division, should establish a coordinated process with the SOI Division to compare the actual sample results to the sampling objectives shortly after the sampling period has ended, to determine if the sample size should be reevaluated for future periods. 

Management’s Response: The Director, CAS, TE/GE Division, will coordinate with the SOI Division on a monthly basis to compare the actual sample results to the sampling objectives to determine if the sample needs to be adjusted (up or down) for the next sampling period to stay within the 5 percent precision margin.

5.      The Director, CAS, TE/GE Division should ensure issue codes are entered for customers’ telephone inquiries.

Management’s Response: TE/GE Division CAS function managers will reemphasize the input of issue codes for use in establishing training needs and for providing helpful feedback to other functions within the TE/GE Division.  If an employee’s issue code input is low, the manager now provides written direction to code calls appropriately.  The managers receive and review reports on the input of issue codes on a weekly basis.  Managers are directed to discuss the results of these reports with their employees, as needed. 

6.      The Director, CAS, TE/GE Division, should evaluate the customer satisfaction survey results, develop action plans to improve service to customers, and monitor whether the changes resulted in improved customer satisfaction in future surveys.

Management’s Response: The Director, CAS, TE/GE Division, has revised the scheduling of employee direct time and implemented new menu options on the toll-free line to respond to customer satisfaction results.  As an ongoing initiative, the TE/GE Division CAS function forwards suggestions to the TE/GE Division Customer Education and Outreach offices for topics or additional information to be put on the web (irs.gov) for enhancing web usage and possibly reducing the call volume, thereby improving customer satisfaction.  The CAS uses customer satisfaction results during the development of the annual training plan and during workshops.

 

Appendix I

 

Detailed Objectives, Scope, and Methodology

 

The overall objectives of this audit were to determine whether the Tax Exempt and Government Entities (TE/GE) Division Customer Account Services (CAS) Quality Assurance (QA) function provides reliable customer accuracy rates and whether the TE/GE Division CAS function effectively uses the quality assurance process as a basis for measuring and improving program effectiveness.  In addition, we determined whether TE/GE Division CAS function CSRs complied with the statutory requirements to properly identify themselves and to prevent unauthorized disclosure when responding to customers’ telephone inquiries. 

To accomplish these objectives, we:

I.                     Determined whether TE/GE Division CAS function CSRs provided accurate and timely responses that complied with applicable disclosure and identification statutes by evaluating the 163[16] TE/GE Division customer telephone inquiries selected for QA function review during the period November 1, 2004, through January 31, 2005.  Specifically, we:

A.    Determined whether TE/GE Division CAS function CSRs properly identified and addressed the customer’s issue(s), including the total number of questions asked, the number of questions answered correctly, and the number of questions answered incorrectly.

B.     Determined whether TE/GE Division CAS function CSRs explained the process or actions taken and confirmed that the customer understood the issue covered.

C.     Determined whether TE/GE Division CAS function CSRs used clear and appropriate language.

D.    Reviewed Data Collection Instruments and compared TE/GE Division CAS QA function personnel determinations of whether the customers received accurate responses with the results from the Treasury Inspector General for Tax Administration’s (TIGTA) review of the same cases.

E.     Determined whether TE/GE Division CAS function CSRs identified themselves by name and unique identifying number, as required by the Internal Revenue Service Restructuring and Reform Act of 1998.[17]

F.      Determined whether the customer’s authorization to receive information was properly determined in accordance with applicable statutes or TE/GE Division procedures.  

G.    Determined the length of the call and whether TE/GE Division CAS function CSRs handled the call timely by directing the conversation and minimizing extraneous dialog.

II.                     Determined whether the TE/GE Division CAS function effectively implemented its sampling plan by interviewing TE/GE Division CAS function personnel and reviewing applicable documentation regarding the sample selection process.  Specifically, we: 

A.    Interviewed TE/GE Division CAS function personnel and reviewed applicable documentation to determine the sampling methodology.

B.     Reviewed documentation of the sample selected during Fiscal Year (FY) 2004 and the first quarter of FY 2005 to determine whether the sample selections followed the sampling plan.

C.     Consulted with the TIGTA statistician to determine whether the samples reviewed by the TE/GE Division CAS QA function during FY 2004 and the first quarter of FY 2005 were statistically valid.

III.                     Determined whether the TE/GE Division CAS function uses the quality assurance process for customers’ telephone inquiries as a basis for measuring and improving program effectiveness. Specifically, we:

A.    Interviewed TE/GE Division CAS function personnel and reviewed applicable documentation for FY 2004 and the first quarter of FY 2005 to determine whether:

1.      The data collected identified defects resulting from call site, systemic, and taxpayer actions or inaction.

2.      Sufficient data were collected and analyzed to determine the reason(s) for defect occurrence.

3.      Sufficient data were collected and analyzed to identify defect trends.

B.     Interviewed TE/GE Division CAS function personnel and reviewed applicable documentation of corrective actions initiated from the quality assurance process for FY 2004 and the first quarter of FY 2005 to determine whether the actions have affected the accuracy of responses.

C.     Analyzed customer satisfaction surveys for FYs 2000 through 2004 to determine whether there were trends regarding the accuracy of responses identified by the surveys and whether actions were taken as a result of feedback from the surveys.

 

Appendix II

 

Major Contributors to This Report

 

Daniel R. Devlin, Assistant Inspector General for Audit (Headquarters Operations and Exempt Organizations Programs)

Nancy A. Nakamura, Director

James V. Westcott, Audit Manager

Gregory W. Holdeman, Lead Auditor

Cheryl Medina, Senior Auditor

Michael R. Van Nevel, Senior Auditor

Donald Martineau, Auditor

 

Appendix III

 

Report Distribution List

 

Commissioner  C

Office of the Commissioner – Attn:  Chief of Staff  C

Deputy Commissioner, Services and Enforcement  SE

Deputy Commissioner, Tax Exempt and Government Entities Division  SE:T

Director, Customer Account Services, Tax Exempt and Government Entities Division  SE:T:CAS

Chief Council  CC

National Taxpayer Advocate  TA

Director, Office of Legislative Affairs  CL:LA

Director, Office of Program Evaluation and Risk Analysis  RAS:O

Office of Management Controls  OS:CFO:AR:M

Audit Liaison:  Director, Communications and Liaison, Tax Exempt and Government Entities

Division SE:T:CL

 

Appendix IV

 

Outcome Measures

 

This appendix presents detailed information on the measurable impact that our recommended corrective actions will have on tax administration.  This benefit will be incorporated into our Semiannual Report to the Congress.

Type and Value of Outcome Measure:

·         Reliability of Information – Actual; 6.13 percent difference between the Treasury Inspector General for Audit (TIGTA) and Internal Revenue Service assessment of inaccurate or incomplete responses (see page 4).

Methodology Used to Measure the Reported Benefit:

During November 2004 through January 2005, the Tax Exempt and Government Entities (TE/GE) Division Customer Account Services (CAS) Quality Assurance (QA) function reviewed a statistical sample of 149[18] customer telephone inquiries for customer accuracy.  We reviewed 134[19] of these calls and compared our results with TE/GE Division CAS QA function determinations of whether the customers received complete and accurate responses to their inquiries.  We identified five additional TE/GE Division customers that did not receive a complete and accurate response to their telephone inquiry. The five additional calls involved three Exempt Organizations (EO) function customers inquiring about their tax-exempt status, one EO function customer inquiring about what activities an organization can engage in prior to receiving a formal determination of tax-exempt status, and one Employee Plans function customer inquiring about abatement of a proposed penalty.  The customer accuracy rate reported for this period was overstated by approximately 6.1 percent (78.52 percent reported by the TE/GE Division CAS QA function versus 72.39 percent determined by our review; see Figure 1 on page 5). The customer accuracy rate is a key method used to determine whether the TE/GE Division CAS toll-free call site is providing quality service to customers. 

Type and Value of Outcome Measure:

·         Taxpayer Burden – Potential; 4,655 taxpayer accounts affected (see page 4).

 

Methodology Used to Measure the Reported Benefit:

During November 2004 through January 2005, we reviewed 148[20] of the TE/GE Division customers’ telephone inquiries for opportunities for the TE/GE Division CAS Customer Service Representatives (CSR) to provide enhanced customer service.  We determined that for 5 of the 148 (3.4 percent) telephone calls, the TE/GE Division CAS function assistors provided accurate responses to customers but could have provided enhanced service and reduced taxpayer burden.  The five telephone calls involved questions about how to become a tax-exempt organization and when tax-deductible contributions could be received.  The TE/GE Division CAS function assistors could have provided enhanced service by more closely following published guidance when explaining the process for applying for recognition of tax-exempt status.

The TE/GE Division CAS QA function selected the sample of telephone calls based on a schedule developed by the Internal Revenue Service Statistics of Income Division to achieve a statistically valid sample.  The sample was designed with a 90 percent confidence level, a precision of +5 percent, and an expected error rate not to exceed 10 percent. 

We projected the 3.4 percent of calls for which enhanced service could have been provided to the population of telephone calls answered by TE/GE Division CAS function assistors during this period (137,763) and determined the range of customers who were calling about becoming a tax-exempt organization.  Based on the results of our sample, we estimate the TE/GE Division CAS function assistors could have provided enhanced service to these customers. 

We determined the number of customers affected by using the United States Army Audit Agency Statistical Analysis System formula for sample precision.  Based upon our sample size of 148, from a population of 137,763, we estimate that 4,655 customers (3.3784 percent) could have received enhanced service if the TE/GE Division CAS function assistors had more closely followed published guidance when explaining the process for applying for recognition of tax-exempt status. 

The outcome measure is determined as follows:

3.3784 percent x 137,763 (population of customer telephone inquiries) = 4,655 customers.

 

Appendix V

 

Management’s Response to the Draft Report

 

The response was removed due to its size.  To see the response, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.



[1] The Aspect Automated Call Distributor’s primary purpose is to direct calls from taxpayers to IRS CSRs for assistance. 

[2] I.R.C. § 501(c)(3) (2004).

[3] I.R.C. § 401(a) (2004).

[4] Toll-free call site quality is the percentage of customers receiving accurate responses to their questions.  Evaluation of accuracy measures administrative accuracy (internal) and customer satisfaction (external).

[5] The Aspect Automated Call Distributor’s primary purpose is to direct calls from taxpayers to IRS CSRs for assistance. 

[6] The total number of calls reviewed was 163 total calls minus 14 calls that were transferred to another IRS division which are not used in calculating customer accuracy rates. 

[7] Percentages were rounded to two decimal places. Actual values were used in computing the estimated number of incorrect or incomplete responses.

[8] There were 137,763 TE/GE Division customer telephone inquiries answered during this period.

[9] The TE/GE Division CAS function confidence level is 90 percent and the sampling error is ±5.53 percent.  Sampling error is the number of errors that occur because only part of the population is directly contacted.  

[10] The total number of calls reviewed by the TIGTA was 134 (163 calls minus 15 recordings that were not available at the time of our review and 14 transferred calls not used in calculating customer accuracy rates).  The percentage of incorrect responses was calculated by dividing the number of incorrect responses identified by the TIGTA (37) by the total number of calls reviewed by the TIGTA (134).

[11] Our confidence level is 90 percent and the sampling error is ±6.35 percent.

[12] There are three key components for an organization to be exempt from Federal income tax under I.R.C. § 501(c)(3) (2004).  A not‑for‑profit (i.e., nonprofit) organization must be organized and operated exclusively for one or more exempt purposes, the most common of which are charitable, educational, and religious.

[13] There is an additional 12-month extension available for filing a Form 1023.

[14] See details in Appendix IV.

[15] Improvements to the Tax Exempt and Government Entities Division’s Telephone Operation Would Enhance Customer Service (Reference Number 2000-10-091, dated June 2000).

[16] The total number of calls included in the TE/GE Division CAS QA function calculation of the customer telephone inquiry accuracy rate was 149 (163 total calls reviewed minus 14 calls that were transferred to another Internal Revenue Service division).  The Treasury Inspector General for Tax Administration review included the 14 calls and determined that the transfer and the QA function’s review of these calls were in accordance with CAS procedures.  The total number of calls included in our calculation was 134 (163 total calls minus 14 transferred calls and 15 recordings that were not available at the time of our review).  There were 137,763 TE/GE Division customer telephone inquiries answered during this period. 

[17] Pub. L. No. 105-206, 112 Stat. 685 (codified as amended in scattered sections of 2 U.S.C., 5 U.S.C. app., 16 U.S.C., 19 U.S.C., 22 U.S.C., 23 U.S.C., 26 U.S.C., 31 U.S.C., 38 U.S.C., and 49 U.S.C.).

[18] The total number of calls included in the TE/GE Division CAS QA function calculation of the customer telephone accuracy rate was 149 (163 total calls reviewed minus 14 calls that were transferred to another IRS division which are not used in calculating customer accuracy rates).  There were 137,763 TE/GE Division customer telephone inquiries answered during this period. 

[19] The total number of calls reviewed by the TIGTA to calculate the customer accuracy rate was 134 (163 calls minus 15 recordings that were not available at the time of our review and 14 transferred calls not used in calculating customer accuracy rates).

 

[20] The total number of calls reviewed by the TIGTA for opportunities for enhanced service was 148 (163 calls minus 15 recordings that were not available at the time of our review).