TREASURY INSPECTOR GENERAL FOR TAX ADMINISTRATION

 

 

Data Used to Report Toll-Free Telephone Quality Are Reliable, but Internal Controls Need to Be Documented

 

 

 

July 21, 2006

 

Reference Number:  2006-40-107

 

 

This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.

 

Phone Number   |  202-927-7037

Email Address   |  Bonnie.Heald@tigta.treas.gov

Web Site           |  http://www.tigta.gov

 

 

July 21, 2006

 

 

MEMORANDUM FOR COMMISSIONER, WAGE AND INVESTMENT DIVISION

 

FROM:                             Michael R. Phillips /s/ Michael R. Phillips

                                          Deputy Inspector General for Audit

 

SUBJECT:                     Final Audit Report – Data Used to Report Toll-Free Telephone Quality Are Reliable, but Internal Controls Need to Be Documented (Audit # 200540008)

 

This report presents the results of our review to determine whether data used by the Wage and Investment Division to calculate the Toll-Free Accounts and Tax Law Customer Accuracy Rates were valid and reliable.

Synopsis

Each year, millions of customers call the Internal Revenue Service (IRS) toll-free telephone lines seeking assistance in understanding the tax law and meeting their tax obligations.  Toll-free telephone assistors provided over 26 million tax account and 5 million tax law services[1] during Fiscal Year 2005.  Because of the complexity of the tax law and the need to serve millions of customers with tax questions, the IRS divides the types of questions commonly asked into categories called “applications.”  Each application is staffed with a group of assistors that have received specialized training to assist customers with specific tax issues.  To ensure assistors provide top-quality service, the IRS subjects all telephone assistor contacts to numerous levels of review.  Quality reviews are used to measure employee, operational, and organizational performance to identify trends, problem areas, training needs, and opportunities for process improvements. 

Quality evaluations in the NQRS database are generally valid and reliable but internal control processes need to be documented to ensure effective and efficient operations.

Organizational performance results are housed in the National Quality Review System (NQRS) database.  The IRS uses data from the NQRS database to report Toll-Free Accounts and Tax Law Customer Accuracy Rates to external stakeholders.  The Centralized Quality Review System (CQRS) function has implemented a variety of processes and internal controls to ensure consistent, valid, and reliable data are input to the NQRS database.  These internal controls include numerous activities that help ensure management directives are carried out and program objectives are met.  Analysis of a sample of 114 Data Collection Instrument records, transcripts, and supporting documentation showed that, generally, the quality evaluations entered into the NQRS database by the CQRS function reviewers are valid and reliable.

Although internal controls are sufficient to ensure the overall quality results of the NQRS database, not all internal controls are documented.  Processes have been developed over time and need to be documented.  For example, the CQRS function Program Manager has established numerous reviews that provide critical internal controls to ensure the consistency of coding and accuracy of the information entered into the NQRS database.  The review requirements or procedures for the reviews have not been documented.  If the Program Manager leaves the CQRS function or the organizational structure changes, there will be no centralized and documented source for ensuring identification and continuation of established processes, which could affect the efficiency and effectiveness of the CQRS function operations.

In addition, in June 2006, the CQRS function plans to implement Contact Recording[2] for call segments; recordings will be retained only at the call site and generally will be erased within 6 months.  Recordings will replace transcripts, and transcripts will no longer be retained.  Without a longer retention period for the recordings, the performance measures for Toll-Free Accounts and Tax Law Customer Accuracy Rates will generally not be auditable.

There are currently no clear guidelines on whether the transcripts or recordings used for reporting quality come under the Federal Government record retention requirements.  The IRS is also implementing Contact Recording in its Taxpayer Assistance Centers (local IRS offices that provide face-to-face tax assistance).  We plan to conduct a review of that implementation and will address whether transcripts and recordings meet the definition of a record that is subject to Federal Government retention requirements.  Therefore, we are making no recommendations regarding record retention at this time.

Recommendation

The Commissioner, Wage and Investment Division, should ensure internal controls are centrally documented to ensure the quality review system continues effective and efficient operations.  

Response

IRS management reviewed the draft report and agreed with our recommendation to ensure internal controls are centrally documented and indicated the CQRS Handbook was revised on May 30, 2006.  Management’s complete response to the draft report is included as Appendix V.

Copies of this report are also being sent to the IRS managers affected by the report recommendation.  Please contact me at (202) 622-6510 if you have questions or Michael E. McKenney, Assistant Inspector General for Audit (Wage and Investment Income Programs) at (202) 622-5916.

 

 

Table of Contents

 

Background

Results of Review

Processes Are in Place to Ensure Data Used to Support Toll-Free Accounts and Tax Law Quality Measures Are Valid and Reliable, but They Need to Be Documented

Recommendation 1:

Contact Recording Is Being Implemented, and Transcripts Will No Longer Be Retained

Appendices

Appendix I – Detailed Objective, Scope, and Methodology

Appendix II – Major Contributors to This Report

Appendix III – Report Distribution List

Appendix IV – National Quality Review System Data Collection Instrument

Appendix V – Management’s Response to the Draft Report

 

 

Background

 

Each year, millions of customers call the Internal Revenue Service (IRS) toll-free telephone lines seeking assistance with understanding the tax law and meeting their tax obligations.  Toll-free telephone assistors provided over 26 million tax account and 5 million tax law services[3] during Fiscal Year 2005.  Assistors answer customer questions involving tax law and other related tax account conditions such as refunds, balance-due billing activity, and changes to the amount of taxes owed.  For example, if a customer calls to find out where to mail a tax return, the call would be routed to an assistor who has been trained to handle IRS procedural issues.  If a customer calls to find out the taxes due on a business tax account, the call would be routed to an assistor who handles balance‑due questions for the business customer.

Embedded Quality is a way of doing business that builds commitment and capability of individuals to continually improve customer service, employee satisfaction, and business results.

The IRS uses the Balanced Measurement System to measure organizational and employee performance.  The Balanced Measurement System satisfies the strategic planning and performance measurement requirements of the Government Performance and Results Act of 1993.[4]  In support of the Balanced Measurement System, the IRS uses a quality measurement system called Embedded Quality, which links employee performance to organizational results related to the quality of customer service.  The Accounts Management function uses the Embedded Quality system to report Customer Accuracy, Professionalism, and Timeliness measures to the IRS Commissioner as part of the Balanced Measurement System.  These measures are used at the strategic level to assess the effectiveness and overall progress of the IRS organization as a whole in achieving its strategic goal of improving customer service.  In addition, the results are used as support for the budgeting process. 

The Customer Accuracy measure is also reported externally to IRS stakeholders (e.g., Congress and the Government Accountability Office) and as part of the reporting requirement of the Government Performance Results Act.  The Regulatory Accuracy and Procedural Accuracy measures are reported internally to IRS management to identify trends and training opportunities.  These quality measures are a numeric indicator of the extent to which completed work meets prescribed standards.  Figure 1 provides details of how business results are reported to stakeholders.

Figure 1:  Business Results Reported to Stakeholders[5]

Figure 1 was removed due to its size.  To see Figure 1, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

 

The Embedded Quality system consists of two parts:  employee performance and organizational results.  The Centralized Quality Review System (CQRS) function in the Customer Account Services organization is responsible for collecting the performance data.  Employee performance results data are housed in the Embedded Quality Review System database.  The prime vehicle for collecting these data is through first-line manager reviews of Accounts Management function employees.  

Organizational performance results data are housed in the National Quality Review System (NQRS) database.  Data from the NQRS database are used to report Toll-Free Accounts and Tax Law Customer Accuracy Rates to external stakeholders.  To collect these data, reviewers in the CQRS function:

 

First.                    Monitor a statistical sample of incoming call segments[6] to each call site by application.  

Second.          Document key points of the conversations.  Generally, procedures require reviewers to remotely monitor a statistical sample of calls using transcription call sheets, also called transcripts.  Figure 2 presents an example of a transcript.

Figure 2:  CQRS Call Transcript

Figure 2 was removed due to its size.  To see Figure 2, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

 

On the transcripts, reviewers will type or write general identifying information (e.g., date, call site, application monitored, call start and stop time, and the record number called the Data Collection Instrument (DCI) Number[7]).  Reviewers will also document the interaction between the assistors and customers. 

Third.                 Perform the necessary research using appropriate references and tools to determine whether assistors followed guidelines. 

Fourth.             Create new DCI records on the NQRS database to record and document whether the assistors provided accurate answers.  The DCI records capture general information for identification purposes (DCI Header section) to identify and evaluate compliance with quality standards (DCI Attribute section) and to document synopses of the customers’ requests, assistors’ responses, and overall results (DCI Remarks section).  See Appendix IV for an example of a DCI record. 

Fifth.                  File the transcripts and related research materials to be retained for subsequent reviews.  Each reviewer attaches the transcript and related printouts of any research performed to a daily information sheet that lists all completed calls reviewed.  These documents are filed and retained in the group for the current and prior fiscal year and are available for subsequent reviews, including reviews in response to a rebuttal of results. 

The five quality measures are:  Timeliness, Professionalism, Customer Accuracy, Regulatory Accuracy, and Procedural Accuracy.

The NQRS database uses attributes to calculate five quality measures:  Timeliness, Professionalism, Customer Accuracy, Regulatory Accuracy, and Procedural Accuracy.  Each attribute is coded as “meeting the quality standard,” “not meeting the quality standard,” or “not appropriate.”  Each coded attribute is the equivalent of an opportunity.  When an attribute is coded as not meeting the quality standard, it is considered a defect.  Coding of the Regulatory and Procedural Accuracy Attributes determines the coding results of the single Customer Accuracy Attribute (correct/complete response/resolution).  Although a defect may be charged to a Procedural or Regulatory Accuracy Attribute, it may not always result in a Customer Accuracy Attribute defect. 

The Customer Accuracy Attribute is based on whether the assistor provided the correct/complete response/resolution to the issue and, if appropriate, took the necessary actions to provide the response or resolution.  The reviewer charges a defect if one of the following is present:

·         The action or response by the assistor is in direct conflict with established IRS guidelines and procedures.

·         The reviewer can definitively determine whether the customer was negatively affected by the assistor’s response.  

The NQRS Remarks section on the DCI record provides a synopsis of the customer’s issue, the employee’s actions/response, and the evaluative feedback.  The volume of text entered in this field is systemically limited; therefore a synopsis of only the key points of the call should be entered.  The reviewer must provide feedback on any attribute coded as a defect and identify the specific Internal Revenue Manual reference citing the error.

Results from the CQRS function’s independent review of a statistical sample of calls are rolled into a weighted national accuracy report, the Weighted Customer Accuracy Report, to provide an assessment of the customer service provided by the assistors responding to customer questions on the toll-free tax accounts and tax law telephone lines.  The Fiscal Year 2005 NQRS Weighted Customer Accuracy Reports shows the CQRS function reviewers sampled and monitored 35,336 toll-free tax account call segments and evaluated 32,527 tax account call segments for accuracy.  The accuracy rate was weighted by call site and projected over a population of 26,067,230 tax account call segments to produce the Customer Accuracy Rate of 91.5 percent.  In addition, the CQRS function reviewers sampled 14,483 toll-free tax law call segments and evaluated 12,041 tax law call segments for accuracy.  The accuracy rate was weighted by call site and projected over a population of 5,026,187 tax law call segments to produce the Customer Accuracy Rate of 89.0 percent. 

This review was performed in the Customer Account Services organization of the Wage and Investment Division in Atlanta, Georgia, and the Centralized Quality Review Site in Philadelphia, Pennsylvania, during the period April 2005 through March 2006.  The audit was conducted in accordance with Government Auditing Standards.  Detailed information on our audit objective, scope, and methodology is presented in Appendix I.  Major contributors to the report are listed in Appendix II.

 

 

Results of Review

 

Processes Are in Place to Ensure Data Used to Support Toll-Free Accounts and Tax Law Quality Measures Are Valid and Reliable, but They Need to Be Documented

The CQRS function has implemented a variety of processes and internal controls to ensure consistent, valid, and reliable data are input to the NQRS database.  These internal controls include numerous activities that help ensure management directives are carried out and program objectives are met. 

Our review of a sample of 114 DCI records, transcripts, and supporting documentation showed CQRS function reviewers’ overall quality evaluations (correct/complete response/resolution) entered into the NQRS database are valid and reliable.  The NQRS database accurately reflected the reviewers’ ratings of their calls for 105 (92 percent) of the 114 DCI records reviewed.  Specifically, the transcripts associated with the DCI records provided the necessary information to identify the key events of the conversations and supported the overall quality attributes entered on the NQRS DCI records reviewed.  For 9 (8 percent) of the 114 DCI records sampled, either the DCI record or transcript contained errors, insufficient information to support the evaluation, or opportunities to provide better customer service.  However, only two of these related to accuracy errors.  This error rate would not significantly affect the reliability of the statistics reported by the IRS on the accuracy of toll-free responses.  Although we are making no recommendations regarding the errors, our concerns with the sample cases reviewed are as follows:

·         Two DCI records were not properly coded to reflect customer accuracy errors.  In both cases, the customer was not provided a correct and/or complete response.  For one DCI record, the CQRS function reviewer accurately identified and coded the appropriate Procedural Accuracy Attributes with defects but did not code the Customer Accuracy Attribute with a defect.  For the other DCI record, the CQRS function reviewer did not accurately identify and code the Procedural Accuracy Attributes and Customer Accuracy Attribute as errors.

·         Four transcripts did not contain enough information to definitively conclude the assistor completely and accurately responded to the customers’ needs.  Because we were not provided access to the recordings and we also did not monitor the calls, we had to rely exclusively on call transcripts to determine if the DCI records were accurate.  The CQRS function reviewers cannot transcribe the calls verbatim.  Some CQRS function reviewers transcribe more information than others.  Some reviewers have the Probe and Response Guide (the Guide)[8] with them when they monitor the calls and identify whether probes or responses are/are not addressed while they are monitoring; other reviewers only listen and transcribe, choosing to research the Guide after the calls.  Some reviewers listen and document at the same time, while others document everything after the call.  Some reviewers transcribe almost every detail, while others limit their transcription to what they consider significant. 

·         Three transcripts showed the assistor could have provided better customer service.  The Embedded Quality process does not take into account the degree or level of service provided to a customer.  If the assistor complies with required IRS guidelines, the CQRS function reviewer records no defect and quality service is considered provided.  Our review identified specific instances where this assumption was not always correct.  For example, one transcript documented a customer asking for a Form 1040[9] and a form for dividends.  The assistor prompted the customer by referring to Schedule D.[10]

We believe the assistor should have asked the customer what type of dividends he or she was reporting before providing the schedule number.  Dividends are reported on either Interest and Ordinary Dividends (Schedule B) or Schedule D.  CQRS function management believes that, because the customer agreed to what the assistor provided, not once but twice, the customer was satisfied.  The customer will not know whether the correct form was received until he or she receives the form and completes the tax return.

In addition, 13 DCI records contained Procedural Attribute errors and 24 DCI records had errors in the DCI Header Section.  However, these Procedural Attribute errors did not affect the final Customer Accuracy Attribute.[11]  CQRS function management stated the general call information is not the main focus of their review and guidelines probably should be clarified for the selection of some header fields.  The errors in the DCI Header section also do not affect the accuracy of the calls.  Nevertheless, these database errors may produce doubt as to the reliability of the NQRS database and its use for other reporting purposes and analyses.

The calls sampled and monitored were statistically selected, were representative of the total population of services provided in each call site, and produced valid projections

The IRS Office of Statistics of Income (SOI) helps the CQRS function select a statistical sample of calls with which to report accuracy results.  It uses an algorithm to determine a statistically valid sample size for each call site and call product line (tax accounts or tax law).  The sample is further stratified by date and time for each reviewer.  CQRS function management appropriately followed the Office of SOI sampling plan and implemented an additional sampling plan by application to ensure the call site sample also reflected the types of services provided by the call site.  CQRS function management adhered to and exceeded the Fiscal Year 2005 sampling plan by evaluating 32,527 tax account and 12,041 tax law call segments for accuracy. 

Established processes and internal controls ensure consistent and reliable results are documented in the NQRS database

The Customer Account Services organization’s quality assurance manual provides procedures for program quality reviews, call site quality reviews, and front-line manager evaluative employee reviews.  The guidance is very broad to allow for all review levels.  Specific day-to-day guidelines are not provided in the manual.  For example, internal controls include:

·         Monitoring adherence to the Office of SOI call site sampling plan provided and implementing an additional sampling plan by application to ensure evaluated call segments are in line with call traffic by service provided/application.

·         Ensuring NQRS database integrity by limiting access to the database and properly establishing user profiles to ensure appropriate capabilities and authorizations for creating, editing, and deleting DCI records. 

·         Developing a web site for CQRS function managers and reviewers that contains the CQRS function’s internal policies and procedures, Office of SOI information, application schedules, monitoring instructions, general and specific attribute coding guidelines, consistency exercises, group meeting minutes, forms, and training links.

·         Establishing various managerial reviews of individual reviewers to ensure compliance with the sampling plan and attribute coding consistency by performing online or joint monitoring with the reviewer and closed case reviews (similar to our review of comparing the transcripts to the DCI records). 

Although internal controls are sufficient to ensure the overall quality results of the NQRS database, not all internal controls are documented.  Numerous processes have been developed over time and need to be documented.  For example, the CQRS function Program Manager has established numerous reviews that provide critical internal controls to ensure the consistency of coding and accuracy of the information entered into the NQRS database.  The review requirements or procedures for the reviews have not been documented.  If the Program Manager leaves the CQRS function or the organizational structure changes, there will be no centralized and documented source with which to ensure identification and continuation of established processes.  This could affect the efficiency and effectiveness of CQRS function operations. 

According to the Government Accountability Office, internal controls and all transactions and other significant events need to be clearly documented, and the documentation should be readily available for examination.  The documentation should appear in management directives, administrative policies, or operating manuals and may be in paper or electronic form.  All documentation and records should be properly managed and maintained.[12] 

The CQRS function has used the method of evaluating telephone operations quality through the use of coding attributes since 1998.  Documentation of internal controls and the experience and expertise gained by CQRS function management should be used for identification of best practices and made available to other functions as the Embedded Quality process is implemented by other functions and programs. 

Recommendation

Recommendation 1:  The Commissioner, Wage and Investment Division, should ensure internal controls are centrally documented to ensure the quality review system continues effective and efficient operations.

Management’s Response:  IRS management agreed with our recommendation and stated they completed the action on May 30, 2006, to document the additional managerial practices and internal controls in the CQRS Handbook.

Contact Recording Is Being Implemented, and Transcripts Will No Longer Be Retained

In June 2006, the CQRS function plans to implement Contact Recording[13] for call segments; recordings will be retained only at the individual call sites and generally will be erased within 6 months.  Recordings will replace transcripts, and transcripts will no longer be retained.  Without transcripts and/or a longer retention period for the recordings, the performance measures for Toll-Free Accounts and Tax Law Customer Accuracy Rates will generally not be auditable.

CQRS function management does not believe the recordings will be needed to support the NQRS database because the DCI records are being retained for 3 years.  However, DCI records do not provide sufficient information with which to validate the results of a live or recorded contact.  Without the paper transcripts or recordings, a third party will be unable to review or validate the performance measures obtained from the NQRS database.  In fact, CQRS function managers currently use reviewers’ transcripts when evaluating CQRS function reviews.  A paper transcript or the future recording will be needed to validate the information in the NQRS database. 

If Contact Recording had been operational in all call sites during Fiscal Year 2005, approximately 31 million call segments would have been recorded.  The CQRS function reviewers evaluated 32,527 tax account and 12,041 tax law call segments for Customer Accuracy.  Approximately 45,000 recordings out of the population of approximately 31 million recorded tax account and tax law call segments would need to be retained to support the 91.5 percent Toll-Free Accounts and 89.0 percent Toll-Free Tax Law Customer Accuracy Rates reported in Fiscal Year 2005.  The CQRS function currently has approximately 45,000 paper transcripts stored in its offices; those transcripts will be destroyed in October 2006.  Once Contact Recording is fully operational, CQRS function management plans to no longer create or maintain transcripts.

There are currently no clear guidelines on whether the transcripts or recordings used for reporting quality come under the Federal Government record retention requirements.  The IRS is also implementing Contact Recording in its Taxpayer Assistance Centers (local IRS offices that provide face-to-face tax assistance).  We will be conducting a review of that implementation and will address whether transcripts and recordings meet the definition of a record that is subject to Federal Government record retention requirements.  Therefore, we are making no recommendations regarding record retention at this time.

 

Appendix I

 

Detailed Objective, Scope, and Methodology

 

The overall objective of this review was to determine whether data used by the Wage and Investment Division to calculate the Toll-Free Accounts and Tax Law Customer Accuracy Rates were valid and reliable.  To accomplish our objective, we:

I.                   Determined the purpose of the Centralized Quality Review Staff (CQRS) function and its role in providing information for the Internal Revenue Service (IRS) Wage and Investment Division Strategy and Program Plan and the IRS Balanced Measurement System.[14]  

II.                Determined whether the sampling plan used by the CQRS function will provide statistically valid quality measures by obtaining and reviewing the Fiscal Year 2005 sampling plan, consulting with a contract statistician, and determining whether the plan was being followed.

III.             Determined whether the Data Collection Instrument (DCI) records[15] on the National Quality Review System (NQRS) database[16] accurately document the monitored calls so quality measures are accurately calculated. 

A.    Accessed the NQRS database to identify the population of 17,952 Accounts-General DCI records and 8,396 Tax Law-General DCI records that were used to calculate Customer Accuracy during the period January 1 through June 30, 2005, and extracted limited header information (DCI Number, CQRS Reviewer Number, Closing Date, and Review Date).

B.     Determined a minimum statistical sample size of 98 Accounts-General DCI records and 98 Tax Law-General DCI records using a sampling methodology similar to that of the IRS (90 percent confidence level, 90 percent expected accuracy rate, and +5 percent precision margin). 

C.     To identify the cases for each sample (Sample DCI Numbers), generated 125 random numbers between 1 and 17,952 to select the Accounts-General DCI records, generated another 125 random numbers between 1 and 8,396 to select the Tax Law-General DCI records, and obtained 250 corresponding call transcripts prepared by CQRS function reviewers.

D.    Evaluated the reliability of the selected DCI records and determined whether third-party reviewers have sufficient facts to arrive at the same conclusions by reviewing and comparing 114 transcripts to the related DCI Header, Attribute, and Remarks sections.  Transcripts were not reviewed in selection order rendering our sample judgmental.  Based on the results of reviewing a judgmental sample of 50 Accounts-General and 64 Tax Law-General transcripts and DCI records and identified internal controls established, we discontinued our review.

E.     Identified potential errors in transcripts and DCI records and discussed potential errors with CQRS function management to obtain agreement to the facts.

IV.             Evaluated CQRS function managerial oversight of the review process (i.e., what internal controls are in place to validate the results and ensure the reliability of the data).

A.    Determined the process used to assign reviewers to product lines and applications and whether the process adhered to the sampling plan.  We also determined how section managers ensured reviewers correctly documented DCI records and evaluated whether the process was effective to ensure valid results.

B.     Determined the procedures used to ensure adherence to the sampling plan (by reviewer, application, and call site).

C.     Determined the procedures used to control call site rebuttals and evaluated whether the process was effective to document changes to the NQRS database. 

D.    Determined the process used to modify and delete DCI records.

V.                Determined what NQRS database reports and processes are used to ensure reliable and valid results.

A.    Generated select reports, evaluated them for accuracy, and identified the specific DCI records that comprise data captured in the reports.

B.     Reviewed DCI records’ attributes to determine whether they were calculated correctly and rolled up into the reports.

C.     Traced the sampled DCI records (from Step III.C.) to determine whether they were captured correctly in respective reports. 

D.    Identified variances and discussed with program analysts the process used to correct discrepancies and determined the impact on the reliability of accuracy measures. 

VI.             Evaluated internal controls over the NQRS database.

A.    Identified the types of validity and maintenance tests performed and how program calculations were validated (e.g., customer accuracy rates, precision margin).

B.     Determined whether internal controls, all transactions, and other significant events were clearly documented and the documentation was readily available for examination.

 

Appendix II

 

Major Contributors to This Report

 

Michael E. McKenney, Assistant Inspector General for Audit (Wage and Investment Income Programs)

Augusta R. Cook, Director

Paula W. Johnson, Audit Manager

Lynn Faulkner, Lead Auditor

Jack Forbus, Senior Auditor

Sharon Shepherd, Senior Auditor

Jerry Douglas, Auditor

Kathy Henderson, Auditor

 

Appendix III

 

Report Distribution List

 

Commissioner  C

Office of the Commissioner – Attn:  Chief of Staff  C

Deputy Commissioner for Operations Support  OS

Deputy Commissioner for Services and Enforcement  SE

Chief, Agency-Wide Shared Services  OS:A

Chief Financial Officer  OS:CFO

Deputy Commissioner, Wage and Investment Division  SE:W

Associate Chief Financial Officer for Corporate Performance Budgeting OS:CFO:CPB

Director, Business Systems Planning, Wage and Investment Division  SE:W:BSP

Director, Customer Account Services, Wage and Investment Division  SE:W:CAS

Director, Strategy and Finance, Wage and Investment Division  SE:W:S

Chief, Performance Improvement, Wage and Investment Division  SE:W:S:PI

Chief, Program Coordination and Support, Wage and Investment Division  SE:W:CAS:PCS

Director, Accounts Management, Wage and Investment Division  SE:W:CAS:AM

Director, Joint Operations Center, Wage and Investment Division  SE:W:CAS:JOC

Chief Counsel  CC

National Taxpayer Advocate  TA

Director, Office of Legislative Affairs  CL:LA

Director, Office of Program Evaluation and Risk Analysis  RAS:O

Office of Management Controls  OS:CFO:AR:M

Audit Liaisons:

Chief, Agency-Wide Shared Services  OS:A:F

Chief Financial Officer  OS:CFO

Chief Information Officer  OS:CIO:SM:PO

Senior Operations Advisor, Wage and Investment Division  SE:W:S

 

Appendix IV

 

National Quality Review System Data Collection Instrument

 

The Centralized Quality Review Staff function is responsible for collecting data that will provide a basis for measuring and improving program effectiveness.  Organizational performance results data are housed in the National Quality Review System (NQRS) database.  Data from the NQRS database are used to report Toll-Free Accounts and Tax Law Customer Accuracy Rates to external stakeholders. 

Centralized Quality Review Staff function reviewers remotely monitor a statistical sample of incoming call segments to evaluate the accuracy of the actions taken and responses provided by assistors to customers and enter the results into the NQRS database using a Data Collection Instrument record.  The NQRS database uses attributes to calculate five quality measures:  Timeliness, Professionalism, Customer Accuracy, Regulatory Accuracy, and Procedural Accuracy. 

Figure 1 provides an example of a Data Collection Instrument record of a call relating to a request for tax forms.  The Quality Assurance Metrics section provides the calculation of the ratio of defects to opportunities.  For example, three Procedural Accuracy Attributes were coded (Attribute Code 003, 624, and 706 as shown on the last page).  The reviewer marked each attribute as “Yes,” indicating the assistor met each quality standard.  Therefore, the assistor received 100 percent for Procedural Accuracy. 

 

Figure 1 was removed due to its size.  To see Figure 1, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

 

Appendix V

 

Management’s Response to the Draft Report

 

The response was removed due to its size.  To see the response, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.



[1] A service is a telephone interaction between an assistor and a customer.  The call is answered by an assistor.  The customer’s question is addressed, so a service is provided.  If the same customer has an additional question or issue and is transferred to another area/assistor, an additional service is provided.

[2] Contact Recording captures and records employee/customer interaction, synchronized with computer screen activity, for replay and quality review.

[3] A service is a telephone interaction between an assistor and a customer.  The call is answered by an assistor.  The customer’s question is addressed, so a service is provided.  If the same customer has an additional question or issue and is transferred to another area/assistor, an additional service is provided.

[4] Pub. L. No. 103-62, 107 Stat. 285 (codified as amended in scattered sections of 5 U.S.C., 31 U.S.C., and 39 U.S.C.).

[5] The reference to “the Service” in Figure 1 refers to the IRS.

[6] The IRS manages call traffic by key segments in the “life” of a call (e.g., the segment from the time a screener answered the call to when it was transferred to an assistor or from the time an assistor connected to the customer to the time the customer was transferred or the call ended).

[7] The DCI Number is systemically generated from the NQRS database when the CQRS function reviewer creates a new record to input his or her evaluation of a monitored call.

[8] This Guide is used by assistors to respond to telephone tax law inquiries on the topics covered in the Guide. 

[9] U.S. Individual Income Tax Return (Form 1040).

[10] Capital Gains and Losses (Schedule D).

[11] Overall, 83 DCI records were fully supported by the transcripts, and 31 DCI records contained 1 or multiple accuracy (9), attribute (13), or header (24) errors.

[12] Standards for Internal Control in the Federal Government (GAO/AIMD-00-21.3.1, dated November 1999).

[13] Contact Recording captures and records employee/customer interaction, synchronized with computer screen activity, for replay and quality review.

[14] The Balanced Measurement System includes three measures balanced among customer satisfaction, employee satisfaction, and business results (comprised of work quantity and quality measures) and reflects the IRS’ mission and strategic goals and priorities. 

[15] A DCI is used to identify and document the quality standard attributes satisfied or not satisfied by the assistors to determine whether the answers provided were accurate.

[16] Organizational performance results are housed in the NQRS database.  The IRS uses data from the NQRS database to report Toll-Free Accounts and Tax Law Customer Accuracy Rates to external stakeholders.