Executive Summary

Objective and Scope

Background

Results

Conclusion

Appendix I – Detailed Objective, Scope, and Methodology

Appendix II – Major Contributors to This Report

Appendix III – Report Distribution List

Executive Summary

The Internal Revenue Service (IRS) audits approximately 1,700 of the largest and most complex taxpayers (primarily corporations) through its Coordinated Examination Program (CEP). Teams of senior revenue agents and specialists, which may include computer analysts, engineers, and economists, audit taxpayers in the CEP. Examinations include 2 or more tax years and require 18 or more months to complete.

The Peer Review is the IRS’ quality review process in the CEP. To be a value-added process, quality review should provide management with an independent assessment of the extent to which the organization complies with its standards, policies and procedures. This assessment should provide valuable feedback to top management on how well the organization achieves its quality goals.

Under the IRS’ new business unit structure, the Quality Assurance and Performance Management function in the Large and Mid-Size Business (LMSB) Division will quality review CEP examinations. The process for quality reviewing CEP examinations in the LMSB Division was recently drafted.

We evaluated the CEP Peer Reviews to determine whether the process was efficient and effective in improving the CEP.

The CEP Peer Reviews did not have program components, such as well-defined goals, objectives, detailed plans, and controls. Neither the Examination function nor we could determine whether the program was effective in improving audit quality. The Examination function was not able to provide us with reliable cost figures, but its estimated salary costs for the last CEP Peer Review ranged from $250,000 to over $1 million; travel figures exceeded $150,000. Consequently, neither the Examination function nor we could ascertain whether the program was cost-beneficial.

The CEP Peer Reviews did not have well-defined goals, program objectives, and detailed plans for conducting reviews. Individual CEP Peer Reviews conducted were not consistent in sample selections and in the issues selected for review. We could not determine overall CEP improvement from one peer review to the next because of the changes in methodologies used to review CEP examinations over an 8-year period.

No follow-up system existed to ensure corrective actions were taken on recommendations made during the CEP Peer Reviews. We could not determine if any corrective actions were taken or whether improvements occurred from one review to the next. In fact, many issues identified as needing improvement were repeated from one CEP Peer Review to another.

No system existed to track costs associated with the CEP Peer Reviews. Resources from Examination, International, Counsel, and Appeals were expended to conduct the last review, which IRS officials estimated cost from $400,000 to over $1 million. Consequently, we could not evaluate the CEP Peer Review effectiveness and cost-benefits.

To improve the quality review of CEP examinations, the LMSB Division’s Director, Quality Assurance and Performance Management, needs to ensure that program goals and objectives are established that integrate into the overall LMSB Division’s goals. Reviews should be based on written planning documents that fully explain the premise for the review and define the scope, objectives, and sample methodology used.

In addition, the Director, Quality Assurance and Performance Management, should establish a system that tracks findings and recommendations from the CEP Peer Reviews to ensure that corrective actions are taken timely. Program costs should also be tracked so they can be associated with program activities to determine whether the CEP Peer Reviews are cost-beneficial.

Management’s Response: Management’s response was due on September 20, 2000. As of September 22, 2000, management had not responded to the draft report.

Objective and Scope

The overall objective of our review was to determine whether the peer reviews were efficient and effective in improving the Coordinated Examination Program (CEP). We concentrated on whether corrective actions to resolve issues identified by management during their CEP Peer Reviews were implemented. We also reviewed available cost information, case selection criteria, and measurable results.

The audit was conducted in the Office of Corporate Examinations during the period December 1999 through May 2000. We also contacted regional offices to secure information for our review. This audit was performed in accordance with Government Auditing Standards.

Details of the audit objective, scope, and methodology are presented in Appendix I. Major contributors to this report are listed in Appendix II.

Background

The Internal Revenue Service (IRS) considers the CEP to be its most productive audit program and dedicates considerable resources to CEP audits. The IRS spent approximately $355 million on the CEP in Fiscal Year (FY) 1999.

The taxpayer population for the CEP includes approximately 1,700 of the largest and most complex taxpayers (primarily corporations). Teams of senior revenue agents and specialists, which may include computer analysts, engineers, and economists, audit taxpayers in the CEP. CEP examinations include 2 or more tax years and require 18 or more months to complete. The CEP strives to conduct quality examinations with the most effective and efficient use of resources, while placing the least burden on the taxpayer and the government.

The IRS uses peer reviews to quality review CEP cases. To be a value-added process, quality review should provide management with an independent assessment of the extent to which the organization complies with its standards, policies and procedures. This assessment should provide valuable feedback to top management on how well the organization achieves its quality goals.

The first CEP Peer Review report was issued in 1990. Results from the other four CEP Peer Reviews were issued between 1991 and 1999. The Examination function’s management told us that CEP Peer Reviews were not conducted at regular intervals because of funding restrictions.

The last CEP Peer Review was a multi-phased quality review that focused on all aspects of a CEP examination. The four-phased review included:

• Data analysis of closed cases.
• In-depth issue review.
• Case manager self-assessment.
• Taxpayer/stakeholder input.

Written results were prepared, and two phases included recommendations for improvements to the CEP examination process.

Under the IRS’ new business unit structure, the Quality Assurance and Performance Management function in the Large and Mid-Size Business (LMSB) Division will quality review CEP examinations. The process for quality reviewing CEP examinations was recently drafted.

Results

The CEP Peer Reviews did not have program components, such as well-defined goals, objectives, detailed plans, and controls. Neither the Examination function nor we could determine whether the CEP Peer Reviews were effective in improving audit quality. The Examination function was not able to provide us with reliable cost figures, but its estimated salary costs for the last CEP Peer Review ranged from $250,000 to over $1 million. Travel figures exceeded $150,000. As a result, neither the Examination function nor we could determine whether the program was cost-beneficial.

The Coordinated Examination Program Peer Reviews Need Program Components

The CEP Peer Reviews did not have the necessary program components, such as well-defined goals, program objectives, and plans for conducting reviews. In particular:

• Sample selection and issues selected for review varied between CEP Peer Reviews. For example, in earlier CEP Peer Reviews, cases were randomly selected with a limit of no more than two cases from any one district, but in a later review, the cases were selected from a stratified sample based on case size.
• The items or issues reviewed varied between reviews. For example, planned utilization of resources was identified as an issue in one CEP Peer Review, yet this issue did not appear in subsequent reviews.

Each review was organized to stand on its own and adopted its own approach for conducting the review. Reviews as a whole did not maintain the consistency that would be invaluable in evaluating whether overall quality is improving.

A program is comprised of specific activities that attempt to accomplish one or more objectives. The objective or objectives define the purpose of the program and impact on the program results. Plans further define the specific activities. The CEP’s auditing standards include written plans as a component in conducting a quality examination. Plans should include the nature and extent of compliance testing, audit scope and methodology, and the basis for key decisions.

The Peer Review Oversight Committee, which directed the CEP Peer Reviews, was an ad-hoc group convened for the duration of individual reviews. The majority of the members changed for each review since funding concerns prevented the staffing of a permanent Oversight Committee. Not having the continuity of a permanent group meant that lessons learned from one CEP Peer Review were not transferred to the next.

Without the necessary program components, the value added from the CEP Peer Review is diminished because it is difficult to determine whether the intended results or benefits are achieved. The inconsistent methodologies used to conduct the reviews over an 8-year period did not allow for the establishment of baselines, which would have given management the starting point for identifying improvement opportunities in CEP processes.

The Director, Quality Assurance and Performance Management, should:

1. Establish goals and objectives for the CEP quality review program that define expected outcomes and serve as the basis for measuring the success of the program. In consultation with the Commissioner, LMSB Division, these goals and objectives should be integrated into the overall LMSB Division’s goals.
2. Develop a planning document establishing the objectives of individual CEP Peer Reviews. This written planning document should incorporate a premise for the review, and define the scope, objectives, and sample methodology of the review.

In designing the new CEP Peer Reviews, LMSB Division management should ascertain the purpose. More specifically, the program purpose needs to identify whether the CEP Peer Review is intended to be a quality assurance process, a quality measurement process, or a combination thereof. Different approaches can affect the organization structure and program cost drivers.

Management’s Response: Management’s response was due on September 20, 2000. As of September 22, 2000, management had not responded to the draft report.

The Coordinated Examination Program Peer Reviews Need a Formal Follow-up System to Ensure Corrective Actions Are Taken

There was not a controlled follow-up system to ensure corrective actions were taken on CEP Peer Review recommendations. We were advised that written CEP Peer Review results were distributed to all CEP personnel. During the five CEP Peer Reviews, issues were identified and recommendations were developed. Although different methodologies were used on the CEP Peer Reviews, similar issues were identified.

Several issues and recommendations repeated from one CEP Peer Review to another. However, a formal tracking system was not established to monitor the implementation of the corrective actions. As a result, we could not determine if any corrective actions were taken or whether improvements occurred from one review to the next.

The following table gives examples of issues identified multiple times in the five CEP Peer Reviews:

The Standards for Control in the Federal Government require that management develop policies and procedures for ensuring that the findings of audits and other reviews are promptly resolved. Managers should:

• Promptly evaluate findings from audits and other reviews.
• Determine proper actions in response to findings and recommendations from reviews.
• Complete, within established time frames, all actions that correct or otherwise resolve matters.

Resolution is completed when action has been taken that:

• Corrects identified deficiencies.
• Produces improvements.
• Demonstrates the findings and recommendations do not warrant management action.

National Headquarters CEP representatives considered visitations made to the regional and district offices as an adequate follow-up system to the CEP Peer Reviews. We reviewed 3 of the most recent regional visitation reports conducted by the National Headquarters and 20 district visitation reports conducted by the former regional offices. While these visitations focused on various CEP issues, such as program accomplishments, the purpose of the visitations was not to follow-up on CEP Peer Review recommendations, and no comments on the CEP Peer Review results were included in the reports.

CEP Peer Review results should be used to effect positive change to the CEP process. Without an effective control system to implement CEP Peer Review recommendations, management cannot assure that conditions identified through the CEP Peer Reviews are corrected.

The Director, Quality Assurance and Performance Management, should:

3. Develop a system to track findings and recommendations identified during the CEP Peer Reviews. This system should include recommendations, responsible officials, planned completion dates of corrective actions, actual completion dates, and comments as to changes or delays regarding the recommendation. This system would assist the IRS in evaluating the status of corrective actions.

The Coordinated Examination Program Peer Reviews Need to Relate All Program Costs to Program Activities

CEP management did not track the costs for conducting the CEP Peer Reviews. CEP representatives stated that costs were never a consideration in conducting the CEP Peer Reviews. However, they considered the reviews as beneficial to the overall CEP program.

Neither the Examination function nor we could determine whether the CEP Peer Reviews were effective and cost-beneficial. The IRS was not able to provide us with reliable cost figures, but the Examination function’s management estimated that the salary costs for the last CEP Peer Review ranged from $250,000 to over $1 million; travel expenses exceeded $150,000. Eighty-two participants from Examination, Counsel, Appeals, and International contributed to the last CEP Peer Review.

The Government Performance and Results Act of 1993 (GPRA) recommends that agencies link program costs with program activities. To accomplish this, agencies must develop appropriate systems for measuring program performance and costs. Reliable systems are critical for collecting relevant data, tracking costs, and monitoring progress toward achieving performance goals.

Since costs were not tracked, management could not determine whether the CEP Peer Reviews were cost-beneficial. Tracking costs would provide the IRS with decision support information to:

• Determine whether the methods to conduct the CEP Peer Reviews were cost-beneficial.
• Make future budget decisions.

3. The Director, Quality Assurance and Performance Management, should develop a system for tracking the CEP Peer Review costs, including staffing, travel, overhead, and other related expenditures. This would allow the IRS to link the program costs with program activities to evaluate whether the program is cost-beneficial. Program costs would also be valuable for fiscal budget planning.

Conclusion

The quality review of CEP examinations needs to be a value-added process that is effective and cost-beneficial. However, the CEP’s current structure does not define program goals and objectives, contain a follow-up system for ensuring corrective actions are taken, and relate program costs to the review process.

Our recommendations regarding weaknesses in past quality reviews of CEP examinations should assist IRS management in the formulation and refinement of the new review process in the LMSB Division.

Detailed Objective, Scope, and Methodology

Our overall objective was to determine whether the peer reviews were efficient and effective in improving the Coordinated Examination Program (CEP). To accomplish this objective, we performed the following steps:

1. Determined whether the Internal Revenue Service (IRS) had an adequate system to track the costs for conducting the CEP Peer Reviews.

1. Interviewed IRS management to determine costs for the CEP Peer Reviews.
2. Secured and reviewed management information reports that recorded CEP Peer Review travel costs.

1. Determined if the IRS’ criteria for selecting cases and ranking issues during the CEP Peer Reviews was statistically sound and allowed the IRS to project the results to all CEP cases.

1. Interviewed appropriate IRS management and reviewed CEP Peer Review reports to identify the process for selecting cases and ranking issues.
2. Analyzed management’s sampling methodology for CEP cases included in the CEP Peer Reviews.
3. Evaluated the system used during CEP Peer Reviews to rank issues.

1. Determined whether the IRS’ system to implement recommendations identified during the CEP Peer Reviews was effective.

1. Reviewed all five CEP Peer Review reports issued since June 1990.
2. Determined whether recommendations identified during the CEP Peer Reviews were prioritized.
3. Determined whether management identified responsible officials to implement the recommendations.
4. Evaluated management’s process to ensure corrective actions for recommendations made by the CEP Peer Reviews were timely completed.
5. When corrective actions were not completed, we determined why. When corrective actions were different from the original actions cited in the CEP Peer Reviews, we determined whether the actions taken by management resolved the problem identified.
6. Determined whether findings and recommendations from the CEP Peer Reviews recurred in later CEP Peer Reviews. When items recurred, we determined the cause of recurrence.

1. Determined whether the CEP Peer Reviews resulted in measurable improvements to the quality of CEP examinations.

1. Interviewed management to determine management’s goals for and expected benefits of the CEP Peer Reviews.
2. Evaluated any reports, studies, or other information that indicated if management’s goals and expected benefits were met or the quality of CEP examinations improved.
3. Reviewed any customer satisfaction studies, surveys, or other information to determine whether the IRS had addressed customer concerns.

Major Contributors to This Report

Gordon C. Milbourn III, Associate Inspector General for Audit (Small Business and Corporate Programs)

Kerry Kilpatrick, Director

Philip Shropshire, Director

Preston Benoit, Audit Manager

Richard Hayes, Audit Manager

Donna Saranchak, Senior Auditor

Lisa Stoy, Senior Auditor

Carole Connolly, Auditor

Sharon Hayes, Auditor

Report Distribution List

Deputy Commissioner, Operations C:DO

Commissioner, Large and Mid-Size Business Division LM

Assistant Commissioner (Examination) OP:EX

National Director, Corporate Examinations OP:EX:C

Director, Legislative Affairs CL:LA

Director, Quality Assurance and Performance Management LM:Q

Director, Office of Program Evaluation and Risk Analysis M:O

Office of the Chief Counsel CC

National Taxpayer Advocate C:TA

Office of Management Controls CFO:A:M

Audit Liaisons:

Assistant Commissioner (Examination) OP:EX

Large and Mid-Size Business Division LM