Executive Summary

Objective and Scope

Background

Results

Conclusion

Appendix I – Detailed Objective, Scope, and Methodology

Appendix II – Major Contributors to This Report

Appendix III – Report Distribution List

Appendix IV – Management’s Response to the Draft Report

Executive Summary

As the Internal Revenue Service (IRS) migrates to four operating divisions, its traditional approach to tax administration is changing to incorporate the needs of a diverse taxpayer population. Within this new approach, it is important for the IRS to effectively develop and use compliance research data to guide its programs. The primary objective of this review was to determine if the IRS’ system of controls provides reasonable assurance that compliance research activities are meeting program objectives and customer needs in a cost-effective manner.

The IRS does not have effective controls over its research activities. From the beginning of Fiscal Year (FY) 1996 to the end of FY 2000, the IRS will have expended over $226 million conducting compliance research. With its current monitoring processes, the IRS cannot accurately quantify its return on investment. Although the IRS has recently implemented some new procedures to better manage its compliance research program, it cannot accurately measure the cost-effectiveness or the impact of the information provided to its functional programs.

The IRS does not have an effective process to measure the costs and impact of its research activities. It does not track recommendations or validate their resulting outcomes. The system used by the IRS tracks costs over the life of a project but does not allow for an effective comparison to yearly budget figures. The IRS cannot effectively and efficiently manage its research program if benefits and costs cannot be accurately identified.

The IRS’ research organization does not adequately address concerns expressed by its functional customers. It also does not always communicate its research results timely and does not document discussions with its customers. In FY 1999, the research organization implemented a survey process to obtain feedback from its functional customers. Even though the results of this survey showed that respondents from four of seven key customers were dissatisfied with the research process, the research organization has done very little to address their specific concerns.

In addition, research results are not reported timely to functional customers because the report approval process is not efficient. As a result, significant information may not be available when managerial decisions are made on key functional processes. The research organization also does not effectively document discussions with its customers. Since research projects often take years to complete, a historical record of customer discussions is critical to ensure research continues to meet customer needs.

The IRS does not maintain enough information to effectively manage its compliance research program at a national level. It has not ensured that key project documentation to support customer involvement and project status is complete and timely stored in electronic project folders. Also, the IRS does not maintain and analyze information on closed research projects. As a result, the IRS may not have the necessary research project information to make effective business decisions.

To ensure that research activities support its mission and provide value to its programs, the IRS needs to improve control over its compliance research program. To do this, the IRS must develop a process to identify and measure actual research outcomes and their related costs. To ensure that research activities provide value to its customers, the IRS should report research results timely to functional customers and involve them more in the research process. The IRS should also maintain adequate project documentation to facilitate effective business decisions.

Management’s Response: IRS management agreed that they must ensure research is being effectively developed and efficiently used to guide management decisions. Since there are many decisions yet to be made as to how the research organization will function in the new operating divisions, the proposed corrective actions are short-term remedies within the current organization structure. Many policies and procedures may change after research organizations begin functioning in the new operating divisions, and embedding a research function in each operating division should substantially eliminate customer concerns. Management has already taken steps to improve controls over annual project costs and essential project documentation. They have planned processes to readdress customer concerns in future customer satisfaction surveys and program reviews. Management is choosing to rely on the new Strategic Planning and Budget process in each operating division to link research activities to strategic measures. They did not agree to document customer discussions, choosing instead to rely on project prospectuses approved yearly by customers to ensure their involvement. Also, management believes current processes to ensure timeliness of reporting and communicating interim results to customers are effective.

Office of Audit Comment: IRS management must ensure that effective controls over research activities continue once research functions are placed in the operating divisions. Management should consider our recommendations when making decisions on how the research organization will function in the new operating structure. If the IRS follows through and links research results to strategic measures, it should be able to quantify the effect of research outcomes on tax administration. Management should also continue reviewing customer satisfaction surveys and conducting program reviews after migrating to the operating divisions. This action will validate whether the embedding of research activities in each operating division substantially eliminates customers’ concerns. Although management has set guidelines and procedures to ensure timely reporting of results to customers, they must provide oversight to ensure that these policies are being followed. In addition, management should ensure that their policies do not discourage the sharing of research results with customers and that significant customer discussions and resulting decisions are effectively documented.

Objective and Scope

This audit was initiated as part of the Treasury Inspector General for Tax Administration’s coverage to evaluate the Internal Revenue Service’s (IRS) compliance research program. The primary objective of this review was to determine if the IRS’ system of controls provides reasonable assurance that compliance research activities are meeting program objectives and customer needs in a cost-effective manner.

We conducted our fieldwork in the National Office from September through December 1999. The majority of our work was done through reviewing documentation on the Compliance Research Intranet, interviewing IRS management, and reviewing specific project documentation. This audit was performed in accordance with Government Auditing Standards.

Details of our audit objective, scope, and methodology are presented in Appendix I. Major contributors to this report are listed in Appendix II.

Background

According to the Guide to the IRS for Congressional Staff, the IRS’ research organization "contributes to improving voluntary compliance through data driven research for the identification and planning of enforcement and non-enforcement activities." It analyzes tax compliance issues and incorporates education, outreach, and traditional enforcement programs in an effort to modify noncompliant taxpayer behavior. These efforts are an integral part of the IRS’ overall taxpayer compliance program.

The Commissioner of the IRS views research activities as an integral part of strategic planning. From the beginning of Fiscal Year (FY) 1996 to the end of FY 2000, the IRS will have expended over $226 million conducting compliance research. This includes over $57 million allocated for FY 2000.

The IRS compliance research plan for FY 2000 contains 138 projects. The Office of Research in the National Office coordinates and oversees all national research activities. Research teams in the National Office and 33 district offices conduct the actual research.

During FY 1998, the Office of Research contracted with an outside vendor to conduct an independent assessment of its organizational processes, competencies, and training. The vendor identified weaknesses in current processes and prescribed recommendations for improvement. Some of these weaknesses included: the current planning process restricted customer involvement, projects were managed with limited and inconsistent use of performance measures, and projects lacked a structured managerial approach.

As a result of this report, the Office of Research has recently developed some new processes that may improve control over its program. However, as the IRS migrates to four operating divisions, its research organization will continue to undergo significant changes.

Results

The IRS does not have effective controls over its research activities. With its current monitoring processes, the IRS cannot accurately quantify its return on investment. Although the IRS has recently implemented some new procedures to better manage its compliance research program, it cannot accurately measure the cost-effectiveness or the impact of the information provided to its functional programs.

The IRS needs to take the following steps to ensure that the research operations are effective:

• The IRS needs to measure the cost-effectiveness of its compliance research program.
• The IRS needs to improve communication between the research organization and its customers.
• The IRS’ research organization needs to improve project documentation.

The Internal Revenue Service Needs to Measure the Cost-Effectiveness of Its Compliance Research Program

The IRS does not have a process to analyze and measure the overall cost-effectiveness of its compliance research program. Even though its Office of Research has implemented several new procedures that measure some aspects of the program, it cannot adequately measure the benefits or the project costs of its research activities on an annual basis.

The Office of Management and Budget Circular A-123 requires all managers in the Federal Government to comply with the General Accounting Office (GAO) Standards for Internal Control in the Federal Government. One standard requires that managers establish a functional level of internal control activities that compare actual performance to planned or expected results. This concept is relevant to both the products and costs of the research organization.

The IRS does not have an effective process to measure the impact of its research on tax administration

There is no process to measure or validate actions taken on recommendations made by its research organization. Once research is completed under the current process, the information is provided to functional IRS management for their use and implementation. If implemented, there is no validation performed to compare the estimated outcomes to actual results.

The IRS has not given priority to validating research project outcomes. However, since the mission of its research organization is to provide analysis on tax compliance issues and to suggest alternative treatments to address those issues, it is imperative that the actual outcomes of its research activities be evaluated.

The IRS cannot effectively measure the annual costs of its research activities

Prior to the Office of Research implementing the Research Measurement System, there was limited accounting information available on specific project costs. While this system now allows the Office of Research to track its project costs and estimated benefits over the life of an entire project, it does not provide annual project costs. Since the IRS’ research budget is determined annually, there needs to be a method to analyze annual project costs and compare them with the yearly budget. Without tying actual project expenditures to the annual budget, there is a risk of costs going uncontrolled.

The IRS will be better able to identify the value of its investment if its research organization documents and tracks research recommendations, the resulting corrective actions, and project costs. This control would help the IRS ensure that research activities are meeting long-term IRS program objectives and overall customer needs.

1. The Chief Operations Officer should assign responsibility for developing a process to track and measure the effect of research outcomes and recommendations on tax administration.

Management’s Response: Management responded that the Strategic Planning and Budget process in the new operating divisions will serve as documentation for the disposition of research recommendations. This process will serve to link research to concrete plans, resource allocations, and strategic measures.

Office of Audit Comment: It is important for the IRS to accurately measure the value of its research activities. If the IRS follows through and links research results to strategic measures, it should be able to quantify the effect of research outcomes on tax administration.

2. The Assistant Commissioner (Research and Statistics of Income) should ensure research costs can be easily determined and accurately tracked to the project level on an annual basis.

Management’s Response: Management agreed with the recommendation and modified their management information system in March 2000 to account for annual costs.

The Internal Revenue Service Needs to Improve Communication Between the Research Organization and Its Customers

The Office of Research did not effectively address customer concerns raised in a survey conducted in FY 1999. It also does not always communicate its research results timely and does not document discussions with its customers.

The Office of Research has developed new processes to improve the sharing of information with its functional customers. It developed the project prospectus to encourage customer buy-in and established research strategy owners (RSO), generally Research managers who have been assigned primary responsibility for customer liaison with senior management in the customer functions, to better communicate with its customers. However, several customers in the IRS functions have expressed dissatisfaction with the research process and the timeliness of reporting research results.

The Office of Research has not adequately addressed concerns expressed by its customers

In FY 1999, the Office of Research sent out a customer satisfaction survey to high-level IRS executives. The results of this survey showed that respondents from four of seven key customers were dissatisfied with the overall research process. Specific concerns expressed in this survey involved timeliness of research activities and reporting, the lack of adequate interaction with the research teams, and the inability to end projects that are no longer a priority.

An internal review conducted by the Office of Research found that poor survey results occurred because the surveys were completed by staff assistants or analysts instead of high-level IRS functional executives. It recommended modifying the survey method and survey design but did not make any recommendation to address the concerns raised by the respondents.

We contacted managers in the Collection, Customer Service, and Examination functions and the Earned Income Tax Credit Project Office. As reported in the surveys, some managers who direct specific national programs in these functions are still dissatisfied with research products.

Office of Research management has repeatedly stressed that their customers are the high-level functional executives. However, it is the national program managers who will use the research data and recommendations to improve their programs.

The Office of Research does not ensure reports are issued timely

The FY 1999 survey results show that three of the four dissatisfied respondents were concerned with the timeliness of research. In our interviews, one customer specifically noted that once research is completed, there is a very long period of time before the results of the project are communicated.

We reviewed 20 research reports issued in FYs 1997, 1998, and 1999. We were only able to verify the draft and final report approval dates for eight of these reports. Six reports took over six months after the draft report was completed to go through the quality review process and be approved by the Assistant Commissioner (Research and Statistics of Income).

The Office of Research has no controls to ensure the timeliness of the quality review process. It does not provide reports to its customers until this process is completed and the final report is approved. If the timeliness of this process is not effectively controlled, the customer functions may not have the information when they can best use it.

The research organization does not document customer contacts

Only 7 of 30 open research projects we reviewed had any documentation of discussions with their customers on the Compliance Research Intranet. Although the research organization has no specific requirement for documenting customer contacts, it should record information gathered in discussions with its customers to ensure that project objectives remain valid and address the current customer needs. Research projects often take several years to complete. The FY 2000 Research Plan has 23 projects that are at least 3 years old.

The GAO Standards for Internal Control in the Federal Government state that management should ensure there is adequate communication with external stakeholders. Managers need relevant, reliable, and timely communications relating to internal as well as external events to adequately control their operations.

Both the Office of Research and its customers need timely and relevant information to effect business decisions. Decisions and analyses concerning key IRS functional processes can be delayed if information is not provided timely to those who need it. This could delay implementation of tax compliance measures or other tax administrative decisions that could increase revenue or reduce operating costs. In addition, effective communication with customers is necessary to ensure that project results will meet customer needs.

Recommendations

3. The Assistant Commissioner (Research and Statistics of Income) should ensure the Office of Research adequately addresses the issues raised in its customer satisfaction surveys.

Management’s Response: Management stated that by embedding a research function in each operating division, the underlying issue of customer concerns should be substantially eliminated. In addition, the RSOs were directed to develop action plans with their customers concerning issues raised in the customer satisfaction surveys. During the mid-year and end-of-year program reviews, steps taken through the RSO action plans to improve customer satisfaction will be evaluated. In June 2000, another customer satisfaction survey will be administered. Results of this survey will be used to measure the impact of any changes since the previous survey.

Office of Audit Comment: The IRS should ensure program reviews continue to address customer concerns. This process should be incorporated into research activities when they are placed under the individual operating divisions.

4. The Director, Office of Research, should set timeliness guidelines for the quality review of research reports and ensure these guidelines are met. In addition, he should consider issuing draft reports to the designated customers to enable timely use of the information and to solicit their input to the final report.

Management’s Response: Management responded that a quality assurance and review process has been in place for several years to track the technical review of research plans and reports. In April 1999, a 30-day turnaround time was instituted for the completion of the technical review process. Current research projects require a project prospectus that has deliverable dates agreed to by the customer. However, there are many instances where a final report or findings may not be delivered by the due date. As a result, in June 1999, interim research findings were allowed to be shared with customers in advance of a fully reviewed and approved final report.

Office of Audit Comment: A September 1999 memorandum from the Acting Director, Office of Research, allows interim findings to be shared with the customer at the discretion of the RSO or Office of Research project manager. It also states that "Such findings should always be shared with the appropriate cautions and at the risk of the RSO or the Office of Research manager." This policy does not seem to encourage the sharing of interim results. Since a lot of this research is being conducted at the request of a customer, access to interim results should be at the discretion of the customer, not the Office of Research.

5. The Director, Office of Research, should require research teams to document information gathered in meetings with their customers to ensure each project remains on target and is still a priority with the customer.

Management’s Response: Management did not agree to keep documentation of all customer contacts. However, they are requiring that project prospectuses for each project be approved by the customer each year to ensure they are involved in project direction.

Office of Audit Comment: It may be possible to manage customer communications with the use of a yearly prospectus in some circumstances. However, the prompt, continuous recording of significant customer contacts provides verification that the customer is adequately involved in project decisions and confirms the customer’s commitment to the project.

The Internal Revenue Service’s Research Organization Needs to Improve Project Documentation

The research organization does not maintain complete research project documentation to support managerial decisions at the national level. Monthly status reports, project prospectuses, and closed project information are not being consistently maintained on the Compliance Research Intranet. As a result, the Office of Research cannot effectively and efficiently manage its resources.

The internal control activities standard set forth in the GAO Standards for Internal Controls in the Federal Government prescribes that documentation of transactions is critical to ensure management has the information they need to make business decisions.

The Office of Research did not ensure that its research teams properly maintained project documentation

Documentation guidelines issued by the Office of Research require monthly status reports for each project to be documented and maintained on the Compliance Research Intranet. In addition, supplemental guidelines require a prospectus to be maintained on this system for every project started after FY 1998.

Out of a judgmental sample of 30 open research projects, only 19 had monthly status reports on the Compliance Research Intranet. In addition, only 10 projects had action items posted to the Research Management System addressing current project status. By not properly documenting the monthly status reports and action items, the Office of Research does not have the information necessary to make effective business decisions on program direction.

From the same sample, 24 open projects were required to have prospectuses, based on Office of Research guidelines. Out of the 17 prospectuses we were able to review, only 7 were found on the Compliance Research Intranet. In addition, the Office of Research has no requirement to document changes made to prospectuses. By not maintaining a historical record of these changes, it will not have adequate documentation of customer involvement.

The Office of Research has no process to identify disposition of closed projects

The Office of Research does not record when projects are closed. We compared research plans for consecutive fiscal years to identify closed projects and provided this information to the Office of Research to obtain reasons for their closure.

Sixty-two out of 158 projects (39 percent) closed in FYs 1997, 1998, and 1999 had been terminated without the issuance of a report. There was no documentation on the Compliance Research Intranet or in project folders to support why these projects were closed. Office of Research staff had to contact field offices to determine what happened to these projects.

The Office of Research cannot effectively analyze the reasons for terminating projects and identify problems in project management if the documentation is not available to make these decisions.

After managerial reviews were conducted in FY 1998, the Office of Research directed its research teams to improve project documentation. However, it did not ensure that project guidelines were followed and that information needed to control business activities was available. The lack of adequate project documentation could result in the Office of Research inefficiently using its resources.

Recommendations

6. The Director, Office of Research, needs to emphasize research project documentation requirements to project teams and conduct periodic reviews for compliance.

Management’s Response: Management responded that a memorandum detailing project management guidelines and documentation requirements is issued to all field employees annually. In addition, a Project Management Council reviews mandatory project documentation approximately every two months and brings deficiencies to the attention of Research managers. The next annual memorandum will emphasize the project documentation requirements for posting to the Compliance Research Intranet. These requirements will also be emphasized at the next RSO meeting.

Office of Audit Comment: Based on the condition of project documentation found during this review, Office of Research management still needs to ensure that the Project Management Council is adequately performing its duties by reviewing project documentation.

7. The Director, Office of Research, should develop a closing document, to be completed for all projects, that includes reasons for closure.

Management’s Response: In February 2000, the Office of Research developed a "project change control process" which includes formal documentation for project termination.

Conclusion

As the IRS migrates to four operating divisions, its traditional approach to tax administration is changing to incorporate the needs of a diverse taxpayer population. Within this approach, it is important for the IRS to effectively and efficiently develop and use compliance research data to guide its programs.

To ensure that research activities support the IRS’ mission and provide value to its programs, the IRS needs to improve control over its compliance research. The IRS must develop a process to identify and measure actual research outcomes and their related costs. The IRS should report research results timely to functional customers and involve them more in the research process. The IRS should also maintain adequate project documentation to facilitate effective business decisions. Without making these changes, the IRS cannot be assured of the benefits derived from its compliance research activities.

Detailed Objective, Scope, and Methodology

The overall objective of this review was to determine if the Internal Revenue Service’s (IRS) system of controls provides reasonable assurance that compliance research activities are meeting program objectives and customer needs in a cost-effective manner.

To accomplish our objective, we interviewed 13 managers and 6 employees from the Office of Research, 8 managers in the IRS functions that are Office of Research "customers," 2 employees of the Assistant Commissioner (Research and Statistics of Income), and 1 employee on the IRS’ modernization team. We also analyzed and tested the controls over the compliance research process. To accomplish our objective, we:

1. Determined whether the IRS had effective controls to ensure that its compliance research process provides beneficial and timely products to its customers.
1. Identified the mission of the Office of Research, evaluated how it relates to the IRS mission, and reviewed how the IRS restructuring may affect the compliance research program.
2. Reviewed implemented or planned corrective actions resulting from a training and organizational assessment study completed by an outside vendor.
3. Identified the products produced by the Office of Research.
4. Determined whether there is a control system in place that measures the effectiveness of the compliance research program.
5. Identified and documented the control system in place that measures the effectiveness of the compliance research program.
6. Determined the methodology used by the Office of Research for computing financial accomplishments for its projects.
7. Determined whether the Office of Research developed procedures to monitor and evaluate the effectiveness of corrective actions implemented as a result of its projects.
8. Determined whether the measurement tools used accurately evaluate the overall effectiveness of the program.
9. Identified and documented key controls in the management of national compliance research projects to meet its mission.
10. Evaluated the management information system used by the Office of Research for managing research activities.
11. Performed a walk-through of the control system.
2. Determined the adequacy of controls over measuring the effectiveness of Office of Research products.
1. Identified all projects conducted by Office of Research teams and where project documentation is stored.
2. Evaluated the effectiveness of controls over closed national projects.
1. Identified the 45 national projects closed since October 1, 1996, by comparing the consecutive research plans for Fiscal Year (FY) 1997 through FY 2000.
2. Reviewed a judgmental sample of 20 reports for the selected projects to identify recommendations and projected financial accomplishments.
3. Reviewed and analyzed the documentation to determine whether reported project accomplishments were valid.
4. Determined if the Office of Research had a method to measure actual changes to specific tax gap categories.
3. Evaluated the effectiveness of controls over open national projects.
1. Selected a judgmental sample of 30 of 48 open projects on the FY 2000 research plan in the following 4 selected research strategies: Improving Earned Income Tax Credit Claimants’ Compliance, Improving Payment Compliance, Developing Compliance and Research Measures, and Improving Filing Compliance.
2. Reviewed and analyzed the documentation for the 30 selected projects to validate the effectiveness of controls.
4. Identified and reviewed 16 Treasury Inspector General for Tax Administration reports relating to the subject areas of the selected projects to determine whether recommendations had already been provided to IRS management.
5. Determined whether other IRS functions or programs were providing research activities and whether the Office of Research is duplicating effort.
3. Determined whether adequate controls were in place to ensure the compliance research program is cost effective.
1. Obtained and reviewed budget figures for the compliance research program since FY 1996.
2. Gathered documentation of any cost comparisons, evaluations, or other measurements used by the Office of Research to evaluate its program.
3. Determined whether the cost of individual research projects could be determined.
4. Determined whether resources were effectively applied to the most critical projects.
5. Determined whether the Office of Research had an effective process to prioritize research activities.
6. Determined the process for assigning priorities given to national projects.

Major Contributors to This Report

Walter E. Arrison, Associate Inspector General for Audit (Wage and Investment Income Programs)

Michael Phillips, Director

Bryce Kisler, Audit Manager

Alan Lund, Senior Auditor

David Hartman, Auditor

Craig Pelletier, Auditor

Steven Stephens, Auditor

Sharon Summers, Auditor

Report Distribution List

Deputy Commissioner Operations C:DO

Chief Operations Officer OP

Assistant Commissioner (Research and Statistics of Income) OP:RS

Director, Office of Research OP:RS:R

Director, Office of Program Evaluation and Risk Analysis M:O

National Director for Legislative Affairs CL:LA

Office of Management Controls M:CFO:A:M

Office of the Chief Counsel CC

Audit Liaison:

Chief Operations Officer OP

Assistant Commissioner (Research and Statistics of Income) OP:RS

Management’s Response to the Draft Report

Response has been removed due to its size. To see the complete response, please go to the Adobe PDF version of this report on the TIGTA Public Web Page.