Review of the Internal Revenue

Review of the Internal Revenue

Service’s Year 2000 End-to-End

System Integration Test Efforts -

Overall Planning and Execution of Test I

April 1999

Reference Number: 094002

April 7, 1999

MEMORANDUM FOR cOMMISSIONER ROSSOTTI

FROM: Lawrence W. Rogers /s/Lawrence W. Rogers

Acting Treasury Inspector General for Tax Administration

SUBJEcT: Final Audit Report - Review of the Internal Revenue Service’s Year 2000 End-to-End System Integration Test Efforts - Overall Planning and Execution of Test I

This report presents the results of our review of the Internal Revenue Service’s (IRS) overall planning and execution of the Year 2000 end-to-end system integration test efforts for Test I.

Despite initial delays in the planning and conducting of end-to-end system integration test activities, IRS’ Information Systems made significant progress in preparing an overall approach to conducting the test. In addition, the end-to-end system integration test team met their limited objectives for executing Test I. We did, however, identify the need for the IRS to prepare a systematic risk analysis of its systems and provide improved oversight over key support systems that will not be included in the IRS end-to-end test.

The Director, Office of Information Resources Management, did not agree to conduct a systematic risk assessment because all IRS mission critical systems will be subject to end-to-end testing. Since the IRS has allotted a three-month period for the actual end-to-end test of over 90 mission critical systems, we feel it would be prudent for the IRS to plan some alternative actions if either time or resources run short during the test period. The Director did agree to identify all key support systems not selected for the end-to-end system integration test. The century Date change Project Office has also established an end-to-end test team to provide technical coordination and support to system owners responsible for their own end-to-end testing.

We also identified the need for the IRS to better coordinate its planning efforts for the end-to-end system integration test. We recommended several ways in which the IRS could improve the end-to-end test planning, including the need to obtain formal commitments from necessary end-to-end system integration test partners in advance of future testing activities and improving project management efforts during the overall end-to-end system integration test planning effort. The Deputy chief Information Officer for Information Resources Management indicated that they would formalize support agreements by issuing Requests for Information Services, Memoranda of Understanding, and Service Level Agreements. They have also taken steps to improve the level of project communication and complete project documentation in more detail.

We also recommended that the IRS appoint an executive-level official solely dedicated to leading the end-to-end system integration test effort. The Deputy chief Information Officer for Information Resources Management disagreed with this recommendation, responding that the end-to-end system integration test was progressing timely and the current management structure for the test is sufficient. IRS Management has taken several steps to mitigate the risk we identified, in part by establishing an end-to-end test team to provide technical support to system owners responsible for their own end-to-end testing and by identifying representatives within Operations responsible for coordinating Year 2000 compliance. We feel that these actions may not suffice to ensure that end-to-end testing efforts are given priority, especially when test activities require testing partners outside of Information Systems to take actions that conflict with their organizational interests. Without an executive-level official, completing a systematic risk assessment becomes more important.

copies of this report are also being sent to IRS managers who are affected by the report recommendations. Please call me at (202) 622-6500 if you have any questions, or your staff may contact Maurice S. Moody, Acting Assistant Inspector General for Audit, at (202) 622-8500.

Table of contents

The Internal Revenue Service Needs to Perform a Detailed Analysis and Risk Review of Its Systems to Support the End-to-End System Integration Test

The Internal Revenue Service Needs to Improve Oversight of the End-to-End Testing of Key Systems Not Included in Its Servicewide Test

Issues Reported during the Review and Responded to by Internal Revenue Service Management

Appendix I - Detailed Objectives, Scope and Methodology

Appendix II - Major contributors to This Report

Appendix III - Report Distribution List

Appendix IV - Management’s Response to the Draft Report

Appendix V - Memorandum #1 Issued on August 11, 1998

Appendix VI - Management’s Response to Memorandum #1

Appendix VII - Memorandum #2 Issued on October 19, 1998

Executive Summary

Within the Internal Revenue Service (IRS), the end-to-end system integration test serves as the final phase for assessing the Year 2000 (Y2K) compliance of its tax processing software, hardware, and communication suites. The IRS’ Y2K end-to-end system integration test is being controlled by the Information Systems Product Assurance function, with significant levels of contractor support. The primary goal of the end-to-end system integration test is to create an off-line test environment that replicates IRS’ tax processing environment. Planning for the end-to-end system integration test began in May 1998, and testing will run through December 1999.

The objective of our review was to assess the IRS’ overall planning efforts for the Y2K end-to-end system integration test and to evaluate the effectiveness of Test I, which was conducted in July and August 1998. Test I was designed to provide a limited assessment of whether the IRS’ systems software components, application components, and commercial off-the-shelf products will work through the year 2000.

Results

Despite initial delays in the planning and conducting of Y2K end-to-end system integration test activities, IRS Information Systems made significant progress in preparing an overall approach to conducting the test. In addition, the end-to-end system integration test team met their limited objectives for executing Test I. We did, however, identify the need for the IRS to prepare a systematic risk analysis of its systems and provide improved oversight for key support systems that will not be included in its Servicewide end-to-end test. We also identified the need for the IRS to better coordinate its planning efforts for the end-to-end system integration test.

The Internal Revenue Service Needs to Perform a Detailed Analysis and Risk Review of Its Systems to Support the End-to-End System Integration Test

The IRS’ basis for selecting the systems included in the end-to-end system integration test was primarily judgmental. Management did not perform a detailed risk analysis or in-depth review of the systems prior to their selection. We identified several systems not selected for the end-to-end system integration test that have similar risk attributes to selected systems. The systems we identified are used to initiate updates to taxpayer accounts and, in some cases, could result in additional contacts with taxpayers. Although the systems to be included in the end-to-end system integration test have already been selected, a detailed analysis of the business value and potential risk exposure of all major IRS systems would provide benefit throughout the entire testing process.

The Internal Revenue Service Needs to Improve Oversight of the End-to-End Testing of Key Systems Not Included in Its Servicewide Test

Additional management attention is needed to ensure key Operations support systems not selected for the coordinated end-to-end system integration test are identified and receive sufficient emphasis and testing resources. Presently, the owners of these systems are expected to perform their own testing based on instructions received from the Product Assurance function. There is no centralized oversight within the Operations area for this process.

Additional benefit could also be gained by mandating the use of the Product Assurance test bed for the review of all key Operations support systems not selected for coordinated testing. This would ensure these systems are analyzed for Y2K compliance in a secure and controlled environment.

Summary of Recommendations

To improve the end-to-end testing of its systems, we recommended that IRS management:

Perform a detailed analysis of the IRS systems inventory to assess the business value and potential risk exposure of all its major systems and establish a priority ranking.
Identify all key Operations support systems not selected for the nationally coordinated end-to-end system integration test.
Establish centralized oversight and control over the testing of key support systems within the Operations functions.
Mandate that key Operations systems owners use the Product Assurance test bed to perform their Y2K compliance testing.

Management’s Response: IRS management agreed to identify all key support systems not selected for the end-to-end system integration test. The century Date change Project Office has established an end-to-end test team to provide technical coordination and support to system owners responsible for their own end-to-end testing. The Project Office will coordinate with system owners to identify resource requirements, support the establishment and implementation of system test plans, assist with the identification and scheduling of test beds and test bed alternatives, and serve as the primary interface among testers, Product Assurance, Telecommunications and other teams which may influence the timeliness and success of the end-to-end testing.

The century Date change Project Office did not agree to perform a detailed analysis of the business value and potential risk exposure of all major systems because all IRS systems will be end-to-end tested. They also did not agree that the use of the Product Assurance test bed should be mandated. Management’s complete response to a draft of this report is included as Appendix IV.

Office of Audit comments: Since IRS has allotted a three-month period from October 1, 1999, through December 31, 1999, to conduct the end-to-end test of over 90 mission critical systems, it would be prudent for IRS to plan some alternative actions if either time or resources run short to complete the test of all systems.

In a memorandum dated August 11, 1998, we also recommended that Information Systems take the following actions to improve the planning and execution of the end-to-end system integration test (management’s response is denoted in italics).

Use the Request for Information Services process or Memoranda of Understanding to obtain, prioritize, and allocate necessary end-to-end system integration test resources. The Deputy chief Information Officer for Information Resources Management stated that IRS would obtain such agreements for all future testing activities, as appropriate.

Oversee the development of more detailed end-to-end system integration test plans and schedules, and improve the level of Servicewide communication for the test effort. The Deputy chief Information Officer for Information Resources Management stated that project documentation would be completed in more detail, and that steps have been taken to improve the level of communication.

Develop and communicate Servicewide an end-to-end system integration test charter showing the members of the inter-organizational test team, the team’s mission, and the team’s roles and responsibilities. The Deputy chief Information Officer for Information Resources Management stated that IRS was completing a revised century Date change End-to-End Integration Testing concept of Operations that will contain this type of information.

consider appointing an executive solely dedicated to leading the end-to-end system integration test team. The Deputy chief Information Officer for Information Resources Management disagreed with this recommendation and stated that the end-to-end system integration test is progressing timely and in a coordinated effort, and that the current management structure for the end-to-end system integration test is sufficient.

Office of Audit comments: IRS Management has taken several steps to mitigate the risk we identified in our memorandum, including the establishment of an end-to-end test team to provide technical support to system owners responsible for their own end-to-end testing and the identification of representatives within Operations responsible for coordinating Y2K compliance. We believe that these actions may not suffice to assure that end-to-end testing efforts are given priority, especially when test activities may require testing partners outside of Information Systems to take actions that conflict with their organizational interests. The decision not to appoint an executive-level official heightens the importance of performing a detailed analysis of the IRS systems inventory to assess the business value and potential risk exposure of all its major systems and establishing a priority ranking.

The complete text of this memorandum and management’s response are included as Appendices V and VI, respectively.

Objective and Scope

The overall objective of this review was to perform an assessment of the overall planning for the Internal Revenue Service’s (IRS) Year 2000 (Y2K) end-to-end system integration test efforts and to evaluate the execution of Test I. Audit work for this review was performed from June 1998 through October 1998. This review was conducted in accordance with generally accepted government auditing standards. currently, we are conducting a review of the planning/execution of Test II and the planning for Test III. During this second end-to-end system integration test review, we will follow up on recommendations presented in this report.

Appendix I contains the detailed objectives, scope and methodology for this review. Management’s response to a draft of this report is included in Appendix IV. A list of the major contributors to this report is presented in Appendix II.

Background

The upcoming century date change is one of the most pervasive problems facing the information systems community. To maximize system-processing capabilities and to preserve data storage space, many date fields in system components and applications have been limited to two-digit year representation (e.g., 97 for 1997). This problem is extremely critical to the IRS, as the majority of its tax processing functions are date driven.

Within the IRS, the end-to-end system integration test serves as the final phase for assessing the Y2K compliance of its tax processing software, hardware, and communication suites. The IRS’ Y2K end-to-end system integration test is being controlled by the Information Systems Product Assurance function, with significant levels of contractor support. The primary goal of the end-to-end system integration test is to create an off-line test environment that replicates the IRS’ tax processing environment, including the tax processing pipeline, tax processing support systems, electronic tax filing, the supporting hardware, and telecommunications infrastructure. Test I was designed to provide a limited assessment of whether the IRS’ systems software components, application components, and commercial off-the-shelf products will work through the year 2000.

Results

Despite initial delays in the planning and conducting of the testing efforts, Information Systems made significant progress in preparing an overall approach to conducting the Y2K end-to-end system integration test. In addition, the test team worked very diligently and met the limited objectives they had set for Test I.

However, key aspects of the end-to-end system integration test, specifically Test II (October 1998 - February 1999) and Test III (February 1999 - December 1999) still need to be performed. In addition, as Product Assurance officials have made clear, the success of the end-to-end system integration test cannot be fully assessed until December 1999, after the 2000 Filing Season applications have been tested.

We identified several issues that need to be addressed to assure that all IRS systems, whether included in the end-to-end system integration test or not, are adequately tested prior to January 1, 2000. These issues have been categorized into the following two recommendations.

The IRS needs to perform a detailed analysis and risk review of its systems to support the end-to-end system integration test.
The IRS needs to improve oversight of the end-to-end testing of key systems not included in its Servicewide test.

We also identified various ways in which the IRS needs to improve the coordination of its planning efforts for the end-to-end system integration test. These concerns were communicated to management in a memorandum dated August 11, 1998. The issues discussed in the memorandum are summarized at the end of the report.

The Internal Revenue Service Needs to Perform a Detailed Analysis and Risk Review of Its Systems to Support the End-to-End System Integration Test

Product Assurance selected the systems to be included in the end-to-end system integration test. These systems were selected primarily on a judgmental basis. Management did not perform a detailed risk analysis or in-depth review of the systems prior to their selection. We identified several systems not selected for the end-to-end system integration test that have similar risk attributes to selected systems. The systems we identified are used to initiate updates to taxpayer accounts and, in some cases, could result in additional contacts with taxpayers. There is no documentation regarding why these systems were not selected.

Although the systems to be included in the coordinated test have already been selected, a detailed analysis of the business value and potential risk exposure of all major IRS systems would provide benefit throughout the entire end-to-end test process. Ongoing benefits of this analysis would include the ability to:

Prioritize the end-to-end system integration testing being coordinated by the Product Assurance function, which could be critical in the event of unforeseen difficulties or schedule slippage.
Establish precedence for the testing of systems not selected for the Product Assurance coordinated test, which would ensure systems with a higher business value receive priority access to limited resources.
Identify all key Operations support systems not selected for testing.

Recommendation

The chief Information Officer should perform a detailed analysis of the business value and potential risk exposure of all major IRS systems and establish a priority ranking.

Management’s Response: The century Date change Project Office does not agree with the recommendation. A priority ranking for each system does not need to be implemented because all IRS systems will be end-to-end tested.

Office of Audit comments: The IRS has allotted a three-month period from October 1, 1999, through December 31, 1999, to conduct the end-to-end test of over 90 mission critical systems. Priority ranking of each system would show where to apply resources if either time or resources run short to complete the test of all systems. In lieu of the decision not to prioritize each system, it would be prudent for the IRS to plan some alternative actions.

The Internal Revenue Service Needs to Improve Oversight of the End-to-End Testing of Key Systems Not Included in Its Servicewide Test

Additional IRS management attention is needed to ensure key Operations support systems, not selected for the coordinated end-to-end system integration test, are identified and receive sufficient emphasis and resources. Presently, the owners of these systems are expected to perform their own testing based on instructions received from the Product Assurance function. There is no centralized oversight within Operations for this process.

Some of the key Operations systems not selected for the coordinated end-to-end system integration test include those responsible for controlling cases and initiating transactions related to the processing of Exempt Plans and Exempt Organization (EP/EO) Determinations, Bankruptcies, and Offers in compromise.

Systems not included in the coordinated end-to-end system integration test are expected to be tested locally for Y2K compliance using instructions prepared by the Product Assurance function. These instructions were not finalized until December 31, 1998.

Additional benefit to the IRS’ Y2K conversion effort could be obtained by mandating the use of the Product Assurance test bed for the review of all key Operations support systems not selected for coordinated testing. This would ensure these systems are analyzed for Y2K compliance in a secure and controlled environment. Use of the Product Assurance test bed for non-selected systems is currently optional for the system owners whose systems are not included in the end-to-end system integration test.

Recommendations

The chief Information Officer should identify all key Operations support systems not selected for the nationally coordinated end-to-end system integration test.

Management’s Response: The century Date change Project Office has identified the following Operations systems to be end-to-end tested by the system owners:

Automated Quarterly Excise Tax Listing (QETL)

Automated criminal Investigation (AcI)

Auto Insolvency System/Insolvency Input Program (AIS/IIP)

Automated Workload control System for Special Procedures Inventory (AWcS)

EP/EO Determination System (EP/EO System)

Excise Tax cell Phone System (Ex-Phone)

Excise Tax Tracking System (ExTRAS)

Exam Management Information System for Large cases (cEMIS)

Midwest Audit classification System (MAcS)

computer Assisted Publishing System (cAPS)

compliance Research Initiative Tracking System (cRITS)

Electronic Transcript Delivery System (ETDS)

Statistics of Income-Distributive Processing System (SOI)

The century Date change Project Office has established an end-to-end test team to provide technical coordination and support to system owners responsible for their own end-to-end testing. Key operations systems and system owners have been identified on the century Date change Project Office Systems List, and end-to-end test commitments and resource requirements are being obtained from system owners. The century Date change Project Office team will coordinate with system owners to identify resource requirements, support the establishment and implementation of system test plans, assist with the identification and scheduling of test bed and test bed alternatives, and serve as the primary interface among testers, Product Assurance, Telecommunications and other teams which may influence the timeliness and success of end-to-end testing.

IRS management should establish within the Operations function centralized oversight and control over the testing of key support systems.

Management’s Response: Operations has identified representatives for coordinating and reporting Y2K compliance activities and issues between the system owners and the century Date change Project Office. The century Date change Project Office team will coordinate with Product Assurance to provide technical guidance and training to system owners as necessary.

The chief Information Officer should mandate that key Operations systems owners use the Product Assurance test bed to perform their Y2K compliance testing.

Management’s Response: The century Date change Project Office and Assistant commissioner (Product Assurance) do not agree that the use of a Product Assurance test bed should be mandated. However, the century Date change Project Office will coordinate closely the scheduling of controlled test beds between systems owners and Product Assurance and has established an end-to-end test team to provide technical coordination and support to system owners responsible for their own end-to-end testing. The team will work closely with system owners and Product Assurance to schedule the controlled test beds for system owner testing wherever possible.

Other options for testing a controlled environment will also be explored on a case by case basis. System owners may be able to use the test facility normally used for system testing. Production platforms may also be scheduled for test time during scheduled system down times, before or after preventive maintenance periods, or by scheduling off-shift operations. In addition, test time may be able to be leased or purchased on third party test facilities.

Office of Audit comments: We concur that this alternative approach addresses our recommendation.

Issues Reported during the Review and Responded to by Internal Revenue Service Management

During our review, we assessed the initial planning efforts for the Y2K end-to-end system integration test and the execution of Test I. In an audit memorandum dated August 11, 1998, we made the following recommendations to improve the overall planning and execution of the end-to-end system integration test (management’s response is denoted in italics):

Information Systems should obtain formal agreements from testing partners for future testing activities for the field and customer functions. The Deputy chief Information Officer for Information Resources Management stated that for all future testing activities, they would issue Requests for Information Services, Memoranda of Understanding, and Service Level Agreements, as appropriate, with project partners to formalize support agreements.

Information Systems should oversee the development of more detailed project plans and schedules. The Deputy chief Information Officer for Information Resources Management stated that project documentation would be completed in more detail, and that steps had already been taken to improve the level of project communication.

The IRS needs to formally identify an end-to-end system integration test inter-organizational team that has representatives from all key testing partners, including field and customer organizations. The mission, roles, and responsibilities of this team should be communicated throughout the IRS in the form of an end-to-end system integration test team charter. The Deputy chief Information Officer for Information Resources Management stated that they were completing a revised century Date change End-to-End Integration Testing concept of Operations, which will provide the information we requested.

Information Systems should consider appointing an executive level official to provide the authority for ensuring that the end-to-end system integration test effort is given sufficient priority in light of other IRS commitments. The Deputy chief Information Officer for Information Resources Management disagreed with this recommendation and felt the end-to-end system integration test was progressing timely and in a coordinated effort, and that the current management structure for the end-to-end system integration test is sufficient.

Office of Audit comments: Recommendations number 2 and 3 in this report are directed at assuring that adequate end-to-end testing is performed for all mission critical systems, not just those included in the end-to-end system integration test. IRS Management has taken several steps to mitigate the risk we identified in our memorandum, in part by establishing an end-to-end test team to provide technical support to system owners responsible for their own end-to-end testing and by identifying representatives within Operations responsible for coordinating Y2K compliance. We believe that these actions may not suffice to assure that end-to-end testing efforts are given priority, especially when test activities may require testing partners outside of Information Systems to take actions that conflict with their organizational interests.

Because IRS management decided not to name an executive-level official, we recommend that the IRS reconsider its decision not to rank and prioritize each system to ensure resources are efficiently used. This relates to recommendation number 1 in the report.

The memorandum and management’s response are included as Appendices V and VI, respectively.

Appendix I

Detailed Objectives, Scope and Methodology

The objective of this review was to assess the Internal Revenue Service’s (IRS) overall planning efforts for the Year 2000 (Y2K) end-to-end system integration test and to evaluate the effectiveness of Test I, which was conducted in July and August 1998. Test I was designed to provide a limited assessment of whether the IRS’ systems software components, application components, and commercial off-the-shelf (cOTS) products will work through the year 2000. Specifically, we:

Reviewed the IRS’ overall end-to-end system integration test planning documentation to determine if the information sufficiently and completely documents all necessary testing activities and the necessary level of coordination between involved functional areas by:

Interviewing the necessary Information Systems officials (e.g., Product Assurance, Y2K Telecommunications Project Office, Office of Technical Support, etc.) and reviewing necessary documentation to determine if the items were in fact completed when scheduled, as items in the planning documents reached scheduled completion dates.

Assessing the impact on the overall testing effort and the IRS’ Y2K conversion efforts if planning items did not meet established deadlines.

Assessing the plans in place for testing external trading partner related components to help determine if the IRS’ trading partner issues were sufficiently addressed.
Reviewing the coordination and planning efforts for systems that were not part of the 93 tax processing systems being tested.
Evaluating the IRS’ methodology for performing a risk assessment of tax processing systems and identifying the ones to be included in the test. For the systems not included in the system test, we assessed the level of guidance and support provided by Product Assurance to the areas responsible for testing the systems.
Reviewing configuration Management efforts to ensure that components being tested were adequately controlled. This included the access, modification, documentation, and storage of tested components.
Evaluated the establishment of the test platforms to determine if the parallel-processing environment would be ready to run tests by:
Evaluating the level of operating responsibility Product Assurance had accepted for the testing sites (Detroit computer center, Martinsburg computer center, Tennessee computer center, and National Office) to determine if Product Assurance had the appropriate authority and staffing for operating the test platforms.

Determining how the IRS’ delays in fully assessing the Y2K impact of telecommunications would affect the establishment of the test platforms.

Reviewing the test system settings (via visit or remotely) to ensure that the systems clocks had in fact been set to the year 2000.
Observed Test I activities and reviewed results to determine if the test provided the IRS with assurance that its system software, applications, telecommunications, and cOTS products would operate correctly in the year 2000.
Determined if TRW, Inc. was providing the IRS with the services required in the contract for the IRS’ end-to-end system integration test effort by:
Reviewing the Statement of Work and deliverables for the end-to-end system integration test effort to identify the services TRW, Inc. was required to provide and if they were providing them.
Interviewing the Technical Point of contact for more specifics on the work expected from TRW, Inc.

Appendix II

Major contributors to This Report

Michael R. Phillips, Acting Director, Office of Audit Projects

Kent T. Sagara, Acting Deputy Director, Office of Audit Projects

Vincent Dell’Orto, Audit Manager

Tony Hubbard, Senior Auditor

Tony Knox, Senior Auditor

Terrey Haley, Senior Auditor

Regina Dougherty, Senior Auditor

Tony choma, Senior Auditor, Southeast Region

Phillip Dearth, Auditor, Southeast Region

Lynda Sanford, Auditor, Southeast Region