Executive Summary

Objectives and Scope

Background

Results

Conclusion

Appendix I – Detailed Objectives, Scope, and Methodology

Appendix II – Major Contributors to This Report

Appendix III – Report Distribution List

Appendix IV – Impediments Encountered by Northrup Grumman Technical Services, Incorporated

Appendix V – Schedule of Delays in Receiving Information and Responses

Appendix VI – Management’s Response to the Draft Report

Executive Summary

The Year 2000 presents a significant challenge to the Internal Revenue Service’s (IRS) operations. Most IRS computer systems employ a two-digit year format in date representations, rather than four digits. Unless the affected software and data files are corrected before January 1, 2000, serious problems may occur during tax processing.

In response to previous Treasury Inspector General for Tax Administration audit recommendations regarding the need for a review of converted application code to ensure Year 2000 (Y2K) compliance, the IRS contracted with Northrop Grumman Technical Services, Incorporated (Grumman). Specifically, Grumman was tasked to analyze 100 percent of the IRS’ application code to ensure conversion efforts were successful and to validate the Y2K compliance of 100 percent of the IRS’ tax processing commercial off-the-shelf (COTS) products with the manufacturer. Grumman’s efforts contribute to lowering the risk that significant errors will go undetected.

The overall objectives of this review were to determine whether the IRS timely implemented corrective actions to previous audit recommendations related to Y2K compliance testing and to assess the effectiveness of the IRS’ efforts to analyze and validate Y2K compliance of application code and tax processing COTS products.

The IRS has made significant progress in implementing corrective actions to previous audit recommendations, related to Y2K compliance testing efforts, to ensure the IRS meets its Y2K readiness objectives. The IRS has also taken actions to analyze all converted application code and to validate all tax processing COTS products. However, due to Information Technology (IT) inventory inaccuracies and delays in the IRS providing information for the contractor-performed analysis and validation, the IRS may not fully achieve its 100 percent analysis and validation objectives for Y2K readiness. As a result, the IRS cannot assure that all critical programs have been identified, are Y2K compliant, and will function properly in the Year 2000.

Our review of 22 Y2K compliance testing-related corrective actions scheduled to be implemented by January 1, 1999, showed that 15 corrective actions were timely implemented. Of the remaining seven corrective actions, five were either implemented late or had their estimated implementation dates extended. The risk of these five corrective actions on the IRS’ Y2K objectives is minimal. However, the remaining two corrective actions, concerning the certification of the IRS’ IT inventory, had not been implemented as of May 12, 1999. These two inventory certification corrective actions have a significant impact on Y2K implementation risk, since the IRS has based its processes for ensuring the timely analysis of all application code and validation of all tax processing COTS products on its IT inventory.

The IRS contracted with Grumman, in August 1998, to analyze all application code associated with the IRS’ seven Y2K conversion phases. As of May 18, 1999, Grumman reported that it had analyzed over 30 million lines of code for phases one through four and identified 8,864 potential errors. Of the potential errors, 7,245 have been explained without corrections needed, 1,456 were confirmed as errors, and 163 had not been responded to by the IRS.

Though Grumman has made significant progress to date, it may not meet its ultimate objective of a 100 percent analysis due to impediments and delays in receiving information and responses from the IRS. Phases one through four were originally scheduled to be completed by December 1, 1998. As of May 18, 1999, Grumman still had not completed the analysis of all the lines of application code in those phases due to the delays in receiving responses and all application code from the IRS. Although the remaining code to be analyzed for the first four phases is small, the IRS does not know which of those components could result in a Y2K processing error.

The IRS has taken actions, such as providing guidance and distributing memoranda to its staff on past-due items, which have improved the IRS’ responsiveness to Grumman’s requests. However, because phases one through four were five months past due as of the end of our audit work and similar delays are being experienced in the analysis of phase five application code, we are concerned that the IRS will be unable to achieve its objective of 100 percent analysis of application code by the Year 2000. By not achieving its objective, the IRS cannot assure that all critical programs have been identified, are Y2K compliant, and will function properly in the Year 2000.

The IRS has also contracted with Grumman to validate Y2K compliance of all tax processing COTS products with the manufacturer. Grumman began the validation process in the beginning of April 1999 and, as of May 19, 1999, had reviewed 2,864 of the approximately 11,300 identified products, which resulted in the identification of 999 potential errors. The majority of the potential errors were related to inaccuracies in the IRS’ IT inventory.

Based on the numerous IT inventory errors which have to be corrected before the validation process can continue, we are concerned that problems similar to those experienced with code analysis will delay the process of validating the Y2K compliance of COTS products. As a result, the IRS may not achieve its objective of 100 percent validation of COTS products by the Year 2000.

By maintaining an IT inventory that is complete and accurate, the IRS can ensure that all application code is analyzed and all COTS products are validated, which reduces the risk that computer systems will not function properly in the Year 2000. Furthermore, by ensuring that all application code and COTS products are Y2K compliant, the IRS reduces the risk that taxpayers will be adversely burdened by system failures in the Year 2000. The proper functioning of the IRS systems will ensure that taxpayers receive quality service, including timely refunds, and the avoidance of erroneous notices.

To ensure that the 100 percent analysis of application code and validation of tax processing COTS products is completed timely, we recommend that IRS executive management ensure those organizations that have not certified their respective IT inventories do so immediately. In addition, management should ensure timely responses are provided to the contractor’s requests for information and responses. Further, in the event that the 100 percent analysis and validation will not be completed, management should prioritize application code and COTS products, to ensure critical applications are Y2K compliant.

Management’s Response: IRS management has initiated a four-fold approach to verification and certification of the IT inventory. This includes a physical wall to wall inventory certification, an independent audit and readiness verification, a contractor performed independent verification and validation, and the issuance of a memorandum instructing IRS executives to provide personal attention to outstanding Y2K issues. Further, the Century Date Change (CDC) Project Office will report to the Chief Information Officer and the Commissioner weekly on open exception items, including completion of code review items. The CDC Project Office will also send a weekly report to the appropriate executives each week for any items that are overdue to the contractor performing the COTS validation. Since the review of applications code conversion work is nearly complete and all COTS products have been through an initial review, IRS management believes that any prioritization would not be beneficial. Management’s complete response to the draft report is included as Appendix VI.

Objectives and Scope

The overall objectives of this review were to determine whether the Internal Revenue Service (IRS) timely implemented corrective actions to previous Treasury Inspector General for Tax Administration audit recommendations related to Year 2000 (Y2K) compliance testing and to assess the effectiveness of the IRS’ efforts to analyze and validate Y2K compliance of application code and tax processing commercial off-the-shelf (COTS) products. We reviewed Y2K testing-related corrective actions scheduled to be implemented between July 1, 1998, and January 1, 1999. We also reviewed the processes in place for the IRS’ 100 percent code analysis and COTS validation to determine whether the processes would accomplish their intended objectives and whether they were progressing in a timely manner.

Audit work was performed at the IRS’ National Office, Delaware/Maryland District Office, and Connecticut/Rhode Island District Office from January to June 1999. This audit was performed in accordance with Government Auditing Standards.

The detailed audit objectives, scope, and methodology of this review are presented in Appendix I. A listing of major contributors to this report is contained in Appendix II.

Background

The Year 2000 presents a significant challenge to IRS operations. Most IRS computer systems employ a two-digit year format in date representations, rather than four digits. Unless the affected software and data files are corrected before January 1, 2000, serious problems may occur during tax processing.

In response to previous audit recommendations regarding the need for a review of converted application code to ensure Y2K compliance, the IRS contracted with Northrop Grumman Technical Services, Incorporated (Grumman). Specifically, Grumman was tasked to analyze 100 percent of the IRS’ application code to ensure conversion efforts were successful and to validate the Y2K compliance of 100 percent of IRS’ tax processing COTS products with the manufacturer. Grumman’s efforts contribute to lowering the risk that significant errors will go undetected.

Results

The IRS has made significant progress in implementing corrective actions to previous audit recommendations to ensure the IRS meets its Y2K readiness objectives. The IRS has also taken actions to analyze all converted application code and to validate all tax processing COTS products. However, due to Information Technology (IT) inventory inaccuracies and delays in the IRS providing information for the contractor-performed analysis and validation, the IRS may not achieve its objectives for Y2K readiness.

The Internal Revenue Service Has Made Progress in Implementing Corrective Actions; However, Information Technology Inventory Certifications Remain Incomplete

We reviewed 22 Y2K compliance testing-related corrective actions scheduled to be implemented between July 1, 1998, and January 1, 1999. Through discussions with personnel responsible for implementation and review of related documentation, we determined that 15 corrective actions were timely completed and consistent with those recommended. Of the remaining seven corrective actions:

• Three were completed between three weeks to 5 1/2 months late.
• Two had their estimated completion date extended from January 1, 1999, to June 1, 1999.
• Two, with an estimated completion date of December 8, 1998 (changed from a prior estimate of September 30, 1998), were not completed as of May 12, 1999.

The three corrective actions completed late related to defining when PIC 9 standards for date fields apply, requiring formal support agreements for test activities, and ensuring project folders for phase three COTS products contain the required documentation. While these corrective actions were not completed by the estimated implementation date, we found no apparent impact on the IRS’ Y2K readiness objective.

The two corrective actions for which the estimated completion dates were extended to June 1, 1999, called for the upgrading of guidelines in the Internal Revenue Manual. While we are concerned that the IRS missed the January 1, 1999, target date, we believe the impact of the delay is minimal, since the information is available to developers and testers in the "Developer’s Testing Guidelines."

The two uncompleted corrective actions that were to be completed by December 8, 1998, required all 38 IRS organizations to certify the accuracy and completeness of their portion of the Integrated Network and Operations Management System (INOMS) inventory. When the estimated completion date was not met, IRS’ Information Systems function issued a memorandum that required a response by January 29, 1999. As of May 12, 1999, 5 of the 38 organizations had not submitted their certifications. The five organizations are the Tennessee Computing Center, Chief Operations Office, Midstates Region, Northeast Region, and the End-User Computing Support Division under the Assistant Commissioner Information Systems Field Operations.

The process for ensuring the timely analysis of all application code and validation of all tax processing COTS products is contingent on the accuracy and completeness of the INOMS inventory. When organizations do not timely certify that the INOMS is accurate and complete, there is the potential that some items may not be tested by the organization or subsequently analyzed and validated as Y2K compliant. As a result, the IRS' objective of completing a 100 percent analysis of application code and validation of COTS products before January 1, 2000, may be jeopardized.

1. IRS executive management should require the organizations that have not certified as to the accuracy and completeness of the INOMS inventory to do so immediately.

Management’s Response: IRS management has initiated a four-fold approach to verification and certification of the IT inventory. This includes a physical wall to wall inventory certification, an independent audit and readiness verification, a contractor performed independent verification and validation, and the issuance of a memorandum instructing IRS executives to provide personal attention to outstanding Y2K issues.

The 100 Percent Analysis of Application Code May Not Be Completed by the Year 2000

The IRS contracted with Grumman to analyze all application code associated with the seven phases of the IRS’ Y2K conversion efforts. Grumman’s work, which we established to be consistent with the contract task order to analyze phases one through four, began on August 24, 1998. As of May 18, 1999, Grumman reported it had analyzed over 30 million lines of code and identified 8,864 potential errors. Of the potential errors, 7,245 have been explained without corrections needed, 1,456 were confirmed as errors, and 163 had not been responded to by the IRS. Despite the progress made, impediments and delays in receiving information and responses from the IRS resulted in Grumman being unable to complete the task by the original target date of December 1, 1998. As of May 18, 1999, Grumman had not completed its analysis of phases one through four.

One of the major problems encountered by Grumman was a delay in receiving system software components. For example, on December 16, 1998, Grumman reported that it was awaiting receipt of 6,044 components to complete phases one through four. As of May 18, 1999, Grumman reported that it had not received 28 components needed to complete its work, which is a significant reduction in outstanding components. However, by taking five months to accomplish this reduction, the IRS exhibited a lack of responsiveness in assisting the contractor in the timely completion of its objective.

Grumman has also experienced delays due to the inaccuracy of the INOMS inventory. Grumman compared the components obtained from the computer systems to the components listed in the INOMS and identified numerous components that were not included in the INOMS. As of January 5, 1999, Grumman reported that it had received 2,372 components that were not on the INOMS. Although this number was reduced to 12 components as of May 18, 1999, the numerous components not on the INOMS created additional work for Grumman, since it required Grumman to independently track the status of these components until they were added to the INOMS.

For additional examples of impediments and a schedule of delays in receiving information and responses encountered by Grumman, see Appendices IV and V, respectively.

The Information Systems Product Assurance Division’s Certification Section has initiated various actions to resolve these issues. For example, the section provided guidance on:

• Sending components to Grumman for analysis.
• Completing the potential error listing document.
• Re-sending components, in which errors were fixed, for analysis.

The Information Systems Century Date Change (CDC) Project Office also appointed a person to work with the organizations to resolve various INOMS issues. In addition, we were informed that when the Y2K Program Director started sending out issue alerts to the organizations, the organizations started inquiring on how to alleviate the situation. As a result of these efforts, which were initiated during the period of our audit, the organizations started resolving outstanding issues.

Despite the steps being taken, we have observed that some of the same impediments and delays have been encountered in the analysis of phase five application code. Grumman began the review of phase five in January 1999 with an estimated completion date of March 15, 1999. Due to the continued impediments and delays, the estimated completion date was changed to May 28, 1999; however, this completion date was also not achieved. Grumman began the review of phase six in April 1999 and has yet to begin phase seven. If the impediments and delays continue, Grumman may be further hindered in the timely completion of its code analysis.

Grumman is capable of reviewing millions of lines of code per week; however, if the code is not available for review, required analysis cannot be completed. Based on the time expended to date on phases one through four, and the fact that similar delays caused by the IRS’ process are being experienced in phase five, we believe the potential exists that the IRS may be unable to achieve the 100 percent analysis of all applications code by the year 2000.

1. IRS executive management should improve its process for ensuring that Grumman receives information and responses that are critical to the successful performance and completion of its task.

Management’s Response: The CDC Project Office is tracking code validation progress through numerous reports during the CDC Progress Report weekly meetings of IRS organizations. The CDC Project Office is tracking code validation progress during the CDC Progress Report weekly meetings of IRS organizations, through numerous reports. The closure of all open exception items, including completion of code review items, will be the subject of weekly reports to the Chief Information Officer and the Commissioner.

2. In the event that the 100 percent analysis will not be completed, IRS executive management should prioritize the application code to ensure that the most critical code is Y2K compliant.

Management’s Response: The review of the applications code conversion work is nearly complete. At this point in the conversion process, trying to divide the last few percent of the application code into mission critical vs. non-mission critical items would not be beneficial. As a result, IRS intends to review all of the application code.

The 100 Percent Validation of Commercial Off-the-Shelf Products May Not Be Completed by the Year 2000

The IRS contracted with Grumman to conduct a validation of all tax processing COTS products that are listed in the INOMS inventory. Grumman was tasked with contacting manufacturers to identify the Y2K compliance of product versions and to verify the information against the INOMS inventory.

The scope of Grumman’s work, which we established to be consistent with the IRS’ Y2K readiness objectives, includes all tier one, two, three, and telecommunication COTS products that are designated in the INOMS product table as Y2K compliant or as having no Y2K compliance impact. Grumman’s process involves verifying COTS at the product milestone level rather than the device level. Per the INOMS product table, there are approximately 11,300 COTS products to be validated by Grumman.

Grumman began the validation of products on April 5, 1999, and, as of May 19, 1999, it had reviewed 2,864 products and identified 999 potential errors. Of the potential errors, 146 were Y2K compliance errors and 853 were errors related to inaccuracies with the INOMS inventory data. Because of the numerous INOMS errors which have to be corrected by the IRS before the validation process can continue, problems similar to those experienced with the code analysis may prevent the completion of a 100 percent validation of COTS products before January 1, 2000. For example, during the review of application code, various organizations were slow in providing required information and responding to error lists submitted by Grumman, which delayed the process. Grumman stated in its COTS project plan that any delays in the IRS part of the process will affect completion times.

In addition, the incompleteness and inaccuracy of the INOMS inventory may affect the IRS’ objective of 100 percent COTS validation, since Grumman relies on the INOMS as the basis for its validation. For instance, we identified a tier three product that was entered in the INOMS under a different product name, with the actual product being described in the comments section of the device record. This was done because the product table did not contain the actual product name. According to the Grumman project manager and the IRS Technical Point of Contact, Grumman does not review device records and, at the present time, it does not have a process to independently identify tier three products not listed on the INOMS. Therefore, the item would most likely not be identified and validated.

In another instance, Grumman identified that certain organizations are not using the INOMS to record telecommunications products. Since the products are not listed on the INOMS, Grumman will have no basis for selecting them for validation. If Grumman identifies the products through other means, it will have to classify them as inventory errors to which the responsible organization will have to respond before the validation process can continue. This activity could further delay Grumman’s validation efforts.

We are concerned that Grumman’s task of validating all COTS products may not be completed prior to the Year 2000 due to the numerous INOMS inventory inaccuracies already identified and the possibility that delays, similar to those experienced during the application code analysis, will occur. As a result, the IRS may not achieve its objective of 100 percent COTS validation.

Recommendations

3. IRS executive management should ensure the organizations that receive error lists understand the importance of timely responding so that Grumman can complete its task on a timely basis.

Management’s Response: The CDC Project Office is tracking code validation progress through numerous reports during the CDC Progress Report weekly meetings. A report is sent to the appropriate executives each week for any items that are overdue to Grumman and the CDC Project Office continues to monitor these remaining items.

4. In the event that the 100 percent validation will not be completed, IRS executive management should prioritize the COTS products to ensure that the most critical COTS products are Y2K compliant.

Management’s Response: COTS products are not always linked to systems and it cannot be determined if they are mission critical or non-mission critical. Therefore, the CDC Project Office set a goal to have all COTS products reviewed. As of August 25, 1999, all the COTS products have been through an initial review. In addition, the CDC Project Office has established a process to control and track the closure of all COTS related problems.

Conclusion

The IRS may not have complete assurance that all applications code and tax processing COTS products will function properly in the Year 2000. The IRS needs to ensure that Grumman receives the information required to perform its analysis and validation tasks on a timely basis. In addition, since Grumman uses the INOMS inventory as its basis for analysis of application code and validation of COTS products, INOMS needs to be accurate and complete to ensure the IRS can timely achieve its Y2K readiness objective of 100 percent analysis and validation.

Detailed Objectives, Scope, and Methodology

The overall objectives of this review were to determine whether the Internal Revenue Service (IRS) timely implemented corrective actions to previous audit recommendations related to Year 2000 (Y2K) compliance testing and to assess the effectiveness of the IRS’ efforts to analyze and validate Y2K compliance of application code and tax processing commercial off-the-shelf (COTS) products. Specifically, we:

1. Determined if the corrective actions related to testing efforts, scheduled to be completed by January 1, 1999, in response to recommendations from recent reviews, adequately met the requirements of the recommendations.

1. Identified the 22 corrective actions that were due to be implemented between July 1, 1998, and January 1, 1999.
2. Established the current status of the corrective actions through interviews of responsible personnel and review of documentation.
3. Through review of supporting documentation, determined whether completed corrective actions were fully consistent with those proposed or recommended.
4. For completed corrective actions not meeting their target dates, interviewed responsible officials to analyze the impact of late completion.
5. For corrective actions not completed, interviewed responsible officials to identify the reason for missed completion dates and to identify planned remedies to address them.

1. Evaluated the process for ensuring that the contractor-performed 100 percent code review of Information System (IS) and non-IS developed applications is progressing as intended.

1. Determined whether the sub-tasks being performed by the contractor were consistent with the scope of the task order.
2. Evaluated the effect that impediments encountered by the contractor had on the progress of the testing and the effectiveness with which they were addressed/remedied.
3. Evaluated the effectiveness with which potential errors identified by the contractor were resolved by the IRS and the timeliness of the resolution process.
4. Evaluated the process that was implemented to review the IRS’ sensitive code and determined its effectiveness and ability to meet established completion times.

1. Assessed the IRS’ efforts to ensure that the Year 2000 compliance initiatives for tax processing COTS products were progressing as planned.

1. Visited the Delaware/Maryland and Connecticut/Rhode Island District Offices to gain an understanding of a typical office configuration and to determine whether all COTS products are accounted for on the INOMS.

Major Contributors to This Report

Scott Wilson, Associate Inspector General for Audit (Information Systems Programs)

Michael Phillips, Director

Thomas Brunetto, Audit Manager

Jill Moore, Senior Auditor

Gwen Bryant-Hill, Auditor

Bobbie Draudt, Auditor

Andrew Harvey, Auditor

Eleonor Lindner, Auditor

Report Distribution List

Deputy Commissioner for Modernization C:DM

Deputy Commissioner Operations C:DO

Chief Information Officer IS

Deputy Chief Information Officer, Operations IS

Deputy Chief Information Officer, Systems IS

Assistant Commissioner (National Operations) IS:O

Assistant Commissioner (Product Assurance) IS:PA

Assistant Commissioner (Program Evaluation and Risk Analysis) M:OP

National Director for Legislative Affairs CL:LA

Office of the Chief Counsel CC

Audit Liaisons:

Deputy Chief Information Officer, Operations IS:O

Impediments Encountered by Northrop Grumman Technical Services, Incorporated

Schedule of Delays in Receiving Information and Responses

Management’s Response to the Draft Report