Based on prior Internal Audit recommendations, Service management has initiated actions to improve the Service’s Year 2000 certification efforts and improve the accuracy of the Service’s Applications Program Registry (APR) data. The Service is also working toward being classified as a capability Maturity Model Level 2 organization to institutionalize effective management processes for software projects, which will allow for repetition of successful practices developed on earlier projects. We recognize these efforts are not yet completed.

Our results emphasize the need for the Service to continue with its efforts to improve the accuracy of the APR. The Service also needs to ensure Service developers and testers document test activities as well as contractors, and the Service needs to give greater consideration to how Year 2000 programming changes will affect hardware capacity and system performance on Tier 2 and Tier 3 systems.

The overall objective of this audit was to perform an assessment of Phase 4 of the Service’s Year 2000 conversion and testing efforts.

In addition to the three specific findings of this audit, we noted that the century Date change (cDc) Project Office has not established any controls to verify the accuracy of Year 2000 compliance certifications. Since these are not validated, the cDc Project Office is unable to fully assess the risk that exists for programs that are not actually Year 2000 compliant.

At the end of our audit fieldwork, efforts to link components tracked on the APR with a standardized project name and phase had not yet been completed. In addition, as previously reported, conversion and testing dates on the APR did not accurately reflect the status of Phase 4 Year 2000 conversion activities.

The Product Assurance Division cannot effectively monitor the status of project conversion and testing efforts, follow up on projects falling behind schedule, accurately report the status of conversion and testing, or make sound project and configuration management decisions when inaccuracies exist on the APR.

Project files prepared by contractors were complete, cross-referenced, and easy to follow, unlike those prepared by Service developers and testers. The Service’s project files did not always reflect testing for invalid conditions or the reasons why invalid testing was not performed.

Insufficient documentation of programming and testing activities can delay Year 2000 project certification and hinder the Service’s efforts to be classified as a capability Maturity Model Level 2 organization. Given the significance of the Year 2000 programming efforts, testing for invalid conditions should be performed to identify and avert any potential system errors or failures.

The Service does not have a corporate plan in place to address the capacity management and performance evaluation issues that may arise due to Year 2000 conversions. Quarterly assessments of Year 2000 capacity and performance issues will be done for Tier 1 systems; however, adequate oversight consideration is not being given to how Year 2000 programming changes will affect hardware capacity and system performance of Tier 2 systems. In addition, no capacity or performance evaluations have been completed for systems that will be running Year 2000 compliant applications software on Tier 3 hardware, such as the Integrated collection System (IcS) which supports over 10,000 workstations.

If capacity and performance evaluations of major Tier 2 and Tier 3 systems are not performed, the potential exists that system timeliness could be impaired or that additional hardware capacity could be needed.

This report makes seven recommendations for the Service to ensure it can effectively meet its century date change needs. In summary, they are:

• The Product Assurance Division should monitor organizational efforts to re-certify the data on the APR.
• Service developers and testers should document and cross-reference programming and testing activities and prepare data to test for invalid conditions or document why such tests are not required.
• The Service needs to proactively consider and evaluate potential capacity and performance issues in preparing for its Year 2000 systems environment.

Management Response: Management’s response was not available for inclusion in the report at the time the final report was issued. We were informed that management is developing actions to address our concerns and will provide us with a written description of their proposed corrective actions at a later date.