The Customer Account Data Engine 2 Program Management Office Implemented Systems Development Guidelines; However, Process Improvements Are Needed to Address Inconsistencies

HIGHLIGHTS

THE CUSTOMER ACCOUNT DATA ENGINE 2 PROGRAM MANAGEMENT OFFICE IMPLEMENTED SYSTEMS DEVELOPMENT GUIDELINES; HOWEVER, PROCESS IMPROVEMENTS ARE NEEDED TO ADDRESS INCONSISTENCIES

Highlights

Final Report issued on September 30, 2011

Highlights of Reference Number: 2011-20-127 to the Internal Revenue Service Chief Technology Officer.

IMPACT ON TAXPAYERS

The mission of the Customer Account Data Engine (CADE) 2 Program is to provide state‑of‑the-art individual taxpayer account processing and technologies to improve service to taxpayers and enhance Internal Revenue Service (IRS) tax administration. Once completed, the new modernization environment should allow the IRS to more effectively and efficiently update taxpayer accounts, support account settlement and maintenance, and process refunds on a daily basis, all of which will contribute to improved taxpayer services.

WHY TIGTA DID THE AUDIT

The overall objective of this review was to determine if the CADE 2 Program Management Office planned and provided oversight for Transition State 1 design activities in accordance with systems development guidelines, including applicable security provisions.

WHAT TIGTA FOUND

The CADE 2 Program Management Office implemented guidelines to cover key systems development processes. Due to the critical nature of the system to the IRS mission, 18 enhanced security controls above those required by security guidelines were added to the CADE 2 system to help protect data from unauthorized access, modification, and corruption. Improvements are still needed to ensure Program consistency. Specifically: 1) systems development guidelines and related processes were not consistently implemented by CADE 2 personnel, and 2) requirements and business rules were not sufficiently developed and traced to their sources before the CADE 2 exit of design activities. The IRS implemented corrective actions; however, some were not developed or completed prior to the conclusion of our audit.

WHAT TIGTA RECOMMENDED

TIGTA recommended that the Chief Technology Officer ensure project test plans are developed timely; the Internal Revenue Manual and other guidelines are revised to include Program-level test plans; and a comprehensive Integrated Master Schedule is developed. TIGTA also made recommendations for the IRS to improve the processes associated with managing business rules and requirements.

In its response, the IRS agreed with four of TIGTA’s five recommendations and indicated that corrective action had been completed. The IRS disagreed with TIGTA’s recommendation to revise the Enterprise Life Cycle guidance, stating it is for project development and is not intended to provide for detailed instructions on developing a Program-level test plan. Rather, the IRS agreed to reconcile two systems development documents it considers as being consistent with the purpose, scope, and timing of the Program Test Plan, and plans to maintain program-level guidance about the process. TIGTA agrees this alternative approach addresses the condition.

The Chief Technology Officer also stated that our finding regarding delays in developing the Program Test Plan appeared inaccurate, citing uncertainty or unfamiliarity with the Program Test Plan’s content was not a factor in the decision to defer delivery. However, during our audit fieldwork, CADE 2 Program Management Office staff advised us that they were considering not completing the Program Test Plan, and only did so after TIGTA brought this to the attention of the CADE 2 Director for Delivery Management.

September 30, 2011

MEMORANDUM FOR CHIEF TECHNOLOGY OFFICER

FROM: Michael R. Phillips /s/ Michael R. Phillips

Deputy Inspector General for Audit

SUBJECT: Final Audit Report – The Customer Account Data Engine 2 Program Management Office Implemented Systems Development Guidelines; However, Process Improvements Are Needed to Address Inconsistencies (Audit # 201020025)

This report presents the results of our review of the Customer Account Data Engine 2 Program Management Office. Our overall objective was to determine if the Program Management Office planned and provided oversight for Transition State 1 design activities in accordance with systems development guidelines, including applicable security provisions. This review was requested by the Chief Technology Officer. It was included in our Fiscal Year 2011 Annual Audit Plan and addresses the major management challenge of Modernization.

Management’s complete response to the draft report is included as Appendix X.

Copies of this report are also being sent to the Internal Revenue Service managers affected by the report recommendations. Please contact me at (202) 622-6510 if you have questions or Alan Duncan, Assistant Inspector General for Audit (Security and Information Technology Services), at (202) 622-5894.

Table of Contents

Background

Results of Review

The Program Management Office Implemented Procedures to Manage Systems Development Activities and Ensure Executive Oversight

Systems Development Processes and Program Guidelines Were Not Always Consistent

Recommendations 1 and 2:

Recommendation 3:

Requirements Management Processes Were Not Performed in Accordance With Established Guidelines

Recommendations 4 and 5:

Appendices

Appendix I – Detailed Objective, Scope, and Methodology

Appendix II – Major Contributors to This Report

Appendix III – Report Distribution List

Appendix IV – Outcome Measure

Appendix V – Enterprise Life Cycle Overview

Appendix VI – Customer Account Data Engine 2 Transition States

Appendix VII – Transition State 1 Integration Reviews

Appendix VIII – The Customer Account Data Engine 2 High and Enhanced Requirements

Appendix IX – Glossary of Terms

Appendix X – Management’s Response to the Draft Report

Abbreviations

CADE	Customer Account Data Engine
CCB	Configuration Control Board
ELC	Enterprise Life Cycle
IMF	Individual Master File
IMS	Integrated Master Schedule
IRS	Internal Revenue Service
ReqPro	Rational RequisitePro
TIGTA	Treasury Inspector General for Tax Administration

Background

The Customer Account Data Engine 2 Program is the highest priority information technology modernization project in the Internal Revenue Service.

In August 2008, the Internal Revenue Service (IRS) Commissioner established the Modernized Taxpayer Account Program Integration Office to manage the transition of the current individual income tax processing, which consists of multiple computer systems for processing tax returns, payments, and other transactions affecting taxpayer accounts, into a more consolidated system. Working in conjunction with IRS business owners, the Program Integration Office decided to integrate elements from both the existing Individual Master File[1] (IMF) and current Customer Account Data Engine (CADE) processes into a new CADE 2 Program. The proposed plan incrementally transfers taxpayer accounts from the current IMF and CADE processing systems to a new CADE 2 relational database.

The CADE 2 Program is the top information technology modernization project in the IRS. The CADE 2 strategy involves three phases:

· Transition State 1. Modifies the IMF from a weekly cycle to daily processing; establishes a new relational database to store all individual taxpayer account information; and provides management tools to more effectively use data for compliance and customer service. The IRS plans to implement Transition State 1 in January 2012.

· Transition State 2. Launches a single processing system where applications directly access and update the taxpayer account database. It will continue efforts toward addressing previously identified financial material weaknesses. The IRS plans to implement Transition State 2 in January 2014.

· Target State. Consists of a single system using elements of the IMF and current CADE, eliminating all transitional applications used to link the current CADE, the IMF, and the Integrated Data Retrieval System. The complete solution is also planned to address all of the financial material weaknesses. As of April 28, 2011, the IRS had not established a Target State implementation date.

Appendix VI presents conceptual models for the As Is, Transition State 1 and 2, and Target State process flowcharts for individual income tax accounts.

The CADE 2 Program Management Office was established with a mission to provide state‑of‑the‑art individual taxpayer account processing and technologies to improve service to taxpayers and enhance IRS tax administration. It published a charter on January 28, 2010. The CADE 2 Program Management Office plans to create a modernized processing environment where applications both access and update an authoritative relational database to manage all individual taxpayer accounts. The CADE 2 Program goals and scope are depicted in Figure 1.

Figure 1: CADE 2 Program Goals and Scope

CADE 2 Program Goals

Establish a solid data foundation for the future by leveraging relational database processing capability.

Address financial material weaknesses, demonstrate compliance with Federal Financial Management System Requirements, and maintain a clean audit opinion.

Improve security and privacy posture by addressing identified weaknesses.

Continue the focus on moving away from 1960’s technology (i.e., aging infrastructure, applications, and sequential flat file processing).

Demonstrate substantive progress toward achieving long-term viability.

CADE 2 Program Scope

Establish the authoritative database for individual taxpayer accounts.

Replace the current IMF and CADE applications with a single, state-of-the art solution.

Expand the Integrated Production Model to include individual taxpayer accounts.

Provide daily outputs to the Integrated Data Retrieval System and other downstream systems in support of daily processing.

Source: CADE 2 Program Charter Version 1.0, dated January 28, 2010.

To implement Transition State 1, the IRS established two systems development projects and completed several prototypes. The objective of each prototype was to demonstrate confidence in the CADE 2 approach by verifying system viability and performance and defining components to serve as the foundation for development activities. The Treasury Inspector General for Tax Administration (TIGTA) issued a report on the results of the prototypes on November 24, 2010.[2] In addition, the TIGTA has recently completed audits covering the two CADE 2 systems development projects—Daily Processing and Database Implementation.[3]

This review was requested by the Chief Technology Officer and was performed at the Modernization and Information Technology Services organization facilities in New Carrollton, Maryland, during the period April 2010 through May 2011. During audit fieldwork, the TIGTA concurrently advised CADE 2 Program officials when issues were identified and suggested corrective actions. The CADE 2 Program Management Office implemented several management corrective actions during the course of the audit. The TIGTA communicated interim audit results and recommendations for improvement to the Associate Chief Information Officer for Modernization – Program Management Office on February 24, 2011, and April 14, 2011.

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objective. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objective. Detailed information on our audit objective, scope, and methodology is presented in Appendix I. Major contributors to the report are listed in Appendix II.

Results of Review

The Program Management Office Implemented Procedures to Manage Systems Development Activities and Ensure Executive Oversight

The CADE 2 Program Management Office has taken initial steps to reduce the risks associated with using new techniques and processes in the Modernization Program. The CADE 2 Program is sponsored by the IRS Commissioner, Deputy Commissioners, Chief Technology Officer, and Wage and Investment Division Commissioner. These sponsors have established organizational commitments and a governance structure to assist in meeting CADE 2 Program goals. Establishment of a governance structure is important to the success of the CADE 2 Program, as it ensures high-level IRS officials oversee and approve critical aspects of systems development.

The CADE 2 Program Management Office ensured the Program Charter established a governance model and procedures and that governance groups, including the Executive Steering Committee and Governance Board, were fully engaged in these processes.

The Chief Technology Officer oversees the Executive Steering Committee, whose members include the Chief Information Officer of the Department of the Treasury and the Commissioner of the IRS Wage and Investment Division. This Committee provides oversight to ensure alignment of the CADE 2 Program and the IRS Strategic Plan and approves decisions having significant organizational or external impact, such as changes to Program goals or policy requirements.

The Associate Chief Information Officer for Modernization – Program Management Office oversees the Governance Board, whose members include the Business Modernization Executive of the IRS Wage and Investment Division. The Board maintains the CADE 2 Program scope, provides guidance, removes obstacles, and cultivates organizational commitment at all levels.

The CADE 2 Program Management Office developed the Program Framework to supplement the Enterprise Life Cycle (ELC).[4] The Program Framework establishes guidance for a single Program Management Office to manage multiple, ongoing information technology systems development projects by defining necessary life cycle phases, activities, and review points. Adherence to Program Framework guidelines is monitored through key systems development processes and recurring Program-level meetings.

Management oversight processes were established

The CADE 2 Program Management Office established oversight meetings and guidelines for key systems development processes to ensure efficient and effective program management. Specifically:

· Program-level oversight meetings: The CADE 2 Program Management Office established oversight processes which include quarterly and monthly briefings to the Chief Technology Officer and a series of weekly, biweekly, and monthly meetings between Program Management Office executives, staff, and project teams. Cybersecurity organization personnel participated in CADE 2 Program weekly meetings to provide early insight into the identification and development of required security controls.

The Executive Steering Committee meets quarterly, while the Chief Technology Officer receives monthly status briefings. In addition, monthly meetings are held to discuss risks and issue management, and weekly meetings are held for a review over activities and progress on the Integrated Master Schedule (IMS). The CADE 2 Program Management Office established several integration reviews to ensure the Program was making appropriate progress across all activities within a transition state and moving in an integrated way towards the defined transition state solution. Appendix VII presents the integration reviews and their purposes.

· Guidelines for key systems development processes were issued: The CADE 2 Program Management Office issued guidelines to cover key processes. The CADE 2 Program’s core description document is presented in the Solution Architecture, which provides a solution that responds to the objectives and capabilities described in the CADE 2 Program Charter. Guidelines were also issued for critical processes such as requirements management, risk and issue management, and configuration management.

Enhanced security controls are planned for the CADE 2 system

The CADE 2 Governance Board approved the Milestone 3 exit in December 2010. As part of this exit process, the Program Management Office prepared two artifacts pertaining to security controls for the CADE 2 system—the Security Strategy and the Security Framework. The Security Framework provides a high-level view of security and sets the tone of the information security solution throughout the Program. More detailed strategies will be presented in other Program-level and project-level documents. The Security Strategy outlines the IRS’s plans for applying resources to mitigate the security risks of developing, implementing, and operating the CADE 2 system.

As previously mentioned, Transition State 1 includes two major changes: establishing a relational database and moving to daily processing of the IMF. According to the Security Framework document, since the IMF will not undergo any major changes to its architecture, the security controls will largely remain as they currently exist. The Security Framework, however, will apply to the new database system and its components.

Both the Security Strategy and Security Framework documents outline the IRS’s plans to provide for enhanced security controls due to the sensitive nature of the taxpayer data stored in the CADE 2 system. The IRS determined that the aggregation of taxpayer information, numbering in excess of 130 million individual records, warrants an enhanced level of security controls to help protect CADE 2 system data. The loss or theft of this data would significantly damage taxpayers, as well as hurt the IRS’s reputation. To mitigate this risk, the IRS intends to implement enhanced security controls. These controls exceed the minimum guidelines required by the Recommended Security Controls for Federal Information Systems and Organizations (National Institute of Standards and Technology Special Publication 800-53, Revision 3). Further, the IRS will adopt a data-centric security approach whereby the focus will be on assessing and mitigating the risk to the data stored on the system versus the risk to the system itself.

The enhanced controls will be chosen using a risk-based approach. In other words, the cost of implementing the control should not exceed the benefit derived from the control. The Cybersecurity organization team identified 18 enhanced security controls that are being added above and beyond the moderate baseline required by National Institute of Standards and Technology.[5] This enhanced set of requirements helps protect CADE 2 system data from unauthorized access, modification, and corruption. Several of these enhanced security control features will protect information from unauthorized use or access to IRS resources and applies access restrictions to changes to the system, including upgrades and modifications that can potentially have significant effects on the security of the system. We plan to evaluate the adequacy of the security controls during a future audit of the CADE 2 system.

Independent government cost estimates were effectively used during contract negotiations

In response to a TIGTA audit recommendation in May 2005, the IRS continues to obtain independent Federal Government cost estimates to provide contracting officers with essential knowledge needed to evaluate and negotiate contract proposals.[6] During a subsequent review in July 2007, the TIGTA reported an actual cost savings that resulted when the IRS obtained an independent Federal Government cost estimate.[7]

During this current audit, we reviewed 6 of 19 CADE 2 Program contracts and determined that all of the acquisition teams prepared an independent Federal Government cost estimate for Fiscal Years 2010 and 2011. However, 2 of the 6 teams provided written documentation for a realized cost savings of approximately $11.5 million as a result of obtaining independent estimates.[8]

Systems Development Processes and Program Guidelines Were Not Always Consistent

The CADE 2 Program Management Office issued guidelines for key systems development processes and convened numerous meetings to provide oversight for the work being performed. As status meetings were convened, it became evident to CADE 2 Program Management Office officials there was a significant challenge involved in assembling diverse processes into a comprehensive set of activities that would be well understood and consistently applied across the Program and the projects. While Program guidelines specified the systems development procedures, the guidelines and the actual processes performed by the project teams were not always consistent.

The CADE 2 Program Management Office needs to improve controls to ensure that systems development guidelines and processes are consistently performed. The CADE 2 Program Management Office stated that two factors contributed significantly to the inconsistent practices identified. Specifically, the CADE 2 Program:

(1) Introduced a new business model for the development of information technology projects within the Modernization and Information Technology Services organization. In summary, the CADE 2 Program represents the first instance a Program Management Office is responsible for providing directions and oversight to multiple, ongoing information technology development projects. As a result, the IRS issued revised guidelines for most of the systems development disciplines.

(2) Created a new way to perform each systems development discipline. This essentially created a cultural change in the way the IRS traditionally developed information technology projects. One critical aspect has been to incorporate and ensure Program‑level and project-level personnel understand new emerging roles and responsibilities; therefore, CADE 2 Program Management Office leadership stated that personnel will need time to mature into these revised roles and processes.

The CADE 2 Program Management Office needs to ensure consistent practices in risk management, configuration management, test guidance, and the IMS.

Risks were not consistently identified and managed

The CADE 2 Program Management Office risk and issue management guidelines are designed to establish a continuous process to identify and mitigate risks as early as possible. Procedures require personnel at both the Program and project levels to identify and assess risks and to control these risks in the Item Tracking Reporting and Control system. Although Program-level risks were being identified and controlled in the system, project-level risks were not. For example, the CADE 2 Program Item Tracking Reporting and Control Risk Log used during the monthly risk and issue management meeting on January 6, 2011, contained 10 Program-level risks and no project-level risks.

We judgmentally selected and reviewed 11 of 13 active risks contained on the CADE 2 Program consolidated Risk Watch List dated February 8, 2011, to determine if risk analysis, risk mitigation plans, and monitoring were performed for each risk. The CADE 2 Program Management Office ensured risks were analyzed, mitigation plans were developed, and actions were monitored for all 11 risks sampled. However, undocumented risks could adversely affect the design activities, requirements development, systems performance, and delivery of the CADE 2 system.

The CADE 2 Program Management Office did not ensure project teams were following the established risk and issue management guidelines. The CADE 2 Director for Program Management and Control acknowledged inconsistencies in risk management practices, stating that there are formal risk tracking procedures at the Program level, but not at the project level. As a result, the Director explained that revisions to the risk management process would include:

Making the process more transparent. Risks identified would no longer be designated as either a Program-level risk or a project-level risk.
Implementing a common process. In the revised process, each risk would be subject to the same Program-level evaluation and review process.
Developing a consolidated list of risks. The list would capture all identified risks and be monitored at monthly risk management meetings. After Program-level evaluation, validated risks will be entered into the Item Tracking Reporting and Control system.

Management Action: The CADE 2 Program Management Office revised both the CADE 2 Risk and Issue Management Process and Risk and Issue Management Plan. In addition, the Risk Watch List was completed following our discussion with management regarding the inconsistent tracking of risks. This list captures all risks and is discussed at the monthly risk and issue management meetings. We reviewed a copy of this list after it was developed, as discussed above.

Configuration management and requirements management guidelines were not aligned with the change management process

The Configuration Management Plan and Requirements Management Plan were not aligned to establish the overall proper Configuration Control Board (CCB) authority. Configuration management involves establishing proper control over approved project products such as documentation, hardware, and software and assuring their changes are authorized, controlled, and tracked. Ensuring proper control over project products involves establishing baselines, which are an agreed-upon description of the attributes or characteristics of a product at a point in time. All baseline products are under configuration control to formally protect them from unwarranted and uncontrolled changes. These baseline products serve as the basis for future development and can be changed only when authorized by the CCB. According to the CADE 2 Configuration Management Plan, proposed changes to baseline products should be documented using a change request form, and no changes are made to the products until the changes are approved by the CCB.

The CADE 2 Program Management Office ensured that changes to baseline products were documented on change request forms. However, the Configuration Management Plan and Requirements Management Plan were not aligned with the change management process. Both guidelines stated there were three CCBs, one for the CADE 2 Program and two for the CADE 2 projects. The CADE 2 Program Management Office Director for Delivery Management stated that only one CCB existed to approve changes to CADE 2 products.

Initially, the CADE 2 Program Management Office believed change requests needed to be addressed at both the Program and project levels separately, but it realized this added an unnecessary extra level of work and decided to use only one CCB for approving changes to baseline products at both Program and project levels. However, the CADE Program Management Office did not ensure the Configuration Management Plan or Requirements Management Plan were timely updated to reflect this new process. When guidelines do not align with the actual processes, unauthorized changes could occur, which may adversely affect the system and delay implementation of the CADE 2 system in January 2012.

Management Action: The CADE 2 Program Management Office revised the Configuration Management Plan and the Requirements Management Plan to reflect that one CCB maintains sole change approval authority for all baseline products developed for the CADE 2 system.

The CADE 2 Program Test Plan was not initially developed to provide needed guidance for testing activities

The CADE 2 Program Management Office did not initially have a Program Test Plan and, as a result, experienced multiple delays in developing the Program Test Plan. The CADE 2 Program Management Office, in partnership with the Enterprise Systems Testing office, plans and executes the testing activities required to verify and validate the overall CADE 2 Transition State 1 solution. The CADE 2 Program Test Strategy requires the development of the CADE 2 Program Test Plan and states that the Plan will describe the next level of detail for testing the system. This testing will include descriptions of the common elements among the testing projects and will specify CADE 2 project test plans. The CADE 2 Program Management Office staff held discussions to consider whether the Program Test Strategy was sufficient to replace the Program Test Plan. Although the Strategy requires detailed testing guidance be provided in a test plan, it focuses on the Program level and does not provide detailed test procedures or information about testing at the project level. The lack of a documented Program Test Plan occurred partially because the Enterprise Systems Testing office had never created a Program Test Plan prior to the CADE 2 Program. Additionally, since this Program Test Plan is new, the Internal Revenue Manual had not yet been updated to include detailed instructions for developing a Program-level test plan. We advised the CADE 2 Director for Delivery Management that the Program Test Plan was a valuable control the project teams need for development of their detailed project test plans. If the CADE 2 project teams do not receive sufficient guidance on developing their test plans, the CADE 2 system may not be properly tested and the system may not work as intended when deployed into IRS operations.

Management Action: The CADE 2 Program Management Office developed the Program Test Plan, which includes due dates for delivery of each project test plan.

Recommendations

The Chief Technology Officer should:

Recommendation 1: Ensure that each project test plan is developed timely to allow sufficient time for preparation of testing materials.

Management’s Response: The IRS agreed with this recommendation. The IRS stated that test plans supporting the CADE 2 and affected systems are prepared in accordance with applicable Internal Revenue Manual guidance, which states when test plans must be prepared and delivered. Specifically, Internal Revenue Manual 2.6.1.4.2.9 states that project-level Systems Acceptability Test Plans must be delivered to all stakeholders at least 14 calendar days before application program delivery. While the Internal Revenue Manual does not provide explicit guidance for Final Integration Test Plan delivery, the same 14-day delivery requirement is maintained. For test types that are not covered by the Internal Revenue Manual or equivalent established guidance, the timing of Test Plan delivery is determined during Program or project planning.

Recommendation 2: Ensure that Internal Revenue Manual 2.16.1, Enterprise Life Cycle Guidance, includes detailed instructions on how to develop a Program-level test plan.

Management’s Response: The IRS disagreed with this recommendation. The IRS stated that the ELC is for project development and is not intended to provide for detailed instructions on developing a Program-level test plan. The CADE 2 Program Management Office will reconcile the System Validation & Verification Plan and the System Test Plan, which are generally consistent in purpose, scope, and timing as the CADE 2 Program Test Strategy and Program Test Plan, respectively, and maintain Program-level guidance regarding their development and usage. In the interim, the existing Transition State 1 Program Test Plan will be used as a template.

Office of Audit Comment: In the memorandum transmitting IRS management’s response to the draft report, the Chief Technology Officer stated that the finding that the Program Test Plan was delivered late appears to be inaccurate. The Chief Technology Officer provided a chronology of activities undertaken to prepare the Program Test Plan, and stated that a decision was made by Enterprise Systems Testing management, and agreed to by the CADE 2 Program Management Office, to defer delivery of the Program Test Plan to allow time to incorporate additional design information as it was being developed. The Chief Technology Officer ends by saying uncertainty or unfamiliarity with the content or structure of the Program Test Plan did not impede development by the Enterprise Systems Testing office and was not a factor in the decision to defer delivery. However, during the TIGTA’s audit fieldwork, the CADE 2 Program Management Office staff advised they were considering not completing the Program Test Plan, and only did so after we brought this to the attention of the CADE 2 Director for Delivery Management. Although the IRS disagreed with our recommendation, management offered an alternative corrective action. We agree this alternative approach addresses the condition.

The Integrated Master Schedule did not include all the activities required by established guidelines

The CADE 2 Program Management Office did not ensure a comprehensive master schedule was developed in accordance with established guidelines. The IMS is designed to capture and maintain tasks, milestones, activities, and dependencies over the course of a program or project lifecycle. The CADE 2 Integrated Schedule Management Process, dated May 26, 2010, defines the approach to developing and maintaining the IMS. Specifically, the Program Management Office is responsible for preparing the IMS, and project teams are responsible for preparing, maintaining, and updating supporting schedules. Currently, the IMS and supporting schedules are managed and maintained on a SharePoint web site. The CADE 2 IMS was not complete, as it did not include significant activities for several milestones. For example, the IMS did not include the CADE 2 Milestone 4b exit date or Milestone 5 deployment date of January 2012. Additionally, participants in several weekly CADE 2 Program meetings were unsure whether they had the most current version of the IMS due to missing activities/tasks.

The CADE 2 Director for Delivery Management stated that the IMS process was new for the CADE 2 Program; therefore, it would take time for stakeholders to use this new process. The critical path is designed to sequence IMS activities for timely completion; however, without a comprehensive IMS, the critical path could be inaccurate. A complete and integrated IMS is necessary at all times to ensure stakeholders are aware of significant systems development activities and to assure the January 2012 CADE 2 system scheduled deployment date is not delayed.

Recommendation

The Chief Technology Officer should:

Recommendation 3: Ensure the IMS includes all key activities associated with the development and deployment of the CADE 2 system, including the Daily Processing and Database Implementation Projects.

Management’s Response: The IRS agreed with this recommendation. The CADE 2 Program Management Office engaged all delivery partners in the rebaseline of the Milestone 4b IMS. This included conducting multiple cross-functional work sessions with stakeholders to ensure all key activities were included and to identify dependencies and alignment of dates.

Requirements Management Processes Were Not Performed in Accordance With Established Guidelines

Requirements are used to define specific business and technical functionalities that are needed from a system. The CADE 2 Requirements Management Plan is the primary source for information on activities, responsibilities, and resources used to manage, monitor, and control requirements of the CADE 2 system. The Requirements Management Plan identifies requirements traceability as a key component of requirements management. It also requires that the CADE 2 Program Management Office report monthly on requirements management measures and metrics. This reporting includes measures such as requirements traceability and metrics that identify requirements changes and the number of untraced requirements.

The Rational RequisitePro (ReqPro) automated tool is the IRS Enterprise Architecture standard for requirements management. All CADE 2 Program, project, and stakeholder personnel should use ReqPro to create, manage, and control requirements and to maintain traceability across the Program and projects. ReqPro can generate a Requirements Traceability Matrix to record and track requirements.

All CADE 2 system requirements were not sufficiently traced prior to the Milestone 3 exit. Additionally, the ELC required business rules be gathered and completed during Milestone 3; however, they were still being developed after the December 2010 Milestone 3 exit. Factors contributing to the untraced requirements include:

Business rules were not timely completed – The CADE 2 Program Management Office did not ensure business rules were gathered from all sources and input into ReqPro prior to the Milestone 3 exit. For example, when the Milestone 3 exit occurred, the business rule that determines eligibility of accounts for daily processing was not developed. This issue can affect any number of individual taxpayer accounts, consequently impinging upon CADE 2 system performance. The January 2011 ReqPro traceability matrix contained approximately 3,664 (93 percent) of 3,944 customer requirements not traced to business rule sets and 8 (9 percent) of 89 business rule sets not traced to customer requirements. Without business rules, these customer requirements may not effectively function. Additionally, new business rules developed after the Milestone exit could also require development of additional customer requirements.

The CADE 2 Director for Delivery Management stated that adherence to the ELC Program Framework resulted in business rules not being fully developed by the Milestone 3 exit. Specifically, the Framework combined both Milestones 3 and 4a into an April 2011 exit, resulting in all activities being structured around that one exit. However, due to budget issues, Milestone 3 was subsequently separated into an exit time period of December 2010. Although the CADE 2 Program Management Office ensured key activities and products were identified, business rules were not completed prior to the new Milestone 3 exit. The risk of incomplete business rules could contribute to untraced requirements, which may adversely impact systems design and testing activities.

Requirements management processes were not followed – Historically, IRS offices managed requirements internally through Excel spreadsheets and automated requirements tools. With the onset of ReqPro, the CADE 2 Program Management Office implemented processes and provided instructions to IRS stakeholders for accurate requirements input into this management tool. However, the CADE 2 Director for Delivery Management stated that these new processes, roles, and responsibilities presented a cultural change for personnel and that some stakeholders were struggling with this adjustment.

For example, after requirements were baselined in ReqPro, Cybersecurity organization and Enterprise Operations personnel continued to develop requirements outside of ReqPro. Primarily, the Cybersecurity organization extracted requirements from ReqPro into an Excel spreadsheet, where they were managed and imported back into ReqPro. Use of this method created a situation where security requirements were very unstable. As a result, 1,137 security requirement discrepancies were created. Additionally, security requirements previously approved in ReqPro prior to their extraction resurfaced as not being approved when they were imported back into the system. During our review, these discrepancies were still being addressed by the Cybersecurity organization.

The risk of incomplete, missing, or invalid requirements could adversely affect CADE 2 system design and testing activities and could delay the scheduled January 2012 system deployment. We plan to review the stability and traceability of all requirements, including security requirements, during our CADE 2 system testing audit.

Recommendations

The Chief Technology Officer should:

Recommendation 4: Ensure all requirements and business rules are identified and sufficiently traced, controlled, and managed in ReqPro prior to initiating any CADE 2 system testing processes to ensure the system functions as designed when deployed into IRS operations. This should include the Daily Processing and Database Implementation Projects.

Management’s Response: The IRS agreed with this recommendation. The IRS stated it has already stood up the CADE 2 Program ReqPro repository, which contains requirements and business rules for the Daily Processing and Database Implementation Projects, in November 2010. All requirements are tracked vertically down the hierarchy, and horizontally to other disciplines such as Configuration Management and Design, through reference requirements. All infrastructure requirements are housed in the Infrastructure Architecture & Engineering logical CADE 2 ReqPro repository, which also includes requirements for the Database Implementation and Daily Processing Projects. These requirements are traced to the CADE 2 Program ReqPro repository through cross‑project traceability.

The CADE 2 Program Management Office also drafted a Program Requirements Management Plan, which outlined the processes for managing requirements and tracing requirements. They also conducted a Program Integrated Requirements Review in December 2010 to ensure that all requirements were traced and complete in the CADE 2 Program ReqPro repository, including ensuring there was cross-project traceability between the Infrastructure Architecture & Engineering repository and the CADE 2 Program ReqPro repository. The CADE 2 Program Management Office also regularly monitors the data in the repository and presents metrics, such as requirements counts, requirements completeness, and untraced requirements, to delivery partners during weekly Integrated Requirements Team meetings. For any requirements that are not traced, an action is given to the project to establish the trace.

Recommendation 5: Implement controls to ensure that CADE 2 Program stakeholders:

Cannot remove and work on requirements outside of ReqPro.
Use ReqPro to create, input, and control requirements.

Management’s Response: The IRS agreed with this recommendation. Since implementation of the CADE 2 Requirements repository, Requirements and Demand Management requirements analysts have begun working directly in ReqPro, using ReqPro to input and control requirements. However, prior to baselining the requirements and establishing configuration control, it was essential that the IRS have a tool to assist with creating the requirements. The Requirements and Demand Management provides business-friendly tools (compatible with ReqPro) which enable creation of requirements that can be imported into ReqPro. Requirements imported into ReqPro are considered baselined and under configuration management control. All changes to requirements are performed using change requests, and the program requirements team ensures that the requirements are input and controlled within ReqPro by using the change requests tracking spreadsheet. The change request tracking spreadsheet records the actions taken to create or update a requirement based on a change request. Requirements can be exported from ReqPro for reporting purposes only and are not manipulated. Requirements analysts have also been trained on working within ReqPro, and a monthly User Group meeting is held to train users on advanced topics on the use of ReqPro.

Appendix I

Detailed Objective, Scope, and Methodology

The overall objective of this review was to determine if the CADE[9] 2 Program Management Office planned and provided oversight for Transition State 1 design activities in accordance with systems development guidelines, including applicable security provisions.

To accomplish the overall objective, we:

I. Determined whether the CADE 2 Program Management Office provided effective oversight and directions for Transition State 1 design activities of both the Program and project activities. As appropriate, we conducted interviews of IRS personnel, attended Program meetings, requested documentation of processes and procedures, and performed analysis.

A. Identified key personnel, including CADE 2 Program executives, directors, and managers, through review of the organization chart and attending meetings.

B. Obtained documentation explaining the roles and responsibilities of key CADE 2 Program personnel.

C. Determined the processes and procedures used by the CADE 2 Program Management Office to manage Program and project activities, including providing formal directions and oversight and monitoring progress, problems, and corrections.

1. Documented Program-level procedures and meeting requirements.

2. Documented formal guidance issued or communicated to project staffs by the CADE 2 Program Management Office.

D. Verified that a governance process was established and that guidance was issued for the CADE 2 Program Management Office and project staffs to follow in fulfilling governance activities and making decisions affecting the CADE 2 Program.

1. Identified the members and names of governance bodies.

2. Reviewed governance guidance that communicated the procedures and processes used by Program and project personnel to make decisions and elevate project changes, risks, and issues from the project level to the Program level.

II. Determined whether the CADE 2 Program Management Office established key Program areas and processes in accordance with systems development guidelines. As appropriate, we conducted interviews of IRS personnel, attended Program/project meetings, requested documentation of processes and procedures, and performed analysis in the following key Program areas.

A. Reviewed the IMS to determine whether it included key project activities.

B. Reviewed the Program Charter and the Program Management Plan.

C. Obtained risk and issue management documentation.

1. Reviewed the Risk and Issue Management Plan, the Risk and Issue Management Process document, and Risks Logs.

2. Judgmentally selected and reviewed 11 of 13 active risks contained on the CADE 2 consolidated Risk Watch List dated February 8, 2011, to determine if risk analysis, risk mitigation plans, and monitoring were performed for each risk. We used a judgmental sample because we were not planning to project our results.

D. Obtained the requirements management documentation.

1. Reviewed the Requirements Management Plan, the Solution Architecture, and the Program Roadmap.

2. Reviewed and analyzed the Milestone 3[10] baseline Requirements Traceability Matrix from the Rational RequistePro application.

E. Obtained and reviewed the Configuration Management Plan, the Change Request Log, and change requests.

F. Reviewed Program testing documentation to ascertain if the CADE 2 Program Management Office provided sufficient guidance for project teams to prepare detailed test plans.

III. Identified and reviewed security guidelines and requirements applicable to the CADE 2 Program Management Office, including the supporting projects. As appropriate, we conducted interviews of IRS personnel (including those in Cybersecurity organization), attended Program meetings, requested documentation of processes and procedures, and performed analysis.

A. Identified key IRS personnel and their roles and responsibilities in designing security features for the CADE 2 system by reviewing organization charts and other CADE 2 system security documents.

B. Reviewed the security and privacy guidelines applicable to the CADE 2 system.

C. Identified and reviewed security requirements in the following CADE 2 Program documentation: the Security Framework, Security Strategy, Program Roadmap, Solution Architecture, and security requirements for supporting projects in the Business Systems Report and Business Systems Requirements Report.

D. Determined whether the security controls were included in CADE 2 system documentation early enough in the systems development life cycle to be cost effective.

E. Determined whether the security categorization the IRS assigned to the CADE 2 system was documented and supported.

IV. Reviewed the CADE 2 Program contracts applicable to both the Program and the projects to determine if the IRS sufficiently protected itself throughout the contracting process. As appropriate, we conducted interviews of IRS personnel, requested documentation of the contracts and procedures, and performed analysis.

A. Obtained all existing contracts and task orders for the CADE 2 Program and the supporting projects.

B. Reviewed a judgmental sample of 6 of 19 contracts (issued in Fiscal Years 2010 and 2011) and determined whether the IRS obtained independent Government cost estimates to ensure the contract costs were economically derived. We used a judgmental sample because we were not planning to project our results.

C. Based on evidence received from the Office of Procurement, developed a cost savings outcome measure related to the use of independent Government cost estimates.

Internal controls methodology

Internal controls relate to management’s plans, methods, and procedures used to meet their mission, goals, and objectives. Internal controls include the processes and procedures for planning, organizing, directing, and controlling program operations. They include the systems for measuring, reporting, and monitoring program performance. We determined the following internal controls were relevant to our audit objective: ELC and related IRS guidelines and the processes followed in the development of information technology projects. We evaluated these controls by conducting interviews and meetings with management and staff, attending meetings of the CADE 2 Program and project teams, and reviewing Program documentation such as the Program Charter, various program plans, and other documents that provided evidence of whether ELC systems development processes were followed.

Appendix II

Major Contributors to This Report

Alan R. Duncan, Assistant Inspector General for Audit (Security and Information Technology Services)

Diana M. Tengesdal, Acting Director

Kimberly R. Parmley, Audit Manager

Wallace C. Sims, Lead Auditor

Suzanne M. Westcott, Senior Auditor

Esther M. Wilson, Senior Auditor

David F. Allen, Program Analyst

Appendix III

Report Distribution List

Commissioner C

Office of the Commissioner – Attn: Chief of Staff C

Deputy Commissioner for Operations Support OS

Deputy Commissioner for Services and Enforcement SE

Chief, Agency-Wide Shared Services OS:A

Commissioner, Wage and Investment Division SE:W

Deputy Chief Information Officer for Strategy/Modernization OS:CTO

Associate Chief Information Officer, Modernization – Program Management Office OS:CTO:MP

Chief Counsel CC

National Taxpayer Advocate TA

Director, Procurement OS:A:P

Director, Risk Management Division OS:CTO:SP:RM

Director, Office of Legislative Affairs CL:LA
Director, Office of Program Evaluation and Risk Analysis RAS:O

Office of Internal Control OS:CFO:CPIC:IC

Audit Liaisons:

Commissioner, Wage and Investment Division SE:W:S:PRA:PEI

Director, Procurement OS:A:P

Director, Program Oversight OS:CTO:SP:RM

Appendix IV

Outcome Measure

This appendix presents detailed information on the measurable impact a prior recommendation has had on tax administration. This benefit will be incorporated into our Semiannual Report to Congress.

Type and Value of Outcome Measure:

· Cost Savings – Funds Put to Better Use – Actual: $11,537,356 (see page 4).

Methodology Used to Measure the Reported Benefit:

The Federal Acquisition Regulation requires the initial negotiation position to be based on the results of the contracting officer’s analysis of the offeror’s proposal, taking into consideration technical analysis, fact-finding results, and independent Federal Government cost estimates.[11] In a May 2005 audit report,[12] the TIGTA determined that the IRS may not have been obtaining requested services at a fair and reasonable cost because independent cost estimates were not required by modernization processes. It was recommended that the IRS promote consistent application of best practices by obtaining independent cost estimates. Additionally, in a July 2007 audit report,[13] the TIGTA reported an actual cost savings outcome measure resulting from the IRS obtaining an independent Government cost estimate.

As part of our current review, the IRS Office of Procurement provided written documentation that it realized a cost savings of $11,537,356 from obtaining independent estimates from 2 of its contracts. The cost savings originated from reductions in the scope for base and option years, labor hours, and labor rates and a revision to the skill mix.[14]

Appendix V

Enterprise Life Cycle Overview

The ELC is the IRS’s standard approach to business change and information systems initiatives. It is a collection of program and project management best practices designed to manage business change in a successful and repeatable manner. The ELC addresses large and small projects developed internally and by contractors.

The ELC includes such requirements as:

· Development of and conformance to enterprise architecture.

· Improving business processes prior to automation.

· Use of prototyping and commercial software, where possible.

· Obtaining early benefit by implementing solutions in multiple releases.

· Financial justification, budgeting, and reporting of project status.

In addition, the ELC improves the IRS’s ability to manage changes to the enterprise; estimate the cost of changes; and engineer, develop, and maintain systems effectively. Figure 1 provides an overview of the phases and milestones within the ELC. A phase is a broad segment of work encompassing activities of similar scope, nature, and detail and providing a natural breakpoint in the life cycle. Each phase begins with a kickoff meeting and ends with an executive management decision point (milestone) at which IRS executives make “go/no-go” decisions for continuation of a project. Project funding decisions are often associated with milestones.

Figure 1: Enterprise Life Cycle Phases and Milestones

Phase	General Nature of Work	Milestone
Vision and Strategy/ Enterprise Architecture Phase	High-level direction setting. This is the only phase for enterprise planning projects.	0
Project Initiation Phase	Startup of development projects.	1
Domain Architecture Phase	Specification of the operating concept, requirements, and structure of the solution.	2
Preliminary Design Phase	Preliminary design of all solution components.	3
Detailed Design Phase	Detailed design of solution components.	4A
Systems Development Phase	Coding, integration, testing, and certification of solutions.	4B
System Deployment Phase	Expanding availability of the solution to all target users. This is usually the last phase for development projects.	5
Operations and Maintenance Phase	Ongoing management of operational systems.	System Retirement

Source: The Enterprise Life Cycle Guide.

Appendix VI

Customer Account Data Engine 2 Transition States

Figures 1 through 4 present conceptual models of the As Is, Transition States 1 and 2, and Target State processing flows for individual income tax accounts.

Figure 1: As Is Processing

Figure 1 was removed due to its size. To see Figure 1, please go to the Adobe PDF version of the report on the TIGTA Public Web Page. .[15]

Figure 2: Transition State 1 Processing Plan

Figure 2 was removed due to its size. To see Figure 2, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

Figure 3: Transition State 2 Processing Plan

Figure 3 was removed due to its size. To see Figure 3, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

Figure 4: Target State Processing Plan

Figure 4 was removed due to its size. To see Figure 4, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

Appendix VII

Transition State 1 Integration Reviews

Integration Review	Outcomes
Integrated Management Planning Review	The Integrated Management Planning Review validates that relationships and integration expectations between the Program and its projects are appropriately defined and well understood.
Integrated Requirements Review	The Integrated Requirements Review validates that the Program‑level requirements allocated to projects are in alignment with the Program solution; that Program-level requirements have been appropriately fulfilled through decomposition to project‑level requirements; and that all requirements dependencies are identified, supported, and fulfilled.
Integrated Solution Planning Review	The Integrated Solution Planning Review validates that Program strategies for solution design are aligned for the transition state solution.
Integrated Logical Design Review	The Integrated Logical Design Review validates that the project‑level designs support the solution’s logical implementation as defined in the Program Roadmap and that the projects collectively will deliver an integrated and cohesive solution.
Integrated Physical Design Review	The Integrated Physical Design Review validates that the project‑level designs support the solution’s physical implementation as defined in the Program Roadmap and that the projects collectively will deliver an integrated and cohesive solution.

Source: IRS Customer Account Data Engine 2, 2^nd Quarter Briefing to the TIGTA, dated July 14, 2010.

Integration Review	Outcomes
Integrated Test Planning Review	The Integrated Test Planning Review validates that Program and project plans for testing the solution components individually and the integrated Program solutions collectively are in alignment and comprehensive.
Integrated Test Readiness Review	The Integrated Test Readiness Review validates that solution components have been accurately and comprehensively tested at the Unit and Developer level and that the Program is ready to begin testing of the integrated Transition State solution.
Integrated Organizational Readiness Review	The Integrated Organizational Readiness Review validates that the Program understands the impact the solution has on the business and validates the organization’s readiness to adopt the new solution.
Integrated Deployment Readiness Review	The Integration Deployment Readiness Review validates that the solution components; production environment; and plans for deployment, back-out, and operations are assessed against defined readiness criteria. The Program will make a “Go/No-Go” decision based on the results of the Integrated Deployment Readiness Review.

Source: IRS Customer Account Data Engine 2, 2^nd Quarter Briefing to the TIGTA, dated July 14, 2010.

Appendix VIII

The Customer Account Data Engine 2
High and Enhanced Requirements

High and Enhanced Security Control Number	NIST 800-53 Security Control Name	Requirement Description	Requirement Purpose
1	Account Management	The information system shall monitor for atypical (abnormal) use of systems accounts.	Monitoring changes outside approved maintenance windows.
2	Account Management	The information system shall establish a role-based user account management process.	Simplifies account management to allow effective enforcement of least privilege principles (i.e., the minimum privileges needed to perform job functions).
3	Account Management	The information system shall monitor and track privilege role(s) assignments.	Existing Internal Revenue Manual requirements must be in place for all IRS systems.
4	Access Enforcement	The information system shall store encrypted backups in a secure location.	Selected to document control enhancement that is already in place.
5	Previous Logon (Access) Notification	The information system shall notify the user, upon successful logon (access), of the date and time of the last logon (access).	Allows detection of unauthorized account access.
6	Previous Logon (Access) Notification	The information system shall notify the user, upon successful logon/access, of the number of unsuccessful logon/access attempts since the last successful logon/access.	Allows detection of unauthorized account access.
7	Response to Audit Processing Failures	The information system shall provide a warning when allocated audit record storage volume reaches a defined percentage of maximum audit record storage capacity.	Intended to prevent loss of audit capabilities due to storage capacity being exceeded either unintentionally or maliciously.
8	Response to Audit Processing Failures	The information system shall provide a real-time alert for intrusions and potential intrusions to IRS networks by unauthorized individuals.	Real-time alerts of unauthorized access are necessary to minimize damage from an attacker or malicious user.
9	Response to Audit Processing Failures	The information system shall provide a real-time alert for unauthorized use or access to IRS resources.	Real-time alerts of unauthorized access are necessary to minimize damage from an attacker or malicious user.
10	Protection of Account Information	The information system shall back up audit records at a defined frequency onto a different system or media than the system being audited.	Intended to reduce the risk of audit compromise.
11	Protection of Account Information	The information system shall authorize access to management of audit functionality to only a limited subset of privileged users.	Intended to prohibit modification of audit records by privileged users.
12	Protection of Account Information	The information system shall protect the audit records of nonlocal accesses to privileged accounts and the execution of privileged functions.	Intended to prohibit modification of audit records by privileged users.
13	Access Restrictions for Change	The information system shall limit information system developer/integrator privileges to change hardware, software, and firmware components and system information directly within the production environment.	Intended to prevent unauthorized changes to production systems.
14	Access Restrictions for Change	The information system shall limit privileges to change software resident within software libraries (including privileged programs).	Existing Internal Revenue Manual requirements must be in place for all IRS systems.
15	Media Storage	The information system shall employ cryptographic mechanisms to protect information in storage.	Existing Internal Revenue Manual requirements must be in place for all IRS systems.
16	Use of Cryptography	The information system shall use Federal Information Processing Standards validated cryptography when cryptography is used to protect information.	Existing Internal Revenue Manual requirements must be in place for all IRS systems.
17	Session Authenticity	The information system shall provide a readily observable logout capability whenever authentication is used to gain access to web pages.	Intended to protect administrative web interfaces and prevent unauthorized access to the application/data.
18	Boundary Protection	The information system shall check incoming communications to ensure that the communications are coming from an authorized source and routed to an authorized destination.	Intended to prevent the introduction of malicious traffic or unauthorized access by an external attacker.

Source: CADE 2 Program Transition State 1 National Institute of Standards and Technology 800-53
High and Enhanced Control Requirements Discussion UPDATE, dated November 24, 2010.

Appendix IX

Glossary of Terms

Term	Definition
Authentication	The process of identifying an individual, usually based on a username and password.
Business Rule	A statement that defines or constrains some aspect of the business (see Business Rule Sets).
Business Rule Sets	A group of business rules related to a common topic or business decision.
Configuration Control Board	Serves as the change approval authority for all baseline products developed at the Program and project levels.
Corporate Files On‑Line	A system that provides online transactional access to IMF and Business Master File data, Information Return Program data, and various other related data collections. These files are accessed via IRS-developed Customer Information Control System command codes.
Critical Path	A sequence of activities that results in the completion of a project in the shortest period of time.
Cryptography	The conversion of data into a secret code for transmission over a public network.
Current Processing Environment	The IRS’s existing entire information technology environment including business applications, data stores, data interfaces and processing flows, infrastructure, and information technology services, as well as involved organizations, locations, processes, policies, and people.
Customer Account Data Engine	A major component of the IRS’s Modernization Program. The system consists of current and planned databases and related applications that work with the IRS Master File system (see Master File).
Daily Processing Project (CADE 2)	A project under the CADE 2 Program that, when completed, will change weekly individual taxpayer account processing to daily processing.
Database Implementation Project (CADE 2)	A project under the CADE 2 Program intended to implement the newest version of the relational database.
Delivery Management Office	Provides Program management oversight and direction to the individual project offices. It coordinates and directs integration activities across supplier organizations to ensure projects are delivered on schedule and within budget. The office ensures that all component projects and affected applications or systems operate inter-dependently at deployment through assuring interfaces and impacts are clearly identified, engineered, and implemented.
Enterprise Architecture	A unifying overall design or structure for an enterprise that includes business and organizational aspects of the enterprise as well as technology aspects. Enterprise Architecture divides the enterprise into its component parts and relationships and provides the principles, constraints, and standards to help align business area development efforts in a common direction. An Enterprise Architecture ensures that subordinate architectures and business system components developed within particular business areas and multiple projects fit together into a consistent, integrated whole.
Enterprise Life Cycle	A structured business systems development method that requires the preparation of specific work products during different phases of the development process.
Executive Steering Committee	Committee with oversight responsibilities for investments, including validating major investment business requirements and ensuring that enabling technologies are defined, developed, and implemented.
Financial Management System Requirements	The Federal Financial Management Improvement Act of 1996 (FFMIA)[16] established financial management systems requirements intended to advance Federal financial management by ensuring that Federal management systems can and do provide reliable, consistent disclosure of financial data. Agencies are required to determine whether their financial management systems comply with the law. If the financial systems do not comply (i.e., they contain financial material weaknesses), the agency is required to develop a remediation plan that describes the resources, remedies, and intermediate target dates for achieving compliance.
Financial Material Weaknesses	If an agency’s financial management systems do not comply with the Federal Financial Management Improvement Act of 1996, the systems contain financial material weaknesses. The agency must develop a remediation plan that describes the resources, remedies, and intermediate target dates for achieving compliance.
Firmware	The fixed, usually rather small, programs that internally control various electronic devices.
Framework	A structure that facilitates understanding of a complex topic by breaking the topic into multiple pieces or features, classifying the features, illustrating relationships between the features, and organizing them in a manner that facilitates visualization and practical usage.
Governance Board	Exists to ensure that the Program goals are achieved and that the Program and component projects are delivering within their defined scope, schedule, and budget. Additionally, the Governance Board approves risk response plans and milestone exits and resolves escalated issues.
Individual Master File	The IRS database that maintains transactions or records of individual tax accounts.
Infrastructure	The fundamental structure of a system or organization. The basic, fundamental architecture of any system (electronic, mechanical, social, political) determines how it functions and how flexible it is to meet future requirements.
Integrated Data Retrieval System	The IRS computer system capable of retrieving or updating stored information; it works in conjunction with a taxpayer’s account records.
Integrated Production Model	Intended to be a data store to meet IRS needs for data analytics and long‑term reporting and as a source for other types of analytic data that supplement the transactional core data store.
Item Tracking Reporting and Control System	An information system used to track and report on issues, risks, and action items in the modernization effort.
Master File	The IRS database that stores various types of taxpayer account information. This database includes individual, business, and employee plans and exempt organizations data.
Milestone	Scheduled time period for providing a “go/no-go” decision point in a program or project (can be associated with funding approval to proceed).
National Institute of Standards and Technology	A nonregulatory Federal agency, within the Department of Commerce, responsible for developing standards and guidelines, including minimum requirements, for providing adequate information security for all Federal Government agency operations and assets.
Rational RequisitePro	An application used for requirements management. The IRS has established ReqPro as its Enterprise Architecture standard for requirements management. It is used to capture detailed requirement data such as the requirement text and any supporting attributes to organize or clarify the requirement. The application also has the capability to create and maintain full requirements traceability within a single project or across multiple projects.
Relational Database	A collection of data items organized as a set of formally described tables from which data can be accessed or reassembled in many different ways without having to reorganize the database tables.
Requirement	A formalization of a need and statement of a capability or condition that a system must have or meet to satisfy a contract, standard, or specification.
SharePoint	A web-based repository that the IRS uses to store and control organizational products and documentation.
Stakeholders	An individual or organization that is materially affected by the outcome of the system. Key stakeholders represent both business and technical functions that fully participate in the architecture development effort to ensure that directional guidance is both accurate and sufficient. These stakeholders are empowered to make project and architectural decisions. Examples of project stakeholders include the customer, the user group, the project manager, the development team, and the testers.
Traceability	Describes the life of a requirement from the initial source through its development and actual deployment into operations.

Appendix X

Management’s Response to the Draft Report

DEPARTMENT OF THE TREASURY

INTERNAL REVENUE SERVICE

WASHINGTON, D.C. 20224

CHIEF TECHNOLOGY OFFICER

SEPTEMBER 13, 2011

MEMORANDUM FOR DEPUTY INSPECTOR GENERAL FOR AUDIT

FROM: Terence V. Milholland /s/ Terence V. Milholland

Chief Technology Officer

SUBJECT: Draft Audit Report - The Customer Account Data Engine 2 Program Management Office Implemented Systems Development Guidelines; However, Process Improvements Are Needed to Address Inconsistencies (201020025)

(e-trak 2011-24467)

I would like to thank you for agreeing to conduct this review at my request. I appreciate the opportunity to review your draft audit report and the approach that your team employed during the audit. As issues were identified by the team during the course of the audit, they were conveyed to the CADE 2 Program Management Office (PMO) in order for corrective actions to be implemented more timely. I believe that this contributed to significant improvements to the overall effectiveness of the Program.

I appreciate TIGTA's recognition of our accomplishments; and found the report very balanced and comprehensive. However, there is one area of concern and some observations that I would like to share with you. My concern is that TIGTA's finding that the CADE 2 Program Test Plan was delivered late appears to be inaccurate. The Program Test Plan was originally scheduled for review and approval by the CADE 2 Governance Board by December 8, 2010, to align with a Program-level Integrated Physical Design Review which was scheduled for December 2010. The Program Review schedules were subsequently adjusted (Part 2 of the Program Integrated Requirements Review was held on December 3, 2010 and the Physical Design walkthrough was held on April 7-8, 2011) and a decision was made by Enterprise Systems Testing (EST) management, and agreed by the CADE 2 PMO, to defer delivery of the Program Test Plan to allow time to incorporate additional design information as it was being developed. The Program Test Plan was completed and delivered to the CADE 2 PMO on February 1, 2011, several months in advance of the first substantive test covered under the plan. While the Program Test Plan was a new testing deliverable, uncertainty or unfamiliarity with its content or structure did not impede development by EST and was not a factor in the decision to defer delivery.

Although I agree with TIGTA's findings with regard to maintaining our Integrated Master Schedule, I would like to share some recent efforts. As the Program Integrated Master Schedule function continues to mature, so does the understanding, engagement, and participation of the stakeholders. In April, 2011, the IMS function's restructuring and updating of the IMS Sharepoint site to ensure access to key information, including the most current IMS, was intuitive and efficient. The function has also documented and implemented a process for managing updates to the IMS and follows the Program Change Management processes for schedule changes. In a separate effort, the PMO developed a high-level, comprehensive timeline of key activities associated with the development and deployment of CADE 2 through Milestone 5 (referred to as "Executive View"). All activities in this "Executive View" (a snapshot as of August 2011) were cross-checked with the IMS to ensure alignment. The reconciliation process also validated that all key activities are in the IMS.

Finally, I want to point out that the Program Management Office was set up as a management effort to oversee multiple projects. The Enterprise Life Cycle is a tool to track individual projects, not a program. I've asked our team to discuss the methodology further with TIGTA, at your convenience.

We are committed to continuously improving our information technology systems and processes and value the continued support and guidance your teams provide. If you have any questions, please contact me at (202) 622-6800 or Karen Mayr at (202) 283-0015.

Attachment

RECOMMENDATION #1: The Chief Technology Officer should ensure that each project test plan is developed timely to allow sufficient time for preparation of testing materials.

CORRECTIVE ACTION #1: The IRS agrees with this recommendation. Project test plans supporting CADE 2 and affected systems are prepared in accordance with applicable IRM guidance, which states when test plans must be prepared and delivered. IRM 2.6.1.4.2.9 states that project-level Systems Acceptability Test (SAT) Plans must be delivered to all stakeholders at least 14 calendar days before application program delivery. While the IRM does not provide explicit guidance for Final Integration Test (FIT) Plan delivery, the same fourteen-day delivery requirement is maintained. The timing of Test Plan delivery for test types that are not covered by IRM or equivalent established guidance is determined during Program or Project planning.

IMPLEMENTATION DATE: Completed February 1, 2011.

RECOMMENDATION #2: The Chief Technology Officer should ensure that Internal Revenue Manual 2.16.1, Enterprise Life Cycle Guidance, includes detailed instructions on how to develop a Program-level test plan.

CORRECTIVE ACTION #2: The IRS disagrees with this recommendation. The ELC is a life cycle for project development and is not intended to provide detailed instructions on how to develop a Program-level test plan.

The CADE program office will look to reconcile two artifacts - the System Validation & Verification Plan (SWP) and System Test Plan (STP), which are generally consistent in purpose, scope and timing as the CADE 2 Program Test Strategy and Program Test Plan, respectively and maintain program level guidance regarding their development and usage. In the interim, the existing TS1 Program Test Plan will be utilized as a template.

IMPLEMENTATION DATE: February 1, 2013

RECOMMENDATION #3: The Chief Technology Officer should ensure the IMS includes all key activities associated with the development and deployment of the CADE 2 system, including the Daily Processing and Database Implementation Projects.

CORRECTIVE ACTION #3: The IRS agrees with this recommendation. The PMO engaged all delivery partners in the re-baseline of the MS4b IMS. This included conducting multiple cross-functional work sessions with stakeholders to ensure all key activities were included, and to identify dependencies and align on dates.

IMPLEMENTATION DATE: Completed and provided to TIGTA August 15, 2011.

RECOMMENDATION #4 The Chief Technology Officer should ensure all requirements and business rules are identified and sufficiently traced, controlled, and managed in ReqPro prior to initiating any CADE 2 system testing processes to ensure the system functions as designed when deployed into IRS operations. This should include the Daily Processing and Database Implementation.

CORRECTIVE ACTION #4: We agree with this recommendation and have already stood up a RequisitePro (ReqPro) Repository in November 2010 which contains requirements and business rules for the DP and DI projects. This repository is referred to as the CADE 2 Program ReqPro repository. All requirements are traced vertically down the hierarchy. Requirements are also traced horizontally to other disciplines such as CM, Design, etc through reference requirements. The IA&E logical CADE 2ReqPro repository houses all infrastructure requirements, which includes requirements for DI and DP. These requirements are traced to the CADE 2 Program ReqPro repository through cross-project traceability.

The PMO also drafted a Program Requirements Management Plan (RMP) which outlined the processes of managing requirements and tracing requirements. There was a Program Integrated Requirements Review (PIRR) conducted in December 2010 to ensure that all requirements were traced and complete in the CADE 2 Program repository. This also ensured that the cross-project traceability existed between the IA&E Repository and the CADE 2 Program repository.

The Program efficiently and regularly monitors the data in the repository. The Program conducts an Integrated Requirements Team (IRT) meeting weekly, where we present metrics to all the delivery partners. These metrics include requirements counts, requirements volatility, requirements completeness, and untraced requirements. If there are requirements that are not traced, an action is given to the project to establish the trace.

IMPLEMENTATION DATE: Completed December 31, 2010.

RECOMMENDATION #5 The Chief Technology Officer should implement controls to ensure that CADE 2 Program stakeholders:

a. Cannot remove and work on requirements outside of ReqPro.

b. Use ReqPro to create, input, and control requirements.

CORRECTIVE ACTION #5:

We agree with the spirit and intent of this recommendation and since the implementation of the CADE 2 Requirements repository, RADM requirements analysts have begun working directly in ReqPro. Using ReqPro to input and control requirements are part of our approach to managing requirements. However, prior to baselining the requirements, it was essential that IRS have a tool to create requirements prior to baselining them for configuration control. RADM provides business-friendly tools (compatible with ReqPro) which enables creation of requirements which can be imported into ReqPro. Requirements imported into ReqPro will be considered baselined. Since all requirements are baselined and are under CM control, all changes to requirements go through CRs. The program requirements team ensures that the requirements are input and controlled within ReqPro by using the CR Tracking spreadsheet. The CR tracking spreadsheet keeps track of actions to create or update a requirement based on a CR. We ensure that these changes are made in the ReqPro repository.

The capability to export requirements does exist; these reports are extracted for reporting purposes only and are not manipulated. We have also provided training to requirements analysts so they are comfortable working within ReqPro. We have a monthly User Group meeting which trains users on advanced topics on the use of ReqPro.

IMPLEMENTATION DATE: Completed December 31, 2010.

[2] Prototype Process Improvements Will Benefit Efforts to Modernize Taxpayer Account Administration (Reference Number 2011-20-001, dated November 24, 2010).

[3] The Customer Account Data Engine 2 Is Making Progress Toward Achieving Daily Processing, but Improvements Are Warranted to Ensure Full Functionality (Reference Number 2011-20-109, dated September 28, 2011), and The Customer Account Data Engine 2 Database Implementation Project Made Progress in Design Activities, but Improvements Are Needed (Reference Number 2011-20-110, dated September 20, 2011).

[4] See Appendix V for an overview of the ELC.

[5] See Appendix VIII for a complete list of the 18 Customer Account Data Engine 2 High and Enhanced Requirements.

[6] While Many Improvements Have Been Made, Continued Focus Is Needed to Improve Contract Negotiations and Fully Realize the Potential of Performance-Based Contracting (Reference Number 2005-20-083, dated May 26, 2005).

[7] While Improvements Continue in Contract Negotiation Methods and Management Practices, Inconsistencies Need to Be Addressed (Reference Number 2007-20-123, dated July 27, 2007).

[10] See Appendix V for an overview of the ELC.

[11] 48 C.F.R. § 15.406-1 (a) (Amended February 2009).

[12] While Many Improvements Have Been Made, Continued Focus Is Needed to Improve Contract Negotiations and

Fully Realize the Potential of Performance-Based Contracting (Reference Number 2005-20-083, dated May 26, 2005).

[13] While Improvements Continue in Contract Negotiation Methods and Management Practices, Inconsistencies Need to Be Addressed (Reference Number 2007-20-123, dated July 27, 2007).

[14] Information obtained from the IRS Office of Procurement. The TIGTA did not verify the accuracy of this information.