TREASURY INSPECTOR GENERAL FOR TAX ADMINISTRATION

 

 

Focusing Management Efforts on Long-Term Project Needs Will Help Development of the Customer Account Data Engine Project

 

 

 

June 2006

 

Reference Number: 2006-20-076

 

 

This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.

 

Phone Number   | 202-927-7037

Email Address   |  Bonnie.Heald@tigta.treas.gov

Web Site           | http://www.tigta.gov

 

June 28, 2006

 

 

MEMORANDUM FOR CHIEF INFORMATION OFFICER

 

FROM:                            Michael R. Phillips /s/ Michael R. Phillips

                                         Deputy Inspector General for Audit

 

SUBJECT:                    Final Audit Report – Focusing Management Efforts on Long-Term Project Needs Will Help Development of the Customer Account Data Engine Project (Audit # 200520012)

 

This report presents the results of our review of the Customer Account Data Engine (CADE) project management and deployment activities.  The overall objective of this review was to determine whether the CADE project management and development activities were effective in assuring CADE releases[1] included all planned capabilities and were implemented within cost and schedule estimates.  This review was part of the Treasury Inspector General for Tax Administration’s Fiscal Year 2005 Information Systems Programs audit plan for reviews of the Internal Revenue Service’s (IRS) modernization efforts.

Synopsis

The CADE project is the foundation for managing taxpayer accounts in the IRS modernization plan.  It consists of databases and related applications that will replace the IRS’ existing Master File processing systems.  The Master Files are the IRS’ central and official repository of taxpayer information.

While the current release of the CADE is performing well, significant challenges await the next release (CADE Release 2), which plans to add several new tax forms and schedules.

In August 2004, the IRS delivered CADE Release 1.1, which successfully processed refund and even-balance Forms 1040EZ[2] for single taxpayers with no pending tax issues.  CADE Release 1.2 started processing tax returns in January 2005.  In January 2006, the IRS and the PRIME contractor[3] successfully delivered the newest release of the CADE (Release 1.3.2) in time for the 2006 Filing Season.[4]  Release 1.3.2 targeted a larger taxpayer population by including U.S. Individual Income Tax Returns (Forms 1040 and 1040A) with no schedules, improving address change capabilities, and including last-minute 2006 Filing Season tax law changes.  As of February 24, 2006, the CADE had processed over 2.7 million tax returns and generated over $1.4 billion in tax refunds, compared to the 1.4 million tax returns processed that generated $427 million tax refunds in of all of 2005.  Although the CADE has experienced cost and schedule overruns in the past, the 2006 Filing Season goals were substantially met.

While the above results are commendable, the IRS has had to reduce the number of tax returns it planned to process during the 2006 Filing Season.  The IRS and the PRIME contractor agreed to defer eight Release 1.3.2 requirements to help ensure the CADE would process tax returns in time for the 2006 Filing Season. This occurred because the PRIME contractor did not effectively and efficiently use computer language (coding practices).  Also, the PRIME contractor’s unit testing[5] did not adequately identify programming defects.

The CADE’s design for managing taxpayer accounts is not fully developed to provide a foundation for the IRS’ current and planned modernized systems.  In April 2005, the Business Systems Modernization Office (BSMO) initiated the CADE Data Architecture and Analysis Study to enable the project team to adopt a flexible release strategy by establishing a long-term plan to organize and manage CADE project data.  Also, the Modernization and Information Technology Services organization is forming the Modernization Vision and Strategy to establish project development priorities with the IRS business operations.  The Modernization Vision and Strategy will provide a 5-year plan for developing modernization projects and processes.  Until these efforts are completed, successful deployment of a long-term CADE release strategy cannot move forward, and the future releases may not be deployed when planned.

Adequate procedures were not in place to ensure all CADE release requirements are timely controlled. As of February 23, 2006, an interim process to control CADE Release 2 requirements was in development.  The BSMO will use this process until a vendor able to develop a repository meeting the IRS’ needs is identified.

The BSMO implemented contracting guidance on April 30, 2004, entitled “Enabling Fixed-Price Contracting for Business Systems Modernization Task Orders,” requiring fixed-price contracts and task orders for Business Systems Modernization acquisition projects at the appropriate life cycle development phase, unless the Federal Government’s interest is best served by other contract types.  The IRS is not using fixed-price contracts for CADE Release 1.3 development activities or for Release 2 design work performed by the PRIME contractor.

Recommendations

The Chief Information Officer (CIO) should provide direction to help ensure progress for the development of current and future CADE releases.  The CIO should ensure current and future CADE contracts provide an assurance to the IRS that the PRIME contractor performed adequate testing.  The CIO should also ensure the CADE Data Architecture and Analysis Study and the Modernization and Information Technology Services organization’s new Modernization Vision and Strategy are expeditiously completed, so the CADE project can implement realistic requirements and development plans.  The CIO should timely implement an interim process to control CADE Release 2 requirements, which are currently in development.  Additionally, the CIO should ensure the BSMO and the IRS’ contracting officers work with the PRIME contractor to adequately define project requirements enabling implementation of fixed-price contracts.

Response

IRS management agreed with our recommendations.  To implement the corrective actions, the CIO is requiring the PRIME contractor to notify the IRS of the completion of unit testing and will incorporate this requirement into the IRS’ application testing guidance.  The Modernization and Information Technology Services organization will use CADE data models and the high‑level data warehousing strategy to develop a data access strategy and transition plan for future CADE release designs.  Further, an interim repository has been developed within the CADE to track and control requirements until the implementation of the planned Enterprise Services repository.  Finally, the Applications Development organization, the Enterprise Services organization, and the Office of Procurement will work to position the IRS with the ability to award fixed-price contracts.  However, after full review, negotiations, and consideration, the IRS made a business decision to continue with the cost-plus-award-fee contract rather than a fixed-price contract for CADE Release 2.  Management’s complete response to the draft report is included as Appendix IX.

Copies of this report are also being sent to the IRS managers affected by the report recommendations.  Please contact me at (202) 622-6510 if you have questions or Margaret E. Begg, Assistant Inspector General for Audit (Information Systems Programs), at (202) 622-8510.

 

 

Table of Contents

 

Background

Results of Review

The Customer Account Data Engine Project Is Making Progress

The PRIME Contractor’s Inadequate Testing Resulted in Reduced Capabilities for Customer Account Data Engine Release 1.3

Recommendation 1:

Successful Development of the Customer Account Data Engine Is Dependent Upon Its Ability to Serve As an Efficient and Effective Foundation for Modernized Systems and Applications

Recommendation 2:

Controls to Manage Customer Account Data Engine Requirements Do Not Allow Ready Access for Project Team Use

Recommendation 3:

Defining Customer Account Data Engine Project Release Requirements Will Help Enable the Use of Fixed-Price Contracts

Recommendation 4:

Appendices

Appendix I – Detailed Objective, Scope, and Methodology

Appendix II – Major Contributors to This Report

Appendix III – Report Distribution List

Appendix IV – Customer Account Data Engine Release Schedules

Appendix V – Customer Account Data Engine Project Costs

Appendix VI – Business Systems Modernization Challenge Issues Relating to the Customer Account Data Engine

Appendix VII – Customer Account Data Engine Corrective Actions to Prior Recommendations

Appendix VIII – Enterprise Life Cycle Overview

Appendix IX– Management’s Response to the Draft Report

 

 

Background

 

The Customer Account Data Engine (CADE) project is the foundation for managing taxpayer accounts in the Internal Revenue Service (IRS) modernization plan.  It consists of databases and related applications that will replace the IRS’ existing Master File processing systems.  The Master Files are the IRS’ central and official repository of taxpayer information.

The IRS initiated the CADE project in September 1999.  CADE Release 1[6] was initially planned for delivery in January 2002.  The IRS and the PRIME contractor[7] postponed the planned delivery date of Release 1 several times.  Setbacks occurred for several reasons, including IRS/PRIME contractor role confusion, noncompliance with management processes, and the absence of consistent, well-understood requirements.  Appendix IV, Table 1, provides a timeline of the adjustments made to the content and planned delivery dates for the CADE release schedule.

In August 2004, the IRS delivered CADE Release 1.1, which successfully processed refund and even-balance Forms 1040EZ[8] for single taxpayers with no pending tax issues.  CADE Release 1.2 started processing tax returns in January 2005.  From January 1 through December 31, 2005, the CADE processed over 1.4 million tax returns and issued 927,044 paper and 478,120 electronic tax return refunds totaling over $427 million.

The Joint Engineering Team was tasked to confirm the CADE architecture and design were correctly implemented and tested throughout the life cycle.

The IRS and the PRIME contractor created the CADE Joint Engineering Team consisting of business architects and engineers.  In the fall of 2004, the Joint Engineering Team recommended the formation of the CADE Data Architecture and Analysis Study to establish a data architecture foundation that would enable the project to adopt a flexible release strategy for current and future CADE releases.  The study began in April 2005 and is on track for completion in June 2006.

Each year the IRS presents to Congress an expenditure plan containing the best estimates regarding the current scope, schedule, and cost of the Business Systems Modernization (BSM) program.  It introduces key activities for the upcoming fiscal year and incorporates the most current information concerning the status of the modernization projects.  The expenditure plan details the successes from prior years and discusses the key activities and steps the Business Systems Modernization Office (BSMO) is taking to strengthen and improve the efficiencies and effectiveness of its program.

After reviewing the IRS’ scope, schedule, and cost estimates contained in the latest BSM 2005 and 2006 Expenditure Plans, Congress authorized $54 million in 2005 and $60 million in 2006 for the CADE.  Additionally, the IRS requested $85 million in 2007 for the CADE, but this amount has been reduced to about $58 million.  Through October 27, 2005, the IRS had obligated $134,168,206 for the design, development, and implementation of the CADE, with an estimated $252.1 million planned to be spent through 2007.  Appendix V presents an analysis of the CADE project release costs.

This review is the fifth in a series of reviews related to the CADE project activities and was performed at the BSMO facilities in New Carrollton, Maryland, during the period August 2005 through February 2006.  The audit was conducted in accordance with Government Auditing Standards.  Detailed information on our audit objective, scope, and methodology is presented in Appendix I.  Major contributors to the report are listed in Appendix II.  Appendix VIII presents an overview of the Enterprise Life Cycle (ELC).[9]

 

 

Results of Review

 

The Customer Account Data Engine Project Is Making Progress

Areas of improvement continue to be identified and actively addressed by CADE project management.

While the CADE project has experienced a series of setbacks and postponements since its beginning in September 1999, the project is making progress.  Specifically, program-level and project-level areas of improvement continue to be identified and actively addressed by project management.  The latest version of the CADE (Release 1.3.2) successfully went into production in time for the 2006 Filing Season.[10]

Actions to address business challenges were beneficial to the CADE project

In 2003, the IRS launched a comprehensive review consisting of three studies and a benchmarking analysis of the BSM program. In addition, the PRIME contractor launched an internal study.  The studies confirmed the IRS modernization effort is a massive, highly complex, high-risk program confronted by a number of critical management and technological challenges.  To address the various assessments, the IRS developed 48 business challenges for itself and the PRIME contractor to complete.  The BSM Challenges Plan Close-Out Report, dated May 1, 2004, provided plans for improved future CADE project development.

We followed up on the 10 business challenges related to the CADE project and determined that each had been adequately addressed.  For example, the Joint Engineering Team, which recommended the formation of the CADE Data Architecture and Analysis Study, is a direct result of addressing an identified business challenge.  Other actions to address business challenges included hiring critical engineering talent and hiring an overall CADE director.  Appendix VI presents descriptions of the 10 business challenges related to the CADE project.

CADE management took corrective actions to prior Treasury Inspector General for Tax Administration recommendations

We followed up on the status of corrective actions taken in response to 13 prior recommendations we made related to the CADE project and determined only 3 corrective actions remained open.  BSMO management implemented or has scheduled corrective action implementation to address all issues.  Appendix VII presents an analysis of the corrective actions to our recommendations.

The IRS and PRIME contractor identified lessons learned to improve future project development activities

In October 2005, the PRIME contractor issued the IRS a lessons-learned workshop report outlining processes that worked well and processes needing improvement during the CADE project’s 2005 development and testing activities.  The lessons-learned workshop report was a joint effort between the IRS and the PRIME contractor.

The report identified 14 processes that worked well and 42 processes that needed improvement.  Items that worked well included aspects of configuration management, transition management, release development, and combined systems integration testing.  Areas that needed improvement also focused on aspects of configuration management and testing, as well as estimating and scheduling the assigned workforce, developing computer operating handbooks, improving communication between architectural design and engineering groups, and improving project schedule management. Several items specifically address our prior recommendations, which are discussed later in this report.

CADE Release 1.3 went into production in time for the 2006 Filing Season

Beginning with the development of CADE Release 1.3, the IRS and the PRIME contractor planned to use a biannual release approach - one release delivery in June and one release delivery in January.  Appendix IV, Table 2, presents the revised CADE biannual release schedule including return types, account characteristics, and estimated tax return volumes for each release.  The first biannual release (Release 1.3.1) was delivered in September 2005.  Release 1.3.1 processed Form 1040EZ tax returns, improved long-term database efficiency, accepted tax returns from taxpayers who changed addresses between filing seasons, implemented interest rate changes, and handled initial 2006 Filing Season tax law changes.

While the current release of the CADE is performing very well, significant challenges await the next release (CADE Release 2), which plans to add several new tax forms and schedules.

In January 2006, the IRS and the PRIME contractor successfully delivered the second biannual release (Release 1.3.2) in time for the 2006 Filing Season.  Release 1.3.2 targeted a larger taxpayer population by including Forms 1040 and 1040A with no schedules, improving address change capabilities, and including last-minute 2006 Filing Season tax law changes.  As of February 24, 2006, the CADE had processed over 2.7 million tax returns and generated over $1.4 billion in tax refunds, compared to the 1.4 million tax returns processed that generated $427 million tax refunds in of all of 2005.

Although the CADE has experienced cost and schedule overruns in the past, the 2006 Filing Season goals were substantially met.  Appendices IV and V, Table 3, present the CADE project cost and schedule variances.

The PRIME Contractor’s Inadequate Testing Resulted in Reduced Capabilities for Customer Account Data Engine Release 1.3

While the above results are commendable, the IRS has had to reduce the number of tax returns it plans to process in 2006.  The latest approved CADE Capital Asset and Business Case Exhibit 300[11] (dated February 2005) estimated Release 1.3.2 would identify 20 million tax returns with characteristics for CADE processing. However, after reconsidering the original projection, the deferral of 8 planned requirements, and the addition of unexpected disaster-related requirements, Release 1.3.2 identified fewer than 14 million tax returns for potential CADE processing.  In October 2005, the BSMO estimated that only 4 million of the 14 million tax returns having CADE processing potential would be processed by the CADE.  This reduction is due to account activity which the current CADE Release cannot process. Had Release 1.3.2 realized the 20 million tax returns envisioned in the Capital Asset and Business Case Exhibit 300, even more taxpayers would have received CADE benefits such as faster refunds.

On October 27, 2005, the IRS and the PRIME contractor renegotiated the Release 1.3.2 task order and deferred eight requirements to future CADE releases.  The task order modification contained a cost adjustment reduction of almost $3.2 million to account for the removal of the eight requirements. The deferred requirements involved the following capabilities:

  • Processing tax returns for taxpayers affected by designated disasters.
  • Processing prior year tax returns.
  • Processing offsets to tax overpayments.
  • Processing requests for extensions to file tax returns.
  • Processing tax returns with errors.
  • Processing refund suspensions for tax returns meeting Criminal Investigation fraud criteria.
  • Processing direct-deposit refunds for tax returns filed on paper versus electronically.
  • Processing improvements for manual refunds.

The IRS and the PRIME contractor agreed to defer the eight Release 1.3.2 requirements to help ensure the CADE would be operational in time for the 2006 Filing Season.  Several factors contributed to the removal of the eight requirements from Release 1.3.2.

 

The Internal Revenue Manual and the ELC provide guidance for developing and documenting project requirements and computer program applications.  The Internal Revenue Manual also provides guidance to plan, execute, and develop several forms of computer program testing, including unit testing and systems acceptance testing.

During development of CADE Release 1.3.1, the PRIME contractor did not effectively and efficiently use computer language (coding practices) and the contractor’s unit testing[12] did not adequately identify programming defects.  This is evidenced by the Release 1.3.1 Combined Systems Integration Testing[13] which experienced a high number of failed tests.  Between June and August 2005, the defect rate averaged over 20 percent.  MITRE[14] provided information on defect rates to measure productivity improvements claimed by the PRIME contractor during Release 1.3.2.  MITRE’s research on defect rates reinforced its observations that the PRIME contractor’s productivity is well below industry standards.  Subsequently, the PRIME contractor did not have adequate staffing to fix the Release 1.3.1 defects, requiring the transfer of designers and developers from Release 1.3.2 startup activities.

The PRIME contractor assessed its performance in developing CADE Release 1.3.1 and compiled lessons learned from its experience.  The PRIME contractor’s lessons-learned report contained the following recommendations that we had previously reported:

  • Computer operations handbooks need more detail.
  • Job Control Language[15] programming needs to be more efficient.
  • The schedule estimation approach needs to be enhanced.

Because of the problems experienced in development, Release 1.3.1 was delivered in September 2005, 2 months behind schedule.  This delay significantly shortened the Release 1.3.2 project schedule, which was time sensitive to the 2006 Filing Season.  Because of the abbreviated development period available for Release 1.3.2, the eight requirements cited previously were removed.  The IRS did not estimate the number of taxpayers affected in making its decision to defer the eight requirements to future CADE releases.  As a result, an unknown number of taxpayers will not receive the benefits associated with CADE processing, primarily faster processing of income tax return refunds.  In addition, Release 1.3.2 taxpayer account balancing, control, and reconciliation process[16] requirements were not finalized until December 9, 2005.  Also, the IRS did not fully develop the requirement to process refund suspensions meeting Criminal Investigation function fraud criteria in time for inclusion in Release 1.3.2.

Release 1.3.2 went into operation on January 17, 2006.  However, all planned Release 1.3.2 requirements were not deployed initially because of the abbreviated development period.  The remaining CADE release requirements were incrementally deployed as their testing was completed.

The impact of the application development problems has had a cascading effect.  The CADE Release 2.1 project team has had to compete with the Release 1.3.2 project team for development resources during startup activities.  This shortfall evolved from the resources the Release 1.3.2 project team had to divert to make Release 1.3.1 operational.

Recommendation

Recommendation 1:  The Chief Information Officer should ensure current and future CADE task order contracts and modifications provide an assurance to the IRS that unit testing was adequately performed by the PRIME contractor and the application provided is ready for integration and testing by the IRS. This assurance should take the form of something similar to an Application Qualification Testing[17] report or other deliverable.  The requirement for this assurance should be incorporated into the Modernization and Information Technology Services (MITS) organization’s testing guidance, which is being revised by the Technology Release Management office.

Management’s Response: The IRS agreed with our recommendation. Prior to IRS testing, the PRIME contractor will be required to provide the IRS with notification regarding the completion of unit testing.  The CADE has placed this requirement of formal notification regarding the completion of unit testing in the Release 2 task order and will do so in future task orders.  Further, the MITS organization’s Test Process Improvement Working Group of Stakeholders will review existing test guidance and make the appropriate modification to ensure that unit testing was adequately performed by the PRIME contractor and the application provided to the IRS is ready for integration and testing.

Successful Development of the Customer Account Data Engine Is Dependent Upon Its Ability to Serve As an Efficient and Effective Foundation for Modernized Systems and Applications

The CADE’s design for managing taxpayer accounts is not fully developed to provide a foundation for the IRS’ current and planned modernized systems.  In a November 2002 briefing to IRS executives, the PRIME contractor stated it had developed the basic infrastructure for CADE Releases 1 through 5 and that 95 percent of the applications for operating Release 1 had been built.  This declaration may have led to a belief that, while Release 1 was experiencing difficulties, the underlying design and infrastructure was almost finished.

Since then, the CADE project has developed Releases 1.1 through 1.3.2. However the IRS and the PRIME contractor have modified the release strategies.  Appendix IV, Table 2, presents the latest release schedules. In April 2005, the BSMO initiated the CADE Data Architecture and Analysis Study, which should be completed in June 2006.  This Study’s overall objective is to enable the project team to adopt a flexible release strategy by establishing a long-term plan to organize and manage project data.  To successfully deliver future releases, the project team needs to address the following topics in the CADE Data Architecture and Analysis Study.

The current CADE logical design release strategy supports only the first two CADE releases

The logical design provides a specific description of the hardware, system software, and network components comprising the IRS’ systems and applications.  The logical design presents a view of the technology being used, including definitions of workstations, hardware components, and interfaces with other IRS systems and applications.

The original CADE release strategy was focused on moving ‘easy’ taxpayers onto the CADE in the early releases and building on this base with more complexity through the later releases.  With this focus, the project’s logical design did not include plans to support the complex tax returns the CADE would process subsequent to Release 2.

Until a long-term logical design is developed, future releases of the CADE cannot begin development.  Additionally, on February 23, 2006, the project team stated an interim solution for the Release 2.1 logical application design had not been completed.

The CADE project has not completed its long-term physical design

The physical design describes the IRS’ systems and applications in physical terms.  It specifies vendors, products, models, versions, releases, capacities, connections, and other options for the planned project.  The purpose of the physical design is to guide the construction and ultimate integration of the IRS’ systems and applications after its logical design is complete.

Because the logical design is in development and a long-term physical design is dependent on the logical design’s completion, the CADE project is building an interim physical design. The interim physical design solution is another area of concern affecting the next CADE Release.  As of February 23, 2006, the interim physical design work for Release 2.1 was several weeks behind schedule.  Until a long-term physical design is developed, testing cannot be performed that will ensure the adequacy of CADE release development.

The IRS did not use the originally selected data conversion plan because of time constraints related to acquisition, evaluation, and implementation

The objective of the data conversion is to convert data between the IRS’ current processing environment formats and the CADE database while minimizing the effect on IRS employees and customers.  Taxpayer account information maintained in the CADE is available to users in the current processing environment format.  Data conversion consists of:

  • Data mapping two mappings are required for the CADE:  1) current processing environment as the source and the CADE as the target and 2) the CADE as the source and current processing environment as the target.
  • Data analysis/data cleansing provides a basis for ensuring completeness of CADE data.
  • Data transformation copies current processing environment data to the CADE.
  • Data validation ensures the transformation of data is correct.

In 2002, we reviewed the IRS’ data conversion plans and reported the plans adequately documented the process necessary to move taxpayer account data to and from existing IRS systems and the modernized CADE.[18]  We cautioned that the key to success was to carefully track the data conversion progress against the project development plans and timely address any potential barriers or risks.

The IRS subsequently abandoned the original data conversion plan because of time constraints related to acquisition, evaluation, and implementation.  As a result, a process termed “quick and dirty” by the IRS was used for the first CADE Release.  The data conversion programs for processing Release 2 taxpayer accounts have not yet been completed.

As part of the CADE Data Architecture and Analysis Study, data conversion recommendations in May and October 2005 proposed several solutions for the long term (CADE Release 3 and beyond).  A definitive approach is still to be determined.  As the CADE project progresses, the data conversion process will become more complex and riskier with the addition of returns, schedules, and taxpayers.

The CADE currently does not store historical financial data

The Office of Management and Budget prescribes[19] policies and standards for executive departments and agencies to follow when developing, operating, evaluating, and reporting on financial management systems.  Several years ago, the IRS initiated a change request to eliminate the requirement to store historical financial data in the CADE for Release 1.  The change request cited overall complexity, and the Release 1 capabilities did not require the storage of financial history. This capability was deferred because financial information for tax returns processed by CADE Release 1 is captured in the current processing environment.

For the 2007 Filing Season, several Form 1040 schedules will be added to CADE Release 2 requiring the storage of historical data.  Examples include Interest and Ordinary Dividends (Schedule B), Profit or Loss From Business (Schedule C), Supplemental Income and Loss (Schedule E), and Profit or Loss From Farming (Schedule F).

The CADE Data Architecture and Analysis Study team is examining the use of the Custodial Accounting Project[20] information as a means of populating the CADE’s historical financial database.  The historical data conversion will be a significant undertaking for the CADE if the financial information for tax returns in the current processing environment needs to be completely rewritten.

The CADE Data Architecture and Analysis Study is integral to the IRS’ future modernization plans

The MITS organization is analyzing the progress of the BSM program and reconsidering the approach to the program’s architecture and development.  To provide a better focus, the MITS organization is forming the Modernization Vision and Strategy to establish project development priorities with the IRS business operations.  The Modernization Vision and Strategy will provide a 5-year plan for developing modernization projects and processes.

The CADE project is integral to the future plans for modernization, and the recommendations resulting from the CADE Data Architecture and Analysis Study will help determine the future plans for developing the CADE project.  Until the issues cited previously are resolved, successful deployment of a long-term CADE release strategy cannot move forward, and the future releases may not be deployed when planned.  For the near term, the risk of untimely implementation of CADE Release 2 is significantly affected by unresolved solutions around the project’s logical and physical designs, the data conversion plans, and the storage of financial information.

Recommendation

Recommendation 2:  The Chief Information Officer should ensure the CADE Data Architecture and Analysis Study and the MITS organization’s new Modernization Vision and Strategy are expeditiously completed, so the CADE project can implement realistic requirements and development plans.

Management’s Response: The IRS agreed with our recommendation. The MITS organization will use the CADE Data Study results as the basis for developing data models to guide designs for CADE Releases 3 and 4.  The MITS organization will also review Modernization and Vision project strategies for extracting operational and analytical data to develop a high-level strategy for data warehousing.  Building upon the CADE data models and the high-level data warehousing strategy, the MITS organization will form a data access strategy and transition plan.

Controls to Manage Customer Account Data Engine Requirements Do Not Allow Ready Access for Project Team Use

The GAO Standards state transactions should be promptly recorded to maintain their relevance and value to management in controlling operations and making decisions.

The ELC’s BSMO Process Description for requirements development and management designates the Acquisition Project Manager as responsible for ensuring project requirements are defined, testable, and traceable.  The requirements for CADE Releases 1.1 and 1.2 are contained in the System Requirements Reports, which were updated to reflect changes as they occurred. For CADE Release 1.3, the IRS and the PRIME contractor agreed to create separate documents called “White Papers” to document development of each specific requirement.  This process change was made to ensure a better understanding of the performance needed to meet the project requirements and to minimize rework during testing.

Adequate procedures were not in place to ensure all CADE release requirements are timely controlled. The project team uses the Document Management System as a database to control the White Papers.  When we requested a complete listing of Release 1.3 requirements, the project team was unable to timely provide the information. The project team and the PRIME contractor needed over 3 months to assemble and provide us all information relating to the population of White Papers.

The Government Accountability Office (GAO) Standards for Internal Control in the Federal Government[21] state control activities occur at all levels and functions of the entity and include the creation and maintenance of related records that provide evidence of execution of these activities as well as appropriate documentation.  Because CADE Release 1.3 requirements are not readily available for project team use, the IRS cannot easily determine whether all agreed-to requirements were developed and tested.

BSMO management stated Release 2 will not use White Papers to document specific requirements, as was the case in Release 1.3.  As of February 23, 2006, an interim process to control Release 2 requirements was in development.  The BSMO will use this process until a vendor able to develop a repository meeting the IRS’ needs is identified.  Once implemented, the new repository will be managed by the MITS organization Requirements Management Office.

Recommendation

Recommendation 3:  Until a new repository is identified and built to control future CADE release requirements, the Chief Information Officer should timely implement an interim process that will effectively and efficiently control CADE Release 2 requirements, which are currently in development.

Management’s Response:  The IRS agreed with our recommendation.  As an interim measure, a project repository has been developed within the CADE to track and control requirements for each release.  This vehicle will remain in place until the implementation of the planned Enterprise Services organization’s repository is completed.

Defining Customer Account Data Engine Project Release Requirements Will Help Enable the Use of Fixed-Price Contracts

The IRS is not using fixed-price contracts for CADE Release 1.3 development or for Release 2 design work performed by the PRIME contractor.  Both releases are using hybrid cost-plus-fixed-fee,[22] cost-plus-incentive-fee,[23] and cost-plus-award-fee contracts.[24]

The BSMO implemented contracting guidance on April 30, 2004, entitled “Enabling Fixed-Price Contracting for Business Systems Modernization Task Orders,” that requires fixed-price contracts and task orders for BSM acquisition projects at the appropriate life cycle development phase, unless the Federal Government’s interest is best served by other contract types.  The ELC established Milestone 4 as the business systems development and enterprise deployment decision point.  The Milestone 4 activities are separated by two checkpoints. Activities leading up to Milestone 4A involve further requirements definition, development of the system’s physical design, and determination of the applicability of fixed-price contracting to complete system development and deployment.  Once the requirements are finalized and agreed to by the IRS and the PRIME contractor, a fixed-price contract can be used to achieve Milestone 4B.

The BSMO’s guidance acknowledges that fixed-price contracts are used when the risk involved can be predicted with acceptable certainty.  A fixed-price contract is suitable for acquiring supplies or services on the basis of “reasonably definite functional or detailed specifications” when the contracting officer can establish fair and reasonable prices at the outset, performance uncertainties can be identified and reasonable estimates of their cost impact can be made, and the contractor is willing to accept a fixed price representing assumption of the risks involved.

The IRS recognized the following challenges in working with the PRIME contractor to use fixed-price contracts when they reach Milestone 4B:

  • The IRS and the PRIME contractor need to reach agreement on the amount of effort necessary to complete each specific CADE release.
  • The IRS needs to get the PRIME contractor to commit to fixed-price contracts close to the IRS’ estimates.
  • The IRS needs to respond to additional unanticipated application requirements from new tax law changes and other special provisions (e.g., modified filing requirements for disaster victims).
  • The IRS needs to overcome funding contingency restrictions.  The BSM Program Management Office retains some CADE funds as a contingency to cover funding shortfalls in other modernization projects. Committing CADE funds to a fixed-price contract will make contingency funding difficult.

The use of fixed-price contracts should lead to more cost-effective acquisitions, better value, and greater competition.  Fixed-price contracts have the net effect of shifting some of the manageable performance risk from the IRS to the contractors.  When the contractor does not bear enough risk under a task order, the contractor does not have an incentive to perform well.  Contractors will be given more latitude for determining methods of performance, with more responsibility for performance quality.  The IRS should experience fewer cost overruns, schedule delays, and performance problems.  These benefits of fixed-price contracting will be realized only when the CADE project requirements are fully developed and agreed to, comprehensive estimates of effort to perform the contract tasks are made, and meaningful negotiations are held.

Recommendation

Recommendation 4:  The Chief Information Officer should ensure the BSMO, the Enterprise Services organization’s Business Rules and Requirements Management office, and the IRS’ contracting officers work with the PRIME contractor to adequately define project requirements enabling implementation of fixed-price contracts when project development reaches Milestone 4B. Hybrid fixed-price contracts incorporating cost-plus provisions should be used only to contract the development of additional unanticipated requirements, such as new tax law changes. Additionally, the BSMO and the IRS’ contracting officers should reevaluate the benefits of using a fixed-price contract, if possible, for the remaining work (Milestone 4B) on CADE Release 2.

Management’s Response:  The IRS agreed with this recommendation.  The Acquisition Development organization, the Enterprise Services organization, and the Office of Procurement will work to position the IRS with the ability to award fixed-price contracts.  CADE project personnel, in conjunction with the IRS contracting officers, reevaluated the benefits of using a fixed-price contract for the remaining work on Release 2.  After full review, negotiations, and consideration, the IRS made a business decision to continue with the cost-plus-award-fee contract.

 

Appendix I

 

Detailed Objective, Scope, and Methodology

 

The overall objective of this review was to determine whether the Customer Account Data Engine (CADE) project management and development activities were effective in assuring CADE releases[25] included all planned capabilities and were implemented within cost and schedule estimates.  This review was part of our Fiscal Year 2005 audit plan for reviews of the Internal Revenue Service’s Business Systems Modernization (BSM) efforts.

To accomplish our objective, we reviewed available documentation and interviewed Business Systems Modernization Office (BSMO) personnel and Modernization Acquisition function analysts from the Agency-Wide Shared Services organization. Specifically, we:

I.                   Assessed the activities of the BSMO and the PRIME contractor[26] to deliver all agreed-to requirements for CADE Releases 1.3 and 2.

A.    Interviewed BSMO staff to determine the final requirements the PRIME contractor delivered for CADE Releases 1.3.1 and 1.3.2.

B.     Interviewed BSMO staff to determine the effect the month-by-month funding process had on CADE Releases 1.3.1 and 1.3.2.

C.     Interviewed BSMO staff to determine the status of the testing schedule activities related to CADE Releases 1.3.1 and 1.3.2.

D.    Reviewed CADE Release 2 planning activities to determine whether the BSMO and the PRIME contractor were on track to successfully deliver additional CADE taxpayer benefits.

E.     Monitored BSMO and PRIME contractor actions related to the CADE Data Architecture and Analysis Study to determine the effect, if any, on CADE Releases 1.3.1, 1.3.2, and 2 activities.

II.                Determined whether the actions taken to address the BSM Challenge Issues have improved the management of the CADE project to successfully move forward.

III.             Followed up on the status of corrective actions taken in response to prior Treasury Inspector General for Tax Administration audit report recommendations.

 

Appendix II

 

Major Contributors to This Report

 

Margaret E. Begg, Assistant Inspector General for Audit (Information Systems Programs)

Gary V. Hinkle, Director

Edward A. Neuwirth, Audit Manager

Phung-Son Nguyen, Senior Auditor

Bruce Polidori, Senior Auditor

Steven Gibson, Auditor

Kim McManis, Auditor

 

Appendix III

 

Report Distribution List

 

Commissioner  C

Office of the Commissioner – Attn:  Chief of Staff  C

Deputy Commissioner for Operations Support  OS

Associate Chief Information Officer, Applications Development  OS:CIO:B

Associate Chief Information Officer, Enterprise Services  OS:CIO:ES

Deputy Associate Chief Information Officer, Applications Development  OS:CIO:AD

Director, Procurement  OS:A:P

Director, Stakeholder Management  OS:CIO:SM

Deputy Associate Chief Information Officer, Business Integration  OS:CIO:ES:BI

Deputy Associate Chief Information Officer, Systems Integration  OS:CIO:ES:SI

Director, Test, Assurance, and Documentation  OS:CIO:AD:TAD

Chief Counsel CC

National Taxpayer Advocate  TA

Director, Office of Legislative Affairs  CL:LA

Director, Office of Program Evaluation and Risk Analysis  RAS:O

Office of Management Controls  OS:CFO:AR:M

Audit Liaisons:

Associate Chief Information Officer, Applications Development  OS:CIO:B

Director, Program Oversight  OS:CIO:SM:PO

 

Appendix IV

 

Customer Account Data Engine Release Schedules

 

Table 1 presents the historical development of the Customer Account Data Engine (CADE) and the original CADE release schedule.  The names of the tax forms planned for CADE processing are presented below Table 1.  All tax forms and schedules listed in Tables 1 and 2 are available at the Internal Revenue Service (IRS) web site (IRS.gov).

Table 1:  Original CADE Release Schedule

 

RELEASE
1.1/1.2

RELEASE
1.3.1

RELEASE
1.3.2

RELEASE
2.1

RELEASE
2.2

RELEASE 3

RELEASE
4

RELEASE 5

Tax Return Types

Form 1040EZ; Single filing status;

refund or even-balance returns

Release 1.1/1.2 plus address change

Release 1.3.1 plus
Forms 1040 and 1040A with no schedules; Forms 4868 and 2688; prior year returns (2003+); limited
name-change returns

Form 1040EZ; Form 1040 Schedules A, B, and R;

Form 1040A Schedules 1 and 3

Release 2.1 plus
Form 1040 Schedules C, D, E, F, and H without an Employer Identification Number (EIN) and their supporting schedules

All Form 1040 family and supporting forms without an EIN;
Form 1040A Schedule 2; refund,

deceased

refund, or fully paid returns

All Form 1040 family and supporting forms with an EIN;

Forms 941, 940, and 720; payroll, unemployment, and excise returns for Form 1040 taxpayers;

refund, fully paid,
balance-due,  and
even-balance returns

All remaining individual tax returns

Filing Status

Single

 

Single

 

Single

 

Single; Married;  Head of Household, limited dependents

Single; Married;  Head of Household, limited dependents

All (including Head of Household)

All

All

Account Characteristics

No account issues (open or closed)

No account issues (open or closed)

No account issues (open or closed)

No account issues (open or closed)

Married once; no open account issues

No open account issues; Earned Income Tax Credit (EITC)

Power of Attorney; Centralized Authorization File;[27] no open account issues

All accounts not included in previous Releases

 

 

RELEASE
1.1/1.2

RELEASE
1.3.1

RELEASE
1.3.2

RELEASE
2.1

RELEASE
2.2

RELEASE 3

RELEASE 4

RELEASE 5

Number of

Returns

Original estimate

Revised Estimate -  February 2005

 

 

 

3 million

 

 

2 million

 

 

 

 

 

 

2 million

 

 

 

 

 

 

To Be Determined (TBD)

 

 

 

 

 

 

TBD

 

 

 

35 Million

 

 

33 million

 

 

 

 

76 Million

 

 

50 Million

 

 

 

110 Million

 

 

80 Million

 

 

 

122 Million

 

 

140 Million

Estimated Delivery:

As of April 2000

As of January 2004

As of December 2004

As of February 2005

 

 

 

January 2002

 

 

 

August 2004

 

 

 

August 2004

 

 

August 2004/ January 2005

 

 

 

 

 

 

 

 

 

 

July 2005

 

 

July 2005

 

 

 

 

 

 

January 2006

 

 

 

January 2006

 

 

January 2006

 

 

 

August 2002

 

 

 

TBD

 

 

 

July 2006

 

 

July 2006

 

 

 

 

 

 

 

TBD

 

 

 

January 2007

 

 

January 2007

 

 

July 2003

 

 

 

TBD

 

 

 

TBD

 

 

Releases 3.1/3.2
July 2007/
January 2008

 

 

July 2004

 

 

 

TBD

 

 

 

TBD

 

 

Releases 4.1/4.2
July 2008/
January 2009

 

 

July 2005

 

 

 

TBD

 

 

 

TBD

 

 

July 2009

Source:  CADE Individual Master File[28] Release Content Master Plan Update, dated June 5, 2003.

Form 720 – Quarterly Federal Excise Tax Return.

Form 940 – Employer’s Annual Federal Unemployment (FUTA) Return.

Form 941 – Employer’s QUARTERLY Federal Tax Return.

Form 1040 – U.S. Individual Income Tax Return.

Form 1040A – U.S. Individual Income Tax Return.

Form 1040EZ – Income Tax Return for Single and Joint Filers With No Dependents.

Form 2688 – Application for Additional Extension of Time to File U.S. Individual Income Tax Return.

Form 4868 – Application for Automatic Extension of Time to File U.S. Individual Income Tax Return.

 

Table 2 presents the current CADE release schedule. The names of additional tax forms planned for CADE processing are presented below Table 2.

Table 2:  Revised Biannual CADE Release Schedule

 


January and
June

2004

RELEASE 1.1

2005

RELEASE 1.2
1.3.1

2006

RELEASE 1.3.2
2.1

2007

RELEASE 2.2
3.1

2008

RELEASE 3.2
4.1

2009

RELEASE 4.2
5.1

2010

RELEASE 5.2
6.1

2011/2012

RELEASE 6.2
7.1/7.2

Tax Return Types

Form 1040EZ

 

1.2 & 1.3.1
Form 1040EZ

(1.3.1 only)
Address changes

 

1.3.2
Forms 1040 and 1040A no schedules

2.2
Form 1040A Schedules 1 and 3;
Form 1040
Schedules A, B, D, and R and supporting forms;
Form 1040 Schedules C, F, and E without an EIN and supporting forms including
Schedule SE; limited name change on return

3.2

Form 1040 decedent returns;

Form 1040 fully paid with remittance; Form 1040 with credit elect;
Form 4868 with remittance;
Form 4868 with no remittance; additional  Form 1040 schedules and forms (TBD); Form
1040-ES;
Form 1040-V; Form 1040X; Form 1040A Schedule 2; Form 1040 Schedule EITC and  supporting forms

4.2

Balance-due returns; math error returns;
Form 6251; additional
Form 1040 schedules and forms (TBD)

5.2

Form 1040 Schedules C, E, F with an EIN and supporting schedules including Schedule SE; Forms 940 and 720 payroll, unemployment, and excise tax returns for Form 1040
self-employed filers; additional
Form 1040 schedules and forms (TBD); delinquent returns

 

6.2

Additional Form 1040 schedules and forms (TBD)

 

7.2

Form 1040NR;
Form 1040 Puerto Rico Resident; Form 1040 Self- Employed; Form 1040 Departing Alien; foreign address; additional Form 1040 schedules and forms (TBD)

 

Filing Status

Single

(never married); no dependents

 

Single

(never married); no dependents

 

2.1

Single, Married (joint), Head of Household, Married (separate)

3.1

Single, Married (joint), Head of Household, Married (separate), Surviving Spouse

4.1

Single, Married (joint), Head of Household, Married (separate), Surviving Spouse

5.1

Single, Married (joint), Head of Household, Married (separate), Surviving Spouse

6.1

Single, Married (joint), Head of Household, Married (separate), Surviving Spouse

7.1

Single, Married (joint), Head of Household, Married (separate), Surviving Spouse

 

January and
June

2004

RELEASE 1.1

2005

RELEASE 1.2
1.3.1

2006

RELEASE 1.3.2
2.1

2007

RELEASE 2.2
3.1

2008

RELEASE 3.2
4.1

2009

RELEASE 4.2
5.1

2010

RELEASE 5.2
6.1

2011/2012

RELEASE 6.2
7.1/7.2

Account Characteristics

Refund or even-balance returns; No account issues (open or closed)

Refund or even-balance returns; no dependents;
No account issues (open or closed)

Refund or even-balance returns; no dependents;
No account issues (open or closed)

2.2
Married once; No open account issues 3.1
No open account issues

EITC

3.2

No open account issues.

EITC
4.1
Power of Attorney; Centralized Authorization File;

no open account issues; EITC

No open account issues EITC

Power of Attorney; Centralized Authorization File;
No open account issues

All accounts not included in previous releases

Estimated
Number of Returns

 

Not applicable

Actual

1,423,517

1.3.2

4 million

2.2

33 million

3.2

50 Million

4.2

70 Million

5.2

90 Million

6.2

110 Million

7.2
135 Million

Source:  Customer Relationship Management Executive Steering Committee, approved October 18, 2005.

Form 1040-ES – Estimated Tax for Individuals.

Form 1040NR – U.S. Nonresident Alien Income Tax Return.

Form 1040-NR-EZ U.S. Income Tax Return for Certain Nonresident Aliens With No Dependents.

Form 1040-V – Payment Voucher.

Form 1040X – Amended U.S. Individual Income Tax Return.

Form 6251 – Alternative Minimum Tax–Individuals.

 

Appendix V

 

Customer Account Data Engine Project Costs

 

Table 1:  Customer Account Data Engine (CADE) Project Task Order Costs Through Release 1.3

Task Order Number

Period of Performance

Cost

0019

1999 – 2000 (specific dates not available)

$2,166,234

0037

April 24, 2000 – August 31, 2000

$3,011,000

0054

September 1, 2000 – April 30, 2001

$13,971,165

0069

June 1, 2001 – August 31, 2001

$5,700,000

0071

May 1, 2001 – June 30, 2001

$1,534,012

0073

September 1, 2001 – August 31, 2004

$60,458,154

0123

October 1, 2004 – December 31, 2006

$47,327,641

 

 

Total  $134,168,206

Source:  CADE contract task orders from the Internal Revenue Service Procurement Office.

Table 2:  Estimated CADE Project Costs and Schedule Through Release 1.3

Release 1 Project Phase

Completed/Scheduled

Cost/Estimate

Milestone 1

December 31, 1999

$5,116,000

Milestones 2 and 3

June 30, 2001

$19,267,000

Milestone 4

July 30, 2004

$58,838,000

2003/2004 Filing Seasons[29] (Release 1.1)[30]

August 5, 2004

$24,550,000

2005 Filing Season (Release 1.2)

December 31, 2004

$23,403,000

Milestone 5

June 30, 2005

$17,450,000

2006 Filing Season (Release 1.3)

December 31, 2005

$28,300,000

 

 

Total  $176,924,000

Source:  Business Systems Modernization (BSM) Expenditure Plans for 2004, 2005 and 2006.

 

Table 3:  CADE Project Cost and Schedule Variance Summary

Release

Milestone

Cost Variance

Cost Variance Percentage

Schedule Variance
(in months)

Schedule
Variance
(Percentage)

1

Operations and Maintenance[31]

$7,510,000

43%

0

0%

1.2

2005
Filing Season
[32]

$10,000

0%

0

0%

1.3.1

2006 Filing Season

$0

                  0%

2

20%

1.3.2

2006
Filing Season
[33]

$0

0%

0

0%

2

4a

In Progress

Program
Management

2005
Calendar Year
Level of Effort

$1,935,000

24%

Not
Applicable

Not
Applicable

2006
Calendar Year
Level of Effort

In Progress

Source:  BSM Expenditure Plans for 2004, 2005 and 2006 and the IRS.

 

Appendix VI

 

Business Systems Modernization Challenge Issues Relating to the Customer Account Data Engine

 

The Business Systems Modernization (BSM) Challenges Plan Close-Out Report, dated
May 1, 2004, provided plans to improve future Customer Account Data Engine (CADE) project development.  The CADE-related BSM Challenge Issues and related close-out plans included:

-        Make business owner and program director accountable for project success.

The Business Systems Modernization Office (BSMO) restructured its management and governance structure to institutionalize the roles of Business Leader[34] (replacing Business Owner), Business Requirements Director,[35] Program Director, and Program Manager.

-        Implement short-duration, discrete “Tiger Teams” in the CADE, Integrated Financial System,[36] and e-Services[37] projects.

The CADE Tiger Team[38] conducted a comprehensive analysis of the CADE project and provided recommendations in a report about managing defects, contracts, and program execution.

-        Designate business architects for all projects to support business insight required throughout the project life.

The role of the Business Architects (title changed to Business Requirements Director – see footnote 2) and other program/project management positions were defined as part of the larger governance structure reorganization.

-        Assign business owners and architects to validate business unit participation and accountability in projects.

Roles of the Business Leader, the Business Requirements Director, the BSM Program Director, the Acquisition Project Manager,[39] and the Information Technology Services Coordinator have been defined and socialized among the BSMO, the Modernization and Information Technology Services organization, and the business (operating division/function) executives.

-        Align critical engineering talent to most critical projects.

Key engineering talent from MITRE,[40] the PRIME contractor,[41] and other appropriate resources were identified and hired.

-        Identify key productivity and quality metrics across the life cycle based on industry standards.

A joint Internal Revenue Service (IRS), PRIME contractor, and MITRE Tiger Team developed the PRIME Enterprise Life Cycle (ELC) – the key productivity and quality measures roll-out plan for effective implementation and institutionalization associated with ELC Milestone 4.[42]

-        Move acceptance testing to be earlier in the life cycle.

The PRIME contractor’s testing and deployment office collaborated with the IRS to develop a combined Systems Integration Test/Systems Acceptance Test[43] for CADE Release 1.1.

-        Establish independent architecture and engineering team of IRS and contractor business architects and system engineers.

The IRS and the PRIME contractor formed the Joint Engineering Team[44] consisting of business architects and system engineers.  The Joint Engineering Team recommended the formation of the CADE Data Architecture and Analysis Study to establish a data architecture foundation that would enable the CADE project to adopt a flexible release strategy for current and future CADE releases

-        Clearly define business requirements and tightly manage them to control scope.

The BSMO worked with the PRIME contractor and representatives from various process teams to define Milestone 4A[45] exit criteria, obtain buy-in from key stakeholders, and redefine ELC requirements and milestone exit criteria.

-        Identify “blockers” on current contracting actions.

Blockers were identified on October 24, 2003, which centered on business integration, systems integration, and the PRIME contractor’s program management office activities.

 

Appendix VII

 

Customer Account Data Engine Corrective Actions to Prior Recommendations

 

Table 1 presents the findings reported by the Treasury Inspector General for Tax Administration (TIGTA) and the Internal Revenue Service’s (IRS) subsequent planned corrective actions and implementation status.  The TIGTA report titles are presented below Table 1.

Table 1:  Status of TIGTA Report Findings and Corrective Actions Related to the Customer Account Data Engine (CADE) Project

Report Number
and Date

Finding and Recommendation

Corrective Action

Status

2003-20-018

11/2002

The Pilot Plan Needs to Include Adequate Defect Reporting Procedures.

To ensure the defect reporting databases provide reliable information for the pilot defect reports, the Business Systems Modernization Office (BSMO) should require the PRIME contractor[46] to provide written procedures directing that the IRS approve defect report resolution actions prior to defect report closure.

The IRS monitors Information Technology Asset Management System (ITAMS)[47] use and requires the PRIME contractor to develop procedures to include defect report resolution.  Under the draft procedures, the PRIME contractor initiates the recommendation to close in the ITAMS and the IRS is responsible for the final closure.  Those procedures are currently under review.  Approval and distribution of the procedures is planned for November 30, 2002.

Closed

 

Report Number
and Date

Finding and Recommendation

Corrective Action

Status

2003-20-018

11/2002

The Pilot Plan Needs to Include Adequate Defect Reporting Procedures.

To ensure the defect reporting databases provide reliable information for the pilot defect reports, the BSMO should require the PRIME contractor to provide detailed procedures for reconciling defect reports in the ClearQuest®[48] database and the ITAMS.

The PRIME contractor has drafted procedures that document the process the IRS will use when interfacing defect reporting information between the ClearQuest® database and the ITAMS defect tracking tools.  These procedures contain specific information on the reconciliation of these reports.  The procedures are currently under review and are scheduled to be approved and distributed by November 30, 2002.

Closed

2003-20-089

3/2003

File and Job Names Need to Be Compatible With Current Tax Processing Systems.

The Deputy Commissioner for Modernization and Chief Information Officer needs to ensure development of job and file naming standards is expeditiously completed by the IRS Enterprise Operations Services organization. The BSMO needs to work with the PRIME contractor to ensure these naming standards are used in the development of future CADE releases and all other IRS modernization projects.

The BSMO agreed with the recommendation and is developing naming standards.  Key naming standards will be published and available for use by the current processing and modernization environments by August 1, 2003.

Closed

Report Number
and
Date

Finding and Recommendation

Corrective Action

Status

2003-20-089

3/2003

The Balancing, Control, and Reconciliation Process Needs to Be Completed and Tested Prior to Release 1.

The BSMO should monitor the completion of the remaining work contained in the detailed schedule to assess the progress in completing the balancing, control, and reconciliation process development.

The BSMO agreed with the recommendation and will continue to monitor the actions taken in balancing, control, and reconciliation in preparation for CADE Release 1 deployment.

Closed

2003-20-089

3/2003

Improvements to the Computer Handbook Will Help to Ensure an Effective Release 1 Deployment.

The BSMO should work with the PRIME contractor to incorporate minimum documentation standards and the Documentation Task Force’s findings into CADE Release 1 and future releases.

The BSMO agreed with the recommendation and will work with the PRIME contractor to incorporate minimum documentation standards and the Documentation Task Force’s findings into the CADE.

Closed

 

Report Number
and
Date

Finding and Recommendation

Corrective Action

Status

2003-20-089

3/2003

Improvements to the Computer Handbook Will Help to Ensure an Effective Release 1 Deployment.

The BSMO should work with the PRIME contractor to incorporate these documentation standards into the Enterprise Life Cycle.[49]  Incorporating standards into the Enterprise Life Cycle will provide guidance in developing documentation for current and future CADE releases, as well as all other IRS modernization projects.

The BSMO is working with the PRIME contractor and Information Technology Service organization as a task group to define standard deliverables that will facilitate improved transition of modernization systems into the current processing environment.  The computer operations handbook subtask group is working to define gaps and requirements for standard deliverables needed by the Information Technology Service organization for operations and maintenance support.

Closed

2003-20-089

3/2003

The Balancing, Control, and Reconciliation Process Needs to Be Completed and Tested Prior to Release 1.

The BSMO should require the PRIME contractor to complete the remaining work on the balancing, control, and reconciliation process and fully test these processes to ensure they meet the design requirements

The BSMO will continue to work with the PRIME contractor to ensure the remaining work for the balancing, control, and reconciliation process for
Release 1 is tested and completed.

Closed

 

Report Number
and
Date

Finding and Recommendation

Corrective Action

Status

2005-20-005

11/2004

Significant Software Changes Were Added to Customer Account Data Engine Release 1.1.

The Chief Information Officer should direct the BSMO to work with the PRIME contractor to ensure future project development changes undergo appropriate performance testing, simulating high-volume processing, before deploying the system.

The BSMO is planning a semiannual release of the CADE in July[50] and January.  The July delivery will involve higher risk, more complex functionality, and the January delivery will include filing season[51] changes combined with additional changes as capacity permits.  Since the returns from earlier in the filing season will be available for testing, the IRS can conduct performance testing on the July release using the highest volume periods.  The IRS will determine whether to conduct additional performance testing on the January release based on the likelihood of the changes affecting performance.

Closed

 

Report Number
and
Date

Finding and Recommendation

Corrective Action

Status

2005-20-005

11/2004

The Customer Account Data Engine Program Does Not Have a Dedicated System Architect.

The Chief Information Officer should ensure the BSMO makes the system architecture resources being acquired available to the CADE program on a full-time basis.

Over the last year, the BSMO has been actively searching for qualified engineering resources.  While the BSMO has had success recruiting a few candidates (some of which are dedicated to the CADE), it is very difficult to attract qualified individuals to work for the Federal Government.  The BSMO’s recruiting efforts will continue.  In the interim, MITRE[52] employees and members from the Enterprise Architecture team are providing CADE engineering and architecture support.  The PRIME contractor is continuing to provide full-time architecture support to the CADE project.

Open

 

Report Number
and
Date

Finding and Recommendation

Corrective Action

Status

2005-20-005

11/2004

Disaster Recovery Capabilities Were Not Tested Prior to Customer Account Data Engine Release 1.1 Implementation.

To ensure the ability of the IRS to restore the CADE after a disaster with the least disruption to the IRS mission, the Chief Information Officer needs to ensure disaster recovery capabilities for future releases of the CADE are fully tested prior to implementation.

Disaster recovery for the CADE needs to be periodically tested.  However, it should be part of an enterprise disaster recovery and testing strategy for mainframe computing operations and databases at the Enterprise Computing Center.[53]  The IRS generally deploys major functionality enhancements for future CADE releases in July and deploys filing season changes and smaller system enhancements as part of CADE filing season releases in January.  The Systems Integration Enterprise Architecture Office will work with the Enterprise Operations Service organization and the Enterprise Computing Center to ensure CADE disaster recovery testing is included in the annual disaster recovery testing performed each fall.  Since most new functionality will be deployed in the July release, the fall testing would be timely prior to the start of the new filing season when the IRS will be encountering high volumes.

Open

 

Report Number
and Date

Finding and Recommendation

Corrective Action

Status

2005-20-005

11/2004

Improvements to the Customer Account Data Engine Operator’s Guide Need to Be Completed.

The Chief Information Officer should direct the BSMO to ensure pilot and production operational documentation for future CADE releases is reviewed, tested, and approved before both the pilot test and live production are allowed to proceed.  This review should also ensure the corrective action to our March 2003 report is incorporated in this review and the IRS’ minimum documentation standards are completed and met in current and future CADE releases.

The BSMO made the required changes to the operational documentation for CADE
Release 1.1; however, the final copy with “no markup” was not available for a few weeks following initial operation.  Going forward, the BSMO plans to have full documentation available at implementation.  There are no plans for additional CADE pilots.

Closed

2005-20-005

11/2004

Manual Processes Within Customer Account Data Engine Release 1.1 Need to Be Automated for Future Releases.

The Chief Information Officer should direct the BSMO to ensure inefficient manual processes are automated in future CADE releases.

Changes will be included in the January 2005 and July 2005 deliveries addressing the specified operational inefficiencies.

Open

 

Report Number
and
Date

Finding and Recommendation

Corrective Action

Status

2005-20-005

11/2004

Disaster Recovery Capabilities Were Not Tested Prior to Customer Account Data Engine Release 1.1 Implementation.

To ensure the ability of the IRS to restore the CADE after a disaster with the least disruption to the IRS mission, the Chief Information Officer needs to ensure all aspects of the CADE disaster recovery capabilities are tested during the Annual IRS Disaster Recovery Test.

The Director, Enterprise Computing Center, will test the recovery and restoration of key components of CADE processing to ensure a high confidence level in the IRS’ ability to recover the CADE in the event of a disaster.  The CADE was part of the annual Enterprise Computing Center Master File[54] Disaster Recovery test.  The test was conducted.

Closed

Source:  TIGTA audit reports and IRS management responses.

2003-20-018 - Improvements in the Customer Account Data Engine Pilot Plan Need to Be Considered to Help Ensure the Pilot’s Success (Reference Number 2003-20-018, dated November 2002).

2003-20-089 - Adhering to Established Development Guidelines Will Help to Ensure the Customer Account Data Engine Meets Expectations (Reference Number 2003-20-089, dated March 2003).

2005-20-005 - To Ensure the Customer Account Data Engine’s Success, Prescribed Management Practices Need to Be Followed (Reference Number 2005-20-005, dated November 2004).

 

Appendix VIII

 

Enterprise Life Cycle Overview

 

The Enterprise Life Cycle (ELC) is the Internal Revenue Service’s (IRS) standard approach to business change and information systems initiatives.  It is a collection of program and project management best practices designed to manage business change in a successful and repeatable manner.  The ELC addresses large and small projects developed internally and by contractors.

The ELC includes such requirements as:

·         Development of and conformance to an enterprise architecture.

·         Improving business processes prior to automation.

·         Use of prototyping and commercial software, where possible.

·         Obtaining early benefit by implementing solutions in multiple releases. [55]

·         Financial justification, budgeting, and reporting of project status.

In addition, the ELC improves the IRS’ ability to manage changes to the enterprise; estimate the cost of changes; and engineer, develop, and maintain systems effectively.  Figure 1 provides an overview of the layers, paths, phases, and milestones (shown as “MS” in Figure 1) within the ELC Framework.

Figure 1:  ELC Framework

Figure 1 was removed due to its size.  To see Figure 1, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

 

ELC Layers

The ELC is a framework for organizing and using IRS directives, processes, procedures, templates, and standards to accomplish business change.  It is organized as a set of 6 interacting layers.

·         The Management Layer specifies how to plan and control business change programs, projects, acquisitions, and solutions throughout the ELC.

·         The Governance Layer specifies additional controls imposed from outside the project or program.

·         The Solution Life Cycle Layer specifies what should be done, but not how to do it.

·         The Solution Layer manages the solution as it is produced, including providing standards for consistent solution specification and formal review of solution content.  This Layer provides control over work products that may be produced by multiple internal and external developers using differing methodologies.

·         The Methodology Layer details how to do the work and specifies a unique set of work products to be produced.  Specific methodologies are not part of the ELC Framework.

·         The Specialty Areas Layer provides additional guidance for areas of particular importance within the IRS.  These areas include Enterprise Integration, Test, and Evaluation;[56] Business Rules Harvesting[57] and Management; Transition Management;[58] Enterprise Architecture; Capital Planning and Investment Control;[59] Security and Privacy; Requirements Development and Management.

ELC Paths

A path specifies a unique “philosophy” or orientation for performing the work.  Although the ELC specifies a standard for the work required to produce and operate business change solutions, there are multiple ways to approach and accomplish the required work.  Paths are like alternate roads, each of which crosses different terrain, but all of which lead to the same destination.  The ELC provides five distinct paths or approaches to developing systems:

·         The Large Custom Path is for large projects.

·         The Small Custom Path is for small projects.

·         The Commercial-Off-the-Shelf Path is a commercial software-based approach.

·         The Joint Application Development/Rapid Application Development Path is a highly accelerated, prototyping-based approach for very small, standalone solutions or solution components.

·         The Iterative Custom Path is a hybrid approach that combines elements of the other approaches.

ELC Phases and Milestones

A phase is a broad segment of work encompassing activities of similar scope, nature, and detail and providing a natural breakpoint in the life cycle. Each phase begins with a kickoff meeting and ends with an executive management decision point (called a milestone) at which IRS executives make “go/no-go” decisions for continuation of a project.  Project funding decisions are often associated with milestones.

Figure 2:  ELC Phases and Milestones

Phase

General Nature

of Work

Concluding Milestone

Vision and Strategy/ Enterprise Architecture Phase

High-level direction setting.  This is the only phase for enterprise planning projects.

0

Project Initiation Phase

Startup of development projects.

1

Domain Architecture Phase

Specification of the operating concept, requirements, and structure of the solution.

2

Preliminary Design Phase

Preliminary design of all solution components.

3

Detailed Design Phase

Detailed design of solution components.

4A

System Development Phase

Coding, integration, testing, and certification of solutions.

4B

System Deployment Phase

Expanding availability of the solution to all target users.  This is usually the last phase for development projects.

5

Operations and Maintenance Phase

Ongoing management of operational systems.

System

Retirement

Source: The ELC Guide.

 

Appendix IX

 

Management’s Response to the Draft Report

 

The response was removed due to its size.  To see the response, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.



[1] A release is a specific edition of software.

[2] Form 1040EZ is the Income Tax Return for Single and Joint Filers With No Dependents.  The initial release of the CADE does not process Forms 1040EZ for joint filers.

[3] The PRIME contractor is the Computer Sciences Corporation, which heads an alliance of leading technology companies brought together to assist with the IRS’ efforts to modernize its computer systems and related information technology.

[4] The filing season is the period from January through mid-April when most individual income tax returns are filed.

[5] Unit testing ensures program modules perform in accordance with requirements.

[6] A release is a specific edition of software.

[7] The PRIME contractor is the Computer Sciences Corporation, which heads an alliance of leading technology companies brought together to assist with the IRS’ efforts to modernize its computer systems and related information technology.

[8] Form 1040EZ is the Income Tax Return for Single and Joint Filers With No Dependents.  The initial Release of the CADE does not process Forms 1040EZ for joint filers.

[9] The ELC establishes a set of repeatable processes and a system of reviews, checkpoints, and milestones that reduce the risks of systems development and ensure alignment with the overall business strategy.

[10] The filing season is the period from January through mid-April when most individual income tax returns are filed.

[11] The Office of Management and Budget Circular A-11 Exhibit 300, Capital Asset Plan and Business Case, dated June 2002, documents business cases used by agencies to request funds, monitor the progress of projects, and improve management decision making over expensive information technology investments.

[12] Unit testing ensures program modules perform in accordance with requirements.

[13] The Combined Systems Integration Test verifies the quality of the integrated system (i.e., verifies the system is integrated into other IRS systems properly and functions as required).

[14] MITRE is the IRS’ modernization strategic support contractor.

[15] Job Control Language is a means of communication between the programmer/analyst and the operating system.  It consists of control statements that introduce a job to the operating system, provide accounting information, direct the operating system, request hardware devices, and execute a job.

[16] The balancing and reconciliation of accumulated and calculated data within the CADE include three major components:  initialization (the transfer of taxpayer account data from existing IRS systems to the modernized CADE system); daily processing (the editing, processing, and record keeping of all transactions); and weekly processing (the external and internal balancing of all processing and transactions for the week).

[17] Application Qualification Testing focuses on verifying that business processes are supported as intended by the application; requirements allocated to the application are met; all tests are defined and executed; and all problems encountered are documented, corrected, and verified.

[18] Improvements in the Customer Account Data Engine Pilot Plan Need to Be Considered to Help Ensure the Pilot’s Success (Reference Number 2003-20-018, dated October 2002).

[19] Office of Management and Budget Circular A-127, Financial Management Systems, dated July 1993.

[20] The Custodial Accounting Project uses a data-warehousing approach for storing, analyzing, and reporting taxpayer accounts and collection information.

[21] GAO/AIMD-00-21.3.1, dated November 1999.

[22] A cost-plus-fixed-fee contract is a cost-reimbursement contract that provides for payment to the contractor of a negotiated fee that is fixed at the inception of the contract.  The fixed fee does not vary with actual cost but may be adjusted as a result of changes in the work to be performed under the contract.  This contract type permits contracting for efforts that might otherwise present too great a risk to contractors, but it provides the contractor only a minimum incentive to control costs.

[23] A cost-plus-incentive-fee contract is a cost-reimbursement contract that provides for an initially negotiated fee to be adjusted later by a formula based on the relationship of total allowable costs to total target costs.

[24] A cost-plus-award-fee contract is a cost-reimbursement contract that provides for a fee consisting of (a) a base amount (which may be zero) fixed at inception of the contract and (b) an award amount, based upon a judgmental evaluation by the Federal Government, sufficient to provide motivation for excellence in contract performance.

[25] A release is a specific edition of software.

[26] The PRIME contractor is the Computer Sciences Corporation, which heads an alliance of leading technology companies brought together to assist with the Internal Revenue Service’s efforts to modernize its computer systems and related information technology.

[27] The Centralized Authorization File contains information about the type of authorizations taxpayers have given their representatives for their tax returns.

[28] The Master Files are the IRS’ central and official repository of taxpayer information.

[29] The filing season is the period from January through mid-April when most individual income tax returns are filed.

[30] A release is a specific edition of software.

[31] CADE Release 1 (Operations and Maintenance) was initially referred to as “CADE Release 1, Milestone 5” in the BSM Expenditure Plan.

[32] CADE Release 1.2 was initially referred to as “CADE Filing Season 2005” in the BSM Expenditure Plan.  The IRS did not provide cost and schedule data for this release in subsequent Expenditure Plans.

[33] CADE Release 1.3.1 and 1.3.2 were initially combined and referred to as “CADE Filing Season 2006” in the BSM Expenditure Plan.

[34] The Business Leader engages with the modernization technology team to help determine how to best apply technology to improve service to taxpayers, support enforcement activities, and improve compliance.

[35] The Business Requirements Director is the point of accountability on behalf of the Business Leader for ensuring business requirements and process engineering are the central focus in making project decisions.

[36] The Integrated Financial System project replaces the Internal Revenue Service’s (IRS) current inventory of antiquated financial systems with a single integrated system that connects the agency’s accounting, performance, budgeting, and procurement functions.

[37] The e-Services project modernizes the way taxpayers transact and communicate with the IRS by creating and marketing easy-to-use electronic products and services.

[38] Tiger teams and subteams consisted of highly experienced professionals from the IRS, Computer Sciences Corporation, IBM, etc. The teams collaborated on findings and recommendations in many areas (e.g., architecture and system design, building and design of code, testing and deployment, and documentation and performance-tuning for the CADE).

[39] This position, under the BSM Program Director, is responsible for successful day-to-day project management and execution, working with the Business Requirements Director and Contractor Project Manager to ensure system requirements are aligned with business requirements.

[40] MITRE is the IRS’ modernization strategic support contractor.

[41] The PRIME contractor is the Computer Sciences Corporation, which heads an alliance of leading technology companies brought together to assist with the IRS’ efforts to modernize its computer systems and related information technology.

[42] See Appendix VIII for an overview of the ELC and its various milestones.

[43] Systems integration testing ensures all system components (hardware and software) are working correctly and collectively with other related or dependant systems.  Systems acceptance testing determines whether a system meets user and contract requirements and objectives.

[44] The Joint Engineering Team was formed as a means to organize the senior engineers assigned to the CADE.

[45] The ELC will establish a new milestone called “Milestone 4A.”  In this Milestone, time and effort will be devoted to ensure business requirements are fully developed and the technical infrastructure is fully detailed.  This detail will significantly increase the opportunities for the use of fixed-price contracts.

[46] The PRIME contractor is the Computer Sciences Corporation, which heads an alliance of leading technology companies brought together to assist with the IRS’ efforts to modernize its computer systems and related information technology.

[47] The ITAMS delivers an inventory system that enables tracking, reporting, and management of information technology assets.

[48] ClearQuest® is a defect and change tracking system that captures and manages all types of change requests throughout the development life cycle, helping organizations quickly deliver higher quality software.

[49] The Enterprise Life Cycle establishes a set of repeatable processes and a system of reviews, checkpoints, and milestones that reduce the risks of systems development and ensure alignment with the overall business strategy.  See Appendix VIII for an overview of the Enterprise Life Cycle.

[50] The BSMO subsequently designated June 30 as its target for deploying the first subrelease of biannual CADE Releases.

[51] The filing season is the period from January through mid-April when most individual income tax returns are filed.

[52] MITRE is the IRS’ modernization strategic support contractor.

[53] IRS Computing Centers support tax processing and information management through a data processing and telecommunications infrastructure.

[54] The Master Files are the IRS’ central and official repository of taxpayer information.

[55] A release is a specific edition of software.

[56] Enterprise Integration, Test, and Evaluation includes processes for integrating multiple components of a solution and conducting various types and levels of testing on the solution.

[57] A business rule is a statement that defines or constrains some aspect of the business.  Harvesting is a general term used to broadly describe the entire set of activities involved in gathering, formalizing, analyzing, and validating business rules for a particular scope.

[58] Transition management helps ensure personnel and organizations are prepared to receive, use, operate, and maintain the business processes and technology provided by business change solutions.

[59] The Capital Planning Investment and Control process manages a central portfolio of information technology investments across the IRS.