TREASURY
INSPECTOR GENERAL FOR TAX ADMINISTRATION
Focusing Management Efforts
on Long-Term Project Needs Will Help Development of the Customer Account Data
Engine Project
June 2006
Reference Number: 2006-20-076
This
report has cleared the Treasury Inspector General for Tax Administration
disclosure review process and information determined to be restricted from
public release has been redacted from this document.
Phone Number |
202-927-7037
Email Address | Bonnie.Heald@tigta.treas.gov
Web Site |
http://www.tigta.gov
June 28, 2006
MEMORANDUM FOR CHIEF INFORMATION OFFICER
FROM: Michael R. Phillips /s/ Michael R. Phillips
Deputy Inspector General for Audit
SUBJECT: Final Audit Report – Focusing Management Efforts on Long-Term Project Needs Will Help Development of the Customer Account Data Engine Project (Audit # 200520012)
This report presents the results of our review of the Customer Account Data Engine (CADE) project management and deployment activities. The overall objective of this review was to determine whether the CADE project management and development activities were effective in assuring CADE releases[1] included all planned capabilities and were implemented within cost and schedule estimates. This review was part of the Treasury Inspector General for Tax Administration’s Fiscal Year 2005 Information Systems Programs audit plan for reviews of the Internal Revenue Service’s (IRS) modernization efforts.
Synopsis
The CADE project is the foundation for managing taxpayer accounts in the IRS modernization plan. It consists of databases and related applications that will replace the IRS’ existing Master File processing systems. The Master Files are the IRS’ central and official repository of taxpayer information.
While the current release of the CADE is performing well, significant challenges await the next release (CADE Release 2), which plans to add several new tax forms and schedules.
In August 2004, the IRS delivered CADE Release 1.1, which
successfully processed refund and even-balance Forms
1040EZ[2] for single taxpayers with no pending tax issues. CADE Release 1.2 started processing
tax returns in January 2005. In January
2006, the IRS and the PRIME contractor[3]
successfully delivered the newest release of the CADE (Release 1.3.2) in time
for the 2006 Filing Season.[4] Release 1.3.2 targeted a larger taxpayer
population by including U.S. Individual Income Tax Returns (Forms 1040 and 1040A)
with no schedules, improving address change capabilities, and including last-minute
2006 Filing Season tax law changes. As
of February 24, 2006, the CADE had processed over
2.7 million tax returns and generated over $1.4 billion in tax refunds, compared to the 1.4 million tax returns
processed that generated $427 million tax refunds in of all of 2005. Although the CADE has experienced cost and schedule overruns in the past, the
2006 Filing Season goals were substantially met.
While the above results are commendable, the IRS has had to reduce the number of tax returns it planned to process during the 2006 Filing Season. The IRS and the PRIME contractor agreed to defer eight Release 1.3.2 requirements to help ensure the CADE would process tax returns in time for the 2006 Filing Season. This occurred because the PRIME contractor did not effectively and efficiently use computer language (coding practices). Also, the PRIME contractor’s unit testing[5] did not adequately identify programming defects.
The
CADE’s design for managing taxpayer accounts is not fully developed to provide
a foundation for the IRS’ current and planned modernized systems. In April 2005, the Business Systems
Modernization Office (BSMO) initiated the CADE Data Architecture and Analysis
Study to enable the project team to adopt a flexible release strategy by establishing
a long-term plan to organize and manage CADE project data. Also, the Modernization and Information
Technology Services organization is forming the Modernization Vision and Strategy
to establish project development priorities with the IRS business
operations. The Modernization Vision and
Strategy will provide a 5-year plan
for developing modernization projects and processes. Until
these efforts are completed, successful deployment of
a long-term CADE release strategy cannot move forward, and the
future releases may not be deployed when planned.
Adequate procedures were not in place to ensure all CADE release requirements are timely controlled. As of February 23, 2006, an interim process to control CADE Release 2 requirements was in development. The BSMO will use this process until a vendor able to develop a repository meeting the IRS’ needs is identified.
The BSMO implemented contracting guidance on April 30, 2004,
entitled “Enabling Fixed-Price Contracting for Business Systems Modernization
Task
Recommendations
The Chief Information Officer (CIO) should provide direction to help ensure progress for the development of current and future CADE releases. The CIO should ensure current and future CADE contracts provide an assurance to the IRS that the PRIME contractor performed adequate testing. The CIO should also ensure the CADE Data Architecture and Analysis Study and the Modernization and Information Technology Services organization’s new Modernization Vision and Strategy are expeditiously completed, so the CADE project can implement realistic requirements and development plans. The CIO should timely implement an interim process to control CADE Release 2 requirements, which are currently in development. Additionally, the CIO should ensure the BSMO and the IRS’ contracting officers work with the PRIME contractor to adequately define project requirements enabling implementation of fixed-price contracts.
Response
IRS management agreed with our recommendations. To implement the corrective actions, the CIO is requiring the PRIME contractor to notify the IRS of the completion of unit testing and will incorporate this requirement into the IRS’ application testing guidance. The Modernization and Information Technology Services organization will use CADE data models and the high‑level data warehousing strategy to develop a data access strategy and transition plan for future CADE release designs. Further, an interim repository has been developed within the CADE to track and control requirements until the implementation of the planned Enterprise Services repository. Finally, the Applications Development organization, the Enterprise Services organization, and the Office of Procurement will work to position the IRS with the ability to award fixed-price contracts. However, after full review, negotiations, and consideration, the IRS made a business decision to continue with the cost-plus-award-fee contract rather than a fixed-price contract for CADE Release 2. Management’s complete response to the draft report is included as Appendix IX.
Copies of this report
are also being sent to the IRS managers affected by the report recommendations. Please contact me at (202) 622-6510 if
you have questions or Margaret E. Begg, Assistant Inspector General for Audit (Information
Systems Programs), at (202) 622-8510.
The Customer Account
Data Engine Project Is Making Progress
Appendices
Appendix
I – Detailed Objective, Scope, and Methodology
Appendix
II – Major Contributors to This Report
Appendix III – Report Distribution List
Appendix IV
– Customer Account Data Engine Release Schedules
Appendix V – Customer
Account Data Engine Project Costs
Appendix VII
– Customer Account Data Engine Corrective Actions to Prior Recommendations
Appendix
VIII – Enterprise Life Cycle Overview
Appendix
IX– Management’s Response to the Draft Report
The Customer Account Data Engine (CADE) project is the foundation for managing taxpayer accounts in the Internal Revenue Service (IRS) modernization plan. It consists of databases and related applications that will replace the IRS’ existing Master File processing systems. The Master Files are the IRS’ central and official repository of taxpayer information.
The IRS initiated the CADE project in September 1999. CADE Release 1[6] was initially planned for delivery in January 2002. The IRS and the PRIME contractor[7] postponed the planned delivery date of Release 1 several times. Setbacks occurred for several reasons, including IRS/PRIME contractor role confusion, noncompliance with management processes, and the absence of consistent, well-understood requirements. Appendix IV, Table 1, provides a timeline of the adjustments made to the content and planned delivery dates for the CADE release schedule.
In August 2004, the IRS delivered CADE Release 1.1, which successfully processed refund and even-balance Forms 1040EZ[8] for single taxpayers with no pending tax issues. CADE Release 1.2 started processing tax returns in January 2005. From January 1 through December 31, 2005, the CADE processed over 1.4 million tax returns and issued 927,044 paper and 478,120 electronic tax return refunds totaling over $427 million.
The Joint
Engineering Team was tasked to confirm the CADE architecture and design were
correctly implemented and tested throughout the life cycle.
The IRS and the
PRIME contractor created the CADE Joint Engineering Team consisting of business
architects and engineers. In the fall of 2004, the Joint Engineering Team recommended the formation of the CADE Data
Architecture and Analysis Study to establish a data architecture foundation
that would enable the project to adopt a flexible release strategy for current
and future CADE releases. The study
began in April 2005 and is on track for completion in June 2006.
Each year the IRS
presents to Congress an expenditure plan containing the best estimates
regarding the current scope, schedule, and cost of the Business Systems
Modernization (BSM) program. It
introduces key activities for the upcoming fiscal year and incorporates the
most current information concerning the status of the modernization projects. The expenditure plan details the successes from
prior years and discusses the key activities and steps the Business Systems Modernization
Office (BSMO) is taking to strengthen and improve the efficiencies and
effectiveness of its program.
After reviewing the IRS’ scope, schedule, and cost estimates contained
in the latest BSM 2005 and 2006 Expenditure Plans, Congress authorized $54
million in 2005 and $60 million in 2006 for the CADE. Additionally, the IRS requested $85 million
in 2007 for the CADE, but this amount has been reduced to about $58
million. Through October 27, 2005, the
IRS had obligated $134,168,206 for the design, development, and implementation
of the CADE, with an estimated $252.1 million planned to be spent through
2007. Appendix V presents an analysis of
the CADE project release costs.
This review is the fifth in a series of reviews related to the CADE project activities and was performed at the BSMO facilities in New Carrollton, Maryland, during the period August 2005 through February 2006. The audit was conducted in accordance with Government Auditing Standards. Detailed information on our audit objective, scope, and methodology is presented in Appendix I. Major contributors to the report are listed in Appendix II. Appendix VIII presents an overview of the Enterprise Life Cycle (ELC).[9]
Areas of improvement continue to be identified and actively addressed by CADE project management.
While the CADE project has experienced a series of setbacks and postponements since its beginning in September 1999, the project is making progress. Specifically, program-level and project-level areas of improvement continue to be identified and actively addressed by project management. The latest version of the CADE (Release 1.3.2) successfully went into production in time for the 2006 Filing Season.[10]
Actions to address business challenges were beneficial to the CADE
project
In 2003, the IRS launched a comprehensive review consisting of three studies and a benchmarking analysis of the BSM program. In addition, the PRIME contractor launched an internal study. The studies confirmed the IRS modernization effort is a massive, highly complex, high-risk program confronted by a number of critical management and technological challenges. To address the various assessments, the IRS developed 48 business challenges for itself and the PRIME contractor to complete. The BSM Challenges Plan Close-Out Report, dated May 1, 2004, provided plans for improved future CADE project development.
We followed up on the 10 business
challenges related to the CADE project and determined that each had been
adequately addressed. For example, the
Joint Engineering Team, which recommended
the formation of the CADE Data Architecture and Analysis Study, is a direct
result of addressing an identified business challenge. Other actions to address business challenges included
hiring critical engineering talent and hiring an overall CADE director. Appendix VI presents descriptions of
the 10 business challenges related to the CADE project.
CADE management took corrective actions to prior Treasury Inspector General for Tax Administration recommendations
We followed up on the status of corrective
actions taken in response to 13 prior recommendations we made related to the
CADE project and determined only 3 corrective actions remained open. BSMO management implemented or has scheduled
corrective action implementation to address all issues. Appendix VII presents an analysis of the
corrective actions to our recommendations.
The IRS and PRIME contractor identified
lessons learned to improve future project development activities
In October 2005, the PRIME contractor issued the IRS a lessons-learned workshop report outlining processes that worked well and processes needing improvement during the CADE project’s 2005 development and testing activities. The lessons-learned workshop report was a joint effort between the IRS and the PRIME contractor.
The report identified 14 processes that worked well and 42 processes that needed improvement. Items that worked well included aspects of configuration management, transition management, release development, and combined systems integration testing. Areas that needed improvement also focused on aspects of configuration management and testing, as well as estimating and scheduling the assigned workforce, developing computer operating handbooks, improving communication between architectural design and engineering groups, and improving project schedule management. Several items specifically address our prior recommendations, which are discussed later in this report.
CADE Release 1.3 went into production in time for the 2006 Filing Season
Beginning with the development of CADE Release 1.3, the IRS and the PRIME contractor planned to use a biannual release approach - one release delivery in June and one release delivery in January. Appendix IV, Table 2, presents the revised CADE biannual release schedule including return types, account characteristics, and estimated tax return volumes for each release. The first biannual release (Release 1.3.1) was delivered in September 2005. Release 1.3.1 processed Form 1040EZ tax returns, improved long-term database efficiency, accepted tax returns from taxpayers who changed addresses between filing seasons, implemented interest rate changes, and handled initial 2006 Filing Season tax law changes.
While the current release of the CADE is performing very well, significant challenges await the next release (CADE Release 2), which plans to add several new tax forms and schedules.
In January 2006, the IRS and the PRIME contractor
successfully delivered the second biannual release (Release 1.3.2) in time for
the 2006 Filing Season. Release 1.3.2
targeted a larger taxpayer population by including Forms 1040 and 1040A with no
schedules, improving address change capabilities, and including last-minute 2006
Filing Season tax law changes. As of
February 24, 2006, the CADE had processed over 2.7 million tax returns and generated
over $1.4 billion in tax refunds,
compared to the 1.4 million tax returns processed that generated $427 million tax
refunds in of all of 2005.
Although the CADE has
experienced cost and schedule overruns in the past, the 2006 Filing Season
goals were substantially met. Appendices
IV and V, Table 3, present the CADE project cost and schedule variances.
While the above results are commendable, the IRS has had to reduce the
number of tax returns it plans to process in 2006. The latest approved CADE Capital Asset and
Business Case Exhibit 300[11] (dated February
2005) estimated Release 1.3.2 would identify 20 million tax returns with
characteristics for CADE processing.
However, after reconsidering the original projection, the deferral of 8
planned requirements, and the addition of unexpected disaster-related
requirements, Release 1.3.2 identified fewer than 14 million tax returns for potential
CADE processing. In October 2005, the
BSMO estimated that only 4 million of the 14 million tax returns having CADE
processing potential would be processed by the CADE. This reduction is due to account activity
which the current CADE Release cannot process.
Had Release 1.3.2 realized the 20 million tax returns envisioned in the
Capital Asset and Business Case Exhibit 300, even more taxpayers would have
received CADE benefits such as faster refunds.
On October 27, 2005, the IRS and the PRIME contractor
renegotiated the Release 1.3.2 task order and deferred eight requirements to
future CADE releases. The task order
modification contained a cost adjustment reduction of almost $3.2 million to account
for the removal of the eight requirements.
The deferred requirements involved the following capabilities:
The IRS and the PRIME contractor agreed to defer the eight Release 1.3.2 requirements to help ensure the CADE would be operational in time for the 2006 Filing Season. Several factors contributed to the removal of the eight requirements from Release 1.3.2.
The Internal Revenue Manual and the ELC provide guidance for developing and documenting project requirements and computer program applications. The Internal Revenue Manual also provides guidance to plan, execute, and develop several forms of computer program testing, including unit testing and systems acceptance testing.
During development of CADE Release 1.3.1, the PRIME contractor did not effectively and efficiently use computer language (coding practices) and the contractor’s unit testing[12] did not adequately identify programming defects. This is evidenced by the Release 1.3.1 Combined Systems Integration Testing[13] which experienced a high number of failed tests. Between June and August 2005, the defect rate averaged over 20 percent. MITRE[14] provided information on defect rates to measure productivity improvements claimed by the PRIME contractor during Release 1.3.2. MITRE’s research on defect rates reinforced its observations that the PRIME contractor’s productivity is well below industry standards. Subsequently, the PRIME contractor did not have adequate staffing to fix the Release 1.3.1 defects, requiring the transfer of designers and developers from Release 1.3.2 startup activities.
The PRIME contractor assessed its performance in developing CADE Release 1.3.1 and compiled lessons learned from its experience. The PRIME contractor’s lessons-learned report contained the following recommendations that we had previously reported:
Because of the problems experienced in development, Release 1.3.1 was delivered in September 2005, 2 months behind schedule. This delay significantly shortened the Release 1.3.2 project schedule, which was time sensitive to the 2006 Filing Season. Because of the abbreviated development period available for Release 1.3.2, the eight requirements cited previously were removed. The IRS did not estimate the number of taxpayers affected in making its decision to defer the eight requirements to future CADE releases. As a result, an unknown number of taxpayers will not receive the benefits associated with CADE processing, primarily faster processing of income tax return refunds. In addition, Release 1.3.2 taxpayer account balancing, control, and reconciliation process[16] requirements were not finalized until December 9, 2005. Also, the IRS did not fully develop the requirement to process refund suspensions meeting Criminal Investigation function fraud criteria in time for inclusion in Release 1.3.2.
Release 1.3.2 went into operation on January 17, 2006. However, all planned Release 1.3.2 requirements were not deployed initially because of the abbreviated development period. The remaining CADE release requirements were incrementally deployed as their testing was completed.
The impact of the application development problems has had a cascading effect. The CADE Release 2.1 project team has had to compete with the Release 1.3.2 project team for development resources during startup activities. This shortfall evolved from the resources the Release 1.3.2 project team had to divert to make Release 1.3.1 operational.
Recommendation
Recommendation 1: The Chief Information Officer should ensure current and future CADE task order contracts and modifications provide an assurance to the IRS that unit testing was adequately performed by the PRIME contractor and the application provided is ready for integration and testing by the IRS. This assurance should take the form of something similar to an Application Qualification Testing[17] report or other deliverable. The requirement for this assurance should be incorporated into the Modernization and Information Technology Services (MITS) organization’s testing guidance, which is being revised by the Technology Release Management office.
Management’s Response: The IRS agreed with our recommendation. Prior to IRS testing, the PRIME contractor will be required to provide the IRS with notification regarding the completion of unit testing. The CADE has placed this requirement of formal notification regarding the completion of unit testing in the Release 2 task order and will do so in future task orders. Further, the MITS organization’s Test Process Improvement Working Group of Stakeholders will review existing test guidance and make the appropriate modification to ensure that unit testing was adequately performed by the PRIME contractor and the application provided to the IRS is ready for integration and testing.
The CADE’s design for managing taxpayer accounts is not fully developed to provide a foundation for the IRS’ current and planned modernized systems. In a November 2002 briefing to IRS executives, the PRIME contractor stated it had developed the basic infrastructure for CADE Releases 1 through 5 and that 95 percent of the applications for operating Release 1 had been built. This declaration may have led to a belief that, while Release 1 was experiencing difficulties, the underlying design and infrastructure was almost finished.
Since then, the CADE project has developed Releases 1.1 through 1.3.2. However the IRS and the PRIME contractor have modified the release strategies. Appendix IV, Table 2, presents the latest release schedules. In April 2005, the BSMO initiated the CADE Data Architecture and Analysis Study, which should be completed in June 2006. This Study’s overall objective is to enable the project team to adopt a flexible release strategy by establishing a long-term plan to organize and manage project data. To successfully deliver future releases, the project team needs to address the following topics in the CADE Data Architecture and Analysis Study.
The current CADE logical design release strategy supports only the
first two CADE releases
The logical design provides a specific description
of the hardware, system software, and network components comprising the IRS’
systems and applications. The logical
design presents a view of the technology being used, including definitions of
workstations, hardware components, and interfaces with other IRS systems and
applications.
The original CADE release strategy was
focused on moving ‘easy’ taxpayers onto the CADE in the early releases and
building on this base with more complexity through the later releases. With this focus, the project’s logical design
did not include plans to support the complex tax returns the CADE would process
subsequent to Release 2.
Until a long-term logical design is developed, future releases of the CADE cannot begin development. Additionally, on February 23, 2006, the project team stated an interim solution for the Release 2.1 logical application design had not been completed.
The CADE project has not completed its long-term physical design
The physical
design describes the IRS’ systems and applications in physical terms. It specifies vendors, products, models,
versions, releases, capacities, connections, and other options for the planned
project. The purpose of the physical
design is to guide the construction and ultimate integration of the IRS’
systems and applications after its logical design is complete.
Because the logical design is in development and a long-term physical design is dependent on the logical design’s completion, the CADE project is building an interim physical design. The interim physical design solution is another area of concern affecting the next CADE Release. As of February 23, 2006, the interim physical design work for Release 2.1 was several weeks behind schedule. Until a long-term physical design is developed, testing cannot be performed that will ensure the adequacy of CADE release development.
The IRS did not use the originally selected data conversion plan because of time constraints related to acquisition, evaluation, and implementation
The objective of the data conversion is to convert
data between the IRS’ current processing environment formats and the CADE
database while minimizing the effect on IRS employees and customers. Taxpayer account information maintained in
the CADE is available to users in the current processing environment
format. Data conversion consists of:
In 2002, we reviewed
the IRS’ data conversion plans and reported the plans adequately documented the
process necessary to move taxpayer account data to and from existing IRS
systems and the modernized CADE.[18] We
cautioned that the key to success was to carefully track the data conversion
progress against the project development plans and timely address any potential
barriers or risks.
The IRS subsequently abandoned the original data conversion plan
because of time constraints related to acquisition, evaluation, and
implementation. As a result, a process termed
“quick and dirty” by the IRS was used for the first CADE Release. The data conversion programs for processing
Release 2 taxpayer accounts have not yet been completed.
As part of the CADE
Data Architecture and Analysis Study, data conversion recommendations in May
and October 2005 proposed several solutions for the long term (CADE Release 3
and beyond). A definitive approach is
still to be determined. As the CADE
project progresses, the data conversion process will become more complex and
riskier with the addition of returns, schedules, and taxpayers.
The CADE currently does not store historical financial data
The Office of Management and Budget
prescribes[19] policies and standards for executive
departments and agencies to follow when developing, operating, evaluating, and
reporting on financial management systems. Several years ago, the IRS initiated a
change request to eliminate the requirement to store historical financial data
in the CADE for Release 1. The change
request cited overall complexity, and the Release 1 capabilities did not
require the storage of financial history.
This capability was deferred because financial information for tax
returns processed by CADE Release 1 is captured in the current processing environment.
For the 2007 Filing Season, several Form
1040 schedules will be added to CADE
Release 2 requiring the storage of historical data. Examples include Interest and Ordinary Dividends (Schedule B), Profit or Loss From
Business (Schedule C), Supplemental Income and Loss (Schedule E), and Profit or Loss From Farming (Schedule F).
The CADE Data Architecture and Analysis Study team is examining the use of the Custodial Accounting Project[20] information as a means of populating the CADE’s historical financial database. The historical data conversion will be a significant undertaking for the CADE if the financial information for tax returns in the current processing environment needs to be completely rewritten.
The CADE Data Architecture and Analysis
Study is integral to the IRS’ future modernization plans
The
MITS organization is analyzing the progress of the BSM program and
reconsidering the approach to the program’s architecture and development. To provide a better focus, the MITS
organization is forming the Modernization Vision and Strategy to establish project
development priorities with the IRS business operations. The Modernization Vision and Strategy will
provide a 5-year plan
for developing modernization projects and processes.
The CADE project is integral to
the future plans for modernization, and the recommendations resulting from the CADE
Data Architecture and Analysis Study
will help determine the future plans for developing the CADE project. Until the issues
cited previously are resolved, successful deployment of a long-term CADE
release strategy cannot move forward, and the future releases may
not be deployed when planned. For the near term, the risk of untimely
implementation of CADE Release 2 is significantly affected by unresolved
solutions around the project’s logical and physical designs, the data
conversion plans, and the storage of financial information.
Recommendation
Recommendation 2: The Chief Information Officer should ensure the CADE Data Architecture and Analysis Study and the MITS organization’s new Modernization Vision and Strategy are expeditiously completed, so the CADE project can implement realistic requirements and development plans.
Management’s Response: The IRS agreed with our recommendation. The MITS organization will use the CADE Data Study results as the basis for developing data models to guide designs for CADE Releases 3 and 4. The MITS organization will also review Modernization and Vision project strategies for extracting operational and analytical data to develop a high-level strategy for data warehousing. Building upon the CADE data models and the high-level data warehousing strategy, the MITS organization will form a data access strategy and transition plan.
The GAO Standards state transactions should be promptly recorded to maintain their relevance and value to management in controlling operations and making decisions.
The ELC’s BSMO Process Description for requirements development and management designates the Acquisition Project Manager as responsible for ensuring project requirements are defined, testable, and traceable. The requirements for CADE Releases 1.1 and 1.2 are contained in the System Requirements Reports, which were updated to reflect changes as they occurred. For CADE Release 1.3, the IRS and the PRIME contractor agreed to create separate documents called “White Papers” to document development of each specific requirement. This process change was made to ensure a better understanding of the performance needed to meet the project requirements and to minimize rework during testing.
Adequate procedures were not in place to ensure all CADE release requirements are timely controlled. The project team uses the Document Management System as a database to control the White Papers. When we requested a complete listing of Release 1.3 requirements, the project team was unable to timely provide the information. The project team and the PRIME contractor needed over 3 months to assemble and provide us all information relating to the population of White Papers.
The Government Accountability Office (GAO) Standards for Internal Control in the Federal Government[21] state control activities occur at all levels and functions of the entity and include the creation and maintenance of related records that provide evidence of execution of these activities as well as appropriate documentation. Because CADE Release 1.3 requirements are not readily available for project team use, the IRS cannot easily determine whether all agreed-to requirements were developed and tested.
BSMO management stated Release 2 will not use White Papers to document specific requirements, as was the case in Release 1.3. As of February 23, 2006, an interim process to control Release 2 requirements was in development. The BSMO will use this process until a vendor able to develop a repository meeting the IRS’ needs is identified. Once implemented, the new repository will be managed by the MITS organization Requirements Management Office.
Recommendation
Recommendation 3: Until a new repository is identified and built to control future CADE release requirements, the Chief Information Officer should timely implement an interim process that will effectively and efficiently control CADE Release 2 requirements, which are currently in development.
Management’s Response: The IRS agreed with our recommendation. As an interim measure, a project repository has been developed within the CADE to track and control requirements for each release. This vehicle will remain in place until the implementation of the planned Enterprise Services organization’s repository is completed.
The IRS is not using fixed-price contracts for CADE Release 1.3 development or for Release 2 design work performed by the PRIME contractor. Both releases are using hybrid cost-plus-fixed-fee,[22] cost-plus-incentive-fee,[23] and cost-plus-award-fee contracts.[24]
The BSMO implemented
contracting guidance on April 30, 2004, entitled “Enabling Fixed-Price
Contracting for Business Systems Modernization Task
The BSMO’s guidance acknowledges that fixed-price contracts are used when the risk involved can be predicted with acceptable certainty. A fixed-price contract is suitable for acquiring supplies or services on the basis of “reasonably definite functional or detailed specifications” when the contracting officer can establish fair and reasonable prices at the outset, performance uncertainties can be identified and reasonable estimates of their cost impact can be made, and the contractor is willing to accept a fixed price representing assumption of the risks involved.
The IRS recognized the following challenges in working with the PRIME contractor to use fixed-price contracts when they reach Milestone 4B:
The use of fixed-price contracts should lead to more cost-effective acquisitions, better value, and greater competition. Fixed-price contracts have the net effect of shifting some of the manageable performance risk from the IRS to the contractors. When the contractor does not bear enough risk under a task order, the contractor does not have an incentive to perform well. Contractors will be given more latitude for determining methods of performance, with more responsibility for performance quality. The IRS should experience fewer cost overruns, schedule delays, and performance problems. These benefits of fixed-price contracting will be realized only when the CADE project requirements are fully developed and agreed to, comprehensive estimates of effort to perform the contract tasks are made, and meaningful negotiations are held.
Recommendation
Recommendation 4: The Chief Information Officer should ensure the BSMO, the Enterprise Services organization’s Business Rules and Requirements Management office, and the IRS’ contracting officers work with the PRIME contractor to adequately define project requirements enabling implementation of fixed-price contracts when project development reaches Milestone 4B. Hybrid fixed-price contracts incorporating cost-plus provisions should be used only to contract the development of additional unanticipated requirements, such as new tax law changes. Additionally, the BSMO and the IRS’ contracting officers should reevaluate the benefits of using a fixed-price contract, if possible, for the remaining work (Milestone 4B) on CADE Release 2.
Management’s Response: The IRS agreed with this recommendation. The Acquisition Development organization, the Enterprise Services organization, and the Office of Procurement will work to position the IRS with the ability to award fixed-price contracts. CADE project personnel, in conjunction with the IRS contracting officers, reevaluated the benefits of using a fixed-price contract for the remaining work on Release 2. After full review, negotiations, and consideration, the IRS made a business decision to continue with the cost-plus-award-fee contract.
Appendix I
Detailed Objective,
Scope, and Methodology
The overall objective of this review was to determine whether the Customer Account Data Engine (CADE) project management and development activities were effective in assuring CADE releases[25] included all planned capabilities and were implemented within cost and schedule estimates. This review was part of our Fiscal Year 2005 audit plan for reviews of the Internal Revenue Service’s Business Systems Modernization (BSM) efforts.
To accomplish our objective, we reviewed available documentation and interviewed Business Systems Modernization Office (BSMO) personnel and Modernization Acquisition function analysts from the Agency-Wide Shared Services organization. Specifically, we:
I. Assessed the activities of the BSMO and the PRIME contractor[26] to deliver all agreed-to requirements for CADE Releases 1.3 and 2.
A. Interviewed BSMO staff to determine the final requirements the PRIME contractor delivered for CADE Releases 1.3.1 and 1.3.2.
B. Interviewed BSMO staff to determine the effect the month-by-month funding process had on CADE Releases 1.3.1 and 1.3.2.
C. Interviewed BSMO staff to determine the status of the testing schedule activities related to CADE Releases 1.3.1 and 1.3.2.
D. Reviewed CADE Release 2 planning activities to determine whether the BSMO and the PRIME contractor were on track to successfully deliver additional CADE taxpayer benefits.
E. Monitored BSMO and PRIME contractor actions related to the CADE Data Architecture and Analysis Study to determine the effect, if any, on CADE Releases 1.3.1, 1.3.2, and 2 activities.
II. Determined whether the actions taken to address the BSM Challenge Issues have improved the management of the CADE project to successfully move forward.
III. Followed up on the status of corrective actions taken in response to prior Treasury Inspector General for Tax Administration audit report recommendations.
Appendix II
Major Contributors
to This Report
Margaret E. Begg, Assistant Inspector General for Audit (Information Systems Programs)
Gary V. Hinkle, Director
Edward A. Neuwirth, Audit Manager
Phung-Son
Nguyen, Senior Auditor
Bruce Polidori, Senior Auditor
Steven
Gibson, Auditor
Kim
McManis, Auditor
Appendix III
Commissioner C
Office of the
Commissioner – Attn: Chief of Staff C
Deputy Commissioner for Operations Support OS
Associate Chief Information Officer, Applications Development OS:CIO:B
Associate
Chief Information Officer,
Deputy Associate Chief Information Officer, Applications Development OS:CIO:AD
Director, Procurement OS:A:P
Director, Stakeholder Management OS:CIO:SM
Deputy Associate Chief Information Officer, Business Integration OS:CIO:ES:BI
Deputy Associate Chief Information Officer, Systems Integration OS:CIO:ES:SI
Director, Test, Assurance, and Documentation OS:CIO:AD:TAD
Chief Counsel
CC
National Taxpayer Advocate TA
Director, Office of Legislative Affairs CL:LA
Director, Office of Program Evaluation and
Risk Analysis RAS:O
Office of Management Controls OS:CFO:AR:M
Audit Liaisons:
Associate Chief Information Officer, Applications Development OS:CIO:B
Director,
Program Oversight OS:CIO:SM:
Appendix IV
Customer Account Data Engine Release
Schedules
Table 1 presents
the historical development of the Customer Account Data Engine (CADE) and the
original CADE release schedule. The
names of the tax forms planned for CADE processing are presented below Table 1. All tax forms and schedules listed in Tables
1 and 2 are available at the Internal Revenue Service (IRS) web site (IRS.gov).
Table 1: Original CADE Release
Schedule
|
|
RELEASE
|
RELEASE
|
RELEASE
|
RELEASE
|
RELEASE
|
RELEASE
3 |
RELEASE
|
RELEASE
5 |
|---|---|---|---|---|---|---|---|---|
|
Tax Return Types |
Form 1040EZ; Single
filing status; refund or even-balance
returns |
Release 1.1/1.2 plus
address change |
Release 1.3.1 plus |
Form 1040EZ; Form 1040
Schedules A, B, and R; Form 1040A Schedules 1
and 3 |
Release 2.1 plus |
All Form 1040 family
and supporting forms without an EIN; deceased refund, or fully paid
returns |
All Form 1040 family
and supporting forms with an EIN; Forms 941, 940, and 720;
payroll, unemployment, and excise returns for Form 1040 taxpayers; refund, fully paid, |
All remaining
individual tax returns |
|
Filing Status |
Single |
Single |
Single |
Single; Married; Head of Household, limited dependents |
Single; Married; Head of Household, limited dependents |
All (including Head of
Household) |
All |
All |
|
Account Characteristics |
No account issues (open
or closed) |
No account issues (open
or closed) |
No account issues (open
or closed) |
No account issues (open
or closed) |
Married once; no open
account issues |
No open account issues;
Earned Income Tax Credit (EITC) |
Power of Attorney;
Centralized Authorization File;[27] no open account issues |
All accounts not
included in previous Releases |
|
|
RELEASE |
RELEASE |
RELEASE |
RELEASE |
RELEASE |
RELEASE 3 |
RELEASE 4 |
RELEASE 5 |
|||
|---|---|---|---|---|---|---|---|---|---|---|---|
|
Number of Returns Original
estimate Revised Estimate - February 2005 |
3 million 2 million |
2 million |
To Be Determined
(TBD) |
TBD |
35 Million 33 million |
76 Million 50 Million |
110
Million 80 Million |
122
Million 140
Million |
|||
|
Estimated Delivery: As of April 2000
As of January 2004 As of December 2004
As of February 2005 |
January 2002 August 2004 August 2004 August 2004/ January 2005 |
July 2005 July 2005 |
January 2006 January 2006 January 2006 |
August 2002 TBD July 2006 July 2006 |
TBD January 2007 January 2007 |
July 2003 TBD TBD Releases
3.1/3.2 |
July 2004 TBD TBD Releases
4.1/4.2 |
July 2005 TBD TBD July 2009 |
|||
Source: CADE
Individual Master File[28]
Release Content Master Plan Update, dated June 5, 2003.
Form 720 – Quarterly Federal Excise Tax Return.
Form 940 – Employer’s Annual Federal Unemployment (FUTA) Return.
Form 941 – Employer’s QUARTERLY Federal Tax Return.
Form 1040 –
Form 1040A –
Form 1040EZ – Income Tax Return for Single and Joint Filers With No
Dependents.
Form 2688 – Application for Additional Extension of Time to File
Form 4868 – Application for Automatic Extension of Time to File
Table 2
presents the current CADE release schedule.
The names of additional tax forms planned for CADE processing are
presented below Table 2.
Table
2: Revised Biannual CADE Release
Schedule
|
|
2004 RELEASE 1.1 |
2005 RELEASE 1.2 |
2006 RELEASE 1.3.2 |
2007 RELEASE 2.2 |
2008 RELEASE 3.2 |
2009 RELEASE 4.2 |
2010 RELEASE 5.2 |
2011/2012 RELEASE 6.2 |
|---|---|---|---|---|---|---|---|---|
|
Tax Return Types |
Form 1040EZ |
1.2 &
1.3.1 (1.3.1
only) |
1.3.2 |
2.2 |
3.2 Form 1040 decedent returns; Form 1040 fully paid with remittance; Form
1040 with credit elect; |
4.2 Balance-due returns; math error returns;
|
5.2 Form
1040 Schedules C, E, F with an EIN and supporting schedules including Schedule
SE; Forms 940 and 720 payroll, unemployment, and excise tax returns for Form 1040
|
6.2 Additional Form 1040 schedules and forms
(TBD) 7.2 Form 1040NR; |
|
Filing Status |
Single (never married); no dependents |
Single (never married); no dependents |
2.1 Single, Married (joint), Head of
Household, Married (separate) |
3.1 Single, Married (joint), Head of
Household, Married (separate), Surviving Spouse |
4.1 Single, Married (joint), Head of
Household, Married (separate), Surviving Spouse |
5.1 Single, Married (joint), Head of
Household, Married (separate), Surviving Spouse |
6.1 Single, Married (joint), Head of
Household, Married (separate), Surviving Spouse |
7.1 Single, Married (joint), Head of
Household, Married (separate), Surviving Spouse |
|
January and |
2004 RELEASE
1.1 |
2005 RELEASE
1.2 |
2006 RELEASE
1.3.2 |
2007 RELEASE
2.2 |
2008 RELEASE
3.2 |
2009 RELEASE
4.2 |
2010 RELEASE
5.2 |
2011/2012 RELEASE
6.2 |
|---|---|---|---|---|---|---|---|---|
|
Account Characteristics |
Refund or even-balance returns; No
account issues (open or closed) |
Refund or even-balance returns; no
dependents; |
Refund or even-balance returns; no
dependents; |
2.2 EITC |
3.2 No open account issues. EITC no open account issues; EITC |
No open account issues EITC |
Power of Attorney; Centralized Authorization
File; |
All accounts not included in previous
releases |
|
Estimated |
Not applicable |
Actual 1,423,517 |
1.3.2 4 million |
2.2 33 million |
3.2 50 Million |
4.2 70 Million |
5.2 90 Million |
6.2 110 Million 7.2 |
Source: Customer
Relationship Management Executive Steering Committee, approved October 18, 2005.
Form 1040-ES – Estimated Tax for Individuals.
Form 1040NR –
Form 1040-NR-EZ
U.S. Income Tax Return for Certain Nonresident Aliens With No Dependents.
Form 1040-V – Payment Voucher.
Form 1040X – Amended
Form 6251 – Alternative Minimum Tax–Individuals.
Appendix V
Customer Account Data Engine Project Costs
Table 1: Customer Account
Data Engine (CADE) Project Task Order Costs Through Release 1.3
|
Task Order Number |
Period of Performance |
Cost |
|---|---|---|
|
0019 |
1999 – 2000
(specific dates not available) |
$2,166,234 |
|
0037 |
April 24, 2000 –
August 31, 2000 |
$3,011,000 |
|
0054 |
September 1, 2000 –
April 30, 2001 |
$13,971,165 |
|
0069 |
June 1, 2001 –
August 31, 2001 |
$5,700,000 |
|
0071 |
May 1, 2001 – June
30, 2001 |
$1,534,012 |
|
0073 |
September 1, 2001 –
August 31, 2004 |
$60,458,154 |
|
0123 |
October 1, 2004 – December 31, 2006 |
$47,327,641 |
|
|
|
Total $134,168,206 |
Source: CADE
contract task orders from the Internal Revenue Service Procurement Office.
Table 2: Estimated
CADE Project Costs and Schedule Through Release 1.3
|
Release 1 Project Phase |
Completed/Scheduled |
Cost/Estimate |
|---|---|---|
|
Milestone 1 |
December 31, 1999 |
$5,116,000 |
|
Milestones 2 and 3 |
June 30, 2001 |
$19,267,000 |
|
Milestone 4 |
July 30, 2004 |
$58,838,000 |
|
August 5, 2004 |
$24,550,000 |
|
|
2005 Filing Season (Release
1.2) |
December
31, 2004 |
$23,403,000 |
|
Milestone 5 |
June 30, 2005 |
$17,450,000 |
|
2006 Filing Season (Release
1.3) |
December 31, 2005 |
$28,300,000
|
|
|
|
Total $176,924,000 |
Source: Business
Systems Modernization (BSM) Expenditure Plans for 2004, 2005 and 2006.
Table 3: CADE Project Cost and Schedule Variance Summary
|
Release |
Milestone |
Cost Variance |
Cost Variance Percentage |
Schedule Variance |
Schedule |
|---|---|---|---|---|---|
|
1 |
Operations and Maintenance[31] |
$7,510,000 |
43% |
0 |
0% |
|
1.2 |
2005 |
$10,000 |
0% |
0 |
0% |
|
1.3.1 |
2006 Filing Season |
$0 |
0% |
2 |
20% |
|
1.3.2 |
2006 |
$0 |
0% |
0 |
0% |
|
2 |
4a |
In Progress |
|||
|
Program |
2005 |
$1,935,000 |
24% |
Not |
Not |
|
2006 |
In Progress |
||||
Source: BSM Expenditure Plans for 2004, 2005 and 2006
and the IRS.
Appendix VI
Business Systems Modernization Challenge
Issues Relating to the Customer Account Data Engine
The Business Systems Modernization
(BSM) Challenges Plan Close-Out Report,
dated
May 1, 2004, provided plans to improve future Customer Account Data Engine
(CADE) project development. The
CADE-related BSM Challenge Issues and related close-out plans included:
- Make business owner and program director accountable for project success.
The
Business Systems Modernization Office (BSMO) restructured its management and
governance structure to institutionalize the roles of Business Leader[34] (replacing Business Owner), Business
Requirements Director,[35] Program Director, and Program Manager.
- Implement short-duration, discrete “Tiger Teams” in the CADE, Integrated Financial System,[36] and e-Services[37] projects.
The
CADE Tiger Team[38] conducted a comprehensive analysis of
the CADE project and provided recommendations in a report about managing
defects, contracts, and program execution.
- Designate business architects for all projects to support business insight required throughout the project life.
The role of the Business Architects
(title changed to Business Requirements Director – see footnote 2) and other
program/project management positions were defined as part of the larger
governance structure reorganization.
-
Assign
business owners and architects to validate business unit participation and
accountability in projects.
Roles of the Business Leader, the Business Requirements Director, the BSM Program Director, the Acquisition Project Manager,[39] and the Information Technology Services Coordinator have been defined and socialized among the BSMO, the Modernization and Information Technology Services organization, and the business (operating division/function) executives.
-
Align
critical engineering talent to most critical projects.
Key engineering talent from MITRE,[40] the PRIME contractor,[41] and other appropriate resources were identified and hired.
-
Identify key
productivity and quality metrics across the life cycle based on industry
standards.
A joint Internal Revenue Service (IRS), PRIME contractor, and MITRE Tiger Team developed the PRIME Enterprise Life Cycle (ELC) – the key productivity and quality measures roll-out plan for effective implementation and institutionalization associated with ELC Milestone 4.[42]
-
Move
acceptance testing to be earlier in the life cycle.
The PRIME contractor’s
testing and deployment office collaborated with the IRS to develop a combined
Systems Integration Test/Systems Acceptance Test[43] for CADE Release 1.1.
-
Establish
independent architecture and engineering team of IRS and contractor business
architects and system engineers.
The IRS and the PRIME
contractor formed the Joint Engineering Team[44] consisting of business
architects and system engineers. The Joint Engineering Team
recommended the formation of the CADE Data Architecture and Analysis Study to establish
a data architecture foundation that would enable the CADE project to adopt a
flexible release strategy for current and future CADE releases
-
Clearly
define business requirements and tightly manage them to control scope.
The BSMO worked with the PRIME contractor and representatives from various process teams to define Milestone 4A[45] exit criteria, obtain buy-in from key stakeholders, and redefine ELC requirements and milestone exit criteria.
-
Identify “blockers”
on current contracting actions.
Blockers were identified on October 24, 2003, which centered on business integration, systems integration, and the PRIME contractor’s program management office activities.
Appendix
VII
Customer Account Data Engine Corrective
Actions to Prior Recommendations
Table 1 presents the
findings reported by the Treasury Inspector General for Tax Administration
(TIGTA) and the Internal Revenue Service’s (IRS) subsequent planned corrective
actions and implementation status. The
TIGTA report titles are presented below Table 1.
Table 1: Status of TIGTA
Report Findings and Corrective Actions Related to the Customer Account Data
Engine (CADE) Project
|
Report
Number |
Finding and Recommendation |
Corrective Action |
Status |
|---|---|---|---|
|
2003-20-018 11/2002 |
The Pilot Plan
Needs to Include Adequate Defect Reporting Procedures. To ensure the
defect reporting databases provide reliable information for the pilot defect
reports, the Business Systems Modernization Office (BSMO) should require the
PRIME contractor[46] to provide written procedures directing
that the IRS approve defect report resolution actions prior to defect report
closure. |
The IRS monitors Information Technology Asset Management
System (ITAMS)[47]
use and requires the PRIME contractor to develop procedures to include defect
report resolution. Under the draft
procedures, the PRIME contractor initiates the recommendation to close in the
ITAMS and the IRS is responsible for the final closure. Those procedures are currently under review.
Approval and distribution of the
procedures is planned for November 30, 2002. |
Closed |
|
Report
Number |
Finding and Recommendation |
Corrective Action |
Status |
|---|---|---|---|
|
2003-20-018 11/2002 |
The Pilot Plan Needs to
Include Adequate Defect Reporting Procedures. To ensure the defect reporting
databases provide reliable information for the pilot defect reports, the BSMO
should require the PRIME contractor to provide detailed procedures for
reconciling defect reports in the ClearQuest®[48] database and the ITAMS. |
The PRIME contractor has drafted procedures that document the process the
IRS will use when interfacing defect reporting information between the ClearQuest® database and the ITAMS defect
tracking tools. These procedures
contain specific information on the reconciliation of these reports. The procedures are currently under review
and are scheduled to be approved and distributed by November 30, 2002. |
Closed |
|
2003-20-089 3/2003 |
File and Job Names Need to Be
Compatible With Current Tax Processing Systems. The Deputy Commissioner for
Modernization and Chief Information Officer needs to ensure development of
job and file naming standards is expeditiously completed by the IRS
Enterprise Operations Services organization. The BSMO needs to work with the
PRIME contractor to ensure these naming standards are used in the development
of future CADE releases and all other IRS modernization projects. |
The BSMO agreed with the recommendation and is developing naming
standards. Key naming standards will
be published and available for use by the current processing and modernization
environments by August 1, 2003. |
Closed |
|
Report
Number |
Finding and Recommendation |
Corrective Action |
Status |
|
2003-20-089 3/2003 |
The Balancing,
Control, and Reconciliation Process Needs to Be Completed and Tested Prior to
Release 1. The BSMO should
monitor the completion of the remaining work contained in the detailed
schedule to assess the progress in completing the balancing, control, and
reconciliation process development. |
The BSMO agreed with the recommendation and will
continue to monitor the actions taken in balancing, control, and
reconciliation in preparation for CADE Release 1 deployment. |
Closed |
|
2003-20-089 3/2003 |
Improvements to
the Computer Handbook Will Help to Ensure an Effective Release 1 Deployment. The BSMO
should work with the PRIME contractor to incorporate minimum documentation
standards and the Documentation Task Force’s findings into CADE Release 1 and
future releases. |
The BSMO agreed with the recommendation and will work
with the PRIME contractor to incorporate minimum documentation standards and
the Documentation Task Force’s findings into the CADE. |
Closed |
|
Report
Number |
Finding and Recommendation |
Corrective Action |
Status |
|---|---|---|---|
|
2003-20-089 3/2003 |
Improvements to the Computer
Handbook Will Help to Ensure an Effective Release 1 Deployment. The BSMO should work with the
PRIME contractor to incorporate these documentation standards into the
Enterprise Life Cycle.[49] Incorporating standards into the Enterprise
Life Cycle will provide guidance in developing documentation for current and
future CADE releases, as well as all other IRS modernization projects. |
The BSMO is working with the PRIME contractor and Information
Technology Service organization as a task group to define standard
deliverables that will facilitate improved transition of modernization
systems into the current processing environment. The computer operations handbook subtask
group is working to define gaps and requirements for standard deliverables
needed by the Information Technology Service organization for operations and
maintenance support. |
Closed |
|
2003-20-089 3/2003 |
The Balancing, Control, and
Reconciliation Process Needs to Be Completed and Tested Prior to Release 1. The BSMO should require the
PRIME contractor to complete the remaining work on the balancing, control,
and reconciliation process and fully test these processes to ensure they meet
the design requirements |
The BSMO will continue to work with the PRIME contractor to ensure the
remaining work for the balancing, control, and reconciliation process for |
Closed |
|
Report
Number |
Finding and Recommendation |
Corrective Action |
Status |
|---|---|---|---|
|
2005-20-005 11/2004 |
Significant Software Changes
Were Added to Customer Account Data Engine Release 1.1. The Chief Information Officer
should direct the BSMO to work with the PRIME contractor to ensure future
project development changes undergo appropriate performance testing,
simulating high-volume processing, before deploying the system. |
The BSMO is planning a semiannual release of the CADE in July[50]
and January. The July delivery will
involve higher risk, more complex functionality, and the January delivery
will include filing season[51]
changes combined with additional changes as capacity permits. Since the returns from earlier in the filing
season will be available for testing, the IRS can conduct performance testing
on the July release using the highest volume periods. The IRS will determine whether to conduct
additional performance testing on the January release based on the likelihood
of the changes affecting performance. |
Closed |
|
Report
Number |
Finding and Recommendation |
Corrective Action |
Status |
|---|---|---|---|
|
2005-20-005 11/2004 |
The Customer Account Data
Engine Program Does Not Have a Dedicated System Architect. The Chief Information Officer
should ensure the BSMO makes the system architecture resources being acquired
available to the CADE program on a full-time basis. |
Over the last year, the BSMO has been actively searching for qualified
engineering resources. While the BSMO
has had success recruiting a few candidates (some of which are dedicated to
the CADE), it is very difficult to attract qualified individuals to work for
the Federal Government. The BSMO’s
recruiting efforts will continue. In
the interim, MITRE[52]
employees and members from the Enterprise Architecture team are providing
CADE engineering and architecture support. The PRIME contractor is continuing to
provide full-time architecture support to the CADE project. |
Open |
|
Report
Number |
Finding and Recommendation |
Corrective Action |
Status |
|---|---|---|---|
|
2005-20-005 11/2004 |
Disaster Recovery Capabilities
Were Not Tested Prior to Customer Account Data Engine Release 1.1
Implementation. To ensure the ability of the
IRS to restore the CADE after a disaster with the least disruption to the IRS
mission, the Chief Information Officer needs to ensure disaster recovery
capabilities for future releases of the CADE are fully tested prior to
implementation. |
Disaster recovery for the CADE needs to be periodically tested. However, it should be part of an enterprise
disaster recovery and testing strategy for mainframe computing operations and
databases at the |
Open |
|
Report
Number |
Finding and Recommendation |
Corrective Action |
Status |
|---|---|---|---|
|
2005-20-005 11/2004 |
Improvements to the Customer
Account Data Engine Operator’s Guide Need to Be Completed. The Chief Information Officer
should direct the BSMO to ensure pilot and production operational
documentation for future CADE releases is reviewed, tested, and approved
before both the pilot test and live production are allowed to proceed. This review should also ensure the
corrective action to our March 2003 report is incorporated in this review and
the IRS’ minimum documentation standards are completed and met in current and
future CADE releases. |
The BSMO made the required changes to the operational documentation
for CADE |
Closed |
|
2005-20-005 11/2004 |
Manual Processes Within
Customer Account Data Engine Release 1.1 Need to Be Automated for Future
Releases. The Chief Information Officer
should direct the BSMO to ensure inefficient manual processes are automated
in future CADE releases. |
Changes will be included in the January 2005 and July 2005 deliveries
addressing the specified operational inefficiencies. |
Open |
|
Report
Number |
Finding and Recommendation |
Corrective Action |
Status |
|---|---|---|---|
|
2005-20-005 11/2004 |
Disaster Recovery Capabilities
Were Not Tested Prior to Customer Account Data Engine Release 1.1
Implementation. To ensure the ability of the
IRS to restore the CADE after a disaster with the least disruption to the IRS
mission, the Chief Information Officer needs to ensure all aspects of the
CADE disaster recovery capabilities are tested during the Annual IRS Disaster
Recovery Test. |
The Director, |
Closed |
Source: TIGTA audit reports and IRS management responses.
2003-20-018 - Improvements
in the Customer Account Data Engine Pilot Plan Need to Be Considered to Help
Ensure the Pilot’s Success (Reference Number 2003-20-018, dated November 2002).
2003-20-089 - Adhering
to Established Development Guidelines Will Help to Ensure the Customer Account
Data Engine Meets Expectations (Reference Number 2003-20-089, dated March
2003).
2005-20-005 - To
Ensure the Customer Account Data Engine’s Success, Prescribed Management
Practices Need to Be Followed (Reference Number 2005-20-005, dated November
2004).
Appendix VIII
Enterprise Life
Cycle Overview
The Enterprise Life Cycle (ELC) is the Internal Revenue Service’s (IRS) standard approach to business change and information systems initiatives. It is a collection of program and project management best practices designed to manage business change in a successful and repeatable manner. The ELC addresses large and small projects developed internally and by contractors.
The ELC includes such requirements as:
·
Development of
and conformance to an enterprise architecture.
·
Improving
business processes prior to automation.
·
Use of
prototyping and commercial software, where possible.
·
Obtaining early
benefit by implementing solutions in multiple releases.
[55]
·
Financial justification,
budgeting, and reporting of project status.
In addition, the ELC improves the IRS’ ability to manage changes to the enterprise; estimate the cost of changes; and engineer, develop, and maintain systems effectively. Figure 1 provides an overview of the layers, paths, phases, and milestones (shown as “MS” in Figure 1) within the ELC Framework.
Figure
1: ELC Framework
Figure
1 was removed due to its size. To see
Figure 1, please go to the Adobe PDF version of the report on the TIGTA Public
Web Page.
ELC Layers
The ELC is a
framework for organizing and using IRS directives, processes, procedures,
templates, and standards to accomplish business change. It is organized as a set of 6 interacting
layers.
·
The Management Layer specifies how to plan
and control business change programs, projects, acquisitions, and solutions
throughout the ELC.
·
The Governance Layer specifies additional
controls imposed from outside the project or program.
·
The Solution Life Cycle Layer specifies
what should be done, but not how to do it.
·
The Solution Layer manages the solution as
it is produced, including providing standards for consistent solution
specification and formal review of solution content. This Layer provides control over work
products that may be produced by multiple internal and external developers
using differing methodologies.
·
The Methodology Layer details how to do the
work and specifies a unique set of work products to be produced. Specific methodologies are not part of the
ELC Framework.
·
The Specialty Areas Layer provides
additional guidance for areas of particular importance within the IRS. These areas include Enterprise Integration,
Test, and Evaluation;[56] Business Rules Harvesting[57] and Management; Transition Management;[58] Enterprise Architecture; Capital Planning and
Investment Control;[59] Security and Privacy; Requirements Development and
Management.
A path specifies a
unique “philosophy” or orientation for performing the work. Although the ELC specifies a standard for the
work required to produce and operate business change solutions, there are
multiple ways to approach and accomplish the required work. Paths are like alternate roads, each of which
crosses different terrain, but all of which lead to the same destination. The ELC provides five distinct paths or approaches
to developing systems:
·
The Large Custom Path is for large
projects.
·
The Small Custom Path is for small
projects.
·
The Commercial-Off-the-Shelf Path is a
commercial software-based approach.
·
The Joint Application Development/Rapid
Application Development Path is a highly accelerated, prototyping-based
approach for very small, standalone solutions or solution components.
·
The Iterative Custom Path is a hybrid
approach that combines elements of the other approaches.
A phase is a broad segment of
work encompassing activities of similar scope, nature, and detail and providing
a natural breakpoint in the life cycle.
Each phase begins with a kickoff meeting and ends with an executive
management decision point (called a milestone) at which IRS executives make
“go/no-go” decisions for continuation of a project. Project funding decisions are often
associated with milestones.
Figure
2: ELC Phases and Milestones
|
Phase |
General Nature of Work |
Concluding Milestone |
|---|---|---|
|
Vision and
Strategy/ |
High-level direction setting. This is the only phase for enterprise
planning projects. |
0 |
|
Project Initiation
Phase |
Startup of development projects. |
1 |
|
Domain Architecture
Phase |
Specification of the operating concept,
requirements, and structure of the solution.
|
2 |
|
Preliminary
Design Phase |
Preliminary design of all solution
components. |
3 |
|
Detailed
Design Phase |
Detailed design of solution components. |
4A |
|
System Development
Phase |
Coding, integration, testing, and
certification of solutions. |
4B |
|
System Deployment
Phase |
Expanding availability of the solution
to all target users. This is usually
the last phase for development projects. |
5 |
|
Operations
and Maintenance Phase |
Ongoing management of operational
systems. |
System Retirement |
Source:
The ELC Guide.
Appendix IX
Management’s
Response to the Draft Report
The response was
removed due to its size. To see the
response, please go to the Adobe PDF version of the report on the TIGTA Public
Web Page.
[1] A release is a specific edition of software.
[2] Form 1040EZ is the Income Tax Return for Single and Joint Filers With No Dependents. The initial release of the CADE does not process Forms 1040EZ for joint filers.
[3] The PRIME contractor is the Computer Sciences Corporation, which heads an alliance of leading technology companies brought together to assist with the IRS’ efforts to modernize its computer systems and related information technology.
[4] The filing season is the period from January through mid-April when most individual income tax returns are filed.
[5] Unit testing ensures program modules perform in accordance with requirements.
[6] A release is a specific edition of software.
[7] The PRIME contractor is the Computer Sciences Corporation, which heads an alliance of leading technology companies brought together to assist with the IRS’ efforts to modernize its computer systems and related information technology.
[8] Form 1040EZ is the Income Tax Return for Single and Joint Filers With No Dependents. The initial Release of the CADE does not process Forms 1040EZ for joint filers.
[9] The ELC establishes a set of repeatable processes and a system of reviews, checkpoints, and milestones that reduce the risks of systems development and ensure alignment with the overall business strategy.
[10] The filing season is the period from January through mid-April when most individual income tax returns are filed.
[11] The Office of Management and Budget Circular A-11 Exhibit 300, Capital Asset Plan and Business Case, dated June 2002, documents business cases used by agencies to request funds, monitor the progress of projects, and improve management decision making over expensive information technology investments.
[12] Unit testing ensures program modules perform in accordance with requirements.
[13] The Combined Systems Integration Test verifies the quality of the integrated system (i.e., verifies the system is integrated into other IRS systems properly and functions as required).
[14] MITRE is the IRS’ modernization strategic support contractor.
[15] Job Control Language is a means of communication between the programmer/analyst and the operating system. It consists of control statements that introduce a job to the operating system, provide accounting information, direct the operating system, request hardware devices, and execute a job.
[16] The balancing and reconciliation of accumulated and calculated data within the CADE include three major components: initialization (the transfer of taxpayer account data from existing IRS systems to the modernized CADE system); daily processing (the editing, processing, and record keeping of all transactions); and weekly processing (the external and internal balancing of all processing and transactions for the week).
[17] Application Qualification Testing focuses on verifying that business processes are supported as intended by the application; requirements allocated to the application are met; all tests are defined and executed; and all problems encountered are documented, corrected, and verified.
[18] Improvements in the Customer Account Data Engine Pilot Plan Need to Be Considered to Help Ensure the Pilot’s Success (Reference Number 2003-20-018, dated October 2002).
[19] Office of Management and Budget Circular A-127, Financial Management Systems, dated July 1993.
[20] The Custodial Accounting Project uses a data-warehousing approach for storing, analyzing, and reporting taxpayer accounts and collection information.
[21] GAO/AIMD-00-21.3.1, dated November 1999.
[22] A cost-plus-fixed-fee contract is a
cost-reimbursement contract that provides for payment to the contractor of a
negotiated fee that is fixed at the inception of the contract. The fixed fee does not vary with actual cost
but may be adjusted as a result of changes in the work to be performed under
the contract. This contract type permits
contracting for efforts that might otherwise present too great a risk to
contractors, but it provides the contractor only a minimum incentive to control
costs.
[23] A cost-plus-incentive-fee contract is a cost-reimbursement contract that provides for an initially negotiated fee to be adjusted later by a formula based on the relationship of total allowable costs to total target costs.
[24] A cost-plus-award-fee contract is a cost-reimbursement contract that provides for a fee consisting of (a) a base amount (which may be zero) fixed at inception of the contract and (b) an award amount, based upon a judgmental evaluation by the Federal Government, sufficient to provide motivation for excellence in contract performance.
[25] A release is a specific edition of software.
[26] The PRIME contractor is the Computer Sciences Corporation, which heads an alliance of leading technology companies brought together to assist with the Internal Revenue Service’s efforts to modernize its computer systems and related information technology.
[27] The Centralized Authorization File contains information about the type of authorizations taxpayers have given their representatives for their tax returns.
[28] The Master Files are the IRS’ central and official
repository of taxpayer information.
[29] The filing season is the period from January through mid-April when most individual income tax returns are filed.
[30] A release is a specific edition of software.
[31] CADE Release 1 (Operations and Maintenance) was initially referred to as “CADE Release 1, Milestone 5” in the BSM Expenditure Plan.
[32] CADE Release 1.2 was initially referred to as “CADE Filing Season 2005” in the BSM Expenditure Plan. The IRS did not provide cost and schedule data for this release in subsequent Expenditure Plans.
[33] CADE Release 1.3.1 and 1.3.2 were initially combined and referred to as “CADE Filing Season 2006” in the BSM Expenditure Plan.
[34] The Business Leader engages with the modernization technology team to help determine how to best apply technology to improve service to taxpayers, support enforcement activities, and improve compliance.
[35] The Business Requirements Director is the point of accountability on behalf of the Business Leader for ensuring business requirements and process engineering are the central focus in making project decisions.
[36] The Integrated Financial System project replaces the Internal Revenue Service’s (IRS) current inventory of antiquated financial systems with a single integrated system that connects the agency’s accounting, performance, budgeting, and procurement functions.
[37] The e-Services project modernizes the way taxpayers transact and communicate with the IRS by creating and marketing easy-to-use electronic products and services.
[38] Tiger teams and subteams consisted of highly experienced professionals from the IRS, Computer Sciences Corporation, IBM, etc. The teams collaborated on findings and recommendations in many areas (e.g., architecture and system design, building and design of code, testing and deployment, and documentation and performance-tuning for the CADE).
[39] This position, under the BSM Program Director, is responsible for successful day-to-day project management and execution, working with the Business Requirements Director and Contractor Project Manager to ensure system requirements are aligned with business requirements.
[40] MITRE is the IRS’ modernization strategic support contractor.
[41] The PRIME contractor is the Computer Sciences Corporation, which heads an alliance of leading technology companies brought together to assist with the IRS’ efforts to modernize its computer systems and related information technology.
[42] See Appendix VIII for an overview of the ELC and its various milestones.
[43] Systems integration testing ensures all system components (hardware and software) are working correctly and collectively with other related or dependant systems. Systems acceptance testing determines whether a system meets user and contract requirements and objectives.
[44] The Joint Engineering Team was formed as a means to organize the senior engineers assigned to the CADE.
[45] The ELC will establish a new milestone called “Milestone 4A.” In this Milestone, time and effort will be devoted to ensure business requirements are fully developed and the technical infrastructure is fully detailed. This detail will significantly increase the opportunities for the use of fixed-price contracts.
[46] The PRIME contractor is the Computer Sciences Corporation, which heads an alliance of leading technology companies brought together to assist with the IRS’ efforts to modernize its computer systems and related information technology.
[47] The ITAMS delivers an inventory system that enables tracking, reporting, and management of information technology assets.
[48] ClearQuest® is a defect and change tracking system that captures and manages all types of change requests throughout the development life cycle, helping organizations quickly deliver higher quality software.
[49] The Enterprise Life Cycle establishes a set of repeatable processes and a system of reviews, checkpoints, and milestones that reduce the risks of systems development and ensure alignment with the overall business strategy. See Appendix VIII for an overview of the Enterprise Life Cycle.
[50] The BSMO subsequently designated June 30 as its target for deploying the first subrelease of biannual CADE Releases.
[51] The filing season is the period from January through mid-April when most individual income tax returns are filed.
[52] MITRE is the IRS’ modernization strategic support contractor.
[53] IRS Computing Centers support tax processing and information management through a data processing and telecommunications infrastructure.
[54] The Master Files are the IRS’ central and official
repository of taxpayer information.
[55] A release is a specific edition of software.
[56] Enterprise Integration, Test, and Evaluation includes processes for integrating multiple components of a solution and conducting various types and levels of testing on the solution.
[57] A business rule is a statement that defines or
constrains some aspect of the business. Harvesting
is a general term used to broadly describe the entire set of activities
involved in gathering, formalizing, analyzing, and validating business rules
for a particular scope.
[58] Transition management helps ensure personnel and organizations are prepared to receive, use, operate, and maintain the business processes and technology provided by business change solutions.
[59] The Capital Planning Investment and Control process
manages a central portfolio of information technology investments across the
IRS.