Adhering to Established Development Guidelines Will Help to
Ensure the Customer Account Data Engine Meets Expectations
March 2003
Reference Number: 2003-20-089
This report has cleared the Treasury
Inspector General for Tax Administration disclosure review process and
information determined to be restricted from public release has been redacted
from this document.
March
27, 2003
MEMORANDUM FOR
DEPUTY COMMISSIONER FOR MODERNIZATION
&
CHIEF INFORMATION OFFICER
FROM: Gordon C. Milbourn III /s/ Gordon C.
Milbourn III
Acting Deputy Inspector
General for Audit
SUBJECT: Final Audit Report - Adhering to Established Development
Guidelines Will Help to Ensure the Customer Account Data Engine Meets
Expectations (Audit # 200320001)
This
report presents the results of our review of the Internal Revenue Service’s
(IRS) efforts to deploy the first release of the Customer Account Data Engine
(CADE). The objective of this review was to determine whether the
IRS and PRIME contractor will timely and effectively deliver the CADE Release
1.
The CADE is the foundation for managing taxpayer
accounts in the IRS’ modernization plan.
It will consist of databases and related applications that
will replace the IRS’ existing Master File processing systems and will include
applications for daily posting, settlement, maintenance, refund processing, and
issue detection for taxpayer tax account and return data. The CADE databases and related applications
will also enable the development of subsequent modernized systems that will
improve customer service and compliance and allow the on-line posting and
updating of taxpayer account and return
data.
The
portion of the CADE related to individual tax accounts will be incrementally
deployed in five releases, each related to a specific taxpayer segment, over
several years. At the conclusion of
Release 5, the CADE will have replaced the Individual Master File, and the IRS
will conduct business much differently than it does today.
In summary, the IRS and PRIME contractor have made progress in delivering the CADE project by building a substantial portion of Release 1 and creating a comprehensive foundation for all five releases. However, the development of the CADE project by the PRIME contractor has experienced significant delays. The Release 1 deployment date is now estimated to be June 2003, which is about 18 months behind its planned delivery date. The IRS and PRIME contractor initially estimated that Release 1 could be delivered for approximately $51.9 million. This estimate was revised upward 6 months later to $64.6 million. The IRS had spent $54.5 million on Release 1 through September 2002.
Project delays can be
attributed to difficulties in identifying and managing the project
requirements. Specifically, these
difficulties occurred in developing the balancing, control, and reconciliation
process; comprehensive documentation for the CADE Computer Operations Handbook;
and computer system naming standards.
Management’s Response: The DCM/CIO
agreed with the recommendations made in our report. The IRS will continue to work with the PRIME contractor to ensure
the remaining work for balancing, control, and reconciliation for Release 1 is
tested and completed. In addition, the
IRS is addressing deficiencies in the CADE Computer Operations Handbook, and it
will continue with the development of job and file naming standards in
coordination with the Enterprise Operations Services organization and the PRIME
contractor. Also, the IRS will work with the PRIME contractor to ensure that the business system
design is completed prior to beginning the development phase for CADE Release 2
and future releases. Management’s complete response to the draft report is
included as Appendix IV.
Copies of this report are also being sent to the IRS
managers who are affected by the report recommendations. Please contact me at (202) 622-6510 if you
have questions or Gary Hinkle, Acting Assistant Inspector General for Audit (Information
Systems Programs), at (202) 927-7291.
File and Job Names Need to Be
Compatible With Current Tax Processing Systems
Release 1 Entered the Development Phase Before the Business System Design Was Complete
Appendix I – Detailed Objective, Scope, and Methodology
Appendix II – Major Contributors to This Report
Appendix III – Report Distribution List
Appendix IV – Management’s Response to the Draft Report
The Customer Account Data Engine (CADE) is the foundation for managing taxpayer accounts in the Internal Revenue Service’s (IRS) modernization plan. The CADE will consist of databases and related applications that will replace the IRS’ existing Master File processing systems. The CADE will include applications for daily posting, settlement, maintenance, refund processing, and issue detection for taxpayer tax account and return data. The CADE databases and related applications will also enable the development of subsequent modernized systems that will improve customer service and compliance. In conjunction with other applications, the CADE will allow employees to post transactions and update taxpayer account and return data on-line from their desks. Updates will be immediately available to any employee who accesses the data and will provide a complete, timely, and accurate account of the taxpayer’s information. In contrast, in the current Master File processing system it can take up to 2 weeks to update tax accounts, and IRS employees may need to access several computer systems to gather all relevant information related to the tax accounts.
The CADE project will include two phases. The first phase involves individual tax accounts and will be incrementally deployed in five releases, each related to a specific taxpayer segment, over several years. For example, Release 1 will process Income Tax Returns for Single and Joint Filers With No Dependents (Form 1040EZ) electronic and paper tax returns for single taxpayers. Release 2 will add the U.S. Individual Income Tax Returns (Form 1040) for families and taxpayers filing as head of household. At the conclusion of Release 5, the CADE will have replaced the Individual Master File, and the IRS will conduct business much differently than it does today. The second phase of the CADE will convert the Business Master File tax accounts; however, development and implementation dates for the second phase have not been established.
The IRS initiated the CADE project in September 1999. In August 2000, the CADE project team completed the architecture phase, and in June 2001 the project exited the design phase. The CADE planning included a Baseline Business Case that targeted January 2002 as the delivery date for Release 1.
The objective of this review was to determine whether the IRS and PRIME contractor could timely and effectively deliver the CADE Release 1. To accomplish our objective, we reviewed available documentation and interviewed PRIME contractor, IRS Information Technology Services (ITS), and Business Systems Modernization Office (BSMO) executives, managers, and analysts located at the National Headquarters in Washington, D.C., the Martinsburg Computing Center, and the New Carrollton, Maryland, offices. We performed the audit from October 2002 through January 2003. The audit was conducted in accordance with Government Auditing Standards. Detailed information on our audit objective, scope, and methodology is presented in Appendix I. Major contributors to the report are listed in Appendix II.
The PRIME contractor drafted an initial CADE planning document for the IRS in March 2001. This document, known as the Baseline Business Case, assists IRS executives in selecting the project’s design for development and testing. It also justifies funding for the project to IRS oversight organizations.
The IRS and PRIME contractor continued with the design of Release 1 and approved its development in June 2001. To date, the project team has built a substantial portion of Release 1 and has created a comprehensive foundation for all five releases. Specifically, the project team validated the conversion of 2002 tax data from the current processing environment to the CADE. The project team also completed building almost one-half of the new database that will serve as a foundation for the first five releases.
Since the creation of the Baseline Business Case, the IRS and PRIME contractor have postponed the planned deployment of Release 1 several times, from January 2002 until June 2003. This setback occurred for several reasons. First, the PRIME contractor could not complete a sub-contract for a software tool in time for the development phase of Release 1. This happened because IRS executives allowed the project to exit the design phase without the necessary contracts in place.
Next, the PRIME contractor underestimated the complexity of developing the balancing, control, and reconciliation process. This, along with inconsistent and incomplete documentation for testing, contributed to pushing the delivery of Release 1 into June 2003, about 18 months later than originally planned.
The Baseline Business Case, developed in March 2001, stated that the cost to design and deploy Release 1 was approximately $51.9 million. In September 2001, the IRS and the PRIME contractor revised this estimate upward to $64.6 million. However, this revised estimate does not include the Release 1 deployment costs. According to IRS contracts and progress reports, the IRS had spent $54.5 million on Release 1 through September 2002. While this amount will not increase for the remainder of the development phase, additional costs will be incurred during the deployment phase.
During our review, the PRIME contractor and the IRS were in the process of negotiating a revised project schedule for the remaining work needed to deliver Release 1. Additionally, a revised CADE strategy outlining the delivery of Release 2 and subsequent releases is being developed.
The balancing, control, and reconciliation process is key to the IRS ensuring the data integrity of the taxpayer information brought into the CADE. The CADE System Requirements Report contains requirements for the balancing, control, and reconciliation of accumulated and calculated data within the CADE. This process needs to be completed and tested prior to the Release 1 deployment.
The PRIME contractor recognized that it underestimated the scope of the CADE balancing, control, and reconciliation process in February 2002. This can be attributed to an absence of complete detail in the CADE Business System Design Report. The absence of adequate detail in this report affected the PRIME contractor’s appreciation of the complexity and extent of this process. The PRIME contractor subsequently pursued further details about the balancing, control, and reconciliation process. Its assessment indicated that it still had to complete significant work activity around the design, development, and testing to satisfy the process.
According to the PRIME contractor’s latest schedule, the balancing, control, and reconciliation process would be completed by February 2003. However, as of December 2002 the PRIME contractor was still working on obtaining a detailed understanding of the process design. Although progress has been made, the following actions must be finished for the process to be completed:
· Conduct reviews to compare balancing formulas to detailed CADE specifications.
· Finalize balancing, control, and reconciliation error message handling and recovery procedures.
· Complete design and implementation of the requirements for the CADE balancing, control, and reconciliation applications.
· Update the balancing, control, and reconciliation details into the CADE Computer Operations Handbook.
· Complete the request for sub-contractor modifications to the CADE reports.
The delay of Release 1 deployment to June 2003 is partially due to the incomplete development of the balancing, control, and reconciliation process. The PRIME contractor does not have a detailed schedule of the remaining work anticipated to complete the process development. In November 2002, the PRIME contractor informed us that it would provide a detailed schedule with the delivery of the revised CADE strategy. We received the revised deployment dates on December 4, 2002; however, the dates did not include a detailed schedule of the remaining work on the balancing, control, and reconciliation process.
Without a detailed schedule outlining the remaining work to complete the balancing, control, and reconciliation process development, the IRS and PRIME contractor cannot ensure that there is sufficient time to meet the targeted Release 1 deployment date. Without a complete and tested balancing, control, and reconciliation process, the IRS cannot ensure that the transactions processed by the CADE are complete and accurate.
To ensure the data integrity of
taxpayer information in the CADE and that critical balancing, control, and
reconciliation processes in Release 1 have been completed, tested, and made
ready for deployment, the BSMO should:
1. Require the PRIME contractor to complete the remaining work on the balancing, control, and reconciliation process and fully test these processes to ensure they meet the design requirements. The BSMO should require the PRIME contractor to provide a detailed schedule of the remaining work, including the time needed to complete the process development and the specific resources necessary to complete each activity. The BSMO should monitor the completion of these action items to assess the progress in completing the process development.
Management’s
Response:
The Deputy Commissioner for Modernization & Chief Information
Officer (DCM/CIO) agreed with
the recommendation and responded that the IRS will continue to work with the
PRIME contractor to ensure the remaining work for balancing, control, and
reconciliation for CADE Release 1 is tested and completed. The IRS has received a detailed schedule
from the PRIME contractor that identifies the planned work to be completed
during this milestone, including balancing, control, and reconciliation. In addition, the BSMO will continue to monitor
the actions taken in balancing, control, and reconciliation in preparation for
the Release 1 deployment.
As part of the design and development work on Release 1, the PRIME contractor produced a draft computer operations handbook to be used by project team members for testing and operating the CADE. However, the IRS project team members found that the handbook did not meet the guidelines developed by the IRS and did not contain sufficient detail to plan activities and schedule resources to perform testing and pilot operations of the CADE application.
The IRS developed the Computer Operations Handbook Guide to act as a template for developing detailed instructions to a computer operator for the proper execution and validation of a program’s operation. It also provides standards and rules applicable to all IRS developers and is a required product for the programming stage of development. The Guide applies to all developers engaged in developing programs for use in the IRS’ Computing Centers. Additionally, the CADE System Requirements Report called for the CADE to follow relevant PRIME and IRS standards for the IRS’ Computing Center operations.
The documentation weaknesses for operating the CADE included:
· Necessary input and/or output files were not identified or were incorrectly named.
· Balancing formulas to validate data transactions did not include descriptive details.
· Instructions for handling processing errors were missing.
· Operating descriptions explaining basic processing objectives and steps were missing.
· Schematics (flowcharts, drawings, or diagrams) describing system processes were missing or not complete.
· Procedures to restart operations following processing interruptions were missing.
The BSMO provided the PRIME contractor with the IRS’ guidance on developing computer operations handbooks during the design phase of the CADE. However, the PRIME contractor prepared the CADE Computer Operations Handbook without referring to the IRS’ guidance. The IRS’ CADE team members notified the PRIME contractor that the Handbook was not sufficient to perform the necessary review, planning, and testing activities and that the Handbook did not follow the IRS’ computer application documentation standards. As a result, the PRIME contractor agreed to begin using the IRS’ guidance for future computer operations handbook development.
The absence of sufficient documentation contributed to delays in the CADE development and testing. The insufficient documentation hindered the CADE team members from performing adequate and insightful reviews of plans and schedules to test the project’s computer operations. These delays have contributed to the overall delay of the Release 1 deployment.
Management Actions: The PRIME contractor created a special review group in August 2002 to improve program documentation. The group’s objective was to build a structured process to develop and release documentation. In November 2002, after addressing many of the initial documentation problems, the PRIME contractor disbanded this group and replaced it with a Documentation Task Force. The Documentation Task Force’s objectives are to address the remaining documentation gaps and identify the root cause of the documentation problems to avoid repeating those problems in future releases.
To address deficiencies in the CADE Computer Operations Handbook, the BSMO should:
2. Work with the PRIME contractor to incorporate minimum documentation standards and the Documentation Task Force’s findings into CADE Release 1 and future releases.
Additionally, the BSMO should work with the PRIME contractor to incorporate these documentation standards into the Enterprise Life Cycle (ELC). These standards will need to include existing IRS documentation requirements in modernization project processes. Incorporating standards into the ELC will provide guidance in developing documentation for current and future CADE releases, as well as all other IRS modernization projects.
Management’s Response: The DCM/CIO agreed with the recommendation and stated that the IRS will work with the PRIME contractor to incorporate minimum documentation standards and the Documentation Task Force’s findings into the CADE. The BSMO is working with the PRIME contractor and the ITS organizations as a task group to define standard deliverables that will facilitate improved transition of modernization systems into the current processing environment. The computer operations handbook sub-task group is working to define gaps and requirements for standard deliverables needed by the ITS for operations and maintenance support.
Naming standards are necessary to provide consistency so that computer system files and jobs are properly identified. This consistency allows programmers to associate the appropriate files and jobs with the correct system. In addition, consistency is necessary in system file and job names to allow for proper interfaces when two or more systems work together.
The CADE System Requirements Report called for the CADE to use IRS enterprise-wide standards and follow relevant PRIME and IRS standards for the Martinsburg Computing Center operations.
Currently, the IRS Enterprise Operations Services organization is developing naming standards for use in all IRS project development. The Martinsburg Computing Center has its own naming standards that it uses for projects operating at the computing center.
The PRIME contractor worked with the IRS to determine the naming standard requirement it needed to use for Release 1 development. The PRIME contractor attempted to incorporate all of the direction it received from the IRS in the use of naming standards for Release 1, but the IRS provided inconsistent direction in the use of the Martinsburg Computing Center’s naming standards. This resulted in the PRIME contractor’s development of its own naming standards. Subsequently, the PRIME contractor provided its CADE naming standards to the IRS; however, there was no official IRS approval or acceptance of these standards for use in the CADE. The PRIME contractor’s naming standards were not fully compatible with the current IRS tax processing systems.
Because of the different job naming standards used by the CADE and current IRS tax processing systems, revisions will have to be made wherever the CADE interfaces with the current systems. Without revisions to the job names, job data will be stored and possibly run incorrectly. For internal CADE interfaces, the names can remain unchanged until a naming standard has been determined.
Once the IRS Enterprise Operations Services organization’s naming standards are finalized, applications already in production may have to be revised to the new naming standard. These changes may result in additional costs to the IRS.
To facilitate project development and limit additional costs on future CADE releases and other IRS modernization projects, the DCM/CIO needs to ensure that:
3. Development of job and file naming standards is expeditiously completed by the IRS Enterprise Operations Services organization. The BSMO needs to work with the PRIME contractor to ensure that these naming standards are used in the development of future CADE releases and all other IRS modernization projects.
Management’s
Response:
The DCM/CIO agreed with
the recommendation and stated that the IRS is developing naming standards. Key naming standards will be published and available
for use by the current processing and modernization environments by August 1,
2003.
In November 2001, we reported that IRS modernization project teams did not follow all critical systems development processes to help projects meet objectives. We cited three separate modernization projects that entered the development phase before the system requirements were complete. These process weaknesses ultimately contributed to cost increases, schedule delays, and performance shortfalls.
We identified similar process weaknesses during the development of the CADE project. The IRS allowed the PRIME contractor to enter the development phase of the CADE project before the business system design was complete. As a result, the IRS and the PRIME contractor are still trying to complete work in the following areas:
· Balancing, control, and reconciliation.
· CADE Computer Operations Handbook documentation.
· Computer system naming standards.
The Software Acquisition Capability Maturity Model is used by organizations such as the IRS that acquire or procure software-intensive systems. The IRS uses this capability maturity model to improve its systems acquisition process for IRS modernization projects, including the CADE. Requirements development and management is a key process in this model. The purpose of this key process is to establish common and clear system requirements that are understood by the project team, IRS end users, and the PRIME contractor.
The IRS also uses the ELC methodology to plan and manage work on its modernization efforts. For those systems under development where the requirements can be adequately defined, the ELC requires the business system design to be complete before beginning the development phase.
The IRS has been processing tax returns with the current system for more than 35 years. The CADE will be replacing this system with additional features. While the programming may be very complicated, the CADE business system design should have been timely completed. However, the IRS did not ensure that the PRIME contractor understood the current processing environment and included input from IRS subject matter experts and testers.
Additionally, the PRIME contractor selected a development approach that is used when requirements are uncertain and a solution needs to be arrived at through a series of prototypes. This approach does not always provide a clear separation of design and development activities.
The use of this development approach allowed the business system design to be added to or amended throughout the development process. This caused the PRIME contractor to design, code, and document against a changing business system design and requirements. Examples include:
· The inability to complete a sub-contract for a business rules engine resulted in the use of a software programming language that was not as versatile.
· The introduction of a CADE Pilot resulted in adding extra work for both the IRS and the PRIME contractor.
These unplanned revisions to the Release 1 project affected the overall
project schedule and budget. Partially
because the business system design was not timely completed and incorporated
into the project, Release 1 is about 18 months behind its planned delivery
date.
Management Actions: The PRIME contractor plans to update the Release 1 database and related applications with an upgraded software tool during the development of Release 2 and future releases.
To address the CADE project design requirement completeness and stability, the BSMO should:
4.
Work
with the PRIME contractor to ensure that the business
system design of the CADE is complete before entering the development phase for
Release 2 and future releases.
Management’s
Response:
The DCM/CIO agreed with
the recommendation and stated the IRS will work with the PRIME contractor to
ensure that it completes the business system design before beginning the
development phase for the CADE Release 2 and future releases.
Appendix I
Detailed
Objective, Scope, and Methodology
The overall objective of this review was to determine whether the Internal Revenue Service (IRS) and PRIME contractor will timely and effectively deliver the Customer Account Data Engine (CADE) Release 1. To accomplish this objective, we performed the following reviews and analyses:
I. Determined the effectiveness of the CADE Release 1 deployment plans by reviewing the deployment guidance documents including the draft deployment plan, final integration testing plans, system application testing plans, and security test and evaluation plans.
A. Determined the status of the Release 1 deployment.
1.
Interviewed
the CADE project managers to identify current status and emerging risks/issues.
2. Participated in Core Data Management Sub-Executive Steering Committee meetings and reviewed status meeting minutes.
B. Interviewed Martinsburg Computing Center (MCC) CADE team members to determine if they received sufficient references and guidance to plan for and perform the Release 1 coding, testing, and deployment activities.
C. Reviewed the work completed to further define objectives and steps necessary for the Release 1 deployment in the MCC.
D. Reviewed the completeness of the Release 1 Computer Operator Handbook with MCC personnel.
E. Determined whether problems with the deployment activities could have been avoided with proper implementation of key system development processes and management disciplines previously presented by the Treasury Inspector General for Tax Administration, the General Accounting Office, and MITRE (the IRS’ modernization strategic support contractor).
II.
Determined the status of the Release 1 task order
activities, including the status of contract negotiations for post December 31,
2002, to allow expeditious deployment.
Appendix II
Major
Contributors to This Report
Gary V. Hinkle, Acting Assistant Inspector General for
Audit (Information Systems Programs)
Scott A. Macfarlane, Director
Edward A. Neuwirth, Audit Manager
W. Allen Gray, Senior Auditor
Bruce
Polidori, Senior Auditor
Glen
J. Rhoades, Senior Auditor
Beverly Tamanaha, Senior Auditor
Appendix III
Acting Commissioner N:C
Associate Commissioner, Business Systems Modernization M:B
Chief, Information
Technology Services M:I
Deputy
Associate Commissioner, Program Management
M:B:PM
Director,
Portfolio Management Division M:R:PM
Chief Counsel CC
National Taxpayer Advocate
TA
Director, Legislative Affairs CL:LA
Director, Office of
Program Evaluation and Risk Analysis
N:ADC:R:O
Office of Management Controls N:CFO:AR:M
Audit Liaisons:
Associate
Commissioner, Business Systems Modernization
M:B
Chief, Information Technology Services M:I
Appendix IV
Management’s Response to the Draft Report
The response was removed due to its
size. To see the complete response,
please go to the Adobe PDF version of the report on the TIGTA Public Web Page.