The Customer Account Data Engine 2 Is Making Progress Toward Achieving Daily Processing, but Improvements Are Warranted to Ensure Full Functionality
September 28, 2011
Reference Number:
2011-20-109
This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.
Phone
Number | 202-622-6500
Email Address | TIGTACommunications@tigta.treas.gov
Web Site |
http://www.tigta.gov
HIGHLIGHTS
THE CUSTOMER ACCOUNT DATA ENGINE 2 IS
MAKING PROGRESS TOWARD ACHIEVING DAILY PROCESSING, BUT IMPROVEMENTS ARE
WARRANTED TO ENSURE FULL FUNCTIONALITY
Highlights
Final
Report issued on September 28, 2011
Highlights
of Reference Number: 2011-20-109 to the Internal Revenue Service
Chief Technology Officer.
IMPACT ON TAXPAYERS
The mission of the Customer
Account Data Engine (CADE) 2 Program is to provide state‑of‑the-art individual taxpayer
account processing and technologies to improve service to taxpayers and enhance
Internal Revenue Service (IRS) tax administration. Once completed, the new modernization
environment should allow the IRS to more effectively and efficiently update
taxpayer accounts, support account settlement and maintenance, and process
refunds on a daily basis, all of which will contribute to improved taxpayer
services.
WHY TIGTA DID THE AUDIT
The
overall objective of this review was to assess the logical and physical design
of the CADE 2 Daily Processing activities and ensure the Daily Processing
design is secure, the design satisfies the documented approved requirements,
and the project management practices adhere to the Enterprise Life Cycle (ELC)
standards and processes for the related milestones (through Milestone 4a).
WHAT TIGTA FOUND
The IRS is closer to achieving one of its modernization
goals, daily processing of taxpayer accounts.
In addition, TIGTA determined the IRS has taken steps to address
security requirements during the Daily Processing Project. The IRS prepared a System Security Plan and
documented 171 security controls, of which 65 were classified as planned
controls (i.e., the control is not in place and there is an activity planned to
implement the control).
However, additional improvements and
adherence to all criteria will help ensure that the CADE 2 functions as intended
and that the January 2012 release date is met.
The CADE 2 Daily Processing Project did not ensure business rules and
requirements were always fully or timely developed, the Electronic Fraud
Detection System was properly recorded in the Item Tracking Reporting and
Control System, and open issues were properly addressed and closed prior to
milestone exits. Further, the Daily
Processing Project did not follow a consistent ELC path, and the Work Breakdown
Schedule was incomplete.
WHAT TIGTA RECOMMENDED
TIGTA
recommended that the Chief Technology Officer ensure a risk mitigation plan is
formally developed and documented and all open issues are addressed and closed
prior to approving milestone exits.
TIGTA also recommended several other system development process
improvements.
In
its response, the IRS agreed with two of TIGTA’s recommendations and plans to
take appropriate corrective action. The
IRS disagreed with TIGTA’s recommendation to ensure that all open issues are
addressed and closed prior to exiting a milestone. The IRS stated its guidance does not list
specifics around what constitutes an open issue, leaving some flexibility to
make risk-based decisions on whether a given open issue will impact efforts in
the following milestones or undermine the success of the project. However, the design meeting documentation
(dated 10 days prior to the milestone exit) indicated the design impact of the
open issues was not known. Exiting a
milestone without properly addressing critical open issues could result in
rework, potentially impact the logical or physical design, and result in
unnecessary costs.
September 28, 2011
MEMORANDUM FOR CHIEF TECHNOLOGY OFFICER
FROM: Michael R. Phillips /s/ Michael R. Phillips
Deputy Inspector General for Audit
SUBJECT: Final Audit Report – The Customer Account Data Engine 2 Is Making Progress Toward Achieving Daily Processing, but Improvements Are Warranted to Ensure Full Functionality (Audit # 201120001)
This report presents
the results of our review of the Customer Account Data Engine 2 Daily Processing. The overall objective of this review was to assess
the logical and physical design of the Customer Account Data Engine 2 Daily
Processing activities and ensure the Daily Processing design is secure, the
design satisfies the stated documented approved requirements, and the project
management practices adhere to the Enterprise Life Cycle standards and
processes for the related milestones (thru Milestone 4a). This review was requested by the Chief
Technology Officer and is included in our Fiscal Year 2011 Annual Audit Plan. It addresses the major management challenge
of Modernization of the Internal Revenue Service.
Management’s
complete response to the draft report is included as Appendix VII.
Copies of this report
are also being sent to the Internal Revenue Service managers affected by the
report recommendations. Please contact
me at (202) 622-6510 if you have questions or Alan Duncan, Assistant Inspector
General for Audit (Security and Information Technology Services), at (202) 622-5894.
Improvements
Are Warranted in Several Management Areas
Appendices
Appendix I – Detailed Objective, Scope, and Methodology
Appendix
II – Major Contributors to This Report
Appendix
III – Report Distribution List
Appendix
IV – Enterprise Life Cycle Overview
Appendix
V – Transition State 1 Integration Reviews
Appendix
VI – Glossary of Terms
Appendix
VII – Management’s Response to the Draft Report
Abbreviations
|
CADE |
Customer Account Data Engine |
|
EFDS |
Electronic Fraud Detection System |
|
ELC |
Enterprise Life Cycle |
|
IMF |
Individual Master File |
|
IRS |
Internal Revenue Service |
|
ITRAC |
Item Tracking Reporting and Control |
The Customer Account Data Engine 2 Program is the highest priority
information technology modernization project in the Internal Revenue Service.
In August 2008, the Internal Revenue Service
(IRS) Commissioner established the Modernized Taxpayer Account Program
Integration Office to manage the transition of current individual income tax
processing, which consists of multiple computer systems for processing tax
returns, payments, and other transactions affecting individual taxpayer
accounts, into a more consolidated system.
Working in conjunction with IRS business owners, the Modernized Taxpayer
Account Program Integration Office decided to integrate elements from both the
existing Individual Master File (IMF)[1] and current Customer Account Data Engine
(CADE) processes into a new CADE 2 Program.
The proposed plan incrementally transfers taxpayer accounts from the
current IMF and CADE processing systems to a new CADE 2 relational database.
The CADE 2 Program is the top information
technology modernization project in the IRS.
The CADE 2 strategy involves
three phases:
·
Transition
State 1. Modifies the IMF from a weekly cycle to daily
processing, establishes a new relational database to store all individual
taxpayer account information, and provides management tools to more effectively
use data for compliance and customer service. The IRS plans to implement Transition State 1
in January 2012.[2]
·
Transition
State 2. Launches a single processing system where
applications directly access and update the taxpayer account database. It will continue efforts toward addressing
previously identified financial material weaknesses. The IRS plans to implement Transition State 2
in January 2014. We recently learned
that lack of funding may put delivery of this phase at risk. The IRS is working to identify funding it
could use to begin high-level planning efforts in an attempt to keep this on
track.
·
Target
State. Consists of a single system using elements of
the IMF and current CADE, eliminating all transitional applications used to
link the current CADE, the IMF, and the Integrated Data Retrieval System. The complete solution also plans to address
all the financial material weaknesses.
As of April 28, 2011, the IRS had not established a Target State
implementation date.
The CADE 2 Daily
Processing Project is not a new application development project. Instead, it will enhance the existing IMF by processing
taxpayer accounts on a daily schedule, rather than weekly schedule. To achieve high efficiency, the CADE 2
Program has decided to leverage existing systems and design artifacts.
The
CADE 2 Daily Processing Project should improve processing and systems
associated with managing individual taxpayer accounts on a daily basis. The Daily Processing Project charter defines
the project purpose, scope, goals, and expected outcomes. The Daily Processing Project will address
risk factors inherent in the current CADE approach, define target-state
applications and processes used for managing the daily processing application
over individual taxpayer accounts, and begin transition to the target state. The Daily Processing Project will be
accountable for achieving the defined goals and managing and integrating all required
components, to include four subprojects:
1)
Daily updates to the Integrated
Data Retrieval System.
2)
Adjustments to notices.
3)
Transaction processing.
4)
Preparing daily loads to update
the CADE 2 database.
By moving to daily processing, the CADE 2 Daily Processing Project will provide immediate and obvious benefits including faster refunds to taxpayers, faster posting of payments, and more efficient adjustments to taxpayer accounts.
This review is one of a series of audits for providing assessments of the CADE 2 Program as part of our Security and Information Technology Audit Strategy for Fiscal Years 2010 and 2011. The Treasury Inspector General for Tax Administration has recently completed audits of the CADE 2 Program Management Office and Database Implementation.[3]
This review was performed at the
Modernization and Information Technology Services organization facility in New
Carrollton, Maryland, during the period January through May 2011. We conducted this performance audit in
accordance with generally accepted government auditing standards. Those standards require that we plan and
perform the audit to obtain sufficient, appropriate evidence to provide a
reasonable basis for our findings and conclusions based on our audit objective.
We believe that the evidence obtained
provides a reasonable basis for our findings and conclusions based on our audit
objective. Detailed information on our
audit objective, scope, and methodology is presented in Appendix I. Major contributors to the report are listed in
Appendix II.
The Internal Revenue
Service Is Nearing Its Modernization Goal to Achieve Daily Processing of Taxpayer
Accounts
The
CADE 2 Daily Processing Project has steadily progressed from project initiation
(Milestone 1) through Physical Design (Milestone 4a). As a result, the IRS is closer to achieving
one of its modernization goals, daily processing of taxpayer accounts. Each milestone presents new risks and
challenges from which the Daily Processing Project team has gained invaluable
information. For example, the IRS stated
for daily processing to be successful, the processing start date needed to be
changed from Friday to Thursday. Additionally,
due to the critical nature of the system to the IRS mission, the IRS documented
security requirements for Daily Processing and will use secure file transfer
protocol to protect data from unauthorized access, modification, and corruption. As a result, the IRS gained a
higher degree of confidence in achieving daily processing.
On
January 29, 2010, the CADE 2 Daily Processing Project initiated efforts to
accomplish objectives for Milestone 3, Logical Design. Specifically, these objectives establish:
·
Context
for CADE 2 Transition State 1, Logical Design.
·
Current
state and high-level relationships to systems affected by the CADE 2 solution.
·
Scope
of CADE 2 Transition State 1 and projects within the context of systems.
·
Changes
required to current processing cycles upon daily processing implementation.
·
CADE
2 data model as a data-centric foundation for the future.
·
Introduction
of architectural components that are the foundation of the logical design.
On September 30, 2010, the CADE 2 Daily Processing Project initiated efforts to accomplish objectives for Milestone 4a, Physical Design. Specifically, these objectives:
·
Build
confidence and demonstrate achievement of a successful completion of CADE 2
Transition State 1, Physical Design.
·
Walk
through the physical design from end to end.
·
Validate
that key physical design components are in place, including data, applications,
infrastructure, security, and performance.
·
Validate
that integration points were identified and incorporated into the physical
design.
·
Ensure
the physical design is sound and consistent with design principles.
·
Discuss
various scenarios associated with daily processing, weekly processing, and
error conditions.
·
Confirm
that outstanding physical design issues have been addressed.
An
independent firm reviewed and evaluated the success criteria for CADE 2 Daily
Processing and validated the IRS’s conclusion that Daily Processing can be
achieved. As a result of the progress
achieved and the independent confidence assessment, the CADE 2 Daily Processing
Project received a “go” decision.
Figure 1: Approach for
Conducting the Go/No-Go Readiness Review
Figure 1 was removed due to its size. To see Figure 1, please go
to the Adobe PDF version of the report on the TIGTA Public Web Page.
Security during the Daily Processing Project was planned
According to the CADE 2 Security
Framework document, because the IMF will not undergo any major changes to its
architecture, the security controls will largely remain as they exist. However, we determined that the IRS has taken
steps to address security during the Daily Processing Project. During the early phases of development, the
IRS prepared a System Security Plan, which documents the planned controls for
the application and addresses the security concerns that may affect the
system’s operating environment. The System
Security Plan highlights 171 security controls, representing 19 control
families, and differentiates between those that are planned controls (i.e., the
control is not in place and there is an activity planned to implement the
control) versus those that are inherited as part of the IRS organizational
level. The CADE 2 Daily Processing Project’s
Controls Requirements Matrix identified 65 planned controls and 106 inherited
controls.
In
addition, the IRS contracted with an independent firm to complete a threat
susceptibility analysis on CADE 2 Transition State 1. The contractor’s report concluded that threats
to CADE 2 Transition State 1 by external interfaces and databases appear
to be minimal. Attacking through an
external partner by corrupting the data imported through the air would be
difficult and could primarily be avoided by proper validity checks during daily
processing. The other attacks will
likely be reasonably mitigated by existing security mechanisms already in place
at the IRS, though conclusions about the Integrated Production Model are
conditional upon database configuration and implementation, and these details were
not known during the time period of this audit.
The
Daily Processing team identified and documented additional security controls
for CADE 2. For example, Daily
Processing will encrypt data moving between systems that are not located in the
same data center. Data moving between
systems within the same data center will not be encrypted. Further, upon reviewing the Design
Specification Report, officials from the Cybersecurity organization presented a
list of about 17 different security concerns for consideration. These concerns were addressed in the final
Design Specification Report.
Given
the timing of our review and the life cycle phase of the CADE 2 Daily
Processing Project, we could not test whether the identified security controls
were operating as designed. We plan to
evaluate the adequacy of these security controls during a future audit of the
CADE 2 system.
Improvements Are Warranted in
Several Management Areas
Overall, the CADE 2 Daily Processing team is
managing the project successfully.
However, the CADE 2 Daily Processing Project did not ensure: 1) business rules and requirements were always
fully and timely developed in adherence to governing criteria; 2) the
Electronic Fraud Detection System (EFDS) was properly recorded in the Item
Tracking Reporting and Control (ITRAC) System for proper tracking, managing,
and completion of a risk mitigation strategy; and 3) open issues were properly
addressed and closed prior to milestone exits.
Efficient management and development over business rules
are needed
The Enterprise Life
Cycle (ELC)[4] requires that business rules be gathered
and completed during the Logical Design Phase (or in this case, Milestone
3). The CADE 2 Requirement Management
Plan is the primary source for information on the activities, responsibilities, and resources
used to manage, monitor, and control requirements for CADE 2. The Requirement Management Plan states that
requirements traceability is a key component of requirements management. Traceability describes the life of a
requirement from its source through development and actual deployment into
operations. Traceability also enables
requirements change management by helping to more clearly identify the impacts
of changing a requirement.
However, the CADE 2 Daily Processing business rules were not
gathered and completed as required and were still being developed after the
December 2010, Milestone 3 exit. For
example, when the Milestone 3 exit occurred, the business rule that determines
eligibility of accounts for daily processing was not developed. Additionally, prior to the Milestone 4a exit,
16 business rules were not written as required by the ELC. These business
rules related to incomplete operational scenarios such as:
·
Elongated day: The completion of a
daily cycle in a day other than when it was initiated could affect the IRS’s ability
to meet scheduled completion dates for dependent processes (e.g., refund
reviews/payment dates, notice mailings, Integrated Data Retrieval System
updates).
·
Elongated week: When the weekly
processing cycle cannot be completed in time for the Integrated Data Retrieval System
weekend update (completed by Monday 6 a.m.), Tax Examiners will not get posted
account information from the last day of the week or from the weekly cycle, the
weekend Taxpayer Information File analysis will not be run timely, and the
availability of the Integrated Data Retrieval System may be affected.
·
Electronic Fraud
Detection System: The
timing of the daily EFDS inputs is different for the filing of electronic
returns (which is from the prior day) than for the filing of paper returns
(which is from the current day).
Originally, both CADE 2 Milestone 3 and 4a exits were combined into an April 2011 exit, with all activities being structured around one single exit. However, due to budget issues, Milestone 3 was subsequently separated into an exit date of December 2010. Although the CADE 2 Daily Processing ensured key activities and products were identified, business rules were not completed prior to this new Milestone 3 exit. Additionally, new business rules developed after the milestone exit could also require development of additional customer requirements. For example, after the Milestone 3 exit, 225 new business requirements were added (55 exclusive to Daily Processing). Without business rules, customer requirements may not effectively function, thereby impinging CADE 2 system performance. The risk of incomplete business rules could contribute to untraced requirements, which may adversely impact systems design and testing activities. We previously reported on this condition in our review of the CADE 2 Program Management Office[5] and recommended that the Chief Technology Officer ensure all requirements and business rules (including those for the Daily Processing Project) are identified and sufficiently traced, controlled, and managed.
A
risk mitigation plan for the EFDS needs development
The EFDS is a
critical upstream system used to detect potentially fraudulent tax returns and
is considered an impacted system that requires the shift from weekly to daily
processing. The CADE 2 Daily
Processing Project utilizes the ITRAC System
to document and monitor project risks.
The Performance Monitoring and Program Reporting Process states project
risks should be documented and monitored in the ITRAC System. Although the EFDS is an open operating
scenario and a known risk, it is not documented in the ITRAC System. Therefore, a risk mitigation plan has not been
formally developed to mitigate the risk in the timing difference identified for
paper returns (as discussed previously, the timing of the daily EFDS inputs is
different for paper returns than for electronic returns).
From Processing Years 2008 through 2010, the EFDS
identified, on average, 603,179 fraudulent refund returns totaling about $4 billion
and intercepted 518,896 returns totaling about $3.7 billion in fraudulent
refunds. Without a proper EFDS risk
mitigation plan to address the timing issue of paper returns, some fraudulent
returns could go undetected, resulting in a monetary loss for the IRS. Figure 2 presents aggregated (i.e., both
paper and electronic) fraudulent refund return statistics for Processing Years
2008–2010.
Figure 2: Fraudulent Returns and Refunds Identified and Stopped
Processing Years 2008–2010
|
Processing Year |
Number of Fraudulent Refunds
Identified |
Number of Fraudulent Refunds
Stopped |
Amount of Fraudulent Refunds
Identified |
Amount of Fraudulent Refunds
Stopped |
|
2008 |
380,656 |
306,128 |
$1,959,992,377 |
$1,683,912,973 |
|
2009 |
457,369 |
369,257 |
$2,988,945,590 |
$2,517,094,116 |
|
2010 |
971,511 |
881,303 |
$7,300,996,194 |
$6,931,931,314 |
|
Average |
603,179 |
518,896 |
$4,083,311,387 |
$3,710,979,467 |
Source: IRS fraudulent tax return statistics for
Processing Years 2008–2010.
Milestone
exit should have been a conditional exit
The purpose of the Milestone Readiness Review is to
determine whether a project is in compliance with ELC requirements and ready to
begin the milestone exit process. The Milestone
Readiness Review eliminates last minute project delays and rework, and helps
streamline decisions made by the project’s governance organization. The Milestone Readiness Review looks for the
process artifacts that were identified for that phase of development in the CADE
2 Project Tailoring Plan. The CADE 2 Program
Executive Steering Committee performed the Milestone Readiness Reviews and gave
them “Unconditional” exits.[6] However, during the Physical
Design Review on April 7 and 8, 2011, there were open issues at the Milestone
3 exit which the Daily Processing Project team stated would be closed prior to
the formal Milestone 4a exit. The CADE 2
Daily Processing Project team ensured the open issues were tracked. Exiting a milestone without properly
addressing critical open issues could result in rework, potentially impact the
logical and/or physical design, and result in unnecessary costs.
Recommendations
The Chief Technology
Officer should ensure:
Recommendation
1: A risk
mitigation plan is formally developed and documented in the ITRAC System for
any impacted systems, such as the EFDS.
Management’s Response: IRS management agreed with this recommendation. A risk (ITRAC #15137) was opened on May 10, 2011, for Timely Processing of Potential Fraud Data for Paper Returns in the EFDS. A mitigation plan was documented and successfully executed.
Recommendation 2: All
open issues are properly addressed and closed during the Milestone Readiness
Review prior to approving an unconditional exit.
Management’s Response: IRS management disagreed with this recommendation. The IRS guidance does not list specifics around what constitutes an “open issue” at milestone exit, leaving some flexibility for the Executive Steering Committee to make risk-based decisions on whether or not a given open issue is at a stage where it will impact efforts in the following milestones or undermine the success of the project. In the case of CADE 2 Daily Processing, there was full understanding by the CADE 2 Executive Steering Committee at the milestone exit that the open issues had been adequately addressed and were near resolution. Each issue had a viable solution and only needed full alignment by all stakeholders and/or a fully documented solution. While it is clearly optimal for all open issues to be properly addressed and fully closed during the Milestone Readiness Review, the IRS has chosen to leave some flexibility in the hands of its Executive Steering Committees in this area, based on the status of the open issues at milestone exit and the impacts the status will have on the outcome of the project. Currently, all Daily Processing design issues have been resolved and closed.
Office of Audit Comment: Documentation of the April 7 and 8, 2011, design meetings defined the open issues as follows, “technical narrative not completed/design impact still being determined.” Yet, on April 18, 2011, an unconditional exit was approved for the project to proceed to the next milestone. We agree there might need to be some flexibility for the IRS to make risk-based decisions on open issues before proceeding to another milestone. However, in this instance, the design meeting documentation indicated the design impact of the open issues was not known. Exiting a milestone without properly addressing critical open issues could result in rework, potentially impact the logical or physical design, and result in unnecessary costs.
The Daily Processing Project Is Not Following a Consistent Enterprise Life Cycle
Path, and the Work Breakdown Schedule Is Incomplete
The CADE 2 Daily Processing Project Tailoring Plan states
the project selected will follow the ELC Iterative
Path (Figure 3). In the Iterative
Path development life cycle (also called an iterative development model),
projects start with initial planning and end with deployment, with repeated
cycles of requirement discovery, development, and testing. In the ELC Iterative
Path, the components are developed through a series of constrained
iterations as the first iteration focuses on the initial requirements and the
subsequent iterations build on the previous iterations by adding to the
functionality. Characteristics of the Iterative
Path include:
·
Fixed Length Iterations (Milestone 1, 2, 3, 4a, 4b,
and 5). Time boxing[7] introduces a
near-term milestone that forces all stakeholders to converge and actually
deliver working software at regular intervals.
·
Just-In-Time Requirements Elaboration. Project planning depends on identifying the
most important capabilities and characteristics of the system.
·
Early and Continuous Testing. Early and continuous testing will deliver
cohesive, valuable increments of functionality.
Figure 3: Illustration of the Iterative Path
Figure 2 was removed due to
its size. To see Figure 2, please go to
the Adobe PDF version of the report on the TIGTA Public Web Page.
IRS Executives stated the Daily Processing Project
is following the ELC Waterfall Path, contradicting
the Daily Processing Tailoring Plan document which uses the ELC Iterative Path. The Waterfall
Path development model is a sequential design process which flows steadily
downward through phases. The Daily
Processing Project team has successfully utilized the Iterative Path methodology through the Business System Requirements
Report and the Lessons Learned document.
Early and continuous testing is another key characteristic of the Iterative Path; however, development and
testing for Daily Processing were
not accomplished throughout the project or through Milestone 4a. The Integrated Master Schedule reflects CADE
2 Daily Processing testing and development as starting in Milestone 4b. Testing at this stage is consistent with the Waterfall Path. As a result, the CADE 2 Daily
Processing Project is being managed
using a combination of both the Waterfall
and Iterative Paths. An ELC path should be selected and documented
to fully realize intended benefits.
The Work Breakdown Schedule does not comply with all
ELC criteria
The CADE
2 Daily Processing Project
Tailoring Plan identified project deliverables through Milestone 5 (System
Deployment Phase) which were not reflected in the Work Breakdown Schedule or subsequent iterations through Milestone 4a. The Work Breakdown Schedule is a
deliverable-oriented grouping (deliverables) of project elements that organize
and define the total work scope of the project.
The Work Breakdown Schedule must incorporate the entire life cycle,
tailored to represent the scope of the project/release from beginning to end,
to the best of the team’s ability at the time of creation. The ELC
guidance requires the Work Breakdown Schedule to be updated and baselined for
each upcoming milestone phase, prior to exiting the previous milestone. In contrast, the CADE 2 Daily
Processing Project Tailoring Plan
states the project will follow the ELC Iterative
Path, whereby system requirements and functionality continually
evolve. Therefore, the Work Breakdown
Schedule is not consistent with the Project Tailoring. However, to ensure consistency with the ELC, the
CADE 2 Daily Processing Project team has performed a baseline of the Work
Breakdown Schedule at the end of each milestone.
Throughout project
duration, all non-milestone Work Breakdown Schedule activities require a
resource owner to provide activity input.
However, there were 32 (of approximately 700) CADE 2 Daily
Processing related non-milestone
activities included in the November 24, 2010, Work Breakdown Schedule that did
not identify a resource owner. Although
the 32 activities were not listed on the November 24, 2010, critical path, if
completion of an activity is excessively delayed, it can become part of the
critical path.
The CADE 2 Daily
Processing Project team should ensure
all deliverables identified in the CADE 2 Daily Processing Project Tailoring Plan are detailed in the
Work Breakdown Schedule and all activities are appropriately traced and
monitored by a resource owner.
Identifying an activity resource owner helps mitigate the risk of
incomplete or delayed work. Further,
delays in completing activities on the critical path can impede the January
2012 scheduled deployment date. We
previously reported on this condition in our review of the CADE 2 Program
Management Office[8] and recommended that the Chief Technology
Officer ensure all key activities associated with the development and
deployment of the CADE 2 system are included in the Work Breakdown Schedule,
including those for the Daily Processing Project.
Recommendation
The Chief Technology Officer should:
Recommendation
3:
Require
a documented and chosen ELC path for CADE 2 Daily Processing to ensure project
consistency and full realization of the benefits of the selected ELC path.
Management’s Response: IRS management agreed with this recommendation for future projects. The ELC is intended for projects, not programs. There is clear guidance and structure in place at the IRS to guide information technology investments in choosing and following an appropriate ELC path. For Transition State 1, the IRS made a business decision to leverage the iterative approach for the Database Implementation project.
Appendix I
Detailed Objective, Scope, and Methodology
The overall
objective of this review was to assess the logical and physical design of the
CADE 2 Daily Processing activities and ensure the Daily Processing design is
secure, the design satisfies the stated documented approved requirements, and
the project management practices adhere to the ELC[9] standards and processes for the related
milestones (thru Milestone 4a).[10] To
accomplish this objective, we:
I.
Determined
whether the Daily Processing Project management team follows the ELC system
development process to achieve daily processing and replace the current IMF
applications and current CADE applications with a single, state-of-the art
solution.
A.
Reviewed
the project charter to verify whether it contained key information (e.g., impacted
systems, identification of stakeholders).
We also compared the project charter with the CADE 2 program charter to
identify any gaps.
B.
Reviewed
the CADE 2 Daily Processing ELC Tailoring Plan to validate whether the plan includes
all of the required phases of project development per the ELC. We also determined whether the Work Breakdown
Schedule contained key deliverables.
C.
Reviewed
the CADE 2 Daily Processing Business Systems Requirements Report to determine
whether requirements and any subsequent changes were documented and approved.
D.
Reviewed
the CADE 2 Daily Processing Requirements Traceability Matrix to identify any
ongoing or open issues that should be addressed before exiting certain project
milestones. In addition, we traced any
open issues to the ITRAC System to determine whether risks were identified and
tracked.
II.
Reviewed
the Program Test Plan, Project Design Specification Reports, and Program
Interface Inventory, to evaluate whether controls are operating as designed to
mitigate risks with modifying the current processing from weekly to daily.
III.
Determined
whether applications, operating system, and hardware for daily processing meet
the requirements of the National Institute of Standards and Technology security
standards, Federal Information Security Management Act,[11] Office of Management
and Budget Memos, Internal Revenue Manuals, and applicable Federal regulations
and laws.
A.
Evaluated
the Daily Processing System Security Plan against the National Institute of Standards
and Technology and Federal Information Security Management Act requirements.
B.
Reviewed
the results from an independent contractor’s Threat Susceptibility Assessment.
C.
Reviewed
the data encryption plan to ensure data is secured from outside threats.
Internal controls methodology
Internal controls
relate to management’s plans, methods, and procedures used to meet their
mission, goals, and objectives. Internal
controls include the processes and procedures for planning, organizing,
directing, and controlling program operations.
They include the systems for measuring, reporting, and monitoring
program performance. We determined the
following internal controls were relevant to our audit objective: the ELC and related IRS guidelines and the
processes followed in the development of information technology projects. We evaluated these controls by conducting
interviews with management and staff, attending CADE 2 meetings of the Program
and project teams, and reviewing project documentation such as the project
charter, various project plans, and other documents that provided evidence of
whether ELC systems development processes were followed.
Appendix II
Major Contributors to This Report
Alan R. Duncan, Assistant Inspector General for Audit (Security and
Information Technology Services)
Diana Tengesdal, Acting Director
Kimberly R. Parmley, Audit Manager
K. Kevin Liu, Lead Auditor
Ryan M. Perry, Senior Auditor
Frank O’Connor, Program Analyst
Michael T. Mohrman, Information Technology Specialist
Appendix III
Commissioner C
Office of the Commissioner – Attn: Chief of Staff C
Deputy Commissioner for Operations Support OS
Deputy Commissioner for Services and Enforcement SE
Chief, Agency-Wide Shared Services OS:A
Commissioner, Wage and Investment Division SE:W
Deputy Chief Information Officer for Strategy/Modernization OS:CTO
Associate Chief Information Officer, Modernization – Program Management Office OS:CTO:MP
Chief Counsel CC
National Taxpayer Advocate TA
Director, Office of Legislative Affairs CL:LA
Director, Office of Program Evaluation and Risk Analysis RAS:O
Director, Procurement OS:A:P
Director, Risk Management Division OS:CTO:SP:RM
Office of Internal Control OS:CFO:CPIC:IC
Audit
Liaisons:
Commissioner,
Wage and Investment Division
SE:W:S
Director, Risk
Management Division
OS:CTO:SP:RM
Appendix IV
Enterprise Life Cycle Overview
The ELC is the IRS’s
standard approach to business change and information systems initiatives. It is a collection of program and project
management best practices designed to manage business change in a successful and
repeatable manner. The ELC addresses
large and small projects developed internally and by contractors. The ELC includes such requirements as:
·
Development
of and conformance to enterprise architecture.
·
Improving
business processes prior to automation.
·
Use of
prototyping and commercial software, where possible.
·
Obtaining
early benefit by implementing solutions in multiple releases.
·
Financial
justification, budgeting, and reporting of project status.
In addition, the ELC
improves the IRS’s ability to manage changes to the enterprise, estimate the
cost of changes, and engineer, develop, and maintain systems effectively. Figure 1 provides an overview of the phases
and milestones within the ELC. A phase is
a broad segment of work encompassing activities of similar scope, nature, and
detail and providing a natural breakpoint in the life cycle. Each phase begins with a kickoff meeting and
ends with an executive management decision point (milestone) at which IRS
executives make “go/no-go” decisions for continuation of a project. Project funding decisions are often
associated with milestones.
Figure 1: Enterprise Life Cycle
Phases and Milestones
|
Phase |
General
Nature of Work |
Milestone |
|
Vision and Strategy/ |
High-level
direction setting. This is the only phase
for enterprise planning projects. |
0 |
|
Project Initiation Phase |
Startup
of development projects. |
1 |
|
Domain Architecture Phase |
Specification
of the operating concept, requirements, and structure of the solution. |
2 |
|
Preliminary Design Phase |
Preliminary
design of all solution components. |
3 |
|
Detailed Design Phase |
Detailed
design of solution components. |
4a |
|
System Development Phase |
Coding,
integration, testing, and certification of solutions. |
4b |
|
System Deployment Phase |
Expanding
availability of the solution to all target users. This is usually the last phase for
development projects. |
5 |
|
Operations and Maintenance Phase |
Ongoing
management of operational systems. |
System Retirement |
Source: The
Enterprise Life Cycle Guide.
Appendix V
Transition State 1 Integration Reviews
|
Integration Review |
Outcomes |
|
Integrated
Management Planning Review |
Validates that
relationships and integration expectations between the Program and its
projects are appropriately defined and well understood. |
|
Integrated
Requirements Reviews |
Validates that the
Program-level requirements allocated to projects are in alignment with the
Program solution; that Program-level requirements have been appropriately
fulfilled through decomposition to project-level requirements; and that all
requirements dependencies are identified, supported, and fulfilled. |
|
Integrated
Solution Planning Review |
Validates that
Program strategies for solution design are aligned for the Transition State
solution. |
|
Integrated
Logical Design Review |
Validates that the
project-level designs support the solution’s logical implementation as
defined in the Program Roadmap and that the projects collectively will
deliver an integrated and cohesive solution. |
|
Integrated
Physical Design Review |
Validates that the
project-level designs support the solution’s physical implementation as
defined in the Program Roadmap and that the projects collectively will
deliver an integrated and cohesive solution. |
|
Integrated Test
Planning Review |
Validates that the
Program and project plans for testing the solution components individually
and the integrated Program solutions collectively are in alignment and
comprehensive. |
|
Integrated Test
Readiness Review |
Validates that
solution components have been accurately and comprehensively tested at the
Unit and Developer level and that the Program is ready to begin testing of
the Integrated Transition State Solution. |
|
Integrated
Organizational Readiness Review |
Validates that the
Program understands the impact the solution has on the business, and
validates organizational readiness to adopt the new solution. |
|
Integrated
Deployment Readiness Review |
Validates that the
solution components, production environment, and plans for deployment are
assessed against defined readiness criteria.
The Program will make a “go/no-go” decision based on the results of
the Deployment Readiness Review. |
Source: IRS Customer Account Data Engine 2, 2nd
Quarter Briefing to the Treasury Inspector
General for Tax Administration officials, dated July 14, 2010.
Appendix VI
|
Term |
Definition |
|
Business Rule |
A statement
that defines or constrains some aspect of the business (see Business Rule Sets). |
|
Business Rule
Sets |
A group of
business rules related to a common topic or business decision. |
|
Customer
Account Data Engine |
A major
component of the IRS modernization program.
The system consists of current and planned databases and related
applications that work with the IRS Master File system. |
|
Daily
Processing Project |
A project
under the CADE 2 Program that, when completed, will change weekly individual
taxpayer account processing to daily processing. |
|
Database
Implementation Project |
A project
under the CADE 2 Program intended to implement the
newest version of the relational database. |
|
Enterprise
Architecture |
A unifying overall design or structure for an
enterprise that includes business and organizational aspects as well as
technology aspects. It divides the
enterprise into its component parts and relationships and provides the
principles, constraints, and standards to help align business area
development efforts in a common direction.
Additionally, it ensures subordinate architectures, business system
components, and multiple projects fit together into a consistent, integrated
whole. |
|
Enterprise
Life Cycle |
A structured business systems development
method that requires the preparation of specific work products during
different phases of the development process. |
|
Executive
Steering Committee |
Committee with
oversight responsibilities for investments, including validating major
investment business requirements and ensuring enabling technologies are
defined, developed, and implemented. |
|
Individual
Master File |
The IRS
database that maintains transactions or records of all individual tax
accounts. |
|
Infrastructure |
The fundamental structure of a system or
organization. The basic, fundamental
architecture of any system (e.g., electronic, mechanical, social, political)
determines how it functions and how flexible it is to meet future
requirements. |
|
Integrated
Data Retrieval System |
The IRS
computer system capable of retrieving or updating stored information; it
works in conjunction with a taxpayer’s account records. |
|
Integrated
Production Model |
Intended to be
a data store to meet IRS needs for data analytics and long-term reporting and
a source for other types of analytic data that supplement the transactional
core data store. |
|
Item Tracking
Reporting and Control System |
An
information system used to track and report on issues, risks, and action
items in the modernization effort. |
|
Master File |
The IRS database that stores various types of taxpayer
account information. This database
includes individual, business, and employee plans and exempt organizations
data. |
|
Milestone |
Scheduled time period for providing a go/no-go
decision point in a program or project.
It can be associated with funding approval to proceed. |
|
National
Institute of Standards and Technology |
A
nonregulatory Federal agency, within the Department of Commerce, responsible
for developing standards and guidelines, including minimum requirements for
providing adequate information security for all Federal Government agency
operations and assets. |
|
Phase |
Broad segment
of work encompassing activities of similar scope, nature, and detail and
providing a natural breakpoint in the life cycle. |
|
Processing
Year |
The calendar year in which the tax return or document
is processed by the IRS. |
|
Project
Tailoring Plan |
ELC Guidance 2.16.1 (pages 66-67) states ELC tailoring must be
documented and approved in a Project Tailoring Plan. This plan must identify all tailoring
decisions and explain the rationale and impact of each decision. Impact should include a discussion of any
risk associated with the change. The
approved tailoring decisions must be reflected in the project Work Breakdown
Structure and the project schedule. All
approved waivers and deferrals for tailoring decisions must be maintained in
the projects repository. |
|
Relational
Database |
A
collection of data items organized as a set
of formally described tables from which data can be accessed or reassembled
in many different ways without having to reorganize the database tables. |
|
Requirement |
A formalization of a need and statement of
a capability or condition that a system must have or meet to satisfy a
contract, standard, or specification. |
|
Stakeholders
|
An individual or organization that is materially
affected by the outcome of the system.
Key stakeholders represent both business and technical functions that
fully participate in the architecture development effort to ensure that directional
guidance is both accurate and sufficient.
They are empowered to make project and architectural decisions. Examples of project stakeholders include
the customer, the user group, the project manager, the development team, and the
testers. |
|
Work Breakdown
Schedule |
A deliverable-oriented grouping of project elements that organize and define total work scope of the project. |
Appendix VII
Management’s Response to the
Draft Report
DEPARTMENT OF
THE TREASURY
INTERNAL
REVENUE SERVICE
WASHINGTON,
D.C. 20224
CHIEF
TECHNOLOGY OFFICER
September 13, 2011
MEMORANDUM
FOR DEPUTY INSPECTOR GENERAL FOR AUDIT
FROM: Terence V. Milholland
/s/ Terence V. Milholland
Chief Technology Officer
SUBJECT: The Customer Account Data
Engine 2 Is Making Progress Toward Achieving Daily Processing, but Improvements
Are Warranted to Ensure Full Functionality (201120001) (e-trak
2011-24369)
Thank you for
the opportunity to review your draft audit report and to discuss earlier draft
report observations with the audit team.
I was pleased
to read your comments and observations acknowledging steady progress on the
CADE 2 Daily Processing Project from project initiation (Milestone 1) through Physical
Design (Milestone 4a). The report also recognizes that the IRS has documented
security requirements for Daily Processing and will use several types of secure
file transfer protocols to protect data from unauthorized access, modification,
and corruption. However, based on performance considerations, we have made a
risk-based decision not to encrypt a small number of machine to machine data
transfers within our Tier 5 Enterprise Computing Centers given that the risk is
minimal.
I agree with
recommendations 1 and 3, and the attachment details our planned actions to
implement. I do not, however, agree with your recommendation around open issues
as it relates to unconditional milestone exits. IRS guidance does not list
specifics around what constitutes an "open issue" at milestone exit,
leaving some flexibility for the Executive Steering Committees (ESCs) to make
risk-based decisions on whether or not a given "open issue" is at a
stage where it will impact efforts in the following milestones or undermine the
success of the project. In the case of CADE 2 Daily Processing, there was full
understanding by the CADE 2 ESC at the milestone exit that the "open
issues" had been adequately addressed and were near resolution. Each issue
had a viable solution and only needed full alignment by all stakeholders and/or
a fully documented solution. While it is clearly optimal for all "open
issues" to be properly addressed and fully closed during the Milestone
Readiness Review, I have chosen to leave some flexibility in the hands of its
ESCs to make risk based choices, based on the status of the "open
issues" at milestone exit and the impacts the status will have on the
outcome of the Program.
We are
committed to continuously improving our information technology systems and processes.
We value your continued support and the assistance and guidance your team
provides. If you have any questions, please contact me at (202) 622-6800 or Karen
Mayr at (202) 283-0015.
Attachment
RECOMMENDATION #1: The Chief Technology Officer should ensure,
a risk mitigation plan is formally developed and documented in the ITRAC System
for any impacted systems, such as the EFDS.
CORRECTIVE ACTION #1: The IRS agrees with this recommendation. A
risk (ITRAC #15137) was opened on 5/10/2011 for Timely Processing of Potential
Fraud Data for Paper Returns (EFDS). A mitigation plan was documented and
successfully executed.
IMPLEMENTATION DATE: Completed August 4, 2011
RESPONSIBLE OFFICIAL: Associate Chief Information Officer,
Modernization and CADE2 PMO
CORRECTIVE ACTION MONITORING PLAN: We enter accepted Corrective Actions into
the Joint Audit Management Enterprise System (JAMES). These Corrective Actions
are monitored on a monthly basis until completion.
RECOMMENDATION #2: The Chief Technology Officer should ensure
all open issues are properly addressed and closed during the Milestone
Readiness Review prior to approving an unconditional exit.
CORRECTIVE ACTION #2: The IRS does not agree with this
recommendation. IRS guidance does not list specifics around what constitutes an
"open issue" at milestone exit, leaving some flexibility for the
Executive Steering Committees (ESC) to make risk-based decisions on whether or
not a given "open issue" is at a stage where it will impact efforts
in the following milestones or undermine the success of the project. In the
case of CADE 2 Daily Processing, there was full understanding by the CADE 2 ESC
at the milestone exit that the "open issues" had been adequately
addressed and were near resolution. Each issue had a viable solution and only
needed full alignment by all stakeholders and/or a fully documented solution.
While it is clearly optimal for all "open issues" to be properly
addressed and fully closed during the Milestone Readiness Review, the IRS has
chosen to leave some flexibility in the hands of its ESCs in this area, based
on the status of the "open issues" at milestone exit and the impacts
the status will have on the outcome of the project. Currently, all DP design
issues have been resolved and closed.
IMPLEMENTATION DATE: Closed August 24, 2011
RESPONSIBLE OFFICIAL: Associate Chief Information Officer,
Modernization and CADE2 PMO
CORRECTIVE ACTION MONITORING PLAN: We enter accepted Corrective Actions into
the Joint Audit Management Enterprise System (JAMES). These Corrective Actions
are monitored on a monthly basis until completion.
RECOMMENDATION #3: The Chief Technology Officer should require
a documented and chosen ELC path for CADE 2 Daily Processing to ensure project
consistency and full realization of the benefits of the selected ELC path.
CORRECTIVE ACTION #3: The IRS agrees with this recommendation for
future projects. The ELC is intended for projects, not programs. There is clear
guidance and structure in place at the IRS to guide IT investments in choosing
and following an appropriate ELC path. However, for TS-1 the IRS made a
business decision to leverage an iterative approach for the Database
Implementation project.
IMPLEMENTATION DATE: Not Applicable.
RESPONSIBLE OFFICIAL: Associate Chief Information Officer,
Modernization and CADE 2 PMO
CORRECTIVE ACTION MONITORING PLAN: We enter accepted Corrective Actions into the Joint Audit Management Enterprise System (JAMES). These Corrective Actions are monitored on a monthly basis until completion.
[1] See Appendix VI for a glossary of terms.
[2] See Appendix V Transition State 1 Integration Reviews.
[3] The Customer Account Data Engine 2 Program
Management Office Implemented Systems Development Guidelines; However, Process
Improvements Are Needed to Address Inconsistencies (Audit Number 201020025,
draft report issued August 11, 2011), and The Customer Account
Data Engine 2 Database Implementation Project Made Progress in Design
Activities, but Improvements Are Needed (Reference Number 2011-20-110, dated September 20, 2011).
[4] Refer to Appendix IV for an overview of the ELC.
[5] The Customer
Account Data Engine 2 Program Management Office Implemented Systems Development
Guidelines; However, Process Improvements Are Needed to Address Inconsistencies
(Audit Number 201020025, draft report issued August 11, 2011).
[6] This means the project has addressed all known risks or conditions.
[7] Architecture development is often time-boxed with an agreement to pass the architecture to the development team after a set period of time as opposed to leaving completion unbounded.
[8] The Customer Account Data Engine 2 Program Management Office Implemented Systems Development Guidelines; However, Process Improvements Are Needed to Address Inconsistencies (Audit Number 201020025, draft report issued August 11, 2011).
[9] See Appendix VI for a glossary of terms.
[10] See Appendix IV for an overview of the ELC.
[11] 44 U.S.C. §§ 3541–3549.