Annual Assessment of the Business Systems Modernization
Program
September 2003
Reference
Number: 2003-20-208
This report has cleared the Treasury
Inspector General for Tax Administration disclosure review process and
information determined to be restricted from public release has been redacted
from this document.
September
29, 2003
MEMORANDUM FOR
CHIEF INFORMATION OFFICER
FROM: Gordon C. Milbourn III /s/ Gordon C.
Milbourn III
Assistant Inspector General for Audit (Small Business and
Corporate Programs)
SUBJECT: Final Audit Report - Annual Assessment of
the Business Systems Modernization Program (Audit # 200320029)
This
report presents the results of our assessment of the progress of the Business
Systems Modernization (BSM) program.
The basis for our assessment is the audit work we have conducted during
Fiscal Year (FY) 2003, as well as reports issued by the General Accounting
Office (GAO). This report also
addresses the requirement in the Internal Revenue Service (IRS) Restructuring
and Reform Act of 1998 that we annually evaluate the adequacy of the IRS’
information technology.
In summary, the BSM program
is in the fifth year of its effort to upgrade and modernize the IRS’
information technology and business systems.
It is expected that BSM will take up to 15 years and cost at least $7
billion to complete. The BSM program is
an extremely complex effort, since many of the IRS’ current business systems
are a mixture of technologies that date back to the 1960s. The GAO has designated the BSM program as a high-risk
area. While difficult, the BSM program must nevertheless be successful if the
IRS is to meet its goals and commitments of improving customer service to
taxpayers. To facilitate success of its
modernization efforts, the IRS hired the Computer Sciences Corporation as the
PRIME contractor and integrator for BSM, and created the Business Systems
Modernization Office (BSMO) to guide and oversee the work of the PRIME
contractor. Since 1999, about $1.35
billion has been appropriated for the BSM program.
The BSMO has made progress
in defining the management processes and capabilities needed to effectively
acquire and implement information technology systems. For example, the BSMO reached its goal of being rated at Level 2
of the Software Engineering Institute’s Capability Maturity Model for software
acquisition. This designation indicates
that the BSMO has documented a set of repeatable, disciplined management
processes that increase the probability of success for the BSM program. The
BSMO has also tasked the PRIME contractor to
build a program-wide cost and schedule estimation system that is compliant with
best practices in the Information Technology industry, and has created a process for independently reviewing
cost and schedule estimates.
The BSMO has
deployed the first release of the infrastructure system on which future BSM applications will
run. Establishing this infrastructure
is a necessary prerequisite to introducing the business applications that are
intended to provide benefits to taxpayers and the IRS. The BSMO also deployed two applications that
immediately produced taxpayer benefits.
The Internet Refund application, upgraded in 2003, was accessed over 12 million
times by taxpayers. The Internet Employer Identification Number application allows
businesses and taxpayers to obtain employer identification numbers online,
eliminating the need to send paperwork to the IRS.
In November 2002, the IRS requested $451 million in its FY 2003
BSM Spending Plan. We, along with the IRS Oversight Board and
the GAO, expressed concerns over the BSMO’s ability to effectively manage its
existing portfolio of projects.
Specifically, the imbalance between the number and pace of the BSM
projects, and available management capabilities, added significant cost,
schedule, and performance risks that continued to escalate as the program
advanced. To address these concerns,
the BSMO scaled back both the scope and number of projects. In March 2003, the IRS submitted a revised FY 2003 BSM Spending Plan
for $378 million. This plan focused the BSMO on a smaller portfolio of existing key
projects, such as the Customer Account Data Engine, Integrated Financial
System, and e-Services, while reducing the planned scope and pace of the FY
2003 BSM program.
Although the BSMO expressed
high confidence in the practicality of the revised plan and assured the
Congress that it could timely deliver the revised FY 2003 project portfolio,
all of the projects experienced schedule delays and deferred capabilities, and
most incurred significant cost increases from FY 2002 estimates. The deferred capabilities, schedule delays, and cost
increases occurred, in part, because defined management and project development
processes have not always been followed by the BSM project teams. In addition, delivery schedules and cost
estimates developed by the BSMO and the
PRIME contractor were very aggressive and overly optimistic. The BSMO and the PRIME contractor have
particularly struggled to develop adequate cost and schedule estimation techniques,
and have not fully implemented disciplined project testing processes and
procedures. While progress has been
made in the IRS’ modernization efforts, the BSMO did not achieve its goals for FY 2003.
This underachievement is disappointing considering that the expectations
for the year were scaled back in hopes of being able to successfully deliver
several key BSM projects.
In September 2002, we
provided an assessment of the progress of the BSM program that cited three
challenges for the BSMO and the PRIME contractor to overcome to be successful
in FY 2003: 1) implement planned
improvements in key management processes and commit necessary resources to
enable success, 2) manage the increasing complexity and risks of the BSM
program, and 3) maintain the continuity of strategic direction with experienced
leadership. We believe these challenges
still exist and have contributed to the problems encountered by the BSMO and
the PRIME contractor in FY 2003.
In addition, we believe the
BSMO is facing a fourth significant challenge.
Because of the continued problems confronting the BSM program, oversight
groups are starting to lose confidence in the PRIME contractor’s ability to
meet its commitments in modernizing the IRS’ business systems, and have raised
concerns about future funding. In light
of this concern, effective contract management, always difficult on a project
of this magnitude, is becoming an increasingly important challenge that needs
to be overcome. Specifically, the BSMO
must ensure PRIME contractor performance and accountability are effectively
managed.
Meeting these challenges is
essential for the BSMO and the PRIME contractor to successfully deliver the BSM
program. The BSMO and the PRIME
contractor have made progress in improving their management capabilities, but
they must improve their performance and avoid the pitfalls of the past while
balancing taxpayer benefits, business priorities, and internal management needs
to successfully execute the large and complex BSM program.
Management’s
Response: Management's
response was due on September 26, 2003.
As of September 26, 2003, management had not responded to the draft
report.
Copies of this report are also being sent to the IRS
managers affected by the report. Please contact
me at (202) 622-6510 if you have questions or Margaret E. Begg, Assistant
Inspector General for Audit (Information Systems Programs), at (202) 622-8510.
Progress
Has Been Made on the Business Systems Modernization Program
Program Accomplishments
Did Not Meet Expectations
Non-Compliance
With Existing Management Processes Contributed to Underachieved Program Goals
Previously Reported Program Challenges Continue
to Confront Modernization Efforts
Appendix I – Detailed Objective, Scope, and Methodology
Appendix II – Major Contributors to This Report
Appendix III – Report Distribution List
The Internal Revenue Service (IRS) Restructuring and Reform Act of 1998 requires the Treasury Inspector General for Tax Administration (TIGTA) to annually evaluate the adequacy and security of the IRS’ information technology. This report provides our assessment of the IRS’ Business Systems Modernization (BSM) program for Fiscal Year (FY) 2003.
The BSM program is in the fifth year of its effort to
upgrade and modernize the IRS’ information technology and business
systems. It is expected that BSM will
take up to 15 years and cost at least $7 billion to complete. The
BSM program is an extremely complex effort, since much of the IRS’ current
business systems are a mixture of technology that dates back to the 1960s. While difficult, the BSM program must be
successful if the IRS is to meet its goals and commitments of improving customer
service to taxpayers. To facilitate success of its modernization efforts, the
IRS hired the Computer Sciences Corporation as the PRIME contractor and
integrator for the BSM program, and created the Business Systems Modernization
Office (BSMO) to guide and oversee the work of the PRIME contractor. Since 1999, about $1.35 billion has been
appropriated for the BSM program.
The BSM program receives extensive scrutiny from various
oversight groups. In addition to our
work, the IRS Oversight Board, the Office of Management and Budget, the General
Accounting Office (GAO), and the Congress monitor the IRS’ modernization
efforts to ensure the successful implementation of the BSM program. The GAO has designated the IRS’
modernization program as a high-risk area.
Our assessment of the BSM program includes information derived from TIGTA and GAO audit reports issued during FY 2003. Detailed information on our audit objective, scope, and methodology, including a listing of previously issued reports, is presented in Appendix I. Each of these audits was conducted in accordance with Government Auditing Standards. Major contributors to the report are listed in Appendix II.
The BSMO has made progress in identifying the modernization management controls needed to effectively acquire and implement information technology systems. For example, the BSMO reached its goal of being rated at Level 2 of the Software Engineering Institute’s Capability Maturity Model for software acquisition. This designation indicates that the BSMO has documented a set of repeatable, disciplined management processes that increase the probability of success for the BSM program. The BSMO has also tasked the PRIME contractor to build a program-wide cost and schedule estimation system that is compliant with best practices in the Information Technology industry, and has created a process for independently reviewing cost and schedule estimates. Focusing on the discipline of following and improving defined management processes will improve the maturity of the project teams and will significantly improve the chances of the BSM program being successful.
In
May 2002, the BSMO put into operation the first release of the infrastructure
system on which future business applications will run.
Establishing the Security and Technology Infrastructure Release was a
necessary prerequisite to introducing the business applications that are
intended to provide benefits to taxpayers and the IRS.
The BSMO also
delivered two applications that immediately produced taxpayer benefits. The
Internet Refund application, upgraded in 2003, was accessed over 12 million
times by taxpayers. The Internet Employer Identification Number
application allows businesses and taxpayers to obtain employer identification
numbers online, eliminating the need to send paperwork to the IRS.
While progress has been made in the IRS’ modernization efforts, the BSMO did not achieve its goals for FY 2003. This underachievement is disappointing considering that the expectations for the year were scaled back in hopes of being able to successfully deliver several key BSM projects.
In November 2002,
the IRS requested
$451 million in its FY 2003 BSM Spending Plan.
We, along with the IRS Oversight Board and the GAO, expressed concerns
over the BSMO’s ability to effectively manage its existing portfolio of
projects. Specifically, the imbalance
between the number and pace of
the BSM projects, and available management capabilities, added significant
cost, schedule, and performance risks that continued to escalate as the program
advanced. To address these concerns, the
BSMO scaled back both the scope and number of projects.
In March 2003, the IRS submitted a revised FY 2003 BSM Spending Plan for $378 million. This plan focused the BSMO on a smaller portfolio of existing key projects, such as the Customer Account Data Engine (CADE), Integrated Financial System (IFS), and e-Services, while reducing the planned scope and pace of the FY 2003 BSM program.
The BSMO expressed high confidence in the practicality of the revised plan and assured the Congress that it could timely deliver the projects in FY 2003. However, all of the projects experienced schedule delays and deferred capabilities, and most incurred significant cost increases from FY 2002 estimates.
The following two tables include the five major BSM projects (Infrastructure Shared Services (ISS), CADE, IFS, Custodial Accounting Project (CAP), and e-Services) that received funding in FY 2003 and show the delivery delays and cost increases. The deferred capabilities, schedule delays, and cost increases occurred, in part, because defined management and project development processes have not always been followed by the BSM project teams. In addition, delivery schedules and cost estimates developed by the BSMO and the PRIME contractor were very aggressive and overly optimistic.
Table 1 –
Changes in Project Delivery Schedules
|
BSM |
Planned Delivery
Date |
Revised |
Delay |
|
ISS |
12/2002 |
9/2003 |
9 months |
|
CADE |
1/2003 |
3/2004 |
14 months |
|
IFS |
10/2003 |
11/2003 |
1 month |
|
CAP |
1/2003 |
11/2003 |
10 months |
|
e-Services |
12/2002 |
8/2003 |
8 months |
Sources:
FY 2001 BSM Spending Plan; GAO Report on BSM (GAO-02-356, dated February
2002); and minutes from the July 25, 2003, Customer Relations Management
Sub-Executive Steering Committee Meeting and the August 4, 2003, PRIME/IRS
Delivery Integrated Conference Call.
The revised delivery dates in the above table were estimates
developed as of August 2003, but we expect that some of the dates will change
again. For example, the CADE Release 1
was initially targeted for delivery in January 2002. Since then, the CADE project has experienced multiple setbacks
and delays, and the latest revised delivery date has not yet been set. In fact, because of the significant problems
encountered with the CADE project, the IRS Commissioner has asked an
independent contractor to conduct a review
to evaluate the project’s progress and determine if changes or
corrections are needed.
The IFS has also encountered significant setbacks and will not be delivered when originally planned. The BSMO and the PRIME contractor are in the process of negotiating revised project schedules for both the CADE and the IFS and may need to revise other project schedules as the planned delivery dates draw near.
Table 2 – Changes in Project Costs ($ Millions)
|
BSM Projects |
Estimated |
Revised |
Change |
Percent |
|
ISS |
$
39.7 |
$
48.8 |
$
9.1 |
23
% |
|
CADE |
$ 51.1 |
$ 50.2 |
$ (0.9) |
(2) % |
|
IFS |
$ 17.3 |
$ 23.0 |
$ 5.7 |
33 % |
|
CAP |
$ 51.4 |
$ 87.1 |
$ 35.7 |
69 % |
|
e-Services |
$ 40.2 |
$ 69.0 |
$ 28.8 |
72 % |
Source: GAO Briefing on Revised FY 2003 BSM Funding Request, April 2003.
The BSMO maintains an integrated set of management processes that cover the entire project life cycle from vision and strategy through development and deployment. However, the BSM project teams did not always follow these existing management processes. The BSMO and the PRIME contractor have particularly struggled to develop adequate cost and schedule estimation techniques and have not fully implemented disciplined project testing processes and procedures. During the past year, we reported that cost and schedule estimates were not well developed, and we identified several concerns with the project testing practices followed by the BSMO and the PRIME contractor.
Another contributing factor for the unachieved program goals is that the BSM program has experienced significant leadership changes due to executive turnover. For example, the former Deputy Commissioner for Modernization & Chief Information Officer (CIO) left the IRS in April 2003, and was replaced with two existing IRS executives. The former Chief Financial Officer was named as CIO (the fourth CIO in the 5 years of the BSM program), and he reports to a newly created Deputy Commissioner for Operations and Support (the former Wage & Investment Division Commissioner). With the hiring of each new set of executives, lessons learned from past mistakes take on added importance to achieve program success.
Established testing processes are not being followed
Testing processes are part of the overall controls to help ensure that systems perform their intended functions accurately and reliably. Equally important, focusing on the discipline of following and improving defined testing processes will improve the maturity of the project teams and will significantly improve the chances of the BSM program being successful.
Testing processes have been substantially revised and refined based on lessons learned during the early testing efforts for BSM projects. However, these enhancements have not been fully implemented by the BSM project teams. Additional controls and further improvements are needed to ensure that BSM project teams incorporate and follow the defined testing processes and practices.
The cost and schedule estimation process has been improved, but additional actions can be taken
The BSM program has been plagued with cost and schedule overruns. The March 2003 BSM Spending Plan acknowledged that most of the BSM initiatives or project milestones experienced cost increases and/or schedule delays exceeding 10 percent of the estimated cost and duration stated in the November 2001 BSM Spending Plan.
Until recently, the PRIME contractor’s cost and schedule estimation process was decentralized at the project level, and the BSMO had not independently reviewed cost and schedule estimates or the PRIME contractor’s estimation system. Recently, the BSMO and the PRIME contractor have made progress in establishing a program-wide cost and schedule estimation system by developing estimation procedures and beginning other estimation activities. Further improvements are needed to effectively capture cost and schedule data, document estimation processes, and monitor contractor compliance.
Overall, FY
2003 has not been as successful a year for the BSMO as it had envisioned. Projects accomplished during the year
focused on customer service. These
projects automated manual processes without retiring any legacy systems. In addition, anticipated enhancements have
not been made to the tax administration process. While the BSMO
and the PRIME contractor have deployed several projects, the successes have
been more the result of the dedication and heroics of the project team members,
rather than from the discipline of following established and effective
processes.
In September 2002, we provided an assessment of the progress of the BSM program. This report cited three challenges for the BSM program to overcome to be successful in FY 2003: 1) implement planned improvements in key management processes and commit necessary resources to enable success, 2) manage the increasing complexity and risks of the BSM program, and 3) maintain the continuity of strategic direction with experienced leadership. We believe these challenges still exist and have contributed to the problems encountered by the BSMO and the PRIME contractor in FY 2003.
In addition, we believe the BSMO is facing a fourth significant challenge. Because of the continued problems confronting the BSM program, oversight groups are starting to lose confidence in the PRIME contractor’s ability to meet its commitments in modernizing the IRS’ business systems, and have raised concerns about future funding. In light of this concern, effective contract management, always difficult on a project of this magnitude, is becoming an increasingly important challenge that needs to be overcome. Specifically, the BSMO must ensure contractor performance and accountability are effectively managed.
Meeting these challenges is essential for the BSMO and the PRIME contractor to successfully deliver the BSM program. The BSMO and the PRIME contractor have made progress in improving their management capabilities, but they must improve their performance and avoid the pitfalls of the past while balancing taxpayer benefits, business priorities, and internal management needs to successfully execute the large and complex BSM program.
Management’s Response: Management's response was due on September 26, 2003. As of September 26, 2003, management had not responded to the draft report.
Appendix I
Detailed Objective, Scope,
and Methodology
The overall objective of this review was to provide our assessment of the progress of the Business Systems Modernization program. The basis for our assessment is the audit work we have conducted during Fiscal Year (FY) 2003, as well as reports issued by the General Accounting Office (GAO). This report also addresses the requirement in the Internal Revenue Service (IRS) Restructuring and Reform Act of 1998 that we annually evaluate the adequacy of the IRS’ information technology.
We
analyzed the following audit reports issued during FY 2003:
· Analysis of Business Systems Modernization Cost, Schedule, and Functionality Performance (Reference Number 2003-20-007, dated October 2002).
· Improvements in the Customer Account Data Engine Pilot Plan Need to Be Considered to Help Ensure the Pilot’s Success (Reference Number 2003-20-018, dated November 2002).
·
U.S.
General Accounting Office, Performance and Accountability Series: Major Management Challenges and Program
Risks, Department of the Treasury (GAO-03-109, dated January 2003).
·
Enhancements to the Internet Refund Project Need to
be Completed to Ensure Planned Benefits to Taxpayers Are Realized (Reference
Number 2003-20-053, dated February 2003).
·
The Business Systems Modernization Quality Assurance
Function Has Established a Solid Set of Policies and Procedures That Can Be
Further Enhanced (Reference Number 2003-20-067, dated February 2003).
·
Adhering to Established Development Guidelines Will
Help to Ensure the Customer Account Data Engine Meets Expectations (Reference
Number 2003-20-089, dated March 2003).
·
U.S. General Accounting Office, IRS Modernization -
Continued Progress Necessary For Improving Service to Taxpayers and Ensuring
Compliance (GAO-03-796T, dated May 2003).
· U.S. General Accounting Office, Business Systems Modernization: IRS Has Made Significant Progress In Improving Its Management Controls, But Risks Remain (GAO-03-768, dated June 2003).
·
Security
Testing and Certification of the Modernized Infrastructure Needs to Be
Strengthened (Reference Number 2003-20-127, dated June 2003).
·
Improvements
to the Modernized Infrastructure Are Needed to Support the Deployment of
Business Systems Modernization (Reference Number 2003-20-161, dated August 2003).
·
Testing
Practices for Business Systems Modernization Projects Need Improvement (Reference Number 2003-20-178, dated
September 2003).
·
The Cost and
Schedule Estimation Process for the Business Systems Modernization Program Has
Been Improved, but Additional Actions Should Be Taken (Reference Number 2003-20-219, dated
September 2003).
·
Review of the
IRS’ e-Services Release 1 Development (Audit
Number 200320023).
·
Review of the Integration and Deployment of the
Custodial Accounting Project, Release 1 (Audit Number 200320025).
·
Review of the Development of the IRS’
Integrated Financial System Release 1(Audit Number 200320038).
Appendix II
Major Contributors to This Report
Margaret E. Begg, Assistant Inspector General for Audit
(Information Systems Programs)
Scott A. Macfarlane, Director
Edward
A. Neuwirth, Audit Manager
James Douglas, Senior Auditor
Bruce Polidori, Senior Auditor
Appendix III
Commissioner C
Deputy Commissioner for Operations Support OS
Associate Commissioner, Business Systems Modernization OS:CIO:B
Chief, Information Technology Services OS:CIO:I
Deputy Associate Commissioner, Program Management OS:CIO:B:PM
Deputy Associate Commissioner,
Systems Integration OS:CIO:B:SI
Acting Director, Portfolio Management Division OS:CIO:R:PM
Chief
Counsel CC
National Taxpayer Advocate TA
Director, Office of Legislative Affairs CL:LA
Director, Office of Program Evaluation and
Risk Analysis RAS:O
Office of Management Controls OS:CFO:AR:M
Audit Liaison:
Associate Commissioner, Business Systems Modernization OS:CIO:B
Chief, Information Technology Services
OS:CIO:I