Management Advisory Report:
Progress Has Been Made in Developing Transition to Support Guidance for
Modernization Projects
August 2002
Reference
Number: 2002-20-146
This report has cleared the Treasury Inspector
General for Tax Administration disclosure review process and information determined
to be restricted from public release has been redacted from this document.
August
2, 2002
MEMORANDUM FOR
DEPUTY COMMISSIONER FOR MODERNIZATION
&
CHIEF INFORMATION OFFICER
FROM: Pamela J. Gardiner /s/ Pamela J. Gardiner
Deputy Inspector General for
Audit
SUBJECT: Final Management Advisory Report - Progress Has Been Made in Developing
Transition to Support Guidance for Modernization Projects (Audit # 200220014)
This report presents the results of our
review of the process for transferring
support and maintenance of deployed modernization projects, known as transition to support. The objective of this review
was to determine whether planning and preparation is adequate to
timely and effectively transfer support and maintenance of Business Systems
Modernization (BSM) projects to the Internal Revenue Service (IRS) support organizations.
In summary, we found that the IRS, in
conjunction with the Computer Sciences Corporation, developed the Enterprise
Life Cycle (ELC) that provides guidance for designing, developing, and
supporting BSM projects. Although the
ELC provided guidance on transition to
support activities, the guidance was not clearly organized and did not
address all activities recommended by the Software Acquisition Capability
Maturity Model. In addition, BSM
project teams were not consistently following the transition to support guidance in the ELC. We found significant variances in planning transition to support activities in four current BSM projects.
To address these shortcomings, the IRS
is establishing a Transition Management
Office (TMO) to
provide transition to support
policy and guidance to BSM project teams and IRS organizations designated to
support the BSM projects. The TMO has drafted guidance to provide a comprehensive and
organized source for planning and completing transition to support activities.
The draft guidance provides
direction for developing transition to support plans and directs BSM project teams to develop these
plans earlier in the projects’ design phase. The draft guidance addresses most of the gaps that exist between
the ELC and the Software Acquisition
Capability Maturity Model. The TMO also plans to review BSM projects to ensure the project
teams are consistently following the transition
to support guidance and will work with the IRS operating divisions to
ensure they are capable and ready to support and maintain the BSM projects once
they are deployed.
Because
the TMO is working toward addressing the transition to
support problems identified, we are not
providing any additional recommendations at this time. However, we suggest you consider the issues
identified in this report to help in your efforts to further improve the transition to support processes,
which will be a major step in improving the overall acquisition management
capabilities of the IRS.
Management’s Response: The Deputy Commissioner for Modernization
& Chief Information Officer responded that he appreciated our insights into
and review of this critical area and plans to incorporate our suggestions into
the transition to support processes and documents. Management’s complete response to the report is included as
Appendix V.
Copies
of this report are also being sent to the IRS managers who are affected by the
report. Please contact me at (202)
622-6510 if you have questions or Scott Wilson, Assistant Inspector General for
Audit (Information Systems Programs), at (202) 622-8510.
Improved Transition to Support Guidance Has Been Developed
Appendix I – Detailed Objective, Scope, and Methodology
Appendix II – Major Contributors to This Report
Appendix III – Report Distribution List
Appendix V –
Management’s Response to the Draft Report
The Internal Revenue Service (IRS) selected the Computer Sciences Corporation to serve as the Prime Systems Integration Services Contractor (PRIME) for the Business Systems Modernization (BSM) program. The IRS and the PRIME contractor developed the Enterprise Life Cycle (ELC) that provides guidance for designing, developing, and supporting BSM projects.
The ELC establishes a set of repeatable processes and a system of reviews that promote delivery of promised business results. The ELC processes are in line with the Software Engineering Institute’s Software Acquisition Capability Model Maturity Model (SA-CMM). The SA-CMM is a widely recognized and adopted framework for improving software and system acquisition processes.
Transition to support (T2S) is identified as one of the SA-CMM key process areas and is included as part of the ELC. The intent of T2S is to provide a disciplined transition of the BSM projects being developed to the IRS organization responsible for their long-term support and maintenance. All BSM project teams need T2S activities to take place in order to successfully transfer the projects to the IRS. The goal of T2S activities is to ensure the support organizations are fully prepared to accept responsibility for supporting and maintaining the projects and related products.
In November 2001, the Deputy Commissioner for Modernization
& Chief Information Officer requested that we review the T2S
efforts. In response, we performed this audit to determine whether planning and preparation
is adequate to timely and effectively transfer support and maintenance of BSM
projects to the IRS support organizations.
To accomplish our objective, we reviewed available documentation and interviewed PRIME contractor staff and IRS executives, managers, and analysts located at the IRS’ National Headquarters and at the Modernization, Information Technology, and Security (MITS) Services offices in New Carrollton, Maryland. We performed these reviews between February and May 2002, in accordance with the President’s Council on Integrity and Efficiency’s Quality Standards for Inspections. Detailed information on our objective, scope, and methodology is presented in Appendix I. Major contributors to the report are listed in Appendix II.
The MITS Services organization is establishing the Transition Management Office (TMO) to provide T2S policy and guidance to BSM project teams and the MITS organization designated to support and maintain the deployed projects. The TMO developed a transition management approach to partner with the PRIME contractor’s Business Transformation Office to oversee, integrate, and provide an IRS-wide view of T2S processes. The TMO has also produced a draft transition management guidance document that provides templates and planning guidance for T2S activities. This document clarifies existing ELC processes and incorporates additional interim guidance developed by the MITS Services organization.
The draft transition management guidance provides a comprehensive and organized source for planning T2S activities. It addresses most of the existing gaps between the ELC and SA-CMM standards, and it directs BSM project teams to develop T2S plans earlier in the projects’ life cycle. The BSM Office’s Solutions Acquisition Process Group is currently reviewing this guidance. Once approved, the TMO plans to issue the new guidance as a program-wide directive that must be implemented by BSM project teams prior to a project’s Development Phase.
In addition to providing T2S guidance to the BSM project
teams and the MITS Services organization, the TMO also plans to provide
transition management guidance to other IRS operating divisions and
functions. The TMO will act as a
facilitator to involve and inform entities at the executive and strategic level
that have a stake in transition management.
While the TMO’s draft guidance on T2S activities addresses most of the gaps between the existing ELC processes and the SA-CMM key process areas, there are still two SA-CMM activities that are not included in the ELC and not fully addressed by the TMO’s draft guidance, as follows:
· Responsibility for the software products is transferred only after the software support organization demonstrates its capability and capacity to modify and support the software products.
·
The project team conducts activities to ensure that
support of the software products is maintained and is effective during the
transition from the contractor to the software support organization.
The current ELC and draft T2S guidance include planning activities to make the necessary resources and training available to the support organizations. However, to address the SA-CMM framework, the TMO needs to consider producing guidance that directs T2S activities to actually demonstrate (test) the MITS Services organizations’ capability and capacity to operate and maintain the BSM projects. Testing the MITS Services organization’s capability and capacity would provide the IRS assurance that appropriate operating experience and adequate resources exist to support a project’s processes prior to the hand-off from the developer. Appendix IV provides a comparison of the SA-CMM framework to the ELC and draft T2S guidance.
Without guidance requiring plans to include tests of the MITS Services organization’s capability and capacity, the project teams risk the ability to effectively and efficiently transfer control of the projects. The MITS Services organization may not be ready to accept control of the projects when they are deployed, which could either delay the projects or increase project costs by requiring the IRS to pay the PRIME contractor to support and maintain the deployed projects until the MITS Services organization is ready to accept control.
Management Actions: The TMO indicated that Version 2.0 of the draft transition management guidance, which is in process, would include procedures to assess T2S readiness of the project teams and the MITS Services organization.
We reviewed the T2S planning activities in the following BSM projects:
· Customer Account Data Engine (CADE) - CADE will create current, complete, and accurate authoritative data stores containing taxpayer accounts and tax returns and construct the related tax administration system processes.
· Security and Technology Infrastructure Release (STIR) - The system infrastructure is envisioned as a combination of custom software modules and commercial-off-the-shelf hardware, software, and security solutions, integrated to create the foundation for a secure, modernized IRS infrastructure. The STIR project objective was to deliver the first installment of the modernized technology infrastructure and security services for the 2002 Filing Season.
· Internet Refund/Fact of Filing (IRFOF) - The IRFOF project will allow taxpayers to access tax return filing and refund information with a secure Internet connection.
· Filing and Payment Compliance (F&PC) - The F&PC project will provide the IRS with the ability to manage its accounts receivables more effectively and significantly increase tax collection.
The T2S plans were not developed as early in the above projects’ life cycles as prescribed by the ELC, and the T2S plans and activities were not timely included in the task orders issued to the PRIME contractor for specific work products.
Project teams did not develop T2S plans early in the project life cycle
ELC guidance directs that T2S activities should begin early in a project’s life cycle and that these activities should be performed in accordance with a T2S plan. The project teams did not develop T2S plans early in the life cycles of the four projects we reviewed. Rather, the projects delivered T2S plans after the completion of the Vision and Strategy and Architecture phases of the ELC.
For the MITS Services organization to be ready and able to maintain the work products that have been developed, it is important that the project teams consider available MITS Services organization resources from project inception. This planning activity allows for an awareness of what is being designed and developed and identifies the necessary preparations and resources for the MITS Services organization to be able to provide the required support upon assumption of responsibility.
The status of T2S plan development for the four projects reviewed was as follows:
· The CADE project team is currently developing a Release 1 pilot but is not scheduled to produce a T2S plan until the development of Release 2 in 2003. However, the PRIME will maintain the CADE project until it is transferred to the IRS after the deployment of Release 5 scheduled for 2006.
· The STIR project team completed its Vision and Strategy and Architecture phases in April 2000 without a T2S plan. The T2S plan was first presented as a Deployment Phase deliverable in January 2002.
· The IRFOF project team began the Development Phase in August 2000, but the T2S plan was not delivered until September 2001.
· The F&PC project team is scheduled to address T2S plans and activities in its Organizational Transition Plan after completing the Vision and Strategy and Architecture phases and before beginning the Development Phase in August 2002.
Project teams did not timely include
T2S plans and activities in task orders
One of the ELC requirements is that T2S plans and activities are required in issued task orders. In the four projects we reviewed, we found that the T2S plans and activities were not timely included in task orders issued to the PRIME contractor. The CADE project does not have a task order for T2S plans for the reasons mentioned above.
· The STIR project team did not task a T2S plan until June 2001, when it was added as a deliverable subsequent to the Development Phase.
· The IRFOF project team did not require a T2S plan deliverable until a task order modification was signed in May 2001 to add it as a deliverable.
· The F&PC project team did not require any T2S plans or activities in its task order. The project team is scheduled to address T2S plans in the project Development Phase in August 2002.
Inconsistency in T2S
planning and task order development can be attributed to varying
interpretations of existing guidance.
The guidance recommends developing T2S plans at the beginning of
a project and then reviewing the progress of the plans at three later points in
the life cycle. Various project teams interpreted this
guidance to mean that T2S plans and task order development need not be
initiated until the Development Phase of the ELC, instead of at the beginning
of the project. There was also some
confusion in interpreting the T2S process, which spans the entire life cycle,
with the actual deployment and transition of a project, which is the final
phase of the life cycle.
The absence of T2S plan requirements in task orders issued to the PRIME contractor could result in delays in transferring BSM projects to the IRS. Also, delivery of T2S plans late in the project life cycle can affect the ability to execute transition activities in time for a seamless transition to the MITS Services organization. Delays may require the PRIME contractor to operate and maintain the deployed projects until the MITS Services organization has the resources and trained personnel available to receive the projects.
Management
Actions: The TMO plans to issue T2S policy and
guidance that is more clearly defined and communicated to allow for consistent
and effective T2S planning and execution.
Future inconsistencies in the timing of T2S planning and contract
tasking should be overcome with the issuance of the pending transition
management guidance document and the TMO’s involvement with the BSM project
teams.
Management’s Response: The Deputy Commissioner for Modernization & Chief Information Officer responded that he appreciated our insights into and review of this critical area and plans to incorporate our suggestions into the transition to support processes and documents.
Appendix I
Detailed Objective, Scope, and Methodology
The objective of this review was to determine whether planning and preparation is adequate to timely and effectively transfer support and maintenance of Business Systems Modernization (BSM) projects to the Internal Revenue Service (IRS) support organizations. This process is known as the transition to support (T2S) process.
I.
Assessed the status and adequacy of the T2S planning
and preparation guidance compared to industry standards and procedures.
A.
Compared the T2S planning and preparation guidance in
the IRS’ Enterprise Life Cycle (ELC) to the Software Engineering Institute’s
Software Acquisition Capability Maturity Model (SA-CMM).
B.
Assessed the extent to which the IRS’ Integration
Management Office draft T2S plan template and related T2S guidance supplements
existing T2S planning and preparation guidelines.
C.
Assessed the scope and content of the IRS’ Transition
Management Office draft T2S guidance for the BSM project teams.
II.
Reviewed the adequacy of task orders and T2S plans of
four BSM projects to determine if necessary T2S planning and preparation
activities were present.
A.
Determined if the selected projects have adequate
provisions in the task orders for developing T2S plans and implementing T2S
activities.
B.
Determined if the selected projects have adequate T2S
plans for project teams to transfer modernized products to the receiving IRS
support organization.
Appendix II
Major Contributors to This Report
Scott E. Wilson, Assistant Inspector General for Audit
(Information Systems Programs)
Scott A. Macfarlane, Director
Edward
A. Neuwirth, Audit Manager
Beverly
Tamanaha, Senior Auditor
Louis
Zullo, Senior Auditor
Appendix III
Commissioner N:C
Deputy Commissioner N:DC
Associate Commissioner, Business Systems Modernization M:B
Chief, Information
Technology Services M:I
Deputy
Associate Commissioner, Program Management
M:B:PM
Chief Counsel CC
National Taxpayer Advocate TA
Director, Legislative Affairs CL:LA
Director, Office of Program Evaluation and Risk Analysis N:ADC:R:O
Office of Management Controls N:CFO:F:M
Audit Liaison:
Associate
Commissioner, Business Systems Modernization
M:B
Chief, Information Technology Services M:I
Appendix IV
Comparison of the Software
Acquisition Capability Maturity Model Framework with the Enterprise Life Cycle
Transition to Support Guidance and Draft Transition Management Guidance
|
CF |
Software
Acquisition |
Addressed in Enterprise Life Cycle Guidance? |
Addressed in Draft
Transition Management Guidance? |
|---|---|---|---|
|
G1 |
The project team ensures the software support
organization has the capacity and capability to provide the required support
upon assumption of responsibility for the support of the software products. |
Yes |
Yes |
|
G2 |
There is no loss in
continuity of support to the software products during transition from the
contractor to the software support organization. |
Yes |
Yes |
|
G3 |
Configuration
management is maintained throughout the transition. |
Yes
|
Yes |
|
C1 |
The acquisition organization has a written policy for the
transfer of software products to the software support organization. This policy typically specifies that: 1.
The
software support organization is identified prior to developing the solicitation
package. 2.
Resources for software support are
included in the appropriate budget(s). 3.
The designated software support
organization is involved, as appropriate, throughout the acquisition. |
Yes |
Yes |
|
C2 |
The acquisition
organization ensures that the software support organization is involved in
planning for transition to support. |
Yes |
Yes |
|
C3 |
Responsibility for
transition to support activities is designated. |
Yes |
Yes |
|
Ab1 |
A group that is
responsible for coordinating transition to support activities exists. |
Yes |
Yes |
|
Ab2 |
Adequate resources are
provided for transition to support activities. (Resources include funding, staff, equipment, and tools.) |
Yes |
Yes |
|
Ab3 |
The organization
responsible for providing support of the software products is identified no
later than initiation of the solicitation package’s development. |
Yes |
No
|
|
Ab4 |
The software support
organization, prior to transition, has a complete inventory of all software
and related items that are to be transferred. |
Yes |
Yes |
|
Ab5 |
Individuals performing
transition to support activities have experience or receive training. |
Yes |
No
|
|
Ab6 |
The members of
organizations interfacing with the transition to support activities receive
orientation on the salient aspects of transition to support activities. |
Yes |
Yes |
|
Ac1 |
The project team performs its activities in accordance
with its documented transition to support plans. The plans typically
cover: 1.
The
objectives and scope of the transition to support activities. 2.
Identification
and involvement of the software support organization. 3.
Support
resource requirements. 4.
A
definition of transition activities. 5.
A
schedule of transition activities. 6.
Allocation of transition responsibilities. 7.
Installation of products that are to be
delivered. 8.
Warranty and data rights provisions. |
Yes, except |
Yes, except for
item #8 |
|
Ac2 |
Responsibility for the software products is transferred only after the software support organization demonstrates its capability and capacity to modify and support the software products. This
capability typically includes the availability of: 1.
Hardware,
software, physical, fiscal, and personnel resources. 2.
Plans,
processes, procedures, and documentation. 3.
An
established configuration management system capable of supporting the
software. 4.
Appropriate
training of all personnel involved. 5.
Software
replication, test, and distribution capabilities. |
No
|
No
|
|
Ac3 |
The project team
conducts activities to ensure that support of the software products is
maintained and is effective during the transition from the contractor to the
software support organization. |
No
|
No
|
|
Ac4 |
The project team oversees the configuration control of the software products throughout the transition. |
Yes
|
Yes |
|
M1 |
Measurements are made
and used to determine the status of the transition to support activities and
resultant products. |
Yes
|
Yes |
|
V1 |
Transition to support
activities are reviewed by acquisition and software support organizations’
management on a periodic basis. |
Yes |
Yes |
|
V2 |
Transition to support
activities are reviewed by the project manager on both a periodic and
event-driven basis. |
Yes |
Yes |
Appendix V
The response was removed due to its size. To see the complete response, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.