Progress in Developing the Customer Communications Project Has Been Made, But Risks to Timely Deployment in 2001 Still Exist
March 2001
Reference Number: 2001-20-055
This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.
March 12, 2001
MEMORANDUM FOR DEPUTY COMMISSIONER FOR MODERNIZATION
AND CHIEF INFORMATION OFFICER
FROM: (for) Pamela J. Gardiner /s/ Scott E. Wilson
Deputy Inspector General for Audit
SUBJECT: Final Audit Report – Progress in Developing the Customer Communications Project Has Been Made, But Risks to Timely Deployment in 2001 Still Exist
This report presents the results of our review of the Internal Revenue Service’s (IRS) development of the Customer Communications Project (CCP). The CCP is expected to increase telephone service to levels similar to the private sector, and is the first intended accomplishment of the IRS’ massive systems modernization initiative. In summary, the IRS has shown significant progress in developing the CCP and in implementing project management capabilities. Lessons are being learned that should improve the effectiveness and efficiency of developing the CCP and other ongoing and planned modernization projects.
However, the IRS is at risk of not meeting its original commitments to the Congress to improve telephone service to taxpayers in the 2001 Filing Season. At risk is the IRS’ expectation of answering an additional 9.6 million taxpayer calls. Also, delays in deploying telephone enhancements could postpone the IRS’ plan to free up telephone assistors for other work, which was projected to be a $5.67 million budget benefit to the IRS in Fiscal Year 2001
.Although immediate benefits to taxpayers associated with the CCP may be at risk, it is our opinion that the IRS will eventually deploy most, if not all, of the project capabilities. However, the growing pains experienced on this project are indicative of an organization that may continue to overestimate its ability to acquire modernized systems in line with its original schedule and cost estimates.
To properly manage the CCP and all other ongoing and future modernization projects, we recommended that the Chief Information Officer (CIO) ensure that project managers timely complete all Enterprise Life Cycle (ELC) recommended work products and not allow projects to exit ELC milestones when significant work products are incomplete. Further, the CIO needs to ensure that project managers develop and communicate realistic project schedules, timely incorporate necessary security considerations, and ensure that risk tracking and reporting is timely, complete and accurate. Management's response was due on March 6, 2001. As of March 9, 2001, management had not responded to the draft report
Copies of this report are also being sent to the IRS managers who are affected by the report recommendations. Please contact me at (202) 622-6510 if you have questions, or your staff may call Scott E. Wilson, Associate Inspector General for Audit (Information Systems Programs), at (202) 622-8510.
Managers Did Not Timely Complete Recommended Work Products
Several Barriers Exist That May Impact the Delivery of Taxpayer Benefits in the 2001 Filing Season
Appendix I – Detailed Objective, Scope, and Methodology
Appendix II – Major Contributors to This Report
Appendix III – Report Distribution List
Appendix IV – Outcome Measures
The Customer Communications Project (CCP) is intended to increase telephone and communication service levels to taxpayers comparable to those of similar customer service operations in the private sector. CCP expectations can be traced back to May 1997, when the Internal Revenue Service (IRS) issued its Modernization Blueprint to define, direct, and control investments in modernized systems and related infrastructure. The Modernization Blueprint identified the CCP as its intended first accomplishment.
The CCP plans to provide a centralized national telephone call management capability so that the more than 150 million yearly taxpayer calls can be directed to available taxpayer assistors at any location. The project also proposes improved self-service telephone and Internet services to taxpayers.
The overall objective of this audit was to assess the IRS’ project management process for implementing the CCP. Our review included work completed by the CCP’s executives, managers and project team members through October 26, 2000.
Results
The IRS is in the early stages of its systems modernization process. In addition to delivering needed improvements in telephone and communication service levels to taxpayers, the IRS considers the CCP as a "test bed" to help develop and institute critical project management and systems modernization processes. The IRS has demonstrated significant progress in implementing project management capabilities by beginning to establish some repeatable processes in developing the CCP. Lessons learned during the initial stages of the CCP should improve the effectiveness and efficiency of developing the CCP and other ongoing and planned modernization projects.
However, the CCP has been scaled back and fallen behind schedule since it first received congressional funding in mid-1999. Heroic efforts by the IRS and the modernization contractor will be needed to meet the commitment of improved telephone service to taxpayers for the 2001 Filing Season. Delays in the delivery of the CCP benefits will impact the IRS’ expectation of answering an additional 9.6 million taxpayer calls during the 2001 Filing Season. In addition, delays in deploying telephone enhancements could postpone the IRS’ plans to free up telephone assistors for other work, which was an estimated $5.67 million benefit to the IRS in Fiscal Year 2001.
In May 1999, the IRS submitted its first Information Technology Investment Account (ITIA) expenditure plan. The expenditure plan showed that for the 2001 Filing Season, the CCP would deliver: 1) centralized call management, systems management, administrative, and management information consolidation and reporting capabilities; and 2) telephone and Internet automated self-service applications.
Since the initial design and development, the IRS significantly modified the scope and schedule of the CCP. The IRS acknowledged that it would not be able to provide taxpayers all of the enhancements and benefits that it originally proposed in its first ITIA expenditure plan. The telephone and Internet self-service applications were postponed until 2002, and the remaining capabilities were rescheduled for deployment during the latter part of the 2001 Filing Season. The CCP fell behind schedule, in part, because some key work products were not timely completed, and several identified barriers to deployment had not yet been overcome.
Although immediate benefits to taxpayers associated with the CCP may be at risk, it is our opinion that the IRS will eventually deploy most, if not all, of the project capabilities. However, the growing pains experienced on this project are indicative of an organization that may continue to overestimate its ability to acquire modernized systems in line with its original schedule and cost estimates.
Managers Did Not Timely Complete Recommended Work Products
The IRS has adopted the Enterprise Life Cycle (ELC) process to manage systems modernization projects. CCP managers decided not to complete several significant ELC work products and asked for waivers from IRS executives to move into the next phase of the development process. Examples of work products that were not timely completed include a cost, benefit and risks analysis; investment technology assessment and strategy reports; acquisition management strategy plans; budget, capacity and investment analysis; the System Operations Concept; and the System Validation and Verification Plan.
Although several key work products were not completed, IRS executives gave a qualified approval for the project to continue as long as the work products were completed by specific dates. CCP managers expected to make up the time lost completing the work products in the next ELC phase. However, the project never did catch up, and in fact, fell further behind. The absence and delay in coordinating and completing work product delivery prior to exiting project milestones negatively impacts project direction and the subsequent ability to meet expectations.
Several Barriers Exist That May Impact the Delivery of Taxpayer Benefits in the 2001 Filing Season
By the completion of our fieldwork in October 2000, the CCP managers had identified three significant issues that could affect the CCP’s ability to deliver improved telephone service to taxpayers in 2001:
Lessons Learned in Managing the Customer Communications Project Will Help the Delivery of Future Modernization Projects
The IRS has made progress in developing the ability to establish some repeatable processes needed for future project successes. These processes relate to using the ELC at the project level and incorporating requirements for overall program management.
For instance, the IRS has improved or standardized many processes related to project reporting, project schedules and tasks, Memoranda of Agreement, proposal reviews, and the Executive sign-off process. In addition, the IRS’ active involvement and participation with activities such as weekly and monthly Project Status meetings is fostering tangible improvements in project oversight capabilities. Lessons learned during the initial stages of the CCP should improve the effectiveness and efficiency of developing the CCP and other ongoing and planned modernization projects.
Summary of Recommendations
To properly manage the CCP and all other ongoing and future modernization projects, the Chief Information Officer (CIO) needs to ensure that project managers timely complete all ELC recommended work products and not allow projects to exit ELC milestones when significant work products are incomplete. Further, the CIO needs to ensure that project managers develop and communicate realistic project schedules, timely incorporate necessary security considerations, and ensure that risk tracking and reporting is timely, complete and accurate.
Management's Response: Management's response was due on March 6, 2001. As of March 9, 2001, management had not responded to the draft report.
The overall objective of this audit was to assess the Internal Revenue Service’s (IRS) project management process for implementing the Fiscal Year (FY) 2001 Release of the Customer Communications Project (CCP). We conducted our review from February through October 2000 in the IRS’ New Carrollton, Maryland, office, and at Computer Sciences Corporation (CSC) facilities in Greenbelt, Maryland. Our review included work completed by the Project executives, managers and team members through October 26, 2000.
This audit was performed in accordance with Government Auditing Standards. Details of our audit objective, scope, and methodology are presented in Appendix I. Major contributors to this report are listed in Appendix II.
The CCP is intended to increase telephone and communication service levels to those of similar customer service operations in the private sector. The project’s expectations can be traced back to May 1997, when the IRS issued its Modernization Blueprint to define, direct, and control investments in modernized systems and related infrastructure.
The Modernization Blueprint identified the CCP as its intended first accomplishment. Project goals to update and improve taxpayer communications included providing a centralized national telephone call management capability so that the more than 150 million yearly taxpayer calls could be directed to available taxpayer assistors at any location. The project also proposed improved self-service telephone and Internet services to taxpayers.
In May 1999, the IRS submitted its first Information Technology Investment Account (ITIA) expenditure plan. The request proposed the following CCP capabilities for the 2001 Filing Season:
In January 2000, the IRS recognized that it did not have the management capacity to control the portfolio of projects initiated in the first half of 1999. The IRS recognized that a real danger existed in allowing the CCP to continue moving ahead of the overall modernization program. As a result, the IRS significantly scaled back the original CCP 2001 Filing Season expectations.
In March 2000, the IRS submitted to the Congress its second ITIA expenditure plan. In this plan, the IRS limited the 2001 CCP capabilities to telephone call center operation improvements. The IRS postponed the following capabilities until the 2002 Filing Season:
In April 2000, MITRE performed an assessment of required CCP work products. The PRIME and the IRS used MITRE’s results as a basis for dividing the remaining planned 2001 Filing Season capabilities into two sub-releases. Sub-Release 1 (filing season), originally scheduled for January 1, 2000, was postponed until February 15, 2001. Sub-Release 2 (post-filing season) remains on schedule for May 31, 2001.
Sub-Release 1 will rely on the existing (legacy) telephone call management systems. Expected capabilities include:
Sub-Release 2 will retire the legacy voice recognition technology and add approximately 4,000 additional speech-enabled ports.
The IRS contracted with experienced information technology companies to design and build the various modernization projects including the CCP. PRIME Alliance is a group of leading companies brought together by the CSC to provide the IRS with access to commercial best practices, guarantee access to viable alternative solutions, and streamline the system acquisition process. One of the processes that the CSC created for the IRS is the Enterprise Life Cycle (ELC), which guides the planning, designing, and implementing of modernization projects.
The ELC establishes a set of repeatable processes and a system of reviews, checkpoints, and milestones that enables delivery of promised business results. All PRIME and IRS personnel involved in systems modernization are mandated to use the ELC. It is intended to improve the acquisition, use, and management of information technology within the IRS, facilitate management of large-scale business change, and enhance the methods of decision making and information sharing.
Key among the benefits of the ELC is an Investment Decision Management (IDM) process. The IRS uses the IDM to prioritize, approve, fund, monitor, and evaluate investment decisions throughout the ELC.
The IRS is in the early stages of its systems modernization process. In addition to delivering needed improvements in telephone and communication service levels, the IRS considers the CCP as a "test bed" to help develop and institute critical project management and systems modernization processes. The IRS has demonstrated significant progress in implementing project management capabilities by beginning to establish some repeatable processes in developing the CCP. Lessons learned during the initial stages of the CCP should improve the effectiveness and efficiency of developing the CCP and other ongoing and planned modernization projects.
However, the CCP has fallen behind schedule, and is at risk of not delivering all intended taxpayer benefits in the 2001 Filing Season. Heroic efforts by the PRIME and the IRS will be needed to meet the commitment of improved telephone service to taxpayers the IRS made to the Congress. Further delays in the delivery of the CCP could postpone taxpayer benefits until the 2002 Filing Season.
At the completion of our fieldwork in late October 2000, the IRS again revised the filing season release (Sub-Release 1) from February 15, 2001, to early April 2001. Most taxpayers would not benefit from this release since the majority of tax returns are filed by April 15. The delivery date for the post-filing season release (Sub-Release 2) was still projected for May 31, 2001.
Managers Did Not Timely Complete Recommended Work Products
The IRS has adopted the ELC process to manage systems modernization projects. However, CCP managers did not ensure that ELC recommended work products were timely completed for the various project milestones. ELC work products are the tangible results of a project. They are used in all aspects of the life cycle to plan, build, deploy, manage, and operate the project.
ELC reviews take place throughout a project’s life cycle in the form of milestones. The PRIME’s project manager and key project personnel have responsibility for conducting the milestone reviews. At the project level, each milestone review is intended to determine if the project should proceed, given the work and findings to date and the strength of the business case.
During the early phases of the CCP, managers decided not to complete several ELC recommended work products for Milestone 1 (Case for Action) and Milestone 2 (Preliminary Business Case). These work products included: a cost, benefit and risks analysis; investment technology assessment and strategy reports; acquisition management strategy plans; budget, capacity and investment analysis; the System Operation Concept; and the System Validation and Verification Plan. During this time, MITRE informed the IRS that the Systems Operation Concept and the Master Test Plan were critical documents for the successful and timely implementation of new capabilities.
The CCP needed waivers from IRS executives to continue to Milestone 3. The CCP received the waivers allowing it to exit Milestone 2 and proceed to Milestone 3 with less than half of the ELC recommended work products completed.
In May 2000, the CCP received a qualified approval from the Core Business Systems Executive Steering Committee to exit the project design phase, known as Milestone 3 of the ELC, and continue on to the development phase. The committee issued a qualified approval because the PRIME needed to complete additional work on ELC work products. The PRIME agreed to complete over 40 additional actions to satisfy all Milestone 3 exit conditions. To allow for completion of these work products, the timing for releasing 2001 Filing Season enhancements needed revisions.
Significant work products receiving qualified approvals include the Project Management Plan, Configuration Management Plan, the Quality Management Plan, the System Life Cycle Management Plan, the System Design Report, the Security Documents, and the Risk Management Plan.
At the time we completed our fieldwork in October 2000, the IRS was actively working to resolve issues related to the Security Documents and the Risk Management Plan (see page 9 and page 10, respectively). The absence and delay in completing work products prior to exiting project milestones negatively impacts project direction and subsequent expectations.
Recommendation
To properly manage the CCP and all other ongoing and future modernization projects, the Chief Information Officer (CIO) should:
Management’s Response: Management's response was due on March 6, 2001. As of March 9, 2001, management had not responded to the draft report.
Several Barriers Exist That May Impact the Delivery of Taxpayer Benefits in the 2001 Filing Season
By the completion of our review in October 2000, the CCP managers had identified three significant issues that could affect the CCP’s ability to deliver improved telephone service to taxpayers in 2001. We discussed these issues with the Business Systems Modernization Office executives, managers and analysts on October 26, 2000.
Software design
The CCP has an extremely aggressive schedule with no reserve or recovery time to allow for unplanned events. For example, PRIME and IRS managers did not meet the October 6, 2000, completion date for the software design. They determined that the completion date might be delayed over 30 days due to unexpected technical issues. Further, subsequent meetings held between the PRIME and the IRS identified additional design issues that may postpone full development until mid January 2001.
If this delay cannot be overcome, systems testing will not be completed until early April 2001. The resulting reduction to taxpayer burden in the 2001 Filing Season will be significantly less than projected. For example, the Milestone 3 Baseline Business Case cited an increase of 9.6 million completed taxpayer calls per year after all product lines are in place. With deployment so late in the filing season, taxpayer benefit will not begin to occur until the 2002 Filing Season. In addition, delays in deploying telephone enhancements may not allow the IRS to free up telephone assistors for other work, which was estimated to be a $5.67 million budget benefit to the IRS in FY 2001.
Recommendation
To ensure that the CCP and other ongoing and future projects have a reasonable opportunity to meet expectations, the CIO should:
2. Ensure project managers build sufficient reserves and recovery time into work schedules to allow for the impact of unplanned events on project delivery.
Security certification schedule requirements
Full security certification requirements should be met before project delivery can begin to ensure adequate protection of taxpayer information. The Office of Security and Privacy Oversight will not grant waivers for operating any modernization project without meeting these requirements. The security certification process usually requires about 3 months after system testing is completed.
CCP team members have met with the Office of Security and Privacy Oversight to pursue security certification. Although many agreements have been reached, security documentation issues and security testing schedules had not been set by the completion of our audit work.
In response to security concerns, the IRS contracted with Booz-Allen & Hamilton (BAH) to perform a security evaluation of the IRS’ automated call routing system on September 21, 2000. BAH’s evaluation found that the call routing system was generally secure. However, in reviewing the BAH documents, we identified and brought to the IRS’ attention a potential security weakness that could allow unauthorized traffic to traverse the system. This risk exposure impacts the project’s ability to receive full security certification.
Recommendation
To ensure the CCP and other ongoing and future projects meet full IRS security requirements, the CIO should:
Risk management
Risk management of modernization projects was made a joint responsibility of the PRIME and the IRS, as detailed in the ELC Joint Risk Management Procedure, dated June 30, 2000. The IRS discusses risks with the PRIME contractor on a periodic basis to identify and reduce known and anticipated risks. However, the ability to track and report project risks, as well as monitor risk reduction efforts, may be severely hampered until an adequate risk tracking system is fully implemented.
The Integrated Tracking System is used to track project risks for modernization projects, including Customer Communications. The database used in this system is inadequate due to the errors and inconsistencies contained in the risk status reports.
At the completion of our audit, the PRIME and the IRS had come to an agreement on the new requirements and procedures necessary to improve the risk database and Joint Risk Management Procedure. The risk management process could be improved if the following weaknesses were reduced:
The risk management process will be greatly enhanced when an adequate tool is implemented to capture and track risks. However, any system or database is only as good as the information it contains. The validity of significant risk reduction results is itself a risk that needs to be considered.
Recommendation
To effectively track and report project risks, as well as monitor risk reduction efforts, the CIO should:
Lessons Learned in Managing the Customer Communications Project Will Help the Delivery of Future Modernization Projects
The CCP is considered a Near Term Release (NTR) project, which are modernization projects intended for release in 2001 and 2002. NTR projects are relatively low risk projects that serve as a pilot or "test bed" for perfecting project management techniques. Besides providing benefits to taxpayers, lessons learned on these projects will help to strengthen the ELC process, and create an effective governance structure.
Although CCP management experienced the project delivery problems presented above, the project management team made progress in developing the ability to establish some repeatable processes needed for future project successes. These processes relate to using the ELC at the project level and incorporating requirements for overall program management.
Enterprise Life Cycle accomplishments
The Business Systems Modernization Office initiated some repeatable processes and capabilities during development of the CCP.
For instance, the Program Oversight for Tax Administration Office (formerly known as the Near Term Projects Office) has improved or standardized many processes related to project reporting, project schedules and tasks, Memoranda of Agreement (MOA), proposal reviews, and the Executive sign-off process, including:
Program level accomplishments
The IRS’ active involvement and participation with activities such as weekly and monthly Project Status meetings is fostering tangible improvements in project oversight. For instance, at the September 2000 PRIME Program Management Review (PMR), improvement and action items requested of the PRIME for the CCP included:
The IRS has demonstrated significant progress in implementing project management capabilities in the CCP. Lessons learned during the initial stages of the CCP should improve the effectiveness and efficiency of developing the CCP and other ongoing and planned modernization projects.
However, the IRS is at risk of not meeting its commitments to the Congress to improve telephone service to taxpayers in the 2001 Filing Season. Heroic actions by the PRIME and the IRS may be needed for taxpayers to begin receiving any benefits from the CCP in 2001.
Although immediate benefits to taxpayers associated with the CCP may be at risk, it is our opinion that the IRS will eventually deploy most, if not all, of the project capabilities. However, the growing pains experienced on this project are indicative of an organization that may continue to overestimate its ability to acquire modernized systems in line with its original schedule and cost estimates.
Appendix I
Detailed Objective, Scope, and Methodology
Our objective was to assess the Internal Revenue Service’s (IRS) project management process for implementing the Fiscal Year (FY) 2001 Release of the Customer Communications Project (CCP).
To accomplish this objective, we:
Appendix II
Major Contributors to This Report
Scott E. Wilson, Associate Inspector General for Audit (Information Systems Programs)
Scott A. Macfarlane, Director
Edward A. Neuwirth, Audit Manager
Bruce Polidori, Senior Auditor
Charles Winn, Senior Auditor
Louis Zullo, Senior Auditor
Appendix III
Commissioner N:C
Commissioner, Wage and Investment Division W
Associate Commissioner, Business Systems Modernization B
Chief Counsel CC
National Taxpayer Advocate TA
Director, Legislative Affairs CL:LA
Director, Office of Program Evaluation and Risk Analysis N:ADC:R:O
Office of Management Controls N:CFO:F:M
Audit Liaisons:
Chief Information Officer IS
Associate Commissioner, Business Systems Modernization B
Appendix IV
This appendix presents detailed information on the measurable impact that our recommended corrective actions will have on tax administration. These benefits will be incorporated into our Semiannual Report to the Congress.
Type and Value of Outcome Measure:
Methodology Used to Measure the Reported Benefit:
The increase in level of service of 9.6 million additional calls answered and the $5.67 million budget benefit to the Internal Revenue Service (IRS) in Fiscal Year (FY) 2001 were projections in the Customer Communications Project (CCP) Milestone 3 Baseline Business Case. These calculations were based on timely deployment of all proposed taxpayer benefits and call management enhancements in the 2001 Filing Season.
The scope of the initial CCP FY 2001 Release encompassed several of the product areas in which taxpayers contact the IRS for general tax assistance and for account-related status information through the telephone. The FY 2001 Release planned to include the following specific toll-free telephone product lines:
The Milestone 3 Baseline Business Case projected an increase in completed calls for the 1040, 8815, 4262, 4477, NTA, EP/EO and ACS product lines by approximately 9.6 million calls per year. Also, due to planned improvements to automated telephone service, the IRS would realize a budget benefit of $5.67 million in FY 2001 by freeing up telephone assistors for other work.