Executive Summary

Objective and Scope

Background

Results

Conclusion

Appendix I – Detailed Objective, Scope, and Methodology

Appendix II – Major Contributors to This Report

Appendix III – Report Distribution List

Appendix IV – Outcome Measures

Executive Summary

The Internal Revenue Service (IRS) has a goal that at least 80 percent of Federal tax and information returns be filed electronically by the year 2007. However, its ability to develop new product features allowing taxpayers and tax preparers to file, pay, and communicate electronically has been limited by its outdated computer systems.

To resolve this problem, the IRS is working with a private contractor, Computer Sciences Corporation (CSC), to modernize IRS business processes and technology. The IRS also established the Business Systems Modernization Office (BSMO) to assist and monitor the modernization efforts. One of the projects selected for development and deployment was the e-Services project. This project will provide the means for tax practitioners and other authorized parties to conduct business electronically with the IRS. The e-Services project will be deployed in various releases, with the first release scheduled in Fiscal Year (FY) 2002. The IRS projected that the project will provide $16 million in benefits in FY 2002.

The objective of our audit was to assess the IRS’ processes for development and implementation of the e-Services FY 2002 Release. To accomplish this objective, we evaluated the CSC’s delivery of goods and services, reviewed the project’s adherence to life cycle requirements, evaluated compliance with the Modernization Blueprint, and determined whether risks existed in the areas of project dependencies and sponsorship.

The BSMO has made progress in establishing critical processes to guide the e-Services and other modernization projects. For example, the BSMO has established a quality review process to evaluate, at the end of each major set of tasks, the products delivered by the contractor to determine whether they meet required standards. Also, the e-Services project team has begun to work more closely with the Architecture and Engineering Office to enable greater compliance with the Modernization Blueprint. In addition, project sponsorship has recently been enhanced, and business executives have become more involved in monitoring the progress of the project team.

Despite the progress that has been made, we identified several issues that have affected the success of the project thus far. Changes in requirements and setbacks in other projects on which the e-Services project is dependent resulted in delays and increased costs. Also, labor costs for IRS employees assigned to the project were not accurately tracked. In addition, we believe contracts issued to the CSC should be strengthened and project tracking measures expanded and validated. Finally, although processes to enable project success have been established, several need to be more effectively implemented to help enable the project team to meet its goals and objectives. Improvements in these areas will be needed to successfully deliver the benefits expected from e-Services as well as other modernization projects.

The IRS initially estimated in April 2000 that the e-Services project would require $3.8 million to complete its design phase by September 2000. As of March 2001, the estimated costs to complete this phase had increased to over $17 million, and the expected date of completion had been delayed until June 2001. Even if current development and deployment schedules are met, over $14 million of the $16 million in projected savings for FY 2002 will not be realized until FY 2003 or later.

Much of this increased cost and delay in project development is due to a significant change in project requirements. IRS executives were not comfortable with how e-Services integrated with other projects, with the Modernization Blueprint, and with the IRS’ overall modernization vision. Another cause for the delay was that the e-Services project was ahead of other key development efforts on which it was dependent. IRS executives decided to slow the development of the project until these other projects were further along, to reduce the risks.

We estimate that the costs for the IRS employees working on the e-Services project were understated by $1.6 million during the period October 1999 through January 2001. Project personnel indicated that it is difficult to ensure that employees, especially those who do not work full-time on the project, use the correct codes for charging their time. As a result, project costs appear to be lower than they actually are, and key measures such as return on investment cannot be accurately calculated.

The BSMO issues contracts (called task orders) to the CSC for specific products and services. Although the BSMO is improving the task order process, the task orders we reviewed did not include performance-based incentives for quality or timeliness. Also, the e-Services project team did not consistently negotiate task orders in a timely manner. Because the IRS identified various quality issues during its review of products received from the CSC, we believe that improvements in processing task orders could better protect the IRS’ interests as it works to ensure quality products at reasonable costs.

The project team uses earned value techniques to measure project progress by individual phase. Although earned value is an appropriate project tracking approach, measures that cover each individual phase are too limited to measure the overall progress of a project that covers five phases. Earned value techniques are designed to measure against a total budgeted amount, not just a current project phase. In addition, no validation of the project measures had been conducted at the time we completed our audit work. The data could be more useful and reliable if changes are made to the manner in which these measures are developed and a validation process is established.

The e-Services project team had developed a configuration management plan that followed the Enterprise Life Cycle (ELC) requirements. The plan outlines proper controls over project documentation and indicates that a project document repository would be established to ensure version control over documentation and system modules.

However, the CSC was not following the configuration management plan. The project document repository, where critical project documentation is maintained, was accessible to anyone on any of the project teams. This repository included both baselined documents and those that were still in process. There was no control over the various versions of key documents that had been accepted and signed by the IRS, and these accepted documents were not easily identified in this repository. Without this control, it will become harder to determine which documents are the official baselined documents. This could lead to project teams following the wrong set of requirements while developing the projects. During our audit, the BSMO and the CSC initiated corrective actions regarding configuration management processes.

The e-Services project team developed a risk management plan that describes the activities and processes it will follow to manage project risks. The project team was working to identify the risks and issues it faces during the project, using the correct forms to documents risks and meeting on a regular basis to discuss project risks.

However, we identified weaknesses in the implementation of the risk management process. For instance, the potential impact of identified risks on the project’s cost, schedule, or technical performance was not adequately documented. In addition, the project team did not adequately document the probable impact date, mitigation actions, measures to determine the effectiveness of mitigation actions, and the rationale for closing risks. These were contributing factors in the project delays and cost increases. The BSMO and the CSC have begun initiating corrective actions regarding risk management processes.

Managing project dependencies is critical to the success of the e-Services project and the overall modernization program. Identifying the risks associated with project dependencies is important so that all project teams and the BSMO can be aware of the criticality of any delays that may occur in completion of each project.

The e-Services project team did a good job of identifying critical dependencies on other projects; however, it was not adequately documenting the risks associated with the identified dependencies so that all project teams were aware of and agreed to the dependencies and necessary product delivery dates. Without clear and accurate documentation of project dependencies, incorrect decisions may be made at a program level and additional delays in the project can occur.

The e-Services project team was employing certain critical project management processes. The Project Manager was using a schedule, known as a Work Breakdown Structure, to manage the project team’s tasks. The schedule listed in detail the specific tasks that the project team needed to complete. Each task was identified with a specific identification number and had an assigned start date, finish date, and estimated duration. However, we found that near-term tasks were not assigned to specific team members. Identifying individuals assigned to tasks is important to ensure those individuals are available when their skills are needed. In addition, the schedule did not include any time to recover from unplanned or unforeseen events that could adversely affect the completion of critical tasks. Reserve or recovery time, while not required on every project, may be necessary if the current schedule is very aggressive or if a project team faces schedule risks, such as dependencies on other projects.

To minimize further delays and cost increases in this project, we recommend that the e-Services project team identify the critical factors the project is dependent upon and ensure BSMO executives are aware of each factor that could cause delays in future development and deployment. In addition, we recommend that the BSMO improve the tracking of IRS labor costs related to the project.

We did not provide recommendations for the other conditions identified in this report because we believe the corrective actions need to be taken at the program level rather than by the individual project teams. Because similar conditions were identified in other projects being audited, we plan to issue a separate report with recommendations that can be implemented at the program level.

Management's Response: Management's response was due on August 30, 2001. As of September 7, 2001, management had not responded to the draft report.

Objective and Scope

The overall objective of our audit was to assess the Internal Revenue Service’s (IRS) processes for development and implementation of the e-Services Fiscal Year (FY) 2002 Release.

To accomplish this objective, we determined whether the contractor that is primarily responsible for designing, developing, integrating, and transferring modernized systems to the IRS was delivering high-quality requested goods and services in a timely and cost-effective manner. To evaluate contractor timeliness and quality, we focused our review on deliverables associated with the system concept phase because that was the last phase the project team had successfully completed. To evaluate cost, we reviewed all the task orders that were associated with the project, focusing primarily on those associated with the system concept and design phases.

We also evaluated whether the project team was following critical processes that had been established to enable its success and whether the team was following the direction in the Modernization Blueprint. Our audit focused primarily on processes such as configuration management, risk management, requirements development, and performance tracking. Lastly, we evaluated whether the project team was effectively managing its dependencies on other projects and whether the IRS business executives were providing appropriate levels of project sponsorship.

We conducted this audit from October 2000 through March 2001, in the National Headquarters’ Business Systems Modernization Office (BSMO) and at the Computer Sciences Corporation (CSC) office. This audit was performed in accordance with Government Auditing Standards.

Details of our objective, scope, and methodology are presented in Appendix I. Major contributors to this report are listed in Appendix II.

Background

The IRS Restructuring and Reform Act of 1998 (RRA 98) set a new direction for the IRS in many areas, including establishing a goal that at least 80 percent of Federal tax and information returns be filed electronically by the year 2007. The IRS has been focused on developing an electronic filing program for over a decade. However, the ability to develop new applications and processes that allow taxpayers and tax preparers to file, pay, and communicate electronically with the IRS has been limited by its outdated computer systems.

To resolve this problem, the IRS is working to modernize its business processes and technology. The IRS hired the CSC to design and integrate the various projects involved in the modernization efforts. The IRS also established the BSMO to assist and monitor the modernization efforts.

One of the first projects the BSMO selected for development and deployment was the e-Services project. This project will provide the means for tax practitioners and other authorized parties to conduct business electronically with the IRS. The e-Services project will be deployed in various releases, with the first release scheduled in FY 2002. The IRS projected that e-Services will provide $16 million in benefits in FY 2002.

Results

The BSMO has made significant progress in establishing critical processes to guide the e-Services and other modernization projects. It has established a quality review process to evaluate products delivered by the CSC to determine whether they meet required standards at the end of each major set of tasks. The e-Services project team has become more involved in the development of the Modernization Blueprint, and project sponsorship has recently been enhanced, as the business executives have become more active in monitoring the progress of the e-Services project.

Despite this progress, significant changes in critical system requirements and setbacks in other development projects on which the e-Service project is dependent have resulted in delays and large increases in costs. Also, contracts need to be strengthened to provide greater emphasis on quality, positive and negative incentives for contractor performance, and shorter negotiation periods.

In addition, the project team has not been consistently following some of the processes that have been established to enable its success. In some cases, the processes are new or are still changing and problems have occurred in understanding and implementing them. In other cases, the project team had not fully incorporated these processes into its activities.

Significant Changes in Requirements and Setbacks in Other Development Efforts Resulted in Delays and Increased Costs

The Congress places funds for the IRS’ systems modernization activities in an Information Technology Investment Account (ITIA). The IRS must submit an expenditure plan requesting that funds from the ITIA be withdrawn for its use. The expenditure plan shows the estimated costs and completion dates for the various phases of the projects. The expenditure plan must be reviewed by the General Accounting Office and approved by the Department of the Treasury, the Office of Management and Budget, and the Congress.

Based on information contained in various ITIA expenditure plans, the e-Services project experienced delays of approximately 9 months and cost increases of over $13 million in completing the project design phase for the e-Services project.

One of the primary reasons for the large discrepancy in cost and due dates is that the IRS changed its requirements after the project team had completed most of the design phase products. The project team appeared before the Core Business Systems Executive Steering Committee (CBS ESC) in November 2000 to present the design phase products. The CBS ESC was not comfortable with the integration of the e-Services project with other projects, the Modernization Blueprint, and the overall modernization vision. The CBS ESC decided to expand the e-Services requirements to design an Internet access point that could be used by all the IRS project teams and to ensure that all requirements were integrated with the overall modernization program.

Project team personnel also indicated that requirements changed because the e-Services project was developing faster than other projects on which it had dependencies. These projects included the Modernization Blueprint, the overall modernization vision, and the Security and Technology Infrastructure Release (STIR). Therefore, the CBS ESC decision to increase the project design scope and delay development efforts until the other projects were further along was intended to reduce risk. While we agree with the decisions made, we believe that strengthening the planning and design processes would help ensure that significant additions to requirements and critical dependencies are addressed earlier.

Delays in the e-Services project will result in limiting taxpayer benefits realized in FY 2002. The project’s new schedule shows that deployment of the earliest segment of the project, originally planned for October 2001, will be delayed until June 2002. Delaying this segment will significantly reduce any benefits realized during the 2002 Filing Season.

The second segment of the project plans to provide significant electronic benefits to taxpayers and tax preparers. These benefits include electronic delivery of tax data, application for authorization to disclose tax data to third parties, matching of taxpayer identification numbers, and resolution of account inquiries. This portion of the project was originally scheduled to begin deployment in July 2002, but it is now scheduled to begin in September 2002.

Therefore, even if this new schedule is met for the 2 project segments, approximately $14 million of the $16 million in benefits projected for FY 2002 will not be realized until FY 2003 or later.

To minimize further delays and cost increases in the e-Services project, we recommend that the BSMO:

1. Ensure the e-Services project team identifies, evaluates, and monitors all significant dependencies on other modernization projects and activities.
2. Ensure the CBS ESC is aware of each factor that could cause delays or cost increases in future development and deployment.

Management's Response: Management's response was due on August 30, 2001. As of September 7, 2001, management had not responded to the draft report.

Costs for Internal Revenue Service Employees Working on the Project Were Not Accurately Tracked

While the IRS accounts for project costs incurred by the contractor, the BSMO does not have a process to accurately track and report the project-related expenses incurred by the IRS. These expenses include labor costs for IRS employees working on the project teams. The ELC indicates that an appropriate measure of total project cost should include any indirect (IRS) costs.

We compared the data provided by the IRS for expenses related to its employees working on the e-Services project to a figure we calculated based on 17 employees working on the project for 16 months. We determined that the reported expenses for IRS employees working on the e-Services project were understated by approximately $1.6 million during the period October 1999 through January 2001.

Project personnel indicated that it is difficult to ensure project employees, especially those who do not work full-time on the project team, use the correct codes for charging their time. However, without an accurate accounting of both contractor costs and the IRS’ internal costs associated with the project, actual benefits and Return on Investment cannot be accurately calculated.

Management Actions: The Tax Administration Modernization (TAM) Program Director’s Office is instituting two measurement criteria in an effort to capture IRS costs for time and travel for e-Services and other TAM teams.

Recommendation

To enable more accurate tracking of total project costs, the BSMO should consider:

3. Reviewing time charges monthly for the IRS employees working on the e-Services project to verify the accuracy of the labor costs charged to the project. The time charges should be compared to estimates to make sure the charges are reasonable.

Contracts Need to Be Strengthened to Ensure Interests Are Protected

The BSMO issues contracts (called task orders) to the CSC for specific products and services. The BSMO has begun to focus on developing procedures to ensure that payments to contractors are tied to contractor performance and that items are negotiated and agreement is obtained from all parties prior to work beginning. This increased focus should help to ensure payments are associated with the work products that are produced according to IRS quality standards.

Although this emphasis on performance is a significant improvement, we identified additional areas where the BSMO can strengthen the task order process to ensure that requirements are met timely:

• The task orders did not include performance-based incentives for quality or timeliness.
• Products were not completed before the project moved to the next development phase.
• The e-Services project team did not consistently negotiate task orders in a timely manner.

We reviewed the e-Services task orders to determine whether they included any positive or negative performance-based incentives for quality or timeliness. We did not identify any incentives to improve timeliness of delivery nor that could have been exercised had the contractor not addressed quality problems in a timely manner.

Government policy states that contracts should include incentive provisions to ensure contractors are rewarded for good performance and quality assurance deductions to discourage unsatisfactory performance. Although the BSMO can withhold payments from contractors until acceptable products are delivered, withholding payments does not provide any redress to the IRS for delays in product delivery. We believe performance-based contracting is a better incentive to achieve timely and quality deliverables.

In February 2001, we reported that the IRS should strengthen the use of performance-based contractor incentives. The IRS responded that it would work with another contractor to identify recommendations to improve the use of incentives.

The ELC indicates that the final acceptance of work products should occur at the completion of the Milestone Readiness Review for each ELC phase and only when all discrepancies and issues in the work products have been corrected.

We evaluated the review process for the e-Services system concept phase and found that the BSMO had identified 70 issues with the quality of the products delivered by the CSC. The BSMO reported these items back to the CSC as conditions to be addressed. Although the BSMO agreed that some of these conditions could be addressed in subsequent phases, correction of 49 of the 70 (70 percent) was required in the system concept documents.

Overall, we found that the CSC addressed these conditions timely. We judgmentally sampled 20 of the 49 conditions associated with system concept documents and found that 19 were timely addressed and completed.

However, we believe some of the conditions whose resolutions were delayed until subsequent phases should have been addressed in the system concept phase. We judgmentally sampled 11 of the more critical conditions that were to be addressed in subsequent phases. Of the 11, we identified 4 conditions that we believe, had they been resolved earlier, would have provided benefit to the project team during the system concept and early design phases. We could not determine whether the Enterprise Life Cycle (ELC) required that these specific conditions be addressed in the system concept phase deliverables because ELC requirements for completion of each phase are vague.

Three of these four conditions related to identification and description of the interfaces, dependencies, and design of the Internet access point. Uncertainties in these areas caused delays in project progress and increases in costs as the project team progressed into the design phase (see pages 3-6). The fourth condition stated that the Modernization and Information Technology Services organization did not have time to complete its review of the system concept documents. We believe it is critical to ensure that documents are thoroughly reviewed prior to completion of a phase. If IRS personnel do not have sufficient time to complete their review, progress into the next phase should be delayed until it is completed.

Management Actions: The BSMO quality review process at the completion of each phase is currently undergoing changes. We received a draft version of the new procedures during our review. This version indicated that each condition should capture the impact, if any, on the project’s cost, schedule, and/or benefits. Although the e-Services project team was tracking the conditions and expected completion dates, the impact of the conditions was not identified because these new procedures had not yet been adopted. In addition, the BSMO has recently begun to require that a structured quality review process be conducted prior to allowing the project team to complete a phase and final payment to be made for the deliverables for that phase.

The BSMO allowed work to begin under each of the e-Services task orders prior to formalizing agreements with the CSC on deliverables, schedules, and costs. For example, 1 task order had a delay of 91 days between the start of work and the completion of negotiation of the requirements, costs, and schedules. The most recent task order had a delay of 22 days between the start of work and the final negotiation of task order items.

While this practice allowed the project members to start work, the BSMO was at a disadvantage because no clearly defined requirements were present during the time the task orders were being negotiated. As a result, the BSMO paid for hours worked by the CSC that may not have contributed directly to the completion of the task order requirements that were later defined.

This practice occurred because the BSMO wanted to retain the significant levels of experience that were present on the CSC’s project staff. Gaps in the contracting process could result in critical staff members leaving to either work on other projects or obtain other employment. We understand this concern but believe the BSMO could address this by requiring that requirements, costs, and schedule are defined and task orders negotiated for the subsequent project phase prior to completing the current phase.

Management Actions: The BSMO is currently developing procedures to require defined task orders prior to allowing the project team to progress into the next phase. The e-Services project staff indicated that a negotiated task order for the next phase would be required for the project team to complete the design phase. The procedures documenting this new effort were not available for our review prior to the completion of our audit fieldwork.

Project Tracking Measures Should Be Expanded and Validated to Be of Greater Value to Program Management Personnel

The e-Services project team, along with the other modernization projects, uses earned value techniques to measure the status of the project. These techniques include measurements of schedule, cost, and value. The CSC is currently preparing these measures by phase or task order.

The Clinger-Cohen Act requires the agency Chief Information Officer to monitor the performance of information technology programs of the agency, evaluate the performance of those programs on the basis of applicable performance measures, and advise the agency head regarding whether to continue, modify, or terminate the program or project.

The ELC indicates that budget and cost measures should include all task orders related to a project across all phases, not just those costs related to the current phase. Measures should be made using total costs to date and estimated future costs, rather than focusing on just those costs related to a single phase, as they are currently being measured. Lengthening the period of time over which project value is measured would enable program management to determine whether the project team is on track to meet original requirements, time frames, and costs and whether investment in the project should be continued.

While measuring value over longer periods could be difficult because projects are funded from the ITIA on a milestone or phase basis, the CSC has not taken actions to improve the measures and incorporate this change. Until the process is changed, the BSMO is paying the contractor to develop measures that are of limited value as a project monitoring tool.

Validation of these measures should also be conducted to ensure they are accurate before reliance is placed on them for project monitoring. In a recent audit report, we recommended that the BSMO assess the CSC’s processes to ensure that performance monitoring data being provided to the IRS are complete and accurate. The BSMO agreed to develop a "surveillance plan" of these processes in December 2000 and begin this "surveillance" in March 2001. However, we did not find indications that these activities took place.

Management Actions: The BSMO has recognized that, to be of the greatest value, the baseline period of time or cost that earned value data are compared against should include the entire project life cycle or as much of that as possible. As a result, the BSMO has tasked the CSC to change the way it captures project measures.

Configuration Management Processes Were Not Consistently Followed

Configuration management involves identifying critical project items (documents, software, and hardware), controlling changes to those items, and recording and reporting any changes to the items. The ELC requires that configuration management procedures be implemented throughout the life of the project.

On modernization projects, one of the more important purposes of configuration management is to assist project management in controlling the content of the developing system. Key activities necessary for proper control include establishing baselines for approved items and assuring that changes to baselined items are authorized, controlled, and tracked. As a result, an effective configuration management process assists project management in developing systems that meet the intended IRS business needs.

Although the e-Services project team had developed a configuration management plan that followed ELC requirements, the CSC was not following this plan. The project’s configuration management plan outlines proper controls over project documentation and indicates that a project document repository would be established to ensure version control over documentation and system modules.

However, the BSMO oversight reviews did not initially include project processes and procedures for configuration management. In addition, the CSC had not conducted any baseline reviews of the project and had cancelled other project level inspections or reviews. Therefore, the lack of access controls noted below was not detected until our review was conducted.

The project document repository (where critical project documentation was stored) was more like a "shared electronic project drawer" and was accessible to anyone on the CSC or IRS project teams. This repository included both baselined documents and those that were still in process. There was no version control over the key documents that had been accepted and signed by the IRS, and these accepted documents were not easily identified in this repository. In addition, change requests to control changes to key documents were not being used and approved as required by the plan.

Without proper configuration management processes over project documentation, confusion can occur within the project team and at the program level about which documents are the official versions. This confusion could result in project team members following the wrong set of project requirements in preparing future documentation or in coding the system modules. Ultimately, inconsistent processes could result in a lack of overall system integration at the program level.

Management Actions: Before we completed our audit, the BSMO and the CSC had initiated corrective actions regarding configuration management processes.

The BSMO implemented Configuration Management Program Instructions. In addition, the BSMO conducted a review and issued a report in February 2001 on the lack of effective configuration management processes. The report confirmed our observation that project baselines were not established for some modernization projects and configuration management inspections (such as baseline configuration audits) were not performed.

The CSC hired a Director to oversee the Configuration Management Office and developed revised procedures for products to be delivered to the IRS. The revised procedures require that baselines be established using formal change requests after products are approved.

Risk Management Processes Were Not Effectively Followed

Risk management is a critical process in project design and development to ensure a project team stays on track and is proactive in addressing potential problems. Effective risk management processes include documenting risk conditions and the risk status. This documentation serves as the basis for development and implementation of risk reduction strategies.

The e-Services project team developed a risk management plan that describes the ELC activities and processes the project will follow to manage project risks. The project team was working to identify the risks and issues it faces during the project, using the correct forms to document risks and meeting on a regular basis to discuss project risks. However, we identified weaknesses in the implementation of the risk management process, and recent changes in this process have resulted in a lack of effective risk and issue management in the project team.

We analyzed e-Services Risk Mitigation documents from July to October 2000 and found a lack of documentation for the identified risks. The risk statements did not adequately describe the potential risk impact, probable impact date, tracking of mitigation actions, use of measures to determine the effectiveness of mitigation efforts, and rationale for closing the risks. For instance, the project team’s actions to reduce or control risks related to integration with the STIR project and the Modernization Blueprint included coordinating requirements by means of discussions, briefings, and meetings. However, the risks were closed in August and October 2000, prior to completion of the necessary actions to reduce impact on project cost, schedule, or technical performance and without detailing how risks were successfully addressed. In November 2000, CBS ESC concerns over these integration issues contributed to project delays and cost increases (see pages 3-6).

The BSMO identified similar weaknesses in its analysis of project risks, along with inadequacies in risk statements, and problems raising critical risks to an appropriate oversight level. We also reported weaknesses in the development of risk management processes and procedures in a report issued in June 2000 and problems with tracking and reporting risks in a separate report issued in March 2001.

IRS project personnel indicated that not all the original risks identified were valid risks because they did not have a proper risk statement and mitigation plan and there was no assessment of such factors as cost, schedule, and functionality. They considered these risks to be just part of day-to-day project management. Therefore, they did not see an impact on the project costs and schedule when probable impact dates passed. In addition, they indicated that any cost projections related to risk impact were also likely not valid for the same reasons as the risks. As a result, in November 2000, the project team cancelled all its previously identified risks and began to identify new risks for the project. As of February 2001, the project team had only two risks that had been validated by the Project Manager, even though many of the conditions present in the original risk list had not been adequately addressed.

We believe that much of the confusion in this area has resulted from the continuous changes in the risk management process. Clearer guidelines, definitions, and requirements would help the project teams ensure that proper risk management occurs in the e-Services and other projects.

Management Actions: After our audit fieldwork was completed, the IRS and the CSC provided information that they agreed with our concerns and were implementing the following actions:

• The CSC developed a revised set of risk management procedures that were accepted by the IRS on March 28, 2001.
• The IRS accepted an Issues Management and an Action Item Management approach on March 12, 2001.
• A process action team was formed in April 2001 to develop a proposed concept for an executive risk management review board.

A Process for Monitoring Dependencies Has Not Been Effectively Implemented

Managing project dependencies is critical to the success of the project and the overall modernization program. The ELC procedures and the draft integrated schedule procedures indicate that it is the Project Manager’s role to ensure that project dependencies are identified and accurately reflected in reports. Identifying the risks associated with project dependencies is important so that all project teams and the BSMO can be aware of the criticality of any delays that may occur in completion of each project.

Dependencies are especially critical with the e-Services project. One of the causes for project delays was that the e-Services project was heavily dependent on the Modernization Blueprint and the STIR project, which were lagging behind the e-Services project. Without clear and accurate documentation of project dependencies, incorrect decisions may be made at a program level and additional delays in the project can occur.

In May 2000, the CSC began using an integrated schedule to identify dependencies between projects. In addition, it established a separate office to manage dependencies and hold weekly alert meetings to discuss upcoming dependencies. E-Services was one of the first projects to work with this office to develop dependencies between projects.

Although the e-Services project did a good job of identifying their critical dependencies on other projects, the team was not always consistently addressing the activities that were necessary to ensure the effective development of the e-Services project. For example, the e-Services project team did not adequately document the risks associated with Cross-Project Dependencies (CPD). Of the 28 CPD Request forms we reviewed, which documented both open and closed dependencies, none included a description of the risk to the e-Services project if the dependencies were not properly managed.

Since a CPD includes activities and products that could impact or delay the project, it is important to document the risk associated with the dependency along with all of the appropriate approval signatures that indicate all parties agree to the dependency.

Project Management Processes Can Be Improved

The e-Services project team was employing certain critical project management processes. The Project Manager used a Work Breakdown Structure (WBS) to manage the project team’s tasks. Each task was identified with a specific identification number and had an assigned start date, finish date, and estimated duration. However, we found evidence that other processes were not being followed consistently, such as assigning tasks to individual team members or allowing for adequate reserve or recovery time in the schedules.

For example, the Project Manager broke down the tasks in the WBS to various levels of detail and, at the lowest level, made assignments to function codes or groups of team members who would complete the task. However, we were unable to obtain names of the individuals in each of these groups who were assigned responsibility for the completion of the tasks that were due within 90 days. In addition, we did not see any indication that adequate reserve or recovery time had been budgeted for unforeseen circumstances that could occur.

The Project Management Institute Work Breakdown Structure Practice Standard states that, at the lowest level in the WBS, an individual should be identified and held accountable for the result. The individual may be an individual contributor creating the deliverable personally or a manager who will in turn create a WBS to plan and manage the results.

Accountability for completion of project tasks is important to ensure that projects are completed on schedule and in a quality manner. The Project Manager is assigned overall accountability for all the tasks, but he/she must delegate responsibility and accountability for the numerous tasks to the individual team members.

We discussed this issue with the e-Services Project Manager and other CSC officials, and they indicated that it was not their practice to assign tasks down to individual staff members. They said they do not find this information to be a useful part of the WBS. However, we believe that accountability should be documented either in the WBS or elsewhere by project personnel for tasks due in the very near future. Identifying specific team members can become critical if a person has specific skills and is working on more than one project. The Project Manager needs to be sure that this person will be available when needed, and this becomes difficult when team members are not specifically identified on the WBS.

We also reviewed the WBS to determine if there was reserve or recovery time allocated to address unplanned events that could occur. We did not identify adequate reserve time for critical tasks, and project management indicated that they do not separately allocate time to recover from unplanned events. Although reserve time is not always required on a project, it may be necessary if the current schedule is very aggressive or if a project team faces schedule risks, such as dependencies on other projects. This issue becomes even more critical as projects move towards development and deployment of systems that affect IRS personnel and taxpayers.

Conclusion

The e-Services project team has made significant progress in its design activities, and the products coming from the project will provide valuable benefits to taxpayers and tax practitioners. However, recent changes in project scope and dependencies on other projects have resulted in projected delays of several months and significant increases in estimated costs. Improved management processes are needed to meet project commitments and expectations.

Detailed Objective, Scope, and Methodology

The overall objective of our audit was to assess the Internal Revenue Service’s (IRS) processes for development and implementation of the e-Services project’s Fiscal Year (FY) 2002 Release. To accomplish this objective, we:

1. Reviewed exit documentation and deliverables for the system concept (also called Architecture) phase and discussed the exit process with the Project Manager to determine whether the project met all pre-determined exit criteria.
2. For the last approved milestone (Milestone 2, the System Concept phase), reviewed milestone exit documentation, Core Business Systems Executive Steering Committee (CBS ESC) meeting minutes, and discussed milestone exit criteria and conditional approval issues with the Business Systems Modernization Office (BSMO) staff and the Project Manager to determine whether this phase was approved conditionally or unconditionally.
0. Identified 70 conditions at the completion of the system concept phase and determined the conditions required to be met before exiting Milestone 2.
1. Judgmentally sampled 20 of the 70 conditions to determine whether the project team met the conditions within the required time period. We choose a judgmental sampling method because the population was too small for mathematical evaluation and we did not intend to project the sample over the population.
2. Determined whether meeting these conditions caused additional delays in subsequent delivery dates.
3. Of the 70 conditions, judgmentally sampled 11 of the 18 conditions that were delayed until future phases to determine whether they should have been addressed in the system concept phase in accordance with the Enterprise Life Cycle (ELC) document standard.
3. Determined that reviewing the current design phase documentation was not feasible because documents were not available due to changes in project requirements. Reviewed the last approved milestone (e.g., Milestone 2, systems concept phase, task order 41) for a list of deliverables and due dates, and compared these dates to the IRS’ review dates to determine whether the contractor met the system concept phase deliverable due dates.
4. Determined whether task orders (covering the past year - task orders 16, 41 and 49) adequately described requirements for the deliverables.
5. Interviewed BSMO and CSC personnel, and reviewed ELC procedures for acceptance of deliverables, and sign-off sheets for each deliverable and task order payment to determine whether a validation process was in place to cover each deliverable prior to the payment for the task order.
6. Reviewed the system concept phase deliverables, signatures for each deliverable listed, and task order payment dates and compared them to the deliverable dates.
7. Determined the cost of the work performed by reviewing task orders 16, 41, and 49, and the Preliminary Business Case, and by having discussions with the Project Manager.
0. Determined if firm fixed price mechanisms were used.
1. Interviewed the Contracting Officer and reviewed task order payment schedules and the FY 2002 project schedule to determine if task orders were defined, agreed to, and approved by all appropriate parties prior to the start of work on the task order. If not, determined how much delay occurred between beginning the work and defining and signing the task order.
2. Interviewed the Contracting Officer and BSMO personnel to determine if task orders included both incentives and disincentives of significant and similar consequence.
3. Determined whether modifications to task orders were approved to increase the original agreed upon firm fixed price cost. If so, identified what additional deliverables were requested and whether the additional deliverables justified the increased costs.
4. Compared the deliverables in task orders 16, 41, and 49 to determine if there were any duplicate deliverables or requirements included (the IRS paying twice for the same thing).
5. Reviewed the Preliminary Business Case (PBC), spending documents, and payment invoices to determine the dollar amount spent as of January 2001 on the project as compared to the projected costs in the PBC. Determined what percentage of the projected costs will be realized in FY 2002 and the amount that would not be realized in FY 2002. In addition, recalculated IRS Labor Development costs and compared them with the actual amount spent per the spending plan.

1. Determined whether the e-Services project was adhering to the ELC requirements.

1. Reviewed the status of configuration management.

1. Determined whether a configuration management plan had been developed for the project. Reviewed it to determine whether it addressed key items required by the ELC.
2. Determined whether a repository for project documentation had been established. Documented the location, how it was maintained, and who was responsible for the maintenance.
3. Determined whether the document list and the configuration items list had been defined and baselined. Reviewed these lists to determine whether they included all items that should be tracked.
4. Determined whether a process had been established for change requests to be initiated and approved/disapproved. Reviewed all change requests initiated by the project to determine whether approvals were documented, changes were not made prior to documented approval, and appropriate levels of approval were required (of the 10 change requests identified, 1 could not be located).
5. Determined whether changes were being tracked using some formal process and how the contractor was keeping the IRS informed of these changes.
6. Determined whether a repository for system modules was required for version control at the project’s current system development stage. Determined if this was addressed in the configuration management plan.
7. Determined whether all of the e-Services project’s approved baselined documents were properly controlled in the document repository.

1. Determined whether all system concept phase approved documents had been baselined and included in the repository.
2. Determined whether access to baselined documents was properly restricted.
1. Obtained and reviewed a printout of the authorization file for the e-Services repository. Determined who had access to this repository.
2. Reviewed the privileges for the individuals in the authorization file. Determined who had write, edit, add, and delete privileges to the baselined documents.

1. Reviewed the status of risk management.

1. Determined whether the project used a risk inventory and assessment worksheet to document risks.
2. Determined whether the project effectively measured the potential quantitative and qualitative effects of the risks and whether the most critical risks were raised to an appropriate level of IRS management in a timely manner.
3. Determined whether all critical documented risks were addressed with mitigation plans. Reviewed mitigation plans for all critical risks to determine whether the plan appeared to effectively address the risks and whether the actions contained in the plan were being tracked and were on schedule. If not, documented the delays in completing mitigation actions for the associated risks and the potential effect of not completing the mitigation actions.
4. Determined whether issues that were identified for the June through October 2000 period were effectively documented, assigned, and tracked until completion. Reviewed a database extract of the entire project’s issues and determined whether issues had been fully and timely addressed. In addition, analyzed oversight reports on project issues reviewed.
5. Reviewed the Work Breakdown Structure (WBS) and Program Management Reviews (PMR) to determine if the project was behind schedule; if so, determined how far. If the project was behind, determined whether this was identified as a risk to the project and was accurately reported to the appropriate officials. In addition, determined whether reserve or recovery time was allocated in the WBS.

1. Reviewed the status of requirements development.

1. Interviewed the e-Services Requirements Manager and Technical Architect, and reviewed Systems Requirements Workshop documentation and Program Management & Control and Leadership Meeting minutes to determine whether a formal process (workshops, etc.) was being used to gather system requirements focusing on stakeholders and individuals at both the executive and technical levels of the business function.
2. Interviewed the e-Services Requirements Manager and Technical Architect and reviewed Customer Technical Reviews and Project Architecture Integration Team meeting documentation to determine whether requirements were being analyzed for consistency, completeness, and feasibility. Reviewed this analysis to determine whether it appeared comprehensive.
3. Determined whether the systems requirements were being documented and approved by IRS officials in the business units and in the Modernization and Information Technology Services (formerly known as Information Systems) organization.
4. Determined whether the project had documented requirements traceability matrices that interrelate the various work products.
5. Reviewed various meeting documentation for the July through November 2000 period, and all other required briefings to identify issues and proper resolution.

4. Reviewed the status of project staffing.

1. Interviewed the CSC and IRS Project Managers and reviewed project team organization charts to determine whether the project had been able to effectively recruit and retain sufficient staff to accomplish its tasks. Identified any significant staff turnover.
2. Selected one WBS, as of November 17, 2000, to analyze and determine whether all tasks scheduled to start within 90 days of that date were assigned to a project staff person. Reviewed the CSC policy on task assignment.
3. Interviewed the CSC Training Program Manager and the CSC and IRS Project Managers and reviewed training plans, skill assessments, and other documentation to determine whether a skills assessment had been performed for the project team and a training plan developed to address any skills deficiencies.

1. Determined whether the project was adhering to Modernization Blueprint requirements.

1. Determined whether the project’s Preliminary Business Case adequately documented how the project fits into the IRS’ Modernization Blueprint.
2. Interviewed the project’s Technical Architect and reviewed key Modernization Blueprint deliverables that relate to the e-Services project and any specific product selections or overall architecture direction information that could affect the project. Compared these to the e-Services project’s design documents to determine whether the project’s direction was consistent with that of the IRS’ overall modernization.
3. Determined whether the Modernization Blueprint provides sufficient direction for this project to proceed.

1. Determined whether project dependencies were effectively and efficiently managed to minimize project delays.
0. Interviewed the Project Manager and Program Release Manager and reviewed the Integrated Master Schedule, Cross Project Dependencies Reports and Request Forms, Alert Meeting Minutes for the August through December 2000 period, WBSs, and PMR Release Overviews.

1. Identified cross-project dependencies and the controls over them.
2. Identified slippage of significant tasks and how any slippage affected the project. Determined whether any technical designs or business processes had been redone or eliminated.
3. Evaluated management’s use of the above documentation, including whether analyses were performed and how a dependency becomes a project issue or risk.

1. Determined if the project had a contingency plan if the Security and Technology Infrastructure Release project (and/or any other project on which the e-Services Project is dependent) was significantly delayed.
2. Determined if a tracking process was used for dependencies.
3. Evaluated the process for adding, changing, or deleting dependencies by reviewing all e-Services project Cross Project Dependencies Request Forms covering the period August 2000 through December 2000. Identified 28 Cross Project Dependencies Forms.
4. Evaluated the coordination efforts between the e-Services project and the other projects that are dependent on each other. Determined if there were mutual understanding and corroboration among the projects.

1. Determined whether the project’s sponsor met his/her responsibilities and time commitments and assisted in bringing the project to its successful completion.

1. Interviewed the Project Manager and IRS personnel; reviewed CBS ESC, Sub ESC, Leadership Meeting, and Program Management & Control meeting minutes; and reviewed Modernization Activity Reports and other documentation to evaluate the sponsor’s commitment of time to the project.

1. Determined the sponsor’s specific responsibilities (e.g., approve project plans, budgets, and schedules; manage project scope; ensure timely availability of needed resources; resolve major issues, etc.) and time devoted to each activity.
2. Determined what sponsor activities were performed to champion the project and the project team.
3. Determined if the sponsor defined a specific set of vital signs for the project.
4. Determined if the Project Manager had quick and easy access to the sponsor for timely resolution of important issues.
5. Determined if the sponsor routinely met with the Project Manager to assess the project’s progress and the status of various vital signs.
6. Determined how the Project Manager felt about the quality of the sponsorship being provided.

1. Reviewed documentation from the project used to monitor the status of critical vital signs.
2. Reviewed CBS ESC and Sub ESC Charters to determine what mutual understanding and corroboration existed between the project and the sponsor.

Major Contributors to This Report

Scott E. Wilson, Assistant Inspector General for Audit (Information Systems Programs)

Scott A. Macfarlane, Director

Tammy L. Whitcomb, Audit Manager

Allen W. Gray, Acting Audit Manager

Jimmie D. Johnson, Senior Auditor

Esther M. Wilson, Senior Auditor

George L. Franklin, Auditor

Suzanne M. Noland, Auditor

Report Distribution List

Commissioner N:C

Associate Commissioner, Business Systems Modernization M:B

Deputy Associate Commissioner, Program Management M:B

Deputy Associate Commissioner, Program Planning and Control M:B

Deputy Associate Commissioner, Systems Integration M:B

Director, Strategic Planning and Client Services M:SP

Chief Counsel CC

National Taxpayer Advocate TA

Director, Legislative Affairs CL:LA

Director, Office of Program Evaluation and Risk Analysis N:ADC:R:O

Office of Management Controls N:CFO:F:M

Audit Liaison:

Associate Commissioner, Business Systems Modernization M:B

Outcome Measures

This appendix presents detailed information on the measurable impact that our recommended corrective actions will have on tax administration. These benefits will be incorporated into our Semiannual Report to the Congress.

Type and Value of Outcome Measure:

• Reliability of Information – Potential; $1.6 million (see page 7).

Methodology Used to Measure the Reported Benefit:

The Business Systems Modernization Office (BSMO) information shows that labor costs for Internal Revenue Service (IRS) employees working on the e-Services project team totaled $199,000 for October 1999 through January 2001. Our calculations of IRS labor costs for the same period totaled $1.8 million, a difference of $1.6 million. We developed our estimates using the following information.

The BSMO reported that there were 17 Full Time Equivalents (FTE), which is equal to approximately 17 IRS employees, assigned to the e-Services project team. We used labor costs of $80,000 per FTE (per the IRS Chief Financial Officer’s office) and divided it by 12 months to obtain $6,666.67 per month. We multiplied this figure by 16 months (October 1999 to January 2001) to obtain $106,666.67, which represents the IRS labor costs per employee. We then multiplied this figure by the 17 FTEs to get the total estimated labor costs of $1,813,334.