Requirements Changes and Testing Delays Have Further Increased the Costs and Delayed the Benefits of the e-Services Project
Reference Number: 2004-20-036
This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.
February 17, 2004
MEMORANDUM FOR CHIEF INFORMATION OFFICER
FROM: Gordon C. Milbourn III /s/ Gordon C. Milbourn III
Acting Deputy Inspector General for Audit
SUBJECT: Final Audit Report - Requirements Changes and Testing Delays Have Further Increased the Costs and Delayed the Benefits of the e-Services Project (Audit # 200320023)
This report presents the results of our review of the Internal Revenue Service’s (IRS) e-Services Release 1.1 Development. The e-Services project will provide a set of Web-based business products as incentives to third parties to increase electronic filing, in addition to providing electronic customer account management capabilities to all businesses, individuals, and other customers. The overall objective of this review was to determine whether the Business Systems Modernization Office (BSMO) was providing adequate oversight on the e-Services project in the critical areas of business case development, requirements management, testing, and transition to support.
In summary, although the BSMO deployed the initial release of the e-Services system in late August 2003, the project continues to experience delays and cost increases due to requirements changes and testing environment problems. We identified opportunities for improvement in business case development, requirements management, and testing oversight.
Cost estimates and planned schedules included in previous business cases have been significantly exceeded throughout the project’s lifecycle. Additionally, business case information provided to gain approval for Fiscal Year 2004 funding did not include updated cost information available at the time of its preparation.
Management of system requirements is critical to ensure the developed system meets user needs. We reviewed the e-Services system requirements to determine whether requirements approved at the end of the design phase were developed and tested. We found that numerous requirements were deleted and documentation of changes made to system requirements was not easily traceable from the requirements database to approval documents.
Several hundred critical defects were identified during acceptance testing. This testing was one of the last phases of testing and occurred after the system had passed other testing phases. Additionally, many of these issues required software changes. Defects identified during both integration and acceptance testing have taken much longer to resolve than initially planned.
To address these issues, we recommended that the Chief Information Officer (CIO) require that system requirements be fully developed before project development begins and that changes after that point meet stringent criteria. Additionally, future submissions of project information should include up-to-date cost and schedule information, and a formal lessons learned document should be developed to fully explore e-Services project issues.
The CIO should require management in the BSMO to ensure that the e-Services requirements database clearly reflects changes and approval for those changes. To improve testing, the CIO should require the BSMO to develop incentives to increase PRIME contractor accountability in the areas of software quality and testing. A process should also be established to review and accept test plans before allowing the PRIME contractor to begin testing.
Management’s Response: Management fully agreed with four of our recommendations, partially agreed to one recommendation, and disagreed with one recommendation. IRS management plans to take the following actions: establish a process that calls for gathering requirements, implementing a requirements traceability verification matrix that maps system requirements or other test components to test cases, and identifying the verification and validation method for each requirement or test component; and implement the recommendations from recent Business Systems Modernization program reviews. IRS management also plans to: continue to submit cost information to the Office of Management and Budget (OMB) according to OMB requirements, review and guidance; develop a thorough lessons learned document; and add additional fields to the requirements database to ensure clear traceability and level of approval indicator.
In addition, the CIO has issued a directive that requires fixed-price contracting for all systems development and implementation projects. This will require the PRIME contractor to provide its written assurance at a newly established checkpoint that due diligence was performed in defining all significant business requirements. Management’s complete response to the draft report is included as Appendix V.
Office of Audit Comment: Management partially agreed with our recommendation to develop incentives to increase PRIME contractor accountability in the areas of software quality and testing. Management has included items to increase contractor accountability in the new Systems Integration Task Order for this fiscal year. However, they did not include the specific incentive for the contractor for systems that successfully pass Government acceptance testing with a minimal level of software defects. We agree with this corrective action because the fixed-priced contract and the written assurance at the new checkpoint will hold the PRIME contractor accountable for systems that do not pass Government acceptance testing due to software defects, although it is not specifically defined in the task order.
Management disagreed with our recommendation to establish a process to review and accept test plans before allowing the PRIME contractor to begin testing. Rather than implement this recommendation, which would require a formal deliverable-based solution, management has implemented an approach that gives the development of the plan, test cases, and other test artifacts to an integrated product team. This allows the team to complete the plan as the required information becomes available and firm, sometimes even into the initial days of the testing, while assuring that all stakeholders approve of and support the plan as completed. While we agree that having an integrated product team environment is a good approach, we believe that the plan should be completed and approved before testing starts to allow more effective testing. If proper planning is implemented, then foreseeable problems can be resolved up front instead of in the last phases of testing, reducing the number of critical defects. While we still believe our recommendation is worthwhile, we do not intend to elevate our disagreement concerning it to the Department of the Treasury for resolution.
Copies of this report are also being sent to IRS managers affected by the report recommendations. Please contact me at (202) 622-6510 if you have questions or Margaret E. Begg, Assistant Inspector General for Audit (Information Systems Programs), at (202) 622-8510.
In the Internal Revenue Service (IRS) Restructuring and Reform Act of 1998 (RRA 98), the Congress challenged the IRS to promote its electronic filing programs to the point where 80 percent of all tax returns would be filed electronically by the 2007 Filing Season. Moreover, by the 2003 Filing Season, most, if not all, returns prepared electronically should be filed electronically as well. Unfortunately, if the IRS’ current systems are not updated and refined, the aforementioned goals will not be met. Therefore, to help meet the Congressional mandate, the IRS created the e-Services project.
The e-Services project will provide a set of Web-based business products as incentives to third parties to increase electronic filing. The project focuses on fostering easy-to-use electronic products and services targeted at specific practitioner segments that inform, educate, and provide service to the taxpaying public. In addition, the e-Services project will provide electronic customer account management capabilities to all businesses, individuals, and other customers.
The e-Services project is broken down into several releases in 2003 – Releases 1.1, 1.2 and 2.0. Collectively, the releases aim to provide agency-wide registration, authorization, authentication, and application procedures to select Electronic Return Originators (ERO). In addition, the EROs will be provided with on-line power of attorney application, Taxpayer Identification Number (TIN) matching, transcript delivery, and electronic account resolution. Also, the 2003 releases will support the Modernized e-file (MeF) project.
We reviewed available documentation and interviewed PRIME contractor, IRS, and Business Systems Modernization Office (BSMO) executives, managers, and analysts located at the IRS National Headquarters and the New Carrollton, Maryland, offices. We performed the audit from November 2002 through October 2003 in accordance with Government Auditing Standards. Detailed information on our audit objective, scope, and methodology is presented in Appendix I. Major contributors to the report are listed in Appendix II.
The IRS deployed the first release of the e-Services system, Release 1.1, in late August 2003. Although the system initially encountered various issues, including limitations on accessibility of the system to the IRS employees who will be processing activity through the system, the project team was working with the system users to correct the issues.
Launching the initial release provided the opportunity for internal and external customers to review, test, and approve specific e-Services products. This initial release allows tax professionals the ability to register online and create an electronic account, and to apply for a Preparer Tax Identification Number to use in place of their Social Security Numbers for submitting returns. In addition, this release will allow for some of the TIN matching capabilities described earlier.
Although the initial release deployment was not a total success, problems identified during the early deployment of this release are currently being resolved and plans to deploy subsequent releases are continuing. Although the deployment of the release occurred in late August 2003, the marketing of the system to external users, primarily tax practitioners, was delayed until November 2003 to ensure that any remaining issues were resolved.
Cost estimates and planned schedules included in previous business cases have been significantly exceeded throughout the project’s lifecycle. Additionally, business case information provided to gain funding approval for Fiscal Year (FY) 2004 did not include updated cost information available at the time of its preparation.
Significant schedule delays and cost increases that were evident during the design phase of the e-Services project are continuing in the development and implementation phases
In our earlier e-Services audit report, we indicated that costs through the design phase of the e-Services project had quadrupled from nearly $4 million to approximately $17 million, and delays of approximately 9 months had been experienced. Current development, testing, and deployment phase cost projections total $119.1 million, which is an increase of $71.9 million (152 percent) from the baseline business case projection of $47.2 million. To date, delays of 14 months have also been experienced.
A key reason for these cost increases and schedule delays was the significant number of changes made to the e-Services system after development began. As of May 2003, we identified 56 separate change requests that were approved for the e-Services project after it started the development phase. According to BSMO management, many of these changes were necessary because of corresponding changes in the tax law. Although a change control process has been established to review all proposed changes, we believe that the process should be more stringent and better budgeting practices should be employed to enable more accurate cost estimation for future changes.
Although negotiations are underway to obtain contractor agreement to a firm fixed-price cost for the remaining development efforts, at least one significant item, continuance of transition to support activities, was removed from contract requirements during these negotiations. These requirements will have to be included in a subsequent contract, even though they were included in the original cost estimates for the development tasks. Delaying these requirements could negatively affect efforts already underway to enable the business organization to have the capability to support the system once it is in operation, and will likely result in extending the length of time that the PRIME contractor is paid to maintain the system.
The schedule slippage with Release 1.1 of the e-Services system will delay deployment of services, such as registration of EROs, TIN matching, and secure electronic mail (e-mail). In addition, since the MeF project is dependent on the implementation of e-Services, such functionalities as allowing tax professionals and businesses to register and make application to file certain business tax forms electronically have also been affected by delays in deploying the e-Services project.
Management Actions: Management plans to improve its budgeting process by increasing the amount of funding allocated to the management reserve account. This account will cover unplanned items, such as changes to the tax laws. In addition, an initiative has been started to better integrate periodic tax law changes into the systems development process.
Business case information presented to justify the FY 2004 budget for the e-Services project did not include updated costs
The FY 2004 Capital Asset Plan and Business Case (Exhibit 300) documents submitted to the Office of Management and Budget (OMB) were not accurate at the time of submission. These documents, submitted to the OMB in the fall of 2002, did not include updated cost information or earned value data that was available at the time it was submitted. Specifically, data in the Alternatives Analysis Section were obtained from the e-Services business case dated July 31, 2001, even though earned value data clearly indicated that these estimates were not accurate and would be greatly exceeded.
The Exhibit 300 also did not adequately reflect the earned value data showing negative performance in excess of 10 percent, which had been ongoing since early 2002. Lastly, the return on investment (ROI) calculations did not include costs incurred to design the system. Although the Enterprise Life Cycle (ELC) guidance allows these “sunk” costs to be excluded, we believe that they should be disclosed in the ROI calculations in the OMB Exhibit 300 submission to provide a more accurate estimate of project costs and benefits.
The OMB establishes policy and provides instruction on budget justification and reporting requirements for major acquisitions and major information technology systems or projects. That policy requires that all of the areas of the Exhibit 300 must be part of the agency’s planning and should be updated as soon as the information is known. While the Exhibit 300 documents are officially submitted to the OMB twice yearly, they should be used as management tools within an agency and updated as the information is available.
In addition, the OMB requires that if any of the Exhibit 300 cost, schedule, or performance variances are a negative 10 percent or more, the Exhibit must provide a complete analysis of the reasons for the variances, the corrective actions that will be taken, and the most likely estimate at completion. The earned value system should be used to identify the specific work packages where problems are occurring, and reasons for problems and corrective actions necessary to return the program, as close as feasible, to the current baseline goals should be disclosed. The rationale for the latest estimate at completion should be disclosed and discussed.
Although the IRS took corrective actions in response to the negative variances, which eventually resulted in a significant Task Order modification based on the e-Services revision of cost and schedule estimates, it did not detail the need for this action in the FY 2004 Exhibit 300. As a result, updated Net Present Value and ROI data, as well as accurate earned value data, were not reflected in the FY 2004 Exhibit 300.
The e-Services team indicated that the guidance used to prepare the FY 2004 Exhibit 300 did not clearly define what was required and has since been updated. Additionally, management stated that although earned value data had not been updated in its FY 2004 submission, the cost overruns that e-Services had experienced were clearly disclosed to representatives from the OMB and other oversight organizations in executive briefings.
Management Actions: The FY 2005 Exhibit 300 submission for the e-Services project was being prepared as we conducted our audit. Management indicated that updated cost and schedule figures were included in this submission; however, ROI data did not include “sunk” costs. Because the submission documents were still in process when we completed our fieldwork, we were unable to review them to ensure latest cost figures were included.
Additionally, new guidance has been drafted by the Information Technology Services organization to assist project teams in preparing their Exhibit 300s for the OMB. We reviewed a draft of this guidance, and the September 2003 version requires inclusion of ROI figures calculated both with and without “sunk” costs.
To better control cost increases and schedule delays, we recommend that the Chief Information Officer (CIO):
1. Establish a more formal discipline to ensure system requirements are established prior to beginning development. Strict criteria should be established to restrict changes once development begins, and any required changes that do not meet the stringent criteria should be made in a subsequent release.
Management’s Response: Management agreed to establish a process that calls for the gathering of requirements, a requirements traceability verification matrix, and the identification of the verification and validation method (e.g., test, demonstration, analysis, etc.) for each requirement or test component. They will also implement the recommendations from recent Business Systems Modernization (BSM) program reviews to further improve the requirements management area. These recommendations will be formally tracked and monitored by their executives and include: clearly define business requirements and tightly manage them to control scope; refine the change request process; and develop criteria for project decision-making including impacts and trade-offs.
2. Ensure that the most current available project costing information, including accurate earned value data and all costs expended to develop the e-Services system, are included in the FY 2005 Exhibit 300 submission to the OMB.
Management’s Response: Management agreed to continue to submit cost information to the OMB according to the OMB’s requirements, review, and guidance.
3. Require that a formal lessons learned document be developed to thoroughly explore the issues experienced in the design and development of the e-Services project so that those issues are not experienced in other projects.
Management’s Response: On December 15, 2003, management completed a lessons learned document to address the issues identified during Releases 1.1 and 1.2. A thorough lessons learned document will be developed as part of the post checkpoint activities as called for by the ELC.
Management of system requirements is critical to ensure the developed system meets user needs. We reviewed the e-Services system requirements to determine whether requirements approved at the end of the design phase were developed and tested. We found that numerous requirements were deleted and documentation of changes made to system requirements was not easily traceable to the approval documents or to the requirements database.
We selected a statistically valid sample of 68 system requirements from Release 1.1 of the e-Services project, which were approved when the e-Services project moved from the design phase to the development phase. We intended to trace these requirements through to testing to ensure that they passed the first phase of integration testing. Of the 68 we selected, 25 (37 percent) were deleted from the requirements as the system was developed. Of these 25 requirements, 20 were originally labeled as “must” requirements, which indicates that the system has to have this functionality. An additional 13 system requirements were delayed to a future release of the e-Services project. Thus, of the 68 requirements we randomly selected, over half were not included in Release 1.1 of the e-Services system.
We attempted to locate documentation of IRS approval of the deletion or deferral of these requirements. Although e-Services personnel indicated that the approvals of these deletions and deferrals were documented in the change request documents, the requirements database, which tracks all system requirements, did not clearly identify the change requests that incorporated the approvals of these deletions and deferrals. Therefore, the IRS will have difficulty ensuring that changes made to the requirements database have been approved. Without requiring a change request number documenting approval for changes to the requirements, unauthorized changes could be made. The resulting system may not meet the users needs.
The IRS indicated that a significant number of additional requirements were added to the system requirements after the e-Services project entered the development phase. Thus, numerous changes were made to the requirements approved at the end of the design phase. Many of these changes were made in meetings, but the minutes did not clearly reflect which specific requirements were affected.
A separate audit effort has been initiated to follow up on these and other system requirements issues. The recommendation in this report is focused solely on ensuring that the e-Services project requirements are adequately addressed.
To ensure that the e-Services project requirements are effectively addressed, the CIO should:
4. Require the BSMO to ensure that the e-Services requirements database clearly reflects when a requirement is changed, and which approval change request documents the change. Additionally, the change request document should indicate which specific requirements are affected by the change.
Management’s Response: Management has prepared a change request to add additional fields to the requirements database to ensure clear traceability and level of approval indicator. As mentioned in management’s response to Recommendation 1, management will be implementing the recommendations from recent BSM program reviews to further improve the requirements management area.
Several hundred critical defects were identified during acceptance testing. This testing was one of the last phases of testing and occurred after the system had passed other phases of testing. Many of these issues required software changes. Although our judgmentally selected sample indicated that these defects eventually appeared to be resolved, it took much longer to resolve them than initially planned.
There were a significant number of defects identified during acceptance testing, after the system had passed the majority of its integration testing
The IRS conducted Systems Acceptance Testing (SAT) for the e-Services project after the PRIME contractor had passed the system through quality testing, project testing, and the majority of its system testing. The SAT identified 362 Priority 1 and 2 issues – the two highest categories of criticality. Many of the issues identified during testing required actual software changes or corrections before they could be resolved. Of the 61 Priority 1 issues, 16 (26 percent) required software corrections, as did 160 (53 percent) of the 301 Priority 2 issues. Other issues included processing problems, necessary documentation changes, and configuration issues.
Because of the high number of software corrections required as a result of SAT testing, the rigor with which the quality and integration testing before SAT was conducted is questionable. The BSMO works with the PRIME contractor to develop and conduct test activities, but places a heavy reliance on the PRIME contractor to ensure these activities are thorough and complete. Even though the ELC requires that reviews of test plans be conducted, the BSMO does not approve test plans before the tests are conducted and does not review all defects to ensure they have been adequately resolved before allowing the PRIME contractor to move to the next test phase.
The BSMO had a close working relationship with the PRIME contractor with respect to the development and testing of the e-Services system. We believe that this close relationship, while a positive attribute in some instances, is a detriment with respect to review and approval of the various testing activities. While the BSMO seems to be closely involved in the development of the test plans, it has not seen the need for a formal process to review and approve the test plans, and to ensure they are conducted effectively.
In the absence of a formal process to review and approve each phase of the test plans, there may not be a clear understanding of the system requirements and the best way to test each requirement. As a result, testing could take longer than initially planned, or there may not be sufficient test cases identified to test each requirement. Failure to test each requirement could result in the system not performing as designed.
The e-Services system testing took several months longer than initially planned, partially because there was not a clear understanding of what was required to prove that requirements were met. Additionally, because the e-Services project has used a cost-plus fee contract, these delays resulted in increased costs to the IRS. Although we were unable to determine the exact amount of additional costs to the IRS for the delays in testing, the cost projections for development, testing, and deployment have increased nearly $72 million from estimates made prior to beginning the development.
The majority of defects identified during integration and acceptance testing were not resolved within planned schedules
We selected a judgmental sample of 44 defects and reviewed them for resolution. These defects appeared to be adequately resolved. However, we further examined the entire population of critical and high-priority defects identified during integration and acceptance testing for Release 1.1 of the e-Services project and found that the vast majority of critical and high-priority issues were not resolved within required time periods. For integration testing, resolution time periods were 24 hours for critical issues and 3 days for high-priority issues. For acceptance testing, resolution time periods were 5 days for critical issues and 10 days for high-priority issues.
Delays in resolving defects resulted in overall testing delays. Originally, acceptance testing was planned to be completed in February 2003. However, this testing was not completed until June 2003. These delays were due in part to the number of defects identified and the length of time to resolve the defects.
We believe the PRIME contractor did not respond to and resolve defects identified during integration and acceptance testing within expected time periods because the system was not adequately tested for quality in the initial test phases. Defects identified in the initial phases of system testing were not always effectively resolved when first identified. Defects become more difficult and costly to resolve as the system moves through various phases of testing.
Not resolving defects timely becomes critical because the PRIME contractor has been working under a cost-plus fee contract. Thus, delays in resolving defects result in additional costs to the IRS. Additionally, these delays have resulted in implementation delays for the e-Services system. Lastly, the PRIME contractor has no contractual obligation to repair defects once the e-Services system goes into operation. Any repair work accomplished by the PRIME contractor once the system is operational is also performed on a cost-plus fee basis or may be negotiated at the time of discovery of the defect.
According to a National Institute of Standards and Technology (NIST) study, it becomes increasingly more expensive to resolve errors the later in the test and/or operational cycle that the errors are discovered. The NIST estimated that, relative to resolving an error during the design phase of a project, it is 90 times more expensive to resolve during the system’s testing phase, but up to 880 times more expensive to resolve after the system is operational.
To ensure that testing for the remaining releases of the e-Services system and other BSMO projects is effectively handled, we recommend that the CIO require the BSMO to:
5. Develop incentives to increase PRIME contractor accountability in the areas of software quality and testing. These incentives should reward the PRIME contractor for systems that successfully pass Government acceptance testing with a minimal level of software defects.
Management’s Response: Management partially agreed with our recommendation. Management agreed that contractor incentives should be applied to the integration and test area. They have included items to increase contractor accountability in the new Systems Integration Task Order for this fiscal year. However, they did not include the specific incentive for the contractor for systems that successfully pass Government acceptance testing with a minimal level of software defects.
The CIO issued a directive that requires fixed-price contracting for all systems development and implementation projects. Since fixed-price contracting requires mutually agreed specifications, management is establishing a new checkpoint, at which point these specifications should be developed. Management will required the PRIME contractor to provide its written assurance at this checkpoint that it performed due diligence in defining all significant business requirements.
Office of Audit Comment: We agree with management’s corrective action to increase PRIME contractor accountability in the new Systems Integration Task Order for this fiscal year. We believe that the fixed-price contracting and the written assurance at the new checkpoint will hold the PRIME contractor accountable for systems that do not pass Government acceptance testing due to software defects, although it is not specifically defined in the task order.
6. Establish a process to review and accept test plans before allowing the PRIME contractor to begin testing.
Management’ Response: Management disagreed with this recommendation. Rather than implement this recommendation, which would require a formal deliverable-based solution, management has implemented a combined integration test approach. This approach gives the development of the plan, test cases, and other test artifacts to a combined team, which includes the PRIME contractor, Product Assurance, the BSMO, and business personnel, in an integrated product team environment. This allows the team to complete the plan as the required information becomes available and firm, sometimes even into the initial days of the testing, while assuring that all stakeholders approve of and support the plan as complete.
Office of Audit Comment: We agree that having an integrated product team environment to approve of and support the completed plan is a good approach. However, the plan should be completed and approved before testing commences to allow more effective testing. If proper planning is implemented, foreseeable problems can be resolved up front instead of in the last phases of testing, reducing the number of critical defects.
The overall objective of this review was to determine whether the Business Systems Modernization Office (BSMO) was providing adequate oversight on the e-Services project in the critical areas of business case development, requirements management, testing, and transition to support.
To accomplish this objective, we performed the following reviews and analyses:
I. Determined if the e-Services project business case was accurate and supportable, and provided information to justify continuing project development and maintenance throughout its life cycle.
II. Determined if requirements management activities would ensure that the e-Services project met user needs by selecting and reviewing a statistically valid sample of 68 e-Services Release 1.1 requirements from a population of 851 requirements approved when the system moved from the design phase to deployment. We intended to project the results of this sample to the population; however, because the requirements changes during system development were so extensive, we decided to simply report on the results of our sample.
III. Determined whether adequate controls were in place over Release 1.1 testing activities to ensure the system delivered would meet user requirements by selecting and reviewing a judgmental sample of 44 out of 611 critical defects identified during testing to determine whether they were adequately resolved. We used a judgmental sample because we did not plan on projecting the results.
IV. Determined whether adequate controls were in place over transition to support activities to ensure Internal Revenue Service readiness for the deployed system.
Margaret E. Begg, Assistant Inspector General for Audit (Information Systems Programs)
Gary V. Hinkle, Director
Scott A. Macfarlane, Director
Tammy Whitcomb, Audit Manager
Cari Fogle, Acting Audit Manager
Glen Rhoades, Acting Audit Manager
Jimmie Johnson, Senior Auditor
Paul Mitchell, Senior Auditor
Jacqueline Ngyuen, Senior Auditor
Esther Wilson, Senior Auditor
George L. Franklin, Auditor
Suzanne Noland, Auditor
Office of the Commissioner – Attn: Chief of Staff C
Deputy Commissioner for Operations Support OS
Associate Commissioner, Business Systems Modernization OS:CIO:B
Chief, Information Technology Services OS:CIO:I
Deputy Associate Commissioner, Business Integration OS:CIO:B:BI
Deputy Associate Commissioner, Program Management OS:CIO:B:PM
Deputy Associate Commissioner, Systems Integration OS:CIO:B:SI
Director, Tax Administration Modernization OS:CIO:B:PM:TAM
Acting Director, Portfolio Management Division OS:CIO:R:PM
Chief Counsel CC
National Taxpayer Advocate TA
Director, Office of Legislative Affairs CL:LA
Director, Office of Program Evaluation and Risk Analysis RAS:O
Office of Management Controls OS:CFO:AR:M
Associate Commissioner, Business Systems Modernization OS:CIO:B
Chief, Information Technology Services OS:CIO:I
This appendix presents detailed information on the measurable impact that our recommended corrective actions will have on tax administration. This benefit will be incorporated into our Semiannual Report to the Congress.
Type and Value of Outcome Measure:
· Reliability of Information – Actual; $71.9 million increase on baseline business case projection (see page 3).
Methodology Used to Measure the Reported Benefit:
We compared the e-Services Baseline Business Case Milestone 4/5 projections of approximately $47.2 million (representing current development, testing, and deployment phase cost projections) with the revised estimates of approximately $119.1 million. There was an increase of approximately $71.9 million (152 percent).
The response was removed due to its size. To see the response, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.