Business Cases for information Technology Projects Remain Inaccurate

TREASURY INSPECTOR GENERAL FOR TAX ADMINISTRATION

Business Cases for Information Technology Projects Remain Inaccurate

January 25, 2007

Reference Number: 2007-20-024

This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.

Redaction Legend:

3(d) = Identifying Information - Other Identifying Information of an Individual or Individuals

Phone Number | 202-927-7037

Email Address | Bonnie.Heald@tigta.treas.gov

Web Site | http://www.tigta.gov

January 25, 2007

MEMORANDUM FOR CHIEF INFORMATION OFFICER

FROM: Michael R. Phillips /s/ Michael R. Phillips

Deputy Inspector General for Audit

SUBJECT: Final Audit Report – Business Cases for Information Technology Projects Remain Inaccurate (Audit # 200620005)

This report presents the results of our follow-up review on the Internal Revenue Service’s (IRS) management of information technology investments. The overall objective of the review was to determine whether the IRS took effective corrective actions to address the recommendations in our previous audit report.[1] Also, we evaluated whether the IRS is managing its information technology investments in compliance with Office of Management and Budget (OMB) and Clinger-Cohen Act of 1996[2] requirements.

Impact on the Taxpayer

The IRS spends approximately $2 billion annually on information technology investments. In this follow-up review, we determined the IRS business cases used to manage and fund specific information technology investments remain inaccurate and unreliable. IRS business case inaccuracies distort the true life-cycle costs of information technology investments and present a false depiction of the IRS’ information technology portfolio, resulting in potential waste and mismanagement of taxpayer dollars.

Synopsis

The Federal Government invests over $60 billion in information technology investments annually. Motivated to improve information technology budget stewardship, Congress enacted legislation directing agencies to use better business practices to manage and report major information technology investments. In April 2005, we issued a report that addressed whether the IRS planned, managed, and controlled its information technology investments in compliance with OMB and Clinger-Cohen Act requirements. We reported project costs had not been reported accurately in the business cases, relevant cost and benefit information had been omitted, progress on development projects had been measured inaccurately, and business cases for operational projects did not demonstrate the results of an E-Government review.[3]

The IRS took several actions to address the weaknesses cited in our previous report; however, most of the weaknesses remain unresolved. Specifically:

· Project costs are still being reported inaccurately. Costs cannot be substantiated, indirect costs for management and overhead are not allocated to projects, and security costs are not reported accurately.

· Progress on development projects continues to be measured inaccurately. Actual costs used in progress calculations were understated and baselines for cost and schedule are continually revised. In addition, the IRS did not review contractors’ procedures for tracking their cost and schedule information, as required.

· The Department of the Treasury’s programming of the software used to prepare business cases continues to contribute to inaccuracies in the business cases. Systemic problems with the software programming contributed to the total costs being reported inconsistently making it difficult to determine the projects’ true financial status.

· Two major systems were not included in the IRS budget submission for Budget Year[4] 2007, and a business case should have been prepared for an additional system based on the expected costs of the system. Failure to provide business cases for all required systems detracts from the OMB’s ability to allocate information technology funding and from the IRS’ ability to adequately monitor and manage the costs and benefits of its major systems.

We believe senior IRS executives and Department of the Treasury and OMB officials still cannot rely on the data in these business cases to manage and fund the projects. Inaccurate information in business cases can distort viable alternative analysis and provide IRS executives with a false assessment of the actual progress and costs of their information technology projects.

Recommendations

To make the business cases more reliable and useful, we recommended the Chief Information Officer provide increased oversight to ensure Project Managers include complete and realistic cost estimates for their projects, coordinate with the Department of Treasury Capital Planning and Investment Control (CPIC) office to follow OMB guidance requiring allocation of all management and labor costs to specific projects, provide additional oversight of Project Managers to ensure sufficient care is taken in developing and reporting progress data, and ensure reviews are conducted to determine whether contractors’ cost and schedule procedures comply with industry standards. In addition, the Chief Information Officer should coordinate with the Department of the Treasury CPIC office to program the ProSight system so the total life-cycle costs are reported consistently and to implement access controls to ensure only authorized users have access to the system. The Chief Information Officer should ensure the Director, CPIC, reviews the IRS Federal Information Security Management Act Master Inventory of major systems annually to ensure business cases are prepared for required projects.

Response

IRS management agreed with all seven of our recommendations. To increase oversight and ensure that realistic costs are reported, the IRS CPIC office will prepare Exhibit 300 policies and provide guidance and training to project managers. The IRS CPIC office will collaborate with the Department of the Treasury to develop policies for allocating management and labor costs to specific projects so they reflect the true cost of the investment, prepare Earned Value Management reporting policies, and provide guidance and training to Project Managers and staff to ensure Earned Value Management data are adequately prepared and disclosed. The IRS also created the Application Development Program Management Office to provide increased oversight over project cost and schedule information.

In addition, the IRS will work with the Department of the Treasury to develop a plan requiring contractors to perform self-assessments and furnish a certificate of compliance or a strategy to achieve compliance so that Earned Value Management data meets industry standards. To ensure total life-cycle costs are reported consistently in business cases, the IRS CPIC office will work with the Department of the Treasury and request modifications to the ProSight system and request that only users authorized by Project Managers have “write” access to project data in the ProSight system. The IRS CPIC office performed an ad hoc review of the IRS’ Federal Information Security Management Act inventory to identify any major systems that are required to have business cases prepared and will develop a process in its CPIC guide to address this recommendation.

The Chief Information Officer did not concur with our outcome measures reported in Appendix IV of the report and disagreed that the IRS was putting millions of taxpayer dollars at risk. The inconsistencies reported were primarily due to the omission of historical data (sunk costs) in the ProSight system and the IRS is meeting with the Department of the Treasury to correct the programming issues. Management’s complete response to the draft report is included as Appendix VI.

Office of Audit Comment

Our position is to sustain the outcome measures. The outcome measures claimed in the report are based on inaccurate, incomplete, and inconsistent cost information found in the sampled business cases. Unreliable cost information reduces the usefulness of the business cases and inhibits management’s ability to make fully informed project finance decisions, potentially putting millions of taxpayer dollars at risk. We agree that correcting the ProSight system should improve the consistency of cost information reported in the business cases.

Copies of this report are also being sent to the IRS managers affected by the report recommendations. Please contact me at (202) 622-6510 if you have questions or Margaret E. Begg, Assistant Inspector General for Audit (Information Systems Programs), at (202) 622-8510.

Table of Contents

Background

Results of Review

Project Costs Remain Inaccurate

Recommendations 1 and 2:

Progress on Development Projects Continues to Be Measured Inaccurately

Recommendations 3 and 4:

The Department of the Treasury’s Programming of the ProSight System Continues to Contribute to Inaccuracies in Business Cases

Recommendation 5:

Recommendation 6:

Major Applications Were Omitted From the Budget Submission

Recommendation 7:

Appendices

Appendix I – Detailed Objective, Scope, and Methodology

Appendix II – Major Contributors to This Report

Appendix III – Report Distribution List

Appendix IV – Outcome Measures

Appendix V – Glossary of Terms

Appendix VI – Management’s Response to the Draft Report

Abbreviations

CADE	Customer Account Data Engine
CPIC	Capital Planning and Investment Control
EMS	Electronic Management System
EVM	Earned Value Management
FISMA	Federal Information Security Management Act
FMS	Financial Management System
ICCE	Integrated Customer Communications Environment
ICS	Integrated Collection System
IRS	Internal Revenue Service
MeF	Modernized e-File
OMB	Office of Management and Budget
SCRIPS	Service Center Recognition/Imaging Processing System

Background

The Federal Government invests over $60 billion in information technology annually. Motivated to improve information technology budget stewardship, Congress enacted legislation directing agencies to use better business practices to manage and report major information technology investments. The Clinger-Cohen Act of 1996[5] requires Federal Government agencies to improve the way they acquire and manage their information technology investments. Agencies are required to put their technology investment decisions in a true business context and analyze investments for their return on investment. The Office of Management and Budget (OMB) published Circular A-11, Preparation, Submission and Execution of the Budget, to assist Federal Government agencies in complying with the Clinger-Cohen Act. This guidance includes two key sections applicable to information technology capital planning. Section 300, Planning, Budgeting, Acquisition, and Management of Capital Assets, provides guidance on the preparation of business cases for information technology systems. Section 53, Information Technology and E-Government, provides guidance on the preparation of an agency’s entire Information Technology Investment Portfolio.

The Internal Revenue Service (IRS) spends approximately $2 billion annually on information technology investments. The IRS uses business cases as the primary tool for capital planning and investment control. These business cases provide a standard format for reporting key details about the investment. The information contained in the business case assists IRS management in evaluating an information technology investment’s costs, benefits, and risks. The business cases also provide support for the IRS’ strategic goals and objectives when compared to other competing information technology requirements. The IRS Capital Planning and Investment Control (CPIC) office[6] is responsible for establishing the processes that support business case preparation and review and maintaining information technology investment process documentation.

In April 2005, we issued a report[7] that addressed whether the IRS planned, managed, and controlled its information technology investments in compliance with OMB and Clinger-Cohen Act requirements. We reported the IRS procedures for preparing information technology business cases had improved, but managers were not complying with these requirements. Specifically, project costs had not been reported accurately in the business cases, relevant cost and benefit information had been omitted, progress on development projects had been measured inaccurately, sufficient information was not provided in the alternatives analysis section, and business cases for operational projects did not demonstrate the results of an E-Government review.[8] Due to the number and significance of the conditions reported, we concluded that the business cases could not be relied on to manage and fund the IRS’ information technology projects.

We conducted this follow-up review to determine whether the IRS had taken corrective actions to address the weaknesses in the report and whether those actions were effective. For this review, we selected Budget Year[9] 2007 business cases for the following six major information technology investment projects. See Appendix V for detailed descriptions of these investment projects:

Customer Account Data Engine (CADE).
Electronic Management System (EMS).
Integrated Customer Communications Environment (ICCE).
Integrated Collection System (ICS).
Modernized e-File (MeF).
Service Center Recognition/Imaging Processing System (SCRIPS).

The CADE and MeF projects are referred to as development projects, which are information technology systems currently being designed and built. The EMS, ICCE, ICS, and SCRIPS projects are referred to as steady state projects, which are existing information technology systems that generally require only maintenance and operational costs.

This review was performed at the IRS CPIC office in New Carrollton, Maryland, during the period November 2005 through September 2006. The audit was conducted in accordance with Government Auditing Standards. Detailed information on our audit objective, scope and methodology is presented in Appendix I. Major contributors to the report are listed in Appendix II.

Results of Review

The IRS took several corrective actions to address the weaknesses cited in our previous report; however, in this review, we identified many of the same significant weaknesses. Project costs remain inaccurate, progress on development projects continues to be measured inaccurately, the software used to prepare business cases contributes to the inaccuracies, and major applications were missing from the budget submission. As a result, we believe senior IRS executives and Department of the Treasury and OMB officials still cannot rely on the data in these business cases to manage and fund the information technology projects. Inaccurate information in business cases can distort viable analysis and provide IRS executives with a false assessment of the actual costs and progress of projects.

Project Costs Remain Inaccurate

Project costs remain inaccurate and could be improved with additional corrective actions and managerial oversight. Specifically, project cost forecasts were not substantiated, indirect management and labor overhead costs were not allocated to specific projects, and security costs were calculated inaccurately. Incomplete and inaccurate cost data inhibit management’s ability to make fully informed project finance decisions, potentially putting millions of taxpayer dollars at risk.

Project cost forecasts were unsubstantiated

Previously, we identified multiple errors in forecasting project costs in IRS business cases. To correct this deficiency, the IRS designated Project Managers as the individuals accountable for all data contained in their business cases and provided training and guidance as appropriate. While this was an important step, we determined during our review that the business cases remained inaccurate and project costs were not substantiated.

For the six projects we reviewed, the Project Managers relied on budget data from the IRS Financial Management System (FMS)[10] office as the source for forecasting project costs. The FMS office derived the costs from the previous year's budget, making adjustments for inflation and other budgetary changes. ****3(d)****

For example, ****3(d)**** relied on the budget figures received from the FMS office and reported $12.87 million for direct labor costs for Budget Year 2007. We determined the actual direct labor cost was approximately $26.59 million, an understatement of $13.72 million, because the costs of Product Assurance organization employees dedicated to testing the ****3(d)**** were not included.

****3(d)**** business case overstated project infrastructure costs by $755,000. The IRS was in the process of eliminating this infrastructure; however, the figure provided by the FMS office still included this cost for Budget Year 2007.

Project cost forecasts for steady state projects also were inaccurate. We estimated the data provided by the FMS office for Budget Year 2007 understated project costs an average of 25 percent for the 4 steady state projects we reviewed, assuming the same level of spending as in prior years. ****3(d)****

Management and labor costs are still not being allocated to the information technology investments

In our last review, we reported the IRS CPIC office did not allocate management and overhead labor costs totaling $79.4 million to development projects. As a result, indirect management and overhead labor costs for each information technology development project were understated. To correct this deficiency, the IRS designated Project Managers as the individuals accountable for all data contained in their business cases and provided training and guidance. However, the IRS did not require the Project Managers to allocate management and overhead labor costs to correct the Budget Year 2007 business cases.

OMB Circular A-11 states Federal Government staffing costs shall include Government indirect labor costs in support of an information technology investment. Indirect labor costs include the information technology investment’s management staff and any other Federal Government effort that contributes to the success of the information technology investment. Persons working on more than 1 information technology investment, whose contributions exceed over 50 percent of their time, should have their time allocated to each information technology investment.

For Budget Year 2007, the IRS reported $51 million for Business Systems Modernization Management and $30 million for Information System Support from business units. These costs are for oversight of the Business Systems Modernization program and staffing support from IRS business units. To comply with OMB guidance, the IRS should have allocated these costs to the information technology investments in Budget Year 2007. However, these labor costs were not allocated to any of the information technology investments we reviewed.

This is a repeat finding and we continue to disagree with this practice. The IRS justifies not allocating its Business Systems Modernization Management and Information System Support labor costs because the OMB has reviewed the IRS’ budget submission and has not raised this issue. Therefore, the IRS believes the OMB has tacitly approved reporting these costs as separate line items and accepted the reporting of them in this format. This practice violates current OMB guidance and understates the IRS’ true cost for each information technology investment. Information technology project costs will continue to be understated when all management and labor costs are not included in the business cases. Over a period of several years, the understatement for each project could be significant.

Security costs continue to be reported inaccurately

We previously determined that three of the four projects reviewed reported security costs inaccurately. The IRS designated Project Managers as the individuals accountable for all data contained in their business cases. This corrective action was not effective as security costs continue to be reported inaccurately in the business cases.

OMB Circular A-11 requires Project Managers to report the cost of providing information technology security for their projects. The calculation of security costs includes two components:

1) The amount representing corporate or “network” security (e.g., a shared network-wide intrusion detection system).

2) The projected security costs specific to the project (e.g., the cost of certification and accreditation or security training).

IRS network security costs were not allocated to any of the six business cases. The IRS claimed network security costs as a separate line item in the budget submission to OMB instead of allocating them to the projects. Although total network security costs may have been accurate, each project’s costs were understated. In addition, $104 million in security costs attributed to the Mission Assurance and Security Services organization were not allocated to any projects. The Mission Assurance and Security Services organization provides many information technology security services that support projects. For example, it assists IRS organizations in certifying their systems, testing system security, and identifying and correcting system security weaknesses.

Furthermore, four of the six projects (EMS, ICCE, ICS, and SCRIPS) did not include any security costs. The IRS CPIC office instructed Project Managers who had not developed their own project-specific security costs to apply 5.3 percent to their total estimated project costs for their security cost estimates for inclusion in their respective business cases. This percentage represents a projection for typical project-specific security costs. However, we found no evidence the security costs were accounted for within the total project cost figures. The understatement of security costs for each project adds to the unreliability of the business cases. We attribute these errors to confusion caused by the IRS CPIC office establishing three different sets of instructions for calculating security costs between July 2005 and August 2005. In addition, ****3(d)****

Recommendations

To ensure project costs are reported accurately, the Chief Information Officer should:

Recommendation 1: Provide increased oversight and training to ensure Project Managers include complete and realistic cost estimates for their projects and more closely review Project Managers’ business cases to ensure realistic cost estimates are provided and costs are substantiated. Particular attention should be paid to the allocation of management, labor, and security costs.

Management’s Response: The IRS agreed with our recommendation. The IRS CPIC office will prepare Exhibit 300 policies and provide guidance and training to project managers and relevant staff. In addition, the IRS created the Application Development Program Management Office to provide additional oversight and review of project information.

Recommendation 2: Coordinate with the Department of the Treasury CPIC office and follow OMB guidance requiring allocation of management and labor costs to specific projects.

Management’s Response: The IRS agreed with our recommendation. The IRS CPIC office will collaborate with the Department of the Treasury to develop policies for allocating management and labor costs to specific projects so they reflect the true cost of the investment. The IRS CPIC office will provide guidance and training to reporting projects when the policy is established.

Progress on Development Projects Continues to Be Measured Inaccurately

In our last review, we identified multiple problems in measuring the progress of the two development project business cases reviewed. In addition, the IRS did not adequately review the business cases and data provided by the contractors. To correct these deficiencies, the Director, CPIC, created an additional table for progress calculations in the Budget Year 2007 business cases. However, the progress on development projects continues to be reported inaccurately. As a result, the usefulness and reliability of the business cases were diminished. ****3(d)****

Earned value continues to be reported inaccurately in business cases

OMB Circular A-11 requires Project Managers of development projects to report Earned Value Management (EVM) data in their business cases. EVM can be defined as a technique to estimate how a project is doing in terms of its budget and schedule. It compares the actual work that has been completed to the estimates made at the beginning of the project and should provide an early warning system for determining whether an investment project is performing on schedule and within budget. EVM data contain information that management should use along with other project indicators to manage a project.

The CADE and MeF projects reported favorable EVM data in their business cases. For example, the CADE project reported that, as of June 2005, the project had spent the precise amount estimated and had completed almost the exact amount of work planned from the time the project was initiated in January 1999. The MeF project also reported near perfect performance in these areas. However, we believe the EVM data in the CADE and MeF business cases did not provide an accurate assessment of the projects’ budget and schedule performance for two reasons.

First, actual costs used in EVM calculations were understated. Actual costs incurred prior to 2003 and selected other costs, such as hardware, software, and Federal Government labor, were not included in EVM calculations. Examples of the understatements include:

· The CADE project omitted $231 million (78 percent) of the total costs of $296 million incurred as of June 2005.

· The MeF project, which began October 2000, omitted $83 million (54 percent) of the total costs of $154 million incurred as of June 2005.

Second, changes to the baselines of estimated costs and completion dates undermine the usefulness of the EVM data. The CADE and MeF projects have frequently obtained approval from the OMB to change their estimated costs and work completion dates. Actual costs are then measured against the new estimates.

The originally approved OMB baseline for the CADE project estimated it would be completed in June 2009 for a cost of $327 million. Currently, the OMB has approved a revised baseline estimate that the CADE project will be completed by December 2012 for a cost of $1.802 billion. This is a nearly $1.5 billion increase and a 3-year overrun; however, the current EVM data show the project is on time and within budget. The CADE project’s Budget Year 2007 business case proposes an additional revised baseline to increase total project cost to $1.829 billion, a $1.502 billion increase over the original OMB baseline.

The originally approved OMB baseline for the MeF project estimated it would be completed in September 2019 for a cost of $509 million. Currently, the OMB has approved a revised baseline estimate that the MeF project will be completed in September 2020 for a cost of $638 million, an increase of $129 million and a 1-year overrun. The MeF project’s Budget Year 2007 business case proposes an additional revised rebaseline to increase total project cost to $673 million, a $164 million increase over the original OMB baseline.

In general, the IRS will request a revised baseline every time the project budget changes. Consequently, very little variance is ever reported between planned and actual costs. Business cases do not disclose the percentage of costs for which earned value is not calculated and the number of times a project has been rebaselined. These disclosures would help explain how the investment is able to achieve its high performance standards. Without this disclosure, the EVM data provided to IRS executives and the OMB will be of little value for tracking progress and costs of information technology projects.

****3(d)****

We had previously found that Project Managers were not verifying whether contractors’ EVM systems were compliant with industry standards. For development projects, the OMB requires business cases to demonstrate the EVM system used by a contractor meets industry standards.[11] After our last review, the IRS designated Project Managers as the individuals accountable for all data contained in their business cases. This corrective action was not effective. ****3(d)****

For example, the IRS reported in the CADE business case that it had conducted reviews of the CADE project contractor’s project management system in 2003 and 2004 and determined the contractor had made progress toward institutionalizing EVM. However, we determined the IRS did not conduct a review in 2004, and there is no evidence to support that the contractor had made progress. The CADE project had planned to conduct a compliance review of the contractor’s project management system in 2004, but cancelled this work due to budget constraints. ****3(d)****

****3(d)**** We determined the IRS has not conducted annual reviews of the MeF project contractors’ project management systems and did not disclose this fact in its business case for the MeF project.

The EVM data for the CADE and MeF projects provide IRS executives and OMB officials little value for making management decisions to track the historical costs and progress of these projects. Until all costs are included and the project management systems used by contractors comply with industry standards, the IRS and the OMB cannot rely on the business case information.

Recommendations

To ensure EVM data are accurately computed and disclosed in IRS business cases, the Chief Information Officer should:

Recommendation 3: Provide additional oversight of Project Managers to ensure sufficient care is taken in developing and reporting EVM data. Each Project Manager should disclose all significant facts related to the EVM data, including the significant percentage of costs omitted from earned value calculations and the number of times the project has been rebaselined. These disclosures would help explain how the investment is able to achieve its high performance standards.

Management’s Response: The IRS agreed with our recommendation. The IRS CPIC office will prepare EVM reporting policies to encompass all aspects of this recommendation, and will provide guidance and training to project managers and relevant staff. In addition, the Application Development Program Management Office will provide increased oversight over EVM data.

Recommendation 4: Ensure reviews are conducted to determine whether contractors’ EVM systems comply with industry standards. Noncompliance and failure to conduct the reviews should be disclosed.

Management’s Response: The IRS agreed with our recommendation. The IRS will work with the Department of the Treasury to develop a plan requiring contractors to perform self-assessments and to furnish a Cognizant Federal Agency certificate of compliance or a strategy to achieve compliance. To ensure contractor compliance, the plan will include the approach for criteria, roles, and review cycle along with the approval, review, and certification process.

The Department of the Treasury’s Programming of the ProSight System Continues to Contribute to Inaccuracies in Business Cases

We had previously identified a problem in the Department of the Treasury’s ProSight system[12] that contributed to inaccuracies in the EVM section of IRS business cases. While the IRS had corrected the programming error in the ProSight system cited in our previous report, additional systemic problems contributed to the IRS’ inconsistent reporting of the total investment costs for all six business cases we reviewed. Total costs were reported inconsistently throughout the business cases, which affected the reliability of the business cases. In addition, a lack of access control to the ProSight system allowed multiple users to make changes to business cases. As a result, accountability for the accuracy of data could not be established.

Total costs were reported inconsistently throughout the business cases

IRS guidance for preparing business cases states there should be no surprises or inconsistencies across the sections of the business case. Inconsistencies create confusion and hinder the readability throughout the business case.

The costs of all six business cases we reviewed were reported inconsistently in different sections of the same business case, which made it difficult to determine the accurate cost of the investment. For example, the Summary of Spending table[13] in the first part of the business case reports the total cost of the CADE investment as $1.829 billion. The Alternatives Analysis table[14] reports the total cost as $1.569 billion. The difference of $260 million represents the costs for Budget Year 2004 and prior years. We found this inconsistency in each business case we reviewed, as illustrated in Table 1.

Table 1: Total Cost of Investment As Reported in
Different Sections of the Business Case

Project	Summary of Spending Table (in millions)	Alternatives Analysis Table (in millions)	Difference (in millions)
CADE	$1.829	$1.569	$260
EMS	$175	$44	$131
ICCE	$460	$198	$262
ICS	$372	$51	$321
MeF	$673	$555	$118
SCRIPS	$142	$129	$13

Source: IRS Budget Year 2007 business cases.

The discrepancies between the Summary of Spending and Alternatives Analysis cost tables are directly attributable to how the Department of the Treasury CPIC office programmed the ProSight system used to prepare the business cases. The ProSight system excluded the cost data for Budget Year 2004 and prior years in the Alternatives Analysis tables. Therefore, the Summary of Spending tables contained the projects’ full life-cycle costs, while the Alternatives Analysis tables did not. A project’s full life-cycle costs should be accounted for and consistent in both tables.

In addition, the Summary of Spending tables and the Actual Performance tables[15] inconsistently reported the actual costs expended for the ICS and EMS projects. The Summary of Spending table for the ICS business case reported the actual cost for 2004 and prior years as $321 million, while the Actual Performance table reported this cost as $34 million, a $287 million difference. Also, the Summary of Spending table for the EMS business case reported the actual cost for 2004 and prior years as $116 million, while the Actual Performance table reported this cost as $125 million, a $9 million difference. ****3(d)**** making it difficult to determine which cost figure was reliable.

A lack of control allowed multiple users to have access to the ProSight system and make changes to business cases

All Department of the Treasury information systems are required to implement system access controls that protect the information from unauthorized modification, loss, or disclosure. Access to information or system resources must be limited to only authorized users, programs, processes, or other systems.

For each of the 6 business cases in our sample, we found that approximately 60 individuals were granted the ability to access and modify the business case on the ProSight system without the approval or knowledge of the Project Manager. Individuals with access included officials in the IRS’ CPIC office, FMS office, IRS Contractors, and the Department of the Treasury’s CPIC office. This lack of control makes it difficult for the Project Manager to be responsible for changes made to the business case. It also increases risk of errors and decreases reliability of information. The IRS’ CPIC office did not effectively coordinate with the Department of the Treasury’s CPIC office to ensure only those with a need to access the ProSight system were given the permission to do so.

Recommendations

To correct the ProSight system and improve business case controls, the Chief Information Officer should:

Recommendation 5: Coordinate with the Department of the Treasury CPIC office to program the ProSight system so total life-cycle costs are reported consistently in the Summary of Spending and Alternatives Analysis tables and actual project costs are reported consistently in the Summary of Spending and Actual Performance tables. Project Managers should ensure the data are reported consistently.

Management’s Response: The IRS agreed with our recommendation. The IRS CPIC office will present this recommendation to the Department of the Treasury CPIC office. It will request modifications to the ProSight system so that total reported lifecycle and actual project costs are consistent in the Summary of Spending and Alternative Analysis tables.

Office of Audit Comment: Our position is to sustain the outcome measures. The outcome measures claimed in the report are based on inaccurate, incomplete, and inconsistent cost information found in the sampled business cases. Unreliable cost information reduces the usefulness of the business cases and inhibits management’s ability to make fully informed project finance decisions, potentially putting millions of taxpayer dollars at risk. We agree that correcting the ProSight system should improve the consistency of cost information reported in the business cases.

Recommendation 6: Coordinate with the Department of the Treasury CPIC office to implement system access controls to ensure only users authorized by Project Managers have access to project data in the ProSight system.

Management’s Response: The IRS agreed with our recommendation. The IRS CPIC office will present this recommendation to the Department of the Treasury CPIC office. It will request modifications to the ProSight system so only users authorized by Project Managers have “write” access to project data in the ProSight system.

Major Applications Were Omitted From the Budget Submission

The OMB requires that all major applications be included in an agency’s budget submissions. The IRS CPIC office has issued guidance that requires business cases to be prepared for any major information technology system with budgetary outlays of over $5 million per year. We compared the list of major systems the IRS uses for the Federal Information Security Management Act (FISMA)[16] to the list of systems reported on the IRS OMB budget submission for Budget Year 2007 to determine whether all systems were reported and business cases had been prepared for the required systems. We identified two major systems (the Electronic Levy System and the Offshore Credit Card Project Application) that were listed on the IRS FISMA Master Inventory list but not included on the IRS OMB budget submission. In addition, a business case had not been prepared for the Automated Collection System although its projected annual costs exceeded $5 million. Failure to provide business cases for all required systems detracts from the OMB’s ability to allocate information technology funding and from the IRS’ ability to adequately monitor the costs and benefits of its major systems. The IRS CPIC office had not ensured business cases were prepared for all major systems.

The Department of the Treasury recently issued guidance that requires bureaus to ensure all FISMA major systems are covered by an investment that is reported in the CPIC inventory submitted to the OMB for the Budget Year 2008 reporting cycle, starting in September 2006. The IRS has started this process by completing a comparison of FISMA and CPIC applications.

Recommendation

To ensure information technology systems are reported consistently:

Recommendation 7: The Chief Information Officer should ensure the Director, CPIC, reviews the IRS FISMA Master Inventory of major systems annually to ensure business cases are prepared for required projects.

Management’s Response: The IRS agreed with our recommendation. The IRS CPIC office performed an ad hoc review of the IRS’ FISMA inventory for Fiscal Year 2007 to identify major systems. The office will develop a process for inclusion in the CPIC guide and will publish the process in the CPIC “Newsflash.”

Appendix I

Detailed Objective, Scope, and Methodology

The overall objective of this review was to determine whether the IRS took effective corrective actions to address the recommendations in our previous audit report.[17] Also, we evaluated whether the IRS is managing its information technology investments in compliance with OMB and Clinger-Cohen Act of 1996[18] requirements. To accomplish this objective, we:

I. Determined whether project costs were accurately reported in the business cases.

A. For the six projects selected for this audit,[19] reviewed the Project Managers’ performance plans to determine whether updates were made to the performance plans to hold Project Managers accountable for ensuring all sections of the business case are consistent, accurate, complete, and supported by documentation. From a total of 28 investments for which business cases were prepared in Budget Year 2007, we judgmentally selected 6 information technology investments with the highest costs according to the September 2005 IRS Budget Year 2007 Exhibit 53. These included two modernization and four operational investments. We used a judgmental sample because we did not plan to project our audit results.

B. Determined whether adequate training and guidance were provided to Project Managers to ensure the accuracy of business cases.

C. Reviewed the IRS Budget Year 2007 Exhibit 53 to determine whether management and overhead labor costs were separately reported.

D. Determined whether security costs were accurately reported in the six business cases reviewed.

E. Determined whether IRS direct labor costs were included in the total project costs in the Summary of Spending table, which is in the first part of the Exhibit 300.

F. Determined whether project-specific infrastructure (hardware and software) costs were included in each project’s business case.

G. Reviewed supporting documentation for key costs reported in the six business cases.

H. Determined whether work performed by contractors was reviewed by the IRS Project Managers before data and calculations were input to the business cases.

I. Determined whether the Enterprise Life Cycle directive signed on August 24, 2004, required a comprehensive evaluation and selection report, including costs, on all commercial off-the-shelf products that are considered for new IRS systems.

II. Determined whether progress on development projects was measured accurately.

A. Determined whether the IRS CPIC office provided training and guidance to Project Managers on how to complete the EVM section of business cases.

B. Reviewed supporting documentation for the key EVM calculations presented in each business case to determine whether the EVM calculations were accurate, up-to-date, and consistently reported throughout the business case. We also determined whether IRS Project Managers verified whether their contractors’ EVM systems were compliant with industry standards.

C. Determined whether the tables in the business cases were correct and consistent where appropriate.

D. Determined whether the IRS coordinated with the necessary parties to correct programming in the ProSight system that caused inaccuracies in the business cases. The ProSight system is used to report IRS and Treasury Department business cases.

III. Determined whether the four operational projects’ business cases demonstrated the results of an E-Government review.

IV. Determined whether the IRS made progress in preparing a business case for each major investment.

Appendix II

Major Contributors to This Report

Margaret E. Begg, Assistant Inspector General for Audit (Information Systems Programs)

Stephen Mullins, Director

Kent Sagara, Acting Director

Thomas Polsfoot, Audit Manager

Jody Kitazono, Lead Auditor

W. Allen Gray, Senior Auditor

Bret Hunter, Senior Auditor

Thomas Nacinovich, Senior Auditor

Appendix III

Report Distribution List

Commissioner C

Office of the Commissioner – Attn: Chief of Staff C

Deputy Commissioner for Operations Support OS

Deputy Chief Information Officer OS:CIO

Chief, Mission Assurance and Security Services OS:MA

Associate Chief Information Officer, Applications Development OS:CIO:B

Director, Capital Planning and Investment Control OS:CIO:M:CP

Director, Financial Management Services OS:CIO:M:FM

Chief Counsel CC

National Taxpayer Advocate TA

Director, Office of Legislative Affairs CL:LA

Director, Office of Program Evaluation and Risk Analysis RAS:O

Office of Internal Control OS:CFO:CPIC:IC

Audit Liaison: Chief Information Officer OS:CIO

Appendix IV

Outcome Measures

This appendix presents detailed information on the measurable impact that our recommended corrective actions will have on tax administration. These benefits will be incorporated into our Semiannual Report to Congress.

Type and Value of Outcome Measure:

· Reliability of Information – Potential; $199.48 million (see page 3). Incomplete and inaccurate cost data inhibit management’s ability to make fully informed project finance decisions, potentially putting millions of taxpayer dollars at risk. Reliability of Information outcome measures are reported on an absolute basis; i.e., both overestimated and underestimated amounts are reported as positive amounts.

Methodology Used to Measure the Reported Benefit:

Understatement of ****3(d)**** labor $13.72 million

****3(d)**** overstated costs $755,000

Management and labor costs (not allocated to projects) $81.00 million

Security costs (not allocated to projects) $104.00 million

Total $199.48 million

Type and Value of Outcome Measure:

· Reliability of Information – Potential; $1.92 billion (see page 6). As a result, the usefulness and reliability of the business cases were diminished.

Methodology Used to Measure the Reported Benefit:

Inaccuracies:

CADE project (omitted from progress calculations) $231 million

MeF project (omitted from progress calculations) $83 million

Total $314 million

Baseline changes:

CADE project (increase from original estimate) $1.475 billion

MeF project (increase from original estimate) $129 million

Total $1.604 billion

Type and Value of Outcome Measure:

· Reliability of Information – Potential; $1.40 billion (see page 9). Total costs were reported inconsistently throughout the business cases, which affected the reliability of the business cases.

Methodology Used to Measure the Reported Benefit:

Spending and Alternatives Tables: Absolute of difference $1.105 billion

Spending and Actual Performance Tables:

a. ICS project difference $287 million

b. EMS project difference $9 million

Total $1.40 billion

Grand Total ($199.48 million plus $1.92 billion plus $1.40 billion) $3.52 billion

Appendix V

Glossary of Terms

Automated Collection System

The Automated Collection System is a computerized inventory system which maintains balance due and nonfiler cases requiring telephone contact for resolution.

Budget Year

The Budget Year refers to a future fiscal year (October 1 – September 30) which is the subject of the budget planning process.

Customer Account Data Engine (CADE)

The CADE is the foundation for managing taxpayer accounts in the IRS’ Business Systems Modernization effort and will eventually replace the existing Master File databases, which are the IRS’ central and official repository of taxpayer information.

Electronic Levy System

The Electronic Levy System enables IRS tax examiners and clerks to review levies prior to printing and requires only levies with flagged errors to be reviewed. The system eliminates time and paper costs associated with a manual review and the retyping of erroneous levies

Electronic Management System (EMS)

The EMS is a processing system that receives, validates, stores, and forwards electronic tax return information to electronic filing systems. This System also provides acknowledgment of electronic tax returns received and is the principle electronic gateway for electronic commerce to and from the IRS.

Enterprise Life Cycle

The enterprise life cycle establishes a set of repeatable processes and a system of reviews, checkpoints, and milestones that reduce the risks of system development and ensure alignment with the overall business strategy.

Integrated Customer Communications Environment (ICCE)

The ICCE provides customer service applications through toll-free telephone service and the Internet. The toll-free telephone service provides automated self-service applications that allow taxpayers to help themselves, as well as avenues to route taxpayers to live customer service representatives. The Internet component of the ICCE allows taxpayers to check refund status.

Integrated Collection System (ICS)

The ICS is an information management system designed to improve revenue collections by providing revenue officers access to the most current taxpayer information, while in the field, using laptop computers for quicker case resolution and improved customer service.

IRS Exhibit 300 Business Case Guide

The Capital Planning and Investment Control Office developed the Exhibit 300 Business Case Guide to assist project team members in preparing “budget quality” Exhibit 300s for submission to higher authority. The Guide addresses content requirements of the business case and provides insight to program and project managers regarding the quality of information required to obtain OMB budget approval.

Modernized e-File (MeF)

The MeF system modernizes the IRS’ existing electronic filing system, providing an Internet-based electronic filing application that taxpayers can use to file IRS forms.

Offshore Credit Card Application

This application is designed to analyze, display, and report information received from summons issued to financial institutions, credit card companies, and third-party processors of financial information which may identify individuals who are illegally sheltering money offshore.

Service Center Recognition/Imaging Processing System (SCRIPS)

The SCRIPS is a data capture, management, and storage system that uses high-speed scanning and digital imaging technology to process tax documents.

Appendix VI

Management’s Response to the Draft Report

The response was removed due to its size. To see the response, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

[1] Business Cases for Information Technology Projects Need Improvement (Reference Number 2005-20-074, dated April 2005).

[2] Pub. L. No. 104-106, 110 Stat. 642 (codified in scattered sections of 5 U.S.C., 5 U.S.C. app., 10 U.S.C., 15 U.S.C., 16 U.S.C., 18 U.S.C., 22 U.S.C., 28 U.S.C., 29 U.S.C., 31 U.S.C., 38 U.S.C., 40 U.S.C., 41 U.S.C., 42 U.S.C., 44 U.S.C., 49 U.S.C., 50 U.S.C.).

[3] The E-Government Act of 2002 (Pub. L. No. 107-347) requires a comprehensive review and analysis to be performed on existing computer systems and information technology investments to identify strategies for smarter and more cost-effective methods of delivering performance.

[5] Pub. L. No. 104-106, 110 Stat. 642 (codified in scattered sections of 5 U.S.C., 5 U.S.C. app., 10 U.S.C., 15 U.S.C., 16 U.S.C., 18 U.S.C., 22 U.S.C., 28 U.S.C., 29 U.S.C., 31 U.S.C., 38 U.S.C., 40 U.S.C., 41 U.S.C., 42 U.S.C., 44 U.S.C., 49 U.S.C., 50 U.S.C.).

[6] The CPIC office reports to the Associate Chief Information Officer, Management, Capital Planning and Investment Control.

[7] Business Cases for Information Technology Projects Need Improvement (Reference Number 2005-20-074, dated April 2005).

[8] The E-Government Act of 2002 (Pub. L. No. 107-347) requires a comprehensive review and analysis to be performed on existing computer systems and information technology investments to identify strategies for smarter and more cost-effective methods of delivering performance.

[10] The FMS office is the budget office that reports to the Associate Chief Information Officer, Management, in the Chief Information Officer organization.

[11] The OMB requires EVM systems to comply with the American National Standards Institute/Electronic Industries Alliance Standard 748-1998, Earned Value Management Systems, approved May 19, 1998, and reaffirmed August 28, 2002. This Standard is a list of 32 guidelines used for determining whether contractors’ EVM systems are acceptable to the Federal Government for large, risky projects. The Federal Government uses these guidelines to validate a contractor’s system.

[12] The ProSight system is the Department of the Treasury software used by the IRS to prepare business cases.

[13] The Summary of Spending table provides an overview of the costs for planning, acquisition, maintenance, and labor for the previous, current, and budget fiscal years.

[14] The Alternatives Analysis table provides a summary of the comparison of viable alternative solutions that includes a general rationale and analysis of the monetary benefits for each alternative presented.

[15] The Actual Performance table compares OMB-approved cost and schedule goals with actual results.

[16] Pub. L. No. 107-347, Title III, 116 Stat. 2946 (2002).

[17] Business Cases for Information Technology Projects Need Improvement (Reference Number 2005-20-074, dated April 2005).

[18] Pub. L. No. 104-106, 110 Stat. 642 (codified in scattered sections of 5 U.S.C., 5 U.S.C. app., 10 U.S.C., 15 U.S.C., 16 U.S.C., 18 U.S.C., 22 U.S.C., 28 U.S.C., 29 U.S.C., 31 U.S.C., 38 U.S.C., 40 U.S.C., 41 U.S.C., 42 U.S.C., 44 U.S.C., 49 U.S.C., 50 U.S.C.).

[19] The CADE, EMS, ICS, ICCE, MeF, and SCRIPS projects. See Appendix V for a glossary of terms.