TREASURY INSPECTOR GENERAL FOR TAX ADMINISTRATION

 

 

Management Oversight Improved, but Expected Benefits and Capabilities for the Tax Exempt Determination System Release 2 Were Not Delivered

 

 

 

December 11, 2007

 

Reference Number:  2008-10-025

 

 

This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.

 

Phone Number   |  202-622-6500

Email Address   |  inquiries@tigta.treas.gov

Web Site           |  http://www.tigta.gov

 

December 11, 2007

 

 

MEMORANDUM FOR COMMISSIONER, TAX EXEMPT AND GOVERNMENT ENTITIES DIVISION

 

FROM:                            Michael R. Phillips /s/ Michael R. Phillips

Deputy Inspector General for Audit

 

SUBJECT:                    Final Audit Report – Management Oversight Improved, but Expected Benefits and Capabilities for the Tax Exempt Determination System Release 2 Were Not Delivered (Audit # 200610047) 

 

This report presents the results of our review of the Tax Exempt Determination System (hereafter referred to as the TEDS or System)[1] Release 2.  The overall objectives of this review were to determine whether the Tax Exempt and Government Entities (TE/GE) Division developed TEDS Release 2 using sound system development practices and whether it managed the Release 2 investment in compliance with Office of Management and Budget and Clinger-Cohen Act of 1996[2] requirements.  This audit was conducted as part of the Treasury Inspector General for Tax Administration Office of Audit Fiscal Year 2007 Annual Audit Plan.

Impact on the Taxpayer

In September 2003, TE/GE Division management began developing TEDS Release 2, which was to include the upfront imaging of Employee Plans function and Exempt Organizations function determination applications, enhanced reporting, and automated case assignment.  The System is currently scheduled for implementation in May 2008.  However, when the System is completed, its cost may far outweigh its benefits.  The high cost of the TEDS compared to the benefits it will deliver brings into question whether sound investment decisions were made during development of the System and whether this was the best use of Federal Government funds.

Synopsis

Overall, the TE/GE Division Investment Executive Steering Committee improved its oversight of the development of TEDS Release 2 from that provided during Release 1 by using certain project management techniques.  In addition, TE/GE Division management began tracking some project costs as we had recommended.[3]  However, management did not fully implement our prior recommendations to evaluate investment decisions and monitor whether business benefits would be realized.  We identified the following:

·           Schedule delays and deletion of some system capabilities resulted in delivering only $33.5 million (less than one-half) of the $73.1 million in expected benefits included in the August 2004 Business Case.[4]

·           Total project costs were not monitored and reported because TE/GE Division management did not have a process in place to appropriately evaluate investment decisions.  The estimated contractor cost was exceeded by $2.1 million (26 percent), and estimated internal labor costs of $5.1 million were not tracked to the August 2004 Business Case.  As a result, we could not verify the reliability of $7.2 million in the Business Case.

·           Very few benefits were received from the development of TEDS Release 1.  Upon completion of Release 1, the technical infrastructure, including computer hardware and software designed to support all future System releases, was replaced because it did not meet new Internal Revenue Service (IRS) standards.  As a result, an inefficient use of resources occurred when little benefit was realized from the $17 million spent on Release 1.

·           Project management oversight may not ensure TEDS Release 2 will align with future IRS systems and processes.  Oversight is necessary to ensure additional costs are not incurred in the future to bring the applications into alignment with other systems.

Recommendations

We recommended the Commissioner, TE/GE Division, ensure costs, schedule delays, and changes in system capabilities on expected benefits are tracked and reviewed against business cases.  In addition, we recommended the Director, Business Systems Planning, complete plans to improve the tracking of actual Federal Government costs back to investments and provide this information to the TE/GE Division Investment Executive Steering Committee for review.

Response

The Commissioner, TE/GE Division, agreed with our recommendations and provided planned actions to address them.  These actions include ensuring costs, schedule delays, and changes in system capabilities on expected benefit are tracked against business cases and actual Federal Government costs are tracked back to investments and provided to the TE/GE Division Investment Executive Steering Committee for review.  However, IRS management disagreed with 1 of the 3 outcome measures described in the report ($17.0 million in inefficient use of resources).  The Commissioner, TE/GE Division, stated the lack of success in a venture such as TEDS Release 1 does not equate to inefficiency.  Instead, lessons learned from Release 1 helped form new IRS standards in Enterprise Architecture that were used in TEDS Release 2 and will be used in all future IRS projects.

In addition, the Commissioner, TE/GE Division, provided perspective on how the TE/GE Division manages systems development projects and how it experienced technical and other challenges in its work on the TEDS project.  Management’s complete response to the draft report is included as Appendix VI.  

Office of Audit Comment

We acknowledge the difficulty in managing a significant information technology project such as the TEDS project and believe the TE/GE Division’s commitment to make additional improvements through its stated corrective actions will improve oversight of future systems development projects.  Furthermore, we agree that lessons learned did provide a benefit to the IRS; however, we believe there was little tangible benefit for the investment in TEDS Release 1.  While we recognize the costs for TEDS Release 1 cannot be recovered at this point, increased controls over tracking costs, delays, or other system capabilities should help the IRS better monitor future projects and prevent implementation of new systems that are unable to accomplish their primary purposes.

In addition, the Commissioner, TE/GE Division, stated the TEDS project management team is working closely with the IRS Enterprise Architecture to comply with emerging standards as they apply to the TEDS and it is premature to conclude that Release 2 will not align with future Enterprise Architecture standards.  We agree with the Commissioner’s conclusion and have made applicable wording changes to the report.  However, we believe completing and approving required Enterprise Architecture documentation is a good preventive measure to ensure compliance with the future IRS Enterprise Architecture.

Copies of this report are also being sent to the IRS managers affected by the report recommendations.  Please contact me at (202) 622-6510 if you have questions or Nancy A. Nakamura, Assistant Inspector General for Audit (Headquarters Operations and Exempt Organizations Programs), at (202) 622-8500.

  

Table of Contents

 

Background

Results of Review

While Some Improvements Have Been Made, Management Oversight Did Not Ensure Significant Expected Savings Were Achieved and Expected Capabilities Were Delivered

Recommendations 1 and 2:

Appendices

Appendix I – Detailed Objectives, Scope, and Methodology

Appendix II – Major Contributors to This Report

Appendix III – Report Distribution List

Appendix IV – Outcome Measures

Appendix V – Glossary of Terms

Appendix VI – Management’s Response to the Draft Report

 

 Abbreviations

 

 

Background

 

The Tax Exempt and Government Entities (TE/GE) Division identified a critical need to modernize and improve its processing of Employee Plans function and Exempt Organizations function determination letter[5] applications.  The Employee Plans/Exempt Organizations Determination System in use at the time was an outdated technology that could not handle the required workload, fulfill statutory Internal Revenue Service (IRS) responsibilities under the Internal Revenue Code, and meet expectations of Employee Plans function and Exempt Organizations function customers.  In December 2001, the TE/GE Division Investment Executive Steering Committee[6] approved a seven-stage release strategy for the Tax Exempt Determination System (hereafter referred to as the TEDS or System) to replace the Employee Plans/Exempt Organizations Determination System, automate certain manual processes, and reduce or eliminate employee labor expense.

The TEDS was needed to address several critical workload factors that threatened to overwhelm the Employee Plans function and Exempt Organizations function Determinations Programs. 

·           Due to several law changes, the TE/GE Division anticipated receiving a significant increase in the number of employee plans restatement applications during Fiscal Years 2002 and 2003.

·           The Determinations Programs were not effective or efficient.  For example, the average time to work determination cases and the cycle time to respond to simple inquiries and status changes were too long; the level of accuracy for determinations was unacceptably low; expected peak volume demands could not be met; significant manual processes, paper handling, and tracking were inefficient; nonautomation of case grading, classification,[7] and assignment was inefficient; and postal costs were excessive because of an inefficient practice of mailing case files. 

The Determinations Programs are a key part of the TE/GE Division and the service provided to its customers.  Without tax-exempt status, plan sponsors may have to operate pension plans without favorable tax treatment, and exempt organizations cannot give donors tax-deductible receipts.

TE/GE Division Business Systems Planning function management started to develop TEDS Release 2 in September 2003, and the Director, Exempt Organizations, became the new executive owner in August 2005.  Release 2 was to include the upfront imaging of Employee Plans function and Exempt Organizations function determination applications, enhanced reporting, and automated case assignment.  It was originally scheduled for implementation in March 2007.  Release 2 implementation is currently scheduled for May 2008.

Project responsibilities for developing TEDS Release 2 are designated as follows:

·           The TE/GE Division Investment Executive Steering Committee has overall responsibility for approving the TEDS Business Cases used to request funding for the new computer system, monitoring and overseeing development of the project, and approving project requests to exit a milestone and continue with development and delivery of the System.  The Committee also has a responsibility to stop the project, if necessary, when the budget is exceeded or the success of the project is in jeopardy.

·           TEDS project management is responsible for managing project costs, schedules, and performance in accordance with IRS policy.  This includes responsibility for acquisition and delivery of products or systems; ensuring the System complies with the IRS Enterprise Architecture; providing status information and recommendations to the TE/GE Division Investment Executive Steering Committee; and managing changes to costs, schedules, and requirements of the project. 

This review was performed at the TE/GE Division National Headquarters Business Systems Planning function and Exempt Organizations function offices in Washington, D.C., during the period January through July 2007.  We conducted this performance audit in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives.  We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.  Detailed information on our audit objectives, scope, and methodology is presented in Appendix I.  Major contributors to the report are listed in Appendix II.

 

 Results of Review

 

While Some Improvements Have Been Made, Management Oversight Did Not Ensure Significant Expected Savings Were Achieved and Expected Capabilities Were Delivered 

The TE/GE Division Investment Executive Steering Committee improved its oversight of the development of TEDS Release 2 from that provided during Release 1 by using certain project management techniques.  In addition, TE/GE Division management began tracking some project costs as we had recommended.[10]  However, management did not fully implement our recommendations to evaluate investment decisions and monitor whether business benefits would be realized.  As a result, when TEDS Release 2 is completed, its cost may far outweigh its benefits.  The high cost of the System compared to the benefits it will deliver brings into question whether sound investment decisions were made during the System’s development and whether this was the best use of Federal Government funds.  We identified the following:

·           Less than one-half of the originally envisioned monetary benefits of Release 2 will be delivered, and the cost of the Release may now exceed the benefits.  At the end of our audit work, TE/GE Division management provided some explanations for the change in value of individual benefits listed in the February 2007 Business Case[11] and the impact schedule delays and deletion of some system capabilities had on overall project benefits.

·           The IRS exceeded by $2.1 million (26 percent) the estimated contractor cost and did not track the estimated $5.1 million in internal labor costs in the August 2004 Business Case.  As a result, we could not verify the reliability of $7.2 million in the Business Case.[12]

·           TE/GE Division management did not ensure required Enterprise Architecture documentation was completed as required.  Therefore, there is no assurance that the System was designed and built to work with other IRS computer systems to improve performance and productivity without the need to make additional future changes.

·           We previously reported that significant benefits from Release 1 were not realized (i.e., reduced cycle times, improved customer satisfaction, and improved customer service).  However, in our previous report, we did not comment on another significant benefit envisioned for Release 1:  providing the initial technical infrastructure (hardware and software) for future releases.  We determined this benefit was not realized because computer hardware, software, and custom code were completely replaced for Release 2.  As a result, an inefficient use of resources occurred when little benefit was realized from the $17 million spent on Release 1.[13]    

Certain project management techniques were generally planned for and applied during development of the TEDS Release 2

The TE/GE Division Investment Executive Steering Committee improved its oversight of the development of Release 2 by using certain project management techniques.  In previous reports concerning the development of Release 1,[14] we reported that project management techniques were not effectively used.

TE/GE Division management adopted the Enterprise Life Cycle-Lite as a process to plan and manage the development of Release 2.  The Enterprise Life Cycle-Lite is a disciplined system development methodology that uses reviews, checkpoints, and milestones to ensure projects are efficiently and effectively planned, designed, developed, and implemented.  TE/GE Division management planned for and generally implemented Enterprise Life Cycle-Lite techniques.  Those techniques and management’s activities included:

·           Risk management – Processes were developed to identify, quantify, respond to, and control potential problems that could severely affect Release 2 development goals.  We reviewed risk reports that identified the critical status of potential problems, as well as the actions identified to reduce the risk that the most likely problems could occur.

·           Configuration management – Processes were developed to identify, control, and approve changes to system documentation, computer source code, and off-the-shelf software. 

·           Requirements management – Processes were developed to gather and document requirements, track the design and programming to the requirements, and manage any changes.

·           Transition management – Processes were developed to transition from the old Employee Plans/Exempt Organizations Determination System to the new TEDS Release 2.  The processes included identifying the gaps between current and future processes for organizational alignment, staffing, training, processing, assets needed to close the gaps, documentation, and communication.

While TE/GE Division management improved their use of certain project management techniques, they did not fully evaluate their investment decisions and did not ensure alignment with the Enterprise Architecture.  If prior Treasury Inspector General for Tax Administration recommendations had been implemented, the cost overruns and decrease in planned benefits may have been avoided or further minimized.

The cost of TEDS Release 2 may exceed the reduced monetary benefits delivered

TEDS Release 2 may be completed with significantly reduced system capabilities and benefits.  The June 2004 TEDS Release 2 Business Case documented 16 high-level system capabilities costing $46 million that would deliver benefits of $58 million.  This Business Case was updated in August 2004 to document estimated benefits of $73.1 million.  However, after work began on Release 2, the TE/GE Division project team incurred delays in delivering benefits (e.g., 4 system capabilities were deleted, some system capabilities have been only partially delivered, and 5 system capabilities have been delayed by up to 2 years).  An additional 8 capabilities were added, but the February 2007 TEDS Release 2 Business Case documents the System is now expected to deliver only $33.5 million (less than one-half) of the $73.1 million August 2004 expected benefits. 

TE/GE Division management advised us the August 2004 TEDS Release 2 Business Case benefits were recalculated in February 2007 to correct:

·           Benefit projections that used double inflation for labor.

·           Benefit projections for changes in scope, schedule, and other financial factors.

·           Cyber Assistant application benefit projections.

In addition, according to TE/GE Division management, part of the large decrease in benefits was due to the elimination of certain TEDS features because functions that were intended to be performed by another system could not be accomplished as planned.  Figure 1 shows a comparison of the cost and benefits calculations for the TEDS Release 2 Business Cases dated June 2004, August 2004, and February 2007.

Figure 1:  Comparison of TEDS Business Case Changes in Costs and Benefits (in millions) Between June 2004 and February 2007

 

Figure 1 was removed due to its size.  To see Figure 1, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

The February 2007 Business Case shows a significant decrease in benefits from both the June 2004 and August 2004 Business Cases.  TE/GE Division management also advised us that updates to the benefits calculations after February 2007 will now put the benefits at over $35 million.

The Clinger-Cohen Act of 1996[15] requires Federal Government agencies to use a disciplined capital and investment control process to manage information technology investments.  Agencies are required to put their technology investment decisions in a true “business context” and analyze investments for their return on investment.  Annually, the Office of Management and Budget publishes Circular A-11, Preparation, Submission and Execution of the Budget, to assist Federal Government agencies in complying with the Clinger-Cohen Act of 1996.  Circular A-11 provides guidance on the preparation of business cases for information technology systems.

In addition, according to organizational policy, the IRS requires tracking, monitoring, and evaluating of all project costs[16] and expected benefits against the approved business case.  Business cases provide decision makers with benchmark information necessary to make evaluations on a project’s costs and benefits and a basis for measuring whether multimillion dollar investments make good business sense. 

We performed additional reviews of documentation to determine whether TEDS project management and the TE/GE Division Investment Executive Steering Committee tracked, monitored, and evaluated project costs as required.  We determined the following:

·           Project management did not have information that associated the value of expected benefits to specific system capabilities.  For example, the updated expected benefits in the February 2007 Business Case show Release 2 is now expected to deliver benefits of only $33.5 million, a decrease of $39.6 million from the August 2004 Business Case projection.  Through our review of documentation and discussions with project management, we could not identify the relationship between the system capabilities and the benefits that make up the $39.6 million that have been deleted.  However, at the end of our audit work, TE/GE Division management provided some information related to the changes in benefits.  Information that relates the value of benefits to any changes in system capabilities is critical for evaluating whether a system should receive continued funding and development.

·           Because project management did not have this information for Release 2, the Investment Executive Steering Committee did not receive the critical periodic updates of cost and benefit information needed to assess the impact that delayed or undelivered system capabilities had on expected benefits. 

·           Review of the Investment Executive Steering Committee’s monthly meeting notes identified the Committee addressed issues that concerned specific system capabilities or funding for specific contracts.  However, we did not find any documented discussion of the collective impact of schedule delays and the deletion of system capabilities on overall project benefits and actual project costs.  When we discussed this with Committee members, they advised us that schedule delays and benefits were regularly discussed at meetings.  However, we were not provided documentation showing these topics were discussed.  The Commissioner, TE/GE Division (a key member of the Investment Executive Steering Committee), also advised us that, while he may not have known the exact dollar figure by which benefits were decreasing, he and the Committee members always knew what capabilities the IRS would be receiving and what capabilities the IRS would be losing when making decisions.  He stated the decision‑making process may not have involved looking back at the August 2004 Business Case.  In addition, despite not knowing the exact amount by which benefits were declining, TE/GE Division management may not have considered other alternatives because the system the TEDS is replacing, the Employee Plans/Exempt Organizations Determination System, is a failing and dying system.

IRS policy provides that business cases may be updated any time during system development,[17] at the discretion of the Investment Executive Steering Committee.  Conditions that may trigger a revision to the business case include changing the scope of the project, increasing the cost of the project by more than 10 percent, or extending the targeted delivery dates by more than 10 percent of the estimate.  Adverse changes in cost, schedule, and performance do not, by themselves, automatically require a business case revision.  However, if changes in cost, schedule, or performance result in modified baseline information and/or increases greater than 10 percent, a revision to the business case and approval from the Investment Executive Steering Committee may be needed to proceed with development.  The revisions are necessary to ensure the most accurate information is available for making investment decisions, monitoring the progress of the system’s development, and evaluating whether business benefits are actually realized. 

The TEDS Business Case was last updated in February 2007.  However, due to the change in the project’s scope and the reduction in benefits of approximately 54 percent, we believe the Business Case should have been updated sooner.  We could not determine the reason for the February 2007 update because actual costs compared to estimated costs and the changes in capabilities and benefits were not tracked.  In addition to the significant reduction in benefits, there was at least an 8-month delay in the Release 2 implementation.  For the TEDS, the TE/GE Division Investment Executive Steering Committee had a responsibility to determine whether the Business Case needed to be revised or updated.  Knowing that the benefits decreased significantly, but not requesting a revision to the Business Case prevented the Committee from having full knowledge of the significant decrease in the benefits that the TEDS would deliver compared to the original estimates in the approved Business Case.  Significant deviations from the Business Case, such as cost overruns or decreased capabilities or benefits, should be discussed as soon as they are identified.

A process to track and monitor total project cost was not implemented, and overruns occurred

Total project costs were not monitored and reported because TE/GE Division management did not have a process in place to appropriately evaluate investment decisions associated with the development of TEDS Release 2.  None of the actual project costs to develop, implement, and evaluate Release 2 were tracked and monitored as required by IRS procedures.  As a result, project management was not aware that actual contractor costs significantly exceeded the estimate in the August 2004 Business Case.

We reviewed the Project Manager’s records and calculated that contractor costs exceeded the estimate by at least $2.1 million (an increase from $8.1 million to $10.2 million, or 26 percent) for Fiscal Years 2004 through 2006.  In addition, IRS internal labor costs estimated at $5.1 million were not tracked.  This information should have been provided to the TE/GE Division Investment Executive Steering Committee so it would be aware of significant deviations from the project’s estimated costs and could determine whether the expected return on investment was being achieved.  As a result, the Committee could not evaluate performance and determine if the project was being delivered within estimated costs.

TE/GE Division management implemented procedures in response to a recommendation[18] we made as part of our review of the TEDS Release 1.  We recommended adopting a business case model to include processes for tracking and monitoring actual project costs.  TE/GE Division management implemented procedures to verify actual contractor costs weekly to ensure the expenses were actually incurred.  However, the procedures did not include tracking, comparing, monitoring, and evaluating contractor costs and IRS internal labor costs to estimates in the approved TEDS Business Case.

In addition, the Project Manager advised us the current IRS timekeeping system does not adequately support tracking internal labor costs.  Release 2 project management advised us Federal Government labor cost is tracked back to the project as actual cost as accurately as possible, considering the limitations of the timekeeping system.  This has been a longstanding weakness in other areas of the IRS, such as in the Modernization and Information Technology Services organization that recently corrected the problem.[19]

We determined there was $7.2 million of potentially unreliable information in the TEDS August 2004 Business Case for Fiscal Years 2004 through 2006.  This includes $2.1 million paid to contractors that exceeded the estimated amount in the Business Case and $5.1 million in estimated IRS internal labor cost not tracked.[20]

Management Action:  Project management informed us the TE/GE Division Business Systems Planning function has been working to improve the tracking of actual Federal Government costs back to investments; however, additional process changes need to be made to facilitate reporting.

Minimal benefits were realized from TEDS Release 1

The TE/GE Division received very few benefits from the development of Release 1.  Approximately $17 million was spent to purchase the technical infrastructure, including computer hardware and the development of computer code.  Upon completion, Release 1 was expected to reduce cycle time, processing time, and processing costs, while implementing process improvements to enhance the quality of determinations and the quality of service delivered to TE/GE Division customers.  The technical infrastructure, including computer hardware and software, was designed to support Release 1 and all future TEDS releases. 

However, according to TE/GE Division management, the initial infrastructure had to be replaced when it was completed because it did not meet new IRS standards.  As a result, an inefficient use of resources occurred and the TE/GE Division realized little benefit from its $17 million investment for Release 1.[21] 

Management oversight may not ensure TEDS Release 2 will align with the future IRS Enterprise Architecture

Project management oversight may not ensure Release 2 will align with the future Enterprise Architecture.  Oversight is necessary to ensure additional costs are not incurred in the future to bring the applications into alignment with the Enterprise Architecture.  We identified the following two documents were not completed or approved as required.

·           The Release 2 Enterprise Architecture Assessment was not completed.  This omission is significant because the purpose of the Assessment is to determine whether the project is consistent with the Enterprise Architecture to ensure computer systems designed and built separately will improve performance and productivity.

·           The Enterprise Architecture Charter for Release 2 was incomplete and had not been approved.  This Charter is used to manage the scope of the project in a manner that is consistent with the IRS Enterprise Architecture.

The development, modernization, and enhancement of projects, such as TEDS Release 2, must show compliance with the IRS Enterprise Architecture.  Enterprise Architecture Directive 15, Project Chartering and Conformance with the IRS Enterprise Architecture, applies to all projects that are part of the future Architecture.  Projects such as TEDS Release 2 require chartering and verification of conformance to the Enterprise Architecture.

Anticipated benefits of Release 2, such as streamlined work processes and improved service delivery and efficiency, may not be realized if management does not ensure Release 2 is in alignment with the approved Enterprise Architecture.  Likewise, incomplete and unapproved project documentation increases the risk that Release 2 will not be successfully integrated into the Enterprise Architecture.

Recommendations

Recommendation 1:  The Commissioner, TE/GE Division, should implement controls to ensure costs and the impact of schedule delays and changes in system capabilities on expected benefits are tracked.  The TE/GE Division Investment Executive Steering Committee should review these impacts against business cases and ensure business cases are timely updated.

Management's Response:  IRS management agreed with the recommendation.  The TE/GE Division Business Systems Planning function governance and control office will implement controls to ensure costs and the impact of schedule delays and changes in system capabilities on expected benefits are tracked.  The TE/GE Division Investment Executive Steering Committee will review these impacts against business cases and ensure business cases are timely updated.  However, IRS management disagreed with our $17 million inefficient use of resources outcome measure representing the funds spent to purchase the technical infrastructure, including computer hardware and the development of computer code.  The Office of Audit Comment in Appendix IV (page 19) describes in more detail management’s response and our position with respect to this outcome measure.

Recommendation 2:  The Director, Business Systems Planning, should complete plans to improve the tracking of actual Federal Government costs back to investments and ensure this information is provided periodically to the TE/GE Division Investment Executive Steering Committee so it has complete information for evaluating system development projects.

Management's Response:  IRS management agreed with the recommendation.  The Director, Business Systems Planning, will complete plans for improving the tracking of actual Federal Government costs back to investments and ensure this information is provided periodically to the TE/GE Division Investment Executive Steering Committee.

 

Appendix I

 

Detailed Objectives, Scope, and Methodology

 

The overall objectives of this review were to determine whether the TE/GE Division developed TEDS Release 2 using sound system development practices and whether it managed the Release 2 investment in compliance with Office of Management and Budget and Clinger-Cohen Act of 1996[22] requirements.  To accomplish these objectives, we:

I.                   Determined if TE/GE Division management applied and adhered to the approved Enterprise Life Cycle-Lite[23] methodology for development of TEDS Release 2.  Specifically, we determined if:

A.    TE/GE Division management’s development approach was adequate to monitor and control the Release 2 activities from project startup through closure.

B.     The risk management process was adequate to identify, quantify, respond to, and control the risks that could prevent successful completion of the project.

C.     The configuration management process was adequate to identify, document, monitor, evaluate, and approve all changes to system requirements.

D.    The requirements management process adequately identified the Release 2 user requirements.

E.     The transition management process was adequate to ensure Release 2 is ready for deployment and transfer to the Modernization and Information Technology Services organization.

II.                Determined if TE/GE Division management ensured alignment with the IRS Enterprise Architecture.  Specifically, we determined if:

A.    The Release 2 project team adhered to Enterprise Architecture requirements.

B.     Project documents identified how Release 2 fits into the IRS architecture of the future.

C.     The project team had coordinated its development efforts and resolved any differences with the Tier II Project Office.

III.             Determined if TE/GE Division management’s investment decisions followed estimates in the TEDS Business Cases. 

A.    Compared the expected and actual Release 2 capabilities and determined if Release 2 was developed as intended to deliver the expected benefits. 

B.     Reviewed project management records and determined if actual expenditures were compared to estimates in the August 2004 Business Case.

C.     Followed up on the development of Release 1 and determined whether benefits outlined in the August 2003 Release 1 Business Case were delivered.

Internal controls methodology

Internal controls relate to management’s plans, methods, and procedures used to meet their mission, goals, and objectives.  Internal controls include the processes and procedures for planning, organizing, directing, and controlling program operations.  They include the systems for measuring, reporting, and monitoring program performance.  We determined the following internal controls were relevant to our audit objectives:  Office of Management and Budget policies, Clinger-Cohen Act of 1996 requirements, Enterprise Life Cycle-Lite requirements, and IRS investment management requirements.  We reviewed these controls by interviewing Business Systems Planning function and Exempt Organizations function management, analyzing Investment Executive Steering Committee minutes, and reviewing various project plans.

 

Appendix II

 

Major Contributors to This Report

 

Nancy A. Nakamura, Assistant Inspector General for Audit (Headquarters Operations and Exempt Organizations Programs)

Troy D. Paterson, Director

Gerald T. Hawkins, Acting Director

Julia M. Collins, Acting Audit Manager

John W. Baxter, Lead Auditor

Andrew J. Burns, Lead Auditor

Deadra M. English, Senior Auditor

Yasmin B. Ryan, Senior Auditor

Michael A. McGovern, Auditor

 

Appendix III

 

Report Distribution List

 

Acting Commissioner  C

Office of the Commissioner – Attn:  Acting Chief of Staff  C

Deputy Commissioner for Services and Enforcement  SE

Deputy Commissioner, Tax Exempt and Government Entities Division  SE:T

Director, Business Systems Planning, Tax Exempt and Government Entities Division  SE:T:BSP

Director, Exempt Organizations, Tax Exempt and Government Entities Division  SE:T:EO

Chief Counsel  CC

National Taxpayer Advocate  TA

Director, Office of Legislative Affairs  CL:LA

Director, Office of Program Evaluation and Risk Analysis  RAS:O

Office of Internal Control  OS:CFO:CPIC:IC

Audit Liaison:  Director, Communications and Liaison, Tax Exempt and Government Entities Division   SE:T:CL 

 

Appendix IV

 

Outcome Measures

 

This appendix presents detailed information on the measurable impact that our recommended corrective actions will have on tax administration.  These benefits will be incorporated into our Semiannual Report to Congress.

Type and Value of Outcome Measure:

·           Reliability of Information – Potential; $39.6 million (see page 4).

Methodology Used to Measure the Reported Benefit:

IRS project management did not have information that associated the value of expected benefits to specific system capabilities.  For example, the updated expected benefits in the February 2007 TEDS Business Case show Release 2 is now expected to deliver benefits of only $33.5 million, a decrease of $39.6 million from the original projection in the August 2004 TEDS Business Case.  Information that relates the value of benefits to the system capabilities is needed to assess the impact that delayed or undelivered system capabilities had on the expected benefits of Release 2.  Because this information was not available, project management could not assess the impact the decrease in expected benefits had on the Release 2 investment.

Benefits expected from Release 2 listed in the
August 2004 Business Case                                                                            $73.1 million

Less:  Benefits expected from Release 2 listed in the
February 2007 Business Case                                                                         $33.5 million

Total                                                                                                               $39.6 million

Type and Value of Outcome Measure:

·           Reliability of Information – Potential; $7.2 million (see page 4).

Methodology Used to Measure the Reported Benefit:

We determined contractor costs were being tracked; however, the IRS did not compare actual contractor costs to the estimated contractor costs listed in the August 2004 Business Case.  We made this comparison and determined contractor costs had exceeded estimates by $2.1 million through Fiscal Year 2006.  We also determined TE/GE Division management was not tracking internal labor costs; therefore, we could not determine whether the $5.1 million estimated for IRS internal labor in the August 2004 Business Case was reliable.  We added these amounts to determine the amount of potential unreliable information in the Business Case.

Amount contractor costs exceeded estimates in the
August 2004 Business Case through Fiscal Year 2006                                              $2.1 million

Add:  Estimated IRS internal labor costs in the
August 2004 Business Case that were not tracked                                                    $5.1 million

Total of potentially unreliable cost information in the Business Case                  $7.2 million

The estimated IRS internal labor costs in the TEDS Business Case were taken directly from the August 2004 Business Case.  The amount of contractor costs that exceeded estimates in the TEDS Business Case through Fiscal Year 2006 were calculated by subtracting the estimated contractor cost to implement Release 2 through Fiscal Year 2006 from the actual contractor cost to implement Release 2 through Fiscal Year 2006.

Actual contractor cost to implement Release 2 through Fiscal Year 2006                 $10.2 million

Less:  Estimated contractor cost to implement Release 2 through
Fiscal Year 2006                                                                                                           $8.1 million

Total by which contractor costs exceeded estimates in the
August 2004 Business Case through Fiscal Year 2006                                            $2.1 million

We calculated the actual contractor cost to implement Release 2 through Fiscal Year 2006 by adding contractor invoice information collected by IRS project management.  We calculated the estimated contract cost to implement Release 2 through Fiscal Year 2006 by subtracting the estimated TE/GE Division and IRS labor cost of $5.1 million listed in the August 2004 Business Case from the $13.2 million total cost to implement Release 2 listed in that Business Case.

Type and Value of Outcome Measure:

·           Inefficient Use of Resources – Actual; $17.0 million (see page 4). 

Methodology Used to Measure the Reported Benefit:

TEDS Release 1 did not significantly improve the processing of Employee Plans function determination applications, and the technical infrastructure designed to support future releases of the System was replaced.  As a result, the TE/GE Division realized little benefit from its $17 million investment.

We determined the total Release 1 costs by adding prior project costs listed in the August 2003 TEDS Business Case to IRS internal labor costs and contractor costs to complete Release 1.

Prior project costs listed in the August 2003 Business Case                          $12.0 million

Add:  TE/GE Division internal labor costs from August 7, 2003, through
February 17, 2006                                                                                               $.5 million

Add:  Contractor costs from August 7, 2003, through February 17, 2006      $4.5 million

Total                                                                                                               $17.0 million

The TE/GE Division labor costs to complete Release 1 were obtained from IRS project management.  The contractor costs from August 7, 2003, through February 17, 2006, were calculated by adding contractor invoice information collected by IRS project management.

Office of Audit Comment

IRS management disagreed with 1 of the 3 outcome measures described in the report ($17.0 million in inefficient use of resources).  The Commissioner, TE/GE Division, stated the lack of success in a venture such as TEDS Release 1 does not equate to inefficiency.  Instead, lessons learned from Release 1 helped form new IRS standards in Enterprise Architecture that were used in TEDS Release 2 and will be used in all future IRS projects. 

We agree that lessons learned did provide a benefit to the IRS; however, we believe there was little tangible benefit for the investment in TEDS Release 1.  Furthermore, while we recognize the costs for TEDS Release 1 cannot be recovered at this point, increased controls over tracking costs, delays, or other system capabilities should help the IRS better monitor future projects and prevent implementation of new systems that are unable to accomplish their primary purposes.

In addition, the Commissioner, TE/GE Division, stated the TEDS project management team is working closely with the IRS Enterprise Architecture to comply with emerging standards as they apply to the TEDS and it is premature to conclude that Release 2 will not align with future Enterprise Architecture standards.  We agree with the Commissioner’s conclusion and have made applicable wording changes to the report.  However, we believe completing and approving required Enterprise Architecture documentation is a good preventive measure to ensure compliance with the future IRS Enterprise Architecture.

 

Appendix V

 

Glossary of Terms

 

 

Appendix VI

 

Management’s Response to the Draft Report

 

The response was removed due to its size.  To see the response, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.