TREASURY INSPECTOR GENERAL FOR TAX ADMINISTRATION

 

 

The Tax Exempt Determination System Release 1 Delivered Only a Small Portion of the Expected Benefits and Significantly Exceeded Cost Estimates

 

 

 

September 26, 2006

 

Reference Number:2006-10-174

 

 

This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.

 

Phone Number ††|202-927-7037

Email Address ††|Bonnie.Heald@tigta.treas.gov

Web Site†††††† |http://www.tigta.gov

 

September 26, 2006

 

 

MEMORANDUM FOR COMMISSIONER, TAX EXEMPT AND GOVERNMENT ENTITIES

†††††††††††††††††††††††††††††††††††††††† DIVISION

 

FROM:††††††††††††††††††††††††††† Michael R. Phillips /s/ Michael R. Phillips

Deputy Inspector General for Audit

 

SUBJECT:††††††††††††††††††† Final Audit Report Ė The Tax Exempt Determination System Release 1 Delivered Only a Small Portion of the Expected Benefits and Significantly Exceeded Cost Estimates (Audit # 200610010)

 

This report presents the results of our review to determine whether implementation of the Tax Exempt Determination System (TEDS) Release 1 improved the Tax Exempt and Government Entities (TE/GE) Divisionís processing of Short Form Application for Determination for Employee Benefit Plan (Form 5307) and provided electronic Employee Plans (EP) determination information to other TE/GE Division programs.

Implementation of TEDS Release 1 was expected to provide significant benefits in the receipt, handling, and processing of Forms 5307 to the EP function and other TE/GE Division programs that need access to Form 5307 information.The enhancements were supposed to assist TE/GE Division management with a large influx of determination applications received as a result of a remedial amendment period.[1]

Impact on the Taxpayer

TEDS Release 1 was designed to reduce the processing time for Form 5307 determination applications and improve service to customers.However, the automation of closing processes for these applications and the automation of processes for issuing determination letters were not operating as envisioned. As a result, the new System did not significantly improve the processing of Forms 5307.This is important because the time to process all types of determination applications increased from 208 calendar days to 324 calendar days between Fiscal Years 2004 and 2005, the period of time in which the new System was operational.

Synopsis

The delivery of TEDS Release 1 on March 16, 2004, did not significantly improve the TE/GE Divisionís processing of Form 5307 determination applications or assist in providing electronic EP determination information to other TE/GE Division programs, as originally envisioned.

Because TEDS Release 1 was implemented a couple of months after the remedial amendment period and the EP function had already started to receive a large number of applications, the EP function would not have realized the full benefits of the System even if it had worked effectively.As a result, the delay in the delivery of the System severely limited the impact TEDS Release 1 had on the processing of Form 5307 determination applications.Also, while electronic Form 5307 information was available to TEDS users, its impact was limited because the use of electronic images had not been fully incorporated into TE/GE Division operating procedures.

Delays in the delivery of TEDS Release 1, as well as the Systemís failure to deliver many of the projected benefits, severely limited its impact on the processing of Form 5307 determination applications.

In addition to the failure of TEDS Release 1 to deliver many of the projected benefits, investments associated with the development of TEDS Release 1 were not appropriately tracked, which prevented TE/GE Division senior management from receiving the information needed to effectively evaluate their investment in the TEDS.TEDS Release 1 was developed from December 2001[2] to March 2004.The TE/GE Division used a business case model for estimating the costs and benefits to justify the need for the project and to provide a basis for estimating the cost associated with the development of the TEDS.The actual cost to develop, implement, and maintain TEDS Release 1 (through February 17, 2006) was over $16.9 million, which was approximately $2.3 million higher than that estimated in August 2003.However, the August 7, 2003, TEDS Business Case was not appropriately updated to reflect the severely curtailed capabilities of the System, the 16 percent increase in cost, and the 5-month delay in delivery of the System.Thus, TE/GE Division management did not use the Business Case to fully evaluate investment decisions.This is especially critical because TEDS Release 2 is under development by the TE/GE Division, where the Director, Exempt Organizations function, is responsible for overseeing its implementation.

Recommendations

We recommended the Director, Business Systems Planning, ensure tasks needed to achieve the system functionality for future releases of the TEDS are clearly established to provide greater assurance that expected benefits will be realized.We recommended the Director, Exempt Organizations, implement processes to ensure the TEDS Business Case is revised when changes to the projectís scope occur, increases in the projectís target completion date are extended by more than 10 percent, or project costs are increased by more than 10 percent, so the most current information is available to make informed investment decisions; monitor the progress of the Systemís development; and evaluate whether business benefits are realized.We also recommended the Director, Exempt Organizations, fully evaluate investment decisions by adopting a business case model that includes processes for tracking actual costs of processing the determination applications to compare with the estimated costs.

Response

TE/GE Division management agreed with the findings, recommendations, and measurable benefits on tax administration included in the report.Specifically, TE/GE Division management has taken corrective actions to ensure the criteria for automating case closure processes are as complete as possible for the available data, to increase the flexibility of the criteria governing automated case closure and automated letter generation systems in TEDS Release 2, and to ensure processes are in place to identify all requirements.TE/GE Division management has implemented corrective actions to ensure information needed to make investment decisions, monitor the progress of the Systemís development, and evaluate whether business benefits are realized is provided to TE/GE Division managers responsible for making decisions regarding TEDS Release 2 investments.Further, TE/GE Division management has taken action to ensure actual costs are tracked and compared against estimated costs.Managementís complete response to the draft report is included as Appendix V.

Copies of this report are also being sent to the Internal Revenue Service managers affected by the report recommendations.Please contact me at (202) 622-6510 if you have questions or Nancy A. Nakamura, Acting Assistant Inspector General for Audit (Headquarters Operations and Exempt Organizations Programs), at (202) 622-8500.

 

 

Table of Contents

 

Background

Results of Review

Critical Portions of the Tax Exempt Determination System Release 1 Did Not Operate As Expected, Which Prevented Expected Benefits From Being Realized

Costs and Benefits for Developing the Tax Exempt Determination System Release 1 Were Not Appropriately Tracked and Monitored

Recommendation 1:

Recommendations 2 and 3:

Appendices

Appendix I Ė Detailed Objective, Scope, and Methodology

Appendix II Ė Major Contributors to This Report

Appendix III Ė Report Distribution List

Appendix IV Ė Outcome Measure

Appendix V Ė Managementís Response to the Draft Report

 

 

Abbreviations

 

CAS

Customer Account Services

EDS

Employee Plans and Exempt Organizations Determination System

EP

Employee Plans

FY

Fiscal Year

IRS

Internal Revenue Service

TEDS

Tax Exempt Determination System

TE/GE

Tax Exempt and Government Entities

 

 

Background

 

The Tax Exempt and Government Entities (TE/GE) Division uses the Employee Plans and Exempt Organizations Determination System (EDS) to process and control Employee Plans (EP) and Exempt Organizations determination letter[3] applications.For EP customers, the determination letter process is a key interaction with the Internal Revenue Service (IRS) because a favorable determination letter gives the employer or plan sponsor the assurance a pension plan is qualified for favorable tax treatment, as long as the plan is operated under the terms on which the determination letter was issued.

However, TE/GE Division management identified shortcomings with the EDS and the determination letter process, which affected their ability to achieve their mission and strategic performance goals.For example, TE/GE Division management determined their performance was constrained by outdated technology that lacked the capacity to handle the required workload, fulfill statutory IRS responsibilities under the Internal Revenue Code, and meet expectations of the EP customers.These shortcomings included:

         Average time to work a case is too long.

         Cycle time[4] to respond to simple inquiries and status changes is too long.

         Level of accuracy in determinations is unacceptably low.

         The EDS can not meet expected peak volume demand.

         Significant manual processes, paper handling, and tracking are inefficient.

         Nonautomation of case grading, classification,[5] or assignment is inefficient.

         Postal costs are excessive because of an inefficient practice of mailing case files.

To address the shortcomings, the TE/GE Division redesigned its business processes in a Business Concept of Operations[6] document dated November 14, 2001.The concept included defining new business processes to be used for processing EP and Exempt Organizations determination letter applications. TE/GE Division management decided to invest in building a new system to replace the EDS and enable the IRS to meet current and future determination letter processing needs of both EP and Exempt Organizations customers.

In December 2001, the TE/GE Division Investment Executive Steering Committee[7] approved a multiple release strategy for the new system, the Tax Exempt Determination System (TEDS).Overall, the TEDS was expected to reduce cycle time, processing time, and processing costs, while implementing process improvements that were to enhance the quality of determinations and the quality of service delivered to customers.TEDS Release 1 was to provide the technical infrastructure for all future releases of the TEDS and would redesign the process for one of the simpler EP determination request forms, Short Form Application for Determination for Employee Benefit Plan (Form 5307).TEDS Release 1 would process the Forms 5307 from receipt by the IRS to issuance of a determination letter or acknowledgment notice.Specifically, implementation of TEDS Release 1 was expected to provide the following system enhancements:

         Control the receipt and handling of Forms 5307.

         Convert hard-copy Forms 5307 and other submitted documents to electronic versions that can be accessed in the TEDS to perform determination processing.

         Automate case grading and classification of Forms 5307 for case assignment.

         Automate case closure for Forms 5307 through the use of business rules.[8]

         Automate the issuance of determination letters for Forms 5307.

         Generate management information reports for monitoring the processing of determination applications.

Due to several tax law changes, the TE/GE Division anticipated receiving a significant increase in the number of EP determination applications associated with a remedial amendment period.[9]TE/GE Division management had set the goal of having TEDS Release 1 operational in Fiscal Year (FY) 2004 to handle the additional EP determination applications.

A prior Treasury Inspector General for Tax Administration audit[10] found that Business Systems Planning program managementís planned use of a modified Enterprise Life Cycle and added project management techniques[11] for the development of the TEDS complied with IRS guidelines for the initial stages of systems development.However, TEDS project management did not implement these techniques as planned.This resulted in the following inefficiencies:

         Requirements gathering led to delays in project development.

         Performance monitoring did not ensure project objectives were completed on schedule or within budget.

         Risk management[12] did not identify potential problems that could affect completion of TEDS development.

In the prior audit, we recommended TEDS project management identify and explain cost, benefit, schedule, and contractor performance variances in the next Business Case[13] update.The Business Case update should include a report of the actual business benefits realized as compared to projected benefits.We also recommended TEDS project management fully implement the planned risk management process.TE/GE Division management agreed with the recommendations and implemented actions to address the problems identified by the audit.

This review was performed at the IRS National Headquarters Business Systems Planning and EP Program offices in Washington, D.C.; the Cincinnati Submission Processing Site[14] in Covington, Kentucky; the EP Rulings and Agreements Determinations program, the EP Determinations Quality Assurance program, and the TE/GE Division Customer Account Services (CAS) call site in Cincinnati, Ohio; and the EP Examinations function in Baltimore, Maryland.The audit work was conducted during the period January through April 2006 in accordance with Government Auditing Standards. Detailed information on our audit objective, scope, and methodology is presented in Appendix I.Major contributors to the report are listed in Appendix II.

 

Results of Review

 

Critical Portions of the Tax Exempt Determination System Release 1 Did Not Operate As Expected, Which Prevented Expected Benefits From Being Realized

The delivery of TEDS Release 1 on March 16, 2004, did not significantly improve the TE/GE Divisionís processing of Form 5307 determination applications or assist in providing electronic EP determination information to other TE/GE Division programs, as originally envisioned. Because TEDS Release 1 was implemented a couple of months after the remedial amendment period and the EP function had already started to receive a large number of applications, the EP function would not have realized the full benefits of the system even if it had worked effectively.As a result, the delay in the delivery of the System severely limited the impact TEDS Release 1 had on the processing of Form 5307 determination applications.In addition, while electronic Form 5307 information was available to TEDS users, its impact was limited because the use of electronic images had not been fully incorporated into TE/GE Division operating procedures.

Enhancements expected from TEDS Release 1 did not significantly improve the processing of EP Form 5307 determination applications

For each of the planned system capabilities for TEDS Release 1, we determined whether the System was operating as intended as well as the reasons why any system capabilities were not operating.The TE/GE Division successfully ensured TEDS Release 1 system capabilities for controlling and converting hard-copy Form 5307 determination applications into electronic case files were implemented.It also successfully automated some of the case assignment processes by systemically grading and classifying Form 5307 determination applications.While no data existed at the time of our audit to assess the impact of these enhancements, the implementation of these additional capabilities improved Form 5307 processing by reducing the number of human resources needed to perform these functions.

However, key portions of the TEDS Release 1 functionality did not deliver the expected benefits projected in the August 7, 2003, TEDS Business Case, which greatly limited the impact the System had on the processing of determination applications.As described in more detail below, the automation of closing processes for Form 5307 determination applications that met selected criteria as well as the automation of processes for issuing determination letters were not operating as envisioned.In addition, management information reports that were to be used to monitor the determination applications after IRS receipt and initial processing were not operating as expected and were not being used.

The TE/GE Division had planned on systemically closing a portion of Form 5307 determination applications by programming selected processing procedures or business rules into the case evaluation processes.Determination applications that met specific criteria were supposed to be automatically closed with a determination letter automatically issued to the customer with no human intervention.However, during pilot testing from July 28, 2003, through March 16, 2004, the EP function determined many of the cases proposed for automatic closure were at a significant risk of being inappropriately closed because sufficient evidence was not available to ensure the plans met all of the criteria for tax-exempt status.

The automated case closure feature was designed to expedite the processing for determination applications and to eliminate the need for EP personnel to prepare and send determination letters to customers.Combined, these enhancements were expected to yield monetary benefits resulting from ďproductivity increases that translate into reinvestment opportunities for the IRS.ĒFor example, the TEDS Business Case from August 2003 showed the TE/GE Division had estimated achieving over $788,000 in monetary benefits for FYs 2004 and 2005 resulting from reduced labor costs[15] for processing determination applications.

Several issues prevented the EP function from implementing the automated closure feature:

At the time of our review, Form 5307 determination applications that originally met the criteria for automated closure were being classified as proposed auto-closures.The proposed auto-closures required a review by revenue agents to determine whether additional actions were needed to approve the application and issue a favorable determination letter.While there is a benefit to using the proposed auto-closure feature, the estimated labor savings of over $788,000 did not materialize because the automated case closure and automated letter generation features were not operating as envisioned.

EP function management and TEDS project management advised us the automated case closure feature described in the March 15, 2002, TEDS Business Concept of Operations will not be available in future releases of TEDS applications unless significant changes are made to either the criteria for applying business rules or the Form 5307.The only EP Form that is expected to be closed automatically is the Notice of Merger, Consolidation or Transfer of Plan Liabilities (Form 5310-A), which is a merger notice that is not required to be reviewed because no determination or acknowledgement letter is required.

TEDS Release 1 was also intended to yield nonmonetary benefits, including the following:

         Reduced cycle time for processing EP determination applications.

         Improved customer satisfaction as measured by the customer satisfaction survey score.

         Improved quality of determination letters.

However, as shown in Figure 1, cycle time for processing applications increased despite the piloting and implementing of the TEDS in FY 2004.

Figure 1: Increased Cycle Time for EP Determination Applications

Figure 1 was removed due to its size.To see Figure 1, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

 

The TE/GE Business Performance Reviews showed the TE/GE Division attributed the increased cycle time to a higher than anticipated volume of determination applications received from adopters of pre-approved plans and the extension of time in which plan sponsors had to submit their determination applications.In FY 2004, the EP function estimated approximately 42,000 determination applications would be received; the actual volume was 70,610 receipts.

However, the increases in cycle time could have been significantly mitigated with successful implementation of the automated case closure feature described by the August 7, 2003, TEDS Business Case.This Business Case showed ďa significant variable impacting the benefits associated with TEDS Release 1 is the percentage of cases that will be closed automatically with TEDS.ĒThe TE/GE Division expected approximately 32 percent of the Forms 5307 would be closed automatically without human intervention.Because the automated case closure feature was not operating as envisioned by the Business Case, the Forms 5307 that would have been auto-closed were identified for proposed auto-closures, which required a review by revenue agents when the System was placed into production in FY 2004.This contributed to increasing cycle time from 176 calendar days to 324 calendar days between FYs 2003 and 2005.

During this same time period, TEDS Release 1 did not contribute towards the performance measures for improving customer satisfaction and improving the quality of determination letters.Figure 2 presents the decline in these two performance measures.

Figure 2: Decreased Customer Satisfaction and Letter Quality

Figure 2 was removed due to its size.To see Figure 2, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

 

The TE/GE Business Performance Reviews attributed a decrease in the level of satisfied customers from 69 percent in FY 2003 to 61 percent in FY 2005 primarily to the large increase in workload resulting from the determination applications received from adopters of pre-approved plans and the corresponding delay in issuing determination letters.During this same period, the quality of determination letters decreased from 80 percent to 76 percent.The TE/GE Business Performance Reviews for the EP function attribute the decrease in quality in FYs 2004 and 2005 to use of a more rigid definition of the timeliness standards, which no longer allowed for reviewer judgment or mitigating circumstances.This was coupled with a large build-up of inventory for determination applications.

The automated letter generation system capability did not deliver the expected benefits described in the August 7, 2003, TEDS Business Case because TEDS Release 1 lacked the functionality to:

TEDS Release 1 also did not include the capability to monitor the processing of the Form 5307 determination applications from receipt by the IRS to issuance of a determination letter or acknowledgement notice.TE/GE Division management advised us the TEDS report feature for the initial receipt and processing of the Form 5307 determination applications was being used. However, the report feature was not being used to control the inventory of determination applications when they are ready for assignment to a revenue agent.TE/GE Division management advised us the report feature used to control the inventory of determination applications was not critical to TEDS Release 1 because the inventory was controlled on the EDS.As a result, the TE/GE Division has been relying exclusively on the EDS for inventory control reports and uses two ďBalancing ReportsĒ[16] to reconcile the two Systems.TE/GE Division management determined the combination of the Balancing Reports and the EDS inventory reports was sufficient to control inventory and that incurring additional costs to create the TEDS Release 1 reports would not be practical because the EDS was the inventory system for processing and controlling the Form 5307 determination applications.However, the EDS was the System TE/GE Division management stated was inefficient in the March 15, 2002, TEDS Business Concept of Operations.

Project management is a carefully planned and organized effort to accomplish a specific one-time effort (e.g., implementation of a new computer system).Project management includes defining project goals and objectives, specifying tasks on how the goals will be achieved and what resources are needed, and associating budgets and timeliness for completion.

Because the effort to implement TEDS Release 1 was not sufficiently planned and monitored to ensure tasks would achieve project goals, there was less assurance that system capabilities would operate as planned, expected benefits would be delivered, and the System would be implemented within estimated project cost.

TEDS Release 1 did not benefit other TE/GE Division operations not directly associated with the processing of Form 5307 determination applications

System documentation showed several TE/GE Division operations not associated with the processing of EP determination applications would also benefit from the implementation of TEDS Release 1.The August 7, 2003, TEDS Business Case identified the TE/GE CAS function as one operation that would be using TEDS functionality, and the March 15, 2002, TEDS Business Concept of Operations identified the EP Examination and EP Quality Assurance functions as other operations that would be using TEDS information.

The functionality of TEDS Release 1 was intended to provide the TE/GE CAS call site with online access to EP Form 5307 determination applications and the ability to reproduce copies of closing letters.However, because the EP Forms 5307 are actually processed on the EDS, TEDS Release 1 did not show whether the EP determination application had actually completed processing.As a result, the TE/GE CAS function used the EDS information, not the TEDS, to answer customer inquiries regarding the status of their determination applications.

While the March 15, 2002, TEDS Business Concept of Operations showed the EP Examination function would be using information from TEDS Release 1, EP Examination management stated TEDS Release 1 would not affect its operation because that functionality was not implemented.Future releases of the TEDS are expected to provide EP revenue agents with advance copies of determination letters and with the applications documents associated with the plans prior to initiating an audit of the employee plan.

The EP Quality Assurance function expected a reduction in the error rates on determination letters and timelier issuance of determination letters with the implementation of TEDS Release 1.However, these benefits were never realized because major portions of the system functionality were not delivered.EP employees have the capability to automatically select cases for review, but they select the cases from the EDS because this is ďtheir system of record.ĒAlthough no benefits were realized with TEDS Release 1 implementation, EP Quality Assurance function management stated the nondelivered system functionality did not affect their ability to meet program goals and provide quality service.

Costs and Benefits for Developing the Tax Exempt Determination System Release 1 Were Not Appropriately Tracked and Monitored

Investments associated with the development of TEDS Release 1 were not appropriately tracked, which prevented TE/GE Division senior management from receiving the information needed to effectively evaluate their investment in the TEDS.The TE/GE Division used a business case model for estimating the cost and benefits.The business case model was intended to provide a basis for making investment funding decisions and to:

         Justify the need for the project to investment decision makers.

         Provide justification for prioritizing, selecting, and funding the investment throughout the life cycle for the project.

         Establish baseline costs, schedule, and performance goals to control and evaluate the investment of resources into the project.

However, TE/GE Division management did not appropriately update the August 7, 2003, TEDS Business Case or demonstrate the business case model was actually used to evaluate investment decisions because there was no process in place to compare the actual cost of processing determination applications with the cost estimated in the Business Case.

Over the last decade, Congress has enacted a variety of legislation to change how the Federal Government conducts its business.This legislation seeks to improve mission performance through more effective strategic, financial, and acquisition management.One law enacted was the Clinger-Cohen Act of 1996,[17] which requires agencies of the Federal Government to focus on the results that are achieved when investing in Information Technology projects.Agencies are required to evaluate technology investment decisions in a true ďbusiness contextĒ and to analyze the investments to determine their return on investment.Even after the system has been deployed, agencies should be comparing the actual business benefits with the projected business benefits for each Information Technology investment.To meet this requirement, agencies must implement processes and maintain information needed to assess whether Information Technology projects are completed at acceptable costs, within reasonable and expected time periods, and are contributing to tangible, observable improvements in mission performance.The Clinger-Cohen Act of 1996 also requires that business benefits from each Information Technology investment be reported to the Office of Management and Budget and Congress annually.The business case model assists in satisfying this requirement.

IRS policy generally requires business cases to be updated at specific points of system development,[18] although revisions to a business case can occur at any time.Conditions that require a revision to the business case include changing the scope of the project, increasing the cost of the project by more than 10 percent, or extending the targeted delivery dates by more than 10 percent of the previous estimate.The revisions are necessary to ensure the most accurate information is available for making investment decisions, monitoring the progress of the systemís development, and evaluating whether business benefits are actually realized.

The original baseline Business Case for the total TEDS project was prepared in June 2001 and updated in August 2003 to include only TEDS Release 1 costs and benefits.However, the Business Case for TEDS Release 1 has not been updated since August 2003, even though all three conditions that require an update to the Business Case have been met.For example, during the pilot testing of TEDS Release 1, problems surfaced with the quality of cases meeting the rules for closing cases using automated business rules processes and with the quality of determination letters that were to be issued using automated processes of TEDS Release 1.This resulted in a significant change to the scope of the TEDS Release 1 project because processes that were designed to close some EP Forms 5307 without human intervention were changed to a process still requiring a revenue agentís review.Problems with the automated letter generation portion of TEDS Release 1 software could not be used, which resulted in a change to the scope and required additional resources to review determination letters for cases meeting selected criteria.

Since issuance of the August 7, 2003, TEDS Business Case, the cost to deliver TEDS Release 1 had also exceeded the estimated project cost by over 10 percent.Total expected project cost through FY 2006 for TEDS Release 1 was estimated at approximately $14.7 million in the Business Case updated in August 2003.However, the actual cost to implement and maintain TEDS Release 1 was about $17 million as of February 17, 2006, which exceeded the estimate by approximately $2.3 million or 16 percent.

The TE/GE Division also did not meet the delivery dates targeted in the updated Business Case, which should have triggered another update to reevaluate the Systemís performance.The August 2003 Business Case showed that TEDS Release 1 was scheduled to be deployed in October 2003, but it was not actually deployed until March 2004.

In addition to not updating the August 7, 2003, TEDS Business Case, TE/GE Division management did not have a process in place to appropriately evaluate investment decisions associated with the development of TEDS Release 1 because the actual cost of processing determination applications was not compared with the cost estimated in the Business Case.The August 7, 2003, TEDS Business Case presented the annual estimated cost of processing determination applications using TEDS Release 1 functionality; however, no process was in place to assess whether the estimates were accurate because the actual cost was not tracked.

TE/GE Division management advised us the Business Case was not updated because they considered TEDS Release 1 to be essentially complete at the time the System was placed into a production mode.However, we believe another update to the Business Case along with an analysis comparing the actual cost of processing determination applications with the estimate in the Business Case would have provided TE/GE Division management with the information necessary to appropriately evaluate investment decisions as required by the Clinger-Cohen Act of 1996.

The TE/GE Division started planning for TEDS Release 2 in September 2003, before TE/GE Division management was fully aware of the limitations of the System.As part of this planning process, TE/GE Division management prepared a June 7, 2004, TEDS Business Case that analyzed the benefits and costs associated with TEDS Release 2.The system functionality to be added with the implementation of this Release includes the ability to image additional documents (i.e., other than those received as part of a new Form 5307 application package) and either add those images to an existing TEDS case file or create a new electronic file for images not related to an existing TEDS case.TEDS Release 2 will also provide the ability to scan and image Exempt Organizations closed case files.We did not perform an assessment of the processes used to plan or develop system functionality for Release 2 in this audit.However, based on the failure of TEDS Release 1 to deliver many of the necessary projected benefits, TE/GE Division management should closely review the expected benefits and costs associated with TEDS Release 2 and update the June 7, 2004, TEDS Business Case as appropriate.

Recommendations

Recommendation 1:The Director, Business Systems Planning, should ensure tasks needed to achieve the system functionality for future releases of the TEDS are clearly established to provide greater assurance that expected benefits will be realized.This includes ensuring:

Managementís Response: The Commissioner, TE/GE Division, reported that each element of Recommendation 1 has been completed. Specifically:

1)      During development of the requirements for TEDS Release 2, the criteria for automating case closure processes were thoroughly reviewed by the TEDS user group and were determined to be as complete as possible for the available data from the form.

2)      Criteria that govern the automated case closure and automated letter generation systems in TEDS Release 2 are significantly more flexible than those in Release 1.The custom-coded business rules engine used in Release 1 has been replaced by a commercial off-the-shelf business rules engine that enables authorized users to modify specific criteria for auto-closure.This commercial off-the-shelf product also enables a programmer to easily create new business rules when needed.The custom-coded letter generation system in TEDS Release 1 has been replaced in TEDS Release 2 by another commercial off-the-shelf-based system that enables authorized users to easily add or modify standard letter paragraphs without reprogramming.

3)      TEDS Release 2 has been divided into distinct phases, and each phase is allocated an amount of time in the project schedule for development, testing, and pilot.The deployment will also be accomplished in a phased approach starting in April 2007 and concluding in October 2007.The project schedule contains time for each phase of the life cycle and is managed by weekly scheduled reviews conducted by the integrated project team.

4)      To ensure effective processes are in place to identify all requirements, a User Group with a wide background of TE/GE Division expertise has been established for TEDS Release 2.The User Group of front-line employees and managers, with additional participants providing input on an as-needed basis, is in place.To control the process, the TEDS 2 Integrated Project Team has developed and implemented a Requirements Management Plan, which includes a Change Control Board imposing configuration management via change requests.

Recommendation 2: The Director, Exempt Organizations, should implement processes that ensure the TEDS Business Case is revised when the changes to the projectís scope occur, increases in the projectís target completion date are extended by more than 10 percent, or project costs are increased by more than 10 percent, so the most current information is available to make informed investment decisions, monitor the progress of the Systemís development, and evaluate whether business benefits are realized.

Managementís Response: The Commissioner, TE/GE Division, reported that all corrective actions for Recommendation 2 have been completed.The Director, Exempt Organizations, ensures TEDS Release 2 follows the IRS-wide Enterprise Life Cycle and Enterprise Life Cycle-Lite methodology and governance processes to ensure the most current information is available to the TE/GE Investment Executive Steering Committee, so it can make informed investment decisions and monitor the progress of the Systemís development and evaluate whether business benefits are realized.As part of this oversight, in the last year, the TEDS Release 2 project team has briefed the Director, Exempt Organizations, and the Commissioner, TE/GE Division, on a regular basis in advance of the monthly review by the TE/GE Investment Executive Steering Committee.At both briefings, project activities; accomplishments; risks and issues; and any significant cost, schedule, and scope issues that require TE/GE Investment Executive Steering Committee review and decision are discussed and evaluated.

Recommendation 3:The Director, Exempt Organizations, should ensure investment decisions are fully evaluated by adopting a business case model that includes processes for tracking actual costs of processing the determination applications to compare with the estimated costs.

Managementís Response: The Commissioner, TE/GE Division, reported that all corrective actions for Recommendation 3 have been completed.On a weekly basis, actual costs are tracked and compared against projected estimates for the same time period.The aggregate weekly actual costs are subsequently verified with the monthly invoices.This tracking mechanism is being used for all three TEDS Release 2 vendor/contractor teams; it ensures the most accurate financial information is available for review and management more timely than in TEDS Release 1 and can be effectively incorporated in any required updates to the TEDS business case.

 

Appendix I

 

Detailed Objective, Scope, and Methodology

 

The overall objective of this audit was to determine whether implementation of TEDS Release 1 achieved its purpose of improving the TE/GE Divisionís processing of Form 5307 and providing electronic EP determination information to other TE/GE Division programs.To achieve this objective, we:

I.                   Determined what system capabilities were scheduled to be delivered as a result of the implementation of TEDS Release 1.

A.    Reviewed the:

1.      September 9, 2002, and August 12, 2003, Statements of Work and determined the system requirements and deliverables for the design and creation of TEDS Release 1.

2.      November 14, 2001, TEDS Concept of Operations and determined the conceptual framework of how the System should facilitate the processing of determination applications.

3.      June 12, 2002, Risk Management Plan[19] used to determine the approach, organization, and procedures TE/GE Division management used to detail the processes for identifying, assessing, and controlling risk to the TEDS project.

4.      Results of the May 8, 2003, System Acceptability Testing[20] and determined what system requirements were tested, the problems identified during the System Acceptability Testing, and the resolution and nonresolution of problems.

5.      August 7, 2003, TEDS Business Case and determined what specific performance measures were expected to be delivered.

6.      May 2004 TEDS Configuration and Change Management Handbook[21] and determined the processes and procedures used during development of the System to establish and ensure the integrity of TEDS products are maintained throughout the projectís life cycle.

B.     Interviewed TEDS Release 1 team project members and determined what system capabilities were scheduled to be delivered with implementation of TEDS Release 1.

C.     Interviewed TE/GE CAS, EP Examination, and EP Determinations Quality Assurance Staff management and determined how TEDS Release 1 was to benefit their operations.

II.                Determined how TEDS Release 1 system functionality is currently operating by assessing the Systemís capability compared to the planned functionality in the August 27, 2003, TEDS Business Case.

III.             Determined whether TEDS Release 1 is functioning as intended and assessed the impact any differences have had on the TE/GE Divisionís programs.

A.Compared the expected TEDS Release 1 system capabilities scheduled to be delivered to the actual delivered TEDS Release 1 functionality.

B.     Reviewed the August 7, 2003, TEDS Business Case and determined what impact the nondelivery of any TEDS Release 1 functionality has had on the TE/GE Division.

C.     Determined what impact the implementation of TEDS Release 1 has had on other TE/GE Division programs by interviewing EP Examination, TE/GE CAS, EP Determinations Quality Assurance, and TE/GE Division Business Systems Planning management.

 

Appendix II

 

Major Contributors to This Report

 

Daniel R. Devlin, Assistant Inspector General for Audit (Headquarters Operations and Exempt Organizations Programs)

Nancy A. Nakamura, Acting Assistant Inspector General for Audit (Headquarters Operations and Exempt Organizations Programs)

Jeffrey M. Jones, Acting Director

James V. Westcott, Audit Manager

John W. Baxter, Lead Auditor

Laura S. Cooper, Auditor

Donald J. Martineau, Auditor

Marjorie A. Stephenson, Auditor

 

Appendix III

 

Report Distribution List

 

CommissionerC

Office of the Commissioner Ė Attn:Chief of StaffC

Deputy Commissioner for Services and EnforcementSE

Deputy Commissioner, Tax Exempt and Government Entities DivisionSE:T

Director, Business Systems Planning, Tax Exempt and Government Entities DivisionSE:T:BSP

Director, Employee Plans, Tax Exempt and Government Entities DivisionSE:T:EP

Director, Exempt Organizations, Tax Exempt and Government Entities DivisionSE:T:EO

Chief CounselCC

National Taxpayer AdvocateTA

Director, Office of Legislative AffairsCL:LA

Director, Office of Program Evaluation and Risk AnalysisRAS:O

Office of Internal ControlOS:CFO:CPIC:IC

Audit Liaison: Director, Communications and Liaison, Tax Exempt and Government Entities

DivisionSE:T:CL

 

Appendix IV

 

Outcome Measure

 

This appendix presents detailed information on the measurable impact that our recommended corrective actions will have on tax administration.This benefit will be incorporated into our Semiannual Report to Congress.

Type and Value of Outcome Measure:

Reliability of Information Ė Potential; $2.3 million (see page 9).Total project costs were not tracked and reported because TE/GE Division management did not have a process in place to appropriately evaluate investment decisions associated with the development of TEDS Release 1.As a result, information on the $2.3 million cost overrun was not available for TE/GE Division management to use in planning, monitoring, and reporting on their investment.

Methodology Used to Measure the Reported Benefit:

We calculated the total estimated TEDS Release 1 project costs through FY 2006 (as of February 17, 2006) by using the information from the August 7, 2003, TEDS Business Case.The $14.7 million estimate includes $13,451,068 to implement TEDS Release 1 and $1,200,055 in FYs 2004 through 2006 for contractor labor and other contractor-related costs.

The $17 million cost to implement and maintain TEDS Release 1 includes $12,028,694 spent to implement the System prior to issuance of the August 7, 2003, TEDS Business Case.The $17 million also includes $475,053 needed by the TE/GE Division to cover labor and other related costs to complete implementation of TEDS Release 1 and $4,488,405 in contractor costs between August 7, 2003, and February 17, 2006.The total cost overrun was determined by subtracting the estimated cost from the actual cost.

1.  Actual cost to implement and maintain TEDS Release 1††
as of February 17, 2006 (see page 11).†††††† ††††††††††††††††††††††††††††††††††††††††††††††† $17.0 million

2.  Estimated TEDS Release 1 project cost through FY 2006[22]
(see page 11).††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††† $14.7 million

Total††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††† $2.3 million

 

Appendix V

 

Managementís Response to the Draft Report

 

The response was removed due to its size.To see the response, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.



[1] Determination letters provide customers assurance that their employee plan or exempt organization is in compliance with applicable tax laws.A favorable determination letter may no longer apply if there is a change in a statute, regulation, or revenue ruling applicable to pension plans.When this occurs, the plan must be amended to comply with the new requirements.The Internal Revenue Service allows plan sponsors time, called the remedial amendment period, to incorporate tax law changes without a penalty or sanction.The remedial amendment period ended on January 31, 2004.

[2] The start date of TEDS Release 1 was determined to be the date the Executive Steering Committee approved the multiple release strategy for the TEDS.

[3] Determination letters provide customers assurance that their employee plan or exempt organization is in compliance with applicable tax laws.

[4] Cycle time is the period from when an EP case is postmarked to the closing of the application and issuance of the determination letter.

[5] The case grading system capability recommends the grade level of the employee assigned to work the case (i.e., General Service 9, 11, 12, or 13).The classification system capability shows the type of case to be worked (i.e., Automated, Merit, Non-Merit, and Washington Office).

[6] The Business Concept of Operations document provides the vision for how the TEDS will facilitate the processing of determinations within the TE/GE Division.

[7] A committee with the authority to approve a project request and determine whether to continue the project.The committee includes executive-level members and advisors.Executive-level members, who have voting privileges, are from each part of the organization that has a stake in the success of their initiative.Advisors are from an organization such as Business Systems Planning.

[8] The business rules are requirements that govern which cases meet the favorable determination criteria.

[9] A favorable determination letter may no longer apply if there is a change in a statute, regulation, or revenue ruling applicable to pension plans.When this occurs, the plan must be amended to comply with the new requirements.The IRS allows plan sponsors time, called the remedial amendment period, to incorporate tax law changes without a penalty or sanction.The remedial amendment period ended on January 31, 2004.

[10] Project Management Techniques Need to Be Followed to Effectively Develop the Tax Exempt Determination System (Reference Number 2003-10-103, dated May 2003).

[11] Project management techniques are used to ensure the development of computer systems is efficient and effective.

[12] Risk management is a continual process that seeks to proactively manage uncertainties to reduce or avoid adverse project impacts.

[13] A business case serves as the document that summarizes numerous technical and business work products, analyses, and studies that provide the basis for making investment funding decisions and for monitoring and evaluating investment performance.

[14] Submission Processing sites process paper and electronic submissions for analysis and posting to taxpayer accounts.

[15] The estimated labor processing savings are presented in constant dollars.Constant dollars are reported in terms of the value they had on a previous date (e.g., the dividend of $5.00 per share paid in 1986 was worth only $2.50 in constant dollars of 1976, when the stock was purchased).

[16] Balancing Reports are inventory reports, one report each from the two systems, used to reconcile the inventory of determination letters between the EDS and the TEDS.

[17] Clinger-Cohen Act of 1996 (Federal Acquisition Reform Act of 1996) (Information Technology Management Reform Act of 1996), Pub. L. No. 104-106, 110 Stat. 642 (codified in scattered sections of 5 U.S.C., 5 U.S.C. app., 10 U.S.C., 15 U.S.C., 16 U.S.C., 18 U.S.C., 22 U.S.C., 28 U.S.C., 29 U.S.C., 31 U.S.C., 38 U.S.C., 40 U.S.C., 41 U.S.C., 42 U.S.C., 44 U.S.C., 49 U.S.C., 50 U.S.C.).

[18] IRS policy generally requires a business case to be updated after successful completion of integration, testing, acceptance, and piloting (testing the system in an actual business environment).

[19] The Risk Management Plan is used to show the TE/GE Division plans to identify, analyze, mitigate, and control the risks for the TEDS project schedule, costs, and/or technical performance.

[20] This is the process of testing a system or program to ensure it meets the original objectives outlined by the user in the requirement analysis document.

[21] The Configuration and Change Management Handbook is used to identify, control, and approve changes to system documentation, custom computer source code, or off-the-shelf software.

[22] The estimated TEDS Release 1 project cost is the amount of money in the August 7, 2003, TEDS Business Case that the TE/GE Division expected to spend to deploy and maintain TEDS Release 1 through FY 2006.