TREASURY INSPECTOR GENERAL FOR TAX ADMINISTRATION

 

 

The Electronic Fraud Detection System Redesign Failure Resulted in Fraudulent Returns and Refunds Not Being Identified

 

 

 

August 9, 2006

 

Reference Number:  2006-20-108

 

This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.

Redaction Legend:

1 = Tax Return/Return Information

3(a) = Identifying Information - Name of an Individual or Individuals
3(d) = Identifying Information - Other Identifying Information of an Individual or Individuals

 

Phone Number   | 202-927-7037

Email Address   |  Bonnie.Heald@tigta.treas.gov

Web Site           | http://www.tigta.gov

 

August 9, 2006

 

 

MEMORANDUM FOR ACTING CHIEF INFORMATION OFFICER

 

FROM:                            Michael R. Phillips /s/ Michael R. Phillips

                                         Deputy Inspector General for Audit

 

SUBJECT:                    Final Audit Report – The Electronic Fraud Detection System Redesign Failure Resulted in Fraudulent Returns and Refunds Not Being Identified (Audit # 200620009)

 

This report presents the results of our review to determine whether the Internal Revenue Service (IRS) effectively managed annual programming changes and requested modifications to the Electronic Fraud Detection System (EFDS) prior to Processing Year[1] (PY) 2006.

Synopsis

The EFDS is the primary information system used to support the Criminal Investigation (CI) Division’s Questionable Refund Program, a nationwide program established to detect and stop fraudulent claims for refunds on income tax returns.  In PY 2005, the CI Division stopped $412.2 million in fraudulent refunds.  In 2001, a contractor was hired to assist the IRS with EFDS operations, maintenance, and enhancements.  As of April 24, 2006, over $37 million had been paid to the contractor for this work, including $18.5 million for system development efforts.  Two other contractors were paid approximately $2 million for system development work, bringing the total EFDS system development cost to $20.5 million.[2]  The January 31, 2006, Business Case shows the EFDS total costs from August 1994 through September 2005 were $185.9 million.

In 2002, the IRS initiated an effort to redesign the EFDS to improve system performance, reliability, and availability.  The redesigned EFDS web-based application[3] (Web EFDS) was to be implemented in January 2005.  Due to system development problems, the implementation date was delayed until January 2006.  However, the implementation date was not met.  On April 19, 2006, all system development activities for the Web EFDS were stopped, and all efforts were focused on restoring the old EFDS for use in January 2007.  Therefore, the IRS has been and will be unable to use the EFDS to prevent fraudulent refunds during PY 2006.  The IRS reported that, due to other leads,[4] $93.9 million[5] in fraudulent refunds had been stopped as of May 19, 2006, without the EFDS being operational.

The lack of adequate executive oversight and monitoring of the Web EFDS project contributed to the EFDS not being implemented for PY 2006.  From June 2002 until July 2003, an EFDS Executive Steering Committee held periodic meetings to review the project’s activities.  After July 2003, executive oversight for the project was provided by Business Systems Development (BSD) office executives who also have responsibility for managing the maintenance and development work for over 325 IRS systems.  The IRS is considering expanding the Senior Management Dashboard Review of projects to include nonmodernization projects such as the EFDS.  The IRS also established an Enterprise Services office in the Modernization and Information Technology Services organization to consolidate common enterprise programs.

In the Business Case required by the Office of Management and Budget, the EFDS is presented as a Steady State project although the Business Case describes ongoing operations and maintenance activities, development of the web-based application, and redesign of the database.  The Business Case also contains several conflicting statements describing the status of the project and the related system development efforts.  Based on IRS guidelines, the EFDS should be categorized as a Development/Modernization/Enhancement project and be governed by an Executive Steering Committee that includes executives from outside the BSD organization. Although the Business Case was reviewed by the IRS, the Department of the Treasury, and the Office of Management and Budget, none of the reviewers questioned the categorization of the project or the conflicting statements.

Because the EFDS was classified as a Steady State project, an initial cost estimate for the Web EFDS development was not prepared. Therefore, management did not monitor for and we cannot determine whether there were cost overruns. However, the Web EFDS was initially scheduled to be implemented in January 2005, and funds continued to be added to the project to pay the contractors for their system development efforts.

Project documentation indicates the Web EFDS project followed the Enterprise Life Cycle-Lite system development methodology and the status of the project was monitored through project team and contractor discussions, status reports, and reviews of the work breakdown structure.  Although numerous indications of potential risks and problems were raised throughout the project, effective corrective actions were not taken.  Also, three key project management documents were not maintained properly or created timely.

During development of the Web EFDS, there were numerous changes in project management and executives responsible for overseeing the project.  Frequent changes in leadership can affect the project continuity and direction and may indicate other problems with the project. There was also excessive turnover of contractor employees working on the project, partially due to mergers of the contractor firms.  The combination of the assignment of new employees who need to become familiar with a project and the loss of highly skilled employees can jeopardize information technology projects and result in less than full performance.

Because the Web EFDS was not implemented as scheduled in 2005 and 2006, and there are no current plans to continue development of the Web EFDS, we estimate the IRS inefficiently used resources totaling $20.5 million from May 25, 2001, to April 24, 2006, for the contractor costs associated with development of the Web EFDS.  An undeterminable amount of internal staffing costs were also incurred in trying to test the new System and monitor the primary contractor’s activities.

We reviewed six work requests written against the EFDS task order and determined they included deliverables written in general terms with no specific due dates.  Because of the contract type used to procure contractor assistance, the contractor continues to be paid for system development work while a critical system used to stop fraudulent tax returns and refunds was not operational during PY 2006.  In addition, the Federal Government’s interest has not been protected, and the contractor cannot be held accountable for not meeting deliverable due dates.

While the contract vehicle used to obtain contractor services is very important, the effectiveness of the Contracting Officer’s Technical Representative (COTR) in monitoring the contractor’s performance is also crucial in assuring successful contract administration.  ****3(d)****

IRS management advised us that, during a meeting between IRS and contractor executives, a Contractor Senior Director commented the old EFDS would be updated and implemented at no cost to the Federal Government because the contractor did not deliver the Web EFDS as scheduled.  However, the contractor submitted charges totaling $459,718 for the additional work to get the old EFDS ready for implementation, and these charges were paid by the IRS.  The EFDS Project Manager advised us the invoices were paid without question because all invoices for the contract type that was used must be paid in full. The contractor had stated only verbally it would not charge the IRS for updating the old EFDS for use in 2005, and the matter was not raised with the contractor.  Therefore, these costs are considered questioned costs because the contractor did not deliver a Web EFDS that worked.

The EFDS Project Executive advised us a decision would be made in the future about whether to continue the Web EFDS development efforts and what contracting approach should be used. The IRS has established a Questionable Refund Program Executive Steering Committee to review and approve changes to the Program, but due dates regarding any Program change decisions have not been determined.

By using cost-reimbursement contracts, ineffectively monitoring contractor performance, and not questioning contractor invoice charges for updating the old EFDS, the IRS did not ensure the Federal Government’s interests were protected and the Web EFDS was implemented timely to identify and stop fraudulent tax returns and refunds.

Recommendations

We recommended the Chief Information Officer ensure the EFDS project is assigned to an Executive Steering Committee for executive oversight; the Business Case and the information technology investment portfolio are revised to categorize the EFDS project properly and include accurate and consistent information; project risks are identified and addressed properly; the proper system development life cycle methodology is implemented for EFDS development; other projects being managed in the new Applications Development organization are assigned to the appropriate oversight process; and high-risk projects, like the EFDS, are included in the Senior Management Dashboard Review process.  We also recommended the Chief Information Officer ensure contractors are held accountable for performance; COTRs are trained adequately and perform their duties properly; discussions are initiated with the Director, Procurement, and the contractor to recover the funds paid to the contractor to restore the old EFDS for PY 2005 and any additional costs resulting from nondelivery of a functional Web EFDS; and additional work on the Web EFDS is deferred until the IRS decides who will perform the EFDS work.

Response

IRS management agreed with all of the recommendations and has begun to implement corrective actions, such as placing the EFDS project under the governance of the Taxpayer Relationship Management Executive Steering Committee on May 31, 2006, and identifying, documenting, and discussing EFDS risks, issues, and mitigation strategies at the appropriate oversight committees.

IRS management also plans to implement several corrective actions, including identifying high-risk projects in the Applications Development organization and assigning them to the appropriate oversight process.  The IRS stated it is revising the Business Cases to categorize the restoration of the old EFDS for 2007 as a Steady State project.  The categorization proposal will be sent to the Department of the Treasury Office of the Chief Information Officer for concurrence and to ensure accuracy and consistency of the information.  In June 2006, the IRS began discussions of tailoring the Enterprise Life Cycle methodology for the project restoring the old EFDS.  In addition, the IRS is negotiating the contract for the EFDS restoration, anticipating that it will be a cost-plus-fixed-fee contract with a percentage of the contractor’s fees dependent upon timely delivery of specified milestones.  In Fiscal Year 2007, the appropriate project office management staff will have a commitment to ensure all COTRs are trained adequately and their duties are performed properly to monitor the contractors’ performance effectively.  In May 2006, the IRS initiated discussions with the contractor regarding cost-sharing for the contract to restore the old EFDS for PY 2007.  An IRS-wide initiative for the Questionable Refund Program has started, and a high-level strategy will be completed by December 31, 2006.  Any further development of the Web EFDS will include requirements identified from this initiative, and any new development work will be opened to competition.  Management’s complete response to the draft report is included as Appendix VIII.

Copies of this report are also being sent to the IRS managers affected by the report recommendations.  Please contact me at (202) 622-6510 if you have questions or Margaret E. Begg, Assistant Inspector General for Audit (Information Systems Programs), at (202) 622-8510.

 

 

Table of Contents

 

Background

Results of Review

The Electronic Fraud Detection System Did Not Have Adequate Executive Oversight

Recommendations 1 and 2:

Recommendation 3:

The Electronic Fraud Detection System Risks Were Not Effectively Managed

Recommendations 4 and 5:

Contractor 1’s Performance Was Not Effectively Monitored, and Performance-Based Contracts Were Not Used

Recommendation 6:

Recommendations 7 through 9:

Appendices

Appendix I – Detailed Objective, Scope, and Methodology

Appendix II – Major Contributors to This Report

Appendix III – Report Distribution List

Appendix IV – Outcome Measures

Appendix V – Chronology of the Electronic Fraud Detection System Development Events - January 1, 1995, Through April 19, 2006

Appendix VI – Glossary of Terms

Appendix VII – Electronic Fraud Detection System Management Turnover

Appendix VIII – Management’s Response to the Draft Report

 

 

Background

 

The Modernization and Information Technology Services (MITS) organization is responsible for providing information technology support and services for the Internal Revenue Service (IRS)  by building and maintaining information systems that will help the IRS achieve its mission, objectives, and business vision.  The MITS Strategic Plan for Fiscal Years (FY) 2005 – 2006 supports the IRS’ objective of discouraging and deterring noncompliance with the tax laws by delivering modernized information systems.

The Criminal Investigation (CI) Division’s Questionable Refund Program is a nationwide program established to detect and stop fraudulent claims for refunds on income tax returns.  The Electronic Fraud Detection System (EFDS) is the primary information system used to support the Questionable Refund Program and is currently maintained by the MITS Applications Development organization.[6]  Figure 1 shows the number of fraudulent refund returns and refunds identified and stopped over the last 4 years.

Figure 1:  Fraudulent Refund Returns and Refunds Identified and Stopped*

Processing Year[7]

Total Returns Filed

Refund Returns Filed

Returns Screened by Fraud Detection Centers

False Refund Returns Identified

False Refund Returns Stopped

False Refunds Identified

False Refunds Stopped[8]

2002

130,341,159

104,367,859

1,942,089

81,486

41,358

$450,023,509

$333,541,138

2003

130,134,276

104,904,543

740,216

96,953

73,400

$349,515,144

$266,423,786

2004

130,459,600

106,420,200

463,222

118,075

82,099

$2,241,612,551

$2,110,454,658

2005

133,933,000

100,276,000

511,805

132,945

103,537

$515,548,186

$412,184,202

Source:  The CI Division.

* The EFDS is the primary source for the identification of leads on fraudulently filed tax returns; however, leads are also received from sources internal to and external from the IRS.  The IRS reports that, due to other leads, $93.9 million in fraudulent refunds had been stopped as of May 19, 2006, without the EFDS being operational.  This amount was derived from a manual process provided by the Fraud Detection Centers on a weekly report; it has not been verified and could contain inconsistencies/inaccuracies due to the manual process.

In 2001, a contractor (Contractor 1) was hired to assist the IRS with EFDS operations, maintenance, and enhancements.  As of April 24, 2006, over $37 million had been paid to the contractor for this work, including $18.5 million for new system development efforts.  In 2002, the IRS initiated an effort to redesign the EFDS to improve system performance, reliability, and availability; the initial plan shows the redesigned EFDS web-based application (Web EFDS) was to be implemented in January 2005. However, due to system development problems, the implementation date was delayed until January 2006.

The IRS also hired two additional contractors to assist Contractor 1 with development of the Web EFDS.  Contracts 1 and 2 are cost-reimbursement contracts.  Contract 3 is a time-and-materials contract.  The responsibilities of Contractors 2 and 3 included the following:

·         Contractor 2 is responsible for determining the effectiveness of and maintaining data-mining techniques and is paid its costs plus a fixed fee.  As of April 24, 2006, this contractor had been paid $1,005,546 for work on the Web EFDS.

·         Contractor 3 is responsible for consulting services on database-related issues and is paid based on the number of hours of consulting services provided.  As of April 24, 2006, this contractor had been paid $1,002,859 for work on the Web EFDS.

Therefore, the total development cost for the Web EFDS as of April 24, 2006, was $20.5 million.[9]  Our review primarily focused on Contractor 1 because it was ultimately responsible for delivering a fully operational Web EFDS.

The IRS has been and will be unable to use the EFDS during PY 2006 to prevent fraudulent refunds, after spending $20.5 million on new system development efforts.

On February 21, 2006, the IRS Deputy Commissioner for Services and Enforcement advised us the EFDS would probably not be available during Processing Year (PY) 2006, due to continued system development problems, and a recovery program to identify fraudulently issued refunds would not be run.  The IRS established a Questionable Refund Program Executive Steering Committee to review and approve changes to the Program, but the due dates regarding any Questionable Refund Program change decisions have not been determined.  On April 19, 2006, all system development activities for the Web EFDS were stopped, and all efforts were focused on restoring the old EFDS for use in January 2007. Therefore, the IRS has been and will be unable to use the EFDS to prevent fraudulent refunds during PY 2006.  A complete chronology of EFDS activities is included in Appendix V.

We initiated this audit at the request of the House Ways and Means Subcommittee on Oversight.  The Subcommittee was interested in:

  • What were the EFDS problems and have they been fixed?
  • How much money did the Federal Government lose as a result of the EFDS not working properly and allowing the issuance of fraudulent refunds?
  • Does the IRS plan to review those refunds that were not evaluated by the EFDS because the System was not implemented?

This audit report will address the first question.  The remaining two questions will be addressed by an audit conducted by the Treasury Inspector General for Tax Administration (TIGTA) Headquarters Operations and Exempt Organizations Programs business unit (Audit Number 200610003), which will determine the effectiveness of the IRS’ procedures for detecting fraudulent and potentially fraudulent refund returns (including inventory controls) and the timely and proper hold and release of refunds. The TIGTA Information Systems Programs business unit has initiated a separate audit (Audit Number 200620040) to assess the EFDS application and infrastructure security certification and accreditation process.

This review was performed at the MITS organization offices in Memphis, Tennessee; New Carrollton, Maryland; and Washington, D.C., during the period February through May 2006. The audit was conducted in accordance with Government Auditing Standards.  Our audit work was delayed because IRS personnel deferred responses to our requests for interviews, documentation, and project status briefings citing other priorities with the Web EFDS development activities.  In addition, due to the lack of documentation of key meetings and decisions, much of the information in the report is based on emails provided to us and on the recollections of the employees we interviewed. Detailed information on our audit objective, scope, and methodology is presented in Appendix I. Major contributors to the report are listed in Appendix II.

 

 

Results of Review

 

The Electronic Fraud Detection System Did Not Have Adequate Executive Oversight

The Clinger-Cohen Act of 1996[10] requires agencies to use a disciplined Capital Planning and Investment Control process to acquire, use, maintain, and dispose of information technology assets.  Office of Management and Budget (OMB) Circular A-11, Preparation, Execution, and Submission of the Budget, dated June 2005, requires each agency to include with its annual budget submission to the OMB an information technology investment portfolio, commonly referred to as an Exhibit 53, containing the information technology investment title, description, amount, and funding source.  Twice each year, the OMB requires agencies to submit an OMB Circular A-11 Exhibit 300, Capital Asset Plan and Business Case, for each major information technology investment.  IRS guidelines require the Business Case to be updated quarterly.  These documents are essentially Business Cases used by agencies to request funds, monitor the progress of projects, and improve management decision making over expensive information technology investments.  Depending on the type of activities occurring in a project, the project is categorized as a Development/Modernization/Enhancement or Steady State investment.[11]

Until November 2004, IRS information technology projects were classified for investment decision purposes as either Tier A, B, or C, as defined below.

·         Tier A – Technical Modernization Projects – Resources devoted to Information Technology Investment Account-funded projects and managed by the IRS Business Systems Modernization Office.  Project scale is large, with a 2-year to 3-year time period.

·         Tier B – Improvement Projects – Resources devoted to new improvements of medium size (1-year to 2-year time period) and funded from the improvement programs budget.

·         Tier C – Enhancements/Stay-In-Business Projects – Resources devoted to all other types of projects (e.g., sustaining operations, legislative changes, and small enhancements to sustaining operations) funded from the regular sustaining operations budget.

In November 2004, the IRS changed the project classification process to be more consistent with OMB Circular A-11 guidance that categorizes investments as either Development/ Modernization/Enhancement or Steady State.

The IRS’ Capital Planning and Investment Control process for managing information technology projects established an executive governance process for monitoring projects.  The process included the MITS Enterprise Governance Committee, MITS Enterprise Governance Committee Investment Management Subcommittee, and Executive Steering Committees responsible for specific projects.  Major projects with costs of over $5 million per year or more than $50 million in total life cycle costs are to be governed by the executive governance process.  Formal agendas, presentations, and meeting minutes (including documentation of key decisions and assignments) should be prepared for each Executive Steering Committee meeting.

The lack of continuous Executive Steering Committee oversight and inadequate documentation of key decisions increases the risks that responsible parties are not held accountable and actions are not implemented

The EFDS project was initially considered a Tier C project; it was later reclassified as a Steady State project.  From June 2002 until July 2003, an EFDS Executive Steering Committee held periodic meetings to review the project’s activities. However, the EFDS Executive Steering Committee stopped meeting at about the time the IRS decided to redesign the EFDS into the Web EFDS.  After July 2003, executive oversight for the project was provided by the Business Systems Development (BSD) organization executives who also have responsibility for managing the maintenance and development work for over 325 IRS systems.

The EFDS project was incorrectly categorized as a Steady State initiative after completion of a reengineering study and the awarding of a task order for software maintenance and development totaling $39 million.  The lack of an executive governance process contributed to the inefficient use of system development funds and fraudulent refunds not being stopped.

As of April 30, 2006, the Web EFDS project did not have an Executive Steering Committee to provide executive oversight as required by the Capital Planning and Investment Control process, although we were advised by the EFDS Project Executive that the IRS is going to reestablish an EFDS Executive Steering Committee. 
On April 24, 2006, the Applications Development organization briefed us on the status of a review of the Business Systems Modernization program conducted by a contractor.  One of the processes considered to be working effectively is the Senior Management Dashboard Review of projects, and the IRS is considering expanding this review process to include nonmodernization projects such as the EFDS.
  The IRS also established an Enterprise Services office in the MITS organization to consolidate common enterprise programs.

While executive oversight of the project was lacking, there was also very limited documentation (mostly in the form of emails) of the discussions and decisions made when the IRS determined Contractor 1 could not deliver the Web EFDS in 2005.  For example, BSD organization management stated that Contractor 1 agreed not to charge the IRS for the cost of updating and implementing the old EFDS, which had been shut down in early December 2005. However, no one could provide documentation showing when the agreement was made and who was present.  If this situation had been documented and elevated to the appropriate executives, action could have been taken to ensure the contractor was held to the agreement.  If the IRS determined the agreement was not legally binding, BSD organization management, at a minimum, would have known to expect the charges.

It is important to document key events and decisions so employee and contractor accountability is established and follow-up can be done to ensure actions are completed as expected. In addition, when new managers and staff are assigned to the project, documentation of key events and decisions will help them understand the history of the project, lessons learned from prior activities, and responsibilities of contractors and employees.  By expanding the Senior Management Dashboard Review and establishing the Enterprise Services office, the IRS hopes to ensure the Web EFDS problems do not happen again.

The management review process over the information technology investment portfolio and Business Case did not resolve conflicting statements regarding the EFDS status

The IRS prepared an information technology investment portfolio and Business Case for the EFDS project.  The information technology investment portfolio for FYs 2005 and 2006 shows the project cost $12.6 million, with $12 million (95 percent) assigned to the Steady State category and $0.6 million (5 percent) assigned to the Development/Modernization/Enhancement category.  However, significantly more funds have been spent on system development activities.  The most current Business Case, dated January 31, 2006, shows the EFDS total costs from August 1994 through September 2005 were $185.9 million.  The EFDS is presented as a Steady State project, although the Business Case describes ongoing operations and maintenance activities, development of the web-based application, and redesign of the database.  The Business Case also contains several conflicting statements[12] describing the project and the system development status:

·         Part I.A. (Investment Description) has the following conflicting project descriptions:

o       The EFDS is a mission critical, web-based automated system designed to maximize fraud detection at the time that tax returns are filed to eliminate the issuing of questionable refunds.

o       The EFDS is currently a client server-based application used by authorized Criminal Investigation Division employees.

o       The EFDS will migrate to a web-based system for PY 2006.

o       The EFDS is a Tier C operational system/maintenance project in the IRS As-built Architecture.  Tier C projects are maintenance projects for existing systems that are critical to sustaining operations, including making the changes required by new tax laws or system improvements that do not significantly change functionality, but improve the process.

·         Part I.E. (Alternative Analysis) and Part I.G. (Acquisition Strategy for existing contract(s) system development), respectively, have the following conflicting statements about the EFDS development status:

o       The EFDS is a Steady State system that is not currently scheduled to be replaced.  The most recent operational analysis of the EFDS was completed on April 15, 2005.  Based on our customer (Criminal Investigation Division) survey . . . there is no indication that the EFDS requires major enhancement or replacement at this time.

o       The current task orders enable the contractors to research and provide solutions for the EFDS, including researching best practices; and defining, developing, and implementing a reengineered EFDS application and database.  Specific tasks include database architecture design, production support, startup/migration activities, loads and application reengineering in the business data model.

Based on information in the Business Case and the cost of the system, the EFDS should have been placed in two different categories. The original System should have been shown in the Steady State category, and the new Web EFDS development project should have been shown in the Development/Modernization/Enhancement category. Therefore, based on IRS guidelines, the EFDS should be categorized as a Development/Modernization/Enhancement project and be governed by an Executive Steering Committee that includes executives from outside the BSD organization.

The inaccuracy of IRS Business Cases was also the subject of a prior TIGTA audit report.[13] In responding to the audit report, the IRS agreed to take several actions to improve the accuracy of the Business Cases, including designating project managers as the individuals accountable for all data contained in their operational and developmental Business Cases and providing training and guidance documents as appropriate.  However, management’s actions did not result in the EFDS Business Case containing accurate and consistent information.

Although the Business Case was also reviewed by the IRS, the Department of the Treasury, and the OMB, none of the reviewers questioned the categorization of the project or the conflicting statements.

The lack of adequate executive oversight and monitoring of the Web EFDS project contributed to the EFDS not being implemented for PY 2006 and the System not being used to identify and stop fraudulent tax returns and refunds.  During PY 2005, $412.2 million in fraudulent refunds were stopped.  The IRS reported that, due to other leads, $93.9 million in fraudulent refunds had been stopped as of May 19, 2006, without the EFDS being operational.  Based on the value of fraudulent refunds stopped in PYs 2005 and 2006 (through May 19, 2006), we estimate approximately $318.3 million in fraudulent refunds may have been issued in 2006, resulting in lost revenue to the Federal Government (see Appendix IV).

Recommendations

Recommendation 1: The Chief Information Officer (CIO) should ensure the EFDS project is assigned to an Executive Steering Committee for executive oversight, including documenting key decisions and assignments.

Management’s Response:  IRS management agreed with this recommendation.  The EFDS was officially placed under the governance of the Taxpayer Relationship Management Executive Steering Committee on May 31, 2006.  The Executive Steering Committee process and procedures include preparation of meeting agendas, minutes, and other appropriate project documentation of key decisions and assignments.

Recommendation 2: The CIO should evaluate other projects being managed in the new Applications Development organization and ensure all are assigned to the appropriate oversight process.  High-risk projects, like the EFDS, should also be included in the Senior Management Dashboard Review process.

Management’s Response:  IRS management agreed with this recommendation.  The Associate CIOs for Applications Development and Enterprise Services have begun initial evaluations of other projects being managed in the new Applications Development organization.  In April 2006, Applications Development projects identified as high risk/high impact were assigned to either the Senior Management Dashboard Review or the Project Health Assessment Review process and to the appropriate governance structure (e.g., Executive Steering Committee) for oversight. The first phase of evaluation was to discuss and apply management judgment to ensure high-risk projects identified to date were assigned to the appropriate oversight process.  All projects will be asked to complete the Health Check Questionnaire.  Responses to the Questionnaire will be assessed and corrective actions taken as needed on at-risk projects.

Recommendation 3: The CIO should ensure the Business Case and the information technology investment portfolio are revised to categorize the EFDS project properly and include accurate and consistent information.

Management’s Response:  IRS management agreed with this recommendation.  Work on the Web portal EFDS has stopped, and no determination has been made concerning implementation of the System.  The old EFDS application is being updated and is planned to be in production in January 2007. Accordingly, the EFDS Business Cases are being revised to categorize the restoration of the old EFDS as a Steady State investment.  The categorization proposal will be sent to the Department of the Treasury Office of the CIO for concurrence and to ensure accuracy and consistency of the information.

The Electronic Fraud Detection System Risks Were Not Effectively Managed

Department of the Treasury Publication 84-01[14] states that general standardization of life cycle management ensures systems are developed, acquired, evaluated, and operated efficiently, within prescribed budget and schedule constraints, and are responsive to mission requirements.  A software development life cycle methodology provides a structured and consistent approach to information technology project development.  In 2001, 2003, and 2004, MITS organization management issued memoranda directing the Enterprise Life Cycle-Lite (ELC-Lite) be used as the required system development methodology for all nonmodernization projects.  The ELC-Lite addresses the life cycle of project management and the phases and milestones of development, review, and approval of project plans.

The Department of the Treasury Information Technology Manual and the IRS system development guidelines stipulate that, as part of the information system life cycle management process, project management should identify project risks early and manage them before they become problems.  The risk management process encompasses the identification of risk issues, assessment of risk to define probability and impact, preparation and implementation of risk mitigation and risk contingency plans, and continuous monitoring of those actions to ensure effectiveness.  Risk management is used to ensure critical areas of uncertainty are surfaced early enough to be addressed without adversely affecting cost, schedule, or performance.

The BSD organization’s EFDS Project Office has responsibility for analyzing and coordinating new or changed requirements with the business operating divisions and contractors supporting the EFDS.  The EFDS Project Office is also responsible for developing, maintaining, and enhancing computer programs that support the business requirements and improve the IRS’ efficiency in detecting potentially fraudulent returns.

The IRS issued a task order to Contractor 1 for the period June 2000 through May 2006 authorizing approximately $39.5 million for EFDS software maintenance and development.  As of April 2006, the IRS had paid Contractor 1 over $37 million, approximately $18.5 million[15] of which was for Web EFDS development.  The Web EFDS was initially scheduled for implementation in January 2005.  However, in October 2004, CI Division and BSD organization management had doubts the System would be ready.  On November 1, 2004, IRS executives decided to use the old EFDS for PY 2005.  During 2005, the contractor continued to promise the Web EFDS would be delivered in January 2006, but it was not operational for PY 2006 because the contractor, again, did not deliver a System that worked.  For example, we were provided an email from the contractor dated October 21, 2005, that stated “[Contractor 1] remains confident in our ability to deliver the EFDS web portal application for use by the IRS Criminal Investigation Division on January 13, 2006.”  Because the IRS believed the contractor’s assurances that the System would be delivered for PY 2006, no contingency plan was developed and no actions were taken to go back to using the old System, similar to what was done for PY 2005.

In an effort to get Web EFDS implemented in 2006, the IRS increased oversight activities in August 2005 by beginning biweekly meetings among the CI Division, the BSD organization, and the contractors.  On October 31, 2005, an EFDS Readiness Executive Briefing was held with the Product Assurance organization and other MITS organizations involved with implementation of the Web EFDS.  In late November or early December 2005, the IRS began holding daily executive meetings.  In December 2005, the IRS again increased oversight activities by holding more frequent technical team meetings with the contractors.  In February 2006, the IRS established a ‘War Room’ and command center to monitor Web EFDS development activities that continued until early April 2006, in hopes of getting the System to work.

The EFDS redesign activities were viewed as an enhancement to an existing System instead of as a complex system development effort.  Project documentation indicates the Web EFDS project followed the ELC-Lite system development methodology and the status of the project was monitored through project team and contractor discussions, status reports, and reviews of the work breakdown structure.  Although numerous indications of potential risks and problems were raised throughout the project, effective corrective actions were not taken, as summarized below.

Problems were identified during preparation for PY 2005

During 2004, the EFDS Project Office and contractors were working to convert the old EFDS to a web-based application and to redesign the System’s databases.  However, signs of development difficulties began to surface by the middle of the year.  Specifically:

  • On June 18, 2004, the Contracting Officer’s Technical Representative (COTR) expressed concerns to the contractor’s development team that they had not completed a walk-through of the application with the users. Therefore, the contractor was not sufficiently familiar with the functionality of the current EFDS, changes were made without considering user needs, and requirements were not followed.
  • In July or August 2004, the Project Manager was informed by the System Acceptability Testing (SAT) team leader that the System was completely unusable when the first versions of the applications were delivered for testing.
  • On October 6, 2004, the EFDS Project Manager advised the CI Director, Refund Crimes, of significant problems with the Web portal application.  On October 18, 2004, the SAT Branch Chief advised the CI Director, Refund Crimes, that late delivery of the Web portal applications to the SAT team put the testing approximately 6 weeks behind schedule.

Due to the continuing problems, IRS executives and the contractor agreed in late October 2004 the System modifications would not be ready by January 2005, and a decision was made by CI Division and MITS organization executives to use the old System for PY 2005 while work continued on the new System.

Problems were identified during preparation for PY 2006

Because the 2004 effort failed to deliver a new System in time for PY 2005 and the old System was restored, the EFDS Project Office and contractors continued to work on converting the old EFDS to a web-based application and redesigning the System’s databases during 2005.  However, signs of development difficulties again began to surface early in the year. Specifically:

  • On April 20, 2005, an EFDS Project Office email cited data loads and other problems had been identified with the System.
  • On May 10, 2005, the CI Director, Refund Crimes, requested an independent study of the Web EFDS requirements to provide some assurance that the contractor would be able to deliver what it agreed to.  The Director advised that the study needed to be done in the summer of 2005, not after the System was implemented.  The BSD Director, Filing Systems Division, replied that the time needed to resolve “a few existing glitches” would not permit them to begin a study until mid-2006 at the earliest, and because the MITS organization did not have funds for the study, the CI Division would have to fund it.  The CI Division agreed to fund the study; however, due to staffing availability, the study was not performed.
  • In August 2005, the CI Division received the training database to begin development of training materials and identified numerous problems with the Web EFDS application.
  • The August 26, 2005, CI Division meeting minutes reported significant performance problems with the computer network, numerous application defects, considerable usability issues, incomplete implementation of requirements, and the need for a contingency plan.  The EFDS Project Office did not have a contingency plan.
  • On October 25, 2005, the BSD Director, Filing Systems Division, issued an EFDS Readiness Review to the Deputy Director, Enterprise Operations; Director, Enterprise Networks; Director, Product Assurance; and Acting Deputy Director, End User Equipment and Services, outlining the problems, ramifications of not reducing the risks, and mitigation plans.
  • On October 28, 2005, the Product Assurance organization informed the CI Division that a significant number of problems with the data conversion had been encountered.
  • On November 2, 2005, the BSD Director, Filing Systems Division, and a BSD Information Technology Specialist made a trip to the CI Division training location to observe the System’s problems.
  • On December 22, 2005, the EFDS Project Information Technology Specialist discovered significant inaccuracies in three recent work breakdown structures submitted by Contractor 1.  The contractor reported the execution of the annual data loads to be 10 percent, 40 percent, and 50 percent completed in the work breakdown structures dated November 28, December 15, and December 20, 2005, respectively.  However, the IRS was advised by the contractor’s technical staff that the data loads had never been started.  We were told the IRS advised the contractor of this situation but did not pursue the matter.

The IRS business units also conduct quarterly Business Performance Reviews to assess the status of programs. The CI Division’s review for the fourth quarter of FY 2005, dated September 30, 2005, raised the first indication of potential problems.  The report stated that, due to the scope of the project, it was impossible to keep both the old and new Systems operational.  This situation could negatively affect fraud detection until System deficiencies are corrected and users become accustomed to the new functionality.  However, the report indicated the CI Division and the MITS organization had been coordinating to reduce the risks, and the CI Division had informed external stakeholders of potential implementation problems.  The first quarter FY 2006 review, dated December 31, 2005, stated the System was currently delayed and remained in SAT, meetings between the MITS organization and CI Division had occurred twice a week to discuss progress and potential risk to the program, and training for Fraud Detection Center personnel had been stopped.

The MITS organization Business Performance Reviews did not identify the Web EFDS implementation as a risk until the System missed the January 2006 implementation date.  The first quarter FY 2006 Business Performance Review, dated February 16, 2006, stated the MITS organization failed to implement the EFDS timely for production startup on January 13, 2006.  Mitigation actions for the risk included holding Senior Executive daily conference calls, a technical team (EFDS Project Office, Enterprise Operations organization, and contractor) working daily on work breakdown structure dates and other issues, stopping prisoner refunds, and delivering a fully functional system to the CI Division per the current work breakdown structure of February 3, 2006.

Key management documents were not prepared or properly maintained

Software development life cycle documentation is an integral part of the information system development process.  The life cycle methodology should specify the documentation to be generated during each phase.  All project plans are considered process documentation necessary to communicate status and direction and allow management to verify if appropriate progress is being made during the development process.  The EFDS Project Office used ELC-Lite as the system development life cycle because the Web EFDS Project was classified as a Tier C or Steady State project.  The life cycle requirements include the preparation and maintenance of three key project documents (project management plan, risk management plan, and work breakdown structure/schedule).  The EFDS Project Office had the three key project documents; however, two of them were not maintained properly and one was not created timely.

·         A project management plan was created on May 8, 2001, and last updated on September 30, 2003.

·         A risk management plan was created June 23, 2004, and had not been updated.  The plan should have been updated after the contractor missed the January 2005 implementation date, included the risk that the contractor would again not implement the Web EFDS timely, and described the activities (e.g., increased project management and executive oversight activities) needed to reduce this risk.

·         A work breakdown structure was not prepared in 2004, but one was created in 2005.  However, it did not include the original completion dates of the activities required to implement the Web EFDS.  Therefore, original baseline dates are not available to determine the actual schedule slippages.

The ELC-Lite system development life cycle also requires milestone reviews during the Architecture, Integration, and Operations and Support Phases of the life cycle. However, no milestone reviews were conducted.

If the EFDS had been properly classified as a Tier B or development project, preparation of additional required documents such as a performance management plan, data management plan, transition to support plan, configuration management plan, and contingency plan would have helped manage the project risks.  A more formalized governance structure, including an Executive Steering Committee to help oversee project activities, would also have been required.

A configuration management plan had been developed; it showed a configuration management board composed of EFDS project team members had been established to control changes to the project. The EFDS Project Executive advised us a new configuration control board will be established including higher level management.

Had a contingency plan been prepared after the contractor failed to deliver a System for use in 2005 and after the CI Division expressed concerns about the contractor’s ability to deliver a System timely for 2006, IRS management may have been better prepared to deal with the problems that occurred in 2006.  Although the BSD Director, Filing Systems Division, emailed the CI Director, Refund Crimes, on October 18, 2005, to confirm that the BSD organization did not have a contingency plan for the EFDS, no other actions were taken.  The BSD Chief, Document Input Branch, stated there was no contingency plan because the BSD organization did not have the funds to run both Systems at the same time.  Documentation we were provided shows the issue was not elevated above Division-level management until October 31, 2005, when the BSD Director, Filing Systems Division, emailed the CIO and others about the Web EFDS status.

Problems encountered during the SAT were indications of the poor quality of the work performed by the contractor

The numbers of problem tickets reported by the SAT team and by the CI Division’s testing efforts were other indications the Web EFDS would not be implemented timely.  Our review of the problem tickets recording application problems identified during testing showed over 900 problems were identified and recorded. Figure 2 provides a breakdown of the recorded problem tickets.

Figure 2: Monthly Totals of Problem Tickets

Month/Year

Problem Tickets[16] As of March 15, 2006

July 2005

10

August 2005

26

September 2005

13

October 2005

21

November 2005

76

December 2005

192

January 2006

534

February 2006

66

March 2006

33

Total

971

Source:  TIGTA analysis of EFDS problem tickets.

The SAT team encountered numerous problems while testing the Web EFDS during July 2005 through April 2006.  The test results were reported in a test status report dated May 11, 2006.  We interviewed the SAT team and reviewed the test status report. Problems encountered included:

·         Documentation of the database characteristics was never delivered, so the SAT team could not verify the accuracy of the tables and columns (data type and format).

·         Corrections to identified problems were not effective and at times created new problems.

·         Daily data loads failed to finish populating the databases; as a result, testing of the Web portal application could not be completed until the daily loads were finished.  Some of the problems with the daily loads remain unresolved.

SAT management stated they would like to have seen a better quality product delivered by the contractor.  The number of problems identified (over 900) and the number of software fixes received were indications of an unsatisfactory product. The lack of coordination among the contractor’s development teams was evident. For example, when one development team made a change, it did not notify other development teams whose portions of the System were affected by the change.  As a result, the SAT team would encounter another problem with the program.  This would cause the SAT team to write another problem ticket, which would go back to the contractor, requiring another software fix.

Cost and schedule variances were not effectively monitored

Because the Web EFDS was classified as a Steady State project, an initial cost estimate for the Web EFDS development was not prepared. Therefore, management did not monitor for and we cannot determine whether there were cost overruns.  However, funds continued to be added to the project to pay the contractors for their system development efforts.

Because a baseline work breakdown structure was not established before the Web EFDS development began, we did not perform an analysis of the work breakdown structure.  However, we identified at least one significant inaccuracy in the work breakdown structure.  As discussed previously, on December 22, 2005, an EFDS Project Information Technology Specialist discovered significant inaccuracies in three recent work breakdown structures submitted by Contractor 1.  We were told the IRS advised the contractor of this situation but did not pursue the matter to determine the impact on the schedule.

BSD organization management stated numerous work breakdown structures were prepared in 2005 and 2006 with deliverable due dates changed several times. However, Contractor 1 was not held accountable for meeting the scheduled due dates.  We were also told by IRS management that Contractor 1 generally had an explanation for the delays and reassured the IRS that it would meet the January 13, 2006, implementation date.

Another indication of schedule variance is the change in the implementation date. The original implementation date for the Web EFDS was January 14, 2005; this was later revised to January 13, 2006, after the contractor missed the first due date.  As of April 19, 2006, the IRS had stopped work on the Web EFDS.

Numerous leadership, project team, and contractor staff changes led to inconsistent and inadequate oversight and development activities

During development of the Web EFDS, there were numerous changes in project management and executives responsible for project oversight.  For example, between 2002 and 2005, there were three Associate CIOs, three Division Directors, and two Project Managers (see Appendix VII).  Frequent changes in leadership can affect a project’s continuity and direction and may indicate other problems with the project.  For example, one Project Manager indicated that reassignment from the project was requested due to project problems and the lack of support from upper management.

There was also excessive turnover of contractor employees working on the project, partially due to mergers of the contractor firms.  The combination of the assignment of new employees who need to become familiar with a project and the loss of highly skilled employees can jeopardize information technology projects and result in less than full performance.  Contractor 1’s biweekly status report included a section showing numerous departures and/or staffing changes, as summarized in Figure 3. Positions with vacancies included testers, developers, and database administrators.

Figure 3:  Contractor Turnover Rate

Year

Total Number of Employees

Number of Employees Who Left

Turnover Rate

2004

46

33

72%

2005

42

20

48%

2006 estimate[17]

43

20

47%

Source: The BSD organization.

Because of the IRS’ failure to effectively manage the above risks and problems, an EFDS was not available for 2006, which significantly reduced the IRS’ ability to identify and stop millions of dollars in fraudulent refunds.  As of April 19, 2006, the IRS had stopped all work on the redesigned Web EFDS, and all efforts were focused on restoring the previous EFDS for use in January 2007.  The EFDS Project Executive advised us that a decision would be made in the future about whether to continue the Web EFDS development efforts and what contracting approach should be used.  Because the Web EFDS was not implemented as scheduled in 2005 and 2006 and there are no current plans to continue development of the Web EFDS, we estimate the IRS inefficiently used resources totaling $20.5 million[18] from May 25, 2001, to April 24, 2006, for the contractor costs associated with development of the Web EFDS (see Appendix IV).  An undeterminable amount of internal staffing costs were also incurred in trying to test the new System and monitor the primary contractor’s activities.

Many of the problems experienced by the Web EFDS project are similar to those we have reported previously related to the IRS Business Systems Modernization program.  Since FY 2002, our annual assessments of the Business Systems Modernization program[19] have cited four specific challenges the IRS needs to overcome to deliver a successful modernization effort.  Three of these challenges are related to the issues presented above, indicating a need for the IRS to address these same challenges on projects outside the Business Systems Modernization program. These challenges include the need for the IRS to:

·         Implement planned improvements in key management processes and commit necessary resources to succeed.

·         Manage the increasing complexity and risks of the modernization program.

·         Maintain continuity of strategic direction with experienced leadership.

Recommendations

Recommendation 4:  The CIO should ensure project risks are identified properly and plans are prepared to reduce the risks affecting the successful development of the project.

Management’s Response:  IRS management agreed with this recommendation.  Risks, issues, and mitigation strategies for the EFDS are identified and documented for the Taxpayer Relationship Management Executive Steering Committee and the Senior Management Dashboard Review.  All items will be documented in the Item Tracking Reporting and Control System and discussed at the Senior Management Dashboard Review and Executive Steering Committee meetings.

Recommendation 5:  The CIO should ensure the proper system development life cycle methodology is implemented for the EFDS development, based on the types of changes being made to the System.

Management’s Response:  IRS management agreed with this recommendation.  Work on the Web portal EFDS has stopped.  The old EFDS is being updated and is planned to be in production in January 2007.  Meetings began the week of June 26, 2006, to discuss tailoring the ELC for the old EFDS. Once the tailoring plan has been completed, ELC milestone reviews will occur as scheduled.  The IRS expects the restoration of the old EFDS will be characterized as Steady State.

Contractor 1’s Performance Was Not Effectively Monitored, and Performance-Based Contracts Were Not Used

The Federal Acquisition Regulation (FAR)[20] holds contractors responsible for timely contract performance; however, the Federal Government is also responsible for monitoring contractor performance, as necessary, to protect its interest.  This monitoring should include comparing a contractor’s performance plans, schedules, controls, and processes against the contractor’s actual performance; determining the contractor’s progress; and identifying any factors that may delay performance.  Agencies are also required to develop quality assurance surveillance plans when acquiring services.  The IRS Office of Procurement Policy best practices state that a planned surveillance effort is necessary to measure contractor performance and ensure successful completion of tasks.  In addition, the FAR requires agencies to prepare evaluations of contractor performance for contracts with a value exceeding $1 million.  Interim evaluations should be prepared for contracts with a period of performance, including options, exceeding 1 year.  Agencies are also required to describe requirements in terms of results rather than process; use measurable performance standards; provide for reductions of fees or price (e.g., for work that will not be or was not done); and include performance incentives, where appropriate.

Contracting Officers are responsible for ensuring performance of all necessary actions for effective contracting, ensuring compliance with the terms of the contract, and safeguarding the interests of the Federal Government in its contractual relationships.  The FAR includes a detailed list of contract administration functions required of Contracting Officers, many of which can be delegated to a COTR.  COTR responsibilities for managing Treasury Information Processing Support Services-2 contracts include developing requirements; monitoring contractor performance and schedule; developing, reviewing, inspecting, and accepting deliverables; reviewing invoices; informing the Contracting Officer when the contractor is behind schedule and coordinating corrective action to ensure the contract schedule is met; and reviewing monthly status reports. Specific responsibilities in the appointment letter of the EFDS COTR include monitoring the contractor’s performance, reviewing the vouchers, performing quarterly technical evaluations, and coordinating with the program office actions relating to funding and changes in scope of work.

We reviewed six work requests written against the EFDS task order and determined they included deliverables written in general terms with no specific due dates.  For example, the due dates for the following documents were May 25, 2005, to January 13, 2006:

·         Requirements Traceability Matrix.

·         Quality Assurance Plan, with revisions.

·         Software Specification Requirement version 1.2a.

·         Test Plans for testing activities associated with 2005 and 2006.

Most of the deliverables in subsequent work requests to cover the contractor’s work from February 5, 2006, to April 6, 2006, also did not have specific due dates, and the work requests were not performance- or incentive-based requests.  As of April 19, 2006, the contractor had not delivered a fully functional EFDS.

Because cost-reimbursement contracts were used without performance-based requirements, the contractor continues to be paid for system development work while a critical system used to stop fraudulent tax returns and refunds could not be used during PY 2006.  The Federal Government’s interest has not been protected, and the contractor cannot be held accountable for not meeting deliverable due dates.

While the contract vehicle used to obtain contractor services is very important, the effectiveness of the COTR in monitoring the contractor’s performance is also crucial in assuring successful contract administration. ****3(d)****

The contractor’s inability to deliver the Web EFDS for PY 2006 is a major concern.  However, this issue is magnified by the fact that the System was originally scheduled for implementation in January 2005.  In October 2004, IRS executives became concerned about the status of the Web EFDS development.  CI Division and MITS organization executives decided that, because the Web EFDS could not be implemented in January 2005 as planned, the old EFDS would be implemented.  Therefore, the old EFDS had to be revised by loading data, making software updates, and testing the System so it could be implemented for PY 2005.  IRS management advised us that, during a meeting between IRS and contractor executives, a Contractor Senior Director commented the old EFDS would be updated and implemented at no cost to the Federal Government because the contractor did not deliver the Web EFDS as scheduled.  However, the contractor submitted charges totaling $459,718 for the additional work to get the old EFDS ready for implementation, and these charges were paid by the IRS.  The EFDS Project Manager advised us the invoices were paid without question because all invoices for the contract type that was used must be paid in full.  The contractor had stated only verbally it would not charge the IRS for updating the old EFDS for use in 2005, and the matter was not raised with the contractor. Therefore, these costs are considered questioned costs (see Appendix IV) because the contractor did not deliver a Web EFDS that worked.

The IRS has established a Questionable Refund Program Executive Steering Committee to review and approve changes to the Program.  One of the Committee’s action items is to analyze the pros and cons of moving all or some of the EFDS work from the CI Division to other business units, to allow the CI Division to focus its efforts on identifying refund schemes worthy of criminal investigation.  A due date for the analysis has not been determined.  If some or all of the EFDS work will be performed by the other business units, the current Web EFDS may not meet their needs.  The change would require the EFDS Project Office to gather requirements from all the new users.

Ensuring contractor performance and accountability is not only a concern for the Web EFDS.  As previously mentioned, our annual assessments of the Business Systems Modernization program have cited four specific challenges the IRS needs to overcome to deliver a successful modernization effort.  The fourth challenge, which also applies to the EFDS, is to ensure contractor performance and accountability are effectively managed. The IRS has taken actions to address the issue of risk and cost sharing between the IRS and contractors in the Business Systems Modernization program. For example, due to problems and delays during the Integrated Financial System development, the IRS and the contractor established a task order cost-sharing agreement in which both parties will pay a percentage of the development costs based on a defined period of time. However, such actions were not taken by the EFDS Project Office.

By using cost-reimbursement contracts without performance-based requirements, ineffectively monitoring contractor performance, and not questioning contractor invoice charges for updating the old EFDS, the IRS did not ensure the Federal Government’s interests were protected and the Web EFDS was implemented timely to identify and stop fraudulent tax returns and refunds.

Recommendations

Recommendation 6: The CIO should ensure contractors are accountable for performance by developing performance-based requirements for new EFDS contracts.  The CIO should also consider employing cost-sharing arrangements for future task orders so both the IRS and contractor share the risk of project development cost overruns.

Management’s Response:  IRS management agreed with this recommendation.  The contract for the old EFDS restoration is under negotiation, but it is anticipated that it will be a cost-plus-fixed-fee contract with a percentage of the contractor’s fees dependent upon timely delivery of specified milestones.  Any future contracts for completion of the Web EFDS system will be performance-based contracts.

Recommendation 7: The CIO should ensure COTRs are trained adequately and their duties are performed properly to monitor contractor performance effectively through planned surveillance efforts and independent inspections of contractor work, as described by IRS Office of Procurement Policy best practices.

Management’s Response:  IRS management agreed with this recommendation.  All current COTRs are certified, having passed the COTR Training conducted by the Office of Procurement prior to being appointed.  Effective in FY 2007, the appropriate project office management staff will have a commitment to ensure all COTRs are trained adequately and their duties are performed properly to monitor the contractors’ performance effectively through the use of planned surveillance efforts and independent inspections of contractor work as described by the IRS Office of Procurement Policy best practices.

Recommendation 8: The CIO and the Director, Procurement, should initiate discussions with the contractor to recover the funds paid to the contractor to restore the old EFDS for use in PY 2005 and any additional costs resulting from nondelivery of a functional Web EFDS.

Management’s Response:  IRS management agreed with this recommendation.  The Federal Government was able to define the performance specifications only in general terms and stated a specific level of effort per labor category in the task order.  The IRS directed the performance of the contractor through issuance of work requests.  Because the task order was awarded on a cost-reimbursement basis, the contractor is expected only to fulfill the defined contractual requirements on a best effort basis.  The contractor fulfilled its contractual obligations of this task order and, therefore, there is no contractual leverage available to negotiate any funds recovery on this task order.  The Federal Government is obligated to pay all allowable and allocable charges invoiced by the contractor against this task order.  In May 2006, the IRS initiated discussions with the contractor regarding cost-sharing for the contract to restore the old EFDS for PY 2007.

Recommendation 9:  The CIO should defer additional work on the Web EFDS until the IRS decides who will perform the EFDS work.  If some or all of the work will transfer to other business units, the CIO should ensure their requirements are identified before initiating a contract for further development of the Web EFDS.  The contract should be opened to competition.

Management’s Response:  IRS management agreed with this recommendation.  Work on the Web EFDS was stopped on April 19, 2006.  An IRS-wide initiative for the Questionable Refund Program has started, and a high-level strategy will be completed by December 31, 2006.  Because of this, no determination has been made concerning the resumption of work on the Web EFDS.  Any further development of the Web EFDS will include requirements identified from this initiative, and any new development work will be opened to competition under the Treasury Information Processing Support Services-3 contract.

 

Appendix I

 

Detailed Objective, Scope, and Methodology

 

The overall objective of this review was to determine whether the Internal Revenue Service (IRS) effectively managed annual programming changes and requested modifications to the Electronic Fraud Detection System (EFDS) prior to Processing Year[21] (PY) 2006. Specifically, we:

I.                   Determined whether policies and procedures for requesting, developing, managing, and implementing system modifications were effective to ensure adequate testing and timely implementation.

A.    Reviewed the Modernization and Information Technology Services (MITS) organization Capital Planning and Investment Control policies, procedures, and documents to determine whether the EFDS was included in the MITS organization information technology investment portfolio.  We also interviewed Business Systems Development (BSD) organization management and reviewed Executive Steering Committee briefings and meeting minutes to determine whether an Executive Steering Committee had oversight responsibility for the Web EFDS, problems were elevated timely to senior management, and measures were taken to reduce risks.

B.     Obtained and reviewed policies and procedures for monitoring contractor progress and performance, and obtained the Requests for Information Services and contract/task order information for the EFDS changes to be implemented for PY 2006, to determine whether the Project Office ensured the Web EFDS changes worked as expected and timely met the users’ needs.  We also determined how problems were identified, elevated, and controlled and where the problems occurred.  We determined why the Web EFDS was not delivered timely and the effect on PY 2006.  We obtained and reviewed status reports and project schedules to determine when critical problems initially occurred and when they were elevated.  We obtained and reviewed correspondence prepared by BSD organization management notifying the Contracting Officer’s Technical Representative (COTR) and users about the Web EFDS problems and the resolution status.

C.     Obtained and reviewed policies and procedures for monitoring contractor progress and performance.  We interviewed the COTRs and identified their process for monitoring the contractor to ensure the work was on schedule and met the contract terms and user requirements; obtained and reviewed status reports and minutes of meetings between the COTRs and contractor(s) working on the Web EFDS project; identified the type of the EFDS contract/task order and determined whether payments were withheld for unacceptable performance; determined the amounts budgeted and actually spent for the Web EFDS changes to be implemented for PY 2006; determined whether the IRS required the contractor to perform the work again, in conformity with contract and specified requirements; and determined the amount of additional costs.

II.                Determined whether system development problems were identified timely and risk mitigation procedures were implemented to minimize the impact to the Questionable Refund Program.

A.    Obtained and reviewed system development policies and procedures for problem identification and resolution.

B.     Interviewed BSD organization and Criminal Investigation Division management and staff responsible for the Web EFDS to determine the status of the Web EFDS; whether the System was ever placed into production and operational during PY 2006; how the problems were identified, tracked, and shared with customers and contractor(s); when the problems occurred, the cause, and the resolution status; whether the problems were associated with annual programming changes and/or System redesign work completed prior to PY 2006; whether meetings/briefings were held to discuss the effect from the problems and resolution status; and what contingency plans were developed and whether they were implemented.

C.     Obtained and reviewed reports used by management to monitor the status of the problems.  We obtained and analyzed a download of the Web EFDS problems reported in the IRS problem ticket database to determine the problem ticket category, date the ticket was opened, number of days open, and number of tickets opened in each priority code and ticket category.

D.    Obtained and reviewed the minutes from meetings/briefings held to discuss the problems associated with the Web EFDS and determined who the attendees were, the frequency of the meetings/briefings, and whether the effects from the application problems and associated resolution status were discussed.

E.     Obtained and reviewed correspondence and emails prepared by the BSD organization to elevate the problems to executive management.

III.             Determined whether measures were being taken to correct the Web EFDS problems timely.

A.    Interviewed IRS personnel, attended conference call status briefings, and received voice mail status briefings to determine the actions taken or recommended to implement as quickly as possible a Web EFDS that met the requirements.

B.     Obtained and reviewed minutes of meetings/discussions or other documentation between contractors and BSD organization management regarding the status of Web EFDS problems and required action items/resolutions to determine whether decisions relating to Step III.A. were documented.

IV.             Determined the validity and reliability of data from computer-based systems.  We used computer-based data to determine the amounts spent and planned to be spent on the Web EFDS.  We reviewed documentation supporting the project costs and the information technology investment portfolio and Business Case to assess the completeness and accuracy of the data.  We determined the data were reliable as it related to our audit objectives.

 

Appendix II

 

Major Contributors to This Report

 

Margaret E. Begg, Assistant Inspector General for Audit (Information Systems Programs)

Gary Hinkle, Director

Danny Verneuille, Audit Manager

Tina Wong, Lead Auditor

Mark Carder, Senior Auditor

Paul Mitchell, Senior Auditor

Phung-Son Nguyen, Senior Auditor

Van Warmke, Senior Auditor

Olivia DeBerry, Auditor

Charlene Elliston, Auditor

Perrin Gleaton, Auditor

Kim McManis, Auditor

Linda Screws, Auditor

 

Appendix III

 

Report Distribution List

 

Commissioner  C

Office of the Commissioner – Attn:  Chief of Staff  C

Deputy Commissioner for Operations Support  OS

Deputy Commissioner for Services and Enforcement SE

Chief, Agency-Wide Shared Services  OS:A

Chief, Criminal Investigation  SE:CI

Deputy Chief, Criminal Investigation  SE:CI

Director, Procurement  OS:A:P

Associate Chief Information Officer, Applications Development  OS:CIO:AD

Director, Capital Planning and Investment Control OS:CIO:R

Director, Refund Crimes  SE:CI:RC

Director, Stakeholder Management Division OS:CIO:SM

Deputy Chief Financial Officer, Department of the Treasury

Audit Liaisons:

Deputy Commissioner for Operations Support  OS

Deputy Commissioner for Services and Enforcement  SE

Chief, Agency-Wide Shared Services OS:A

Director, Procurement  OS:A:P

Manager, Program Oversight Office  OS:CIO:SM:PO

 

Appendix IV

 

Outcome Measures

 

This appendix presents detailed information on the measurable impact that our recommended corrective actions will have on tax administration.  These benefits will be incorporated into our Semiannual Report to Congress.

Type and Value of Outcome Measure:

  • Revenue Protection – Potential; $318.3 million (see page 4).

Methodology Used to Measure the Reported Benefit:

The Electronic Fraud Detection System (EFDS) was not operational during Processing Year[22] (PY) 2006.  Therefore, Internal Revenue Service (IRS) management does not know how many fraudulent tax returns and refunds have not been and will not be stopped during PY 2006.  Figure 1 shows the amount of fraudulent refunds stopped has generally increased over the last 4 years.

Figure 1: Fraudulent Refund Returns and Refunds Identified and Stopped

Processing Year

Total Returns Filed

Refund Returns Filed

Returns Screened by Fraud Detection Centers

False Refund Returns Identified

False Refund Returns Stopped

False Refunds Identified

False Refunds Stopped[23]

2002

130,341,159

104,367,859

1,942,089

81,486

41,358

$450,023,509

$333,541,138

2003

130,134,276

104,904,543

740,216

96,953

73,400

$349,515,144

$266,423,786

2004

130,459,600

106,420,200

463,222

118,075

82,099

$2,241,612,551

$2,110,454,658

2005

133,933,000

100,276,000

511,805

132,945

103,537

$515,548,186

$412,184,202

Source:  The Criminal Investigation Division.

During PY 2005, $412.2 million in fraudulent refunds were stopped.  The IRS reported that, due to other leads,[24] $93.9 million[25] in fraudulent refunds had been stopped as of May 19, 2006, without the EFDS being operational.  Therefore, based on the value of fraudulent refunds stopped by the EFDS in PYs 2005 and 2006 (through May 19), we estimate approximately $318.3 million in fraudulent refunds may have been issued in 2006, resulting in lost revenue to the Federal Government.

Type and Value of Outcome Measure:

  • Inefficient Use of Resources – Potential; $20.5 million (see page 9).

Methodology Used to Measure the Reported Benefit:

Because the Web EFDS was not implemented as scheduled in 2005 and 2006, and there are no current plans to continue development of the Web EFDS, we estimate the IRS inefficiently used resources totaling $20.5 million from May 25, 2001, to April 24, 2006, for the contractor costs associated with the development of the Web EFDS.  The EFDS Project Executive provided a spreadsheet with the payments made to Contractor 1 for system maintenance and Web EFDS development.  We determined $18.5 million should be considered the development costs charged by Contractor 1.[26] Contractors 2 and 3 were paid approximately $2 million between February 6, 2004, and April 24, 2006, for Web EFDS development costs.  Thus, the costs for Web EFDS development totaled $20.5 million.

Type and Value of Outcome Measure:

  • Questioned Costs – Actual; $459,718 (see page 18).

Methodology Used to Measure the Reported Benefit:

In October 2004, IRS executives became concerned about the status of the Web EFDS development.  They determined the Web EFDS could not be implemented in January 2005 as planned and decided the old EFDS would be implemented. Therefore, the old EFDS needed to receive software updates, complete testing, etc. to be implemented for PY 2005.  IRS management advised us that, during a meeting between IRS and contractor executives, a Contractor Senior Director commented the old EFDS would be updated and implemented at no cost to the Federal Government because the contractor did not deliver the Web EFDS as scheduled. However, the contractor submitted invoice charges totaling $459,718 for the additional work to get the old EFDS ready for implementation and these charges were paid by the IRS.  The EFDS Project Manager advised us the invoices were paid without question because all invoices for the contract type that was used must be paid in full.  The contractor had stated only verbally it would not charge the IRS for updating the old EFDS for use in 2005, and the matter was not raised with the contractor.

 

Appendix V

 

Chronology of the Electronic Fraud Detection System Development Events January 1, 1995, Through April 19, 2006

 

 1995

 

1995

The Electronic Fraud Detection System (EFDS) was prototyped at the Cincinnati Service Center[27] in 1994 and implemented in the five electronic filing service centers in 1995.  Prior to implementation of the EFDS, the Questionable Refund Program Computer Identification Program began analyzing paper tax returns in 1977 and electronically filed tax returns in 1990.

1996 

 

1996

Contractor 1 began work on the EFDS under the Treasury Information Processing Support Services contract.  The EFDS was implemented at all 10 Fraud Detection Centers.

2001 

 

June 20, 2001

The Internal Revenue Service (IRS) approved a task order for Contractor 1 for the period
June 2000 through May 2006 authorizing approximately $39.5 million for software maintenance and development.  As of April 2006, the IRS had spent over $37 million.

2002 

 

January 31, 2002

A new Director, Business Systems Development (BSD), was appointed.

June 12, 2002

The Contractor 1 reengineering study, which dealt mostly with redesign of the database, stated the EFDS database required a major reengineering effort to meet the needs of the Criminal Investigation (CI) Division to identify quickly and stop tax refund fraud.

June 14, 2002

The EFDS Executive Steering Committee (ESC) was established in the Modernization and Information Technology Services (MITS) organization.

2003 

 

February 14, 2003

The Reengineering 2003 Technical Approach and Plan showed Web EFDS applications were to be implemented in January 2005.

July 14, 2003

Last known meeting of the EFDS ESC.  This meeting included the first ESC discussion of the Web portal and database redesign projects.  Project monitoring responsibility was transferred to the EFDS Project Office in the BSD organization because the planning and scheduling phase was completed (it lasted 13 months) and the performance problems were resolved.

2004 

 

June 2004

The MITS organization appointed a new BSD Director, Filing Systems Division, who is responsible for the EFDS.

June 18, 2004

The EFDS Contracting Officer’s Technical Representative (COTR) expressed concerns to the contractor’s development team that they had not completed a walk-through of the application with the users. Therefore, the contractor was not sufficiently familiar with the functionality of the current EFDS, screen changes were made without considering user needs, and the requirements document was not followed.

July – August 2004

The EFDS software was delivered to the Product Assurance organization for System Acceptability Testing.  Problems were identified, and the System was deemed unusable.

September 9, 2004

The IRS EFDS Project Manager sent an email to the CI Division advising there is no contingency plan to keep the old programs functional or to use them for Processing
Year (PY) 2005 production.

October 2004

The IRS EFDS Project Manager left the EFDS project sometime between August 2004 and October 2004.  IRS executives and Contractor 1 agreed the System modifications would not be ready by January 2005.  A decision was made to use the old EFDS for PY 2005 while work continued on the new System.

October 18, 2004

The Product Assurance organization advised the CI Division the EFDS Web portal applications were not delivered for System Acceptability Testing on October 9, 2004, as planned.

October 31, 2004

A new Project Manager was appointed for the EFDS.

November 2, 2004

The CI Director, Refund Crimes, advised the Chief, CI, that, per a meeting between the BSD Director, Filing Systems Division, and the Associate Chief Information Officer (CIO), Information Technology Services, on November 01, 2004, the IRS would be going back to the 2004 EFDS for PY 2005.

****3(d)****

****3(d)****

2005

 

February 17, 2005

The CI Director, Refund Crimes, signed the Web Portal Requirements Package.

March 11, 2005

The Web Portal Work Breakdown Structure was signed.  This provided for a commitment from Contractor 1 senior management to the dates noted in the work breakdown structure.

April 20, 2005

An EFDS Project Office email cited data loads and other problems (first indication of Web EFDS problems in 2005).

July 2005

The Product Assurance organization began System Acceptability Tests and identified System problems.

August 2005

The training database was delivered to the CI Division, which began developing training materials and surfacing problems.  Biweekly meetings among the CI Division, the BSD organization, and contractor executives began.

September 28, 2005

CI Division executives and staff expressed to BSD organization management (BSD Filing Systems Division Director, BSD Document Input Branch Chief, EFDS Project Office) their doubts that the Web EFDS would be implemented in January 2006.

October 18, 2005

The CI Director, Refund Crimes, submitted a briefing paper to the Chief, CI, advising there is no contingency plan for the EFDS.

October 21, 2005

In an email, Contractor 1 assured the IRS it was confident that the Web EFDS would be delivered on January 13, 2006.

October 28, 2005

The Chief, CI, emailed the Deputy CIO advising that their staffs had come to a collective understanding of the Web EFDS risks and planned to discuss any unresolved issues or pending risks the following week.

October 31, 2005

The BSD Director, Filing Systems Division, emailed an update on the Web EFDS performance and data conversion problems to the CIO, other MITS organization executives, CI Division executives, and BSD organization management and staff.  (This is the first time we can substantiate the CIO was notified of problems.) Contractor 1 maintained the System would be ready on January 13, 2006, as scheduled.  The first EFDS Readiness Executive Briefing was held.

Late October or November 2005

The Deputy Commissioner for Operations Support and the Deputy Commissioner for Services and Enforcement advised us they became aware of the Web EFDS issues in late October or November 2005.

November 2, 2005

The BSD Director, Filing Systems Division, visited the CI Division’s training location to observe the problems with the training database.

November 4, 2005

The CIO advised us the first indication of potential Web EFDS issues was raised during the CI Division’s Business Performance Review meeting for the fourth quarter of Fiscal Year 2005.

November 7, 2005

The CIO advised us the MITS organization Business Performance Review meeting for the fourth quarter of Fiscal Year 2005 included a presentation on Web EFDS that indicated there were issues with the System, but it would be delivered on schedule.

November 2005

The Associate CIO, BSD, and others decided they could not go back to the old EFDS.  They thought they could run the new System, even though it had known flaws.

November 29, 2005

A meeting was held with the following IRS executives to discuss progress made to prepare the CI Division for PY 2006 as it related to the Web EFDS: Deputy CIO; Associate CIO, BSD; Deputy Associate CIO, BSD; Deputy Director, Submission Processing; Director, Filing Systems Division; CI Director, Refund Crimes; and CI Deputy Director, Technology Operations and Investigative Services.

Late November or December 2005

Senior executive briefings began daily on the status of the Web EFDS.  Participants in these briefings were generally the Associate CIO, BSD; Deputy Associate CIO, BSD; Director and Assistant Director, Filing Systems Division; and representatives from the contractor, Project Office, and Enterprise Operations organization.

The Commissioner advised us he became aware of the Web EFDS problems in late November or December 2005.

December 2005

An acting EFDS Project Manager was appointed.

December 2, 2005

The old EFDS was shut down.

December 22, 2005

The EFDS Project Office determined data loads had not started although the
December 20, 2005, work breakdown structure prepared by Contractor 1 indicated the work was 50 percent complete.

December 31, 2005

The CI Division Business Performance Review for the first quarter of Fiscal Year 2006 stated the Web EFDS was currently delayed and remained in testing, meetings between the MITS organization and CI Division had occurred to discuss progress and potential risk to the Program, and training for Fraud Detection Center personnel had been stopped.

2006

 

January 2006

IRS executives realized the project would not be delivered on time.

January 4, 2006

The Web EFDS project schedule slipped 2 weeks since prior day’s work breakdown structure.  The problems at this time were related to data volume/data conversion/daily loads; they were not application-related problems.

January 13, 2006

Planned delivery date for the Web EFDS.

January 25, 2006

After missing the January 13, 2006, implementation date, Contractor 1 management stated a fully functioning System would be delivered by February 1, 2006.

January 30, 2006

The BSD organization informed the CIO that a System would not be delivered by
February 1, 2006.

February 13, 2006

The IRS initiated a ‘War Room’ to coordinate communications among the Enterprise Operations and BSD organizations and the CI Division.

February 15, 2006

The Compliance Domain Director, Applications Development, took over the Web EFDS project.

February 16, 2006

The MITS organization Business Performance Review for the first quarter of Fiscal
Year 2006 stated it failed to timely implement the Web EFDS for production startup on
January 13, 2006.

February 21, 2006

The IRS Deputy Commissioner for Services and Enforcement advised the Treasury Inspector General for Tax Administration that the EFDS would not be available during PY 2006.  The IRS does not plan to conduct a recovery program to identify potentially fraudulent refunds already issued.

March 2006

The old EFDS became available for nationwide research purposes only.

April 10, 2006

War Room and testing efforts for the new System were stopped.

April 17, 2006

A new acting EFDS Project Manager was appointed.  Due to the risks, MITS organization executives (e.g., the EFDS Project Executive), with IRS Commissioner concurrence, decided to work only on restoring the old EFDS for use in PY 2007.

April 19, 2006

All system development activities for the new EFDS (i.e., Web portal application and database redesign efforts) were stopped; the contractor began finalizing a plan for revising the old EFDS for use in PYs 2006 and 2007.

Source:  Our interviews and documents acquired during fieldwork.

 

Appendix VI

 

Glossary of Terms

 

As-built Architecture

Documents Internal Revenue Service (IRS) current production information technology systems.

Business Case

Required by Office of Management and Budget Circular A-11 (Preparation, Execution, and Submission of the Budget; dated June 2005) and commonly called Exhibit 300, Capital Asset Plan and Business Case.  Each agency must submit a Business Case twice a year for each major information technology investment.

Cincinnati Service Center (currently known as the Cincinnati Campus), located in
Covington, Kentucky

Data processing arm of the IRS. The campuses process paper and electronic submissions, correct errors, and forward data to the Computing Centers for analysis and posting to taxpayer accounts.

Client Server

A network architecture in which clients are personal computers or workstations on which users run applications.  Clients rely on servers for resources such as files, devices, and even processing power.

Computing Centers

Support tax processing and information management through a data processing and telecommunications infrastructure.

Contracting Officer’s Technical Representative

Furnishes technical direction, monitors contract performance, and maintains an arm’s-length relationship with the contractor.

Cost-Plus-Fixed-Fee Contract

A cost-reimbursement contract that provides for payment to the contractor of a negotiated fee that is fixed at the inception of the contract.  This contract type permits contracting for efforts that might otherwise present too great a risk to contractors, but it provides the contractor only a minimum incentive to control costs.

Cost-Reimbursement Contract

Provides for payment of allowable incurred costs, to the extent prescribed in the contract.

Data Loads

Process of placing data into a system or database.

Data-Mining Technique

Process of automatically searching large volumes of data for patterns.

Development/Modernization/Enhancement

Any new information technology investment being proposed, developed, or acquired.

Enterprise Life Cycle-Lite (ELC-Lite)

Required system development methodology for all nonmodernization projects.

ELC-Lite – Architecture Phase

Establishes the concept/vision, requirements, high-level design, project management processes, and plans for a particular business area or target system.  It also defines the releases for the business area or system.

ELC-Lite – Integration Phase

Includes the integration, testing, piloting, and acceptance of a system release.

ELC-Lite – Operations and Support Phase

Addresses the ongoing support of the system and maintenance of the applications.  It begins after the business process and system(s) have been installed and have begun performing business functions.  It encompasses all of the operations and support processes necessary to deliver the services associated with managing all or part of a computing environment.

Executive Steering Committee

Oversees investments, including validating major investment business requirements and ensuring that enabling technologies are defined, developed, and implemented.

Fraud Detection Centers

Ten Criminal Investigation Division Centers whose mission is to identify and detect refund fraud, prevent the issuance of false refunds, and provide support for the Criminal Investigation Division field offices.

Information Technology Investment Portfolio

Required by Office of Management and Budget Circular A-11 and commonly referred to as an Exhibit 53.  This portfolio must be submitted with each agency’s annual budget submission and contain the information technology investment title, description, amount, and funding source.

Integrated Financial System

An administrative accounting system used by the IRS.

Item Tracking Reporting and Control

An information system used by the Business Systems Modernization Office to track and report on issues, risks, and action items in the modernization effort.

Milestone Review

A formal review process conducted to determine whether key activities within a project’s life cycle have been completed prior to proceeding from milestone to milestone.

Modernization and Information Technology Services (MITS) Enterprise Governance Committee

Highest level recommending and
decision-making body to oversee and enhance enterprise management of information systems and technology.  It ensures strategic modernization and information technology program investments, goals, and activities are aligned with and support 1) the business needs across the enterprise and 2) the modernized vision of the IRS.

MITS Enterprise Governance Committee Investment Management Subcommittee

Supports the MITS Enterprise Governance Committee in the realization of the IRS Capital Planning and Investment Control process and with the management of the IRS information technology investment portfolio.  This Subcommittee provides general information technology investment portfolio oversight, including operational analysis reviews and reports, investment prioritization recommendations, and recommendations for adjustments to the IRS portfolio.

Performance-Based Contract

Provides for acquiring services on the basis of required results rather than the methods of performing the work and uses measurable performance standards (e.g., in terms of quality, timeliness, quantity).

Processing Year

The year in which tax returns and other tax data are processed.

Product Assurance

A MITS organization function that independently assesses the quality of the applications software by conducting System Acceptability Testing with controlled data to aid the customer in determining the system’s production readiness.

Quality Assurance Surveillance Plan

Ensures services provided by the contractor meet contract requirements.  It should specify the work requiring surveillance and the method of surveillance.

Questionable Refund Program Computer Identification Program

An application running on the mainframe computer.  The Program was originally developed by the IRS Inspection Service and run by the Internal Audit function (now the Treasury Inspector General for Tax Administration Office of Audit).

Questioned Cost

A cost that (1) violated a provision of a law, regulation, contract, or other requirement governing the expenditure of funds; (2) was not supported by adequate documentation; or
(3) was unnecessary or unreasonable.

Request for Information Services

A formal memorandum requesting MITS organization support for changes to current or planned programming, hardware, system testing, etc.

Security Certification and Accreditation

A security certification is an independent technical evaluation, for the purpose of accreditation, that uses security requirements as the criteria for the evaluation.  An accreditation is an authorization granted by a management official to operate the system based on the evaluation of the security controls.

Senior Management Dashboard Review

A review attended by senior executives, contractors, program directors, and project managers to ensure program directors and project managers are held accountable for the project status (e.g., risk, cost, schedule).  Emphasis is placed on only problem areas or notable status changes.

Steady State

Any information technology investment that is fully operational.

System Acceptability Testing

Process of testing a system or program to ensure it meets the original objectives outlined by the user in the requirement analysis document.

Task Order

An order for services placed against an established contract or with Federal Government sources.

Tier A - Technical Modernization Projects

Resources devoted to projects managed by the IRS Business Systems Modernization Office.  Project scale is large, with a 2-year to 3-year time period.

Tier B - Improvement Projects

Resources devoted to new improvements of medium size (1-year to 2-year time period) and funded from the improvement programs budget.

Tier C - Enhancements/Stay-In-Business Projects

Resources devoted to all other types of projects (e.g., sustaining operations, legislative changes, and small enhancements to sustaining operations) funded from the regular sustaining operations budget.

Time-and-Materials Contract

Provides for acquiring supplies or services on the basis of direct labor hours at specified fixed hourly rates that include wages, overhead, general and administrative expenses, profit, and materials at cost.

Treasury Information Processing Support Services-2

Contracts, awarded in 2000, that provide a broad range of information technology-related services.

War Room

Set up at the Enterprise Computing Center in
Memphis, Tennessee, to serve as a centralized repository for all incoming and outgoing information regarding the Web Electronic Fraud Detection System (Web EFDS).  The War Room was staffed primarily by technical staff so information could be disseminated as it became available, questions could be answered immediately, and necessary assistance could be provided.  The War Room sent daily voice mail messages two to three times per day regarding the Web EFDS status.

Web EFDS

EFDS development effort allowing users to access the system via the IRS Intranet.

Web Portal

An Internet site or service that functions as a major starting site for users to connect to a broad array of resources and services, such as email, forums, research tools, online shopping malls, etc.

Work Breakdown Structure

Project schedule used to manage the tasks, task relationships, and resources needed to meet project goals.

Work Request

Contains the specific details of the work to be performed in a task order including the skill categories, estimated number of hours, required work products, and acceptance criteria.

 

Appendix VII

 

Electronic Fraud Detection System Management Turnover

 

Table 1:  Management Responsible for the Electronic Fraud Detection System (EFDS) Prior to the August 8, 2005, Modernization and Information Technology Services (MITS) Reorganization

Title

Employee’s Name

Date

Chief Information Officer (CIO)[28]

****3(a), 3(d)****

****3(a), 3(d)****

****3(a), 3(d)****

****3(a), 3(d)****

****3(a), 3(d)****

****3(a), 3(d)****

****3(a), 3(d)****

****3(a), 3(d)****

Associate CIO, Information Technology Services[29]

****3(a), 3(d)****

****3(a), 3(d)****

****3(a), 3(d)****

****3(a), 3(d)****

Director, Business Systems Development (BSD)[30]

****3(a), 3(d)****

****3(a), 3(d)****

Deputy Director, BSD[31]

****3(a), 3(d)****

****3(a), 3(d)****

****3(a), 3(d)****

****3(a), 3(d)****

Director, Submission Processing Division[32]

****3(a), 3(d)****

****3(a), 3(d)****

Director, Filing Systems Division

****3(a), 3(d)****

****3(a), 3(d)****

****3(a), 3(d)****

****3(a), 3(d)****

Acting Director, Filing Systems Division

****3(a), 3(d)****

****3(a), 3(d)****

Document Perfection Branch Chief, Filing Systems Division

****3(a), 3(d)****

****3(a), 3(d)****

EFDS Project Managers/Acting Project Managers

****3(a), 3(d)****

****3(a), 3(d)****

Acting Project Managers

2000 - October 2000

****3(a), 3(d)****

****3(a), 3(d)****

****3(a), 3(d)****

****3(a), 3(d)****

Source:  Our analysis of MITS organization documents.

Table 2: Management Responsible for the EFDS After the
August 8, 2005
MITS Reorganization

Title

Employee's Name

Date

CIO

****3(a), 3(d)****

****3(a), 3(d)****

Deputy CIO

****3(a), 3(d)****

****3(a), 3(d)****

Associate CIO, BSD

****3(a), 3(d)****

****3(a), 3(d)****

Associate CIO, Applications Development

****3(a), 3(d)****

****3(a), 3(d)****

Deputy Associate CIO, Applications Development

****3(a), 3(d)****

****3(a), 3(d)****

Director, Filing Systems Division

****3(a), 3(d)****

****3(a), 3(d)****

Document Perfection Branch Chief, Filing Systems Division

****3(a), 3(d)****

****3(a), 3(d)****

EFDS Project Managers/Acting Project Managers

****3(a), 3(d)****

****3(a), 3(d)****

****3(a), 3(d)****

****3(a), 3(d)****

****3(a), 3(d)****

****3(a), 3(d)****

Source:  Our analysis of MITS organization documents.

 

Appendix VIII

 

Management’s Response to the Draft Report

The response was removed due to its size.  To see the response, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

 



[1] See Appendix VI for a Glossary of Terms.

[2] The $18.5 million and $20.5 million are based on our analysis of the total costs obtained from the EFDS Project Office and the IRS web-based requisition tracking system.

[3] The system development effort will allow users to access the System via the IRS Intranet.

[4] The EFDS is the primary source for the identification of leads on fraudulently filed tax returns; however, leads are also received from sources internal to and external from the IRS.

[5] This amount was derived from a manual process provided by the Fraud Detection Centers on a weekly report; it has not been verified and could contain inconsistencies/inaccuracies due to the manual process.

[6] The MITS organization recently combined the former Business Systems Development and Business Systems Modernization organizations to create one Applications Development organization.  As of March 2006, executive management had been realigned into the new organization structure.

[7] See Appendix VI for a Glossary of Terms.

****1****

[9] The $18.5 million and $20.5 million are based on our analysis of the total costs obtained from the EFDS Project Office and the IRS web-based requisition tracking system.

[10] Pub. L. No. 104-106, 110 Stat. 642 (codified in scattered sections of 5 U.S.C., 5 U.S.C. app., 10 U.S.C., 15 U.S.C., 16 U.S.C., 18 U.S.C., 22 U.S.C., 28 U.S.C., 29 U.S.C., 31 U.S.C., 38 U.S.C., 40 U.S.C., 41 U.S.C., 42 U.S.C., 44 U.S.C., 49 U.S.C., 50 U.S.C.).

[11] The Development/Modernization/Enhancement and Steady State investment types are broken down into the subinvestment types Major, Non-Major, and Small-Other.

[12] The bolded words in the italicized direct quotes show the conflicting information.

[13] Business Cases for Information Technology Projects Need Improvement (Reference Number 2005-20-074, dated April 2005).

[14] Information System Life Cycle Manual (dated March 2002).

[15] This amount is based on our analysis of the total costs obtained from the EFDS Project Office and the IRS web-based requisition tracking system.

[16] On December 1, 2005, unresolved problem tickets recorded on the CI Division’s database were transferred to the Product Assurance organization’s problem ticket database.  We were unable to determine the number of problem tickets opened and closed by the CI Division from August to December 2005 that were not included in the Product Assurance organization’s problem ticket database.

[17] The total number of employees (43) and the number of employees (5) who left the project were determined through March 2006 (i.e., first quarter of 2006).  The number of employees who left the project through March (5) was multiplied by 4 to determine an annual estimate (20).

[18] This amount was based on our analysis of the total costs obtained from the EFDS Project Office and the IRS web-based requisition tracking system.

[19] Annual Assessment of the Business Systems Modernization Program (Reference Number 2005-20-102, dated August 2005).

[20] 48 C.F.R. ch. 1 (2005).

[21] See Appendix VI for a Glossary of Terms.

[22] See Appendix VI for a Glossary of Terms.

****1****

[24] The EFDS is the primary source for the identification of leads on fraudulently filed tax returns; however, leads are also received from sources internal to and external from the IRS.

[25] This amount was derived from a manual process provided by the Fraud Detection Centers on a weekly report; it has not been verified and could contain inconsistencies/inaccuracies due to the manual process.

[26] The $18.5 million and $20.5 million are based on our analysis of the total costs obtained from the EFDS Project Office and the IRS web-based requisition tracking system.

[27] See Appendix VI for a Glossary of Terms.

[28] Names of the Acting CIOs during the gaps in the time period are not listed.

[29] This position was abolished ****3(d)****

[30] Position title changed to Associate CIO, BSD, as a result of the MITS organization’s reorganization, effective August 8, 2005 (see Table 2).

[31] Position title changed from Deputy Director, BSD, to Deputy Associate CIO, Applications Development, as a result of the MITS organization’s reorganization (see Table 2).

[32] The Division responsible for EFDS development prior to the creation of the BSD organization.