TREASURY INSPECTOR GENERAL FOR TAX ADMINISTRATION

 

 

Oversight of the Electronic Fraud Detection System Restoration Activities Has Improved, but Risks Remain

 

 

 

March 29, 2007

 

Reference Number:  2007-20-052

 

 

This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.

Redaction Legend:

3(a) = Identifying Information - Name of an Individual or Individuals
3(d) = Identifying Information - Other Identifying Information of an Individual or Individuals

Phone Number   |  202-927-7037

Email Address   |  Bonnie.Heald@tigta.treas.gov

Web Site           |  http://www.tigta.gov

 

March 29, 2007

 

 

MEMORANDUM FOR CHIEF INFORMATION OFFICER

 

FROM:                            Michael R. Phillips /s/ Michael R. Phillips

                                         Deputy Inspector General for Audit

 

SUBJECT:                    Final Audit Report – Oversight of the Electronic Fraud Detection System Restoration Activities Has Improved, but Risks Remain (Audit # 200620042)

 

This report presents the results of our review to determine whether the Internal Revenue Service (IRS) adequately monitored the contractors’ development efforts in 2006 to ensure the Electronic Fraud Detection System (hereafter referred to as EFDS or System)[1] was delivered in time for the 2007 Filing Season.  This audit is a follow-up to a prior Treasury Inspector General for Tax Administration audit.[2]

Impact on the Taxpayer

The EFDS is the primary information system used to support the Criminal Investigation Division’s Questionable Refund Program, which is a nationwide program established in January 1997 to detect and stop fraudulent and fictitious claims for refunds on income tax returns.  During Processing Year 2006, the System was not operational because the IRS and its contractors were unable to launch a web-based version of the EFDS application (Web EFDS), resulting in an estimated $318.3 million in fraudulent refunds being issued as of May 19, 2006.  The IRS has improved controls over the EFDS restoration activities including executive governance and project management.  As a result, project risks are being identified and mitigation actions are being taken to ensure the System is implemented and fraudulent refunds stopped during Processing Year 2007.

 Synopsis

On April 19, 2006, all system development activities for the Web EFDS were stopped and all efforts were focused on restoring the client-server EFDS for use in January 2007.  The restoration effort requires the contractors to prepare the System and the related databases for Processing Year 2007 by starting with the Processing Year 2005 EFDS and updating it with the 2006 and 2007 tax law changes.  Therefore, the System restoration work to be completed by the contractors involves the routine annual update of the System with tax law changes and does not contain the level of complexity involved in redesigning it into a web-based system.

In the prior EFDS audit, we reported the IRS did not ensure the EFDS project had the required executive oversight, manage the System risks effectively, monitor contractor performance effectively, and use performance-based contracts.  The EFDS project also was improperly classified as a steady state project in the business case.  During this audit, we determined that IRS management completed several corrective actions in response to our prior audit report.

IRS management implemented executive oversight and improved project management controls.  However, the Federal Government may not receive the full amount of the equitable adjustment.

The IRS improved executive oversight of the EFDS project by requiring the status and risks of the project be reported at various meetings.  Additionally, project management controls were improved.  For example, regular meetings are held with stakeholders and contractors to ensure tasks are on target for timely completion and risks are addressed.  If tasks are not completed when scheduled, the effect on the overall schedule is determined and remedial actions are taken, if needed.

The EFDS Project Office also obtained project management support from contractor Booz Allen Hamilton, Inc., and obtained independent assessments of the System from the MITRE Corporation at an estimated cost of $1,722,132.  These expenses are considered inefficient use of resources because the expenses would not have been incurred if the Web EFDS had been implemented in Processing Year 2006 (see Appendix IV).

Although project management controls have improved, as of the time of our review on December 8, 2006, risks remained as several critical tasks had not been completed.  For example, the EFDS (applications and 3 years of data) must be loaded into the production environment, final integration testing must be completed, and the required Enterprise Life Cycle documents must be prepared.

This audit was conducted while the IRS was performing restoration activities to implement the System in Processing Year 2007.  Any changes that occurred since we completed our analysis in December 2006 are not reflected in this report.  As a result, this report may not reflect the most current status of the EFDS project.  According to the IRS, the System was placed into production on January 16, 2007.

During this audit, we also determined the Contracting Officer’s Technical Representative oversight of the Computer Sciences Corporation (CSC) had not changed significantly and the EFDS Project Office is in the process of drafting procedures for monitoring acquisitions.  Meanwhile, compensating controls, such as the improvements in project management, mitigate the oversight risks.

The IRS recently issued a contract for an estimated amount of $3,080,004 for restoration work to be performed from November 1, 2006, through February 24, 2007.  We reviewed the contract and found that payment of the contractor’s fee is not dependent on the timely delivery of specific System deliverables or milestones.  The contract also established a cost sharing amount not to exceed $3,080,004 as an equitable adjustment amount to compensate the IRS for the cost to restore the client-server EFDS.  However, the agreement does not include a provision that would refund the unused equitable adjustment to the IRS and the cost sharing commitment is exclusively related to delivering a client-server EFDS in January 2007.

Based on our review of the EFDS project work breakdown structure (i.e., a list of all tasks required to complete the project) it does not appear the CSC has $3,080,004 worth of work remaining on the restoration project.  The EFDS Executive agreed with this conclusion and stated the CSC has verbally agreed to work on two application changes unrelated to the restoration work to ensure the IRS will receive the $3,080,004 equitable adjustment.  However, the contract states the CSC’s cost sharing commitment is exclusively related to delivering a client-server-based System and will not apply to any Federal Government directed scope increases.  Therefore, the IRS will be obligated to pay the contractor’s fee if a functional EFDS is not implemented timely and the IRS may not receive the entire equitable adjustment.

Recommendation

We recommended the Chief Information Officer work with the Director, Procurement, to ensure the IRS receives all of the $3,080,004 equitable adjustment from the CSC.  If the entire adjustment is not received by the end of the original period of performance stated in the contract, the IRS should request the CSC pay the IRS the difference between the $3,080,004 and the credit the IRS received during the period of performance.  Alternatively, the IRS should request the application of the remaining equitable adjustment credit owed to the IRS to invoices for future EFDS-related task orders or for other work being performed by the CSC.

Response

IRS management agreed with the recommendation and prepared a modification to the task order to ensure the IRS receives the full equitable adjustment.  The modification, signed by the IRS and the CSC on February 23, 2007, extends the base period of performance and includes additional work within the scope of the cost sharing agreement.  Management’s complete response to the draft report is included as Appendix IX.

Copies of this report are also being sent to the IRS managers affected by the report recommendation.  Please contact me at (202) 622-6510 if you have questions or Margaret E. Begg, Assistant Inspector General for Audit (Information Systems Programs), at (202) 622-8510.

 

 Table of Contents

 

Background

Results of Review

Executive Oversight of the Electronic Fraud Detection System Has Improved

Electronic Fraud Detection System Restoration Project Management Controls Have Been Improved, but Risks Remain

Contracting Activities Have Improved, but a Cost Reimbursement Issue Remains

Recommendation 1:

Appendices

Appendix I – Detailed Objective, Scope, and Methodology

Appendix II – Major Contributors to This Report

Appendix III – Report Distribution List

Appendix IV – Outcome Measures

Appendix V – Electronic Fraud Detection System Management

Appendix VI – Electronic Fraud Detection System Oversight

Appendix VII – Electronic Fraud Detection System Project Timeline

Appendix VIII – Glossary of Terms

Appendix IX – Management’s Response to the Draft Report

 Abbreviations

 

CIO

Chief Information Officer

COTR

Contracting Officer’s Technical Representative

CSC

Computer Sciences Corporation

EFDS, System

Electronic Fraud Detection System

ESC

Executive Steering Committee

IRS

Internal Revenue Service

MITRE

MITRE Corporation

MITS

Modernization and Information Technology Services

PY

Processing Year

Web EFDS

Web Electronic Fraud Detection System

 

 

Background

 

The Electronic Fraud Detection System (hereafter referred to as the EFDS or System)[3] is an automated compliance system designed to maximize fraud detection when tax returns are filed and prevent the issuance of fraudulent refunds.  The EFDS is the primary information system used to support the Criminal Investigation Division’s Questionable Refund Program, which is a nationwide program established in January 1977 to detect and stop fraudulent and fictitious claims for refunds on income tax returns.

In January 2006, the Internal Revenue Service (IRS) planned to launch a web-based version of the EFDS application (Web EFDS) after failing to implement the Web EFDS in January 2005 because of system development problems.  However, the IRS and its contractors were unable to provide a functioning Web EFDS to prevent fraudulent refunds during Processing Year (PY) 2006.  During PY 2006, the System was not operational, resulting in an estimated $318.3 million in fraudulent refunds being issued as of May 19, 2006.

On April 19, 2006, all system development activities for the Web EFDS were stopped, and all efforts were focused on restoring the client-server EFDS for use in January 2007.  The restoration effort requires the contractors to prepare the System and the related databases for PY 2007 by starting with the PY 2005 EFDS and updating it with the 2006 and 2007 tax law changes.  Therefore, the System restoration work to be completed by the contractors involves the routine annual update of the System with tax law changes and does not contain the level of complexity involved in redesigning the System into a web-based system.

Five contractors are involved in various EFDS activities.  Three of the contractors are working to restore the System for PY 2007, while the remaining two contractors provide program management support.  The responsibilities of the five contractors include the following:

·         Computer Sciences Corporation (CSC), the primary contractor, is responsible for delivering a fully operational client-server-based System in January 2007.  As of December 11, 2006, the total amount paid to the CSC for System restoration work was $2,613,953.  In addition, a task order with an estimated cost of $3,080,004 was approved on October 24, 2006, for restoration work to be performed through February 24, 2007.

·         Systems Research and Applications Corporation is responsible for providing and maintaining data-mining techniques used by the EFDS.  As of December 11, 2006, the total amount paid to the Systems Research and Applications Corporation for the System restoration was $167,584.  In addition, a task order with an estimated cost of $420,648 was approved on July 28, 2006, for work to be performed through July 31, 2007.  The remaining funds available for this task order are $336,859.

·         Anteon Corporation is responsible for providing maintenance support for the EFDS client-server application and database.  A task order was approved on August 15, 2006, with an estimated cost of $1,500,000 for work to be performed between April 11, 2006, and February 24, 2007.  Because the work performed by Anteon Corporation is critical to the System restoration, it was allowed to begin work before the task order was approved.  As of December 11, 2006, the total amount paid to the Anteon Corporation for the EFDS restoration was $707,006.  The remaining funds available for this task order are $792,994.

·         Booz Allen Hamilton, Inc. is responsible for providing EFDS Project Office support.  A task order with an estimated cost of $1,201,378 for project management support services was awarded July 6, 2006, for work to be performed through July 1, 2007.

·         MITRE Corporation (MITRE) is responsible for providing independent assessments of the System restoration activities.  A task order with an estimated cost of $103,024 was approved on September 14, 2006, for work to be performed through December 31, 2006.

This review is a follow-up to a prior Treasury Inspector General for Tax Administration audit.[4]  This review was performed at the Modernization and Information Technology Services (MITS) organization offices in New Carrollton, Maryland, and Washington, D.C., during the period October through December 2006.  The audit was conducted in accordance with Government Auditing Standards.  Detailed information on our audit objective, scope, and methodology is presented in Appendix I.  Major contributors to the report are listed in Appendix II.

 

Results of Review

 

Executive Oversight of the Electronic Fraud Detection System Has Improved

The Clinger-Cohen Act of 1996[5] requires agencies to use a disciplined Capital Planning and Investment Control process to acquire, use, maintain, and dispose of information technology assets.  The Office of Management and Budget Circular A-11, Preparation, Execution, and Submission of the Budget, dated June 2006, requires each agency to include with its annual budget submission an information technology investment portfolio, commonly referred to as an Exhibit 53, containing the information technology investment title, description, amount, and funding source.  For each major information technology investment, the Office of Management and Budget requires agencies to include Circular A-11 Exhibit 300, Capital Asset Plan and Business Case, with their budget submissions.

The IRS’ Capital Planning and Investment Control process for managing information technology projects established an executive governance process for monitoring projects that included the MITS Enterprise Governance Committee, the MITS Enterprise Governance Investment Management Subcommittee, and Executive Steering Committees (ESC) responsible for specific projects.  Major projects with costs of more than $5 million per year or total lifecycle costs of more than $50 million were to be governed by the executive governance process.  Formal agendas, presentations, and meeting minutes are prepared for each ESC meeting including documenting key decisions and assignments.  To assess the controls over the EFDS project, we reviewed the policies and procedures applicable to the project and determined whether they were implemented effectively.

In the prior EFDS audit, we reported the Exhibit 300 improperly classified the EFDS as a steady state project.  This was improper because, at the time, the IRS was in the process of developing the Web EFDS.  In addition, information in the Exhibit 300 was not consistent and presented the EFDS as both a steady state system and a system under development.

The EFDS project did not have continuous ESC oversight as required by the Capital Planning and Investment Control process.  Instead, there was ESC oversight from June 2002 until July 2003.  Afterwards, oversight was provided by Business Systems Development organization executives who were also responsible for managing the maintenance and development work for more than 325 IRS systems.

We also reported that key decisions relating to the Web EFDS development were not adequately documented.  Consequently, we made the following recommendations:

  • Recommendation 1:  The Chief Information Officer (CIO) should ensure the EFDS project is assigned to an ESC for executive oversight, including documenting key decisions and assignments.
  • Recommendation 2:  The CIO should evaluate other projects being managed in the new Applications Development organization and ensure all projects are assigned to the appropriate oversight process.  High-risk projects, like the EFDS, should also be included in the Senior Management Dashboard Review process.
  • Recommendation 3:  The CIO should ensure the business case and the information technology investment portfolio are revised to categorize the EFDS project properly and include accurate and consistent information.

IRS management implemented executive oversight and completed several corrective actions in response to our prior audit report.  As of December 8, 2006, the EFDS Project Office reported the project was on schedule and implementation was expected to occur on January 16, 2007.

During this audit, we determined that IRS management implemented executive governance oversight and completed several corrective actions in response to our prior audit report.  For example, the EFDS project was assigned to the Compliance ESC.[6]  Discussion items, actions, and decisions resulting from these meetings are documented in the meeting minutes.  This corrective action addresses Recommendation 1 from the prior audit report.  The System is also included in the Senior Management Dashboard Review.  The System risks, issues, and mitigation strategies identified at the Senior Management Dashboard Review meetings are documented and tracked.  The Senior Management Dashboard Review is attended by one or more executives from the Enterprise Services organization and representatives of the projects under review.  This corrective action partially addresses Recommendation 2 from the prior audit report.  The IRS Commissioner is also briefed monthly on the status of the System activities by the CSC (see Appendix VI for a list of recurring executive briefings).

The IRS stopped the Web EFDS development and is restoring the client-server EFDS for use in PY 2007.  The EFDS Project Office revised the Exhibit 300 to correctly support classifying the System restoration as a steady state project.  On September 11, 2006, the IRS submitted a revised Exhibit 300 which was approved by the Department of the Treasury.  This corrective action addresses prior audit Recommendation 3.  The remaining corrective actions, which are to evaluate other projects and assign them the appropriate oversight, are in process and scheduled to be completed by April 1, 2007.

Continuous executive oversight of a project helps to ensure risks are identified and mitigated.  As of December 1, 2006, the EFDS Project Office reported the project is on schedule and implementation is expected to occur on January 16, 2007.

Electronic Fraud Detection System Restoration Project Management Controls Have Been Improved, but Risks Remain

The Department of the Treasury Publication 84-01, Information System Life Cycle Manual, dated March 2002, states that general standardization of life cycle management ensures systems are developed, acquired, evaluated, and operated efficiently, within prescribed budget and schedule constraints, and are responsive to mission requirements.  In addition, the IRS system development guidelines (currently, the Enterprise Life Cycle - Lite) stipulate that, as part of the information system life cycle management process, project management should identify project risks early and manage them before they become problems.  The risk management process encompasses the identification of risk issues, assessment of risk to define probability and impact, preparation and implementation of risk mitigation and risk contingency plans, and continuous monitoring of those actions to ensure effectiveness.  Risk management is used to ensure critical areas of uncertainty are surfaced early enough to be addressed without adversely affecting cost, schedule, or performance.

In the prior EFDS audit, we reported that the risks were not managed effectively; status meetings with stakeholders were held, but the meeting results were not documented sufficiently, if at all; individuals were not held accountable for timely completion of tasks; a process to adequately and independently confirm the completion of tasks had not been established nor documented; and key management documents were not prepared or properly maintained.  As a result, we made the following recommendations:

·         Recommendation 4:  The CIO should ensure project risks are identified properly and plans are prepared to reduce the risks affecting the successful development of the project.

·         Recommendation 5:  The CIO should ensure the proper system development life cycle methodology is implemented for the EFDS development, based on the types of changes being made to the system.

For the current client-server EFDS restoration project, the EFDS Project Manager monitors the contractor and IRS progress and performance to ensure the project is on schedule.  During the Web EFDS development, the EFDS Project Manager was also the EFDS/Questionable Refund Program Section Chief and performed some of the CSC Contracting Officer’s Technical Representative (COTR) duties.  For the EFDS restoration project, the three duties were assigned to separate individuals.  See Appendix V for a list of the individuals responsible for the EFDS project.  IRS management stated that spreading out the assignments made the project less difficult to manage.  In addition, the CSC no longer maintains the project work breakdown structure (i.e., a list of all tasks and the tasks completion dates required to complete the project timely).  Instead, it is maintained by Booz Allen Hamilton, Inc. which is providing program management support to the EFDS Project Office.  The project tasks are divided into shorter manageable increments to facilitate task monitoring, validation, and inclusion on the project schedule.

Improvements in project management controls include holding regular meetings with stakeholders and contractors to ensure tasks are on target for timely completion and risks are addressed.  If tasks are not completed when scheduled, the impact on the overall schedule is determined and remedial actions are taken, if needed.  Stakeholder involvement ensures that activities and decisions adequately address the business concerns and completed tasks are satisfactory.  Examples of meetings held include the Weekly Stakeholder Status Meetings, the Weekly Technical Meetings, and the Filing Season Readiness Meetings (see Appendix VI for a list of recurring meetings).

EFDS project management and risk identification and mitigation have improved.  For example, if tasks are not completed when scheduled, the impact on the overall schedule is determined and remedial actions are taken, if needed.

In addition, a process was established, documented, and implemented to monitor the status and verify the satisfactory completion of tasks.  Risks and mitigation activities discussed during the weekly stakeholder status meetings are documented in status reports and/or meeting minutes and a database maintained by Booz Allen Hamilton, Inc.  This corrective action and the executive oversight discussed above address prior audit Recommendation 4.

The Enterprise Life Cycle Project Office also performed an analysis to determine what Enterprise Life Cycle - Lite documents should be produced for a steady state project.  On October 30, 2006, the Compliance ESC gave the EFDS Project Office approval to limit the Enterprise Life Cycle documents to five required documents (Business Systems Requirements Report, Requirements Traceability Matrix, Test Plan, Transition Management Plan, and 508 Compliance).  Figure 1 provides the status of the EFDS Project Office’s preparation of the documents as of October 31, 2006.  This corrective action addresses prior audit Recommendation 5.

Figure 1:  Status of Enterprise Life Cycle Documents

Document Name

Preparation Status

Business Systems Requirements Report

Completed

Requirements Traceability Matrix

Completed

Test Plan

Completed

Transition Management Plan

In Planning

508 Compliance

In Process

Source:  EFDS Project Office.

As previously stated, the EFDS Project Office contracted with Booz Allen Hamilton, Inc. to provide program/project management support for the restoration of the client-server EFDS application.  The estimated cost of this program management support is $1,201,378.  The contract states Booz Allen Hamilton, Inc. employees will:

·         Help to accurately monitor and timely report risks, issues, project status, and action items.

·         Provide technical architecture support to help assess technical issues caused by the PYs 2006 and 2007 changes.

·         Maintain the work breakdown structure and enter start and completion dates into the schedule.  When a completion date has not been met or it appears that a completion date will not be met, contractor support determines the effect of the delay on other tasks, the overall schedule, and the stakeholders.

MITRE was also hired as the IRS’ Federally Funded Research and Development Center to perform two independent studies of the System.  The first study, dated June 9, 2006, cost the IRS $417,730 and determined the root causes of the Web EFDS performance issues and recommended actions to address those issues.  The study also assessed the EFDS Web Portal system, rendered an opinion on its future viability, and recommended actions to apply the lessons learned from the System situation across the information technology portfolio to improve the delivery of other projects of similar size, scope, and complexity.  MITRE stated the Web EFDS application and database were good products and with additional work, could be implemented.  Due to the focus on the System restoration, the EFDS Project Office only implemented recommendations that would help in the System restoration efforts and that could be done quickly.  Decisions on whether the other recommendations will be implemented have not been made.

The second MITRE study, dated October 5, 2006, will cost an estimated $103,024 and assessed the client-server EFDS’ readiness to successfully perform refund fraud detection functions in PY 2007.  MITRE issued a preliminary assessment stating the project is on a path for successful implementation and there were no significant issues or risks that would prevent delivery of a functioning system by January 2007.  MITRE planned to reassess EFDS project readiness on or after November 17, 2006, after PY 2006 data loads were completed and after the Criminal Investigation Division completed its data quality reviews.  However, on December 6, 2006, the EFDS Executive advised us the IRS will not be inviting MITRE to perform another readiness assessment because the project is on schedule and he did not want to subject the EFDS Project Office to another third-party review as it would not provide any new information.

The IRS will spend an estimated $1,722,132 for Booz Allen Hamilton, Inc. project management support and MITRE independent assessments.  These expenses are considered inefficient use of resources because the expenses would not have been incurred if the Web EFDS had been implemented in PY 2006 (see Appendix IV).

Overall, oversight of the client-server EFDS restoration project has improved because management implemented effective project management controls and completed several corrective actions in response to our prior audit report.  However, as of December 8, 2006, risks remained as several critical tasks had not been completed.

·         The EFDS (applications and 3 years of data) must be loaded into the production environment.  The planned completion date is December 29, 2006.

·         Final integration testing must be completed.  The planned completion date is December 29, 2006.

·         Security Certification and Accreditation must be completed.  The planned completion date is January 8, 2007.

·         Disaster recovery testing will not be performed prior to the January implementation.  It is scheduled to occur after the filing season.  The tentative test date is September 2007 and this test is included in a broader IRS disaster recovery test.

As a result of improved project management, risks identified thus far have been mitigated and the System restoration is on schedule for the January 16, 2007, implementation.

This audit was conducted while the IRS was performing restoration activities to implement the System in PY 2007.  Any changes that occurred since we completed our analysis in December 2006 are not reflected in this report.  As a result, this report may not reflect the most current status of the EFDS project.  According to the IRS, the System was placed into production on January 16, 2007.

Contracting Activities Have Improved, but a Cost Reimbursement Issue Remains

The Federal Acquisition Regulation[7] holds contractors responsible for timely contract performance; however, the Federal Government is also responsible for monitoring contractor performance, as necessary, to protect its interest.  This monitoring should include comparing a contractor’s performance plans, schedules, controls, and processes against the contractor’s actual performance; determining the contractor’s progress; and identifying any factors that may delay performance.  Agencies are also required to develop quality assurance surveillance plans when acquiring services specifying the work requiring surveillance and the method of surveillance.  The IRS Office of Procurement Policy best practices state that a planned surveillance effort is necessary to measure contractor performance and ensure successful completion of tasks.

Contracting Officers are responsible for ensuring performance of all necessary actions for effective contracting, ensuring compliance with the terms of the contract, and safeguarding the interests of the Federal Government in its contractual relationships.  Since many of the Contracting Officers’ responsibilities can be delegated to a COTR, the COTR plays a critical role in the technical administration of Federal Government contracts to assure that the Government receives the supplies or services in accordance with the contracts’ specifications.  COTR responsibilities usually include monitoring contractor performance and schedule; acknowledging receipt of supplies or services with an acceptance certificate; reviewing, commenting, and accepting or rejecting deliverables, as well as providing written evaluation of each major deliverable; and reviewing and verifying the contractor’s invoices for hours expended and costs incurred.

While contracting officials should always check the mathematical accuracy of invoices to avoid any overpayment to the contractor, cost-reimbursement contracts require a more indepth review of invoices to ensure costs are not incurred prematurely and relate to progress under the contract.  As a result, COTR activities should include checking the invoice date against the contract performance period to ensure costs are being billed for the proper time period; comparing the contractor’s billing rates against the contract rates to ensure indirect costs are being properly billed; reviewing the contractor’s time cards, sign-in sheets, and overtime records to help assess the reasonableness of direct costs; and maintaining monthly reports or spreadsheets on costs incurred against the contract amount.

****3(d)**** we made the following recommendations:

·         Recommendation 6:  The CIO should ensure contractors are accountable for performance by developing performance-based requirements for new EFDS contracts.  The CIO should also consider employing cost-sharing arrangements for future task orders so both the IRS and contractor share the risk of project development cost overruns.

·         Recommendation 7:  The CIO should ensure COTRs are trained adequately and their duties are performed properly to monitor contractor performance effectively through planned surveillance efforts and independent inspections of contractor work, as described by IRS Office of Procurement Policy best practices.

·         Recommendation 8:  The CIO and the Director, Procurement, should initiate discussions with the contractor to recover the funds paid to the contractor to restore the old EFDS for use in PY 2005 and any additional costs resulting from nondelivery of a functional Web EFDS.

·         Recommendation 9:  The CIO should defer additional work on the Web EFDS until the IRS decides who will perform the EFDS work.  If some or all of the work will transfer to other business units, the CIO should ensure their requirements are identified before initiating a contract for further development of the Web EFDS.  The contract should be opened to competition.

COTR oversight has not significantly changed, but compensating controls mitigate the risks

****3(d)**** We determined that COTR oversight has not changed significantly.  As in the prior audit, the new COTR attends meetings with the contractors but still depends on EFDS Project management to provide confirmation of the status of tasks and receipt and acceptability of deliverables.  The EFDS Project Office is aware of this dependency and has mitigated this risk by obtaining the confirmations from the stakeholders through its weekly System status reporting process.

The COTR now reviews invoices and obtains feedback from the IRS technical points of contact and EFDS Project Office personnel to confirm technical accuracy of deliverables.  However, our review of the controls over the procurement process identified issues similar to those found in the prior audit.  ****3(d)****  EFDS Project Office is in the process of drafting procedures for monitoring acquisitions.  Corrective actions for Recommendation 7 are scheduled to be completed by January 1, 2007.

The equitable adjustment agreement does not ensure the IRS will receive the full amount of the cost reimbursement

The IRS recently issued a Treasury Information Processing Support Services-3 cost-plus-fixed-fee contract that established a base period of performance of November 1, 2006, through February 24, 2007, for EFDS restoration work at an estimated cost of $3,080,004.  The IRS reported in the Joint Audit Management Enterprise System that this contract award completed the corrective action for Recommendation 6 and originally stated that a percentage of the contractor’s fee would be dependent upon timely delivery of specified milestones.  The Joint Audit Management Enterprise System was updated subsequently to state that a percentage of the contractor’s fee was associated with specific deliverables.  However, we reviewed the signed contract and found that payment of the contractor’s fee was not dependent on the timely delivery of EFDS milestones or specific deliverables, and the contract did not include milestones.  As a result, the Federal Government’s interest is not protected because it would be obligated to pay the contractor’s fee if a functional EFDS is not implemented timely.  Regarding Recommendation 6 to use performance-based contracts, the IRS stated that future contracts for completion of the Web EFDS will be performance-based.

Based on the contract and the remaining CSC work identified in the work breakdown structure, the Federal Government may not receive the full equitable adjustment.  However, the EFDS Executive stated the CSC has verbally agreed to work on additional application changes (unrelated to the restoration work) to ensure the IRS receives the $3,080,004 equitable adjustment.

The contract also established a cost sharing amount not to exceed $3,080,004 ($2,859,253 cost reimbursement amount and $220,751 fee) as an equitable adjustment amount.  The CSC agreed to credit each invoice submitted to the IRS for work performed during the base period of performance for the cost incurred plus a fee.  However, the agreement did not include a provision that would refund the unused equitable adjustment to the IRS.  The equitable adjustment was included in the contract as a response to Recommendation 8 from the prior audit report.

Based on our October 25, 2006, meeting with the EFDS Project Manager and our review of the work breakdown structure, most of the CSC’s work was completed by October 2006.  Thus, it does not appear the CSC has $3,080,004 of work remaining.  This is also supported by the EFDS Executive’s August 3, 2006, comment to CSC and MITS executives, “Since much of the cost for restoring the EFDS will likely have been incurred before this agreement is finalized, some of the CSC’s cost sharing will likely be in force after the restoration is complete and the EFDS is in operations and maintenance.”  On December 6, 2006, the EFDS Executive agreed with our conclusion and explained that, if the contract had been signed timely, this would not have been a problem.  The EFDS Executive stated the CSC was aware of this and has verbally agreed to work on two application changes (unrelated to the restoration work) to ensure the IRS will receive the $3,080,004 equitable adjustment.  However, the contract states the CSC’s cost sharing commitment is related exclusively to delivering a client-server-based System and will not apply to any Federal Government directed scope increases.  Again, the Federal Government’s interest has not been protected because the CSC could bill the IRS for the work that is unrelated to the System restoration without crediting the IRS for the unused equitable adjustment.

Recommendation

Recommendation 1:  The CIO should work with the Director, Procurement, to ensure the IRS receives all of the $3,080,004 equitable adjustment from the CSC.  If there is not enough work to be completed by the CSC during the November 1, 2006, through February 24, 2007, period of performance to enable the IRS to receive the full adjustment, the IRS should request that the CSC pay the IRS the difference between the $3,080,004 and the credit the IRS received during the period of performance.  Alternatively, the IRS should request the application of the remaining equitable adjustment credit owed to the IRS to invoices for future EFDS-related task orders or for other work being performed by the CSC.

Management’s Response:  IRS management agreed with the recommendation and prepared a modification to the task order to ensure the IRS receives the full equitable adjustment.  The modification, signed by the IRS and the CSC on February 23, 2007, extends the base period of performance and includes additional work within the scope of the cost sharing agreement.

 

Appendix I

 

Detailed Objective, Scope, and Methodology

 

The overall objective of this review was to determine whether the IRS adequately monitored the contractors’ development efforts in 2006 to ensure the EFDS[8] was delivered in time for the 2007 Filing Season.  To accomplish our objective, we:

I.                   Determined whether the executive monitoring and project management processes were effective to ensure 2007 Filing Season implementation.

A.    Obtained and reviewed the minutes and briefing materials for the Compliance ESC and Senior Management Dashboard Reviews; the monthly briefings presented by the CSC (i.e., the PRIME contractor) to the IRS Commissioner; the MITS and Criminal Investigation Division Business Performance Reviews; the Enterprise Life Cycle Gap Analysis; and the results of the CIO’s program review of the EFDS project.

B.     Determined the effectiveness of project management controls.

1.      Interviewed EFDS Project Office management to determine how they monitored contractor progress and performance.  We also obtained and reviewed status reports and project schedules used to monitor contractor progress and performance and determined whether the status reports documented when critical problems occurred initially, when they were elevated for resolution, and how management validated the accuracy of the schedule.

2.      Obtained and reviewed minutes of the monthly meetings between the Compliance Domain Director, EFDS Project Manager, Criminal Investigation Division representative, and CSC personnel to determine the issues and related resolutions that were discussed.

3.      Determined what risks were identified and whether risk mitigation plans were prepared.

4.      Interviewed Criminal Investigation Division management to determine whether they needed the System in advance to prepare and conduct their training.

5.      Interviewed EFDS Project Office and Criminal Investigation Division management to determine what contingency plans were developed to minimize the effect to the Questionable Refund Program in the event the client-server EFDS was not implemented timely or with full functionalities.

6.      Interviewed the IRS employees responsible for conducting the System Acceptability Testing for the restored EFDS to determine the status and results of testing as well as whether the contractor submitted quality products.

C.     Identified the contractor support that was obtained to assist the EFDS Project Office in the System restoration.

1.      Interviewed EFDS Project Office management to identify the contractors and the scope of their work on the restoration.

2.      Obtained the contracts/task orders/statements of work for the restoration efforts to determine the scope of work and restoration costs for each contractor and reviewed the CSC contract/task order/statement of work to determine the amount the IRS would receive as an equitable adjustment for the Web EFDS not being implemented in 2006.

3.      Validated the invoice amounts supplied by the COTR by comparing the invoice to information in the IRS’ Web Request Tracking System.

4.      Reviewed the MITRE report assessing the System restoration efforts to determine the effect, if any, on our audit work.

II.                Determined whether the COTRs for the contracts and task orders were effectively monitoring and documenting the contractors’ progress and performance on the System restoration work.

A.    Obtained and reviewed policies and procedures for monitoring contractor progress and performance.

B.     Interviewed the COTRs and identified their process for monitoring the contractors and performing independent inspections to ensure the work was on schedule and met the contract terms and user requirements.  We also obtained and reviewed documentation of independent inspections, if performed.

C.     Obtained and reviewed status reports and minutes of meetings between the COTRs and contractors working on the EFDS project, if taken.

III.             Determined whether effective corrective actions were implemented to address the recommendations in the prior EFDS audit report, The Electronic Fraud Detection System Redesign Failure Resulted in Fraudulent Returns and Refunds Not Being Identified (Reference Number 2006-20-108, dated August 9, 2006) and the MITRE report, Electronic Fraud Detection System (EFDS) Project Final Assessment Report Version 1.0, dated June 9, 2006.

A.    Reviewed the Joint Audit Management Enterprise System to determine the status of the corrective actions.

B.     Obtained documentation to verify closed corrective actions were implemented.

C.     Interviewed the EFDS Project Manager to determine the IRS’ decision on implementing the MITRE recommendations (e.g., the number of recommendations agreed to, implemented, rejected, etc.).

 

Appendix II

 

Major Contributors to This Report

 

Margaret E. Begg, Assistant Inspector General for Audit (Information Systems Programs)

Gary Hinkle, Director

Danny Verneuille, Audit Manager

Tina Wong, Lead Auditor

Phung-Son Nguyen, Senior Auditor

Van Warmke, Senior Auditor

Olivia DeBerry, Auditor

Linda Screws, Auditor

 

Appendix III

 

Report Distribution List

 

Commissioner  C

Office of the Commissioner – Attn:  Chief of Staff  C

Deputy Commissioner for Operations Support  OS

Deputy Commissioner for Services and Enforcement  SE

Chief, Agency-Wide Shared Services  OS:A

Chief, Criminal Investigation  SE:CI

Deputy Chief Information Officer  OS:CIO

Deputy Chief, Criminal Investigation  SE:CI

Associate Chief Information Officer, Applications Development  OS:CIO:AD

Director, Procurement  OS:A:P

Director, Refund Crimes  SE:CI:RC

Director, Stakeholder Management  OS:CIO:SM

Chief Counsel  CC

National Taxpayer Advocate  TA

Director, Office of Legislative Affairs  CL:LA

Director, Office of Program Evaluation and Risk Analysis  RAS:O

Office of Internal Control  OS:CFO:CPIC:IC

Audit Liaisons:

Deputy Commissioner for Operations Support  OS

Deputy Commissioner for Services and Enforcement  SE

Chief, Agency-Wide Shared Services  OS:A

Director, Procurement  OS:A:P

Director, Program Oversight Office  OS:CIO:SM:PO

 

Appendix IV

 

Outcome Measures

 

This appendix presents detailed information on the measurable impact that our recommended corrective actions will have on tax administration.  These benefits will be incorporated into our Semiannual Report to Congress.

Type and Value of Outcome Measure:

·         Inefficient Use of Resources – Potential; $1,201,378 (see page 5).

Methodology Used to Measure the Reported Benefit:

The EFDS Project Office has obtained program management support from Booz Allen Hamilton, Inc.  The support that contract employees will provide includes helping to accurately monitor and timely report risks, issues, project status, and action items; providing technical architecture support to help assess technical issues caused by the PYs 2006 and 2007 changes; and maintaining the work breakdown structure.  If the Web EFDS had been implemented in PY 2006, program management support would not have been required.  The estimated cost of the project management support services is $1,201,378.

Type and Value of Outcome Measure:

·         Inefficient Use of Resources – Potential; $417,730 (see page 5).

Methodology Used to Measure the Reported Benefit:

The IRS hired the MITRE to perform a study to determine the root causes of the Web EFDS project performance issues and recommend actions to address those issues, assess the EFDS Web Portal system and render an opinion on its future viability, and recommend actions to apply the lessons from the System situation across the information technology portfolio to improve the delivery of other projects of similar size, scope and complexity.  If the Web EFDS had been implemented timely and successfully in PY 2006, the IRS would not have requested the study which is estimated to cost $417,730.

Type and Value of Outcome Measure:

·         Inefficient Use of Resources – Potential; $103,024 (see page 5).

Methodology Used to Measure the Reported Benefit:

The IRS hired the MITRE to perform a study to assess the client-server EFDS’ readiness to successfully perform refund fraud detection functions in PY 2007.  If the Web EFDS had been implemented timely and successfully in PY 2006, the IRS would not have requested the study which is estimated to cost $103,024.

 

Appendix V

 

Electronic Fraud Detection System
Management

 

Title

Employee’s Name

Date

CIO/Acting CIO

****3(a), 3(d)****

****3(a), 3(d)****

****3(a), 3(d)****

****3(a), 3(d)****

****3(a), 3(d)****

****3(a), 3(d)****

Deputy CIO

****3(a), 3(d)****

****3(a), 3(d)****

Associate CIO, Applications Development

****3(a), 3(d)****

****3(a), 3(d)****

****3(a), 3(d)****

****3(a), 3(d)****

Deputy Associate CIO, Applications Development

****3(a), 3(d)****

****3(a), 3(d)****

****3(a), 3(d)****

****3(a), 3(d)****

Compliance Director/Acting Compliance Director

****3(a), 3(d)****

****3(a), 3(d)****

Chief Enforcement Division/Acting Chief Enforcement Division

****3(a), 3(d)****

****3(a), 3(d)****

EFDS/Questionable Refund Program Section Chief

****3(a), 3(d)****

****3(a), 3(d)****

EFDS Project Manager/Acting EFDS Project Manager

****3(a), 3(d)****

****3(a), 3(d)****

****3(a), 3(d)****

****3(a), 3(d)****

Source:  Meetings with EFDS management and our analysis of MITS organization documents.

 

Appendix VI

 

Electronic Fraud Detection System Oversight

 

Figure 1:  Meetings Attended by the EFDS Project Staff, Stakeholders, Contractors and/or Executives Assigned to Oversee the EFDS Project

Meetings

Frequency

Stakeholder status meetings are held to discuss and analyze the project status and schedule, risks, and risk mitigation strategies.

Weekly

Technical meetings are held to review and propose solutions to technical issues regarding the EFDS restoration effort
(e.g., application change requests, user or system requirements, etc.).

Weekly

The COTRs and contractors meet to discuss the status of the project (e.g., whether work is on schedule and meets the users’ needs).

Bi-weekly

Senior Management Dashboard Review meetings are held to facilitate common understanding of the status of each project among Government and contractor representatives.  Only problem areas or notable status changes are discussed.

Monthly

ESC meetings are held to oversee investments and ensure business risks are known and quantified.

Monthly

Filing Season Readiness meetings are held to discuss the status and issues regarding requests for application changes needed for the filing season.

Weekly - prior to the filing season.
Daily - during the filing season.

Source:  EFDS Project Office and various IRS documents.

 

Figure 2:  Meetings the EFDS Project Office Reported It Provides Project Status Briefings

Meetings

Frequency

Commissioner’s Monthly Meeting

Monthly

Filing Season Executive Meeting

Monthly

Business Performance Reviews

Quarterly

Operational Reviews of the Applications Development Domain

Quarterly

Project Status Review

Quarterly

Source:  EFDS Project Office.

 

Appendix VII

 

Electronic Fraud Detection System Project Timeline

 

April 17, 2006

MITS executives and the IRS Commissioner made the decision to restore the client-server EFDS.

April 19, 2006

All system development activities on the Web EFDS were stopped.

May 31, 2006

The System was assigned to the Compliance ESC.

****3(d)****

****3(d)****

June 27, 2006

The Senior Management Dashboard Review began including the EFDS project in its reviews.

July 6, 2006

The IRS approved a task order for Booz Allen Hamilton, Inc. for the period July 2, 2006, through July 1, 2007, with an estimated cost of $1,201,378 for providing project office support.

July 28, 2006

The IRS approved a task order for Systems Research and Applications Corporation for the period August 1, 2006, through July 31, 2007, with an estimated cost of $420,648 for providing and maintaining data-mining techniques used by the System.

August 15, 2006

The IRS approved a task order for Anteon Corporation for the period
April 11, 2006, through February 24, 2007, with an estimated cost of $1,500,000
for maintenance of the EFDS client-server application and the database supporting the application.

September 14, 2006

The IRS approved a modification to an existing task order for the MITRE for the period July 3, 2006, through December 31, 2006.  The modification had an estimated cost of $103,024 for independent assessments of the restoration activities.

October 24, 2006

The IRS approved a task order under the Treasury Information Processing Support Services 3 contract for the CSC for the period November 1, 2006, through February 24, 2007, with an estimated cost of $3,080,004 for delivering a fully operational client-server-based EFDS for PY 2007.

November 6, 2006

The IRS completed loading the 2006 daily tax return information into the EFDS databases.

December 6, 2006

The IRS completed its test of the System application that will be used in
PY 2007.

December 29, 2006

The IRS is scheduled to complete the loading of the System applications and 3 years of data into the production environment.

January 8, 2007

The EFDS Security Certification and Accreditation is scheduled to be completed.

January 16, 2007

The System is scheduled for implementation in the production environment.

 

Appendix VIII

 

Glossary of Terms

 

Business Case

Required by Office of Management and Budget Circular A-11 (Preparation, Execution, and Submission of the Budget; dated June 2005) and commonly called Exhibit 300, Capital Asset Plan and Business Case.  Each agency must submit a business case twice each year for each major information technology investment.

Client-server

A network architecture in which clients are personal computers or workstations on which users run applications.  Clients rely on servers for resources such as files, devices, and even processing power.

Contracting Officer’s Technical Representative

Furnishes technical direction, monitors contract performance, and maintains an arm’s-length relationship with the contractor.

Cost-Plus-Fixed-Fee Contract

A cost-reimbursement contract that provides for payment to the contractor of a negotiated fee that is fixed at the inception of the contract.  This contract type permits contracting for efforts that might otherwise present too great a risk to contractors, but it provides the contractor only a minimum incentive to control costs.

Cost-Reimbursement Contract

A contract that provides for payment of allowable incurred costs, to the extent prescribed in the contract.

Data Loads

A process of placing data into a system or database.

Data-Mining Technique

A process of automatically searching large volumes of data for patterns.

Enterprise Life Cycle - Lite

A required system development methodology for all nonmodernization projects.

Executive Steering Committee

A committee that oversees investments, including validating major investment business requirements and ensuring that enabling technologies are defined, developed, and implemented.

Federally Funded Research and Development Center

An organization that uses private resources to accomplish tasks that cannot be effectively completed by existing Federal Government employees or contractors.

Filing Season

The period from January through mid-April when most individual income tax returns are filed.

Information Technology Investment Portfolio

A portfolio required by Office of Management and Budget Circular A-11 and commonly referred to as an Exhibit 53.  This portfolio must be submitted with each agency’s annual budget submission and contains the information technology investment title, description, amount, and funding source.

Joint Audit Management Enterprise System

A system used to document and track the status of recommendations from audit reports and their corresponding corrective actions.

MITS Enterprise Governance Committee

The highest level recommending and decision-making body to oversee and enhance enterprise management of information systems and technology.  It ensures strategic modernization and information technology program investments, goals, and activities are aligned with and support  1) the business needs across the enterprise and
2) the modernized vision of the IRS.

MITS Enterprise Governance Committee Investment Management Subcommittee

A body that supports the MITS Enterprise Governance Committee in the realization of the IRS Capital Planning and Investment Control process and with the management of the IRS information technology investment portfolio.  This Subcommittee provides general information technology investment portfolio oversight, including operational analysis reviews and reports, investment prioritization recommendations, and recommendations for adjustments to the IRS portfolio.

Performance-based Contract

A contract that provides for acquiring services on the basis of required results rather than the methods of performing the work and uses measurable performance standards
(e.g., in terms of quality, timeliness, quantity).

Processing Year

The year in which tax returns and other tax data are processed.

Quality Assurance Surveillance Plan

A plan that ensures services provided by the contractor meet contract requirements.  It should specify the work requiring surveillance and the method of surveillance.

Questionable Refund Program Computer Identification Program

An application running on the mainframe computer.  The Program was originally developed by the IRS Inspection Service and run by the Internal Audit function (now the Treasury Inspector General for Tax Administration Office of Audit).

Security Certification and Accreditation

A security certification is an independent technical evaluation, for the purpose of accreditation, that uses security requirements as the criteria for the evaluation.  An accreditation is an authorization granted by a management official to operate the system based on the evaluation of the security controls.

Senior Management Dashboard Review

A review attended by senior executives, contractors, program directors, and project managers to ensure program directors and project managers are held accountable for the project status (e.g., risk, cost, schedule).  Emphasis is placed only on problem areas or notable status changes.

Steady State

Any information technology investment that is fully operational.

System Acceptability Testing

The process of testing a system or program to ensure it meets the original objectives outlined by the user in the requirement analysis document.

Task Order

An order for services placed against an established contract or with Federal Government sources.

Treasury Information Processing Support Services-3

Contracts, awarded in 2006, that provide a broad range of information technology-related services.

Web EFDS

The EFDS development effort allowing users to access the EFDS via the IRS Intranet.

Web Portal

An Internet site or service that functions as a major starting site for users to connect to a broad array of resources and services, such as email, forums, research tools, online shopping malls, etc.

Web Request Tracking System

A web-based application that allows IRS personnel to prepare, approve, fund, and track requests for the delivery of goods and services.  It also allows for electronic acceptance of items delivered and provides an electronic interface with the automated financial system for payment processing.

Work Breakdown Structure

A project schedule used to manage the tasks, task relationships, and resources needed to meet project goals.

 

Appendix IX

Management’s Response to the Draft Report

 

The response was removed due to its size.  To see the response, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.



[1] See Appendix VIII for a Glossary of Terms.

[2] The Electronic Fraud Detection System Redesign Failure Resulted in Fraudulent Returns and Refunds Not Being Identified (Reference Number 2006-20-108, dated August 9, 2006).

[3] See Appendix VIII for a Glossary of Terms.

[4] The Electronic Fraud Detection System Redesign Failure Resulted in Fraudulent Returns and Refunds Not Being Identified (Reference Number 2006-20-108, dated August 9, 2006).

[5] Pub. L. No. 104-106, 110 Stat. 642 (codified in scattered sections of 5 U.S.C., 5 U.S.C. app., 10 U.S.C., 15 U.S.C., 16 U.S.C., 18 U.S.C., 22 U.S.C., 28 U.S.C., 29 U.S.C., 31 U.S.C., 38 U.S.C., 40 U.S.C., 41 U.S.C., 42 U.S.C., 44 U.S.C., 49 U.S.C., 50 U.S.C.).

[6] On October 16, 2006, the MITS Enterprise Governance Committee approved the reconfiguration of the Compliance ESC into the Reporting Compliance and the Filing and Payment Compliance ESCs.  On November 15, 2006, the MITS Enterprise Governance Committee approved keeping the EFDS in the Reporting Compliance ESC until the filing season is complete, then the EFDS will be moved to the Criminal Investigation ESC.

[7] 48 C.F.R. ch. 1 (2005).

[8] See Appendix VIII for a Glossary of Terms.

[9] The EFDS was placed in the Compliance domain in the new Applications Development organization.