TREASURY INSPECTOR GENERAL FOR TAX ADMINISTRATION

 

 

Improvements Are Needed to Ensure Successful Development and System Integration for the Return Review Program

 

 

 

July 26, 2013

 

Reference Number:  2013-20-063

 

 

 

This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.

 

Phone Number  /  202-622-6500

E-mail Address /  TIGTACommunications@tigta.treas.gov

Website           /  http://www.treasury.gov/tigta

 

HIGHLIGHTS

Improvements Are Needed to Ensure Successful Development and System Integration for the Return Review Program

Highlights

Final Report issued on July 26, 2013  

Highlights of  Reference Number:  2013-20-063 to the Internal Revenue Service Chief Technology Officer.

IMPACT ON TAXPAYERS

Based on the fraud it currently detects, the IRS estimates that tax refund fraud is more than $19.2 billion per fiscal year.  The IRS is developing a new Return Review Program (RRP) system to implement the IRS’s new business model for a coordinated criminal and civil tax noncompliance system.  Once developed and implemented, the new system will significantly enhance the IRS’s capabilities to prevent, detect, and resolve tax refund fraud, including identity theft.

WHY TIGTA DID THE AUDIT

The IRS’s current system to detect fraud is the Electronic Fraud Detection System (EFDS).  The IRS determined that the EFDS, which was implemented in 1994, is outdated and would be inefficient to maintain, upgrade, or operate beyond Calendar Year 2015.  Successful implementation of the new RRP system would increase the dollar amount of fraudulent tax refunds identified annually.  TIGTA’s overall audit objective was to determine whether the IRS’s Information Technology Applications Development organization was adequately managing RRP Transition State 1 systems development risks to achieve stated business and information technology requirements.  

WHAT TIGTA FOUND

Roles for program-level governance were not yet established for the RRP and the key role of system integrator was not documented or clearly communicated.  From January to December 2012, prototype activities were conducted to validate that technology product solutions integrated successfully.  However, RRP Prototype Management Plans, critical systems development products, were not completed or approved by major stakeholders before significant resources were committed.  Uncertainty about the systems development path for the RRP and the absence of Enterprise Life Cycle guidance for prototypes hindered initial systems development efforts.  Further, alternative commercial software products were not fully considered prior to selecting technology solutions for the RRP system.

WHAT TIGTA RECOMMENDED

TIGTA recommended that the Chief Technology Officer:  1) establish appropriate program-level governance with enterprisewide authority for the RRP; 2) clearly document the RRP systems integrator roles and responsibilities; 3) complete the RRP Prototype Management Plans, clarify how to measure prototype success, map prototype activities to requirements, incorporate lessons learned, and obtain approval from governance bodies; 4) document, for approval by RRP governance bodies, the decided systems development path; 5) establish sufficient Enterprise Life Cycle guidance for prototypes; and 6) take appropriate steps to ensure that change requests include alternative analyses and impact assessments and also establish and implement Enterprise Architecture guidelines for evaluating later versions of tested commercial products.

In its response, the IRS agreed with our recommendations and reports that it has implemented two corrective actions.  The IRS established two new enterprisewide governance entities to oversee the RRP, and it updated its RRP Prototype Management Plans and individual RRP Prototype Reports for performance measures criteria and relevant functional and performance requirements.  In addition, the IRS plans to document system integrator roles and responsibilities in the RRP Project Management Plan and to document the approved RRP systems development path.  The IRS also plans to update the Internal Revenue Manual with prototype guidance and to develop a process for analyzing and processing Enterprise Architecture Change Requests in a standard, repeatable process.

 

July 26, 2013

 

 

MEMORANDUM FOR CHIEF TECHNOLOGY OFFICER

 

FROM:                       Michael E. McKenney /s/ Michael E. McKenney

                                  Acting Deputy Inspector General for Audit

 

SUBJECT:                  Final Audit Report – Improvements Are Needed to Ensure Successful Development and System Integration for the Return Review Program (Audit # 201220011)

 

This report presents the results of our review of the Internal Revenue Service’s (IRS) efforts on developing the new system for replacing the fraud detection system.  The overall objective of this review was to determine whether the IRS is effectively and efficiently implementing its continuous monitoring tool to monitor security settings on employee workstations and laptop computers.  This audit is included in the Treasury Inspector General for Tax Administration’s Fiscal Year 2013 Annual Audit Plan and addresses the major management challenge of Modernization. 

Management’s written response to the draft report is included as Appendix VI.

Copies of this report are also being sent to the IRS managers affected by the report recommendations.  If you have any questions, please contact me or Alan R. Duncan, Assistant Inspector General for Audit (Security and Information Technology Services).

 

 

Table of Contents

 

Background

Results of Review

Roles for Program-Level Governance Were Not Yet Established and Clarification Is Needed for the Key Role of System Integrator

Recommendation 1:

Recommendation 2:

Return Review Program Prototype Management Plans Were Not Completed or Approved by Major Stakeholders

Recommendation 3:

Uncertainty About the Systems Development Path for the Return Review Program and the Absence of Enterprise Life Cycle Guidance for Prototypes Hindered Initial Systems Development Efforts

Recommendations 4 and 5:

Alternative Commercial Software Products Were Not Fully Considered Prior to Selecting Technology Product Solutions for the Return Review Program

Recommendation 6:

Appendices

Appendix I – Detailed Objective, Scope, and Methodology

Appendix II – Major Contributors to This Report

Appendix III – Report Distribution List

Appendix IV – Timeline – Return Review Program Project Events

Appendix V – Glossary of Terms

Appendix VI – Management’s Response to the Draft Report

 

 

Abbreviations

 

CTO

Chief Technology Officer

EFDS

Electronic Fraud Detection System

ELC

Enterprise Life Cycle

IBM

International Business Machines

IRS

Internal Revenue Service

IT

Information Technology

RRP

Return Review Program

SAS

Statistical Analysis System

TS

Transition State

 

 

Background

 

The Internal Revenue Service’s (IRS) current fraud detection system is the Electronic Fraud Detection System (EFDS).[1]  The IRS has determined that numerous inefficiencies and operational challenges render the EFDS, implemented in Calendar Year 1994, too risky to maintain, upgrade, or operate beyond Calendar Year 2015.  The IRS reports that the long-term limitations of the EFDS include its inability to keep pace with increasing levels of fraud or to serve the organization’s evolving compliance needs. 

The RRP system will implement the IRS’s new business model for a coordinated criminal and civil tax noncompliance approach to prevent, detect, and resolve tax refund fraud, estimated by the IRS to be more than $19.2 billion per fiscal year.

In February 2009, the IRS Commissioner approved a program charter authorizing the formation of the Return Review Program (RRP) under joint leadership provided by the Wage and Investment Division and Criminal Investigation.  The Wage and Investment Division is responsible for RRP requirements development, risk management, governance, project management, and deployment support.  Criminal Investigation is responsible for supporting the RRP by identifying and developing schemes to refer and support high-impact criminal tax and related financial investigations.  In September 2010, the Applications Development organization awarded a cost-plus-incentive-fee contract to International Business Machines (IBM) for leading RRP prototyping activities.

A successful RRP system is critical to the IRS mission.  It will be the key automated component of the IRS’s prerefund initiative.  The RRP system will implement the IRS’s new business model for a coordinated criminal and civil tax noncompliance approach to prevent, detect, and resolve tax refund fraud.  Based on fraud detected by the EFDS and supplemented by manual detection methods, the IRS estimates that tax refund fraud is more than $19.2 billion per fiscal year.  Successful implementation of the new RRP system would increase the dollar amount of fraudulent tax refunds identified annually.  Figure 1 shows that the RRP system will use new data analytics techniques to determine and identify possible noncompliance and fraud.  

Figure 1:  RRP Solution Concept

Figure 1 was removed due to its size.  To see Figure 1, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

Source:  IRS-developed document “RRP Technical Overview,” dated September 14, 2011.

In August 2011, the Wage and Investment Division and the Applications Development organization redirected RRP systems development efforts to include new technology product solutions and to incorporate Patient Protection and Affordable Care Act (Affordable Care Act)[2] functionality into the planned scope for the system.  Benefits of the RRP system include:

This review was performed at the IRS Information Technology (IT) organization’s facilities in New Carrollton, Maryland, during the period May through October 2012.  We conducted this performance audit in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objective.  We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objective.  Detailed information on our audit objective, scope, and methodology is presented in Appendix I.  Major contributors to the report are listed in Appendix II. 

 

 

Results of Review

 

Roles for Program-Level Governance Were Not Yet Established and Clarification Is Needed for the Key Role of System Integrator

In February 2008, the IRS established the Criminal Investigation Executive Steering Committee as the governance body to oversee Criminal Investigation systems development projects.  The Criminal Investigation Executive Steering Committee held monthly governance meetings to discuss RRP topics, approve decisions affecting the RRP, and ensure that the RRP system satisfied milestone exit requirements.  However, the Criminal Investigation Executive Steering Committee does not have the authority to govern enterprisewide and program-level activities pertaining to tax fraud detection, resolution, and prevention, which directly affect RRP systems development.  More specifically, it does not have the authority to ensure that fraud programs stay aligned with the IRS Strategic Plan, resolve enterprisewide issues for fraud programs and projects, and ensure that RRP objectives are met. 

During our audit, the IRS took initial steps, including drafting a proposal, to establish a new executive governance body to oversee both RRP and EFDS fraud initiatives.  IRS management recognizes that a change in governance is needed to provide more accountability and assurance of success for systems development of the RRP system.  However, the necessary governance charter and plan were still undergoing review across IRS business units at the time of our review.

We recognize that the IRS has taken appropriate management action by reconsidering the initial governance structures established for the RRP.  However, during our review, roles for program‑level governance, including an RRP Program Management Office, were not established, and it was not clear how the IRS will govern the systems development process for this mission-critical system, resolve enterprisewide issues for fraud projects and programs, and resolve escalated disputes and issues from fraud projects and programs.  Suitable responsibilities for this program-level governance body include:

Further, roles and responsibilities for integrating the various components of the RRP system during its initial development activities were not well established or clearly communicated as needed.  A systems integrator is a person or company that specializes in bringing together component subsystems into a whole and ensuring that those subsystems function together as intended.  Systems integrator roles and responsibilities often involve managing scope, schedule, staffing, risks, configuration management, testing, procurement, performance, reporting, and release management.  The IRS officials we met with indicated that the Applications Development organization was the systems integrator for the RRP system.  However, neither the September 2012 Project Management Plan for RRP Transition States (TS) 1 through 4 nor the June 2011 RRP TS1 Tailoring Plan stipulated organizational roles and responsibilities for integrating the various component subsystems that must work together for a successful RRP system. 

Further, the IRS September 2010 contract with IBM does not specify or delineate key systems integrator roles and responsibilities.  The contract simply states, “The contractor shall lead the development efforts and coordinate the stand-up of the physical components of the RRP environment.”  As a result, specific systems integrator roles and responsibilities for the development of this mission-critical system have not been clearly established.  Without clearly defined systems integrator roles and responsibilities, there is limited assurance that they will be understood, performed, successfully completed, or appropriately monitored.  Accordingly, there is limited assurance that RRP systems development activities will achieve expected benefits or meet time-sensitive business and information technology requirements for addressing the IRS’s evolving tax refund fraud risks.

Recommendations

Recommendation 1:  The Chief Technology Officer (CTO) should establish appropriate program-level governance with enterprisewide authority for meeting RRP objectives, managing program risks, and ensuring that the expenditure of enterprise resources is fiscally sound. 

Management’s Response:  The IRS agreed with this recommendation.  The RRP was originally governed by the Criminal Investigation Executive Steering Committee.  To provide enterprisewide governance of revenue protection initiatives, the Revenue Protection Technology Governance Board and Executive Steering Committee are two new governance entities that were established and approved by the Information Technology Enterprise Governance Committee on November 29, 2012.

 

Recommendation 2The CTO should clearly document and communicate RRP systems integrator roles and responsibilities.

Management’s Response:  The IRS agreed with this recommendation.  The system integrator roles and responsibilities will be documented in the Project Management Plan.

Return Review Program Prototype Management Plans Were Not Completed or Approved by Major Stakeholders  

The IRS plans to deliver the RRP system incrementally through four TSs as indicated below.

In June 2009, the Wage and Investment Division submitted the initial budget request for the RRP through the Exhibit 300 budget submission process.  Exhibit 300 depicts a required business case for a proposed investment, including expected benefits, costs, and risks for the budget proposal.  In May 2012, the Wage and Investment Division and the Applications Development organization submitted a baseline change request to incorporate new technology products, add the Affordable Care Act requirements into the new system’s scope, and reflect these changes in the RRP’s cost, schedule, and performance goals.  In September 2012, the Department of the Treasury approved the RRP’s revised funding request for $147 million, which included a total of $89 million for TS1 prototyping activities and hardware and software for the RRP solution. 

In September 2010, the Applications Development organization awarded a cost-plus-incentive‑fee contract to IBM for leading prototyping activities for RRP TS1.  According to the contract, “the contractor’s primary tasks will include:  developing the milestone deliverables/work products, providing technical leadership for predictive analytics, business rules, architecture development, physical design, prototype architecture, and subject matter experts during the detailed design phase for the RRP TS1 system.”  As part of the RRP systems development process, IRS prototyping activities were scheduled to occur during the period January to December 2012.  The purpose of the prototyping activities is to successfully integrate the technology product solutions selected for the RRP, assess their performance, and provide confidence that the RRP TS1 architecture and preliminary design will meet the RRP TS1 functional and performance requirements.  The Applications Development organization had established prototype teams and responsibilities for the data, analytics, business rules, Java, reports, and infrastructure teams. 

During our review, the prototype projects were in various stages of completion, and prototype project results were not yet available for our review.  Figure 2 describes the prototypes included in RRP TS1 systems development efforts and identifies the prototype tasks, the products to be prototyped and integrated, and a description of each prototype.

Figure 2:  Description of RRP TS1 Prototypes

Prototyping Task

Products

Task Description

1.      Implement the Physical Data Model for the RRP Database

Greenplum

Convert the logical data model to the physical data model.  Resolve performance issues in the logical data model.  Create the physical RRP database.

2.      Develop Predictive Models

Statistical Analysis System (SAS)

Start developing predictive models for income and withholding, identity theft, and frivolous filer anomaly areas.

3.      Integrate Enterprise Informatica Platform With the RRP Database[3]

Informatica

Greenplum

Extract and transform data from other IRS data sources and load data into the RRP database.  Explore design issues regarding data transfer over wide-area network between IRS Enterprise Computing Centers. 

4.      Integrate Tools to Track and Control Software Changes

ClearCase

Blaze Advisor

Facilitate team software development and setup and integrate software change management and rule management tools. 

5.      Develop End-to-End Application Flow

Informatica

Java

Springbatch

Blaze Advisor SAS

Greenplum

Business Objects

Develop an end-to-end prototype application that flows from the Enterprise Informatica Platform to the RRP database, then to batch processing that executes Java code business rules and predictive models and saves those screening results to the RRP database, and then to the desktop where users can query the results.

6.      Execute SAS As
In-Database Analytics Against the RRP Database

SAS

Greenplum

Publish and execute predictive models using SAS in-database screening.  Assess system performance.

7.      Assess
Day-in-a-Day Performance

Informatica

Java

Springbatch

Blaze Advisor

SAS

Greenplum

Business Objects

Assess the performance of the end-to-end application.

8.      Use SAS Social Network Analysis

SAS

Explore the usage of SAS social network analysis in linked return analysis.  Assess system performance. 

9.      Integrate Rule Management Application With Web Application

Blaze Advisor

Develop rule management application and deploy web application.

10.   Integrate Desktop Reporting With the RRP Database

Business Objects

Greenplum

Develop prototype reports and ad hoc queries.

Source:  RRP TS1 MS4A Prototyping Approach and Strategy.

The purpose of the RRP Prototype Management Plans is to manage the development, execution, and evaluation of these 10 RRP prototype projects.  However, only draft versions of the RRP Prototype Management Plans were completed as of July 2012.  Major RRP stakeholders did not approve these critical systems development life cycle documents before significant resources were committed to the RRP prototype activities. 

Further, our review of the Draft RRP Prototype Management Plans found that the plans did not:

·       Include criteria for measuring the success of the RRP prototypes.

·       Map prototype activities back to RRP TS1 functional and performance requirements.

·       Consider or incorporate lessons learned during other prototypes, such as:

o   Each prototype team must define the detailed expected outcome of the prototype. 

o   Prototype teams need a technical lead assigned to the team.

o   The technical lead needs to have a strong technical applications development background and skills relevant to the platform and products being prototyped. 

o   The prototype time period should allow for analysis and design. 

Senior Applications Development organization project management personnel review and approve key deliverable documents, like the RRP Prototype Management Plans.  The lack of complete and approved RRP Prototype Management Plans could contribute to inefficiencies in the RRP systems development process, increase project risks and cost overruns, and introduce schedule delays. 

During our review, we also considered results from a March 2012 internal evaluation of the Applications Development organization’s systems development processes.  This internal evaluation concluded that a weakness existed in the Applications Development organization’s project planning activities.  More specifically, the evaluation team concluded that it was not evident that all projects were conducting sufficient planning activities by including project planning activities in their project schedules and holding project-planning meetings. 

Recommendation

Recommendation 3:  The CTO should ensure that RRP Prototype Management Plans:

Management’s Response:  The IRS agreed with this recommendation.  RRP TS1 prototype success criteria and measures are outlined in TS1 Architecture Prototype reports and other documentation on the RRP SharePoint site.  The RRP TS1 Milestone 4a Prototyping Approach and Strategy document describes the tasks to achieve the expected prototype results and performance goals.  The strategy document outlines how the RRP TS1 architecture and preliminary design, combined with the prototype using the new technology stack, will meet the RRP TS 1 functional requirements and performance requirements.  By April 26, 2013, technical documentation, including the Prototype Management Plan, was provided to RRP stakeholders and management for review and concurrence.  Prototype development status and results were communicated to project stakeholders.

Uncertainty About the Systems Development Path for the Return Review Program and the Absence of Enterprise Life Cycle Guidance for Prototypes Hindered Initial Systems Development Efforts

The IRS applies its Enterprise Life Cycle (ELC) framework to manage and implement business change through information systems initiatives.  The ELC provides the direction, processes, tools, and assets necessary to accomplish business change through software development in a consistent and repeatable manner.  The objectives of the ELC process are to:

·       Enhance chances for successfully achieving the desired business change. 

·       Standardize the approach for managing and governing business change and supporting information system projects throughout the IRS.

·       Help ensure project success by reducing risk and ensuring compliance with applicable internal and external standards.

To achieve these objectives, the ELC supports multiple software development approaches called paths.  A path is a unique technical or systems engineering approach to accomplish new systems development.  The ELC recognizes five paths for new development projects, covering all phases of development from project initiation through system deployment.  Projects are required to select one or more paths that provide the best fit for their project solution.  The ELC paths include Waterfall, Commercial Off-the-Shelf, Iterative, Joint Application Design, or Rapid Application Development, and Maintenance.

The Waterfall path requires sequential development of a solution with required reviews and formal approvals before continuing work.  These reviews occur within each of the six ELC phases, and approvals must occur at the end of each phase in order to allow project work to continue in the subsequent phase.  The following are characteristics of a Waterfall development path approach:  sequential development, evolving teams, formal documentation, and formal approvals.  The Iterative path is an adaptive development approach in which projects start with initial planning and end with deployment, with repeated cycles of requirement discovery, development, and testing in between.  Iterative development path is fundamentally different from sequential development path approaches, such as Waterfall. 

Based on our review of RRP project documents and discussions with Applications Development organization personnel, the RRP project is using the Waterfall development path.  However, the Criminal Investigation Executive Steering Committee approved the RRP project to use an Iterative development path methodology.  The chronological list of events are: 

This chronology of events indicates uncertainty between the Applications Development organization’s RRP project team and its governance body regarding which development path the RRP project is using and which systems development deliverable documents the RRP project can be held accountable to produce.  Project management risks exist that the RRP project team is not using the development path that the Criminal Investigation Executive Steering Committee intended.  Further, the RRP project team may not be using the development path that the Applications Development organization identified as best suited for the RRP system.

Our review also noted that while the IRS ELC describes the Waterfall development path, there is no ELC guidance for prototyping technology products during systems development following the Waterfall development path.  Absent prototyping guidance, there is limited assurance that RRP prototyping activities were carried out in accordance with management’s intentions.  Based on our review of the RRP systems development, we believe that guidance regarding prototyping products planned and completed within the Waterfall development path is needed to address risks areas including: 

Recommendations

Recommendation 4:  The CTO should document for approval by established RRP governance bodies the decided RRP systems development path.

Management’s Response:  The IRS agreed with this recommendation.  RRP will request formal approval and decision documentation to follow the Waterfall systems development path from the newly established governance body, the Revenue Protection Technology Executive Steering Committee, based on the previous approval obtained from the Criminal Investigation Executive Steering Committee.

Recommendation 5:  The CTO should ensure that the IRS IT organization establishes sufficient ELC guidance for managing prototype efforts during systems development. 

Management’s Response:  The IRS agreed with this recommendation.  The ELC office will update Internal Revenue Manual 2.16.1 to incorporate additional guidance for prototype, pilot, and proof-of-concept efforts.  The guidance will be captured in one particular section within the Internal Revenue Manual instead of multiple sections.

Alternative Commercial Software Products Were Not Fully Considered Prior to Selecting Technology Product Solutions for the Return Review Program

The IRS IT Enterprise Architecture organization maintains an approved software products list referred to as the Enterprise Standards Profile.[4]  The Enterprise Architecture organization’s change request process is intended to ensure that the IRS IT Enterprise Architecture organization reviews and approves all new technology products added to the IRS computing environment.  Adding a product to the Enterprise Standards Profile requires the Applications Development organization to complete a change request, including alternative product analyses and impact assessments.  We reviewed five change requests related to the RRP system and identified the following inconsistencies in two of the five change requests:

Figure 3 lists the hardware and software products approved by the CTO in August 2011 for the RRP TS1–TS4.  It also shows the Enterprise Architecture organization’s approval status of the respective products.

Figure 3:  RRP Technology Hardware/Software Products Being Prototyped

Product Function

Type

Product

Enterprise Standards
Profile Status

Developer

Server Hardware

x86 Servers

Hewlett-Packard ProLiant

Approved 

Hewlett-Packard

Operating System

Linux

Red Hat

Enterprise Linux

Approved

Red Hat Corporation

Database

Massively Parallel Processing

Greenplum

Approved

EMC Corporation

Program Language

Java Virtual Machine

Java

Approved

Oracle Corporation

Data Mining Tools

Predictive Modeling and Linked Return Analysis

SAS Fraud Framework (SAS Enterprise CASE Management

SAS Enterprise Guide

SAS Enterprise Miner

SAS Grid Manager

SAS Forecast Server

SAS Enterprise Business Intelligence Server

SAS Social Network Analysis

SAS Model Manager

SAS Text Miner

Scoring Accelerator for Greenplum Access to Oracle)

In Process 

SAS Institute

Java Application Server

Java 2 Platform Enterprise Edition

JavaBeans Open Source Software Application Server and Development Environment

Approved 

Red Hat Corporation

Batch Processing

Spring Batch

Spring Framework (Spring Batch Component) adds utilities to JavaBeans Open Source Software

The Cybersecurity organization disapproved the change request; however, the Change Control Board accepted the security risks and approved the request. 

EMC Corporation

Business Rules

Business Rules Management System for Predictive Analytics

Fair Isaac Corporation Blaze Advisor Builder

Decision Simulator 

Approved

Fair Isaac Corporation

Extract, Transform, and Load

The Enterprise Informatica Platform is a tool to extract, transform, and load data from existing data sources to application databases

Enterprise Informatica Platform 

Approved

Informatica Corporation

Reports

Web-enabled ad hoc query and data reporting tool

Business Objects

Approved

System Analysis and Program Development

Source:  Obtained from the IRS Enterprise Architecture organization’s Enterprise Standards Profile database as of May 2012.

Again, we considered the Applications Development organization’s March 2012 internal evaluation of its systems development processes.  The evaluation team concluded that a weakness existed in the Applications Development organization’s processes for evaluating alternative products.  More specifically, the evaluation team concluded that it was not clear when alternative analyses were required and how the Applications Development organization should conduct the analyses.  If alternative products are not fully considered as required with complete alternative analyses and impact assessments, the IRS risks buying software and building systems that are duplicative, incompatible, insecure, and unnecessarily costly to integrate and maintain. 

Recommendations

Recommendation 6:  The CTO should take appropriate steps to:

·       Ensure that the RRP project team complies with existing guidance that requires change requests to include alternative analyses.

·       Ensure that the RRP project team complies with existing guidance that requires change requests to include an impact assessment.

·       Establish and implement Enterprise Architecture guidelines for evaluating later versions of tested commercial products included in the Enterprise Standards Profile.

Management’s Response:  The IRS agreed with this recommendation.  The Enterprise Architecture organization’s Standards and Technology Management Team will develop a Data Item Description template for analyzing and processing Enterprise Architecture Change Requests in a standard, repeatable process.  In accordance with the process, the Standards and Technology Management Team will evaluate all requests to add or update products in the Enterprise Standards Profile.

 

Appendix I

 

Detailed Objective, Scope, and Methodology

 

Our overall objective was to determine whether the IRS’s IT Applications Development organization was adequately managing the RRP TS1 systems development risks to achieve stated business and information technology requirements.  To accomplish our objective, we:

I.                 Considered status of key milestones, criteria, and guidance relevant to the RRP TS1 systems development effort.

II.               Evaluated how the Applications Development organization managed RRP project management risks to achieve stated business and information technology requirements.

A.    Systems Development Methodology.  We asked what systems development methodology the Applications Development organization was using.

B.    Governance.  We evaluated if the IRS IT organization had implemented sufficient program-level governance for the RRP. 

C.    Alternative Solutions.  We assessed how the Applications Development organization considered alternative technology product solutions for the design of RRP TS1.

III.             Determined if the Applications Development organization was adequately managing RRP TS1 prototype risks to achieve stated business and information technology requirements.

A.    Internal Revenue Manual Policy and Applicable Guidance.  We assessed the adequacy of existing IRS IT organization guidance regarding initiating and managing prototype projects. 

B.    Prototype Plan.  We assessed if the Applications Development organization had completed the RRP Prototype Management Plans and obtained proper management approval.

C.    Enterprise Architecture.  We reviewed the IRS IT organization’s process for updating the existing enterprise architecture for new systems development technology products.  We assessed if the Applications Development organization had followed stated processes for the RRP.

Internal controls methodology

Internal controls relate to management’s plans, methods, and procedures used to meet their mission, goals, and objectives.  Internal controls include the processes and procedures for planning, organizing, directing, and controlling program operations.  They include the systems for measuring, reporting, and monitoring program performance.  We determined the following internal controls were relevant to our audit objective:  the ELC and related IRS information technology guidelines and the processes followed in the development of information technology projects.  We evaluated these controls by reviewing the guidelines, conducting interviews and meetings with management and staff, and reviewing project documents.

 

Appendix II

 

Major Contributors to This Report

 

Alan Duncan, Assistant Inspector General for Audit (Security and Information Technology Services)

Gwendolyn McGowan, Director, Systems Modernization and Applications Development

Carol Taylor, Audit Manager

Charlene Elliston, Lead Auditor

Andrea Barnes, Senior Auditor

Cari Fogle, Senior Auditor

Sylvia Sloan-Copeland, Senior Auditor

Robert Carpenter, Senior Information Technology Specialist

 

Appendix III

 

Report Distribution List

 

Principal Deputy Commissioner

Office of the Commissioner – Attn:  Chief of Staff  C

Office of the Deputy Commissioner for Services and Enforcement  SE

Deputy Commissioner for Operations Support  OS

Commissioner, Wage and Investment Division  SE:W

Deputy Chief Information Officer for Operations  OS:CTO

Director, Privacy, Governmental Liaison and Disclosure  OS:P

Associate Chief Information Officer, Applications Development OS:CTO:AD

Associate Chief Information Officer, Enterprise Services OS:CTO:ES

Associate Chief Information Officer, Strategy and Planning OS:CTO:SP

Director, Business Performance Solution, Wage and Investment Division  SE:W:BMO:BPS

Chief Counsel CC

National Taxpayer Advocate TA

Director, Office of Legislative Affairs  CL:LA

Director, Office of Program Evaluation and Risk Analysis  RAS:O

Office of Internal Control  OS:CFO:CPIC:IC

Audit Liaison:  Director, Risk Management Division  OS:CTO:SP:RM

 

Appendix IV

 

Timeline – Return Review Program Project Events

 

This appendix provides a detailed timeline for specific RRP system events and milestones that were considered during this review.  

2009

2/2009 - The IRS Commissioner-approved RRP charter.

2010

 

5/1/2010 - Executive Steering Committee approved iterative System Development Life Cycle

5/5/2010 - The RRP TS1 exited Milestone 1 (Project Initiation Phase)

9/2010 - The IRS and IBM entered into a contract for Leading RRP TS1 prototype activities.

2011

 

8/2011 - The CTO approved RRP new technologies.

5/2011 - The Criminal Investigation Executive Steering Committee approved for the RRP project to combine Preliminary Design, Detailed Design, and Development.

8/2011 - The Wage and Investment Division and Application Development Organization redirected scope to include new technologies and Affordable Care Act functionality.

2012

 

1/2012 - The RRP project began prototype activities.

3/2012 - The RRP TS1 Project Plan indicated the project was following the Waterfall path.

7/2012 - The RRP project had not completed or approved TS1 Prototype Management Plans.

9/2012 - The Department of the Treasury approved the RRP revised Fiscal Year 2013 budget request for $147 million.

 

Appendix V

 

Glossary of Terms

 

Term

Definition

Alternative Product Analyses

Process of assessing different products.

Business Case

Required by Office of Management and Budget Circular A-11 (Preparation, Execution, and Submission of the Budget, dated June 2005) and commonly called Exhibit 300, Capital Asset Plan and Business Case.  Each agency must submit a Business Case twice a year for each major information technology investment. 

Change Request

The form for requesting approval to change a baselined product or other controlled item.

Cost-Plus-Incentive-Fee

A type of cost-plus contract in which the fee is based on either cost savings or performance.  It varies according to the level the contractor achieves in meeting such cost or performance criteria.

Electronic Fraud Detection System

The primary information system used to support the IRS Criminal Investigation’s Questionable Refund Program, which is a nationwide program established to detect and stop fraudulent and fictitious claims for refunds on income tax returns.

Enterprise Architecture

A unifying design or structure for an enterprise that includes business and organizational aspects of the enterprise as well as technology aspects.  Enterprise Architecture divides the enterprise into its component parts and relationships and provides the principles, constraints, and standards to help align business area development efforts.  An Enterprise Architecture ensures that subordinate architectures and business system components developed within particular business areas and multiple projects fit together into a consistent, integrated whole.

Enterprise Life Cycle

A structured business systems development methodology that requires the preparation of specific work products during different phases of the development process.

Executive Steering Committee

A committee that oversees investments, including validating major investment business requirements and ensuring that enabling technologies are defined, developed, and implemented.

Fiscal Year

A 12-consecutive-month period ending on the last day of any month. 
The Federal Government’s fiscal year begins on October 1 and ends on September 30.

Impact Assessment

A process aimed at structuring and supporting the development of policies.  It identifies and assesses the problem at stake and the objectives pursued.

Java

A set of several computer software products and specifications from Sun Microsystems (which has since merged with Oracle Corporation) that together provide a system for developing application software and deploying it in a cross-platform computing environment.

Master File

The IRS database that stores various types of taxpayer account information.  This database includes individual, business, and employee plans and exempt organizations data.

Milestone

Scheduled time for providing a “go/no-go” decision point in a program or project. 

Red Hat Enterprise

Linux

A Linux-based operating system developed by Red Hat and targeted toward the commercial market.  Red Hat Enterprise Linux is released in server versions for x86, x86-64, Itanium, PowerPC and IBM System z, and desktop versions for x86 and x86-64.

Stakeholders

An individual or organization that is materially affected by the outcome of the system.  Examples of project stakeholders include the customer, the user group, the project manager, the development team, and the testers.

 

Appendix VI

 

Management’s Response to the Draft Report

 

DEPARTMENT OF THE TREASURY

INTERNAL REVENUE SERVICE

WASHINGTON, D.C. 20224

 

CHIEF TECHNOLOGY OFFICER

 

June 26, 2013

 

 

 

MEMORANDUM FOR DEPUTY INSPECTOR GENERAL FOR AUDIT

 

FROM:                            Terence V. Milholland /s/ Terence V. Milholland

            Chief Technology Officer

 

 

SUBJECT:                       Draft Audit Report - Improvements Are Needed To Ensure Successful Development and System Integration for the Return Review Program (Audit# 201220011) (e-trak # 2013-42284)

 

Thank you for the opportunity to review and respond to the subject audit report.

 

We agree with the recommendations in the report, and appreciate your suggestions for organizational and process improvements to ensure the success of the Return Review Program.  As indicated in the attachment, most of the recommendations were implemented prior to the end of your review.  The IRS is moving forward expeditiously to resolve the outstanding matters, and the attachment describes our planned actions.

 

We value your continued support and the assistance your organization provides.  If you have any questions, please contact me at (202) 622-6800, or a member of your staff may contact Lisa Starr, Senior Manager of Program Oversight, at (202) 283-3607.

 

Attachment

 

RECOMMENDATION #1:  The Chief Technology Officer (CTO) should establish appropriate program-level governance with enterprise-wide authority for meeting RRP objectives, managing program risks, and ensuring the expenditure of enterprise resources is fiscally sound.

 

CORRECTIVE ACTION #1:  The IRS agrees with the recommendation.  The Return Review Program (RRP) Program was originally governed by the Criminal Investigation ESC.  To provide enterprise wide governance of Revenue Protection initiatives, the Revenue Protection Technology (RPT) Governance Board and Executive Steering Committee (ESC) are two new governance entities that were established and approved by the Information Technology Enterprise Governance Committee on November 29, 2012.

 

IMPLEMENTATION DATE:  November 29, 2012

 

RESPONSIBLE OFFICIAL: Associate Chief Information Officer, Applications Development

 

CORRECTIVE ACTION MONITORING PLAN: We enter accepted Corrective Actions into the Joint Audit Management Enterprise System (JAMES) and monitor them on a monthly basis until completion.

 

RECOMMENDATION #2:  The CTO should clearly document and communicate RRP systems integrator roles and responsibilities.

 

CORRECTIVE ACTION #2:  The IRS agrees with the recommendation.  The system integrator roles and responsibilities will be documented in the Project Management Plan.

 

IMPLEMENTATION DATE:  August 25, 2013

 

RESPONSIBLE OFFICIAL:  Associate Chief Information Officer, Applications Development

 

CORRECTIVE ACTION MONITORING PLAN:  We enter accepted Corrective Actions into the Joint Audit Management Enterprise System (JAMES) and monitor them on a monthly basis until completion.

 

RECOMMENDATION #3:

 

The CTO should ensure that RRP Prototype Management Plans:

 

CORRECTIVE ACTION #3:  The IRS agrees with the recommendation.  RRP Transition State 1 (TS1) prototype success criteria and measures are outlined in TS1 Architecture Prototype reports and other documentation on the RRP SharePoint site.  The RRP TS1 Milestone 4a (MS4a) Prototyping Approach and Strategy document describes the tasks to achieve the expected prototype results and performance goals.

 

The RRP TS1 MS4a Prototyping Approach and Strategy document outlines how the RRP TS1 Architecture and Preliminary Design, combined with the prototype using the new technology stack, will meet the RRP TS1 functional requirements and performance requirements.

 

By April 26, 2013, technical documentation, including the Prototype Management Plan, was provided to RRP stakeholders and management for review and concurrence.  Prototype development status and results were communicated to project stakeholders.  Sections 2 and 4 within each Prototype Report detail prototype objectives and results which clarify how to measure the success of the RRP TS1 prototypes.  Section 2.1 “Prototype Objectives” within each Prototype Report, documents RRP TS1 functional and performance requirements which map back to and address key RRP TS1 functional and performance requirements.  Section 5.4 within each Prototype Report captures recommendations and incorporates IRS prototype lessons learned.

 

IMPLEMENTATION DATE:  April 26, 2013

 

RESPONSIBLE OFFICIAL: Associate Chief Information Officer, Applications Development

 

CORRECTIVE ACTION MONITORING PLAN:  We enter accepted Corrective Actions into the Joint Audit Management Enterprise System (JAMES) and monitor them on a monthly basis until completion.

 

RECOMMENDATION #4:  The CTO should document for approval by established RRP governance bodies the decided RRP systems development path.

 

CORRECTIVE ACTION #4:  The IRS agrees with the recommendation.  RRP

will request formal approval and decision documentation to follow the waterfall systems development path from the newly established governance body, the Revenue Protection Technology (RPT) Executive Steering Committee, based on the previous approval obtained from the Cl Executive Steering Committee.

 

IMPLEMENTATION DATE:  July 17, 2013

 

RESPONSIBLE OFFICIAL:  Associate Chief Information Officer, Applications Development

 

CORRECTIVE ACTION MONITORING PLAN:  We enter accepted Corrective Actions into the Joint Audit Management Enterprise System (JAMES) and monitor them on a monthly basis until completion.

 

RECOMMENDATION #5:  The CTO should ensure the IRS IT organization establishes sufficient ELC guidance for managing prototype efforts during systems development.

 

CORRECTIVE ACTION #5:  The IRS agrees with the recommendation.  The Enterprise Life Cycle (ELC) office will update Internal Revenue Manual 2.16.1 to incorporate additional guidance for prototype, pilot and proof-of-concept efforts.  The guidance will be captured in one particular section within the IRM instead of multiple sections.

 

IMPLEMENTATION DATE:  January 25, 2014

 

RESPONSIBLE OFFICIAL:  Associate Chief Information Officer, Strategy and Planning

 

CORRECTIVE ACTION MONITORING PLAN:  We enter accepted Corrective Actions into the Joint Audit Management Enterprise System (JAMES) and monitor them on a monthly basis until completion.

 

RECOMMENDATION #6:  The CTO should take appropriate steps to:

·        Ensure that the RRP project team complies with existing guidance that requires change requests to include alternative analyses.

·        Ensure that the RRP project team complies with existing guidance that requires change requests to include an impact assessment.

·        Establish and implement Enterprise Architecture guidelines for evaluating later versions of tested commercial products included in the Enterprise Standards Profile.

 

CORRECTIVE ACTION #6:  The IRS agrees with the recommendation.  The Enterprise Architecture (EA) Office, Standards and Technology Management (S&TM) Team will develop a Data Item Description template for analyzing and processing EA Change Requests (CR) in a standard, repeatable process.  In accordance with the process, S&TM will evaluate all requests to add or update products in the Enterprise Standards Profile.

 

An alternative analysis is a mandatory step for the CR initiator before a CR request is evaluated.  This evaluation includes a review of existing products that are of similar nature and in the same Service area of the Technical Reference Model for overlap/ duplication.  All the analysis documentation, including impact assessments, are currently tracked within the Change Request Tracking System and will be managed with the Enterprise Change Management System in the future. The IT Change Control Board will disposition a CR for each process when it receives a defined minimum number of impact assessments to the CR.

 

IMPLEMENTATION DATEAugust 25, 2013

 

RESPONSIBLE OFFICIALAssociate Chief Information Officer, Enterprise Services

 

CORRECTIVE ACTION MONITORING PLANWe enter accepted Corrective Actions into the Joint Audit Management Enterprise System (JAMES) and monitor them on a monthly basis until completion.



[1] See Appendix V for a glossary of terms.

[2] Pub. L. No. 111-148, 124 Stat. 119 (2010) (codified as amended in scattered section of the U.S. Code), as amended by the Health Care and Education Reconciliation Act of 2010, Pub. L. No. 111-152, 124 Stat. 1029.

[3] Informatica is an enterprisewide tool used by the IRS for extracting data from existing systems and loading into new repositories.

[4] Internal Revenue Manual 2.15.1.3.5.1 (Aug. 1, 2003) pertains to the Enterprise Standards Profile and identifies approved products and guidelines applicable to the IRS’s target architecture.