The Internal Revenue Service Should Ensure the Root Causes of Business Systems Modernization Performance Problems Are Successfully Addressed

 

December 2004

 

Reference Number:  2005-20-014

 

 

This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.

 

 

December 3, 2004

 

 

MEMORANDUM FOR CHIEF INFORMATION OFFICER

 

FROM:     Gordon C. Milbourn III /s/ Gordon C. Milbourn III 

                 Acting Deputy Inspector General for Audit

 

SUBJECT:     Final Audit Report - The Internal Revenue Service Should Ensure the Root Causes of Business Systems Modernization Performance Problems Are Successfully Addressed (Audit # 200420033)

 

This report presents the results of our review of the Business Systems Modernization (BSM) Challenges Plan.  The overall objectives of this review were to determine whether the Internal Revenue Service (IRS) had taken actions to address the recommendations made by various studies during Calendar Year 2003 for improving the BSM program and had developed a plan to measure the success of these actions.

In summary, the IRS is currently engaged in the BSM program to modernize its systems and associated processes.  All of the BSM projects initiated to this point have experienced cost overages and schedule delays.  In mid-2003, the IRS and the PRIME contractor initiated four studies to help identify the root causes of the problems hindering the BSM effort and make recommendations to remedy the problems identified.

Key IRS executives and stakeholders reviewed the results of the four studies, quickly acknowledged BSM shortcomings, and developed actions to address the studies’ recommendations and resolve longstanding BSM issues.  Collectively, these actions became known as the BSM Challenges Plan.

However, we determined the studies’ recommendations were not fully addressed by the BSM Challenges Plan; no measurement plan was created to determine if the actions taken resulted, or will result, in actual improvements in the BSM program; and many BSM Challenges Plan actions were closed before all significant activities were completed.

Upon presenting our preliminary results to the IRS, we were informed the Associate Chief Information Officer (ACIO), Modernization Management, met with and interviewed a number of individuals in the IRS business units, the Treasury Inspector General for Tax Administration (TIGTA), the Government Accountability Office (GAO), and the Office of Management and Budget.  The ACIO also reviewed the four studies, as well as various reports prepared by the TIGTA and the GAO.  As a result, the ACIO determined what he believed to be the key barriers to success in the BSM program and created a plan to address these barriers.  This plan includes 7 high-level key focus areas and 16 detailed highest priority initiatives to deal with the root causes that are creating the barriers.

The ACIO, Modernization Management, and his staff also performed an analysis of the BSM Challenges Plan and the activities taken to close each of its 48 actions.  We were provided with documentation showing several BSM Challenges Plan actions have been reopened and others have additional follow-up activities planned; however, IRS officials informed us these new and additional follow-up activities were not being tracked using the Item Tracking, Reporting, and Control (ITRAC) database.

To assist the IRS in improving the BSM program, we recommended the Chief Information Officer (CIO) reevaluate the study recommendations that were not addressed by the BSM Challenges Plan to determine whether corrective actions should be taken, create an overall measurement plan to determine whether activities are leading to improvements in the BSM program, ensure stakeholders understand additional work is necessary to complete actions in the BSM Challenges Plan beyond what was originally reported, and place new activities in the ITRAC database for tracking and closure purposes.

Management’s Response:  The CIO agreed with our recommendations and stated there was a lot of work ahead to continue to mature the BSM program.  However, the CIO pointed to several improvements and accomplishments reflecting improved program performance.  For instance, the CIO stated cost and schedule performance had dramatically improved in 2004 and strides were being made in the hiring of outside experts and aligning BSM workload with IRS and PRIME contractor management capacity.

The CIO also responded to our specific recommendations.  The new ACIO, BSM, has thoroughly reviewed the study recommendations and is actively working many of them.  Justifications have been documented for those study recommendations not being pursued. In addition, the BSM program will be developing measures similar to those outlined in our report.  Also, presentations have been made to the stakeholder community on the status of improvement plans, and the ITRAC database has been updated.  Management’s complete response to the draft report is included as   Appendix IX.

Copies of this report are also being sent to the IRS managers affected by the report recommendations.  Please contact me at (202) 622-6510 if you have questions or Margaret E. Begg, Assistant Inspector General for Audit (Information Systems Programs), at (202) 622-8510.

 

Table of Contents

Background

The Business Systems Modernization Challenges Plan Did Not Address All Study Recommendations

Recommendation 1:

The Business Systems Modernization Challenges Plan Did Not Include Measures to Determine Whether Actions Taken Were Successful

Recommendation 2:

Recommendations 3 and 4:

Appendix I – Detailed Objectives, Scope, and Methodology

Appendix II – Major Contributors to This Report

Appendix III – Report Distribution List

Appendix IV – Study Recommendations Not Addressed or Only Partially Addressed by the Business Systems Modernization Challenges Plan

Appendix V – Enterprise Life Cycle Overview

Appendix VI – Key Focus Areas and Highest Priority Initiatives

Appendix VII – Example Measures for Evaluating Business Systems Modernization Challenges Plan Actions

Appendix VIII – Business Systems Modernization Challenges Plan Actions That Were Closed Prematurely

Appendix IX – Management’s Response to the Draft Report

Background

The Internal Revenue Service (IRS) is currently engaged in an effort, known as Business Systems Modernization (BSM), to modernize its systems and associated processes.  To facilitate the success of its modernization efforts, the IRS hired the Computer Sciences Corporation as the PRIME contractor and integrator for the BSM program and created the BSM Office (BSMO) to guide and oversee the work of the PRIME contractor.

Since its inception, the BSM program has experienced management problems at both the program and project levels, as well as problems between the IRS and the PRIME contractor. All of the BSM projects initiated to this point have experienced cost overages and schedule delays.

For example, by May 2003, the Customer Account Data Engine (CADE) project had missed its deployment date for the third time and was over 2 years behind schedule.  By the summer of 2003, another high profile project, the Integrated Financial System (IFS), was also experiencing significant cost increases and the project team was concerned about potential schedule delays.  

In mid-2003, the IRS and the PRIME contractor initiated the following four studies to help identify the root causes of the problems hindering the BSM effort and make recommendations on remedying the problems identified.

·         CADE Study – The Software Engineering Institute (SEI) conducted a study of the CADE project.

·         PRIME Contractor Internal Study – The Bain and Company conducted an internal study of the PRIME contractor.

·         IRS Root Cause Study – The BSMO conducted an internal study of the BSM program.

·         High-Level Assessment of the IRS Office of Procurement Study – The Acquisition Solutions, Incorporated, conducted a study of the IRS’ acquisition program. 

Key IRS executives and stakeholders reviewed the results of the 4 studies in October 2003 and created 46 actions to address the study recommendations.  These 46 actions collectively became known as the BSM Challenges Plan.  In December 2003, the IRS Oversight Board issued a special report containing 9 recommendations, 2 of which were added to the 46 actions already identified.  The 48 actions were each assigned an implementation deadline of 30, 90, or 180 days and were entered into the Item Tracking, Reporting, and Control (ITRAC) database in order for the IRS to monitor activity on each action.

The IRS selected the Deputy Commissioner, Tax Exempt and Government Entities (TE/GE) Division, to oversee the implementation of the actions, beginning on November 10, 2003.  On May 1, 2004, the Deputy Commissioner, TE/GE Division, issued a close-out report detailing the activities taken to close 44 of the 48 actions plus planned activities and projected closing dates for the remaining 4 actions.  Further oversight responsibilities were then assumed by the Associate Chief Information Officer (ACIO), Modernization Management.

This review was performed at the BSMO facilities in New Carrollton, Maryland, during the period May through September 2004.  The audit was conducted in accordance with Government Auditing Standards.  The scope of this audit did not include verifying the actions taken as part of the BSM Challenges Plan were completed or ensuring completed actions achieved the intended results.  Detailed information on our audit objectives, scope, and methodology is presented in Appendix I.  Major contributors to the report are listed in Appendix II.

This audit was conducted while changes were being made at the BSM program level.  We communicated the results of our fieldwork to the BSMO in August 2004.  Any changes that have occurred since we concluded our analyses are not reflected in this report.  As a result, this report may not reflect the most current status.

The Business Systems Modernization Challenges Plan Did Not Address All Study Recommendations

IRS executives and key stakeholders quickly acknowledged BSM shortcomings and developed actions, as part of the BSM Challenges Plan, to resolve longstanding BSM issues.  However, we determined that the study recommendations were not fully addressed by the BSM Challenges Plan.  In addition, some of the activities undertaken as part of the BSM Challenges Plan did not fully address the actions in the Plan.

Some study recommendations were not addressed and others were only partially addressed by the BSM Challenges Plan

The results of the four studies were presented to key IRS executives and stakeholders on October 9, 2003.  We reviewed the 4 studies and identified 21 BSM-related recommendations.  We matched the 48 actions in the BSM Challenges Plan to the 21 recommendations and determined 9 of the recommendations were adequately addressed by the BSM Challenges Plan, 9 were partially addressed, and 3 were not addressed at all, as shown in Chart 1.  See Appendix IV for a list of these study recommendations.

Chart 1:  Study Recommendations Addressed by the BSM Challenges Plan

 

Chart 1 was removed due to its size.  To see Chart 1, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

For example, the IRS root cause study recommended eliminating funding emergencies by timely requesting spend plan changes.  We could find no mention of this action in either the BSM Challenges Plan or the ITRAC database.  As another example, the PRIME contractor internal study recommended enforcing compliance with requirement definition and program management processes.  Part of this recommendation included increasing formal Enterprise Life Cycle (ELC) training for the IRS and the PRIME contractor. While we could find documented activities for several parts of the overall recommendation, we could not find any documented activities requiring ELC training for either the IRS or the PRIME contractor.  

When we presented our preliminary observations to IRS officials, we were informed the BSM Challenges Plan was done in a somewhat random manner, meaning key IRS executives and stakeholders did not take each study recommendation and generate an action item to address it.  In addition, the ACIO, Modernization Management, later informed us he believed the IRS did not address all recommendations because certain recommendations were deemed lower priority and actions were already underway to address the intent of other study recommendations.  Thus, some study recommendations did not get addressed as part of the BSM Challenges Plan.  Without addressing or creating a plan for addressing all conditions within the studies, the IRS runs the risk of having past problems continue to plague the BSM program.

Additional activities are needed to fully address BSM Challenges Plan actions

We reviewed the activities taken, or scheduled, to close the 48 actions listed in the BSM Challenges Plan Close-Out Report and identified 3 instances where the completion activities did not fully address the action.  The BSM Challenges Plan action title and the gaps we identified follow.

·         Ensure projects strictly follow the ELC and appropriate alternatives within the ELC are selected.  While we identified activities taken to ensure appropriate alternatives within the ELC are selected, we could not find any activities to ensure that projects strictly follow the ELC.

·         Designate business architects for all projects to support business insight throughout the project life.  Instead of designating business architects for each project, the IRS designated business requirements directors for each project.  For example, we determined the CADE business requirements director had no systems architecture experience.  During discussions on a preliminary version of this report, BSMO officials stated it was never their intention to elicit systems architecture assistance from the IRS business units.  Therefore, the IRS decided to perform activities under this BSM Challenges Plan action that would better align with the IRS modernization model.

·         Strengthen the development environment by expanding capacity, greater automating testing [sic], separating IF [infrastructure] development.  This is one of the four actions that remained open when the BSM Challenges Plan Close-Out Report was issued.  Our review of the activities scheduled to close this action identified no activities for automating testing or separating the infrastructure environment.

Management Actions:  The ACIO, Modernization Management, met with and interviewed a number of individuals in the IRS business units, the TIGTA, the Government Accountability Office (GAO), and the Office of Management and Budget (OMB).  The ACIO also reviewed the four studies, as well as various reports prepared by the TIGTA and the GAO.  As a result, the ACIO, Modernization Management, determined what he believed to be the key barriers to success in the BSM program and created a plan to address these barriers.  The ACIO established 7 high-level key focus areas (KFA) and developed 16 detailed highest priority initiatives (HPI) to deal with these root cause problems.  See Appendix VI for a list of the 7 KFAs and the 16 HPIs.  Several of the HPIs are BSM Challenges Plan actions.

Recommendation

To ensure all of the conditions in the studies are addressed, the Chief Information Officer (CIO) should:

1.      Reevaluate the study recommendations that were not completely addressed by the BSM Challenges Plan and determine whether corrective actions should be taken as part of either the 7 KFAs and16 HPIs or future action items.  If it is determined no actions are needed, the reasons should be documented for future reference.

Management’s Response:  The new ACIO, BSM, has thoroughly reviewed the study recommendations and is actively working many of them.  In addition, justifications have been documented for those study recommendations not being pursued.

The Business Systems Modernization Challenges Plan Did Not Include Measures to Determine Whether Actions Taken Were Successful

Management best practices indicate measures should be developed to determine whether actions taken actually result in intended improvements.  While the BSM Challenges Plan included reasonable actions, no measurement plan was created to determine whether the actions taken resulted, or will result, in actual improvements in the BSM program.  In addition, many actions were closed before all significant activities were completed.

The BSM Challenges Plan contained no criteria for measuring the effect of actions 

The BSM Challenges Plan included several management controls such as details of specific activities that needed to be completed, assignment of responsibility for completing specific activities, and planned completion dates.  However, the BSM Challenges Plan Close-Out Report, generally, did not include plans for determining whether the implemented actions were, or would be, effective in meeting their designated goal.

Management expert W. Edward Deming gained widespread attention in the 1980’s after being featured in a National Broadcasting Corporation documentary titled “If Japan Can, Why Can’t We?”  Deming expanded the idea that quality can be measured and developed what is known as the Deming Cycle of Plan-Do-Check-Act (PDCA).  The PDCA cycle is a well-known model for continuous process improvement that includes, 1) Plan:  Design or revise business processes to improve results, 2) Do:  Implement the plan and measure its performance, 3) Check:  Assess the measurements and report the results to decision makers, and 4) Act:  Decide on and implement changes needed to improve the process.

Chart 2: Plan-Do-Check-Act Visual Representation

Chart 2 was removed due to its size.  To see Chart 2, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

The IRS planned the actions that need to be taken for improvement; however, it did not make plans to measure the success of its efforts.  If measures are not employed to determine the effectiveness of BSM Challenges Plan actions, the IRS will only be able to subjectively determine whether the BSM Challenges Plan activities were worthwhile.  In addition, the IRS will have no insight into what activities were successful as it moves forward with future improvement efforts to address the root causes of BSM program challenges.

Our analysis of the BSM Challenges Plan found there were no overall criteria for determining whether taking the actions set forth in the Plan will lead to any improvement, and many activities in the Plan did not lend themselves easily to the use of quantitative measurement.  In addition, most action items were mapped to multiple study recommendations, which made it difficult to understand what study conditions were being addressed. Without the condition to be corrected clearly in mind, it is not feasible to identify suitable measurement activities.

Based on the interviews we conducted with individuals associated with the formulation and implementation of the BSM Challenges Plan, we determined the IRS had not analyzed the BSM Challenges Plan actions to determine what metrics would be used to measure the effectiveness of the actions.  We were advised that the success of BSM Challenges Plan actions could be determined by assessing whether future project cost and schedule estimates were met.

We concluded more detailed measures could also be used to determine which of the multiple challenges within the BSM program were being adequately addressed and which challenges still needed further work.  To assist the IRS with identifying measures for future activities, we have created potential measures that could have been used to measure the short-term and long-term success of BSM Challenges Plan actions.  See Appendix VII for a list of these suggested measurements.

Actions taken under the BSM Challenges Plan were closed prematurely

In addition to not providing measures to determine the overall success of BSM Challenges Plan actions, we determined several actions within the BSM Challenges Plan were closed before significant activities were completed.  The BSM Challenges Plan Close-Out Report listed 48 actions, 44 of which were classified as ‘closed’ and 4 as ‘open.’  This status was reported to the Congress and the press.  We determined 10 of the ‘closed’ actions still had significant work activities to be completed.  Seven of these 10 actions concerned staffing issues (funding, hiring, locating individuals) and 3 concerned completing documentation.  See Appendix VIII for the listing of these 10 actions.

Management Actions:  Upon presenting our preliminary results to the ACIO, Modernization Management, we were informed he and his staff had also performed an analysis of the BSM Challenges Plan and the activities taken to close each of the 48 actions.  We were provided with documentation showing, as of August 10, 2004, some BSM Challenges Plan actions were reopened and others had additional follow-up activities planned (see Chart 3).  However, IRS officials informed us these new and additional follow-up activities were not being tracked using the ITRAC database.

During preliminary discussions of this report, we were also informed the IRS had taken actions after our fieldwork was complete to inform the OMB and the GAO of the remaining work that needed to be performed on the BSM Challenges Plan.

Chart 3:  Comparison of BSM Challenges Plan Actions on May 1, 2004, and August 10, 2004

 

Chart 3 was removed due to its size.  To see Chart 3, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

We agree with the IRS’ action to reopen and add new activities to ensure BSM Challenges Plan actions are truly complete.  Based on this new analysis, 8 of the 10 actions we determined were closed prematurely had new or follow-up activities concerning the areas we identified as needing to be addressed before the actions could be closed.  For the two remaining actions, we provided information to the ACIO, Modernization Management, for his consideration.  See Appendix VIII for further information on these 10 actions.

Recommendations

To ensure the IRS can objectively determine whether process improvement actions are leading to improvement in the BSM program, stakeholders are updated on the BSM Challenges Plan status, and new BSM Challenges Plan activities are monitored and tracked to closure, the CIO should:

2.      Create an overall measurement plan to determine whether the activities in the seven KFAs are leading to improvements in the BSM program by developing specific measurement criteria for each of the HPIs (where possible).

Management’s Response:  The BSM program will be developing measures similar to those outlined in our report. For example, measures concerning cost and schedule variance, requirements stability, and requirements delivered will be implemented in Fiscal Year 2005.

3.      Ensure stakeholders understand additional work is necessary to complete actions in the BSM Challenges Plan beyond what was originally reported.

Management’s Response:  Presentations have been made to the stakeholder community, such as the OMB and the GAO, on the status of improvement plans.

4.      Place new activities in the ITRAC database for tracking and closure purposes based on the recent analysis of the BSM Challenges Plan status.

Management’s Response:  The ITRAC database has been updated with information on new, ongoing, and closed activities.

 

Appendix I

 

Detailed Objectives, Scope, and Methodology

 

The overall objectives of this review were to determine whether the Internal Revenue Service (IRS) had taken actions to address the recommendations made by various studies during Calendar Year 2003 for improving the Business Systems Modernization (BSM) program and had developed a plan to measure the success of these actions.

To accomplish these objectives, we:

     I.      Determined whether the 48 actions in the BSM Challenges Plan fully satisfied the recommendations from the 4 recent studies.

A.    Determined the mission and roles/responsibilities of the BSM Challenges Plan oversight team.

B.     Determined whether the 48 actions were mapped to the recommendations from the 4 recent studies.  

C.     Reviewed the 48 actions to determine whether the activities included under each action completely addressed the related conditions identified in the 4 recent studies.

  II.            Determined whether the IRS identified measures to ensure that conditions noted in the four recent studies were corrected.

A.    Determined whether the IRS identified metrics that could measure the effect of the corrective actions on each study recommendation.

B.     Determined what process the IRS was using to follow up on these actions.

 

Appendix II

 

Major Contributors to This Report

 

Margaret E. Begg, Assistant Inspector General for Audit (Information Systems Programs)

Gary V. Hinkle, Director

Troy D. Paterson, Audit Manager

Paul M. Mitchell, Lead Auditor

Steven W. Gibson, Auditor

Perrin T. Gleaton, Auditor

 

Appendix III

 

Report Distribution List

 

Commissioner  C

Office of the Commissioner – Attn: Chief of Staff  C

Deputy Commissioner for Operations Support  OS

Associate Chief Information Officer, Business Systems Modernization  OS:CIO:B

Deputy Associate Chief Information Officer, Business Integration OS:CIO:B:BI

Deputy Associate Chief Information Officer, Program Management  OS:CIO:B:PM

Deputy Associate Chief Information Officer, Systems Integration OS:CIO:B:SI

Director, Stakeholder Management  OS:CIO:SM

Chief Counsel  CC

National Taxpayer Advocate  TA

Director, Office of Legislative Affairs  CL:LA

Director, Office of Program Evaluation and Risk Analysis  RAS:O

Office of Management Controls  OS:CFO:AR:M

Audit Liaisons: 

Associate Chief Information Officer, Business Systems Modernization  OS:CIO:B

Manager, Program Oversight Office  OS:CIO:SM:PO

 

Appendix IV

 

Study Recommendations Not Addressed or Only Partially Addressed by the Business Systems Modernization Challenges Plan

 

We determined three study recommendations were not addressed as part of the Business Systems Modernization (BSM) Challenges Plan.  We also determined nine study recommendations were only partially addressed by the BSM Challenges Plan.

Study Recommendations Not Addressed by the BSM Challenges Plan

1.      Better allocate responsibility for certain BSM management functions (budget, audits, personnel, contracting).

2.      Use more realistic assumptions for testing estimates/plans/schedules.

3.      Request spend plan changes in a timely manner – eliminate funding emergencies.

Study Recommendations Only Partially Addressed by the BSM Challenges Plan

1.      Enforce compliance for requirement definition and program management processes.  This recommendation was unsupported by any documented activity to require Enterprise Life Cycle (ELC) training for both Computer Sciences Corporation (CSC) and Internal Revenue Service (IRS) personnel.

2.      Clarify and streamline decision authority.  This recommendation was unsupported by any documented activity to explain how to split client management, program management, and technical leadership (CSC and IRS). 

3.      Bolster broadly defined skill areas/personnel (focus on business integration/transition).  This recommendation was unsupported by any documented activities to (1) expand tax administration experts at the CSC at the program level, (2) assign domain experts and change agents to the “Tiger Teams,” and (3) integrate Information Technology Services personnel into PRIME development and testing.

4.      Improve subcontractor accountability and delivery by restructuring subcontractor model.  This recommendation was unsupported by any documented activities to (1) have the CSC conduct technical audits of all current work, (2) require “Tiger Teams” to drive resolution of key decisions at all project levels, (3) have the CSC reinvigorate the escalation process with subcontractor senior management, (4) have the CSC review the conditions/criteria for subcontractor engagement, and (5) have the CSC reinvigorate the ongoing communications process with subcontractors.

5.      Institutionalize Systems Engineering.  This recommendation was unsupported by any documented activities to (1) have the IRS lead an integrated team to define, monitor, manage, and support operational security across the Customer Account Data Engine (CADE), Modernization, and Current Processing Environment; (2) have the IRS and the PRIME contractor define customer requirements or analyze requirements for implications on the CADE architecture and infrastructure; and (3) have the PRIME contractor use prototyping during the development process to validate complex requirements.

6.      Validate Business Rules Engines.  This recommendation was unsupported by any documented activity to have the PRIME contractor explore C++, another high-level language, or other business rules engine approaches as risk mitigation design alternatives.

7.      Institutionalize Management Discipline.  This recommendation was unsupported by any documented activities to have the IRS and the PRIME contractor (1) improve their ability to jointly create realistic cost and schedule estimates and (2) establish mechanisms to independently validate and monitor cost and schedule estimates.

8.      Examine the Office of Modernization Acquisition and centrally locate all essential resources necessary to fully support the PRIME in one organization, headed by a Senior Executive Service Department of Defense (DoD) DAWIA Level 3 System Acquisition Program and Contracting Specialist. This recommendation was unsupported by any documented activity to require the BSM Acquisition Executive to be a DoD DAWIA Level 3 System Acquisition Program and Contracting Specialist. Also, the selection of this individual had not yet occurred.  

9.      Redefine the PRIME contract with the CSC to reflect desired roles.  This recommendation was unsupported by any documented activity to have the IRS redefine the PRIME contract. 

 

Appendix V

 

Enterprise Life Cycle Overview

 

The Enterprise Life Cycle (ELC) defines the processes, products, techniques, roles, responsibilities, policies, procedures, and standards associated with planning, executing, and managing business change. It includes redesign of business processes; transformation of the organization; and development, integration, deployment, and maintenance of the related information technology applications and infrastructure.  Its immediate focus is the Internal Revenue Service (IRS) Business Systems Modernization (BSM) program.  Both the IRS and the PRIME contractor must follow the ELC in developing/acquiring business solutions for modernization projects.

The ELC framework is a flexible and adaptable structure within which one plans, executes, and integrates business change. The ELC process layer was created principally from the Computer Sciences Corporation’s Catalyst^® methodology.  It is intended to improve the acquisition, use, and management of information technology within the IRS; facilitate management of large-scale business change; and enhance the methods of decision making and information sharing. Other components and extensions were added as needed to meet the specific needs of the IRS BSM program.

ELC Processes

A process is an ordered, interdependent set of activities established to accomplish a specific purpose.  Processes help to define what work needs to be performed. The ELC methodology includes two major groups of processes:

            Life-Cycle Processes, which are organized into phases and subphases and which address all domains of business change.

            Management Processes, which are organized into management areas and which operate across the entire life cycle.

 

Enterprise Life-Cycle Processes

The chart was removed due to its size.  To see the chart, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

Life-Cycle Processes

The life-cycle processes of the ELC are divided into six phases, as described below:

·                     Vision and Strategy - This phase establishes the overall direction and priorities for business change for the enterprise.  It also identifies and prioritizes the business or system areas for further analysis.

·                     Architecture - This phase establishes the concept/vision, requirements, and design for a particular business area or target system.  It also defines the releases for the business area or system.

·                     Development - This phase includes the analysis, design, acquisition, modification, construction, and testing of the components of a business solution.  This phase also includes routine planned maintenance of applications.

·                     Integration - This phase includes the integration, testing, piloting, and acceptance of a release.  In this phase, the integration team brings together individual work packages of solution components developed or acquired separately during the Development phase. Application and technical infrastructure components are tested to determine whether they interact properly. If appropriate, the team conducts a pilot to ensure all elements of the business solution work together.

·                     Deployment - This phase includes preparation of a release for deployment and actual deployment of the release to the deployment sites.  During this phase, the deployment team puts the solution release into operation at target sites.

·                     Operations and Support - This phase addresses the ongoing operations and support of the system.  It begins after the business processes and system(s) have been installed and have begun performing business functions.  It encompasses all of the operations and support processes necessary to deliver the services associated with managing all or part of a computing environment.

The Operations and Support phase includes the scheduled activities, such as planned maintenance, systems backup, and production output, as well as the nonscheduled activities, such as problem resolution and service request delivery, including emergency unplanned maintenance of applications.  It also includes the support processes required to keep the system up and running at the contractually specified level.

Management Processes

Besides the life-cycle processes, the ELC also addresses the various management areas at the process level.  The management areas include:

·                     IRS Governance and Investment Decision Management - This area is responsible for managing the overall direction of the IRS, determining where to invest, and managing the investments over time.

·                     Program Management and Project Management - This area is responsible for organizing, planning, directing, and controlling the activities within the program and its subordinate projects to achieve the objectives of the program and deliver the expected business results.

·                     Architectural Engineering/Development Coordination - This area is responsible for managing the technical aspects of coordination across projects and disciplines, such as managing interfaces, controlling architectural changes, ensuring architectural compliance, maintaining standards, and resolving issues.

·                     Management Support Processes - This area includes common management processes, such as quality management and configuration management that operate across multiple levels of management.

Milestones

The ELC establishes a set of repeatable processes and a system of milestones, checkpoints, and reviews that reduce the risks of systems development, accelerate the delivery of business solutions, and ensure alignment with the overall business strategy. The ELC defines a series of milestones in the life-cycle processes.  Milestones provide for “go/no-go” decision points in the project and are sometimes associated with funding approval to proceed.  They occur at natural breaks in the process where there is new information regarding costs, benefits, and risks and where executive authority is necessary for next phase expenditures.

There are five milestones during the project life cycle: 

·                     Milestone 1 - Business Vision and Case for Action.  In the activities leading up to Milestone 1, executive leadership identifies the direction and priorities for IRS business change.  These guide which business areas and systems development projects are funded for further analysis.  The primary decision at Milestone 1 is to select BSM projects based on both the enterprise-level Vision and Strategy and the Enterprise Architecture.

·                     Milestone 2 - Business Systems Concept and Preliminary Business Case.  The activities leading up to Milestone 2 establish the project concept, including requirements and design elements, as a solution for a specific business area or business system.  A preliminary business case is also produced.  The primary decision at Milestone 2 is to approve the solution/system concept and associated plans for a modernization initiative and to authorize funding for that solution.

·                     Milestone 4 - Business Systems Development and Enterprise Deployment Decision.  In the activities leading up to Milestone 4, the business solution is built.  The system is integrated with other business systems and tested, piloted (usually), and prepared for deployment.  The primary decision at Milestone 4 is to authorize the release for enterprise-wide deployment and commit the necessary resources.

·                     Milestone 5 - Business Systems Deployment and Postdeployment Evaluation.  In the activities leading up to Milestone 5, the business solution is fully deployed, including delivery of training on use and maintenance.  The primary decision at Milestone 5 is to authorize the release of performance-based compensation based on actual, measured performance of the business system.

 

Appendix VI

 

Key Focus Areas and Highest Priority Initiatives

 

The Associate Chief Information Officer, Modernization Management, established 7 high-level key focus areas and developed 16 detailed highest priority initiatives to improve the Business Systems Modernization (BSM) program. 

            Key Focus Areas

1.      Staffing and Skills.

2.      Contractor Management.

3.      Requirements and Demand Management.

4.      Systems Engineering.

5.      Project Management Disciplines.

6.      Communication and Collaboration.

7.      Empowerment/Accountability.

 

Highest Priority Initiatives (Completion over 6 months 8/1/2004 - 1/30/2005)

1.      Staffing and Skills.

a.       Strengthen Internal Revenue Service (IRS) system engineering capability through external hiring or leverage of additional [contractor] capabilities.

b.      Strengthen IRS program and project manager cadre.

c.       Complete the human capital strategy (develop detailed recruiting plan).

2.      Contractor Management. 

a.       Assess the elements of the PRIME contractor’s program role and make modifications as required.

b.      Strengthen the negotiation process through establishment of negotiating teams and use of performance-based contracting techniques.

3.      Requirements and Demand Management.

a.       Refine the change request process.

b.      Improve the configuration management practices.

c.       Modify the Modernization and Information Technology Services (MITS) Configuration Management (CM) Directive and procedures . . . ensuring that the MITS organization CM processes are implemented throughout the IRS . . . and coordinated with the BSM Office (BSMO) and that CM deficiencies are appropriately addressed.

d.      Modify the MITS CM Directive and procedures to establish governance policies, similar to those used by the BSMO, for defining the authority level and threshold criteria to approve and control changes to the production environment in the Information Technology Services (ITS) organization.

e.       Create a Requirements Management Office.

f.       Hire/contract for a requirements management lead.

4.      Systems Engineering.

a.       Hire a technical lead for the Program.

b.      Start revalidation of the Customer Account Data Engine (CADE) approach and architecture.

5.      Project Management Disciplines.

a.       Institutionalize BSM program reviews, including ongoing reviews of projects, infrastructure, engineering, process and financial elements.

6.      Communication and Collaboration.

a.       Establish periodic communications between BSM Program leadership to all BSM personnel (including contractors).

b.      Establish an ITS project office to support new application transition to operations.

7.      Empowerment/Accountability.

 

Appendix VII

 

Example Measures for Evaluating Business Systems Modernization Challenges Plan Actions

 

The following table shows a few examples of possible measures that could have been used to measure the effect of actions taken as part of the Internal Revenue Service’s (IRS) Business Systems Modernization (BSM) Challenges Plan.

 

Table 1:  Possible Measures That Could Have Been Used to Measure

the Effectiveness of BSM Challenges Plan Actions

Source: BSM Challenges Plan Close-Out Report and Treasury Inspector General for Tax Administration analysis.

 

Appendix VIII

 

Business Systems Modernization Challenges Plan Actions That Were Closed Prematurely

 

We determined that 10 Business Systems Modernization (BSM) Challenges Plan actions were closed with significant activities remaining.  The Associate Chief Information Officer (ACIO), Modernization Management, and his staff performed an analysis of the BSM Challenges Plan and the activities taken to close each of the 48 actions. We were provided with documentation showing, as of August 10, 2004, some BSM Challenges Plan actions were reopened and others had additional follow-up activities planned.  Based on this new analysis, 8 of the 10 actions we determined were closed prematurely had new or follow-up activities concerning the areas that we identified as needing to be addressed before the actions could be closed. For the two remaining issues, we provided information to the ACIO, Modernization Management, for his consideration.

Table 1:  Ten of the ‘Closed’ Actions That Still Had Significant Work Activities to Be Completed

Source: Treasury Inspector General for Tax Administration analysis.

 

Appendix IX

 

Management’s Response to the Draft Report

 

The response was removed due to its size.  To see the response, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.