TREASURY INSPECTOR GENERAL FOR TAX ADMINISTRATION

 

 

Strengthened Management Processes Are Needed to Assure the Usefulness of Products and Services Received Through the Infrastructure Shared Services Task Order

 

 

 

March 2006

 

Reference Number:  2006-20-063

 

 

This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.

 

Phone Number   |  202-927-7037

Email Address   |  Bonnie.Heald@tigta.treas.gov

Web Site           |  http://www.tigta.gov

 

March 30, 2006

 

 

MEMORANDUM FOR CHIEF INFORMATION OFFICER

 

FROM:                            Michael R. Phillips /s/ Michael R. Phillips

                                         Deputy Inspector General for Audit

 

SUBJECT:                    Final Audit Report – Strengthened Management Processes Are Needed to Assure the Usefulness of Products and Services Received Through the Infrastructure Shared Services Task Order (Audit # 200520018)

 

This report presents the results of our review of the Fiscal Year (FY) 2004/2005 Infrastructure[1] Shared Service (ISS) task order.[2]  The overall objective of this review was to determine whether the Internal Revenue Service (IRS) is receiving full value from the PRIME contractor[3] under the terms and conditions of the FY 2004/2005 ISS task order.

Synopsis

The IRS is currently in the midst of a multiyear, multibillion dollar effort to update its core business systems, known as Business Systems Modernization (BSM).  According to IRS estimates, this effort involves integrating thousands of hardware and software components over 15 years.  The success of the BSM program depends on establishing a strong foundation from which to build business applications supporting tax return processing and other critical mission-related functions.  In February 2002, the Business Systems Modernization Office began building this foundation by establishing the ISS program to install the infrastructure.  These critical infrastructure areas are being relied upon by other modernized projects and systems.  For FYs 2004 and 2005, the IRS issued a task order to the PRIME contractor to work with the IRS to maintain the current infrastructure environment and adapt to new changes and upgrades that are inherent in information technology components.

We determined the IRS has improved its ability to perform detailed reviews of the contractor’s invoices, the PRIME contractor is making corrections to work items as requested by the IRS, and significant work items and special projects provided through the ISS task order are providing value to the IRS.  While value is being received for many of the work items and special projects in the FY 2004/2005 ISS task order, we determined the IRS is not obtaining the anticipated value for some work items and special projects, and it is difficult to determine the value being received for other work items and special projects.

We reviewed a judgmental sample of 27 work items produced by the PRIME contractor and received by the IRS between February 2004 and May 2005 and determined the IRS received the anticipated value for 12 work items and received value for an additional 3 work items as part of different projects.  However, the value received for the remaining 12 work items was more difficult to determine.  Initially, we concluded the IRS did not receive full value for these work items.  Upon meeting with IRS officials in September 2005, December 2005, and January 2006, the IRS provided additional information concerning whether these remaining work items provided value.  The IRS agreed seven work items did not provide the IRS with their anticipated value.  However, the IRS stated it believed that, while the original anticipated value may not have been obtained, the IRS did receive value from the remaining five work items.  In addition, we reviewed all 11 special projects performed under the FY 2004/2005 ISS task order and determined 2 (18 percent) did not provide the anticipated value to the IRS.[4]

The IRS does not individually evaluate the value obtained from specific work items or special projects during the completion of activities within the FY 2004/2005 ISS task order.  We determined the IRS could do more to proactively assess if value is being received from the PRIME contractor.  Currently, the IRS does not have all the information necessary to perform value measurements because 1) users/owners are not documenting the purpose and planned results for work items, 2) the IRS does not have cost estimates for individual work items, and 3) the IRS is not performing quality reviews of deliverables or making overall performance assessments of the PRIME contractor.

The IRS has spent Federal Government resources for Infrastructure Shared Services work items that are not being used.

Recommendations

To ensure users/owners are appropriately identified and actively participate in measuring the value received from work items delivered by the PRIME contractor, we recommended the Chief Information Officer (CIO) implement a consistent process to formally document users/owners and require users/owners to 1) document the purpose, 2) create success criteria, and 3) document whether each work item met the stated purpose and defined success criteria.  To ensure adequate cost estimates are available, we recommended the CIO require the PRIME contractor to provide original and revised (if applicable) cost estimates for all task order activities.  To ensure quality ratings and measurements are performed, we recommended the CIO require all Acquisition Project Managers and Project Account Managers[5] to follow the ISS task order monitoring plan.  Finally, to ensure an overall contractor performance evaluation process is available to create a rating for the entire PRIME contract, we recommended the Business Systems Modernization Office complete activities to implement an overall contractor performance evaluation process.

Response

IRS management fully agreed with three of our five recommendations and partially agreed with two recommendations.  To ensure users/owners are appropriately identified and actively participate in measuring the value received from work items delivered by the PRIME contractor, the IRS will modify templates to define expected outcomes and ensure users/owners are identified and documented.  This will allow the IRS to determine whether stated objectives were met.  To ensure adequate cost estimates are available, the IRS will ensure certain cost estimate revisions are documented as separate estimates.  To ensure quality ratings and measurements are performed and an overall contractor performance evaluation process is available to create a rating for the entire PRIME contract, the IRS will perform quarterly surveillance of expected outcomes, has implemented a contractor performance evaluation process, and is completing an evaluation for the PRIME contractor.  Management’s complete response to the draft report is included as Appendix VIII.

Office of Audit Comment

The CIO stated the partial agreement for two recommendations reflected the IRS’ view that the cost to fully implement the recommendations outweighs the benefits.  However, the IRS believes the planned corrective actions reflect a cost-effective approach to addressing the recommendations.

For our recommendation to require users/owners to 1) document the purpose, 2) create success criteria, and 3) document whether each work item met the stated purpose and defined success criteria, the IRS will modify its templates to define expected outcomes.  However, the IRS stated it did not plan to implement our recommendation at the work item level.  Based on a preliminary version of this report, IRS officials explained they planned to evaluate logical groupings of work items against expected outcomes instead of defining expected outcomes for every work item.  During our preliminary discussions, we agreed the IRS’ planned corrective action would meet the intent of our recommendation.

For our recommendation to require the PRIME contractor to provide original and revised cost estimates for all task order activities, the IRS stated it would strengthen its processes for certain cost estimate revisions.  However, the IRS does not believe a separate accounting of each task order activity is needed.  As stated in the report, the IRS process for initiating work items and special projects within the contract does not require estimates of hours or dollar amounts for specific work items.  For example, one group of products and services requested from the PRIME contractor was support for project management, business management, risk management, and quality management, as well as the creation of a Project Management Plan.  The request included an estimate of over 3,500 hours.  Since the estimate is for all products and services, it is not possible to determine whether the PRIME contractor planned to spend a relatively small or large amount of time creating the Project Management Plan.  As a result, the IRS can not determine the relative cost of each work item or service, which makes it difficult to judge the value being received for any specific item within the contract.

Copies of this report are also being sent to the IRS managers affected by the report recommendations.  Please contact me at (202) 622-6510 if you have questions or Margaret E. Begg, Assistant Inspector General for Audit (Information Systems Programs), at (202) 622-8510.

 

 

Table of Contents

 

Background

Results of Review

The Modernized Infrastructure Is Being Maintained by the Internal Revenue Service Infrastructure Shared Services Team

The Internal Revenue Service Did Not Receive the Anticipated Value for All Work Items It Received

Recommendations 1 and 2:

Recommendation 3:

Recommendations 4 and 5:

Appendices

Appendix I – Detailed Objective, Scope, and Methodology

Appendix II – Major Contributors to This Report

Appendix III – Report Distribution List

Appendix IV – Outcome Measure

Appendix V – Major Components and Special Projects for the Fiscal Year 2004/2005 Infrastructure Shared Services Task Order

Appendix VI – Enterprise Life Cycle Overview

Appendix VII – Review of Value Received From Sample Work Items

Appendix VIII – Management’s Response to the Draft Report

 

 

Background

 

The Internal Revenue Service (IRS) is currently in the midst of a multiyear, multibillion dollar effort to update its core business systems, known as Business Systems Modernization (BSM).  According to IRS estimates, this effort involves integrating thousands of hardware and software components over 15 years.  The IRS hired the Computer Sciences Corporation as its PRIME contractor to head an alliance of leading technology companies in assisting with the BSM program.  The IRS also established the Business Systems Modernization Office (BSMO) to manage the BSM program and oversee the work of the PRIME contractor.

The success of the BSM program depends on establishing a strong foundation from which to build business applications supporting tax return processing and other critical mission-related functions.  In February 2002, the BSMO began building this foundation by establishing the Infrastructure Shared Services (ISS) program to install the following infrastructure areas:

1)      The Security and Technology Infrastructure Release provides the hardware and software necessary to deploy and run a BSM project.  It provides key features such as web servers, security, messaging, and directory services.

2)       Enterprise Systems Management provides the overlying management capabilities to the Security and Technology Infrastructure Release and network and systems management to improve infrastructure availability and performance.  It also manages hardware and software inventories and monitors the performance of various systems.

3)      The Development, Integration, and Test Environment provides a complete project development and testing environment meant to simulate actual operating conditions.  It provides labs to evaluate potential solutions and their impact on business processes (Solutions Demonstration Lab), support all phases of information systems development (Virtual Development Environment), and test systems integration and final acceptance (Enterprise Integration and Test Environment).

These critical infrastructure areas have been implemented and are being relied upon by other modernized projects and systems.  For Fiscal Years (FY) 2004 and 2005, the IRS issued a task order[6] to the PRIME contractor to work with the IRS to maintain the current infrastructure environment and adapt to new changes and upgrades that are inherent in information technology components.  Major components and special projects have been detailed within the task order and are being executed by the PRIME contractor.[7]  The FY 2004/2005 ISS task order has a period of performance from February 2004 through November 2005 with an estimated cost of $36.9 million.[8]

This audit focused on the value received from products and services provided through the FY 2004/2005 ISS task order.  The Treasury Inspector General for Tax Administration is conducting an additional audit focusing on whether vouchers submitted and paid under the FY 2004/2005 ISS task order were appropriate and in accordance with the task order’s terms and conditions.

We discussed our audit results with the BSMO in September 2005.  At that time, changes were being made at both the BSM program level and the ISS program level.  Therefore, any changes that occurred after we completed our analyses are not reflected in this report.

This review was performed at the BSMO facility in New Carrollton, Maryland, during the period April through November 2005.  The audit was conducted in accordance with Government Auditing Standards.  Detailed information on our audit objective, scope, and methodology is presented in Appendix I.  Major contributors to the report are listed in Appendix II.

 

Results of Review

 

The Modernized Infrastructure Is Being Maintained by the Internal Revenue Service Infrastructure Shared Services Team

The ISS team, through execution of the FY 2004/2005 ISS task order, is maintaining the modernized infrastructure to support new and current modernized projects such as the Modernized e-File[9] and the e-Services projects.[10]  The BSMO and the PRIME contractor have been working together to continue improving on the products and services received from the ISS program and to ensure the modernized infrastructure is available for other modernized projects and systems.

The IRS has improved its ability to perform detailed reviews of the contractor’s invoices

The PRIME contractor submits monthly invoices with supporting vouchers[11] for the FY 2004/2005 ISS task order.  Upon review and substantiation, the IRS will pay the invoice.  Between February 2004 and March 2005, the IRS worked with the PRIME contractor to revise the voucher format.  In April 2005, the IRS began receiving from the PRIME contractor a revised voucher format that included data the IRS had not previously received (e.g., specific contractor employees with the hours and travel costs they charged).  By obtaining this additional information, the IRS has improved its ability to perform detailed reviews of the PRIME contractor’s invoices.

The PRIME contractor is making corrections to work items as requested by the IRS

Issues and comments may arise during the IRS’ review of work items produced by the PRIME contractor.  We determined the ISS Project Account Managers[12] and the Office of Procurement Contracting Officer’s Technical Representative are taking steps to ensure the PRIME contractor is making corrections to work items by addressing those comments and issues.

Significant work items and special projects provided through the ISS task order are providing value to the IRS

We reviewed a judgmental sample of 27 (14 percent) of 195 work items produced by the PRIME contractor and received by the IRS between February 2004 and May 2005 and determined the IRS received the anticipated value for 12 work items and received value for an additional 3 work items as part of different projects.  For example, the PRIME contractor delivered a system security plan and an outline for disaster recovery.[13]  We also determined the IRS received the anticipated value from 9 (82 percent) of 11 special projects within the FY 2004/2005 ISS task order.[14]  Many of these special projects were necessary to provide enhancements, upgrades, and improvements to current infrastructure systems.  While value is being received for many of the work items and special projects in the FY 2004/2005 ISS task order, we determined the IRS is not obtaining the anticipated value for other work items and special projects, and it is difficult to determine the value being received for other work items and special projects.

The Internal Revenue Service Did Not Receive the Anticipated Value for All Work Items It Received

As mentioned previously, we determined 15 of 27 sampled work items have provided or are providing value to the IRS either for their original purpose or as part of different projects.  However, the value received for the remaining 12 (44 percent) work items was more difficult to determine.  Initially, we concluded the IRS did not receive full value for these work items.  Upon meeting with IRS officials in September 2005, December 2005, and January 2006, the IRS provided additional information concerning whether these remaining work items provided value.  The IRS agreed 7 (26 percent) work items did not provide the IRS with the anticipated value of the work items.  However, the IRS stated it believed that, while the original anticipated value may not have been obtained, the IRS did receive value from the remaining five work items.  In two instances, the IRS provided additional information after the conclusion of our fieldwork that we were unable to verify.  In the other three cases, the IRS believed the value of the items had either yet to be realized or was obtained by allowing the organization to determine an alternate, less costly path.  Figure 1 summarizes the results of the 27 sampled items.

Figure 1:  Selected Sample Work Items and Results[15]

Figure 1 was removed due to its size.  To see Figure 1, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

In determining value for the sampled items, we performed a detailed review to identify each item’s purpose, results, and actual IRS users/owners.  We also interviewed users/owners to determine if the sampled items met their stated purposes and needs.  Our conclusions that some work items did not provide the anticipated value were based on one or more of the following results found during the review:

• The IRS could not identify users/owners for the sampled items.
• Actual IRS users/owners stated the sampled items were of no value.
• Sampled items were not being used by the IRS and/or did not have a documented purpose.

We attempted to calculate and report the costs (dollar amounts) for the 195 total work items, the 27 sampled work items, and each of the 7 sampled work items that did not provide the anticipated value to the IRS.  However, the IRS could not provide cost data on individual work items paid to the PRIME contractor because multiple work items and services were acquired together.  For each set of work items and services, the PRIME contractor provides a total estimate (number of hours) of what it would take to complete all work items and services for the IRS; however, the IRS does not request a breakdown by work item (see The IRS does not have cost estimates of individual work items section in this report).

The IRS has spent Federal Government resources for Infrastructure Shared Services work items that are not being used.

In addition to the sampled items, we reviewed all 11 special projects performed under the FY 2004/2005 ISS task order.[16]  These special projects varied in dollar amounts, with the largest coming from the Development, Integration, and Test Environment General Lab Support special project that cost the IRS $6.7 million.[17]  We determined 2 (18 percent) of the 11 special projects did not provide the anticipated value to the IRS.  The IRS spent $2,995,676 on these 2 special projects.[18]  According to the IRS, the PRIME contractor’s performance on the Electronic File Transfer Utility special project was not worth the cost during the last period we reviewed, and funding priorities made it wise to take what the PRIME contractor had produced to date and transition it to the IRS for completion.  Regarding the Portal Upgrade and Infrastructure Development Sandbox,[19] the IRS stated the special project was never completed and was ultimately cancelled by the IRS in December 2004.  According to the IRS, it estimates it spent $310,863 on the portions of the 2 special projects that did not provide anticipated value.[20] 

In both cases, the IRS stated that, while it did not receive the anticipated value from these special projects, it did receive value by either completing work internally or as part of another project.

The IRS could do more to proactively assess whether value is being received from the PRIME contractor.

The IRS does not individually evaluate the value obtained from specific work items or special projects during the execution of activities within the FY 2004/2005 ISS task order.  We determined the IRS could do more to proactively assess whether value is being received from the PRIME contractor.  The IRS needs to enhance awareness and provide the information necessary to incorporate value within task order monitoring activities.  Currently, the IRS does not have all the information necessary to perform value measurements because 1) users/owners are not documenting the purposes and planned results for work items, 2) the IRS does not have cost estimates of individual work items, and 3) the IRS is not performing quality reviews of deliverables or making overall performance assessments of the PRIME contractor.

Users/owners are not documenting the purposes and planned results for work items

According to the American Society for Quality^TM, one way of achieving performance measures is through the implementation of a continuous quality management process called the Plan-Do-Check-Act cycle, which is a continuous cycle of planning, implementing, reviewing, and improving the processes and actions that an organization undertakes to meet its business goals.

Figure 2:  Plan-Do-Check-Act Cycle

Figure 2 was removed due to its size.  To see Figure 2, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

 

Work items and special projects from the FY 2004/2005 ISS task order did not always list the specific purpose, criteria, or details as to what the end results should contain.  Furthermore, acceptance and success criteria for those work items and special projects were not always defined.  This made it difficult to evaluate the work items and does not provide the IRS information necessary to determine if value is being received or to hold the contractor accountable for the work items and services.  In addition, adequate user involvement helps ensure user requirements are met.  However, users/owners for work items and special projects in the FY 2004/2005 ISS task order were not always identified or known by the IRS.

Due to the complexity and size of the ISS program, multiple Acquisition Project Managers[21] and Project Account Managers are using different techniques to execute their portions of the task order.  As a result of the differing techniques, users/owners are not always identified, acceptance or success criteria are not always defined, and users/owners do not always participate in measuring the success of work items and services received from the PRIME contractor.

Recommendations

Recommendation 1:  To ensure users/owners are appropriately identified, the Chief Information Officer (CIO) should implement a consistent process to formally document users/owners for all task order activities (work products, deliverables, and management and support services).

Management’s Response:  The CIO agreed with this recommendation.  To ensure users/owners are appropriately identified, the IRS will modify templates to ensure users/owners are identified and documented.

Recommendation 2:  To ensure users/owners actively participate in measuring the value received from work items delivered by the PRIME contractor, the CIO should require users/owners to 1) document the purpose, 2) create success criteria, and 3) document whether each work item met the stated purpose and defined success criteria.

Management’s Response:  The CIO partially agreed with this recommendation.  To ensure users/owners actively participate in measuring the value received from work items delivered by the PRIME contractor, the IRS will modify templates to define expected outcomes.  This will allow the IRS to determine whether stated objectives were met.

Office of Audit Comment:  While the IRS plans to modify templates to define expected outcomes, it does not plan to implement our recommendation at the work item level.  Based on a preliminary version of this report, IRS officials explained they planned to evaluate logical groupings of work items against expected outcomes instead of defining expected outcomes for every work item.  During our preliminary discussions, we agreed the IRS’ planned corrective action would meet the intent of our recommendation.

The IRS does not have cost estimates of individual work items

The Office of Federal Procurement Policy document entitled A Guide to Best Practices for Performance-Based Service Contracting[22] states costs must be computed for each service or output based on available data.  These costs are used in preparing the Federal Government estimate, evaluating proposals, and determining positive and negative incentives.  In addition, the Enterprise Life Cycle[23] states the IRS should assess the contractor’s basis of estimate in response to an acquisition document issued to establish a task order or in response to tasking from an existing task order.  While complying with the full extent of these practices may not be appropriate in all instances for the ISS task order, we believe the IRS should know the relative cost of individual special projects or work items to be able to determine whether it is cost beneficial to acquire individual special projects or work items and whether value is being received.

The IRS process for initiating work items and special projects within the contract does not require estimates of hours or dollar amounts for specific work items.  For example, one group of products and services requested from the PRIME contractor was support for project management, business management, risk management, and quality management, as well as the creation of a Project Management Plan.  The request included an estimate of over 3,500 hours.  Since the estimate is for all products and services, it is not possible to determine whether the PRIME contractor planned to spend a relatively small or large amount of time creating the Project Management Plan.  As a result, the IRS can not determine the relative cost of each work item or service, which makes it difficult to judge the value being received for any specific item within the contract.  Without this information, the IRS increases the chances that Federal Government resources are not used effectively.

Recommendation

Recommendation 3:  To ensure adequate cost estimates are available, the CIO should require the PRIME contractor to provide original and revised (if applicable) cost estimates for all task order activities (work products, deliverables, and management and support services).

Management’s Response:  The CIO partially agreed with this recommendation.  To ensure adequate cost estimates are available, the IRS will ensure certain cost estimate revisions are documented as separate estimates.

Office of Audit Comment:  While the IRS plans to ensure certain cost estimate revisions are documented as separate estimates, it does not believe a separate accounting of each task order activity is needed.  As stated previously, the IRS process for initiating work items and special projects within the contract does not require estimates of hours or dollar amounts for specific work items.  As a result, the IRS can not determine the relative cost of each work item or service, which makes it difficult to judge the value being received for any specific item within the contract.

The IRS is not performing quality reviews of work items or making overall performance assessments of the PRIME contractor

The Office of Federal Procurement Policy document entitled Best Practices for Collecting and Using Current and Past Performance Information[24] states recording contractors’ current performance periodically during contract performance and discussing the results with contractors is a powerful motivator for contractors to maintain high-quality performance or improve inadequate performance.  It also states current performance assessment is a basic “best practice” for good contract administration and is one of the most important tools available for ensuring good contractor performance.  The IRS uses the Office of Procurement Surveillance Reports to report on the quality of deliverables.  The ISS task order monitoring plan defines specific quality ratings for the Surveillance Report.  These criteria include evaluating cost control, timeliness of performance, and quality of product or service.

We determined the ISS Surveillance Reports do not provide measurements on the quality of work items as required by the ISS task order monitoring plan.  In addition, the PRIME contractor’s overall performance for the FY 2004/2005 ISS task order is not measured or reported.  As a result, the IRS may not know how well the PRIME contractor is performing or if the PRIME contractor is meeting the needs and requirements of users/owners to adequately measure the value received.  Additionally, the risk of wasted resources is increased.

The Acquisition Project Managers and Project Account Managers are not required to follow the quality ratings process, and they do not have a process which details specific steps for measuring and reporting the contractor’s specific performance on the task order or to provide input for the PRIME contractor’s overall performance on all modernization task orders.  In addition, the Associate CIO, BSM, stated the IRS stopped reporting on the overall performance of the PRIME contractor due to inconsistent application of the prior contractor performance evaluation process.

Management Action:  The Associate CIO, BSM, stated the BSMO is currently testing a revised contractor performance evaluation process over a 6-month period.  The revised process will create a rating for the entire PRIME contract, instead of providing individual ratings for each task order.  According to the Office of Procurement, the IRS will complete the contractor performance evaluation after January 2006.

Recommendations

Recommendation 4:  To ensure quality ratings and measurements are performed, the CIO should require all Acquisition Project Managers and Project Account Managers to follow the ISS task order monitoring plan.

Management’s Response:  The CIO agreed with this recommendation.  To ensure quality ratings and measurements are performed, the IRS will perform quarterly surveillance of expected outcomes.

Recommendation 5:  To ensure an overall contractor performance evaluation process is available to create a rating for the entire PRIME contract, the BSMO should complete activities to implement an overall contractor performance evaluation process.

Management’s Response:  The CIO agreed with this recommendation.  To ensure an overall contractor performance evaluation process is available to create a rating for the entire PRIME contract, the IRS has implemented a contractor performance evaluation process and is completing an evaluation for the PRIME contractor.

 

Appendix I

 

Detailed Objective, Scope, and Methodology

 

The overall objective of the review was to determine whether the Internal Revenue Service (IRS) is receiving full value from the PRIME contractor[25] under the terms and conditions of the Fiscal Year 2004/2005 Infrastructure[26] Shared Services task order.[27]  To accomplish this objective, we:

I.                   Determined whether work items were clearly identified and adequate acceptance criteria had been developed.

II.                Determined whether a judgmental sample of work items provided value to the IRS by:[28]

A.    Comparing work items to product descriptions.

B.     Interviewing users/owners.

C.     Determining whether requested corrections were made by the PRIME contractor.

III.             Determined whether the IRS evaluated the PRIME contractor’s overall performance on the task order by regularly monitoring and measuring performance for specific work items and special projects.

A.    Determined whether the IRS evaluated the PRIME contractor’s performance on specific work items.

B.     Reviewed the 11 special projects within the Fiscal Year 2004/2005 Infrastructure Shared Services task order and determined the IRS’ process for managing, measuring, and reporting performance and results.

 

Appendix II

 

Major Contributors to This Report

 

Margaret E. Begg, Assistant Inspector General for Audit (Information Systems Program)

Gary Hinkle, Director

Troy Paterson, Audit Manager

Phung Nguyen, Lead Auditor

Tina Wong, Senior Auditor

Charlene Elliston, Auditor

 

Appendix III

 

Report Distribution List

 

Commissioner  C

Office of the Commissioner – Attn:  Chief of Staff  C

Deputy Commissioner for Operations Support  OS

Associate Chief Information Officer, Business Systems Modernization  OS:CIO:B

Associate Chief Information Officer, Enterprise Services  OS:CIO:ES

Director, Procurement  OS:A:P

Director, Stakeholder Management  OS:CIO:SM

Deputy Associate Chief Information Officer, Systems Integration  OS:CIO:B:SI

Chief Counsel  CC

National Taxpayer Advocate  TA

Director, Office of Legislative Affairs  CL:LA

Director, Office of Program Evaluation and Risk Analysis  RAS:O

Office of Management Controls  OS:CFO:AR:M

Audit Liaisons:

Associate Chief Information Officer, Business Systems Modernization  OS:CIO:B

Director, Procurement  OS:A:P

Manager, Program Oversight Office  OS:CIO:SM:PO

 

Appendix IV

 

Outcome Measure

 

This appendix presents detailed information on the measurable impact our recommended corrective actions will have on tax administration.  This benefit will be incorporated into our Semiannual Report to Congress.

Type and Value of Outcome Measure:

·         Inefficient Use of Resources – Actual; $310,863 (see page 4).

Methodology Used to Measure the Reported Benefit:

We interviewed the Acquisition Project Manager and Project Account Managers[29] assigned to implement and monitor special projects from the Fiscal Year 2004/2005 Infrastructure[30] Shared Services task order.[31]  According to the Internal Revenue Service (IRS), the PRIME contractor’s performance was not worth the cost near the conclusion of its involvement in the project, and funding priorities made it wise to take what the PRIME contractor had produced and transition it to the IRS for completion.  Regarding the Portal Upgrade and Infrastructure Development Sandbox,[32] the IRS stated the special project was never completed and was ultimately cancelled by the IRS in December 2004.

According to the IRS, it estimates it spent approximately $310,863 ($83,613 for the Enterprise File Transfer Utility Support Services special project and $227,250 for the Portal Upgrade and Infrastructure Development Sandbox special project) on the portions of the 2 special projects that did not provide anticipated value.  In both cases, the IRS stated that, while it did not receive the anticipated value from these special projects, it did receive value by either completing work internally or as part of another project.

 

Appendix V

 

Major Components and Special Projects for the Fiscal Year 2004/2005 Infrastructure Shared Services Task Order

 

The Fiscal Year (FY) 2004/2005 Infrastructure[33] Shared Services (ISS) task order[34] contains many different components and special projects for which the PRIME contractor[35] is tasked to perform activities and provide work items and services.  Each component and special project has a total estimated cost associated for each contracted period.  Table 1 provides a snapshot of the major components and Table 2 provides a snapshot of the special projects from the FY 2004/2005 ISS task order dated May 17, 2005.  The total estimated costs shown are provided for the contracted base period of February 21 through November 30, 2004, and the option period of December 1, 2004, through November 30, 2005.

Table 1:  Estimated Costs for FY 2004/2005 ISS Task Order Major Components

Source:  Treasury Inspector General for Tax Administration analysis of the FY 2004/2005 ISS task order (modification 40, dated May 17, 2005).

Table 2:  Estimated Costs for FY 2004/2005 ISS Task Order Special Projects[37]

Source:  The FY 2004/2005 ISS task order (modification 40, dated May 17, 2005).

 

Appendix VI

 

Enterprise Life Cycle Overview

 

The Enterprise Life Cycle (ELC) defines the processes, products, techniques, roles, responsibilities, policies, procedures, and standards associated with planning, executing, and managing business change.  It includes redesign of business processes; transformation of the organization; and development, integration, deployment, and maintenance of the related information technology applications and infrastructure.  Its immediate focus is the Internal Revenue Service (IRS) Business Systems Modernization (BSM) program.  Both the IRS and the PRIME contractor[41] must follow the ELC in developing/acquiring business solutions for modernization projects.

The ELC framework is a flexible and adaptable structure within which one plans, executes, and integrates business change.  The ELC process layer was created principally from the Computer Sciences Corporation’s Catalyst^® methodology.[42]  It is intended to improve the acquisition, use, and management of information technology within the IRS; facilitate management of large-scale business change; and enhance the methods of decision making and information sharing.  Other components and extensions were added as needed to meet the specific needs of the IRS BSM program.

ELC Processes

A process is an ordered, interdependent set of activities established to accomplish a specific purpose.  Processes help to define what work needs to be performed.  The ELC methodology includes two major groups of processes:

Life-Cycle Processes, which are organized into phases and subphases and address all domains of business change.

Management Processes, which are organized into management areas and operate across the entire life cycle.

Enterprise Life-Cycle Processes

The chart was removed due to its size.  To see the chart, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

Life-Cycle Processes

The life-cycle processes of the ELC are divided into six phases, as described below:

·         Vision and Strategy - This phase establishes the overall direction and priorities for business change for the enterprise.  It also identifies and prioritizes the business or system areas for further analysis.

·         Architecture - This phase establishes the concept/vision, requirements, and design for a particular business area or target system.  It also defines the releases for the business area or system.

·         Development - This phase includes the analysis, design, acquisition, modification, construction, and testing of the components of a business solution.  This phase also includes routine planned maintenance of applications.

·         Integration - This phase includes the integration, testing, piloting, and acceptance of a release.  In this phase, the integration team brings together individual work packages of solution components developed or acquired separately during the Development phase.  Application and technical infrastructure components are tested to determine whether they interact properly.  If appropriate, the team conducts a pilot to ensure all elements of the business solution work together.

·         Deployment - This phase includes preparation of a release for deployment and actual deployment of the release to the deployment sites.  During this phase, the deployment team puts the solution release into operation at target sites.

·         Operations and Support - This phase addresses the ongoing operations and support of the system.  It begins after the business processes and system(s) have been installed and have begun performing business functions.  It encompasses all of the operations and support processes necessary to deliver the services associated with managing all or part of a computing environment.

The Operations and Support phase includes the scheduled activities, such as planned maintenance, systems backup, and production output, as well as the nonscheduled activities, such as problem resolution and service request delivery, including emergency unplanned maintenance of applications.  It also includes the support processes required to keep the system up and running at the contractually specified level.

Management Processes

Besides the life-cycle processes, the ELC also addresses the various management areas at the process level.  The management areas include:

·         IRS Governance and Investment Decision Management - This area is responsible for managing the overall direction of the IRS, determining where to invest, and managing the investments over time.

·         Program Management and Project Management - This area is responsible for organizing, planning, directing, and controlling the activities within the program and its subordinate projects to achieve the objectives of the program and deliver the expected business results.

·         Architectural Engineering/Development Coordination - This area is responsible for managing the technical aspects of coordination across projects and disciplines, such as managing interfaces, controlling architectural changes, ensuring architectural compliance, maintaining standards, and resolving issues.

·         Management Support Processes - This area includes common management processes, such as quality management and configuration management that operate across multiple levels of management.

Milestones

The ELC establishes a set of repeatable processes and a system of milestones, checkpoints, and reviews that reduce the risks of systems development, accelerate the delivery of business solutions, and ensure alignment with the overall business strategy.  The ELC defines a series of milestones in the life-cycle processes.  Milestones provide for “go/no-go” decision points in the project and are sometimes associated with funding approval to proceed.  They occur at natural breaks in the process where there is new information regarding costs, benefits, and risks and where executive authority is necessary for next phase expenditures.

There are five milestones during the project life cycle: 

·•         Milestone 1 - Business Vision and Case for Action.  In the activities leading up to Milestone 1, executive leadership identifies the direction and priorities for IRS business change.  These guide which business areas and systems development projects are funded for further analysis.  The primary decision at Milestone 1 is to select BSM projects based on both the enterprise-level Vision and Strategy and the Enterprise Architecture.

·•         Milestone 2 - Business Systems Concept and Preliminary Business Case.  The activities leading up to Milestone 2 establish the project concept, including requirements and design elements, as a solution for a specific business area or business system.  A preliminary business case is also produced.  The primary decision at Milestone 2 is to approve the solution/system concept and associated plans for a modernization initiative and to authorize funding for that solution.

·•         Milestone 3 - Business Systems Design and Baseline Business Case.  In the activities leading up to Milestone 3, the major components of the business solution are analyzed and designed.  A baseline business case is also produced.  The primary decision at Milestone 3 is to accept the logical system design and associated plans and to authorize funding for development, test, and (if chosen) pilot of that solution.

·•         Milestone 4 - Business Systems Development and Enterprise Deployment Decision.  In the activities leading up to Milestone 4, the business solution is built.  The system is integrated with other business systems and tested, piloted (usually), and prepared for deployment.  The primary decision at Milestone 4 is to authorize the release for enterprise-wide deployment and commit the necessary resources.

·•         Milestone 5 - Business Systems Deployment and Postdeployment Evaluation.  In the activities leading up to Milestone 5, the business solution is fully deployed, including delivery of training on use and maintenance.  The primary decision at Milestone 5 is to authorize the release of performance-based compensation based on actual, measured performance of the business system.

 

Appendix VII

 

Review of Value Received From Sample Work Items

 

The Treasury Inspector General for Tax Administration (TIGTA) reviewed a judgmental sample of 27 work items from a population of 195 work items produced by the PRIME contractor[43] and received by the Internal Revenue Service (IRS) between February 2004 and May 2005.  Table 1 provides an analysis of the value received by the IRS for each of the sample work items.

Table 1:  Independent Assessment of Value Received From the 27 Sample Work Items

Source:  TIGTA analysis of the selected sample work items and IRS input on a preliminary version of this report.

 

Appendix VIII

 

Management’s Response to the Draft Report

 

The response was removed due to its size.  To see the response, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.