TREASURY INSPECTOR GENERAL FOR TAX ADMINISTRATION

 

 

The Modernization Vision and Strategy Program Is Achieving Desired Results, but Risks Remain

 

 

 

October 31, 2008

 

Reference Number:  2009-20-008

 

 

This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.

 

Phone Number   |  202-622-6500

Email Address   |  inquiries@tigta.treas.gov

Web Site           |  http://www.tigta.gov

 

October 31, 2008

 

 

MEMORANDUM FOR CHIEF INFORMATION OFFICER

 

FROM:                            Michael R. Phillips /s/ Michael R. Phillips

                                         Deputy Inspector General for Audit

 

SUBJECT:                    Final Audit Report – The Modernization Vision and Strategy Program Is Achieving Desired Results, but Risks Remain (Audit # 200820003)

 

This report presents the results of our review to determine whether the Internal Revenue Service (IRS) Modernization Vision and Strategy (MV&S) Program is achieving the desired results.  This review was included in our Fiscal Year 2008 Annual Audit Plan coverage of IRS Business Systems Modernization efforts.

Impact on the Taxpayer

Since the Business Systems Modernization program was initiated in Fiscal Year 1999, substantial changes in the information technology and budget environments have forced the IRS to shift its thinking and original assumptions.  The MV&S Program was developed to implement a fundamentally different approach to the IRS modernization effort and to provide guidance in the development of an integrated portfolio of information technology investments.  The MV&S Program has improved the information technology investment decision processes, resulting in information technology projects that 1) better support the IRS goals of improving customer service and enforcement, and 2) make better use of technology funds.  However, process improvements need to be implemented to ensure the continued improvement of future investment proposal cost estimates and standardized reporting of investment decisions’ performance measures.

Synopsis

Overall, the decision-making processes used in the MV&S Program provide IRS executives with effective tools and information with which to make informed information technology decisions.  Developed in October 2006, the MV&S Plan is an annually updated 5-year plan for the governance and planning processes for IRS information technology decisions.  This approach leverages the IRS strategic plan and associated business plans to drive information technology decisions, address the priorities around modernizing front-line tax administration, and enable technical capabilities provided by the information technology infrastructure.  The IRS also created several guiding principles for this new vision and strategy.  The MV&S Program will guide the investment priorities of the Business Systems Modernization program for Fiscal Years 2007 through 2011.

Our review of the MV&S Program determined that, overall, processes and procedures have been effectively implemented.  For example, the MV&S Plan is updated annually; strategic planning is effective; collaborative alliances have been established; and focused, more frequent delivery of information technology project releases is emphasized.

In addition, the IRS is using an approved estimation model for determining total cost of ownership.  We determined that the cost estimates for a sample of 4 of 23 projects in the approved investment portfolio appear to be reasonable based on 1) the documentation provided, 2) the validation of the estimates by employees with expertise in the area, and 3) reviews by the team working on the proposed investments.  However, IRS internal estimation guidance does not require comparison of actual investment costs to the original estimates.  Therefore, the reliability of estimated total costs of ownership might be at risk because the estimates for prior projects were not validated by comparing the estimates to the actual costs.

MV&S Program performance measures are being collected, but there is no standardized procedure for compiling and reporting these measures.  Through review of documentation and discussions with personnel, we could not determine whether performance measurement and reporting were being thoroughly accomplished as required.  The lack of MV&S Program performance measures and reporting compliance could result in inefficient accomplishment of strategic information technology acquisitions, objectives, and expectations.

Recommendations

The Chief Information Officer should 1) ensure the reliability of the cost estimation process by implementing procedures to compare actual project operations and maintenance costs to initial estimates and revising the estimation process, if necessary, and 2) include guidelines for performance measurement and reporting in the updated MV&S Plan.

Response

IRS management agreed with all of our recommendations.  Corrective actions planned include 1) implementing procedures to compare actual project operations and maintenance costs to initial estimates and revising the estimation process when necessary, and 2) updating the MV&S Plan to reference performance measures and reporting.  This update will be included in the Calendar Year 2009 annual publication.  Management’s complete response to the draft report is included as Appendix V.

Copies of this report are also being sent to the IRS managers affected by the report recommendations.  Please contact me at (202) 622-6510 if you have questions or Margaret E. Begg, Assistant Inspector General for Audit (Security and Information Technology Services), at (202) 622-8510.

 

 

Table of Contents

 

Background

Results of Review

The Modernization Vision and Strategy Program Is an Effective Process for Making Information Technology Investment Decisions

The Process for Developing Total Cost Estimates Can Be Improved

Recommendation 1:

Modernization Vision and Strategy Program Performance Measurement Standards Need Further Development

Recommendation 2:

Appendices

Appendix I – Detailed Objective, Scope, and Methodology

Appendix II – Major Contributors to This Report

Appendix III – Report Distribution List

Appendix IV – Glossary of Terms

Appendix V – Management’s Response to the Draft Report

  

Abbreviations

 

IRS

Internal Revenue Service

MITS

Modernization and Information Technology Services

MV&S

Modernization Vision and Strategy

 

Background

 

Since the Business Systems Modernization program was initiated in Fiscal Year 1999, substantial changes in the information technology and budget environments have forced the Internal Revenue Service (IRS) to shift its thinking and original assumptions.  In particular, the original goal to completely replace the existing IRS information technology environment within 10 years - 15 years has been deemed unrealistic due to resource limitations.  The Modernization Vision and Strategy (MV&S) Program was developed to implement a fundamentally different approach to the IRS modernization effort and to provide guidance in the development of an integrated portfolio of information technology investments.  Developed in October 2006, the MV&S Plan is an annually updated 5-year plan for the governance and planning process of IRS information technology decisions.  This approach leverages the IRS strategic plan and associated business plans to drive information technology decisions, address the priorities around modernizing front-line tax administration, and enable technical capabilities provided by the information technology infrastructure.  The MV&S Program will guide the investment priorities of the Business Systems Modernization program for Fiscal Years 2007 through 2011.  It includes the following guiding principles:

  • Joint business and information technology leadership and involvement throughout the process.
  • Unified approach to setting strategic priorities and selecting major and non-major projects (investments)[1] as an integrated portfolio.
  • Smaller, incremental releases delivered more frequently.
  • Existing systems (or components of existing systems) leveraged where appropriate.
  • A portfolio view of investmentsnot simply a collection of individual projects.
  • Application of end-to-end costing models to ensure a full view of all costs associated with each project.

The Portfolio Planning, Estimation, and Delivery Services Office within the Modernization and Information Technology Services (MITS) organization leads and facilitates the MV&S Program.  This Program consists of reviewing, prioritizing, and preparing proposed information technology projects for funding approval through the MITS organization Enterprise Governance Committee.  Three offices facilitate the MV&S Program:

  • The MV&S Program Office enables the prioritization and selection of future information technology investments.  This is accomplished by proactively engaging stakeholders to develop investment strategies aligned with IRS enterprise-wide strategic plans.  This Office coordinates and gathers information required in making informed investment decisions and presents results to senior executives.
  • The Solution Concept Office develops and employs processes and proactively engages and collaborates with stakeholders to deliver investment ideas when new projects or new releases are initially proposed for the investment portfolio.
  • The Estimation Program Office develops and uses processes and tools to deliver the following:  1) estimates of scope, cost, schedule, staffing, and quality; 2) MV&S Program pre-select phase estimates of candidate investments; 3) independent estimates for selected, high-priority projects; and 4) human capital profiles and the numbers and types of skill sets required to deliver selected major and non-major projects.

This review was performed at the MITS organization office in New Carrollton, Maryland, during the period February through August 2008.  We conducted this performance audit in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objective.  We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objective.  Detailed information on our audit objective, scope, and methodology is presented in Appendix I.  Major contributors to the report are listed in Appendix II.

  

Results of Review

 

The Modernization Vision and Strategy Program Is an Effective Process for Making Information Technology Investment Decisions

 

Overall, the decision-making processes used in the MV&S Program provide IRS executives with effective tools and information with which to make informed information technology decisions.  For example:

·         The IRS annually updates the MV&S Plan.

·         The MITS organization has established an effective approach for strategic planning.

·         The MITS organization has established collaborative alliances internally and with other IRS business organizations.

·         The IRS emphasizes focused, more frequent delivery of information technology project releases.

The IRS annually updates the MV&S Plan

The MV&S Program Office has a detailed project plan outlining efforts from MV&S Plan preparation to communicating information affecting the Program.  The first MV&S Plan was prepared in October 2006 and was updated in October 2007.  Annual updates to the Plan will be ongoing throughout the MV&S Program.  The IRS advised us that the Plan is updated to reflect changes that affect the business organizations.  In addition, the annual update includes changes in strategic objectives and incorporates new requirements.  For example, our review of the 2006 and 2007 Plans determined that the 2007 Plan was revised to reflect updated lists of potential and future projects and to include information technology as a common service.

The MITS organization has established an effective approach for strategic planning

The IRS begins the strategic planning process by having the business organizations define their goals and initiatives to meet their business objectives, which then aids in the accomplishment of IRS strategic goals.  We confirmed that the Portfolio Planning, Estimation, and Delivery Services Office conducted portfolio investment planning efforts.  In addition, the MITS organization completed several portfolio planning efforts for Fiscal Years 2007 through 2011.  The Strategy processes involve a series of activities aimed at preparing, executing, and communicating the Strategy.

For each business area (e.g., the Submission Processing, Managing Taxpayer Accounts, Customer Service, Chief Counsel, and Internal Management organizations), the MV&S Plan defines the goals, benefits, opportunities, and expected business outcomes related to technology that will yield the greatest benefits to taxpayers and the IRS.  The Plan identifies the specific projects, systems, and investments that will be necessary to achieve the business outcome.  The business areas collaborate to develop a joint strategy for information technology investments.  After strategic plans are reviewed, investment proposals are identified, evaluated, and prioritized for each business area.  The investment proposals are then prioritized based on the IRS’ strategic needs.  We reviewed the most recent enterprise portfolio prioritization document available and found that all internal business areas included in the MV&S Plan prioritized and presented their specific lists of projects.

Mapping of investment proposals to IRS and other strategic goals is being accomplished.  Information technology vision sessions are conducted within the business areas to identify scope, goals, opportunities, recommendations, and strategic views for each office.  Each investment proposal that goes through the MV&S Program is given an investment review that shows 1) the business proposal’s alignment with business area goals, 2) initiatives in the President’s Management Agenda, and 3) Department of the Treasury and IRS strategic goals.

The MITS organization has established collaborative alliances internally and with other IRS business organizations

Collaborative alliances were formed between organizations for the MV&S Program.  The Portfolio Planning, Estimation, and Delivery Services Office staff have developed relationships internally and with other business organizations to engage participants in the various activities in the MV&S Program.  Additionally, the staff work closely with the Chief Financial Office and the Application Development Budget, Capital Planning and Investment Control, and Financial Management Services organizations to determine how to improve departmental processes.  The Capital Planning and Investment Control and the Financial Management Services organizations are members of a working group structured to improve investment processes.  The MV&S Program Office continues to search for ways to improve these processes.  For example, the MITRE Corporation conducted lessons-learned sessions concerning the 2009 investment planning processes.  The participants in these sessions reported that the planning process had improved from that in the prior year.

Another example of the collaborative alliances that were developed is demonstrated by the effort to ensure that all proposed solutions are in compliance with the Enterprise Architecture and support business processes.  We determined that a collaborative system integration review was being accomplished and included representatives from several organizations such as the Systems Architecture and Engineering, Enterprise Architecture, Enterprise Data Management, and Information Technology Security Engineering organizations.

The IRS emphasizes focused, more frequent delivery of information technology project releases

The IRS continues to emphasize smaller, more frequent delivery of information technology project releases.  In our annual assessment of the Business Systems Modernization program for Fiscal Year 2008,[2] we concluded that the IRS and its contractors were performing better against cost and schedule estimates as a result of breaking releases down into smaller, more manageable pieces.  While our annual assessment included an analysis of major investments in the Business Systems Modernization program, we determined that the October 2007 MV&S Plan included language relevant to all investment projects.  Further, our review of meeting minutes and other supporting documents determined that the IRS is following a policy of making more frequent releases.  This policy is the focus of continued emphasis within the IRS.  As recently as July 2008, the Enterprise Services organization–in an MV&S Program presentation–included “definition of project scope in smaller incremental releases delivered more frequently” as one of the Key Success Factors for Investment Proposal Development.

The Government Accountability Office Standards for Internal Control in the Federal Government[3] states that internal management control is 1) comprised of plans, methods, and procedures used to meet departmental mission, goals, and objectives, and 2) supports performance-based management.  The Clinger-Cohen Act of 1996[4] addresses controls over information technology investments and encourages streamlined, phased approaches for information technology acquisitions.  The IRS Enterprise Life Cycle methodology stresses focus on delivering quality results that produce smaller benefits early and incrementally, throughout the life of the project.

The MV&S Program has improved the information technology investment decision processes, resulting in information technology projects that 1) better support the IRS goals of improving customer service and enforcement, and 2) make better use of technology funds.  However, process improvements need to be implemented to ensure the continued improvement of future investment proposal cost estimates and standardized reporting of investment decisions’ performance measures.

The Process for Developing Total Cost Estimates Can Be Improved

The IRS is using internal guidance to estimate the total cost of ownership for information technology investment projects.  However, the guidance does not require comparison of actual investment costs to the original estimates.  Thus, these comparisons do not occur.  We reviewed the cost estimates for 4 of the 23 projects in the approved investment portfolio and determined that the Estimation Program Office uses 1 of the approved estimation models for determining total cost of ownership.  The cost assumptions and the resulting cost estimates are validated by employees with expertise in the area participating in the development of the Solution Concept document, and Estimation Program Office personnel conduct indepth reviews of the Basis of Estimate with the team working on the proposed investment.  Based on the extent of the estimation process and the Basis of Estimate documents, the cost estimates appear to be reasonable.  However, the reliability of the estimated total cost of ownership might be at risk because the estimates for prior projects were not validated by comparing the estimates to the actual costs.

Internal guidance, IRS Basis for Parametric Estimation Guidance, dated October 2007, provides a basic requirement to update operations and maintenance estimates to actual costs for future estimating purposes.  However, it does not include specific provisions for comparisons of actual costs to original estimates.  Additionally, in July 2007, the Government Accountability Office released an Exposure Draft, Cost Assessment Guide:  Best Practices for Estimating and Managing Program Costs,[5] that provides an outline for successful cost estimating in a 12-step estimation process.  The process provides specific procedures to update cost estimates to actual costs.

IRS management informed us that there is no documented process to track and compare actual operations and maintenance costs to the original estimates.  The systems modernization projects (i.e., major projects) track actual costs compared to the Earned Value Management budget for Capital Planning and Investment Control, but they do not consider original cost estimates.  Management further indicated that processes to document other projects (i.e., non-major projects) focus on only development, modernization, and enhancement costs and schedules and do not consider operations and maintenance.

By not comparing initial cost estimates to the actual project costs to validate the cost estimation model, the IRS is increasing the risk that its future investment proposal cost estimates will not be reliable.

Recommendation

Recommendation 1:  The Chief Information Officer should ensure the reliability of the cost estimation process by implementing procedures to compare actual project operations and maintenance costs to initial estimates and revising the estimation process, if necessary.

Management’s Response:  The IRS agreed with this recommendation.  It will implement procedures to compare actual project operations and maintenance costs to initial estimates and revise the estimation process when necessary.

Modernization Vision and Strategy Program Performance Measurement Standards Need Further Development

MV&S Program performance measures are being collected, but there is no standardized procedure for compiling and reporting these measures.  Through review of documentation and discussions with personnel, we could not determine whether performance measurement and reporting were being thoroughly accomplished as required.  Specifically, the October 2007 MV&S Plan does not include guidelines for measuring and reporting compliance.  Also, when we reviewed the performance measurement briefing slides and templates, it was unclear whether the data collection and reporting cycle was accomplished.  Additionally, personnel stated that because the MV&S Program was in its early stages and was just gaining momentum, they are now able to determine the types of data measurement and reporting that are needed.

The Government Performance and Results Act of 1993[6] requires agencies to set goals, measure performance, and report on their accomplishments.  A key tenet of the Act is that agencies will develop 1) strategic plans–as well as annual performance plans that are linked to the strategic plan–that establish the organizational goals and objectives, and 2) strategies for achieving these goals.  Additionally, agencies are required to establish performance measures and benchmarks to identify gaps between actual and desired performance levels and mission outcomes.

The MV&S Program is a new program that began with the first MV&S Plan in October 2006.  Over the next 2 years, as the Program gained momentum, management stated that formalized procedures were being consolidated based on integrated involvement and lessons learned.

Determining performance goals for information technology investments is a critical factor for requesting funding in the budget formulation and submission process.  Performance measures are necessary to support the establishment of a sound strategic direction to ensure that decisive information technology investment and acquisition decisions are accomplished through business priority shifting and technological evolvement.  The lack of MV&S Program performance measures and reporting compliance could result in inefficient accomplishment of strategic information technology acquisitions, objectives, and expectations.

Recommendation

Recommendation 2:  The Chief Information Officer should include guidelines for performance measurement and reporting in the updated MV&S Plan.

Management’s Response:  The IRS agreed with this recommendation and will update the MV&S Plan to reference performance measures and reporting.  This update will be included in the Calendar Year 2009 annual publication.

 

Appendix I

 

Detailed Objective, Scope, and Methodology

 

The overall objective of this review was to determine whether the IRS MV&S Program is achieving the desired results.  To accomplish our objective, we:

I.                   Determined whether the enterprise-wide investment portfolio approved by the MITS organization Enterprise Governance Committee[7] supports modernizing front-line tax administration and the technical infrastructure.

A.    Conducted interviews with IRS personnel and requested and reviewed business organizations’ strategic plans.

B.     Interviewed IRS personnel and determined whether technical, functional, and integration reviews were conducted by the IRS to ensure that the system projects identified in the investment portfolio support the technical infrastructure.

II.                Determined whether the business organizations were involved in the investment strategy formulation process.

A.    Interviewed Portfolio Planning, Estimation, and Delivery Services Office management to determine the process for soliciting input from the business organizations for the budget formulation process.

B.     Reviewed documentation to determine whether data requests were sent to all business organizations.

C.     Reviewed the participants list for the investment strategy formulation meeting.

III.             Reviewed the coordination between the MV&S Program Office and other organizations responsible for portfolio management/governance, including the Capital Planning and Investment Control Office, Executive Steering Committees, and Financial Management Services organization.

A.    Interviewed MV&S Program Office and Capital Planning and Investment Control Office personnel to determine the roles and responsibilities regarding monitoring, tracking, and reporting the progress of the MV&S Program.

B.     Interviewed Financial Management Services organization personnel to determine the roles and responsibilities regarding budget formulation and adjustments within the MV&S Program.

C.     Reviewed the Executive Steering Committees’ meeting minutes to assess coordination with the MV&S Program Office.

IV.             Determined whether the IRS is frequently delivering smaller, more incremental releases.

A.    Interviewed MV&S Program Office personnel and reviewed the investment portfolio and Executive Steering Committees’ meeting minutes to determine whether projects are developed and frequently delivered via incremental releases.

B.     Reviewed project documentation for supporting costs and schedules that are being estimated and approved at the release level.  We selected a judgmental sample of 4 projects from the population of 23 projects in the approved investment portfolio.  This list of 23 projects provided by the IRS included only projects for the MV&S Program for the Fiscal Year 2009 cycle.  We obtained estimated costs and business area and respective owner information to determine whether projects were being estimated and approved at the release level.  The four projects were judgmentally selected based on the projects’ titles, time period from development to operations and maintenance, total cost, and project business area.  We used a judgmental sample because we did not plan to project the audit results.

V.                Determined whether the IRS is creating costing models that include investment, operations, and maintenance amounts.

A.    Interviewed personnel in the MV&S Program Office; the Portfolio Planning, Estimation, and Delivery Services Office; the Solution Concept Office; the Estimation Program Office; and the Capital Planning and Investment Control Office regarding the estimation process and models used to prepare project costs estimates.

B.     Reviewed the Government Accountability Office Assessing Risks and Returns:  A Guide for Evaluating Federal Agencies’ Information Technology Investment Decision-making[8] and the IRS operations and maintenance guidance.[9]

C.     Reviewed the supporting project cost documentation for the judgmental sample of four projects selected in Step IV.B.

VI.             Determined whether guidelines, processes, and procedures were in place for measuring and reporting results delivered by the MV&S Program.

A.    Interviewed MV&S Program Office personnel and reviewed the MV&S Plan to identify MV&S Program goals, objectives, outcomes, and performance measures to ensure program accomplishment.

B.     Verified whether measures are linked to IRS long-term strategic goals as defined in the 2005-2009 IRS Strategic Plan.

C.     Interviewed MV&S Program Office personnel to determine whether performance measures are incorporated into the Business Performance Management System.

D.    Interviewed MV&S Program Office personnel to determine whether specific guidelines were developed for periodically measuring and reporting MV&S Program accomplishments.

VII.            Determined the validity of data.  The IRS provided us with a report, identified as the project Basis of Estimate report, that contained support for each project’s total cost and timeline.  We reviewed the information in each Basis of Estimate report for reasonableness.  However, we did not conduct audit steps to verify the accuracy or completeness of the information because we did not use the information to make projections.

 

Appendix II

 

Major Contributors to This Report

 

Margaret E. Begg, Assistant Inspector General for Audit (Security and Information Technology Services)

Scott A. Macfarlane, Director

Danny R. Verneuille, Audit Manager

Kimberly R. Parmley, Lead Auditor

Paul M. Mitchell, Senior Auditor

Wallace C. Sims, Senior Auditor

Charlene L. Elliston, Auditor

 

Appendix III

 

Report Distribution List

 

Commissioner  C

Office of the Commissioner – Attn:  Chief of Staff  C

Deputy Commissioner for Operations Support  OS

Associate Chief Information Officer, Applications Development  OS:CIO:AD

Associate Chief Information Officer, Enterprise Operations  OS:CIO:EO

Director, Stakeholder Management  OS:CIO:SM

Chief Counsel  CC

National Taxpayer Advocate  TA

Director, Office of Legislative Affairs  CL:LA

Director, Office of Program Evaluation and Risk Analysis  RAS:O

Office of Internal Control  OS:CFO:CPIC:IC

Audit Liaison:  Director, Program Oversight Office  OS:CIO:SM:PO

 

Appendix IV

 

Glossary of Terms

 

Basis of Estimate

A tool used to estimate all task order proposals.  It includes specific sections that must be documented to justify the estimate proposal, such as work to be performed, scheduling estimates, and staffing summaries.

Business Performance Management System

A System designed to support the IRS’ Strategic Planning and Budgeting and Performance Management processes.  It supports executives and the senior leadership team, division managers, division planners, and analysts.  The System also brings value to the organization by providing 1) a centralized source of agency-wide planning and performance management data, 2) a “guided analysis tool” to facilitate planning and performance management at the enterprise, division, and program level, and 3) a “one-stop” source for information.

Earned Value Management

Involves measuring actual cost and work accomplished against the budgeted cost and planned work scheduled.  Variances are analyzed for decision making.

Enterprise Architecture

A unifying overall design or structure for an enterprise.  Enterprise architecture divides an enterprise into its component parts and relationships and provides the principles, constraints, and standards to help align business area development efforts in a common direction.  An enterprise architecture ensures that subordinate architectures and business system components developed within particular business areas and multiple projects fit together into a consistent, integrated whole.

Enterprise Governance Committee

The highest level recommending and decision-making body to oversee and enhance enterprise management of information systems and technology.  It ensures that strategic modernization and information technology program investments, goals, and activities are aligned with and support 1) the business needs across the enterprise, and 2) the modernized vision of the IRS.  It oversees investments, including validating major investment business requirements and ensuring that enabling technologies are defined, developed, and implemented.

Enterprise Life Cycle

A structured business systems development method that requires the preparation of specific work products during different phases of the development process.

Major Project (Investment)

Department of the Treasury specific criteria state that major information technology investments (or projects) have an annual cost equal to or greater than $5 million, or total lifecycle costs exceeding $50 million.

MITRE Corporation

Provides the IRS with independent, expert, and objective advice and guidance on strategic, technical, and program management issues.

Non-Major Project (Investment)

Any initiative or investment not meeting the definition of a major project.

Release

A specific edition of software.

Solution Concept Document

Provides the characteristics of the ability of the system.

 

Appendix V

 Management’s Response to the Draft Report

 

The response was removed due to its size.  To see the response, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.


[1] See Appendix IV for a glossary of terms.

[2] Annual Assessment of the Business Systems Modernization Program (Reference Number 2008-20-129, dated June 24, 2008).

[3] GAO/AIMD-00-21.3.1, dated November 1999.

[4] (Federal Acquisition Reform Act of 1996) (Information Technology Management Reform Act of 1996), Pub. L. No. 104-106, 110 Stat. 642 (codified in scattered sections of 5 U.S.C., 5 U.S.C. app., 10 U.S.C., 15 U.S.C., 16 U.S.C., 18 U.S.C., 22 U.S.C., 28 U.S.C., 29 U.S.C., 31 U.S.C., 38 U.S.C., 40 U.S.C., 41 U.S.C., 42 U.S.C., 44 U.S.C., 49 U.S.C., 50 U.S.C.).

[5] GAO 07-1134SP, dated July 2007.

[6] Pub. L. No. 103-62, 107 Stat. 285 (codified as amended in scattered sections of 5 U.S.C., 31 U.S.C., and 39 U.S.C.).

[7] See Appendix IV for a glossary of terms.

[8] GAO/AIMD-10.1.13, dated February 1997.

[9] IRS Basis for Parametic Estimation Guidance, dated October 23, 2007.