Analysis of Business Systems Modernization Cost, Schedule,
and Functionality Performance
October 2002
Reference Number: 2003-20-007
This report has cleared the Treasury
Inspector General for Tax Administration disclosure review process and
information determined to be restricted from public release has been redacted
from this document.
October
31, 2002
MEMORANDUM FOR
DEPUTY COMMISSIONER FOR MODERNIZATION
&
CHIEF INFORMATION OFFICER
FROM: Pamela J. Gardiner /s/ Pamela J. Gardiner
Acting Inspector General
SUBJECT: Final Audit Report - Analysis of Business Systems Modernization
Cost, Schedule, and Functionality Performance (Audit # 200220037)
This report presents the
results of our review of the cost, schedule, and functionality performance of
the Business Systems Modernization (BSM) program. The overall objective of this review was to determine how well
the Business Systems Modernization Office (BSMO) and modernization contractors
were performing against cost, schedule, and near-term functionality goals for
developing BSM projects. The BSM
program is one of the most complex and expensive efforts ever undertaken by the
Internal Revenue Service (IRS). The BSM
program is a very high-risk effort, but one that is essential if the IRS is to
meet the reforms mandated by the Congress and taxpayers, such as improved
customer service and increased productivity.
In summary, we found that
beginning in 2001, the BSM program has delivered business results by deploying
projects and learning valuable lessons that should help improve future
projects. Deployed projects have
increased the capacity of the IRS’ telephone system, improved the ability to
receive, route, and respond to taxpayer telephone calls, and provided refund
information via the Internet. However,
as reported in previous Treasury Inspector General for Tax Administration and
General Accounting Office reports, the BSM program has been experiencing
difficulties meeting the original cost, schedule, and functionality estimates
included in the BSM Spending Plans submitted to the Congress.
While the BSM
projects have cost more, taken longer, and delivered less functionality than
originally estimated, our analysis shows that cost and schedule performance is
improving. The BSMO forecasts that
project costs and schedules should be coming in closer to the estimates, and
the majority of the originally planned functionality is still scheduled for
delivery in the future. It is not
uncommon for organizations that are attempting to modernize their computer
systems to experience cost increases and schedule delays. According to a study performed by the
Standish Group, only 9 percent of large companies complete computer
modernization projects within cost and schedule budgets. However, because the IRS hired an
experienced contractor to lead the development and integration efforts for BSM,
we would have expected the cost and schedule estimates to have been more
accurate.
So far, the 8
BSM projects that are currently being developed and deployed have experienced cost
increases of 26 percent and delays averaging 13 months over initial
estimates. The majority of the cost
increases and delays occurred during the planning and design phases of these
projects, where cost increases were approximately 89 percent over original
estimates. The BSMO forecasts that
future project costs and schedules will be much closer to the estimates
detailed in BSM Spending Plans.
Although the anticipated costs and schedules are expected to be closer
to the estimates provided to the Congress, actual costs and schedules could
still vary significantly from the estimates.
Since the purpose of this
review was to identify and analyze the cost, schedule, and functionality
performance compared to the original project estimates, we do not make any
recommendations in this report.
However, we believe the information provided in this report can be
helpful to the BSMO in its efforts to improve the management and delivery of
BSM projects and benefits.
Management’s
Response: The Deputy Commissioner for Modernization
& Chief Information Officer agreed with our assessment and provided
comments on how the IRS is making improvements to further reduce cost increases
and schedule delays. Management’s
complete response to the draft report is included as Appendix XIV.
Copies of this report are also being sent to the IRS
managers affected by the report. Please
contact me at (202) 622-6510 if you have questions or Scott E. Wilson,
Assistant Inspector General for Audit (Information Systems Programs), at (202)
622-8510.
The Majority of Originally Planned Near-Term Functionality Is Still Planned for Future Delivery
Overall Observation on Business Systems Modernization Performance
Appendix I – Detailed Objective, Scope, and Methodology
Appendix II – Major Contributors to This Report
Appendix III – Report Distribution List
Appendix IV – Time-Line and Details on Business Systems
Modernization Spending Plans
Appendix V – Background Details (Methodology)
Appendix VI – Background Details (Project Groupings)
Appendix VII – Background Details (Cost and Schedule Data
Breakdown)
Appendix VIII – Cost Performance Details
Appendix IX – Schedule Performance Details
Appendix X – Functionality Performance Details
Appendix XI – Industry Performance Statistics
Appendix XII – Previous Audit Reports
Appendix XIII – Selected Modernization Project Descriptions
Appendix XIV – Management’s Response to the Draft Report
The Internal Revenue Service (IRS) is challenged with modernizing its computer systems and business processes and practices. This effort, known as the Business Systems Modernization (BSM), is projected to last up to 15 years. The BSM program is one of the most complex and expensive efforts ever undertaken by the IRS. The BSM program is a very high-risk effort, but one that is essential if the IRS is to meet the reforms mandated by the Congress and taxpayers, such as improved customer service and increased productivity.
The IRS hired the Computer Science
Corporation (known as the PRIME contractor) as the lead integrator for
BSM. The PRIME contractor heads up
several leading technology and management companies that form the PRIME
Alliance. The IRS established the
Business Systems Modernization Office (BSMO) to manage the BSM program and to
provide oversight of the PRIME contractor.
Executives with a wide range of experience in the private sector as well
as seasoned IRS executives were brought in to manage the BSMO.
The BSMO provides the Congress
justification to release funds specifically set aside for the BSM effort by submitting
spending plans. These BSM Spending
Plans contain the estimated costs to deliver set features for each project
within a scheduled period of time.
There have been 5 BSM Spending Plans and 2 interim BSM Spending Plans
requesting a total of $968 million between April 1999 and December 2001. See Appendix IV for a timeline with details
on the BSM Spending Plans.
Beginning in 2001, the BSM program has delivered results through projects such as the Customer Relationship Management Exam, Customer Communications, Enterprise Systems Management, and Security and Technology Infrastructure Release. For example, it installed an upgraded telephone communications system that improved the IRS’ ability to receive, route, and respond to the more than 150 million taxpayer telephone calls each year. It also deployed an application that allows taxpayers to check on the status of their refunds via the Internet.
Based on these early projects, the BSM program has and continues to learn valuable lessons in order to improve its processes and deliver timely, quality results. However, as reported in previous Treasury Inspector General for Tax Administration and General Accounting Office reports, the majority of BSM projects require spending more money and time to deliver fewer features than originally planned.
The objective of this review was to determine how well the BSMO and modernization contractors were performing against cost, schedule, and near-term functionality goals. Our review consisted of analyzing 20 BSM projects, which are presented and grouped throughout this report according to their current developmental status. A detailed description of the methodology and analyses of the data used to provide the reported results is included in Appendix V, and a listing of the projects reviewed is provided in Appendix VI.
It is important to note that the information and conclusions
we present in this report are subject to change due to the types of data used
during our analyses. Since none of the
20 BSM projects were fully completed, a majority of the results were calculated
based upon BSMO forecasted data.
Forecasted data are estimates to complete each phase of the 20 projects
based upon the BSMO’s estimation processes.
Therefore, the information presented in this report may change as
projects mature and actual figures become available for future analyses or as
the BSMO revises its forecasted data. A
detailed breakdown on the availability of actual cost and schedule data versus
BSMO forecasted cost and schedule data is available in Appendix VII.
To accomplish our objective, we obtained and reviewed documentation and conducted interviews with individuals from the BSMO. We relied on information accumulated by the BSMO as of February 2002 but did not verify the accuracy of the information. The audit was conducted at the IRS’ National Headquarters Office and the BSMO facilities in New Carrollton, Maryland, between January 2002 and June 2002 in accordance with Government Auditing Standards. Detailed information on our audit objective, scope, and methodology is presented in Appendix I. Major contributors to the report are listed in Appendix II.
We analyzed the BSM projects’
actual and forecasted cost and schedule data and found that the projects were
costing more and taking longer than originally estimated. However, based on the BSMO forecasts for
estimated time and costs to complete the projects, the cost increases and
schedule delays are decreasing over time.
To
provide further insight, we analyzed the cost increases and schedule delays by
BSM Spending Plan and development phase.
The majority of cost increases and schedule delays have occurred in the
planning and design phases, while more limited cost increases and schedule
delays are forecasted for the development and deployment phases. Also, cost increases are trending significantly
downward and schedule delays are trending slightly downward between Spending
Plans 1 and 4.
Based
on BSMO actual and forecasted cost data, the BSM program has experienced a $75
million or 24 percent cost increase for the 20 projects that have been
initiated since 1999, as shown in the summary table below. Detailed data on the cost increases are
provided in Appendix VIII.
Table 1: Cost
Performance Results to Date
|
BSM Projects By Current Developmental Status |
Cost Increase (Cumulative) |
Percentage Increase |
|---|---|---|
|
8 in
Development and Deployment |
$62.56 million |
25.60% |
|
5 in Planning and Design |
$ 8.16 million |
27.10% |
|
3 Cancelled |
$ 2.47 million |
13.35% |
|
3 Testing Laboratories |
$ 1.57 million |
6.93% |
|
1 Delayed |
N/A |
N/A |
|
|
$74.76
million |
23.69% |
Source: Data summarized from Appendix VIII.
The projects’ cost increases are
decreasing by Spending Plan, with the majority of cost increases occurring
during the planning and design phases.
While the trends are encouraging, they could change in the future since
cost results from the later BSM Spending Plans contain only forecasted
data. The accuracy of the forecasts
will affect the trend results shown in the following graph.
Graph 1 was removed due to its
size. To see Graph 1, please go to the Adobe
PDF version of the report on the TIGTA Public Web Page.
Cost increases in the first 2 BSM
Spending Plans were significant (83 and 47 percent, respectively). However, current BSMO forecasts show that
cost increases are less than 15 percent for the fourth BSM Spending Plan. While the current trend line is encouraging,
it may be unrealistic to expect such a sharply declining trend will
continue. We expect that the cost trend
will be less dramatic once actual cost figures are obtained, although we are optimistic
that the downward trend for cost increases will continue.
The 8 projects that have completed
the planning and design phases experienced an approximately 89 percent cost
increase ($52 million) during those phases.
This equates to the majority of the cost increases that have been
incurred on these projects to date.
BSMO officials state that the reliability of costs estimates for the
development and deployment phases is much greater than that for the planning
and design phases.
It is not uncommon for
organizations that are attempting to modernize their computer systems to
experience cost increases and schedule delays.
According to a study performed by the Standish Group, only 9 percent of
large companies complete computer modernization projects within cost and
schedule budgets. However, because the
IRS hired an experienced contractor to lead the development and integration
efforts for BSM, we would have expected the cost and schedule estimates to have
been more accurate. Both the BSMO and
PRIME contractor may have underestimated the enormous size and complexity of
the BSM effort. According to the
Standish Group study, the average cost of a computer system development project
for a ‘large’ company is $2.3 million.
The BSM program has already requested approximately $1 billion for its
projects, and it is expected that about $2.5 billion will be needed over the
next 5 years.
Based on BSM actual and forecasted
schedule data, the projects are experiencing schedule delays of 6 to 13 months,
as shown in the following table.
Table 2: Schedule Performance by Project Groupings
|
BSM Project
Groupings |
Avg. Delay
per Project |
|---|---|
|
8 projects currently in Development and Deployment
Phase Planning and Design Phase Development and Deployment Phase |
13
months 7.7
months 5.5
months |
|
5 projects currently in Planning and Design Phase |
6 months |
Source: Comparison of the originally scheduled
completion dates in the BSM Spending Plans to actual or forecasted completion
dates provided by the IRS.
The projects currently in the development and deployment
phases are experiencing an average delay of 13 months. However, 7.7 months of this delay occurred
during the planning and design phases compared to the 5.5 months delay to date
for the development and deployment phases.
BSMO officials state that the reliability of
schedule estimates for the development and deployment phases is much greater
than that for the planning and design phases.
Our analysis shows that schedule delays are declining slightly by BSM Spending Plan, with more delays occurring during the planning and design phases. For the first 2 BSM Spending Plans, the projects were experiencing an average schedule delay of approximately 9 months. For BSM Spending Plans 3 and 4, the average schedule delay is between 5 and 6 months. While this trend is also encouraging, we again do not expect the trend to continue at the same rate. The schedule figures for BSM Spending Plans 3 and 4 contain mainly forecasted data, and the accuracy of the forecasts will affect the trend results. In addition, the IRS Commissioner has stressed the need for quality deliverables versus schedule accuracy. Additional detailed information on schedule analysis is provided in Appendix IX.
While faced with many challenges,
the BSM program was able to deliver business features from projects such as
Customer Relationship Management (CRM) Exam, Customer Communications (CC), Security and
Technology Infrastructure Release (STIR), and Enterprise Systems Management (ESM). Several projects were planned for delivery in the 2001 and 2002
Filing Seasons; however, only the CRM Exam project has delivered 100 percent of
its originally planned functionality. A
listing of the projects that were planned for delivery in the 2001 and 2002
Filing Seasons is provided in Appendix X.
The CC project was one of the
first to be deployed. The project
delivered features such as Internet and telephone access to refund and filing
information. The CC project also
increased the capacity of the IRS’ existing telephone system. However, not all originally planned features
have been delivered. For instance,
systems management, administrative, and management information systems
capabilities have all been delayed for possible future delivery.
As shown in the following table,
for all the projects that were planned for completion by the 2002 Filing
Season, only 46 percent of their features have been either fully or partially
delivered. Fourteen percent of the
features have been suspended, while the remaining almost 40 percent are still
in progress. According to the BSMO, a
majority of the functionality originally envisioned is still planned for future
delivery in Fiscal Years 2002 and 2003.
A listing of the current projects’ planned delivery and their projected
future delivery dates is provided in Appendix X.
Table 3:
Planned Functionality Delivered
|
Project |
Total
Planned |
Full |
Partial |
Suspended |
In
Progress |
|---|---|---|---|---|---|
|
CADE |
1 |
0 |
0 |
0 |
1 |
|
CRM Exam |
1 |
1 |
0 |
0 |
0 |
|
CC |
10 |
4 |
2 |
4 |
0 |
|
e-Services |
5 |
0 |
0 |
0 |
5 |
|
ESM |
4 |
0 |
2 |
0 |
2 |
|
STIR |
7 |
4 |
0 |
0 |
3 |
|
TOTAL |
28 |
9 |
4 |
4 |
11 |
|
PERCENT |
100% |
32.1% |
14.3% |
14.3% |
39.3% |
Source: This table includes a comparison of major functionality listed in the BSM Spending Plans (see Appendix X) to actual delivery as confirmed by the IRS.
The
BSMO has identified cost and schedule estimation as 1 of the 12 processes
needing improvement during 2002. As
process improvements are implemented, cost increases and schedule delays could
be reduced as the BSM program matures.
However, risks associated with building
systems (such as complexity and integration) significantly increase as projects
progress through their life cycle (from the planning and design phases to the
development and deployment phases). The
majority of the BSM projects are entering or currently in the high-risk
phases. As project risks increase, the
probability of cost overruns and schedule delays will also increase.
Because the BSM program is still maturing, it is difficult to determine whether improvements to processes have occurred until the projects are completed and actual cost and schedule data are available. If the BSMO and PRIME contractor successfully implement process improvements, we would expect to see the BSM program make significant strides to deliver quality products within a reasonable amount of time and money. Alternately, if the BSMO and PRIME contractor are not successful in implementing process improvements, the BSM program will continue to be plagued with cost increases and schedule delays that prevent it from effectively and efficiently delivering the modernized technology that the IRS needs to improve service to taxpayers.
Management’s Response: The Deputy Commissioner for Modernization & Chief Information Officer agreed with our assessment and provided comments on how the IRS is making improvements to further reduce cost increases and schedule delays. Management’s complete response to the draft report is included as Appendix XIV.
Appendix I
Detailed Objective, Scope, and Methodology
The overall objective of this review was to determine how well the Business Systems Modernization Office and modernization contractors were performing against cost, schedule, and near-term functionality goals. To accomplish our objective, we:
I. Determined the percentage of modernization projects that were over/under or on target with budgeted amounts with respect to dollar cost.
A. Identified and documented the planned dollar amounts for each project by milestone or time period.
B. Identified and documented the actual dollar amounts that had been spent for each project by milestone.
C. Processed the information that had been collected and performed variance analyses.
D. Verified all analyses with appropriate Internal Revenue Service (IRS) representatives.
II. Determined the schedule variances for each project with respect to its projected completion dates versus its actual completion dates.
A. Identified
and documented the planned completion dates for each project by spending plan.
B. Identified
and documented the actual completion dates for each project.
C. Processed
the information that had been collected and performed variance analyses.
D. Verified
all analyses with appropriate IRS representatives.
III. Determined the amount of unfulfilled tangible deliverables/benefits for near-term projects versus planned tangible deliverables/benefits.
A.
Identified and documented the estimated deliverables/benefits
for each project originally planned to be delivered in Fiscal Year 2002 or
earlier.
B.
Identified and documented the actual functionality and
deliverables/benefits for each project originally planned to be delivered in
Fiscal Year 2002 or earlier.
C.
Processed the information that had been collected and
performed variance analyses.
D.
Verified all analyses with appropriate IRS representatives.
Appendix II
Major Contributors to This
Report
Phung Son Huu
Nguyen, Senior Auditor
Perrin T. Gleaton, Auditor
Appendix III
Commissioner N:C
Deputy Commissioner N:DC
Associate Commissioner, Business Systems Modernization M:B
Deputy Associate Commissioner, Program Management M:B:PM
Deputy Associate Commissioner, Systems Integration M:B:SI
Director, Budget Policy, Planning, and Programs M:BP
Chief Counsel CC
National Taxpayer Advocate TA
Director, Legislative Affairs CL:LA
Director, Office of Program Evaluation and Risk Analysis N:ADC:R:O
Office of Management Controls N:CFO:F:M
Audit Liaison:
Associate
Commissioner, Business Systems Modernization
M:B
Appendix IV
The chart and table were removed due to their size. To see the chart and table, please go to the Adobe PDF
version of the report on the TIGTA Public Web Page.
Appendix V
There are many different ways to
analyze and present the data provided by the Internal Revenue Service
(IRS). Depending on the methodology
used, the outcomes can be significantly skewed in support of or contrary to the
IRS’ results. The Treasury Inspector
General for Tax Administration’s (TIGTA) analytical methods attempt to report
the results in a logical and impartial manner.
The TIGTA’s methodology takes actual
results for cost, schedule, and functionality and compares them to
the originally planned cost, schedule, and functionality
to determine if any variances exist.
This methodology differs from the IRS’ variance methodology performed
within Business Systems Modernization (BSM) Spending Plans 3, 4, and 5. It is not the TIGTA’s intention to judge the
IRS’ methods of variance analysis but to present our perspective on the
data. The difference in methodology is
provided using a basic example of a cost variance analysis. Suppose the following data are available for
analysis:
Example of Planned Data for Project A
|
Spending Plan |
Amount Requested |
For Project Phase |
|---|---|---|
|
Plan 1 |
$10 Million |
Planning Phase |
|
Interim 1 |
$5 Million* |
Planning Phase |
|
Plan 2 |
$30 Million |
Development Phase |
|
Interim 2 |
$7 Million* |
Planning Phase |
|
Plan 3 |
$25 Million* |
Development Phase |
|
Plan 4 |
None |
None |
|
Plan 5 |
$5 Million* |
Development Phase |
* The IRS revises its
originally planned amount and requests additional amounts to complete that
particular project phase.
Example of Actual Data for Project A
|
Project Phase |
Actual Amount |
|---|---|
|
Planning Phase |
$30 Million |
|
Development Phase |
$60 Million |
Forecasted Amounts
The tables were removed due to their size. To see the tables, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.
A majority of the analyzed
projects had not yet been completed.
Therefore, the actual amounts for those projects were unavailable. However, the IRS had been forecasting each
project’s actual amounts at completion within the BSM Spending Plans. When appropriate, we used the forecasted
amounts as the actual amounts during the analyses. Based on this fact, it is important to know that the results
within this report may change at any time due to the following reasons:
·
Project phases come to
completion and the actual amounts become available. This will replace the forecasted amounts with the actual amounts
within the analyses causing the variance to increase or decrease based upon the
new data.
·
The IRS releases another BSM
Spending Plan and revises its forecasted amounts for the projects. This will cause the current variance amounts
to increase or decrease based upon the new forecasted amounts.
Using the IRS’ forecasted cost amounts during the analysis
may produce distorted results because they are amounts derived by the IRS’
estimation abilities. However,
according to the IRS, its estimation processes have matured and improved over
time. With this in mind, the forecasted
cost amounts are the best available to perform these types of analyses. A detailed breakdown of the availability of
actual costs versus IRS forecasted costs used for the report is provided in
Appendix VII.
Appendix VI
A total of 20 projects have been funded through the Business Systems
Modernization (BSM) Spending Plans.
Projects start at different times and are currently at different phases
of design and development. There are
five basic phases that a project undergoes throughout its life cycle. These life cycle phases are called
milestones and are defined as:
Planning and Design Phase
Ø
Milestone 1:
Strategic Plan
Ø
Milestone 2:
Concept Definition
Ø
Milestone 3:
System Design
Development and
Deployment Phase
Ø
Milestone 4: Enterprise Deployment
Ø Milestone 5:
Post-Deployment Evaluation
In order to appropriately and accurately present the
data analyses on the BSM projects, we grouped them according to their current
phases. Project name abbreviations are
defined on the next page.
The chart and table were removed due to their size.
To see the chart and table, please go to the Adobe PDF version of the report on
the TIGTA Public Web Page
Appendix VII
We relied on information accumulated by the Internal Revenue
Service (IRS) as of February 2002 and did not verify the accuracy of the
information. The majority of the data
consisted of forecasted cost amounts and completion dates since most of the
projects were still in progress.
Forecasted numbers and dates are based upon the IRS’ estimation
processes and those figures may be different from the actual amounts at
completion.
The detailed breakdown of available actual versus forecasted cost amounts
is shown in Figure A: Cost
Breakdown. The detailed breakdown
of available actual versus forecasted completion dates is shown in Figure
B: Schedule Breakdown. These analyses provide a picture of where
changes to cost and schedule results may occur due to the possibility of future
changes to forecasted amounts and dates.
The breakdown is shown based on data points. A data point is considered an originally planned figure (cost or
schedule) compared to its actual or forecasted figure.
Figures A and B
were removed due to their size. To see
these Figures, please go to the Adobe PDF version of the report on the TIGTA
Public Web Page.
Appendix VIII
Tables 1 and 2 were removed due to their size. To see the Tables, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.
Appendix IX
Table 1 and Graph 1 were removed due to
their
size. To see the Table and Graph, please go to
the Adobe PDF version of the report on the TIGTA Public Web Page.
Appendix X
Appendix XI
Industry Performance Statistics
From the Standish Group:
·
Only
16 percent of projects in the United States are considered successful.
·
Only
9 percent of large companies complete projects on schedule and on budget.
·
Fifty-three
percent of all projects cost 189 percent over original estimates.
·
Average
schedule overrun for projects in trouble was 222 percent.
·
Delivered
products contained only 61 percent of originally specified features.
·
Thirty-one
percent of all projects are cancelled.
From the
Parsons Group:
·
Fifty percent of finished
projects contain less than 70 percent of their planned functionality.
·
Sixty to 80 percent of
Enterprise Resource Planning (ERP) systems will either fail or not meet
expectations.
Appendix XII
Modernization Project Teams Need to Follow Key Systems Development Processes (Reference Number 2002-20-025, dated November 2001).
IRS Needs to Better Balance Management Capacity with Systems Acquisition Workload (GAO-02-356, dated February 2002).
The Business Systems Modernization Office
Needs to Strengthen Its Processes for Overseeing the Work of the PRIME Contractor (Reference Number 2002-20-059, dated
March 2002).
Continued Progress Modernizing IRS Depends on Managing Risks (GAO-02-715T, dated May 2002).
Appendix XIII
Selected Modernization
Project Descriptions
Customer
Relationship Management (CRM) Exam
– The CRM Exam project is a software package that computes complex taxes.
Customer
Communications (CC) –
The CC project increased the capacity of the Internal Revenue Service’s (IRS)
existing telephone system. The project
also delivered features such as Internet and telephone access to refund and
filing information. The Internet refund
portion was delivered by a project called Internet Refund Fact of Filing, which
was originally planned to be part of the CC project.
Enterprise
Systems Management (ESM)
– The ESM project will help the IRS monitor all IRS computer systems and
networks to ensure they are consistently available to the employees relying on
them. The project also provides for the
consolidation of 19 help desks throughout the IRS into a single help desk to
better serve the users of the systems and networks.
Security
and Technology Infrastructure Release (STIR) – The STIR project will provide the secure technical
infrastructure to support and enable the delivery of the IRS’ modernized
business systems.
Appendix XIV
Management’s Response to the Draft Report
The response was removed due to its
size. To see the complete response,
please go to the Adobe PDF version of the report on the TIGTA Public Web Page.