Inadequate Planning Leads to Costly Enterprise License
Agreements for Software
August 2003
Reference
Number: 2003-10-170
This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.
August
22, 2003
MEMORANDUM FOR
CHIEF, AGENCY-WIDE SHARED SERVICES
CHIEF
INFORMATION OFFICER
FROM: Gordon C. Milbourn III /s/ Gordon C.
Milbourn III
Assistant Inspector General for Audit (Small Business and
Corporate Programs)
SUBJECT: Final Audit Report - Inadequate Planning
Leads to Costly Enterprise License Agreements for Software (Audit # 200210022)
This
report presents the results of our review of the Internal Revenue Service’s
(IRS) use of Enterprise License Agreements (ELA). The overall objective of our review was to evaluate whether the
use of ELAs is cost effective for the IRS.
In
Fiscal Year 2001, the Treasury Inspector General for Tax Administration (TIGTA)
received an allegation from an IRS employee regarding the IRS’ use of the
ELAs. The IRS employee alleged there
were several ELAs that the Office of Procurement has in place which are not
cost-effective to the IRS. While the
TIGTA Office of Investigations did not find evidence of criminal activity, our
review showed there is some basis to the allegation regarding the
cost-effectiveness of these agreements.
In
summary, the IRS is not utilizing the ELAs in a cost-effective manner. We reviewed four of the nine ELAs the IRS
currently has in place. Two of the four
ELAs included in our review were significantly underutilized and one was
overutilized. For another ELA, we could
not identify how many software packages were in use. Our reviews of the contract files did not reveal any
justifications for establishing these ELAs or analyses documenting how many
software products and/or how much maintenance the IRS needed.
We
recommended that the Director, Office of Procurement, take the appropriate
action to ensure the IRS properly analyzes and documents the business case,
cost justification, Federal Government cost estimate, and required terms and
conditions for all future ELAs. The
Director, Enterprise Operations, should review and, if necessary, redefine
requirements for all the ELA contracts the IRS has established and work with
the Director, Office of Procurement, to seek appropriate price reductions. Additionally, the Director, Enterprise
Operations, should monitor use of the ELAs on a regular basis and work with the
Office of Procurement to adjust agreements accordingly.
Management’s Response: IRS
management agreed with the recommendations presented in our report and has
completed corrective actions.
Specifically, the Offices of Procurement and Enterprise Operations are
working cooperatively to ensure ELAs issued reflect sound business judgment and
are documented in accordance with applicable Federal acquisition regulations,
policy, and guidance. Additionally, the
Office of Enterprise Operations has worked with the Office of Procurement to
implement activities to adjust requirements for ELAs, seek appropriate price
reductions, and monitor the use of ELAs.
While IRS management agreed
with our recommendations, they disagreed with our conclusion that the IRS’ use
of ELAs is not cost-effective. IRS
management also disagreed with the audit findings that hold them to a higher
standard for file documentation than required by regulation or policy, the
methodology used to project measurable benefits, and the measurable benefits
projected.
Management’s complete
response to the draft report is included as Appendix V.
Office of Audit Comment: IRS
management did not provide documentation during the time of our review to
support the significant cost savings they claim these ELAs have achieved. Additionally, we believe it is important for
the IRS to properly document the business reasons why the IRS needs the ELAs,
how much software and/or maintenance is needed, and the cost estimates associated
with the requirements. Without this
documentation, the IRS has no justification for establishing the ELA and
expending Government funds. Further,
while we recognize the IRS may not be able to negotiate the full cost savings
presented in our outcome measures because the price per million instructions per second may be lower
depending upon volume, we believe it can be a starting point to determine what
is a reasonable amount to pay for the ELA.
Because the IRS took
appropriate corrective actions in response to our recommendations, we do not
intend to elevate our disagreement with management over our findings to the
Department of the Treasury for resolution.
Copies of this report are also being sent to the IRS
managers who are affected by the report recommendations. Please contact me at (202) 622-6510 if you
have questions or Daniel R. Devlin, Assistant Inspector General for Audit
(Headquarters Operations and Exempt Organizations Programs), at (202) 622-8500.
The
Internal Revenue Service Is Not Utilizing the Enterprise License Agreements
Cost-Effectively
Appendix I – Detailed Objective, Scope, and Methodology
Appendix II – Major Contributors to This Report
Appendix III – Report Distribution List
Appendix IV – Outcome Measures
Appendix V – Management’s Response to the Draft Report
In keeping with the latest approach to delivering Information Technology services, the Internal Revenue Service (IRS) has implemented the use of Enterprise License Agreements (ELA). An ELA is a means to acquire commonly used, standards-compliant software or software maintenance under specified terms and conditions for “enterprise-use,” which means agency-wide. ELAs give the IRS the ability to lease software licenses for the whole organization and entitle the IRS to all the software upgrades during the period of the agreement. The IRS pays a set fee annually for maintenance until the completion of the agreement.
The primary purpose of entering into an ELA is to obtain lower prices and other favorable terms by leveraging the organization’s buying power under one supplier agreement. In addition to obtaining lower prices on software leases, the ELAs should also reduce acquisition and support costs. While cost savings is the biggest benefit in establishing ELAs, there are other advantages. They include:
While the ELAs can offer significant cost savings, they need to be properly planned. If not, ELAs can be costly to an organization. Proper planning is important because, while organizations can add products and licenses to the ELA, they often cannot remove them from the agreement if downsizing occurs. Another possible disadvantage, depending on the agreement, is the organization does not own or have the right to use the software when the agreement either expires or is terminated.
According to Procurement function personnel, ELAs help the IRS in two ways: savings and communications. The communication among the end users at the IRS will be improved because everyone will be using the same software and upgrades. Software on the ELAs is purchased by the number of seats needed for desktop applications or the number of million instructions per second (MIPS) for mainframe usage, instead of by individual packages for each computer/mainframe.
The Federal Acquisition Regulation (FAR) stipulates that agencies shall perform acquisition planning and conduct market research for all acquisitions. The planning must address all the technical, business, management, and other significant considerations that will control the acquisition. Considerations that should be addressed include a statement of need, cost goals, rationale for quantity, performance standards, and delivery or performance period. The FAR further states that agencies are required to procure supplies in such quantity that will both result in the total cost and unit cost most advantageous to the Federal Government, where practicable, and not exceed the quantity reasonably expected to be required by the agency.
The Internal Revenue Manual states that the Director, Office of Procurement, is responsible for planning, directing, coordinating, and controlling the procurement program of the IRS. Specifically, the Director, Office of Procurement:
· Ensures that all contractual commitments for equipment, supplies, and services are made within the framework of Federal and Departmental statutes and regulations, internal policy, and sound business judgment.
· Procures, in a timely, ethical manner, goods and services that meet the needs of the IRS, providing the best value to the Federal Government considering cost, price, and other factors.
The IRS Office of Procurement provides guidance outlining factors that should be considered when planning an acquisition. These include determining:
· What is needed.
· The quantities or level of effort that will be needed during the current fiscal year and subsequent fiscal years.
· Why the requirement (product/service) is needed.
· Who needs the requirement (product/service).
Additionally, a memorandum was issued in July 1999 from the Director, Technical Contract Management Division, providing procedures for the preparation and processing of information technology acquisitions. The procedures require that requisitions over $1 million include a requirements analysis package, business case/business needs document, or other project approval/investment decision documentation.
The Information Technology Services (ITS), Enterprise Operations organization, includes two branches involved with ELAs and/or with determining the IRS’ computer capacity: the Capacity Management Branch and the Contracts and Orders Management Branch. Responsibilities of the Capacity Management Branch include:
· Performing capacity planning, performance analysis, and modeling for all IRS mainframe Tier I systems and the distributed Tier II servers.
· Monitoring systems activity and resource utilization.
· Collecting and maintaining systems resource utilization data.
· Providing technical support and guidance to business areas.
The Contracts and Orders Management Branch manages and optimizes contractual acquisitions in support of the IRS modernization effort, Tier I and Tier II, and telecommunications infrastructures. This Branch also manages and facilitates enterprise maintenance, software licenses, and systems training and operations to achieve the best possible service at the lowest possible total cost of ownership.
We selected a sample of four ELAs from the nine ELAs the IRS had established during the time of our review.
· The first ELA provides software services for automation and control of IBM Tier I Multiple Virtual Storage and Tier II UNIX and NT operating systems.
· The third ELA offers the IRS a complete family of database engines and supporting products on which to deploy agency-wide, mission critical applications.
· The fourth ELA provides software for Tier II systems. The IRS uses this software for key applications, including the Integrated Network Operations Management System, Integrated Personnel System, and Web-Travel Reimbursement and Accounting System.
In Fiscal Year (FY) 2001, the Treasury Inspector General for Tax Administration (TIGTA) received an allegation from an IRS employee regarding the IRS’ use of ELAs. The IRS employee alleged there were several ELAs the IRS currently has in place which are not cost-effective. When the allegation was received, we worked with the TIGTA Office of Investigations (OI) to determine if any criminal activity existed in the procurement of these ELAs. The OI did not find evidence of criminal activity. However, based on the TIGTA’s assessment, there is some basis to the allegation regarding the cost-effectiveness of the agreements.
The audit work was performed from March 2002 to March 2003 in the IRS’
Office of Procurement in Oxon Hill, Maryland, and the Office of ITS in New Carrollton,
Maryland. The audit was conducted in
accordance with Government Auditing
Standards.
Detailed information on our audit objective, scope, and methodology is presented in Appendix I. Major contributors to the report are listed in Appendix II.
The IRS is not utilizing the ELAs in a cost-effective manner. They have significantly underutilized two of the four ELAs we reviewed. In a third ELA, the IRS is overutilizing the software authorized by the agreement. For the fourth ELA, the IRS could not provide adequate support for the number of users.
In April 2002, the IRS established a Tier I Review Group, which included representatives from the Offices of Procurement and ITS, to address issues concerning contract limitations, software licensing, and operations support for modernization projects. Because of severe budget constraints making it imperative to reduce costs, the Group focused on software licensing issues, specifically two of the ELAs included in our review. These ELAs were selected by the Group because of their importance to the Tier I environment and because the Group believed the ELAs were underutilized.
The study revealed, in an August 2002 interim report, that the IRS is underutilizing the number of MIPS available for the two ELAs. The study showed that the 2 ELAs are very costly and commit the IRS to expenditures exceeding $100 million over an extended period for MIPS levels that have not been and may not be realized for years to come, and for products that are either underutilized or not in use at all. An advantage to the agreements is that they provide maximum flexibility for use of covered products across the IRS computing environment without having to closely track and manage their use or administer more complex licensing arrangements. While the cost per MIPS or seat may be low, the advantage may be lost when products are not being used at authorized levels.
Specifically, this study revealed for 1 ELA that the IRS anticipated using only an average of 14 percent of the MIPS available in FYs 2002 and 2003 and only 24 percent in FY 2004. The IRS paid approximately $14.6 million for the ELA in FYs 2002 and 2003 and will pay $10.5 million in FY 2004. Further, our review of the agreement shows the number of contracted MIPS increases approximately 30 percent for the last 3 years of the contract. The IRS is currently in year 3 of a 10-year contract with a value of approximately $81 million.
For the second ELA, the study identified that the IRS anticipated using only an average of 23 percent of the MIPS available in FYs 2002 and 2003 and only 38 percent in FY 2004. The IRS paid the contractor $6.7 million in FYs 2002 and 2003 and will pay $4.7 million in FY 2004 for the ELA. The IRS is currently in year 4 of a 6-year contract with a value of approximately $22.4 million.
Using the average percentages of unused MIPS for these 2 ELAs, we determined that the IRS has potentially paid as much as $17.6 million in FYs 2002 and 2003 for MIPS not used. Additionally, the IRS will potentially pay as much as $8 million in FY 2004 for MIPS not used, based on current usage projections. See Appendix IV for further details. While we recognize the IRS may not be able to negotiate the full cost savings based on these percentages of costs because the price per MIPS may be lower depending upon volume, it can be a starting point to determine what is a reasonable amount to pay for the ELA.
The Tier I Review Group provided general recommendations addressing procedures for establishing ELAs and specific options to address the two ELAs. These recommendations included developing a comprehensive strategy for future contractual support of the computing centers with respect to hardware and software maintenance, and revamping the way future software ELAs and other software licensing arrangements are pursued and established. The options available for the specific ELAs in the IRS study include possibly renegotiating the agreements, terminating them for convenience, or allowing them to go to term and concentrating efforts on replacement agreements.
Our review of the third ELA indicated that the IRS did not properly estimate the amount of MIPS needed for the software provided by this agreement and is currently overutilizing the MIPS. Additionally, the IRS did not have any documentation for, nor could it explain, how the number of estimated MIPS was determined. The IRS is currently paying approximately $5 million each year for the use of this software; the ELA covers a 6-year period. However, IRS personnel advised us that they are currently using over the amount of MIPS authorized by the agreement. Therefore, the IRS needs to renegotiate with the vendor for additional MIPS and will possibly have to pay for the MIPS at a higher rate than the original negotiated price. If the requirements had been adequately established before the agreement was signed, the IRS may have been in a better negotiating position and may have been able to receive a bigger discount (on a MIPS basis) for the software product.
On the last ELA in our review, the IRS purchased software licenses and maintenance for 130,000 seats. While IRS personnel explained that this estimate was based on 105,000 IRS employees and 25,000 vendor personnel using these licenses, we could not find any documentation supporting how it was determined that all IRS employees needed this software product. Additionally, the IRS could not provide us with support for how many employees and vendor personnel currently have access to and are using the software product. IRS staff advised that this software license is for an enterprise-wide application that allows access to various web-based programs used widely in the IRS. The value of this ELA is approximately $34 million.
Procurement function officials advised us that justifications were documented for these agreements. However, our reviews of the contract files did not reveal any justifications for establishing these ELAs, analyses documenting how many software products and/or how much maintenance the IRS needs, or estimates regarding the costs to acquire the software products and maintenance.
The FAR requires agencies to perform acquisition planning for all acquisitions. The FAR further states that the purpose of the planning is to ensure that the Federal Government meets its needs in the most effective, economical, and timely manner. Additionally, we believe it makes good business sense to properly document and justify the necessity for all purchases of significant value.
Procurement personnel explained that these analyses were not necessary because the IRS already had this software and maintenance covered under other contracts and the required planning and analysis were completed for the prior contracts. Because of the magnitude of these ELAs, the planning and analysis performed for prior contracts, in our opinion, were not sufficient to determine what the IRS required and whether the costs associated with the new requirements were appropriate.
The IRS needs to develop benchmarks (e.g., estimated discounts,
special terms and conditions, etc.) to ensure it achieves the best results for
the Federal Government when establishing the ELAs. A benchmark will provide the necessary standard for negotiation
objectives and can be used as a reference to judge success. Procurement function officials believe they
have saved millions of dollars in software costs. However, we could not find any analyses documenting this cost
savings. Additionally, our review of
the contract files and discussions with IRS personnel indicate that these
agreements could be improved to potentially realize more cost savings and
benefits for the IRS.
1. The Director, Office of Procurement, should take appropriate action to ensure the IRS properly analyzes and documents the business case and cost justification for all future ELAs.
Management’s
Response:
The Offices of Procurement and Enterprise Operations are working
cooperatively to ensure ELAs that are issued reflect sound business judgment
and are documented in accordance with the applicable Federal acquisition
regulations, policy, and guidance.
2.
The Director, Office of Procurement, should take appropriate
action to document the Federal Government cost estimate and required terms and
conditions to ensure the IRS is getting the best price and terms and conditions
for future ELAs.
Management’s
Response:
The Offices of Procurement and Enterprise Operations will work
cooperatively to ensure ELAs that are issued reflect sound business judgment
and are documented in accordance with Federal acquisition regulations, policy,
and guidance.
3.
The Director, Enterprise Operations, should review and, if
necessary, redefine requirements for all the ELA contracts the IRS has
established and work with the Director, Office of Procurement, to seek
appropriate price reductions.
Management’s
Response:
The Office of Enterprise Operations has worked with the Office of Procurement
to implement activities to adjust requirements for ELAs, seek appropriate price
reductions, and monitor the use of the ELAs.
4.
The Director,
Enterprise Operations, should monitor use of the ELAs on a regular basis and
work with the Office of Procurement to adjust agreements accordingly.
Management’s
Response:
The Office of Enterprise Operations has worked with the Office of
Procurement to implement activities to adjust requirements for ELAs, seek
appropriate price reductions, and monitor the use of the ELAs.
Office
of Audit Comment: While IRS management agreed with the
recommendations, they disagreed with the conclusion of our review that the IRS’
use of ELAs for software is not cost-effective. IRS management indicated in their response that they have
achieved greater value at significant cost savings through these ELAs than with
their former software contract arrangements.
However, no documentation to validate these cost savings was provided to
us during the time of our review.
IRS
management also disagreed with the audit findings that hold them to a higher
standard for file documentation than required by regulation or policy, the
methodology used to project measurable benefits, and the measurable benefits
projected. We believe it is important
for the IRS to properly document the business reasons why the IRS needs the
ELAs, how much software and/or maintenance is needed, and the cost estimates
associated with the requirements.
Without this documentation, the IRS has no justification for establishing
the ELA and expending Government funds.
Additionally, while we recognize the IRS may not be able to negotiate
the full cost savings presented in our outcome measures because the price per
MIPS may be lower depending upon volume, we believe it can be a starting point
to determine what is a reasonable amount to pay for the ELA.
Appendix I
Detailed Objective, Scope,
and Methodology
The overall objective was to
evaluate whether the use of Enterprise License Agreements (ELA) is cost
effective for the Internal Revenue Service (IRS). To accomplish our objective, we:
I. Determined whether the IRS’ ELA terms were effectively planned and negotiated so they are in the best interest of the Federal Government.
A. Interviewed procurement and operations personnel and determined the process used to establish the requirements and for planning and negotiating the ELAs.
1. Judgmentally selected a sample of the established ELAs and determined the cost effectiveness of the agreements. A judgmental sample was used because we did not plan on projecting the results to the universe. A sample size of four ELAs was selected from the nine ELAs the IRS had established during the time of our review.
2. Reviewed the documentation included in the contract/blanket purchase agreement (BPA) files to determine whether the ELAs were properly planned, negotiated, and prepared.
3. Analyzed the terms, conditions, and costs associated with the ELAs.
II. Determined whether the proper procurement procedures were followed in awarding the ELAs.
A. Reviewed the Information Technology Acquisition Procedures and determined whether the ELAs followed this process for award.
B. Reviewed the Federal Acquisition Regulation and procurement procedures for BPAs and competition requirements to determine whether the ELAs were properly competed and awarded.
Appendix II
Major Contributors to This Report
Daniel R. Devlin, Assistant Inspector General
for Audit (Headquarters Operations and Exempt Organizations Programs)
John R. Wright, Director
Debra L. Gregory, Audit Manager
Theresa
Haley, Senior Auditor
Kent
Johnson, Senior Auditor
Gary Pressley, Senior Auditor
Chinita Coates, Auditor
Appendix III
Commissioner N:C
Deputy Commissioner
for Operations Support N:OS
Director,
Procurement A:P
Director,
Enterprise Operations M:I:EO
Chief Counsel CC
National Taxpayer
Advocate TA
Director, Office of Legislative Affairs CL:LA
Director, Office of Program Evaluation and Risk
Analysis N:ADC:R:O
Office of Management Controls N:CFO:AR:M
Audit Liaisons:
Agency-wide
Shared Services A
Procurement A:P
Information Technology Services M:I:E
Appendix IV
This appendix presents detailed information on the measurable impact that our recommended corrective actions will have on tax administration. These benefits will be incorporated into our Semiannual Report to the Congress.
Type and Value of Outcome Measure:
· Inefficient Use of Resources – Potential; $17.6 million (see page 5).
· Cost Savings, Funds Put to Better Use – Potential; $8 million (see page 5).
Methodology Used to Measure the Reported Benefit:
Inefficient Use of Resources: The $17.6 million in potential inefficient use of resources resulted from the costs associated with underutilizing the Million Instructions Per Second (MIPS) established for information technology products in 2 of the Enterprise License Agreements (ELAs) included in our review. In calculating these costs for the first ELA, we multiplied the anticipated average of unused MIPS of 7,611 (86 percent of 8,850, the total MIPS included in the agreement) for Fiscal Years (FY) 2002 and 2003 by the price per MIPS of $954.58, resulting in potential inefficient costs for the 2 years of $14.5 million. For the second ELA, we multiplied the anticipated average of unused MIPS of 3,095 (77 percent of 4,020, the total MIPS included in the agreement for FY 2002) and 4,153 (77 percent of 5,394, the total MIPS included in the agreement for FY 2003) by the price per MIPS of $429.92, resulting in potential inefficient costs of $3.1 million. The potential inefficient use of resources for the 2 ELAs is $17.6 million.
Cost Savings, Funds Put to Better Use: The $8 million in potential cost savings resulted from the costs associated with underutilizing the MIPS for the 2 ELAs in FY 2004; payment for FY 2004 is not yet due. For the first ELA, we multiplied the anticipated average of unused MIPS for FY 2004 of 6,726 (76 percent of 8,850, the total MIPS included in the agreement) by the price per MIPS of $954.58, resulting in potential cost savings of $6.4 million. For the second ELA, we multiplied the anticipated average of unused MIPS for FY 2004 of 3,884 (62 percent of 6,264, the total MIPS included in the agreement) by the price per MIPS of $429.92, resulting in potential cost savings of $1.7 million. The total potential cost savings for these 2 ELAs in FY 2004 is $8 million.
Appendix V
Management‘s Response to the Draft Report
The response was removed due to its size. To see the complete response, please go to the Adobe PDF version
of the report on the TIGTA Public Web Page.