TREASURY INSPECTOR GENERAL FOR TAX ADMINISTRATION

 

 

Increased Oversight of Information Technology Hardware Maintenance Contracts Is Necessary to Ensure Against Paying for Unnecessary Services

 

 

 

September 24, 2013

 

Reference Number:  2013-22-094

 

 

This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.

 

 

Phone Number  /  202-622-6500

E-mail Address /  TIGTACommunications@tigta.treas.gov

Website           /  http://www.treasury.gov/tigta

 

HIGHLIGHTS

INCREASED OVERSIGHT OF INFORMATION TECHNOLOGY HARDWARE MAINTENANCE CONTRACTS IS NECESSARY TO ENSURE AGAINST PAYING FOR UNNECESSARY SERVICES

Highlights

Final Report issued on September 24, 2013

Highlights of Reference Number:  2013-22-094 to the Internal Revenue Service Chief Technology Officer.

IMPACT ON TAXPAYERS

Coordination among acquisition team members is a key to ensuring that the contractor is meeting the Government’s interest in terms of providing deliverables that are of high quality, complete, timely, and cost effective.  However, the IRS’s administration of selected information technology hardware maintenance contracts could be enhanced.  The IRS cannot ensure that taxpayer dollars are not being misspent to service information technology hardware assets that are no longer in use.

WHY TIGTA DID THE AUDIT

This audit is included in TIGTA’s Fiscal Year 2013 Annual Audit Plan and addresses the major management challenge of Achieving Program Efficiencies and Costs Savings.  The overall objective was to determine whether the IRS has adequate controls over its hardware maintenance contracts and is actively mitigating contract fraud risks.

WHAT TIGTA FOUND

TIGTA found several weaknesses in the oversight of selected information technology hardware maintenance contracts.  Specifically, TIGTA found instances where contracting personnel were not always effectively monitoring the contracts.  TIGTA also found an instance where the IRS did not receive contract deliverables in accordance with the contract’s requirements or submit written modifications when necessary to update an existing contract.  As a result of the lack of coordination and oversight, the IRS paid for services it did not receive or need. 

Further, TIGTA found incomplete or inaccurate asset data in three of the seven information technology hardware maintenance contracts reviewed.  A current TIGTA review provided the IRS with several recommendations for improving internal controls and overall reliability of the data.

WHAT TIGTA RECOMMENDED

TIGTA recommended that the Chief Technology Officer ensure that Contracting Officer’s Representatives provide Contracting Officers notification to timely process a contract modification when information technology assets are retired or removed from service or changes to performance requirements are made.  Additionally, Contracting Officer’s Representatives should make any necessary adjustments with respect to receipt and acceptance.  In addition, the Chief Technology Officer should ensure that Contracting Officer’s Representatives work closely with Technical Points of Contact to periodically reconcile assets associated with hardware maintenance contracts and provide necessary updates to User and Network Services Asset Management personnel.  

In its response to the report, the IRS agreed with TIGTA’s recommendations.  The IRS plans to communicate and emphasize expectations with Information Technology organizations so that managers can take appropriate action to ensure that hardware maintenance contracts are administered, acquisition duties are performed, and acquisition staff effectively coordinate in reconciling and providing updates about assets associated with hardware maintenance contracts, all in accordance with existing IRS policy.

 

September 24, 2013

 

 

MEMORANDUM FOR CHIEF TECHNOLOGY OFFICER

 

FROM:                       Michael E. McKenney /s/ Michael E. McKenney

Acting Deputy Inspector General for Audit

 

SUBJECT:                  Final Audit Report – Increased Oversight of Information Technology Hardware Maintenance Contracts Is Necessary to Ensure Against Paying for Unnecessary Services (Audit # 201220224)

 

This report presents the results of our review of Information Technology Hardware Maintenance Contracts.  The overall objective of this review was to determine whether the Internal Revenue Service has adequate controls over its hardware maintenance contracts and is actively mitigating contract fraud risk.  This review addresses the major management challenge of Achieving Program Efficiencies and Costs Savings and is included in TIGTA’s Fiscal Year 2013 Annual Audit Plan. 

Management’s complete response to the draft report is included as Appendix V.

Copies of this report are also being sent to the IRS managers affected by the report recommendations.  If you have any questions, please contact me or Alan R. Duncan, Assistant Inspector General for Audit (Security and Information Technology Services).

 

 

Table of Contents

 

Background

Results of Review

Some Deficiencies Were Identified in the Administration of Selected Information Technology Hardware Maintenance Contracts

Recommendations 1 and 2:

Asset Data for Selected Maintenance Contracts Were Inaccurate and Incomplete

Appendices

Appendix I – Detailed Objective, Scope, and Methodology

Appendix II – Major Contributors to This Report

Appendix III – Report Distribution List

Appendix IV – Glossary of Terms

Appendix V – Management’s Response to the Draft Report

 

 

Abbreviations

 

CO

Contracting Officer

COR

IRS

IT

TIGTA

TPOC

Contracting Officer’s Representative

Internal Revenue Service

Information Technology

Treasury Inspector General for Tax Administration

Technical Point of Contact

 

 

 

 

Background

 

The Internal Revenue Service (IRS) Information Technology (IT) organization delivers information technology services and solutions that drive effective tax administration to ensure public confidence.  Its goals include improving service, delivering modernization, increasing value, and assuring the security and resilience of IRS information systems and data.  The IRS IT organization consists of nine different functional areas, e.g., User and Network Services, Cybersecurity, and Applications Development and works closely with each operating division and functional area to deliver quality, world-class information technology support, services, and solutions. 

As part of its annual budget, the IRS includes funding for infrastructure costs, such as hardware maintenance.  The IRS spent about $39.8 million on hardware maintenance during Fiscal Year 2009.  The amount increased to $44.9 million in Fiscal Year 2010 and remained relatively steady at $44.1 million in Fiscal Year 2011.  In Fiscal Year 2012, the IRS spent $47.8 million on hardware maintenance.

There are generally two scenarios that occur within the IRS when purchasing maintenance for its information technology hardware assets.  First, when most hardware assets, e.g., laptops, desktops, and servers are purchased, those assets come with a manufacturer’s warranty to cover replacement and repairs.  In this scenario, the IRS will usually wait until the warranty coverage nears expiration before it purchases maintenance if needed for those assets.  Second, when the IRS purchases information technology hardware assets that support critical infrastructure projects, e.g., loggers to monitor Internet and e-mail traffic, it will often immediately purchase upgraded maintenance coverage to ensure that the asset can be promptly serviced or replaced without causing a significant disruption to ongoing operations.

Management of the IRS information technology environment is organized into four tier levels:  Tier 1 (supercomputers and mainframes), Tier 2 (minicomputers), Tier 3 (microcomputers), and Tier 4 (data and voice telecommunications).  Before a requisition for hardware services, i.e., maintenance, is forwarded to the Office of Procurement for processing, it will undergo various levels of management and technical review within the IT organization.  First, it undergoes review by one of the four tier levels depending on the type of good/service being acquired.  For example, if the IRS needs to purchase hardware maintenance for laptops or desktops, the requisition would need approval by Tier III (microcomputers).  After tier review, the requisition is routed to the User and Network Services organization’s Asset Management group to research and ensure that the assets associated with the requisition are tracked in the inventory system.  According to personnel from the User and Network Services organization, contract information, e.g., contract number, coverage dates, and applicable vendor information is added to the inventory system’s Vendor Contract Module to help with associating information technology hardware assets with their maintenance contract.  The process to create the linkage and validate the information is manual and requires that the Asset Management team be provided identifying data for the impacted information technology hardware assets.

When the IRS awards a contract, the acquisition team is responsible for the various aspects of the contract administration.  This team consists of a Contracting Officer (CO), Contracting Officer’s Representative (COR), and Technical Point of Contact (TPOC).  The responsibilities of the acquisition team are as follows:

·       Contracting Officer’s Representative – designated by the CO to perform certain administrative tasks related to a specific contract.  The primary role of the COR is to monitor the contractor’s performance, ensure that the contractor delivers what is called for in the contract, and serve as the technical liaison between the contractor and the CO. Most CORs are not co-located with the TPOCs at the various sites and do not work directly with the assets or contractors.  Therefore, the COR relies on the TPOC for the assurance of delivered goods or rendered services.

·       Technical Point of Contact responsible for providing technical assistance, input and direction to the CO and COR throughout the life cycle of the contract.  Regarding information technology hardware maintenance contracts, the TPOC facilitates the process of confirming contractor services were performed prior to notifying the COR to pay the invoice.  The TPOC also ensures that asset records are accurate.

The IRS IT organization’s Vendor and Contract Management office was created to help maximize the value of information technology investments by implementing effective sourcing strategies, monitoring vendor performance and contract management, and facilitating strong acquisition governance processes.  This office consists of a director and 13 staff divided among three different groups.  The Program Management group specifically has responsibility for conducting research and analyses on the information technology contract portfolio and evaluating contract management processes to identify opportunities for cost savings.  Due to limited staffing, the Vendor and Contract Management office reviews all requisitions where the contract will total $5 million or more.

This review was performed at the offices of the IRS IT organization’s Cybersecurity, Enterprise Operations, and Strategy and Planning organizations located in Lanham, Maryland, and the Agency-Wide Shared Service’s Office of Procurement located in Oxon Hill, Maryland, during the period of April 2012 through May 2013.  We conducted this performance audit in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objective.  We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objective.  Detailed information on our audit objective, scope, and methodology is presented in Appendix I.  Major contributors to the report are listed in Appendix II.

 

 

Results of Review

 

Some Deficiencies Were Identified in the Administration of Selected Information Technology Hardware Maintenance Contracts

Our review identified several weaknesses in the oversight of selected information technology hardware maintenance contracts.  We judgmentally selected seven maintenance contracts for review,[1] interviewed members from the acquisition team for each contract to determine their roles and responsibilities, and reviewed the contract file documentation.  During our review, we found instances where contracting personnel were not always effectively monitoring contracts and issuing a contract modification when necessary.  For example, in one of the contracts reviewed, we identified 10 assets that were retired (June 2012) prior to the end of the contract period (January 2013).  This contract provided maintenance for the 64 hardware components, e.g., switches, located in the Development, Integration and Test Environment.[2]  The contract, totaling $80,310, was awarded in January 2012 and expired January 2013.  In the contract documents reviewed, we did not find any evidence where a modification was submitted to remove these assets from the contract.  However, the IRS was still being billed the same monthly maintenance amount ($6,692) even though the 10 assets had been retired.  The TPOC for this contract advised that the usual process is to review and reconcile the assets associated with the contract prior to the contract’s annual renewal.

In another contract we reviewed, we identified an outdated information technology asset list that was being used to manage the contract and pay the vendor.  This particular contract provided maintenance for storage devices that required coverage since the original manufacturer’s warranty coverage elapsed.  When the contract was originally awarded in December 2009, it covered 54 storage devices with an average annual hardware maintenance cost of about $2.5 million.[3]  The current list, dated April 1, 2013, showed 32 storage devices requiring maintenance.  The decrease in the number of storage devices is due to the retirement of those hardware assets or the migration to a separate storage contract as part of IRS’s efforts to consolidate and share storage across the IRS.

The current TPOC for the storage contract we reviewed was assigned in November 2012.  Since then, the TPOC worked closely with the vendor to reconcile and correct the old asset inventory list and provided that to the COR.  Even after doing the reconciliation, another incorrect list, representing the storage devices that should receive coverage during the final period of this contract, was presented to the TPOC to assist with performing receipt and acceptance.  It is not clear from the documentation provided how this occurred.  It may have been caused by a lack of oversight and coordination among responsible parties.

Further, this same contract stipulated 34 different performance standards along with associated deliverables, i.e., the documents used to monitor the standards.  The performance standards included tasks such as providing 24/7 availability to communicate with the IRS on critical errors, keeping the system in working order, and delivering an inventory.  When we inquired about the performance standard that requires the contractor to provide a list of assets, the acquisition team stated it had not received this report since being assigned oversight responsibilities for this contract and that very few of the electronic documents associated with this contract were provided to them by the prior acquisition team.  The current TPOC also advised that these reports are no longer necessary since the IRS can input its maintenance requests directly into the vendor’s helpdesk system for tracking/monitoring purposes.  If the acquisition team deems that several of the performance standards no longer apply, then it should proceed with submitting a modification to remove these requirements and negotiate a new contract cost from the vendor.

The COR has responsibility for the day-to-day oversight of the contract and the responsibilities are documented in a letter of appointment.  The responsibilities include:

·       Assuring that changes in work or services are included in the contract through a written modification issued by the CO.

·       Monitoring the contractor’s performance of the contract’s technical requirements to assure that performance is strictly within the scope of the contract.

Further, Federal Acquisition Regulation sections 52.243-1 – 52.243-7 authorize the CO to make changes within the general scope of a contract when changes cause an increase or decrease in the cost or when property is obsolete or excessed.  These changes must be done through a written contract modification issued by the CO.  In addition, the Federal Acquisition Regulation requires a contract modification to be executed before a work scope change is implemented, if practicable.  These actions can only be taken if there is ongoing coordination between the TPOC and COR regarding changes to the information technology hardware assets, i.e., asset retirement.  This ongoing communication assists the COR with fulfilling his/her responsibility to notify the CO to modify the contract. 

As a result of this lack of proper coordination and oversight, the IRS paid for services it did not need or did not receive.  The IRS also did not receive contract deliverables in accordance with the contract’s requirements or submit written modifications when necessary to update an existing contract.  These scenarios could potentially cause the IRS to unnecessarily pay for maintenance on assets that have been retired and no longer need this service.  When contracts are not properly administered, the IRS may not receive the desired outcome or the best return on its investment.  The Treasury Inspector General for Tax Administration (TIGTA) previously reported similar deficiencies in its review of another contract.[4]

We did not identify any potential fraudulent activity among the contracts reviewed.  We conducted interviews of the contracting personnel to ensure that procedures were in place to mitigate fraud risk.  Many individuals interviewed confirmed they would contact their managers, the CO, or the TIGTA’s Office of Investigation if they suspected any fraudulent activity.  Further, an internal website used by contracting personnel provides information on where to report suspected fraudulent activity.  We also performed tests to determine if any of the selected vendors were involved in any legal proceedings.

Management Action:  IRS management agreed that there was a gap in the understanding and management of the storage contract selected in our review due to COR attrition.  The IRS has taken steps to mitigate these issues over the last two months.  The CO has been made aware of the TPOC’s work to reconcile the hardware list with the vendor and will be contacting the vendor for the required documentation supporting a reduction in costs.  The COR is also reviewing the current performance standards to identify non-applicable performance standards to be removed.

Recommendations

The Chief Technology Officer should:

Recommendation 1:  Ensure that the CORs provide the CO notification to timely process a contract modification, if appropriate, when information technology assets are retired or removed from service, or changes to performance requirements are made.  Additionally, the CORs should make any necessary adjustments with respect to receipt and acceptance. 

Management’s Response:  The IRS agreed with our recommendation.  The IRS will communicate and emphasize expectations with IT organizations so that managers can take appropriate action to ensure that hardware maintenance contracts are administered and acquisition duties are performed in accordance with existing IRS policy.

Recommendation 2:  Ensure that the CORs work closely with the TPOCs to periodically reconcile assets associated with hardware maintenance contracts to the vendor’s independent records and provide necessary updates about the assets to User and Network Services Asset Management personnel. 

Management’s Response:  The IRS agreed with our recommendation.  The IRS will communicate and emphasize expectations with IT organizations so that managers can take appropriate action to ensure that acquisition staff effectively coordinate in reconciling and providing updates about assets associated with hardware maintenance contracts in accordance with existing IRS policy.

Asset Data for Selected Maintenance Contracts Were Inaccurate and Incomplete

A recent TIGTA review[5] of the IRS IT organization’s asset inventory system identified several deficiencies in the internal controls designed to ensure accurate and complete inventory records.  We also identified incomplete or inaccurate asset data in three of the seven information technology hardware maintenance contracts reviewed. 

These data discrepancies occurred because the internal controls for proper asset management need to be strengthened and due to a lack of available resources to monitor and oversee the inventory.  The discrepancies identified in this review further underscore the need to periodically reconcile the assets associated with hardware maintenance and ensure that the IRS accounts for all of its assets and only pays for maintenance coverage on those assets that are still in service.  Further, it is equally important that the various users of the information technology asset inventory management system data have confidence in and can rely on the data maintained within the system.  TIGTA’s current review[6] provided the IRS with several recommendations for improving internal controls and overall reliability of the data. 

 

Appendix I

 

Detailed Objective, Scope, and Methodology

 

The overall objective of this review was to determine whether the IRS has adequate controls over its hardware maintenance contracts and is actively mitigating contract fraud risk.  To achieve our objective, we:

I.                 Determined whether IRS personnel have adequate controls in place to prevent payment for maintenance services for assets covered by warranty or that have been disposed. 

A.    Identified the universe of 208 hardware maintenance contracts awarded during Fiscal Years 2009 through 2012.  We judgmentally[7] selected seven contracts to review based on the following monetary criteria:  two contracts, each worth more than $5 million; two contracts worth between $700,000 and $4 million; and three contracts each worth less than $700,000.  One of the contracts we reviewed was a closed contract.  We used judgmental sampling because we did not intend to project our results to the contract universe.  Further, the scope of our review was limited due to an ongoing TIGTA investigation.

B.    Reviewed all contract documentation such as the Statement of Work[8] (also called a Performance Work Statement), invoices, modification, and asset listing of each sampled contract to identify the contractor’s requirements for providing maintenance. 

C.    Compared a copy of the asset listing associated with the maintenance contract with the data recorded in the asset inventory system to obtain the current status of the assets.  In order to assess the reliability of the inventory data, data were reviewed for reasonableness.  We reviewed the assets associated with our selected contracts to verify their accuracy in the inventory system and identified several with incorrect statuses.  However, we found the data to be reliable for the limited purposes of this audit and performed no other validity tests.     

D.    For the sampled contracts, we interviewed the CO, COR, and TPOC to determine their roles and responsibilities in providing oversight for the selected contracts and their awareness of fraud.

II.               Assessed actions taken by the IT organization to enhance vendor, contract, and asset management activities.

III.             Interviewed individuals involved with overseeing maintenance contracts to determine their awareness in detecting and reporting potential fraud.

Internal controls methodology

Internal controls relate to management’s plans, methods, and procedures used to meet their mission, goals, and objectives.  Internal controls include the processes and procedures for planning, organizing, directing, and controlling program operations.  They include the systems for measuring, reporting, and monitoring program performance.  We determined the following internal controls were relevant to our audit objective:  the Federal Acquisition Regulations, IRS procurement policies and procedures, and the User and Network Services organization’s policies and procedures relating to information technology asset management.  We evaluated these controls by interviewing acquisition team members and User and Network Services organization personnel and reviewing relevant contract documentation. 

 Appendix II

 

Major Contributors to This Report

 

Alan R. Duncan, Assistant Inspector General for Audit (Security and Information Technology Services)

Danny Verneuille, Director

Diana Tengesdal, Audit Manager

Anthony Morrison, Program Analyst

Sarah Shelton, Program Analyst

Linda Nethery, Information Technology Specialist

 

Appendix III

 

Report Distribution List

 

Acting Commissioner 

Office of the Commissioner – Attn:  Chief of Staff  C

Deputy Commissioner for Operations Support  OS  

Deputy Chief Information Officer for Operations  OS:CTO

Chief, Agency-Wide Shared Services  OS:A

Associate Chief Information Officer, User and Network Services  OS:CTO:UNS

Director, Procurement  OS:A:P

Director, Vendor Contract Management  OS:CTO:SP:VCM

Chief Counsel  CC

National Taxpayer Advocate  TA

Director, Office of Legislative Affairs  CL:LA

Director, Office of Program Evaluation and Risk Analysis  RAS:O

Office of Internal Control  OS:CFO:CPIC:IC

Audit Liaison:  Director, Risk Management Division  OS:CTO:SP:RM

 

Appendix IV

 

Glossary of Terms

 

Term

Definition

24/7

The contractor maintains a helpdesk function staffed 24 hours a day, seven days a week to communicate with the IRS on critical errors. 

Delivery Order

An order for supplies placed against an established contract or with Government sources.

Development, Integration, and Test Environment

Provides a standardized modernized development, integration, and test environment and the associated tools and processes needed to ensure the successful production deployment of IRS modernization projects.

Invoice

A contractor’s bill or written request for payment under the contract for supplies delivered or services performed.

Modification

Any formal change to the terms and conditions of a contract, delivery order, or task order, either within or outside the scope of the original agreement.

Option

A unilateral right in a contract by which, for a specified time, the Government may elect to purchase additional supplies or services called for by the contract, or may elect to extend the term of the contract.

Receipt and Acceptance

The point at which the Government accepts ownership of specifically identified supplies or approves the performance of specific services.

Statement of Work

Documents the work to be performed by the contractor, the period of performance, performance standards, and special requirements. 

Switch

A small hardware device that joins multiple computers together with one local area network. 

Task Order

An order for services placed against an established contract or with Government sources.

 

Appendix V

 

Management’s Response to the Draft Report

 

DEPARTMENT OF THE TREASURY

INTERNAL REVENUE SERVICE

WASHINGTON, D.C. 20224

 

 

CHIEF TECHNOLOGY OFFICER

 

 

September 12, 2013

 

 

MEMORANDUM FOR DEPUTY INSPECTOR GENERAL FO.R AUDIT

 

FROM:                             Terence V. Milholland /s/ Terence V. Milholland

                                          Chief Technology Officer

 

SUBJECT:                       Draft Audit Report - Increased Oversight of Information Technology Hardware Maintenance Contracts is Necessary to Ensure Against Paying for Unnecessary Services (Audit# 201220224) (e-trak #2013-46225)

 

Thank you for the opportunity to review and respond to the subject audit report.

 

The IRS recognizes the significance of ensuring complete accountability for hardware maintenance coverage and costs through proper oversight of contracts and asset inventory records.

 

While the IRS agrees with the recommendations in the report, we take exception to the conclusion that the IRS paid for services it did not need or did not receive.  As we discussed during the audit, the contracts identified in the report were firm fixed price contracts that did not allow for midterm changes to product inventory from maintenance coverage.  The decision to use firm fixed price contracts is a risk based decision intended to lower overall contracting costs.  Of course the risk with using these type contracts is that in some situations the expected benefits are not realized.

 

The Service will seek continual improvements in these areas to reduce risks.  The attachment to this memo describes our planned actions to implement the audit recommendations.

 

We value your continued support and the assistance your organization provides.  If you have any questions, please contact me at (202) 622-6800 or a member of your staff may contact Lisa Starr, Senior Manager, Program Oversight at (202) 283-3607.

 

Attachment

 

Attachment

 

RECOMMENDATION #1Ensure that the CORs provide the CO notification to timely process a contract modification, if appropriate, when information technology assets are retired or removed from service, or changes to performance requirements are made.  Additionally the CORs should make any necessary adjustments with respect to receipt and acceptance.

 

CORRECTIVE ACTION #1:  The IRS agrees with this recommendation.  The CTO's Vendor and Contract Management division will communicate and emphasize expectations with IT organizations, so that managers can take appropriate action to ensure hardware maintenance contracts are administered and acquisition duties are performed in accordance with existing IRS policy.

 

IMPLEMENTATION DATE:  January 25, 2014

 

RESPONSIBLE OFFICIALAssociate Chief Information Officer, Strategy and Planning

 

CORRECTIVE ACTION MONITORING PLANWe enter accepted Corrective Actions into the Joint Audit Management Enterprise System (JAMES) and monitor them on a monthly basis until completion.

 

RECOMMENDATION #2:  Ensure that the CORs work closely with the TPOCs to periodically reconcile assets associated with hardware maintenance contracts to the vendor's independent records and provide necessary updates about the assets to User and Network Services Asset Management personnel.

 

CORRECTIVE ACTION #2:  The IRS agrees with this recommendation.  The CTO's Vendor and Contract Management division will communicate and emphasize expectations with IT organizations, so that managers can take appropriate action to ensure acquisition staff effectively coordinate in reconciling and providing updates about assets associated with hardware maintenance contracts in accordance with existing IRS policy.

 

IMPLEMENTATION DATEJanuary 25, 2014

 

RESPONSIBLE OFFICIALAssociate Chief Information Officer, Strategy and Planning

 

CORRECTIVE ACTION MONITORING PLANWe enter accepted Corrective Actions into the Joint Audit Management Enterprise System (JAMES) and monitor them on a monthly basis until completion.



[1] A judgmental sample is a nonstatistical sample, the results of which cannot be used to project to the population. 

[2] See Appendix IV for a glossary of terms.

[3] The contract included six-month option/renewal periods to provide flexibility to reduce the contract. 

[4] TIGTA, Ref. No. 2012-10-075, An Independent Risk Assessment of Facility Physical Security Was Not Performed in Compliance With Contract Requirements (Jul. 2012).

[5] TIGTA, Ref. No. 2013-20-089, Weaknesses in Asset Management Controls Leave Information Technology Assets Vulnerable to Loss (Sept. 2013).

[6] TIGTA, Ref. No. 2013-20-089, Weaknesses in Asset Management Controls Leave Information Technology Assets Vulnerable to Loss, pp. 7, 11, and 15 (Sept. 2013).

[7] A judgmental sample is a nonstatistical sample, the results of which cannot be used to project to the population.

[8] See Appendix IV for a glossary of terms.