TREASURY INSPECTOR GENERAL FOR TAX ADMINISTRATION

RECOVERY ACT

Recovery Act Provisions for the Health Coverage Tax Credit Were Implemented, but Development Processes Could Be Improved

May 25, 2010

Reference Number: 2010-21-057

This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.

Redaction Legend:

1 = Tax Return/Return Information

Phone Number | 202-622-6500

Email Address | inquiries@tigta.treas.gov

Web Site | http://www.tigta.gov

HIGHLIGHTS

RECOVERY ACT PROVISIONS FOR THE HEALTH COVERAGE TAX CREDIT WERE IMPLEMENTED, BUT DEVELOPMENT PROCESSES COULD BE IMPROVED

Highlights

Final Report issued on May 25, 2010

Highlights of Report Number: 2010-21-057 to the Internal Revenue Service Commissioner for the Wage and Investment Division.

IMPACT ON TAXPAYERS

The improvements to the Health Coverage Tax Credit included increasing the Government’s portion of health premiums paid from 65 to 80 percent, reimbursing the taxpayers’ portion of premiums paid during the enrollment period, allowing family members to continue receiving benefits after certain life events, and expanding taxpayer eligibility in the Program. Effective implementation of these provisions was critical to provide taxpayers the benefits mandated by the American Recovery and Reinvestment Act of 2009 (Recovery Act).

WHY TIGTA DID THE AUDIT

This audit was initiated because the Recovery Act included provisions mandating improvements to and expansion of the Health Coverage Tax Credit Program. The Health Coverage Tax Credit Program provides income tax credits to subsidize health insurance coverage for individuals displaced from their jobs under specific circumstances, which ultimately resulted in loss of health insurance coverage benefits. Our review assessed the extent to which the Recovery Act provisions were implemented, Health Coverage Tax Credit system development was efficient and effective, and benefits were provided to taxpayers.

WHAT TIGTA FOUND

The Internal Revenue Service (IRS) properly used Recovery Act funds to implement mandated provisions to the Health Coverage Tax Credit Program. The provisions were completed as mandated by legislation, the system was ready to process health care claims, and benefits were provided to taxpayers. Specifically, in April 2009, the portions of health premiums paid by the Government were increased from 65 to 80 percent. In August 2009, retroactive reimbursements for premiums paid during enrollment were implemented. The remaining features will be rolled out during Fiscal Years 2010 and 2011.

Although the Health Coverage Tax Credit Program Office appropriately executed provisions, some project management practices needed improvement. Specifically, the Program’s system development processes for requirements and configuration management need strengthening, testing practices need improvement, and performance measures targets should be updated using current Program estimates. Actions taken to strengthen the system development processes will improve the effectiveness and efficiency of future upgrades to the Health Coverage Tax Credit Program.

WHAT TIGTA RECOMMENDED

TIGTA recommended that the Commissioner, Wage and Investment Division, ensure 1) full traceability on the requirements traceability matrix, 2) all configuration changes are properly documented and tracked, 3) in-depth analysis of test plans, and 4) performance measures targets are suitable for assessing the effectiveness of the Program.

The IRS agreed with our recommendations and stated corrective actions have been taken or started. However, the IRS stated that the inability to estimate targets did not diminish its ability to measure outcomes or demonstrate Health Coverage Tax Credit Program Office effectiveness. Because the final Fiscal Year 2009 results included one performance measure (Customer Service Representative Level of Service) that did not meet the performance measures targets for the Health Coverage Tax Credit Program, TIGTA questions whether the IRS had sufficient and timely information to develop accurate performance measures targets with which to make risk-based decisions in the Health Coverage Tax Credit Program.

May 25, 2010

MEMORANDUM FOR COMMISSIONER, WAGE AND INVESTMENT DIVISION

FROM: Michael R. Phillips /s/ Michael R. Phillips

Deputy Inspector General for Audit

SUBJECT: Final Audit Report – Recovery Act Provisions for the Health Coverage Tax Credit Were Implemented, but Development Processes Could Be Improved (Audit # 200920130)

This report presents the results of our review of the Internal Revenue Service’s (IRS) implementation of changes to the Health Coverage Tax Credit (HCTC) Program. The American Recovery and Reinvestment Act of 2009 (Recovery Act)[1] included provisions for upgrading the HCTC Program. The overall objective of this review was to determine whether the IRS established and followed adequate system development processes to ensure efficient and effective upgrades of the HCTC Program. This audit was part of our Fiscal Year 2009 annual audit plan and addresses the major management challenge of Modernization of the IRS.

The Recovery Act provides separate funding to the Treasury Inspector General for Tax Administration through September 30, 2013, to be used in oversight activities of IRS programs. This audit was conducted using Recovery Act funds.

Management’s complete response to the draft report is included as Appendix V.

Copies of this report are also being sent to the IRS managers affected by the report recommendations. Please contact me at (202) 622-6510 if you have questions or Alan R. Duncan, Assistant Inspector General for Audit (Security and Information Technology Services), at (202) 622-5894.

Table of Contents

Background

Results of Review

The Health Coverage Tax Credit Program Office Effectively Implemented Mandated Recovery Act Provisions and Managed Funds

Requirements Management Processes Need Improvement

Recommendation 1:

The Configuration Management Process Needs Strengthening

Recommendation 2:

Testing Practices Need Improvement

Recommendation 3:

Performance Measures Targets Should Be Updated Using Current Estimates

Recommendation 4:

Appendices

Appendix I – Detailed Objective, Scope, and Methodology

Appendix II – Major Contributors to This Report

Appendix III – Report Distribution List

Appendix IV – Glossary of Terms

Appendix V – Management’s Response to the Draft Report

Abbreviations

FY	Fiscal Year
HCTC	Health Coverage Tax Credit
IRS	Internal Revenue Service

Background

The Health Coverage Tax Credit (HCTC) Program is intended to address the long-term needs in delivering health insurance benefits to a select group of individuals who might otherwise be uninsured. For eligible taxpayers, the HCTC Program makes heath care insurance more affordable by currently paying 80 percent of the health insurance premiums. The HCTC was created by the Trade Adjustment Assistance Reform Act of 2002,[2] and became effective in August 2003. The HCTC is a unique tax credit, whereby participants can either receive payments on a monthly basis to help pay for health insurance, or on a yearly basis when filing their income tax return, where the credit will be applied as a refund or toward any taxes due.

The American Recovery and Reinvestment Act of 2009 (Recovery Act)^{^[3]} mandated new provisions to the HCTC. The Recovery Act changed the HCTC by:

The HCTC is designed to provide a Federal income tax credit subsidizing health coverage for individuals who might otherwise be uninsured.

· Increasing the portion of health insurance premiums paid by the Federal Government from 65 to 80 percent.

· Reimbursing the participants’ portion of premiums paid during the enrollment period.

· Allowing family members to continue receiving the HCTC after certain life events.

· Expanding taxpayer eligibility in the Program.

The Internal Revenue Service (IRS) created an HCTC Program Office to oversee the design, development, and overall operations of the HCTC Program. This includes tracking and paying the Government’s portion of health insurance premiums due for HCTC Program participants. The HCTC Program Office is also responsible for all phases of the HCTC software development life cycle, beginning with a detailed analysis of business and technical requirements and ending with system deployment.

Because the population of taxpayers enrolling in the HCTC Program is continually increasing, outdated system hardware presented a risk. The HCTC Program Office deemed it necessary to upgrade the HCTC system and worked with a contractor to make the needed improvements. The system upgrades to implement the Recovery Act mandates were implemented in three phases. HCTC Release[4] 9 focused on delivering the increased HCTC and the retroactive health care premium reimbursements, and Release 10 focused on the remaining Recovery Act provisions and hardware and software upgrades.

The Recovery Act activities require a high level of scrutiny and taxpayer dollars spent on economic recovery must be subject to unprecedented levels of transparency and accountability. Federal agencies are required to ensure Recovery Act funds are used for authorized purposes and appropriate measures are taken to prevent fraud, waste, and abuse. As such, the Treasury Inspector General for Tax Administration is required to monitor the IRS implementation of Recovery Act provisions, and this audit was conducted to meet those requirements.

This review was performed in the IRS HCTC Program Office in Washington, D.C., and the contractor site in Reston, Virginia, during the period May through December 2009. We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objective. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objective. Detailed information on our audit objective, scope, and methodology is presented in Appendix I. Major contributors to the report are listed in Appendix II.

Results of Review

The Health Coverage Tax Credit Program Office Effectively Implemented Mandated Recovery Act Provisions and Managed Funds

The IRS effectively used Recovery Act funds to implement the mandated provisions to the HCTC Program. Specifically, the Recovery Act provisions were completed as mandated by legislation, the HCTC system was ready to process health care claims, and benefits were provided to taxpayers. In March 2009, the HCTC Program Office, working in conjunction with the IRS business owners, evaluated the Recovery Act legislative provisions and initiated a plan for implementation. The HCTC Program Office monitored and met the development milestone dates for both HCTC Release 9 and 10.

Recovery Act provisions were successfully implemented

The IRS successfully developed three HCTC system releases to implement the Recovery Act provisions. Beginning in April 2009, the HCTC Program Office increased the credit percentage paid from 65 to 80 percent of qualified health insurance premiums. In August 2009, Release 9 delivered the increased tax credit and allowed for retroactive reimbursements for health care premiums paid while enrolling in the monthly HCTC Program. Release 10 was implemented in December 2009 and increased the eligibility for qualifying family members who would otherwise become ineligible and lose health insurance coverage due to a change in the recipient’s Medicare status or as a result of family death or divorce. Release 10 also included hardware and software upgrades to the HCTC system.

The HCTC Program Office completed a number of HCTC project management business processes to ensure provisions required by the Recovery Act were implemented and provided the intended benefits to taxpayers. Review of available documentation demonstrated the HCTC project team successfully implemented the HCTC project management plan. This plan served as the overall guiding document used by the HCTC project team to ensure the system materialized as mandated by the Recovery Act. The project plan outlined the project’s objectives, parties involved in project planning and execution, and the overall time period and delivery strategy for the project. These business practices aided in ensuring the HCTC system was available to process health care claims. These practices included:

· Justifying the need for the HCTC enhancements and reporting needed funding.

· Managing requirements.

· Managing risks.

· Managing project schedule.

· Performing/conducting peer reviews.

· Reporting activities.

· Testing strategy.

· Developing a project management plan.

Benefits from changes to the HCTC Program are being realized

The IRS effectively promoted and provided HCTC benefits to taxpayers. The IRS redesigned the HCTC Program Kit and Registration Form to simplify customer registration and increase benefits to taxpayers, and enrollment in the HCTC Program increased from 14,765 in April 2009 to 20,722 by January 2010. Additionally, the new HCTC Program Kit reduced printing costs by one-half and created a cost savings of approximately $100,000, with an additional cost savings of more than 40 percent for future print orders.

In order to determine whether the HCTC applies, there are education and outreach activities designed to assist taxpayers (i.e., public service announcements were developed and underway in certain States). The HCTC Program Office reports positive feedback from use of these activities. ******1*******.

The HCTC Program Office is responsible for informing potential eligible taxpayers about the credit, enrolling eligible applicants into the Program, and making payments to health plans on their behalf. The intent of the Recovery Act legislation related to the HCTC is to provide those American taxpayers who are eligible with the opportunity to take advantage of this Program in order to ease their economic stress.

Recovery Act funding for the HCTC is being managed

The HCTC Program Office is actively monitoring expenditures to ensure they do not exceed the funds obtained and executing measures to implement Program requirements. Federal agencies are required to ensure Recovery Act funds are used for authorized purposes and appropriate measures are taken to prevent fraud, waste, error, and abuse. The Recovery Act reporting activities are designed to track Recovery Act tax provision benefits.

As of September 30, 2009, the HCTC Program Office obligated and/or expended $15.2 million, or 67 percent of its $22.8 million base year budget.[5] A spending plan was completed to project and track costs for each year of the investment contract. By February 2011, the HCTC Program Office forecasts the use of all Recovery Act funding on future releases and additional functionalities. Calculating the appropriate funding needed for the potential health coverage effort was a challenge for HCTC Program executives. The funding needs were based on potential enrollment figures, which were difficult to estimate. Although specific funding for the HCTC Program was designated in the IRS 2009 budget, HCTC Program executives were able to obtain additional Recovery Act funding of $80 million for Fiscal Years (FY) 2009 – 2010 HCTC business and technology investments.

While the mandated changes were successfully implemented and Recovery Act funds were monitored, the development processes followed by the IRS to make these changes could be improved.

Requirements Management Processes Need Improvement

Requirements Management is used for analyzing and verifying whether an organization’s developed solution meets the needs of customers and internal/external stakeholders. In a combined effort, project team members and stakeholders identify organizational objectives and constraints, as well as develop and facilitate program changes and requirements. It also includes requirements traceability which documents the life cycle of a requirement.

The requirements traceability matrix for Release 9 was not all inclusive and needed further development. Specifically, the matrix:

· Did not trace to specific work products.

· Did not trace all requirements to a test case.

· Contained one approved requirement erroneously tested by using a description different from that as prescribed on the matrix.

Without complete traceability, the IRS cannot ensure adequate development of requirements and tracing, potentially resulting in requirements not being satisfied. Missing or invalid requirements could jeopardize the IRS’ ability to deliver health insurance benefits to taxpayers and/or noncompliance with mandated HCTC legislature.

The Software Engineering Institute’s Capability Maturity Model Integration states projects should maintain traceability between all source and product requirements. The Enterprise Life Cycle directs project teams to verify contractors maintain traceability over project life cycle application requirements, thus ensuring each one is mapped to its higher level requirement.

A complete tracing of requirements was not accomplished due to the aggressive time period required to implement the HCTC provision. The discrepancies involving insufficient matrix traceability to work products and test cases were discussed with the HCTC Program Office on September 11 and 25, 2009, but a subsequent review of the final requirements traceability matrix determined the issues were still not corrected.

Signatures annotating matrix approval are needed from all stakeholders

There was no documented evidence that the baseline requirements traceability matrix was approved. Specifically, the April 24, 2009, matrix did not have signatures from either the Wage and Investment Division business sponsor or internal stakeholders endorsing their review of the baseline requirements. To ensure the system delivers the intended benefits and provides credible information, it is imperative internal stakeholders approve the baseline requirements.

According to the Enterprise Life Cycle, requirements management is the process where baseline requirements are defined, formalized, managed, controlled, and verified. The HCTC Program Office indicated there was not a formal process requiring business sponsors or internal stakeholders to provide signatures on the matrix. However, effective requirements management builds upon the appropriately documented review and approval of baseline requirements.

Management Action: During a September 11, 2009, discussion, the HCTC project executive stated her single signature on the matrix represented an overall acceptance by the HCTC Program Office business operations team lead personnel. Subsequently, the HCTC Program Office agreed the matrix should contain documented signatures from the business sponsor and internal stakeholders, concluding this change would take effect for the Release 10 matrix. The IRS subsequently reported that the signatures of the Director, Electronic Tax Administration and Refundable Credits; Systems Operations Lead; Business Operations Lead; Payment Operations Lead; and Program Management Office Lead were included on the Release 10 baseline requirements by December 4, 2009.

Recommendation

Recommendation 1: The Commissioner, Wage and Investment Division, should ensure that for ongoing and future system upgrades, the HCTC Program Office completes full traceability on the requirements traceability matrix, including mapping of all requirements to high-level requirements, and tracing to the final product requirements. These include changes, source codes, and any manual referencing accomplished.

Management’s Response: The IRS agreed with the recommendation. The IRS stated the recommendation outlines its standard processes and has already taken action. The HCTC Program Office completed full traceability of the Release 9 requirements traceability matrix on August 19, 2009, following system implementation per its standard process. The requirements traceability report contains some requirements that do not call for mapping to test cases or build components.

Office of Audit Comment: The IRS noted that corrective action was accomplished during the review. However, documentation supporting the corrective action was not provided to us during the review, so we were unable to validate the actions taken. As a result, corrective action will be validated during the subsequent review of the HCTC Program Office (Audit Number 201020101) currently in process.

The Configuration Management Process Needs Strengthening

The Change Control Log (a process used by the HCTC Program Office for documenting all change requests) did not capture complete data for documenting and tracking change requests. For example, the change control log did not include the originator’s name or impact of the change request. Specifically, after creation of baseline requirements, individual change request forms were not retained on requirements modified. Rather, the Change Control Log, which provided a brief description and/or comment, was used for documenting change requests. Additionally, the information supporting approved change requests was not captured on the contractor’s database. The September 25, 2009, control log contained 50 approved change requests; however, according to the HCTC Program Office, information was not always captured in the contractor’s database.

Change request forms contain project data and information necessary to forecast program costs and schedules, resulting in a greater assurance for successful delivery of the program’s intended benefits. The Configuration Management Plan outlines the following processes for effecting change requests:

Identify potential change/error.
Document change/error.
Analyze change/error.
Obtain authorization.
Implement and migrate change.
Update Requirements Traceability Matrix.

Although the Configuration Management Plan outlines change control procedures, the HCTC configuration management process does not contain specific guidelines to capture change request information, such as requestor’s name, cost, and schedule impacts. The HCTC Program Office maintained a Change Control Log which does not give specific or pertinent change request information. Additionally, they relied on the contractor’s Application Management and Operating System to track and support information on change requests. The Application Management Operating System database is used to capture items that are changed in the production system, in accordance with the Configuration Management Plan.

Management Action: During a November 30, 2009, discussion, HCTC Program management agreed the Change Control Log should be modified to capture more data, specifically to incorporate the impact on cost and schedule changes. Subsequently, the January 27, 2010, updated Change Control Log template reflected a modification to include the impact of cost and schedule changes.

Recommendation

Recommendation 2: The Commissioner, Wage and Investment Division, should ensure that for ongoing and future releases, the HCTC Program Office updates the Configuration Management Plan to include procedures to verify all configuration changes are properly documented and tracked.

Management’s Response: The IRS agreed with the recommendation, stating it has already implemented additional corrective actions. As part of the standard process, all change requests are documented in a change control log during the development phase. All HCTC Release 9 and Release 10 change requests were reviewed and approved by the HCTC Program Office and had no effect on cost and schedule. Therefore, the IRS did not include those metrics. Cost and schedule were included in the change control log effective November 11, 2009. Further, the Configuration Management Plan was updated January 29, 2010, to include a clear differentiation of how change requests are managed during development, maintenance, and operations.

Testing Practices Need Improvement

Various testing and requirements management processes are needed to ensure functional and technical specifications are met. These processes are key management controls IRS executives use as credible information upon which to base managerial decisions.

Additional actions are needed to ensure a structured approach to testing

Results for Release 9 User Acceptance Testing disclosed that test practices needed to be developed and test documents contained missing or incomplete information. The HCTC Program Office stated the timing of requests and versioning of documentation created errors. Although HCTC Program personnel created a central repository spreadsheet to collect test script data for all test cases, the information was incomplete. On a query of the universe of 37,602 test scripts, 1,410 were missing expected test results which are used by testers to conclude whether the test cases passed, failed, or were not tested. Additionally, there were two incomplete test cycle control sheets. Further, in the final test results documentation, it was noted that many test scripts were either not clear or outdated.

Documentation for all tests must be complete and contain sufficient information to ensure the test results can be relied on to ensure the system is delivering its intended benefits and that there are not any usability issues. To ensure effective and efficient Program operation, the HCTC Program Office relies on contractor support in key areas, one of which is testing. The Software Engineering Institute Software Capability Maturity Model Integration outlines a series of processes and tasks for project software testing. These processes include planning, resource allocation, test development, and test effort and result monitoring. Tasks include defining test cases, expected results, procedural steps, and tracing test cases to their requirements.

Management Actions: During User Acceptance Testing in August 2009, the HCTC Program Office and contractor jointly tested and validated functional and business requirements. On September 22, 2009, we informed the HCTC Program Office of the insufficient strategy practices used during User Acceptance Testing. The HCTC Program Office agreed that for Release 10, individuals selected to perform the testing would have a greater knowledge of the HCTC system. Participants selected to perform Release 10 testing during the week of November 30, 2009, were composed of HCTC business users (the IRS and contractor) representing each functional area of the scripts being tested. Also noteworthy, a review of tests developed for Release 10 was comprehensive in terms of scope, functionality, technical coverage, and business participation. Test results were sufficiently documented.

Waived and deferred test cases should be approved and documented

Numerous waived and deferred test cases did not have a corresponding Test Case Waiver and Deferral form, which is used to state the reason a test was not performed and document the waiver/deferral approval via a relevant stakeholder’s signature. Without proper support and approval for changes to test cases, business and functional requirements may not be adequately tested to ensure the system provides the needed functionality.

The IRS’ Test Case Waiver and Deferral procedures define a waiver request as eliminating the obligation, due to various reasons, for executing a test case and thus altering the test plan. Test case requirements must either be approved for deletion and/or verified as being included for testing in another test case. The IRS’ procedures outline processes for tailoring a program specific Test Case Waiver and Deferral form to include the necessary format for use in removing a particular test case or phase. Additionally, procedures require that all changes to test cases have a corresponding rationale and approval.

During a September 11, 2009, audit discussion, the HCTC Program Office stated it was unaware of IRS procedures requiring waived and deferred test cases to have a corresponding Test Case Waiver and Deferral form. Additionally, personnel stated the waived test cases were a result of the test scripts having already been tested and listed in templates for prior HCTC releases. As such, performing an in-depth analysis of test plans prior to beginning each Program release would eliminate erroneously listing previously tested requirements. In the event a test case is erroneously included, a corresponding Test Case Waiver and Deferral needs to document the status.

Management Action: A copy of the Modernization and Information Technology Services organization Test Case Waiver and Deferral Procedure, dated November 2006, was provided to HCTC Program Office personnel during an October 26, 2009, audit discussion. On January 27, 2010, the HCTC Program Office provided two HCTC specific Test Case Waiver and Deferral forms that were used to approve deferred test cases for Release 10. Management action taken should resolve this issue.

Recommendation

Recommendation 3: The Commissioner, Wage and Investment Division, should ensure that the HCTC Program Office performs an in-depth analysis of test plans prior to beginning each Program release for elimination of previously tested requirements and, in the event a test case is erroneously included, completes the Test Case Waiver and Deferral form for approving its removal.

Management’s Response: The IRS agreed with the recommendation to implement a waiver and deferral process and has implemented the corrective actions within HCTC Release 10. The IRS stated it completed the implementation of the waiver and deferral process, peer review of test cases, and removal of test cases from test plans that were no longer applicable. The process documented with Release 10 outlines the procedures for authorizing the deferral or waiver of a test case, condition, script, or scenario, and addresses documentation, submission, approval, and signatures.

There were scripts included in the Release 9 test plan that are no longer in scope for the HCTC. These scripts represented historical requirements that were never implemented, or are no longer necessary, as well as requirements with no system impact. During Release 9, test scripts that were not applicable were marked as “not tested” in the test script spreadsheet, per existing processes. At the time of the review, Release 9 scripts were correctly tested in accordance with the existing process in place at the time. Prior to Release 10 test execution, the test plan and detailed test scripts were updated to remove the test scripts and other references that were no longer applicable.

Office of Audit Comment: The IRS noted that corrective action was accomplished during the review. However, documentation supporting the corrective action was not provided to us during the review, so we were unable to validate the actions taken. As a result, corrective action will be validated during the subsequent review of the HCTC Program Office (Audit Number 201020101) currently in process.

Performance Measures Targets Should Be Updated Using Current Estimates

Although the HCTC Program Office used satisfactory processes to determine performance measures, they were unable to meet all performance measures targets. As documented in the results of the monthly performance measure reviews, established measures did not consistently meet monthly or FY 2009 targets. Specifically, the final FY 2009 results included one performance measure (Customer Service Representative Level of Service) that did not meet the FY 2009 performance measures targets for the HCTC Program. As such, the IRS has already established a risk stating the HCTC Program is not on track to meet annual performance measures targets forecast for FYs 2009 through 2011.

The Internal Revenue Manual requires targets being set for any measure to include a review of statistics for the previous year’s results, historical patterns, and availability of resources. Additionally, as priorities change and new program requirements evolve, the suite of measures included in the budget must adjust to changing circumstances.

The original performance measures targets were established using Program statistics prior to the new HCTC legislative requirements being passed in the Recovery Act. Although these requirements significantly increased the volume of enrollees in the HCTC Program, performance measures targets were not created using the new data as a baseline. An accurate measurement of the Program’s effectiveness cannot be accomplished when using obsolete targets.

Due to the inaccurate targets used for performance measures, the HCTC Program Office planned to conduct an assessment addressing the accuracy of the baseline for FY 2011 once they determined population and operational volumes stabilized. However, launching a system of this nature in a live setting is inherently risky and, therefore, adjusting the performance baseline as soon as possible would provide a more accurate snapshot of system effectiveness.

Recommendation

Recommendation 4: The Commissioner, Wage and Investment Division, should ensure that the HCTC Program Office uses current FY 2010 data to develop performance measures targets for FY 2011. Additionally, applicable current fiscal year data should be used when developing performance measures targets for future years to ensure accurate representation of the HCTC Program’s effectiveness.

Management’s Response: The IRS agreed with the recommendation. The IRS acknowledges that new performance measures targets should be established for FY 2011 and intends to baseline several of the performance measures targets using FY 2010 data in order to develop new targets for FY 2011. However, the IRS stated that its inability to estimate targets did not diminish its ability to measure outcomes or demonstrate HCTC Program Office effectiveness. In FY 2009, agreement was obtained from the Wage and Investment Division Operations Strategy Board and the Department of the Treasury that FY 2010 would be a baseline year for many of the performance measures targets. Many of the HCTC measures are based on workload and enrollee volumes, which fluctuated widely due to the uncertain nature of the expanded HCTC population. The HCTC Program Office collects monthly information and shares the results with IRS leadership and the Department of the Treasury.

Office of Audit Comment: The audit recommendation is intended to ensure the HCTC Program Office uses current FY 2010 data to develop performance measures targets for FY 2011, which would provide better performance measures. Because the final FY 2009 results included one performance measure (Customer Service Representative Level of Service) that did not meet the performance measures targets for the HCTC Program, we question whether the IRS had sufficient and timely information to develop accurate performance measures targets with which to make risk-based decisions in the HCTC Program. Updating performance measures targets provides complete historical information needed for accurate rebaselining.

Appendix I

Detailed Objective, Scope, and Methodology

The overall objective of this audit was to determine whether the IRS had established and followed adequate system development processes to ensure efficient and effective upgrades of the HCTC Program. Specifically, we determined whether the upgrades to the HCTC system 1) were completed as mandated by legislation, 2) were ready to process HCTC claims, and 3) provided the intended benefits to the IRS and taxpayers for the filing season. To accomplish this objective, we:

I. Determined whether the HCTC Release 9 development provided the planned capabilities.

A. Determined whether testing support processes for the releases were adequately planned for deployment.

1. Determined whether all entrance and exit criteria were being met prior to initiating and exiting the various testing cycles by obtaining and reviewing the final test reviews for each testing cycle.

2. Determined whether the HCTC project team adequately controlled waivers and deferrals during the various test cycles.

a) Obtained copies of the test case deviation and waiver forms and determined whether they were processed and approved. All requests should be submitted in writing using the test case deviation and waiver form. All approved requests should have Program-level signatures.

b) Determined the effect or impact of each test waiver or deviation approved.

c) Ensured that deviation test cases from a test cycle were subsequently tested.

3. Determined whether all aspects of the application were tested.

a) Matched the requirements listed in the Program Improvement Form Requirement Report to the requirements listed in the Requirements Traceability Verification Matrix. If there were mismatches, determined why, e.g., a change request added or deleted a requirement, or the Requirements Traceability Verification Matrix was not complete.

b) Verified that all requirements listed in the Requirements Traceability Verification Matrix have a test case. If not, determined why and how these requirements were verified and validated.

c) If specific requirements were not testable, determined what type of assurance was provided to ensure that specific features function as intended.

d) Reviewed test results to determine whether the system performed as intended (e.g., unit testing, string testing, compatibility testing, user testing, test plan results, end of test status reports).

B. Determined whether requirements management activities were adequately planned for and whether the system is ready to process HCTC claims and provide the intended benefits to the IRS and taxpayers as mandated by authorizing legislation.

1. Interviewed the HCTC Program Office to determine how requirements were identified and validated based on the Recovery Act.

2. Reviewed how requirements were baselined (e.g., requirements specification documents, legislation requirements traceability matrix, program improvement form).

3. Reviewed documentation showing how changes to the requirements were tracked, monitored, and approved (e.g., the requirements specification document, requirements traceability matrix).

4. Reviewed documentation to support management’s oversight in the requirements management process (e.g., meeting minutes, email, management reports).

II. Determined whether the HCTC Program Office had adequately implemented the new legislation mandates.

A. Interviewed the HCTC Program Office to determine how performance measures were used to determine Program success.

B. Reviewed the performance measures results to determine whether service levels were met.

Internal controls methodology

Internal controls relate to management’s plans, methods, and procedures used to meet their mission, goals, and objectives. Internal controls include the processes and procedures for planning, organizing, directing, and controlling program operations. They include the systems for measuring, reporting, and monitoring program performance. We determined the following internal controls were relevant to our audit objective: sound project management guidelines for developing and testing requirements. We evaluated these controls by interviewing management and reviewing requirements, testing documentation, and performance measures.

Appendix II

Major Contributors to This Report

Alan R. Duncan, Assistant Inspector General for Audit (Security and Information Technology Services)

Scott A. Macfarlane, Director

Kimberly R. Parmley, Audit Manager

Charlene L. Elliston, Lead Auditor

Tina Wong, Senior Auditor

Melinda H. Dowdy, Auditor

Appendix III

Report Distribution List

Commissioner C

Office of the Commissioner – Attn: Chief of Staff C

Deputy Commissioner, Wage and Investment Division SE:W

Director, Electronic Tax Administration and Refundable Credits, Wage and Investment Division

SE:W:ETARC

Program Manager, Health Coverage Tax Credit, Wage and Investment Division SE:W:HCTC

Chief Counsel CC

National Taxpayer Advocate TA

Director, Office of Legislative Affairs CL:LA

Director, Office of Program Evaluation and Risk Analysis RAS:O

Office of Internal Control OS:CFO:CPIC:IC

Audit Liaisons:

Commissioner, Wage and Investment Division SE:W

Chief, GAO/TIGTA/Legislative Implementation Branch SE:S:CLD:PSP:GTL

Appendix IV

Glossary of Terms

Change Request	The medium for requesting approval to change a baselined product or other controlled item.
Configuration Management	Involves establishing proper control over approved project documentation, hardware, and software and assuring changes are authorized, controlled, and tracked.
Deferral	An approved request for verification of a requirement or set of requirements to be moved to another phase of testing.
Enterprise Life Cycle	A structured business systems development method that requires the preparation of specific work products during different phases of the development process. The Enterprise Life Cycle establishes a set of repeatable processes and a system of reviews, checkpoints, and milestones that reduce the risks of system development and ensure alignment with the overall business strategy.
Filing Season	The period from January through mid-April when most individual income tax returns are filed.
Release	A specific edition of software.
Requirement	A formalization of a need and is the statement of a capability or condition that a system, subsystem, or system component must have or meet to satisfy a contract, standard, or specification.
Requirements Traceability Matrix	A matrix developed and continually updated that links each requirement to a test case. The matrix provides important information on system testing and functionality status.
Software Engineering Institute	A Federally Funded Research and Development Center whose purpose is to help others make measured improvements in their software engineering capabilities.
Software Engineering Institute’s Capability Maturity Model Integration	A process improvement approach that provides organizations with the essential elements of effective processes.
Test Scripts	A series of instructions written for, and specific to, performing each test case.
Traceability	Traceability of requirements can be established from the source requirement to its lower level requirements and from the lower level requirements back to their source. Such traceability helps determine that all source requirements have been completely addressed and that all lower level requirements can be traced to a valid source.

Appendix V

Management’s Response to the Draft Report

DEPARTMENT OF THE TREASURY

INTERNAL REVENUE SERVICE

ATLANTA, GA 30308

COMMISSIONER

WAGE AND INVESTMENT DIVISION

APR 28, 2010

MEMORANDUM FOR MICHAEL R. PHILLIPS

DEPUTY INSPECTOR GENERAL FOR AUDIT

FROM: Richard Byrd. Jr. /s/Richard Byrd

Commissioner, Wage and Investment Division

SUBJECT: Draft Audit Report - Recovery Act Provisions for the Health Coverage Tax Credit Were Implemented, but Development Processes Could Be Improved (Audit # 200920130)

I reviewed the subject draft report and appreciate your acknowledgement that the IRS' Health Coverage Tax Credit (HCTC) Program Office effectively managed funds and implemented Recovery Act provisions. As your report states, the Recovery Act provisions were completed as manda1ed by legislation, the HCTC system was ready to process health care claims, and benefits were provided to taxpayers. Your report also found that the HCTC Program Office is actively monitoring expenditures to ensure they do not exceed the funds obtained, and is executing measures to implement program requirements. We agree with your recommendations and have already implemented several development process corrective actions based on this review.

The IRS has administered the HCTC since it was created in 2002. The tax credit is designed to provide assistance in paying for health insurance to a select group of individuals who might otherwise be uninsured. For eligible taxpayers, the HCTC makes health care insurance more affordable by paying 80 percent of the health insurance premiums (up from 65 percent prior to the Recovery Act).

The Recovery Act provisions for HCTC included increasing the IRS' portion of health premiums paid from 65 to 80 percent; reimbursing the taxpayers' portion of premiums paid during the enrollment period; allowing family members to continue receiving benefits after certain life events; and expanding taxpayer eligibility in the Program. Effective implementation of these provisions was critical to provide taxpayers the benefits mandated by the Recovery Act.

We agree with most of your findings about the development processes, including the need to complete test case waiver and deferral forms. We have completed implementation of the corrective actions to address the recommendations around requirements traceability, the configuration management plan, and test plan analysis. For any issues found, and not addressed, during Release 9.0, we implemented the corrective actions during Release 10.0.

Finally, the HCTC Program received unqualified approval from the Wage and Investment Division Modernization Advisory Council for Recovery Act implementation during both Releases 9.0 and 10.0. Independent Verification & Validation by the Wage and Investment Business Systems Planning staff provides continuous oversight for the HCTC Program and ensures all applicable Enterprise Life Cycle milestones are satisfied.

Attached are our specific comments on your recommendations. If you have any questions, please contact me, or a member of your staff may contact Rinaldi Boykin, Acting Program Manager, Health Coverage Tax Credit, Wage and Investment Division, at (202) 283-9600

Attachment

RECOMMENDATION 1

The Commissioner, Wage and Investment Division, should ensure that for ongoing and future system upgrades, the HCTC Program Office completes full traceability on the requirements traceability matrix, including mapping of all requirements to high-level requirements, and tracing to the final product requirements. These include changes, source codes, and any manual referencing accomplished.

CORRECTIVE ACTION

This recommendation outlines our standard processes; therefore, we agree with the recommendation and have already addressed it. We completed full traceability of the Release 9.0 requirements traceability matrix, August 19, 2009, following system implementation, per our standard processes. The requirements traceability report contains some requirements that do not call for mapping to test cases or build components.

IMPLEMENTATION DATE

N/A

RESPONSIBLE OFFICIAL

N/A

CORRECTIVE ACTION MONITORING PLAN

N/A

RECOMMENDATION 2

The Commissioner, Wage and Investment Division, should ensure that for ongoing and future releases, the HCTC Program Office updates the Configuration Management Plan to include procedures to verify all configuration changes are properly documented and tracked.

CORRECTIVE ACTION

We agree with this recommendation and have already implemented additional corrective actions. At the time of the Treasury Inspector General for Tax Administration (TIGTA) review, we had many standard configuration management processes in place.

As part of our standard process, we document all change requests in a change control log, during the development phase. All Release 9.0 and Release 10.0 change requests were reviewed and approved by the Health Coverage Tax Credit (HCTC) Office and had no effect on cost and schedule; therefore, we did not include those metrics. Cost and schedule were included in the change control log template, effective November 11, 2009

The Application Management Operating System database is used to capture items that are changed in the production system, in accordance with our Configuration Management Plan. During development phase, the approved change requests are documented in the change control log, per our standard processes.

The Configuration Management Plan was updated January 29, 2010, to include a clear differentiation of how change requests are managed during development, maintenance, and operations.

IMPLEMENTATION DATE

N/A

RESPONSIBLE OFFICIAL

N/A

CORRECTIVE ACTION MONITORING PLAN

N/A

RECOMMENDATION 3

The Commissioner, Wage and Investment Division, should ensure the HCTC Program Office performs an in-depth analysis of test plans prior to beginning each program release for elimination of previously tested requirements and. in the event a test case is erroneously included, completes the Test Case Waiver and Deferral form for approving its removal.

CORRECTIVE ACTION

We agree with the recommendation to implement a waiver and deferral process, and have implemented the corrective actions with Release 10.0. At the time of your review, we had vigorous testing processes, and as noted below, we completed the implementation of the waiver and deferral process, peer review of test cases, and removal of test cases, from test plans, that were no longer applicable. As these processes existed, or were created for Release 10.0, they are part of our standard procedures for future releases.

There were scripts that were included in the Release 9.0 test plan that are no longer in scope for HCTC. These scripts represented historical requirements that were never implemented, or are no longer necessary, as well as requirements with no system impact. During Release 9.0, test scripts that were not applicable were marked as "not tested" in the test script spreadsheet, per existing processes. At the time of your review, Release 9.0 scripts were correctly tested in accordance with the existing process in place at the time. Prior to Release 10.0 test execution, the test plan and detailed test scripts were updated to remove the test scripts and other references that were no longer applicable. A waiver and deferral process was implemented to document and approve the reasons behind the decision for not executing a particular test script. A peer review was conducted on this test documentation, to confirm the removal of the invalid test scripts.

The process documented with Release 10.0 outlines the procedures for authorizing the deferral or waiver of a test case, condition, script or scenario; and addresses documentation, submission, approval, and signatures. In addition, at the HCTC System Operations level, the process specifies the procedures for creating a Change Request to track and document the rationale for the waiver or deferral, the impacts on the system or business teams, the test item for waiver or deferral consideration, and a process for obtaining a decision on approval.

IMPLEMENTATION DATE

N/A

RESPONSIBLE OFFICIAL

N/A

CORRECTIVE ACTION MONITORING PLAN

N/A

RECOMMENDATION 4

The Commissioner, Wage and Investment Division, should ensure the HCTC Program Office use current Fiscal Year (FY) 2010 data to develop performance measures targets for FY 2011. Additionally, applicable current fiscal year data should be used when developing performance measures targets for future years to ensure accurate representation of the HCTC Program's effectiveness.

CORRECTIVE ACTION

We acknowledge that new performance measures targets should be established for Fiscal Year 2011. However, we do not agree that our inability to estimate targets diminishes our ability to measure outcomes or demonstrate HCTC Program effectiveness. We intend to baseline several of our performance measure targets using FY 2010 data in order to develop new targets for FY 2011. Baselining should be completed by November 1, 2010.

In FY 2009, we obtained agreement from the Wage and Investment (W&I) Division Operations Strategy Board and the Department of the Treasury that FY 2010 would be a baseline year for many of our performance measures targets. Many of the HCTC measures are based on workload and enrollee volumes, which fluctuated widely this year due to the uncertain nature of the expanded HCTC population. The HCTC Office collects up-to-date information, monthly, and shares the results with Program leadership, W&I Senior Leadership, and the Department of the Treasury. Our measures are very robust, and we look forward to analyzing new baselines and establishing new performance measure targets for the coming years.

IMPLEMENTATION DATE

November 15, 2010

RESPONSIBLE OFFICIAL

Program Manager, Health Coverage Tax Credit, Wage and Investment Division

CORRECTIVE ACTION MONITORING PLAN

Implementation will be monitored by the Wage and Investment Division Operations Strategy Board and the Department of the Treasury as we establish FY 2011 performance measure targets.

[5] The base year of the task order goes from February to September 2009.