Enhancements to the Internet Refund Project Need to Be
Completed to Ensure Planned Benefits to Taxpayers Are Realized
February 2003
Reference
Number: 2003-20-053
This report has cleared the Treasury
Inspector General for Tax Administration disclosure review process and
information determined to be restricted from public release has been redacted
from this document.
February
11, 2003
MEMORANDUM FOR DEPUTY COMMISSIONER FOR MODERNIZATION
&
CHIEF INFORMATION OFFICER
FROM: Gordon C. Milbourn III /s/ Gordon C. Milbourn
Acting Deputy Inspector
General for Audit
SUBJECT: Final Audit Report - Enhancements to the
Internet Refund Project Need to Be Completed to Ensure Planned Benefits to
Taxpayers Are Realized (Audit #
200320002)
This
report presents the results of our review of the plans and actions to deploy
the Internal Revenue Service’s (IRS) Internet Refund Fact of Filing (IRFOF) modernization
project. The
objective of this review was to determine whether the Business Systems
Modernization Office (BSMO) and the PRIME contractor will complete the needed
enhancements to the IRFOF application to provide the capabilities planned for
2003.
The IRFOF application provides secure Internet access for taxpayers to determine whether their tax returns have been received and processed by the IRS, and the status of their refunds. A pilot version of the IRFOF application was deployed in May 2002, and it has handled over 1 million tax return and refund status requests. Taxpayer responses to survey questions about their use of the IRFOF application show that 78 percent of them were at least somewhat satisfied with this service.
The PRIME contractor has
provided enhancements to the IRFOF application since it was initially
deployed. The enhancements entitled
Release Package 1 were installed in October 2002 and provided application fixes
and an improved encryption level.
Further enhancements are included in Release Package 2. These enhancements include application
performance changes, 2003 tax law changes, and changes to enable the IRFOF
application to operate with the IRS’ upgraded Internet application. The PRIME contractor has stated it has
completed Release Package 2 development and project application testing, and
the IRS is completing its testing to determine whether to accept the enhanced
version of the IRFOF application.
In summary, although the
IRFOF application has been providing refund and fact of filing information to
taxpayers, the application is not currently providing the level of performance
required by the IRS. The product the
IRS received has not been tested to determine whether it can meet the demand of
taxpayer inquiries during the peak of the 2003 Filing Season. Additionally, the IRS does not expect to
realize the application’s economic benefits previously projected, and cost
overruns make up about 27 percent of the overall development and deployment
costs.
We
are not providing recommendations at this time since the corrective actions
needed to address the issues identified have been recommended in previous
Treasury Inspector General for Tax Administration reports. As this and the previous reports identify,
project management disciplines need to be followed to ensure projects can and
do meet realistic and attainable requirements within a reasonable budget
schedule.
Management’s Response: The Deputy Commissioner for Modernization
& Chief Information Officer provided comments on the progress made on the
IRFOF application. He stated that the
IRS has worked hard to ensure that the IRFOF application meets taxpayer needs,
and that the IRS is committed to deliver enhancements to the project as it
moves forward. He also stated that the
IRFOF application delivers business value to IRS operations and to American
taxpayers. It is a major step forward
for modernization and has been a popular customer service enhancement. Management’s complete response to the draft
report is included as Appendix IV.
Copies
of this report are also being sent to the IRS managers who are affected by the
report. Please contact me at (202)
622-6510 if you have questions or Gary V. Hinkle, Acting Assistant Inspector
General for Audit (Information Systems Programs), at (202) 927-7291.
Appendix
I – Detailed Objective, Scope, and Methodology
Appendix
II – Major Contributors to This Report
Appendix
III – Report Distribution List
Appendix
IV – Management’s Response to the Draft Report
The Internet Refund Fact of Filing (IRFOF) application provides secure Internet access for taxpayers to determine whether their tax returns have been received and processed by the Internal Revenue Service (IRS), and the status of their refunds. To protect the taxpayers’ privacy, the IRFOF application requires that the taxpayers provide their social security numbers, filing status, and the refund amount shown on the tax returns. The IRFOF project objective is to improve customer service and increase the number of IRS customer communications by providing instant fact of filing and refund status via the Internet to taxpayers expecting refunds from their tax returns. The project capability plans also include the ability for online users to receive information (such as links to other Internet sites, forms, and frequently asked questions) to assist them in resolving refund associated questions or problems.
The IRS selected the IRFOF application as one of its first modernization projects because it was a relatively uncomplicated project that would allow the IRS to gain experience and confidence with developing modernization projects, provide immediate benefits to taxpayers, and generate monetary benefits to the IRS. At the end of its design stage, the IRS expected that the IRFOF application would cost approximately $35 million to develop, deploy, and operate for 10 years. In addition to providing benefits to taxpayers, the IRS estimated that the IRFOF application would also generate almost $42 million in benefits to the IRS by reducing the number of telephone assistors needed to answer calls related to the status of tax refunds. These telephone assistors could then help taxpayers with more difficult tax questions.
The Business Systems Modernization Office (BSMO) and the PRIME contractor deployed the pilot version of the IRFOF application in May 2002. This application provides worldwide Internet access 24 hours a day, 7 days a week. Since going into production in May 2002, the IRFOF application has handled over 1 million tax return and refund status requests. Taxpayer responses to survey questions about their use of the Internet application show that 78 percent of them were at least somewhat satisfied with this application. During the latter part of 2002, the BSMO and the PRIME contractor have been working on enhancing the performance and capabilities of the IRFOF application.
The objective of this review was to determine whether the BSMO and the PRIME contractor will complete the needed enhancements to the IRFOF application to provide the capabilities planned for 2003. To accomplish our objective, we reviewed available documentation and interviewed executives from the PRIME contractor, and IRS executives, managers, and analysts located at the IRS’ National Headquarters in Washington, D.C., and New Carrollton, Maryland. We performed these reviews from October through December 2002 in accordance with Government Auditing Standards. Detailed information on our objective, scope, and methodology is presented in Appendix I. Major contributors to the report are listed in Appendix II.
The PRIME contractor has provided enhancements to the IRFOF application since it was initially deployed. The enhancements entitled Release Package 1 were installed in October 2002 and provided application fixes and an improved encryption level. Further enhancements are included in Release Package 2. These enhancements include application performance changes, 2003 tax law changes, and changes to enable the IRFOF application to operate with the IRS’ upgraded Internet application. The PRIME contractor has stated it has completed Release Package 2 development and project application testing, and the IRS is completing its testing to determine whether to accept the enhanced version of the IRFOF application.
Although the IRFOF application has been providing refund and fact of filing information to taxpayers, the application is not currently providing the level of performance required by the IRS. In addition, the IRFOF project team does not anticipate realizing all benefits shown in the documents used to get approval for the IRFOF project development, and the project has incurred material unanticipated costs.
Improved performance is needed for the IRFOF application to meet the IRS’ requirements
The IRFOF project was originally scheduled to be operational in March of the 2002 Filing Season. However, project testing delays and infrastructure problems postponed the IRFOF application deployment. The BSMO and the PRIME contractor deployed the application in May 2002, with the knowledge that performance response time and user capacity needed improvement to meet the peak demands for return filing and refund information during the filing season. The BSMO and the PRIME contractor ran the application during the summer when the demand was lower to ensure it was working as expected and to get feedback from taxpayers using the application. Enhancements and fixes could then be made before the 2003 Filing Season.
During the IRFOF project design stage, the project team developed a set of performance requirements based on IRS business needs, independent statistical analysis, and modeling approaches. The performance requirements were developed as a part of the overall set of performance requirements for the IRS’ modernized computer system infrastructure. The IRFOF project performance requirements include handling 320,000 tax return and refund status requests per hour during peak demand periods. This requirement translates to supporting an average of 89 request transactions per second during a peak hour.
Although the IRFOF application was designed to support up to 89 tax return and refund status requests per second, the pilot application deployed in May 2002 was not meeting this requirement. The PRIME contractor formulated a performance tuning plan to meet the intended requirement. Meanwhile, the BSMO delayed decisions to approve the completion of the deployment phase of the project until the performance tuning issues were resolved.
The PRIME contractor initially focused its performance tuning efforts on the efficiency of the IRFOF application. It identified efficiency opportunities with the IRFOF application software coding, and made changes to improve the tax return and refund status request transactions per second processed through the mainframe computer. In October 2002, the PRIME contractor reported that the IRFOF application within the mainframe computer was meeting the system requirements.
However, the PRIME contractor found that during an end-to-end test (i.e., testing the IRFOF application from the website to the mainframe computer), the IRFOF application was still unable to meet the target 89 transactions per second. In fact, it was only handling approximately 36 transactions per second, less than half of the capacity required. The PRIME contractor determined that the web application servers currently in place as part of the IRS’ modernized infrastructure are unable to provide the capacity needed for the IRFOF application system requirements. According to the PRIME contractor, the current web application servers could only accommodate taxpayer refund requests at 36 transactions per second.
The PRIME contractor and the IRS considered the 89 transactions per second requirement as an estimate of the highest capacity that would be needed, but which would likely not be reached. The PRIME contractor suggested that doubling the number of web application servers would provide the capacity to handle an average of 72 transactions per second, which it stated should be sufficient to meet peak demands during the filing season.
At the request of the BSMO Tax Administration Modernization Director, MITRE (the IRS’ modernization strategic support contractor) analyzed the PRIME contractor’s recommendation to double the number of web application servers. MITRE reviewed the end-to-end test the PRIME contractor conducted and determined that doubling the web application servers would not necessarily result in doubling the transactions per second handled by the IRFOF application. MITRE concluded that to meet the business system requirements would require tripling the number of web application servers. However, MITRE did conclude that the 89 transactions per second business system requirement may not occur during the 2003 Filing Season and, therefore, a lower requirement could be sufficient.
The IRS subsequently followed the PRIME contractor’s recommendation and purchased web application server upgrades at a cost of $401,000. The IRS completed the web application server upgrade installation in December 2002. However, no testing was planned prior to deployment to determine whether the IRFOF application works with the additional web servers, or whether the expected performance level will be reached. Additionally, there is no assurance that the application performance level will be sufficient to meet taxpayer demand during the filing season.
While we understand that the design stage requirements were based on estimates from statistical analysis and models, the current situation indicates that the PRIME contractor’s IRFOF application will not meet the contracted requirements, including handling up to 89 tax return and refund status request transactions per second. Subsequently, the IRS waived this requirement to allow the project to proceed with its deployment and to transition the IRFOF application operations to the IRS.
Management’s Action: Subsequent to the completion of our audit field work, the PRIME contractor and IRS management informed us that they have developed plans to test the web application server upgrades. These tests will include assessments of the IRFOF performance during the deployment site readiness testing phase.
The IRFOF project does not anticipate realizing the originally expected monetary benefits
The IRFOF project’s September 2001 Baseline Business Case (BBC), supporting the basis for moving forward with project development, presented economic benefits to the IRS in addition to improved customer service. The BBC showed that providing refund information to taxpayers via the Internet would reduce the number of refund-related telephone calls to the IRS, thus freeing up Customer Service Representatives (CSR) for more difficult tax questions. The BBC estimated that the IRFOF application would reduce the number of refund status telephone calls that get routed to CSRs by 150,000 within the first year of IRFOF deployment, growing to a reduction of 395,000 by year 3 and staying stable at about 395,000 through year 10. This translates into freeing up approximately 23 full time CSRs in year 1, growing to an equivalent of 89 CSRs in year 10.
Reducing the number of calls related to refund status would allow CSRs to concentrate on other types of customer tax account requests. This redirection of CSR effort is an off-budget monetary benefit – meaning that although the IRS is not reducing costs by reducing staff, it is anticipating a benefit by freeing up CSRs to better service taxpayers. The estimated off-budget monetary benefits from the BBC were almost $42 million, as presented in the following table.
Estimated Off-Budget Monetary Benefits
(in Millions)
|
Off-Budget (Monetary)
Benefits |
Total |
Year 1 |
Year 2 |
Year 3 |
Year 4 |
Year 5 |
Years |
|
|
|
|
|
|
|
|
|
|
Cost
Avoidance |
$41.72 |
$1.44 |
$2.17 |
$3.98 |
$4.18 |
$4.39 |
$25.49 |
Source: IRFOF Baseline Business Case, Table 3-4, Version
1.0D, Document No. PRIME-CC2002-BBC-IRFOF, September 20, 2001.
Since this BBC was delivered, the IRFOF project team has reconsidered the expected monetary benefits from the application. Analysis of telephone calls to the IRS shows the number of refund related inquiries has not changed since the deployment of the IRFOF application. The IRFOF project team initially thought that telephone traffic would decrease with use of the IRFOF webpage, but after researching this topic with state tax agencies that had similar applications already in place, it was discovered that no near-term telephone reductions occurred. As a result, the IRS does not anticipate realizing the expected $42 million in monetary benefits tied to a reduction in the number of CSRs needed to handle tax return/refund status telephone calls.
The IRS gave the IRFOF project team approval to exit its development and integration stage of the project life cycle in November 2002, even though work on the application had not been completed, nor a revised business case approved. In exchange for approving the exit of the project from development and full payment on the contract, the PRIME contractor agreed to provide the IRS with 80 hours of software development time to be used to try and fix any problems that may arise with the new web application servers or the installed application enhancements. This approval was contingent upon the Chief Financial Officer’s approval of an updated BBC.
The updated BBC was approved in December 2002. This BBC confirmed the project team’s reconsideration of benefits to the IRS and presents that:
“…observations from the experiences of U.S. State tax
agencies indicate that the IRFOF application will not reduce CSR workload
initially, therefore although IRFOF will increase the number of contacts
handled by the IRS, IRFOF is not expected to provide any on-budget or
off-budget monetary benefits in the near-term.”
This document focuses the project justification on non-monetary benefits in terms of improved customer service.
We are concerned that the information related to potential monetary benefits to the IRS changed so dramatically during the development of the IRFOF application. We understand that the projections were based on estimates, but the information gathered from the various state tax agencies was available for inclusion in the original BBC and should have been considered before including the projected monetary benefits in the BBC. Because the BBC is used to gain approval to start development of a project, it is critical that estimates of costs and benefits be as complete and accurate as possible.
We are also concerned that the IRS conditionally approved the IRFOF project team exiting the development and integration stage or milestone. We believe that approval to exit a milestone should only be given when all significant and material products have been delivered to and accepted by the IRS. In this case, the IRS approved the exit of the milestone and the payment to the PRIME contractor even though the IRFOF application has not yet met the requirements of the contract, the latest enhancements have not yet been fully tested, and the updated baseline business case was not finalized and approved.
The IRFOF project incurred material unanticipated costs
Up to the IRFOF project development stage, the PRIME contractor incurred costs of approximately $10.8 million. Subsequently, the IRFOF project targeted costs for development and deployment activities to total approximately $7 million, plus a fee to the PRIME contractor of almost $455,000.
The project had to incur additional costs to develop its “transition to support” plan, amounting to $465,404. It also incurred costs of $308,791 to update the IRFOF application for the 2003 Filing Season.
As of November 19, 2002, the project development and deployment costs totaled $11.2 million. This total includes cost overruns of almost $3 million. The cost overruns are largely attributable to application testing delays and additional labor costs to conduct the performance tuning activities mentioned above.
Summary
In summary, the IRFOF project has provided an additional service to taxpayers interested in determining the status of their tax return filing and refund. However, the product the IRS received from the PRIME contractor does not meet the contracted performance standards and has not been tested to determine whether it can meet the demand of taxpayer inquiries during the peak of the 2003 Filing Season. Additionally, the IRS does not expect to realize the project’s economic benefits previously projected, and cost overruns make up about 27 percent of the overall development and deployment project costs.
We are not providing recommendations in this report since corrective actions needed to address the issues identified have been recommended in previous Treasury Inspector General for Tax Administration reports. We previously issued two reports about the development of the IRFOF project, and another report that included the project’s management of transition to support activities.
As this and the previous reports identify, the IRFOF project team needs to implement and follow disciplined project management activities and processes that have been developed by the IRS and PRIME contractor. These project management activities and processes are clearly detailed in the IRS’ Enterprise Life Cycle, and include activities such as risk management, requirements management, coordination and integration with other BSM projects, and cost/schedule estimation processes. These project management activities and processes need to be followed to ensure projects can, and do, meet realistic and attainable requirements within a reasonable budget schedule.
Management’s Response: The Deputy Commissioner for Modernization & Chief Information Officer responded to our draft report and provided comments on the progress made on the IRFOF application. He stated that the IRS has worked hard to ensure that IRFOF meets taxpayer needs, and that the IRS is committed to deliver enhancements to the project as it moves forward. He also stated that the IRFOF application delivers business value to IRS operations and to American taxpayers. It is a major step forward for modernization and has been a popular customer service enhancement. Management’s complete response to the draft report is included as Appendix IV.
Appendix I
Detailed Objective, Scope, and
Methodology
The objective of this review was to determine whether the
Business Systems Modernization Office and the PRIME contractor would complete the needed enhancements to the
Internet Refund Fact of Filing (IRFOF) application to provide the capabilities
planned for 2003.
To accomplish this objective, we performed the following audit tests:
I. Reviewed the IRFOF application deployment plans and activities to determine if they would provide for timely delivery of intended taxpayer benefits for 2003.
A. Interviewed IRFOF project managers to determine the status of the project, delays affecting deployment, performance issues, and steps being taken to resolve them in 2003.
B. Participated in Customer Relationship Management Sub-Executive Steering Committee meetings and reviewed the minutes of status meetings for potential problems and delays.
C. Evaluated the current IRFOF application performance data to validate how closely the performance goals are being met.
D. Evaluated the planned IRFOF Maintenance Release Package changes to determine the impact to performance and timely deployment in 2003.
E. Evaluated the defect reports that were identified related to the performance tuning.
F. Obtained and reviewed the Baseline Business Case to analyze expected and actual performance goals and capabilities of the IRFOF project.
G. Determined what contingency plans were in place should planned IRFOF application performance goals not be achieved in time for the 2003 Filing Season.
II. Determined if the IRFOF development and
deployment task order contract included adequate activities and funding to
ensure the IRFOF application would be performing up to planned expectations in
2003.
Appendix II
Major Contributors to This Report
Gary V. Hinkle, Acting Assistant Inspector General for
Audit (Information Systems Programs)
Scott A. Macfarlane, Director
Edward A. Neuwirth, Audit Manager
James A. Douglas, Senior Auditor
Michael Garcia, Senior Auditor
Louis
Zullo, Senior Auditor
Appendix III
Acting Commissioner N:C
Associate Commissioner, Business Systems Modernization M:B
Chief, Information
Technology Services M:I
Deputy
Associate Commissioner, Program Management
M:B:PM
Chief
Counsel CC
National
Taxpayer Advocate TA
Director, Legislative Affairs CL:LA
Director, Office of Program Evaluation and Risk Analysis N:ADC:R:O
Office of Management Controls N:CFO:F:M
Audit Liaisons:
Associate
Commissioner, Business Systems Modernization
M:B
Associate Commissioner, Information Technology Services M:I
Appendix IV
Management’s
Response to the Draft Report
The response was removed due to its
size. To see the complete response,
please go to the Adobe PDF version of the report on the TIGTA Public Web Page.