TREASURY INSPECTOR GENERAL FOR TAX ADMINISTRATION

 

 

Despite Steps Taken to Increase Electronic Returns, Unresolved Modernized e-File System Risks Will Delay the Retirement of the Legacy e-File System and Implementation of Business Forms

 

 

 

September 27, 2012

 

Reference Number:  2012-20-121

 

 

This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document..

 

Phone Number  /  202-622-6500

E-mail Address /  TIGTACommunications@tigta.treas.gov

Website           /  http://www.tigta.gov

 

 

HIGHLIGHTS

DESPITE STEPS TAKEN TO INCREASE ELECTRONIC RETURNS, UNRESOLVED MODERNIZED E-FILE SYSTEM RISKS WILL DELAY THE RETIREMENT OF THE LEGACY E-FILE SYSTEM AND IMPLEMENTATION OF BUSINESS FORMS

Highlights

Final Report issued on September 27, 2012

Highlights of Reference Number:  2012-20-121 to the Internal Revenue Service Chief Technology Officer.

IMPACT ON TAXPAYERS

The Modernized e-File (MeF) system, which will replace the existing electronic filing system (the Legacy e-File system), is a critical component of the IRS initiative to meet the needs of taxpayers, reduce taxpayer burden, and broaden the use of electronic interactions.  Unresolved performance issues with MeF Release 7.0 and planned Calendar Year 2012 infrastructure changes have impaired the IRS’s efforts to retire the existing Legacy e-File system and have delayed plans for receiving employment tax forms through MeF Release 8.0.  Deferring the implementation of employment tax forms and schedules will further delay the ability to detect employment tax return errors on a real-time basis.

WHY TIGTA DID THE AUDIT

This review was part of our Fiscal Year 2011 Annual Audit Plan and addresses the major management challenge of Modernization.  The overall audit objective was to evaluate the IRS’s progress toward having transmitters fully participate in the MeF system and retiring the Legacy e-File system in October 2012.

WHAT TIGTA FOUND

The IRS took steps to increase the volume of returns transmitted to the MeF system and increased the number of vendors’ software packages available to transmit electronic tax returns.  However, our audit findings indicate that unresolved performance issues with MeF Release 7.0 existed as of its deployment.  In addition, the IRS Information Technology organization is planning significant infrastructure changes in Calendar Year 2012 that will introduce uncertainty and may affect MeF system reliability.  Further, the IRS has not developed a retirement plan for the Legacy e‑File system that includes the conditions that will need to be met before the system can be shutdown.  Lastly, the MeF system has not yet fully demonstrated the ability to process all electronically filed returns for an entire filing season, projected to be over 121 million combined individual and business returns.

WHAT TIGTA RECOMMENDED

TIGTA recommended that the Chief Technology Officer:  1) advise the Wage and Investment Division to defer retirement of the Legacy e‑File system until the increased risk associated with its retirement can be addressed;  2) update the Internal Revenue Manual to include improved performance testing processes, ensure system performance test teams obtain approved waivers or deferrals when performance tests are not executed, and ensure performance test teams submit End of Test Status Reports for senior management review; and 3) advise the Wage and Investment Division to complete a retirement plan for the Legacy e-File system as well as communicate retirement milestones and timelines to key stakeholders.

In their response to the report, IRS officials partially agreed with the recommendations.  The IRS is taking steps to build a contingency plan for the MeF system and updating Internal Revenue Manuals.  However, IRS officials did not state they would develop a Legacy e-File system retirement plan, along with associated implementation dates and monitoring plans.

 

September 27, 2012

 

 

MEMORANDUM FOR CHIEF TECHNOLOGY OFFICER

 

FROM:                       Michael E. McKenney /s/ Michael E. McKenney

Acting Deputy Inspector General for Audit

 

SUBJECT:                  Final Audit Report – Despite Steps Taken to Increase Electronic Returns, Unresolved Modernized e-File System Risks Will Delay the Retirement of the Legacy e-File System and Implementation of Business Forms (Audit # 201120025)

 

This report presents the results of our review of the Internal Revenue Service’s (IRS) progress toward having transmitters fully participate in the Modernized e-File System and retiring the Legacy e-File system in October 2012.  This audit was included in the Treasury Inspector General for Tax Administration’s Fiscal Year 2011 Annual Audit Plan and addresses the major management challenge of Modernization.

Management’s complete response to the draft report is included in Appendix VI.

Copies of this report are also being sent to the IRS managers affected by the report recommendations.  Please contact me at (202) 622-6510 if you have any questions or Alan R. Duncan, Assistant Inspector General for Audit (Security and Information Technology Services), at (202) 622-5894.

 

                                                

Table of Contents

 

Background

Results of Review

The IRS Took Positive Steps to Increase the Volume of Tax Returns Transmitted to MeF Release 7.0 During the 2012 Filing Season

An Increased Number of Transmitters’ Software Packages Passed MeF Release 7.0 Software Assurance Testing Prior to Deployment

The IRS Introduced Compensating Controls to Mitigate Risks Associated With Failed MeF Release 7.0 Systems Acceptability Tests

Unresolved Performance Issues and Infrastructure Changes Have Impaired Efforts to Retire the Legacy e-File System and Caused the IRS to Defer the Implementation of Employment Tax Forms in MeF Release 8.0

Recommendation 1:

Guidance Does Not Require Test Waivers, Test Deferrals, and End of Test Status Reports for Performance Testing

Recommendation 2:

The IRS Has Not Developed a Retirement Plan, Including Measurable Shutdown Conditions, for the Legacy e-File System

Recommendation 3:

Appendices

Appendix I – Detailed Objective, Scope, and Methodology

Appendix II – Major Contributors to This Report

Appendix III – Report Distribution List

Appendix IV – Comparison of the Modernized e-File and Legacy e-File Systems

Appendix V – Glossary of Terms

Appendix VI – Management’s Response to the Draft Report

 

 

Abbreviations

 

ATS

Assurance Testing System

e-File

Electronically File

IRS

IRS IT

Internal Revenue Service

IRS Information Technology

MeF

Modernized e-File

SAT

Systems Acceptability Testing

W&I

Wage and Investment

 

 

Background

 

The IRS deployed MeF Release 7.0 into production on January 17, 2012, for the 2012 Filing Season. 

Release 7.0 included all Form 1040 (U.S. Individual Income Tax Return) series forms and schedules.

The mission of the Internal Revenue Service (IRS) Information Technology (IRS IT)[1] organization is to deliver information technology services and solutions that drive effective tax administration to ensure public confidence.  The Applications Development function within the IRS IT organization is responsible for building, testing, delivering, and maintaining integrated information systems to support systems and their production environment.  This includes developing, testing, and maintaining the Modernized e-File (MeF) system.  The IRS Wage and Investment (W&I) Division serves as the system owner for the MeF system.

The MeF system directly supports the IRS’s strategic plan to improve taxpayer service, enhance enforcement of the tax law, and modernize the IRS through its people, processes, and technology.  It is also a critical component of the W&I Division initiative to meet the needs of taxpayers, reduce taxpayer burden, broaden the use of electronic interactions, and support the mission of the Small Business/Self-Employed Division.  

In December 1998, the IRS announced a strategic goal to revolutionize the way taxpayers transact and communicate with the IRS.  In order to achieve this goal, the IRS initiated plans to replace the existing electronic filing system, referred to as the Legacy e-File system, with the MeF system.  Once completely developed and deployed, the MeF system will accept electronic tax returns for corporations, partnerships, nonprofit/tax-exempt businesses, and individuals via a modernized Internet-based platform.  The MeF system currently enables real-time processing of tax returns while improving error detection, standardizing business rules, and expediting acknowledgments to taxpayers.  The MeF system also allows for standard electronic attachments with tax forms.  Appendix IV provides a detailed comparison of key features for the Legacy e‑File and MeF systems.  Figure 1 shows key processes and systems for accepting, rejecting, and forwarding both paper and electronic returns through IRS systems and subsystems.

Figure 1:  Key IRS Processes and Systems for
Accepting, Rejecting, and Forwarding Tax Returns

Figure 1 was removed due to its size.  To see Figure 1, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

Source:  Combination of IRS As-Built Architecture data flows and application development documentation.

Note:  EMS = Electronic Management System.  FTP = File Transfer Protocol.  GMF = Generalized Mainline Framework.  XML = Extensible Markup Language.

In February 2004, the IRS deployed the initial release[2] of the MeF system, which provided for electronic filing of Form 1120, U.S. Corporation Income Tax Return, and associated corporate forms.  In February 2010, the IRS deployed MeF Release 6.1 to begin electronically processing the Form 1040, U.S. Individual Income Tax Return, series along with 22 other forms and schedules.  MeF Release 6.2 deployed in January 2011 and focused on improving system performance and capacity.  MeF Release 7.0 deployed on January 17, 2012, for the 2012 Filing Season.  The latest release included the remaining Form 1040 series forms and schedules.  Since February 2004 when the IRS first deployed the MeF system, the IRS has also maintained the Legacy e-File system while it has developed and deployed subsequent MeF system releases to accommodate more returns. 

The Submission Processing Executive Steering Committee is chartered by the IRS IT Enterprise Governance Committee and governs projects within the Submissions Processing function portfolio, including the MeF system.  Its primary objective is to ensure project objectives are met, risks are managed appropriately, and the expenditure of enterprise resources is fiscally sound.  On January 10, 2012, the IRS reported to the Executive Steering Committee that MeF Release 7.0 cost $62.1 million; this includes Fiscal Year 2012 MeF Release 7.0 development costs but does not include IRS employee salaries and travel costs.

We previously reported[3] that the volume of returns processed by the MeF system had fallen short of expectations for the 2010 and 2011 Filing Seasons.  Specifically, as of  April 18, 2011, the number of electronically filed individual tax returns transmitted through the MeF system for the 2011 Filing Season was 8.7 million.  This was significantly lower than the 35 million individual tax returns the IRS had projected to be filed electronically through the system for the 2011 Filing Season.  The low volume of returns processed by the MeF system at that time affected the IRS’s ability to test the system’s capacity and performance with a volume of transactions representative of production processing.

This review was performed at the IRS IT organization in New Carrollton, Maryland, during the period August 2011 through April 2012.  We conducted this performance audit in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objective.  We believe the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objective.  Detailed information on our audit objective, scope, and methodology is presented in Appendix I.  Major contributors to the report are listed in Appendix II.

 

 

Results of Review

 

The IRS took steps to increase the volume of tax returns that transmitters would submit to the MeF system during the 2012 Filing Season.  In addition, as of MeF Release 7.0 deployment on January 17, 2012, the IRS doubled the number of vendors’ software packages that were available to submit electronic tax returns through the MeF system compared to the same time last year. 

Although the IRS took these positive steps, our audit findings indicate that unresolved performance issues existed with MeF Release 7.0 at the time of deployment.  In addition, the IRS IT organization is planning significant infrastructure changes in Calendar Year 2012 that will introduce uncertainty and may affect the MeF system’s reliability.  Unresolved MeF Release 7.0 performance issues and the planned infrastructure changes have impaired the IRS’s efforts to retire the Legacy e-File system and have caused the IRS to defer the implementation of employment tax forms in MeF Release 8.0.  Deferring the implementation of employment tax forms and schedules in MeF Release 8.0 will postpone the IRS from detecting employment tax return errors on a real-time basis.  Further, the IRS has not developed a retirement plan for the Legacy e-File system, including measurable shutdown conditions and milestones.  Lastly, MeF Release 7.0 has not yet fully demonstrated the ability to process all electronically filed returns for an entire filing season, which for the 2012 Filing Season was projected to be over 121 million[4] combined individual and business returns.  During the 2012 Filing Season, the MeF system experienced problems that delayed refunds for about a week for millions of taxpayers.  As a result, the IRS instructed transmitters to fall back to filing electronic returns to the Legacy e‑File system during this time.  

Our review of IRS Systems Acceptability Testing (SAT) of MeF Release 7.0 found that the IRS took necessary steps to introduce compensating controls aimed at mitigating risks associated with 30 failed SAT tests.  However, our review of performance testing for MeF Release 7.0 found that IRS guidance does not require test waivers, test deferrals, and End of Test Status Reports for performance testing.  As a result, MeF Release 7.0 performance test documentation was not sufficient for us to determine the significance of performance test results, how the test results related to performance requirements, and if performance requirements were sufficiently satisfied. 

The IRS Took Positive Steps to Increase the Volume of Tax Returns Transmitted to MeF Release 7.0 During the 2012 Filing Season

Prior to the 2012 Filing Season, the low volume of tax returns processed by the MeF system impaired the IRS’s ability to test the system for sufficient capacity and performance to meet a steadily increasing number of electronic tax return transmissions.  Specifically, during the 2010 and 2011 Filing Seasons, the actual volume of returns transmitted to the MeF system fell considerably short of IRS expectations.  To address this condition, the IRS took steps to secure commitments from large transmitters (e.g., transmitters with one million or more electronically filed returns in the 2011 Filing Season) for the percentage of returns each transmitter agreed to transmit to the MeF system during the 2012 Filing Season.  During the 2012 Filing Season, the IRS operated both the Legacy e-File and MeF systems and planned to fully retire the Legacy e‑File system in October 2012.  To help ensure that the MeF system received the volume needed to test system capacity and performance during the 2012 Filing Season, the IRS put early parameters in place to limit the volume of tax returns transmitted to the Legacy e-File system.  The IRS communicated to transmitters that companies that had electronically transmitted one million or more electronic returns (including all Forms 1040 and extensions) during the 2011 Filing Season were required to transmit returns through the MeF system for the 2012 Filing Season.

By securing MeF system volume commitments prior to the start of the 2012 Filing Season and limiting the use of the Legacy e-File system, tax return volumes transmitted to the MeF system will likely increase, which will enable the IRS to better assess the MeF system’s capacity and performance capabilities.  For the 2011 Filing Season, the IRS received only 8.7 million individual tax returns through the MeF system, of which 6.2 million were filed through the system by April 18, 2011.  Subsequent to our audit fieldwork, the IRS reported that as of April 17, 2012, the MeF system received 72.4 million individual tax returns, whereas the Legacy e-File system received 49.8 million returns for the same period.

An Increased Number of Transmitters’ Software Packages Passed MeF Release 7.0 Software Assurance Testing Prior to Deployment  

The IRS reported that as of January 17, 2012, 41 vendor software packages had successfully passed Tax Year 2011 MeF Assurance Testing System (ATS) tests.  The preliminary results of ATS testing through January 17, 2012, represented a significant increase over the number of software packages (23) that the IRS reported as having passed ATS testing as of the same time in 2011. 

Software developers participating in the MeF system software testing process must complete ATS testing in accordance with the applicable publication for each type of tax return that is transmitted through the MeF system.  Publication 1436, Test Package for Electronic Filers of Individual Income Tax Returns for Tax Year 2011, details the ATS test requirements for Tax Year 2011.  Specifically, Publication 1436 outlines numerous tax return scenarios that each software developer must successfully complete in order for its software package to be accepted into the MeF program for the 2012 Filing Season.  This testing began on October 31, 2011.  We reviewed all periodic ATS status reports from the IRS Electronic Products and Services Support organization.  We judgmentally reviewed test case files for seven of 25 transmitters that had submitted approved test cases.  Each test case file contained test data on software packages for a particular transmitter participating in ATS testing.  We evaluated the test case files and verified that each MeF software package designated as having passed ATS testing met the ATS test requirements outlined in Publication 1436. 

The emphasis on promoting the MeF system as the preferred platform for electronic filing has encouraged transmitters to complete ATS testing for their respective software packages.  In addition, the increased number of software packages passing ATS testing provided tax preparers a greater selection of software packages to use when preparing and electronically filing their 2012 Filing Season tax returns.

The IRS Introduced Compensating Controls to Mitigate Risks Associated With Failed MeF Release 7.0 Systems Acceptability Tests

The SAT is required to help ensure that designed and delivered software meets customer requirements.  Most of the 4,491 SAT test cases met stated functional requirements for the MeF system.  However, to mitigate the risk that 30 failed test cases may adversely affect the processing of tax returns, the W&I Division and IRS IT organization took steps to implement compensating controls prior to and during deployment of MeF Release 7.0.  Specifically, the IRS prioritized test cases for resolution handling, patched code prior to the January 17, 2012, deployment date, relied on another IRS error detection system until the code could be patched, or correctly explained to transmitters that certain error responses were in fact not errors.  

For MeF Release 7.0, SAT took place between July 13, 2011, and January 13, 2012, and included four code releases.  The Enterprise Systems Testing function within the IRS IT organization is responsible for independently conducting quality assessments of application software, including MeF system software.  This includes testing the MeF system with controlled data to determine conformance of the system to customer requirements and aiding the customers and developers in determining the system’s production readiness.  For MeF system testing, Enterprise Systems Testing function personnel relied on requirements derived from Unified Work Requests to prepare test scenarios, data, and expected test results.  A Unified Work Request is a mechanism to document, control, monitor, and track requests to the IRS IT organization for changes to IRS computer systems and for support.  There were a total of 145 Unified Work Requests and 4,491 test cases for MeF Release 7.0. 

According to the End of Test Status Report, dated January 20, 2012, there were 30 failed SAT test cases requiring resolution at the conclusion of the SAT process.  Thirteen of the failed test cases related to specific MeF business rules that help to prevent fraud and detect errors.  The remaining 17 failed test cases related to MeF application code and edit rules that primarily affect downstream processing of tax returns after transmission through the MeF system.

Unresolved Performance Issues and Infrastructure Changes Have Impaired Efforts to Retire the Legacy e-File System and Caused the IRS to Defer the Implementation of Employment Tax Forms in MeF Release 8.0

The IRS must conduct successful performance testing of the MeF system to provide assurance that the Legacy e-File system can be retired as planned.  It is important that the system demonstrate the ability to process all business and individual returns filed electronically for an entire filing season, which was projected to be over 121 million returns for the 2012 Filing Season.  Internal Revenue Manual 2.5.2, Systems Development – Software Testing Standards and Procedures, dated September 3, 2010, states “Performance testing is required to determine whether the system undergoing testing can effectively process transactions under expected normal and peak workload conditions and within acceptable response times.”  If performance and capacity tests are not successful, the risk that the system may fail to meet stated business performance requirements is increased.  Specific risks include the possibility that an application could become unavailable to process mission-critical transactions or the system could experience unexpected outages or degraded response times.

MeF Release 7.0 performance testing disclosed unresolved performance issues as of deployment in January 2012.  These issues involved 100 percent central processing unit utilization, time‑consuming events, and poor input and output of MeF data.  More specifically, the test results indicated that the system was running at capacity, having problems returning acknowledgements in a timely manner, experiencing a high volume of traffic to its database, and experiencing time-consuming get/send services transactions that posed a possible denial of service condition.  The cause of the performance issues is multifaceted and involves the architecture of the application code, the database configuration, memory and disk management, and the processing capacity of MeF system processors.

In addition to improving MeF Release 7.0 system performance, the IRS must be prepared to respond to planned enterprise-wide infrastructure changes that include upgrading Service-wide storage devices, deploying enterprise-wide software to run Java programs, deploying a new web portal, and upgrading to Internet Explorer 8.®  Further, the IRS IT organization must identify and implement code fixes as its W&I Division business partner identifies production issues and creates new work requests for the MeF system.  Uncertainty regarding MeF system reliability due to these upcoming planned enterprise-wide infrastructure changes will affect many IRS systems and applications.  Resolving MeF system performance and reliability weaknesses within the planned environment is critical to ensure the IRS meets its goal of transitioning to the MeF system as the single system for processing electronic tax returns. 

MeF Release 8.0 was initially scheduled to deploy employment tax forms and schedules in January 2013.  However, on January 10, 2012, the IRS IT organization requested a scope change for MeF Release 8.0.  The change included the resolution of MeF performance issues and preparation for planned infrastructure changes.  The Submission Processing Executive Steering Committee approved the change request and determined that MeF Release 8.0 will not contain the new forms and schedules initially planned for the release.  Under this new approach for MeF Release 8.0 development and implementation, the following tasks are underway to improve system performance and reliability:

·       Modify the architecture of MeF application code to reduce high processor utilization, improve timeliness of generating acknowledgements, and reduce the likelihood of creating a denial of service condition. 

·       Investigate data caching products that could allow the application to reduce the level of work on the database.

·       Modify the database configuration and upgrade the database patch-level.

·       Improve database disk management.

·       Upgrade MeF processors to accommodate larger files.

Of the 29.7 million employment tax returns filed in the 2010 Filing Season, 77 percent were filed on paper, with the remainder e-filed to the Legacy e-File system.  Deferring the implementation of employment tax forms and schedules in MeF Release 8.0 will postpone the IRS from detecting employment tax return errors on a real-time basis.  In addition, MeF Release 7.0 has not fully demonstrated the ability to process all electronically filed tax returns for an entire filing season.  Coupled with the uncertain impact of planned infrastructure changes, these unresolved system performance and reliability issues necessitate a careful reevaluation of the IRS’s initial plans to retire the Legacy e-File system in October 2012. 

For information regarding the 2012 Filing Season, please refer to our audit report, While Use of the Modernized e-File System for Individual Tax Returns Has Increased, the Legacy e-File System Is Still Needed As a Backup.[5]

Recommendation

Recommendation 1:  To deliver information technology services and solutions that drive effective tax administration, we recommend that the Chief Technology Officer advise the W&I Division to defer the retirement of the Legacy e-File system until the increased risk associated with retiring the system can be addressed. 

Management’s Response:  The IRS agreed with our recommendation and is taking the prudent step of building a contingency for the Legacy e-File system to accept a limited set of individual tax return forms and schedules should the MeF system experience serious productions issues.  Given the results of tests to date, the IRS does not expect to invoke this contingency during the 2013 Filing Season.

Guidance Does Not Require Test Waivers, Test Deferrals, and End of Test Status Reports for Performance Testing

Internal Revenue Manual 2.16.1, Enterprise Life Cycle, dated September 4, 2010, pertains to systems tests.  Tests for IRS systems include system acceptability tests, system integration tests, and final integration tests.  Testing is undertaken to determine that the complete integrated solution works as intended.  Further, system tests determine that the system executes properly without errors, expected results are obtained, performance is acceptable, and requirements are satisfied.  While general guidance provided by the Internal Revenue Manual addresses what systems tests and performance tests should accomplish, it does not provide guidance for or require End of Test Status Reports for performance testing or approval for test waivers or test deferrals.  These specific process controls, however, are necessary to ensure that:  (a) system performance tests are executed or are authorized by management to be waived or deferred; (b) system performance test results are documented and mapped to the related system performance requirements, thereby ensuring performance requirements are satisfied; and (c) valid performance test conclusions are reached, documented, and reported to management. 

Because an End of Test Status Report was not required for performance testing, MeF system performance testing documentation simply showed test results by topic and corrective actions that the IRS planned to take.  Performance test documentation was not sufficient to determine the significance of the performance test results, how the test results related to performance requirements, and whether or not performance requirements were sufficiently satisfied through the testing process.  For example, the IRS identified and approved 16 performance requirements in its MeF Release 7.0 Business System Requirements Report.  Eleven of those requirements represented performance capabilities required of the Registered User Portal and Employee User Portal web servers for handling MeF transactions.  The Performance Enhancement Team Test Plan specifically requires that MeF Release 7.0 performance tests were to include portal, enterprise, backend, and mainframe components.  During our review, officials revealed that the 11 web server requirements were not tested prior to MeF Release 7.0 deployment.  Further, responsible IRS IT organization personnel informed us that they did not request or obtain a deferral or waiver to exclude these requirements from testing, even though this could have provided for management’s authorization not to test all system requirements as planned.  Moreover, an End of Test Status Report for MeF system performance testing was not prepared, even though this report would have provided information regarding test results to allow customers, developers, and management to assess the quality of the product and its readiness for deployment.

Recommendation

Recommendation 2:  The Chief Technology Officer should update relevant Internal Revenue Manual sections and ensure that system performance test teams:  a) obtain approved waivers or deferrals if performance tests are not executed, and b) prepare and submit End of Test Status Reports to senior management for performance tests completed.

Management’s Response:  The IRS agreed with our recommendation and is updating Internal Revenue Manual 2.127, Software Testing Standards and Procedures, to reflect these additional requirements for performance testing.

The IRS Has Not Developed a Retirement Plan, Including Measurable Shutdown Conditions, for the Legacy e-File System  

The IRS has not yet developed a Legacy e-File system retirement plan that includes measureable shutdown conditions that should be achieved before the system can be retired.  Further, the IRS has not obtained approval from the Submission Processing Executive Steering Committee for its retirement plan and has not clearly communicated actionable retirement plans to internal and external stakeholders. 

Internal Revenue Manual 2.16.1, Enterprise Life Cycle, dated September 4, 2010, states that retirement or termination status represents the end of a project or system’s life cycle.  It provides for the systematic termination of a project or system to ensure that vital information is preserved for potential future access or reactivation.  The emphasis of this status is to ensure that the project or system (e.g., equipment, parts, software, data, procedures, and documentation) is packaged and disposed of in accordance with appropriate regulations and requirements. 

In the Electronic Tax Administration Advisory Committees Annual Report[6] dated June 2011, the committee recommended that “By October 31, 2011, the IRS should develop specific shutdown conditions for retiring Legacy after the 2012 Filing Season.  The IRS should develop an internal fallback plan if the retirement deadline is not met or the MeF platform is unable to handle the volume.”  We met with IRS management to discuss their system retirement planning efforts and, specifically, what actions they planned in response to these recommendations.  They indicated that a retirement plan for the Legacy e-File system was not in place because they were focusing on deploying MeF Release 7.0 for the 2012 Filing Season.  The officials also noted that the IRS is not required to follow the recommendations from the committee and stressed the intent to reassess the situation after the 2012 Filing Season.

When specific and measurable performance metric shutdown conditions are documented, agreed upon, and achieved, the IRS will have valid criteria on which to base its decision to recommend the retirement of this mission-critical system.  By identifying key stakeholder steps and timelines for shutting down the Legacy e-File system, IRS management can better implement a controlled, orderly shutdown of the system and help ensure a smooth transition to its goal for sole reliance on the MeF system to accept all electronic returns.

Recommendation

Recommendation 3:  The Chief Technology Officer should advise the W&I Division to develop a Legacy e-File system retirement plan, obtain governance approval of the retirement plan, and communicate to key internal and external stakeholders the activities and timeline for retiring the Legacy e-File system.  The retirement plan should include:  a) specific and measurable qualitative and quantitative shutdown conditions that should be achieved in order for the Applications Development function to recommend that the Legacy e-File system be retired, and b) a fallback plan if the shutdown conditions to retire the system are not achieved or if the MeF system is unable to handle peak filing season volume.

Management’s Response:  The IRS agreed with our recommendation and is developing a Legacy e-File contingency with a limited set of individual forms and schedules.  The legacy system will be dormant and will only be invoked should the MeF system experience production issues.  To ensure that MeF will be able to process all of the 2013 Filing Season volume, the IRS is focusing Release 8.0 efforts on system performance and stability.  Furthermore, although discussions regarding the future of the Legacy e-File environment have taken place, a final decision to retire the Legacy e-File system has not been made.  When the IRS is ready for a final decision, it will be made through the existing formal governance process established over this environment.  The IRS is implementing technology solutions to increase processing throughput and proactively monitor the health of the system for any bottlenecks.

Office of Audit Comment:  The IRS management response is silent on whether they agree or disagree with our recommendation to develop a Legacy e-File system retirement plan, obtain governance approval of the retirement plan, and communicate to key internal and external stakeholders the activities and timeline for retiring the Legacy e-File system.  We believe a Legacy e-file system retirement plan is important because it will prompt the IRS to define criteria on which to base its decision to recommend the retirement of this system.  By not including an implementation date and corrective action monitoring plans, this recommendation has not been adequately addressed.

 

Appendix I

 

Detailed Objective, Scope, and Methodology

 

Our overall audit objective was to evaluate the IRS’s progress toward having transmitters[7] fully participate in the MeF System and retiring the Legacy e-File system in October 2012.  

I.                 Testing and Performance Readiness.  Determined whether the IRS is ready to completely transition from the Legacy e-File system to the MeF system.

A.    Determined whether the MeF system is having performance issues.

1.     Observed Performance Enhancement Team testing to verify that best practices were being followed.

2.     Reviewed the scope of performance testing and determined whether performance testing included stress testing, load testing, and capacity testing.

3.     Verified that the test environment mimicked the production environment.

4.     Verified that performance testing encompassed returns with different forms.

5.     Verified that testing encompassed returns with various attachments (the size and number of attachments).

B.    Determined whether the performance measures adequately supported the submission processing requirements.

1.     Compared the MeF Release 7.0 SAT Test Plan to SAT Test results.  Ensured that failed SAT test cases at deployment were satisfactorily resolved.

2.     Verified tax forms and schedules planned for MeF Release 7.0 were tested.

C.    Determined whether the IRS SAT tested the additional Form 1040, U.S. Individual Income Tax Return, series forms and if the results were positive.

1.     Reviewed the performance requirements for MeF Release 7.0 and compared them with testing parameters and test cases.

2.     Reviewed and verified the process for documenting performance testing results.

II.               Transmitter Readiness for Filing Season.  Determined whether transmitters were ready to participate in the 2012 Filing Season. 

A.    Determined whether software developers’ software packages were adequately tested and approved prior to transmitting returns to the MeF system.

1.     Obtained and reviewed an approved software developers list.

2.     Judgmentally[8] selected and reviewed the test results for seven of 25 approved transmitters’ software packages.

B.    Determined whether transmitters using the Legacy e-File system will be able to transition to using the MeF system.  We obtained and analyzed a list of transmitters that met the mandate to file tax returns electronically regarding the following data points: 

1.     The total number of tax returns filed for Tax Years 20092011.

2.     Whether the transmitter filed returns exclusively through the Legacy e-File system, exclusively through the MeF system, or through a combination of the Legacy e-File and MeF systems for Tax Years 20092011.

Internal Controls Methodology

Internal controls relate to management’s plans, methods, and procedures used to meet their mission, goals, and objectives.  Internal controls include the processes and procedures for planning, organizing, directing, and controlling program operations.  They include the systems for measuring, reporting, and monitoring program performance.  We determined the following internal controls were relevant to our audit objective:  the Enterprise Life Cycle and related IRS guidelines and the processes followed in the development of information technology projects.  We evaluated these controls by reviewing the guidelines, conducting interviews and meetings with management and staff, and reviewing project documents.

 

Appendix II

 

Major Contributors to This Report

 

Alan R. Duncan, Assistant Inspector General for Audit (Security and Information Technology Services)

Gwendolyn McGowan, Director, Systems Modernization and Applications Development

Carol Taylor, Audit Manager

Ryan Perry, Lead Auditor

Charlene Elliston, Senior Auditor

Cari Fogle, Senior Auditor

Sylvia Sloan-Copeland, Senior Auditor

Trisa Brewer, Auditor

Kevin Liu, Information Technology Specialist

Michael Mohrman, Information Technology Specialist

Robert Carpenter, Information Technology Specialist

 

Appendix III

 

Report Distribution List

 

Commissioner  C

Office of the Commissioner – Attn:  Chief of Staff  C

Deputy Commissioner for Operations Support  OS

Deputy Chief Information Officer for Strategy/Modernization  OS:CTO

Director, Privacy, Information Protection and Data Security  OS:P

Associate Chief Information Officer, Enterprise Operations  OS:CIO:EO

Deputy Associate Chief Information Officer, Applications Development  OS:CTO:AD

Director, Program Management  OS:CTO:AD:PM

Director, Submission Processing  OS:CTO:AD:SP

Chief, Program Evaluation and Improvement, Wage and Investment Division  SE:W:S:PRA:PEI

Chief Counsel  CC

National Taxpayer Advocate  TA

Director, Office of Legislative Affairs  CL:LA

Director, Office of Program Evaluation and Risk Analysis  RAS:O

Office of Internal Control  OS:CFO:CPIC:IC

Audit Liaisons: 

Chief, Program Evaluation and Improvement, Wage and Investment Division  SE:W:S:PRA:PEI

            Director, Risk Management Division  OS:CTO:SP:RM

 

Appendix IV

 

Comparison of the Modernized e-File and Legacy e-File Systems

 

Condition

MeF e-File System

Legacy e-File System

Data Format

Extensible Markup Language (XML)

American Standard Code for Information Interchange (ASCII)

Attachments

Accepted in Portable Document Format (PDF)

No attachments accepted

Transmissions

Transaction based system – processed on receipt

Batch return system using three drain times to process transmissions

Acknowledgements

Acknowledgments returned in real-time

Acknowledgments returned within two days of receipt

Tax Return Errors

Business rules for each error condition

Error rejects codes that may address multiple conditions

Processing Year

Year-round processing

No electronically filed tax returns after October 15

Prior Year Returns

Prior year tax returns in Filing Season 2011 

Prior year tax returns are not accepted

Source:  IRS MeF system and Legacy e-File system documentation.

 

Appendix V

 

Glossary of Terms

 

Term

Definition

Capacity Testing

Testing used to determine how many users and/or transactions a given system will support and still meet performance goals.

Central Processing Unit

An internal component of the computer that performs arithmetic and logical operations, extracts instructions from memory, decodes instructions, and executes instructions.

Controlled Data

Data developed to test a specific set of conditions and expected results for a particular test objective.

Data Caching

Specialized, high-speed storage used to store frequently accessed or recently accessed date.

Denial of Service

When a machine or network resource becomes unavailable to its intended users.

Enterprise Life Cycle

A structured business systems development method that requires the preparation of specific work products during different phases of the development process.

Filing Season

The period from January through mid-April when most individual income tax returns are filed.

Internet Explorer

A series of graphical web browsers developed by the Microsoft Corporation and included as part of the Microsoft Windows operating system.

Java

A general purpose, concurrent, class-based, object-oriented language that is specifically designed to have as few implementation dependencies as possible.

Load Testing

The process of putting demand on a system or device and measuring its response.  Load testing is performed to determine a system’s behavior under both normal and anticipated peak load conditions.

Mainframe Computer

A term that is used to distinguish high-end commercial machines, with large-scale computer system architectures, from less powerful units.

Memory Management

The act of managing computer memory.  The essential requirement of memory management is to provide ways to dynamically allocate portions of memory to programs at their request, freeing it for reuse when no longer needed.

Milestone

Milestones provide for “go/no-go” decision points in a project and are sometimes associated with funding approval to proceed.

Patch

A code change to fix a program bug. 

Portal

A point-of-entry into a network system that includes a search engine or a collection of links to other sites, usually arranged by topic. 

Release

A specific edition of software.

Requirement

A statement of capability or condition that a system, subsystem, or system component must have or meet to satisfy a contract, standard, or specification.

Risk

A potential event that could have an unwanted impact on the cost, schedule, business, or technical performance of an information technology program, project, or organization.

Stress Testing

A form of testing that determines the stability of a given system to operate beyond normal capacity, often to a breaking point, in order to observe the results.

Tax Year

The year for which taxpayers file their Federal income tax return.

Transmitter

A firm, organization, or individual that receives returns and Personal Identification Number registrations electronically from clients (Electronic Return Originators, Reporting Agents, or taxpayers), reformats the data (if necessary), batches the data with data from the returns and electronic Personal Identification Number registrations of other clients, and then transmits the data to the IRS.

 

Appendix VI

 

Management’s Response to the Draft Report

 

DEPARTMENT OF THE TREASURY

INTERNAL REVENUE SERVICE

WASHINGTON, D.C. 20224

 

CHIEF TECHNOLOGY OFFICER

 

September 11, 2012

 

 

MEMORANDUM FOR DEPUTY INSPECTOR GENERAL FOR AUDIT

 

FROM:                            Terence V. Milholland /s/ Terence V. Milholland

    Chief Technology Officer

 

SUBJECT:                       Draft Audit Report - Despite Steps Taken to Increase Electronic Returns, Unresolved Modernized e-File System Risks Threaten Goals to Retire the Legacy e-File System and Delay Implementation of Business Forms (Audit # 201120025) e-trak # 2012-34871

 

Thank you for the opportunity to review your draft audit report and to discuss earlier draft report observations with the audit team.

 

I was pleased to read your comments and observations acknowledging the steps we took to mitigate risks associated with the small number (30) of failed Systems Acceptance Testing (SAT) test casesI also want to acknowledge your favorable comments regarding the IRS's efforts to increase the volume of individual tax returns filed on Modernized e-File (MeF) during the 2012 Filing Season.

 

MeF's Release 7, the largest release in MeF history, deployed on schedule on January 17, 2012.  This Release included the rollout of more than 129 remaining Form 1040- family schedules and forms, including Forms 1040A and 1040EZ, which expanded MeF's reach to the entire e-File population of approximately 98.3 million filers.  In addition, all tax law changes for Business Master File returns were deployed with Release 7, and extensive system performance testing was also conducted.  Release 7 is exceeding established performance thresholds with acknowledgements being returned to transmitters well within the five minutes established in the Service Level Agreement with the business owner.

 

As of August 18, 2012, more than 116 million individual and business returns were submitted to MeF with more than 81 million accepted.  These 2012 Filing Season statistics compare with more than 21.9 million individual and business returns submitted and more than 17 million accepted for the same period during the 2011 Filing Season.  During the last peak filing period in April 2012, the MeF system showed that it can consistently process large volumes of tax returns for an extended period of time.

 

Attached you will find our planned corrective actions resulting from the audit recommendations in your report.  We are committed to continuously improving our information technology systems and processes.  We are confident in the Modernized E-File system's ability to process the electronic tax returns based on our test and production experience.  We value your continued support and the assistance and guidance your team provides.  If you have any questions, please contact me at (202) 622-6800 or Karen Mayr at (202) 283-0015.

 

Attachment

 

RECOMMENDATION #1:  To deliver information technology services and solutions that drive effective tax administration, we recommend that the Chief Technology Officer advise the W&I Division to defer the retirement of the Legacy e-File system until the increased risk associated with retiring the system can be addressed.

 

CORRECTIVE ACTION #1:  We are already taking the prudent step of building a contingency for the Legacy e-File system to accept a limited set of individual tax return forms and schedules should MeF experience serious production issues.  Given the results of our tests to date, we do not expect to invoke this contingency during the 2013 Filing Season.

 

IMPLEMENTATION DATE:  N/A

 

RESPONSIBLE OFFICIAL:  Associate Chief Information Officer, Applications Development

 

CORRECTIVE ACTION MONITORING PLAN: N/A

 

RECOMMENDATION #2:  The Chief Technology Officer should update relevant Internal Revenue Manual sections and ensure that system performance test teams:  a) obtain approved waivers or deferrals if performance tests are not executed and b) prepare and submit End of Test Status Reports to senior management for performance tests completed.

 

CORRECTIVE ACTION #2:  The IRS agrees with this recommendation.  IRM 2.127 (Software Testing Standard and Procedures) will be updated to reflect these additional requirements for performance testing.

 

IMPLEMENTATION DATE: January, 31, 2013

 

RESPONSIBLE OFFICIAL:  Associate Chief Information Officer, Applications Development

 

CORRECTIVE ACTION MONITORING PLAN:  We enter accepted Corrective Actions into the Joint Audit Management Enterprise System.  These Corrective Actions are monitored on a monthly basis until completion.

 

RECOMMENDATION #3:  The Chief Technology Officer should advise the W&I Division to develop a Legacy e-File system retirement plan, obtain governance approval of the retirement plan, and communicate to key internal and external stakeholders the activities and timeline for retiring the Legacy e-File system.  The retirement plan should include:  a) specific and measurable qualitative and quantitative shutdown conditions that should be achieved in order for the Applications Development function to recommend that the Legacy e-File system be retired and b) a fallback plan if the shutdown conditions to retire the system are not achieved or if the MeF system is unable to handle peak filing season volume.

 

CORRECTIVE ACTION #3:  As we noted in response to recommendation #1, we are developing a Legacy e-File contingency with a limited set for individual forms and schedules, the legacy system will be dormant and will only be invoked should MeF experience production issues.  To ensure that MeF will be able to process all of the 2013 Filing Season volume, we are focusing our Release 8 efforts on system performance and system stability.  Furthermore, although discussions regarding the future of the Legacy e-File environment have taken place, a final decision to retire Legacy e-File has not been made.  When IRS is ready for a final decision, it will be made through the existing formal governance process established over this environment.  We are implementing technology solutions to increase our processing throughput and proactively monitor the health of the system for any bottlenecks.

 

IMPLEMENTATION DATE: N/A

 

RESPONSIBLE OFFICIAL:  Associate Chief Information Officer, Applications Development

 

CORRECTIVE ACTION MONITORING PLAN: N/A



[1] On July 1, 2012, the Modernization and Information Technology Services organization officially changed its name to the Information Technology organization.  The name change reflects a shift in the organization’s way of thinking and operating as it collaborates with the business and functional operating divisions to deliver the IRS’s mission.  Instead of modernization being treated as a separate and distinct strategic offering within the IRS IT organization, it will now be incorporated into the overall portfolio.

[2] See Appendix V for a glossary of terms.

[3] TIGTA, Ref. No. 2011-40-131, Low Participation and Tax Return Volumes Continue to Hinder the Transition of Individual Income Tax Returns to the Modernized e-File System (Sept. 2011).

[4] As of April 17, 2012, the IRS reported the following statistics for the 2012 Filing Season:  49 million returns were transmitted to the Legacy e‑File system and 72 million returns were transmitted to the MeF system, for a total of 121 million electronically filed returns. 

[5] Ref. No. 2012-40-012, dated Aug. 2012.

[6] The Internal Revenue Service Restructuring and Reform Act of 1998 (RRA 98), Pub. L. No 105-106, 112 Stat. 685 (codified as amended in scattered sections of 2 U.S.C, 5 U.S.C. app., 16 U.S.C., 19 U.S.C., 22 U.S.C., 23 U.S.C., 31 U.S.C., 38 U.S.C., and 49 U.SC.) authorized the creation of the Electronic Tax Administration Advisory Committee.  This committee has a primary duty of providing input to the IRS on its strategic plan for electronic tax administration.  Its responsibilities involve researching, analyzing, and making recommendations on a wide range of electronic tax administration issues, as well as annually reporting to Congress. 

[7] See Appendix V for a glossary of terms.

[8] A judgmental sample is a nonstatistical sample, the results of which cannot be used to project to the population.