Management Advisory Report: The Internal Revenue Service Needs a Reliable Measure of the Quality of Electronic Tax Law Assistance Provided to Small Businesses and Self-Employed Taxpayers
Reference Number: 2002-30-120
This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.
July 5, 2002
MEMORANDUM FOR COMMISSIONER, SMALL BUSINESS/SELF-EMPLOYED DIVISION
COMMISSIONER, WAGE AND INVESTMENT DIVISION
FROM: (for) Pamela J. Gardiner /s/ Scott E. Wilson
Deputy Inspector General for Audit
SUBJECT: Final Management Advisory Report - The Internal Revenue Service Needs a Reliable Measure of the Quality of Electronic Tax Law Assistance Provided to Small Businesses and Self-Employed Taxpayers (Review # 200130044)
This management advisory report presents the results of our review to determine whether the Internal Revenue Service (IRS) has a valid method for reliably measuring the quality of responses to tax law questions received from small businesses and self-employed taxpayers through the Electronic Tax Law Assistance (ETLA) Program. We performed this review in response to the continuing Congressional concern with the IRS’ ability to provide accurate tax law information to small business taxpayers.
In summary, the quality measurement system for the ETLA Program has not been changed to reflect the IRS’ new organizational structure. While a statistically valid measure of quality for the ETLA Program is available at the IRS-wide level, separate measures are not available on the quality of responses to questions received from each major customer segment. Without a separate measure, the IRS is precluded from reliably determining the quality of service being provided to small businesses and self-employed taxpayers.
We recommended that the IRS take action to develop reliable measures of the quality of service provided to each customer segment through the ETLA Program. One way that this could be accomplished is by redesigning the IRS’ Internet web site, Digital Daily, so that taxpayers are required to select a specific customer segment before submitting tax law questions through the ETLA Program. This would enable the IRS to more precisely identify the population of questions received from each customer segment and, in turn, design a statistical sampling plan that can reliably measure the quality of service being delivered to each segment.
The Commissioner, Small Business/Self-Employed (SB/SE) Division, responded that, while he agrees with the recommendation, there is a concern that capturing data about the type of customer (i.e., wage earners, self-employed, etc.) moves away from the Commissioner’s intention for the public portal to provide a totally anonymous, one-way interaction with the IRS. The SB/SE Division will work with the Internet Services Division of the Electronic Tax Administration organization to determine if the recommendation is feasible. The planned completion date of the feasibility study is January 1, 2003.
Office of Audit Comment: We appreciate the IRS’ concern with wanting to maintain the anonymity of taxpayers when they interact with the Digital Daily. Since the IRS already requires taxpayers to provide their Email address when submitting questions to the ETLA Program, however, we do not believe that also requiring taxpayers to indicate a particular customer segment would further diminish the anonymity that they are already provided. Nevertheless, to avoid any perception that this additional information might be used to identify taxpayers, the IRS could include a disclosure statement on the Digital Daily that states that the customer segment information is solely used for measuring the quality of service provided.
Management’s complete response to the draft management advisory report is included as Appendix VI.
Copies of this report are also being sent to the IRS managers who are affected by the report recommendation. Please contact me at (202) 622-6510 if you have questions or Gordon C. Milbourn III, Assistant Inspector General for Audit (Small Business and Corporate Programs), at (202) 622-3837.
Appendix I – Detailed Objective, Scope, and Methodology
Appendix II – Major Contributors to This Report
Appendix III – Report Distribution List
Appendix IV – Senator Bond’s Letter to the Internal Revenue Service
Appendix V – Internal Revenue Service’s Response to Senator Bond
Appendix VI – Management’s Response to the Draft Report
The Internal Revenue Service (IRS) Restructuring and Reform Act of 1998 (RRA 98) mandated that the IRS do a better job of meeting the needs of its customers. To comply with this Congressional mandate, the IRS revised its mission statement to refocus its emphasis on helping taxpayers understand and meet their tax responsibilities, and modernized its organizational structure from one that was based on function and geography to one that is aligned with its major customer segments. Four new customer-focused business operating divisions were created with full end-to-end responsibility for serving distinct groups of taxpayers with similar needs.
One of these, the SB/SE Division, serves about 7 million small businesses, including corporations and partnerships with assets under $10 million, and 33 million individuals that are fully or partially self-employed. Small businesses and self-employed taxpayers need specialized services from the IRS to comply with an increasingly complex tax code. The National Taxpayer Advocate’s Fiscal Year (FY) 2000 Annual Report to the Congress rated the complexity of business tax laws as one of the most serious problems facing taxpayers, and stated that the complexity of the tax code can drive some small businesses into technical noncompliance.
One of the IRS’ performance goals is to provide taxpayers with greater access to assistance before they file their tax returns. The IRS provides a variety of pre-filing assistance to taxpayers through its tax publications, its toll-free telephone system, and its Internet web site called the Digital Daily. The IRS’ Electronic Tax Law Assistance (ETLA) Program provides a means for taxpayers to submit tax law and procedural questions through the Digital Daily. Each question is routed to an assistor who responds to the taxpayer in the form of an Email message. Taxpayers will generally receive a response from the IRS within 2 business days. During FY 2001, the IRS answered approximately 264,000 questions that were submitted through the ETLA Program.
During both the 2000 and 2001 Filing Seasons, auditors from the Treasury Inspector General for Tax Administration (TIGTA) submitted questions on small business and self-employment issues through the Internet to the IRS and commercial web sites that offered free tax advice. While the IRS outperformed the commercial web sites in both years, the IRS’ accuracy rates in answering the questions were significantly below its official quality measure for the ETLA Program. However, the prior reviews involved only 50 test questions and, thus, were not a statistically reliable measure of the quality of service provided to small businesses and self-employed taxpayers.
In reaction to our 2001 Filing Season review, the Ranking Member of the Senate Committee on Small Business and Entrepreneurship issued a letter (see Appendix IV) to the IRS Commissioner expressing disappointment with the results. The Senator further stated that the relatively low accuracy rate puts small businesses and self-employed taxpayers at substantial risk if they rely on the IRS for assistance in complying with the tax law. The Commissioner, SB/SE Division, responded (see Appendix V) to the Ranking Member that the IRS’ own statistically valid sampling of ETLA Program responses showed the IRS was accurate almost 83 percent of the time.
This review was performed at the SB/SE Division’s offices in Oxon Hill, Maryland, and at the IRS’ Centralized Quality Review Site (CQRS) in Philadelphia, Pennsylvania. We performed our review between December 2001 and March 2002 in accordance with the President’s Council on Integrity and Efficiency’s Quality Standards for Inspections. Detailed information on our review objective, scope, and methodology is presented in Appendix I. Major contributors to the report are listed in Appendix II.
The quality measurement system for the ETLA Program has not been changed to reflect the IRS’ new organizational structure. As a result, the IRS does not have a separate, statistically reliable measure of the quality of service that the ETLA Program is providing to small businesses and self-employed taxpayers.
The IRS has a centralized quality review site that samples Email responses for accuracy. The reviewers follow a sampling plan that is designed by the IRS Statistics of Income (SOI) function. The sampling plan requires the selection of Email responses without regard to the type of taxpayer or tax law category. The SOI function advised us that the IRS’ quality measure for the ETLA Program is statistically valid at only the IRS-wide level. Therefore, the IRS’ accuracy rate for the ETLA Program is not statistically reliable for gauging the quality of assistance provided to a specific customer segment.
The Digital Daily does not enable the IRS to separate the tax law questions received from each customer segment
Although the IRS modernized its structure in 2000 by aligning its operations along specific customer segments, the current design of the Digital Daily prevents the IRS from precisely determining which customer segment submitted a question. Presently, all taxpayers access the same site on the Digital Daily to submit a question. Every taxpayer submitting a tax law question on the Digital Daily must select one of the 17 categories shown in Figure 1.
Figure 1 was removed due to its size. To see the figure, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.
The computer system used to support the ETLA Program retains the tax law category that was selected by the taxpayer. However, this information is not particularly useful for determining the customer segment to which the taxpayer belongs. While some of the tax law categories, such as Small Business/Self-Employed/Other Business, specifically target small businesses and self-employed taxpayers, many others are applicable to multiple customer segments. For example, questions submitted in the tax law category – Individual Retirement Arrangement (IRA) – could be received from wage earners, self-employed, or businesses. In addition, questions submitted in the tax law category – Other – cover a broad spectrum of topics that pertain to all customer segments.
At the CQRS site, we examined 994 electronic tax law questions that were selected for quality review during the 2001 Filing Season and determined that 190 (19 percent) involved issues that generally would be associated with small businesses or self-employed taxpayers. Overall, as shown in the following table, these 190 taxpayers selected 11 (65 percent) of the 17 tax law categories shown on the Digital Daily when submitting questions to the IRS. Of these 190 taxpayers, 34 (18 percent) selected Other as the tax law category when submitting their question. Only 105 (55 percent) of the 190 taxpayers selected the two tax law categories (Sale or Trade of Business, Depreciation, Rentals and Small Business/Self-Employed/Other Business) that would seemingly focus on small businesses and self-employed taxpayers.
Table 1 was removed due to its size. To see the table, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.
The CQRS has implemented a manual coding procedure to separate tax law questions by customer segment
The quality review system used to measure the quality of service provided by the ETLA Program does not systemically capture the tax law category selected by the taxpayer. In recognition of the need to know the accuracy rate achieved for each customer segment, the IRS implemented a manual tracking procedure in February 2002. For each question selected for review, the quality review staff is now manually recording the tax law category selected by the taxpayer and, based on the content of the question, manually assigning a business operating division code to the electronic quality review document.
While this initiative is a positive first step toward giving IRS management feedback on the quality of service delivered to each customer segment, it is subjective since it depends on each reviewer’s analysis and interpretation of the taxpayer’s question. This method of coding questions by customer segment is also inexact since many questions received through the ETLA Program, such as those previously mentioned about IRAs, involve tax law issues that affect wage-earner taxpayers as well as small businesses and self-employed taxpayers.
Any future attempt by the IRS to measure the quality of responses to small businesses and self-employed taxpayers based on this method of identifying the customer segment would be statistically unreliable. One of the fundamental requirements of statistical sampling is that the population to be sampled must be clearly defined. Unsatisfactory results may be obtained when statistical sampling is applied to an improperly identified population.
The IRS needs reliable, more specific measures of the quality of service that the ETLA Program provides to different segments of customers
The ETLA Program offers small businesses and self-employed taxpayers a convenient communication medium for obtaining specialized tax law assistance. With the IRS’ new focus on preventing problems rather than addressing them after filing, the ETLA Program supports this change in direction by providing one-on-one assistance electronically, potentially resulting in fewer filing errors, decreased processing costs, and reduced taxpayer burden. Further, the ETLA Program reflects the vision of “government of the 21st century,” using information technology to communicate with its customers online.