Management Advisory Report:
The Internal Revenue Service Needs a Reliable Measure of the Quality of
Electronic Tax Law Assistance Provided to Small Businesses and Self-Employed
Taxpayers
July 2002
Reference Number: 2002-30-120
This report has cleared the Treasury
Inspector General for Tax Administration disclosure review process and
information determined to be restricted from public release has been redacted
from this document.
July
5, 2002
MEMORANDUM FOR
COMMISSIONER, SMALL BUSINESS/SELF-EMPLOYED DIVISION
COMMISSIONER,
WAGE AND INVESTMENT DIVISION
FROM: (for) Pamela J. Gardiner /s/ Scott E.
Wilson
Deputy Inspector General for
Audit
SUBJECT: Final Management Advisory Report - The
Internal Revenue Service Needs a Reliable Measure of the Quality of Electronic
Tax Law Assistance Provided to Small Businesses and Self-Employed Taxpayers
(Review # 200130044)
This
management advisory report presents the results of our review to determine
whether the Internal Revenue Service (IRS) has a valid method for reliably
measuring the quality of responses to tax law questions received from small
businesses and self-employed taxpayers through the Electronic Tax Law
Assistance (ETLA) Program. We performed
this review in response to the continuing Congressional concern with the IRS’
ability to provide accurate tax law information to small business taxpayers.
In
summary, the quality measurement system
for the ETLA Program has not been changed to reflect the IRS’ new
organizational structure. While a
statistically valid measure of quality for the ETLA Program is available at the
IRS-wide level, separate measures are not available on the quality of responses
to questions received from each major customer segment. Without a separate measure, the IRS is
precluded from reliably determining the quality of service being provided to
small businesses and self-employed taxpayers.
We recommended that the IRS
take action to develop reliable measures of the quality of service provided to
each customer segment through the ETLA Program. One way that this could be accomplished is by redesigning the
IRS’ Internet
web site, Digital Daily, so that taxpayers are required to select a
specific customer segment before submitting tax law questions through the ETLA
Program. This would enable the IRS to
more precisely identify the population of questions received from each customer
segment and, in turn, design a statistical sampling plan that can reliably
measure the quality of service being delivered to each segment.
The
Commissioner, Small Business/Self-Employed (SB/SE) Division, responded that,
while he agrees with the recommendation, there is a concern that capturing data
about the type of customer (i.e., wage earners, self-employed, etc.) moves away
from the Commissioner’s intention for the public portal to provide a totally
anonymous, one-way interaction with the IRS.
The SB/SE Division will work with the Internet Services Division of the
Electronic Tax Administration organization to determine if the recommendation
is feasible. The planned completion
date of the feasibility study is January 1, 2003.
Office of
Audit Comment: We appreciate the
IRS’ concern with wanting to maintain the anonymity of taxpayers when they
interact with the Digital Daily.
Since the IRS already requires taxpayers to provide their Email address
when submitting questions to the ETLA Program, however, we do not believe that
also requiring taxpayers to indicate a particular customer segment would
further diminish the anonymity that they are already provided. Nevertheless, to avoid any perception that
this additional information might be used to identify taxpayers, the IRS could
include a disclosure statement on the Digital Daily that states that the
customer segment information is solely used for measuring the quality of
service provided.
Management’s
complete response to the draft management advisory report is included as
Appendix VI.
Copies of this
report are also being sent to the IRS managers who are affected by the report
recommendation. Please contact me at
(202) 622-6510 if you have questions or Gordon C. Milbourn III, Assistant
Inspector General for Audit (Small Business and Corporate Programs), at (202)
622-3837.
Recommendation 1:
Appendix I – Detailed Objective, Scope, and Methodology
Appendix II – Major Contributors to This Report
Appendix III – Report Distribution List
Appendix IV – Senator Bond’s Letter to the Internal Revenue
Service
Appendix V – Internal Revenue Service’s Response to Senator
Bond
Appendix VI – Management’s Response to the Draft Report
The Internal Revenue Service (IRS) Restructuring and Reform
Act of 1998 (RRA 98) mandated that the IRS do a better job of meeting the needs
of its customers. To comply with this
Congressional mandate, the IRS revised its mission
statement to refocus its emphasis on helping taxpayers understand and meet
their tax responsibilities, and modernized its organizational structure from
one that was based on function and geography to one that is aligned with its
major customer segments. Four new
customer-focused business operating divisions were created with full end-to-end
responsibility for serving distinct groups of taxpayers with similar
needs.
One of these, the SB/SE Division,
serves about 7 million small businesses, including corporations and
partnerships with assets under $10 million, and 33 million individuals that are
fully or partially self-employed. Small
businesses and self-employed taxpayers need specialized services from the IRS
to comply with an increasingly complex tax code. The National Taxpayer Advocate’s Fiscal Year (FY) 2000 Annual
Report to the Congress rated the complexity of business tax laws as one of the
most serious problems facing taxpayers, and stated that the complexity of the
tax code can drive some small businesses into technical noncompliance.
One of the IRS’ performance goals is to provide taxpayers
with greater access to assistance before they file their tax returns. The IRS provides a variety of pre-filing
assistance to taxpayers through its tax publications, its toll-free telephone
system, and its Internet web site called the Digital Daily. The IRS’ Electronic Tax Law Assistance
(ETLA) Program provides a means for taxpayers to submit tax law and procedural
questions through the Digital Daily.
Each question is routed to an assistor who responds to the taxpayer in
the form of an Email message. Taxpayers
will generally receive a response from the IRS within 2 business days. During FY 2001, the IRS answered
approximately 264,000 questions that were submitted through the ETLA Program.
During both the 2000 and 2001 Filing Seasons, auditors from
the Treasury Inspector General for Tax Administration (TIGTA) submitted
questions on small business and self-employment issues through the Internet to
the IRS and commercial web sites that offered free tax advice. While the IRS outperformed the commercial
web sites in both years, the IRS’ accuracy rates in answering the questions
were significantly below its official quality measure for the ETLA Program. However, the prior reviews involved only 50
test questions and, thus, were not a statistically reliable measure of the
quality of service provided to small businesses and self-employed taxpayers.
In reaction to our 2001 Filing Season review, the Ranking
Member of the Senate Committee on Small Business and Entrepreneurship issued a
letter (see Appendix IV) to the IRS Commissioner expressing disappointment with
the results. The Senator further stated
that the relatively low accuracy rate puts small businesses and self-employed
taxpayers at substantial risk if they rely on the IRS for assistance in
complying with the tax law. The
Commissioner, SB/SE Division, responded (see Appendix V) to the Ranking Member
that the IRS’ own statistically valid sampling of ETLA Program responses showed
the IRS was accurate almost 83 percent of the time.
This review was performed at the SB/SE Division’s offices in
Oxon Hill, Maryland, and at the IRS’ Centralized Quality Review Site (CQRS) in
Philadelphia, Pennsylvania. We
performed our review between December 2001 and March 2002 in accordance with
the President’s Council on Integrity and Efficiency’s Quality Standards for
Inspections. Detailed information
on our review objective, scope, and methodology is presented in Appendix
I. Major contributors to the report are
listed in Appendix II.
The quality measurement system for the ETLA Program has not
been changed to reflect the IRS’ new organizational structure. As a result, the IRS does not have a
separate, statistically reliable measure of the quality of service that the
ETLA Program is providing to small businesses and self-employed taxpayers.
The IRS has a centralized quality review site that samples
Email responses for accuracy. The
reviewers follow a sampling plan that is designed by the IRS Statistics of
Income (SOI) function. The sampling
plan requires the selection of Email responses without regard to the type of
taxpayer or tax law category. The SOI
function advised us that the IRS’ quality measure for the ETLA Program is
statistically valid at only the IRS-wide level. Therefore, the IRS’ accuracy rate for the ETLA Program is not
statistically reliable for gauging the quality of assistance provided to a
specific customer segment.
The Digital Daily does not enable the IRS to
separate the tax law questions received from each customer segment
Although the IRS modernized its structure in 2000 by
aligning its operations along specific customer segments, the current design of
the Digital Daily prevents the IRS from precisely determining which
customer segment submitted a question.
Presently, all taxpayers access the same site on the Digital Daily
to submit a question. Every taxpayer
submitting a tax law question on the Digital Daily must select one of
the 17 categories shown in Figure 1.
Figure 1 was
removed due to its size. To see the
figure, please go to the Adobe PDF version of the report on the TIGTA Public
Web Page.
The computer system used to support the ETLA Program retains
the tax law category that was selected by the taxpayer. However, this information is not
particularly useful for determining the customer segment to which the taxpayer
belongs. While some of the tax law
categories, such as Small Business/Self-Employed/Other Business,
specifically target small businesses and self-employed taxpayers, many others
are applicable to multiple customer segments.
For example, questions submitted in the tax law category – Individual
Retirement Arrangement (IRA) – could be received from wage earners,
self-employed, or businesses. In
addition, questions submitted in the tax law category – Other – cover a
broad spectrum of topics that pertain to all customer segments.
At the CQRS site, we examined 994 electronic tax law
questions that were selected for quality review during the 2001 Filing Season
and determined that 190 (19 percent) involved issues that generally would be
associated with small businesses or self-employed taxpayers. Overall, as shown in the following table,
these 190 taxpayers selected 11 (65 percent) of the 17 tax law categories shown
on the Digital Daily when submitting questions to the IRS. Of these 190 taxpayers, 34 (18 percent)
selected Other as the tax law category when submitting their
question. Only 105 (55 percent) of the
190 taxpayers selected the two tax law categories (Sale or Trade of
Business, Depreciation, Rentals and Small Business/Self-Employed/Other
Business) that would seemingly focus on small businesses and self-employed
taxpayers.
Table 1 was
removed due to its size. To see the
table, please go to the Adobe PDF version of the report on the TIGTA Public Web
Page.
The CQRS has implemented a manual coding procedure to
separate tax law questions by customer segment
The quality review system used to measure the quality of service
provided by the ETLA Program does not systemically capture the tax law category
selected by the taxpayer. In
recognition of the need to know the accuracy rate achieved for each customer
segment, the IRS implemented a manual tracking procedure in February 2002. For each question selected for review, the
quality review staff is now manually recording the tax law category selected by
the taxpayer and, based on the content of the question, manually assigning a
business operating division code to the electronic quality review document.
While this initiative is a positive first step toward giving
IRS management feedback on the quality of service delivered to each customer
segment, it is subjective since it depends on each reviewer’s analysis and
interpretation of the taxpayer’s question.
This method of coding questions by customer segment is also inexact
since many questions received through the ETLA Program, such as those
previously mentioned about IRAs, involve tax law issues that affect wage-earner
taxpayers as well as small businesses and self-employed taxpayers.
Any future attempt by the IRS to measure the quality of
responses to small businesses and self-employed taxpayers based on this method
of identifying the customer segment would be statistically unreliable. One of the fundamental requirements of
statistical sampling is that the population to be sampled must be clearly
defined. Unsatisfactory results may be
obtained when statistical sampling is applied to an improperly identified
population.
The IRS needs reliable, more specific measures of the
quality of service that the ETLA Program provides to different segments of
customers
The ETLA Program offers small businesses and self-employed
taxpayers a convenient communication medium for obtaining specialized tax law
assistance. With the IRS’ new focus on
preventing problems rather than addressing them after filing, the ETLA Program
supports this change in direction by providing one-on-one assistance electronically,
potentially resulting in fewer filing errors, decreased processing costs, and
reduced taxpayer burden. Further, the
ETLA Program reflects the vision of “government of the 21st
century,” using information technology to communicate with its customers
online.