The Revised Collection Case Selection Criteria That
Expedites Trust Fund Workload to the Field Appears Effective
September 2004
Reference Number: 2004-30-173
This report has cleared the Treasury
Inspector General for Tax Administration disclosure review process and
information determined to be restricted from public release has been redacted
from this document.
September
17, 2004
MEMORANDUM FOR
COMMISSIONER, SMALL BUSINESS/SELF-EMPLOYED DIVISION
FROM: Gordon C. Milbourn III /s/ Gordon C.
Milbourn III
Acting Deputy Inspector
General for Audit
SUBJECT: Final Audit Report - The Revised
Collection Case Selection Criteria That Expedites Trust Fund Workload to the
Field Appears Effective (Audit # 200330037)
This
report presents the results of our review of the Internal Revenue Service’s
(IRS) collection case selection criteria.
The overall objective of this review was to determine whether new
Collection Field function (CFf) case selection criteria helped the Small
Business/Self-Employed (SB/SE) Division meet its goal of effectively resolving
delinquent trust fund accounts.
Between Fiscal Years (FY) 1996
and 2001, the IRS Collection function’s business results declined, due in part
to the 36 percent decrease in revenue officer staffing during the period. For example, the number of balance due tax
accounts closed decreased and the number of accounts not being worked
increased.
In an effort to improve
operations, the Commissioner, SB/SE Division, established teams to review
Collection function operations and suggest methods for improvement. One team recommended changing the risk levels used to identify workload for the CFf to place
more emphasis on assigning trust fund cases and assigning the cases earlier in
the balance due stream. Addressing
employment tax noncompliance is an operational priority included in the SB/SE
Division’s Business Action Plan.
In summary, our review of a
sample of closed trust fund cases and analysis of the Collection Activity
Reports (CAR) both showed improved business results since FY 2001 (our baseline
period and the last full fiscal year prior to implementation of the new
criteria) for the CFf. The number of
trust fund cases closed increased by 42 percent, and the number of additional
tax periods that accumulated (pyramided) while cases were assigned in the CFf
decreased by 17 percent. In addition, it does not appear that this shift in CFf
workload caused an adverse impact on overall Taxpayer Delinquent Account (TDA)
closures by the Collection function. The
number of TDAs closed, the percentage closed as fully paid, and the amount
collected all increased. These
improvements appear to indicate that the change in risk level criteria for
inventory selection was successful.
However, expected impacts of
the revised risk level case selection criteria were not adequately
measured. Efforts to measure the impact of
the revised risk level case selection criteria primarily involved an analysis
of the CARs. These reports do not
provide results on individual reengineering efforts or effective measures for
all of the expected benefits from the efforts, such as a decrease in
pyramiding. Limitations such as these
inhibit management’s ability to measure the success of reengineering efforts, thus
hampering decision-making ability when considering the impact of the
implementation of recommendations.
While there was an overall
improvement in business results from FYs 2001 to 2003, a year-to-year analysis
showed that some results declined from FYs 2002 to 2003. During FY 2003, other reengineering efforts
were being implemented that could also have affected some of the same
productivity indicators as the revised risk level case selection criteria. Without an effective process to measure the
impact of individual reengineering projects, it is not known whether the
overall improvements from FY 2001 and declines during FY 2003 were attributable
to the risk level criteria change, additional reengineering recommendations
that were being implemented, or other unrelated factors. In a prior overview report on Collection
Reengineering, we advised management of the potential for this condition to
exist.
We
recommended the Commissioner, SB/SE Division, implement a process to ensure all
future reengineering and process improvement teams develop a means to measure
the impact of any recommendations.
Management’s Response: SB/SE Division
management agreed with our recommendation and plans to take corrective
action. They will work with the SB/SE Division
Research function to develop a methodology to quantify the attainment of
benefits for future initiatives. Once
they have the methodology, they will weigh the costs against the potential
benefits of the knowledge or information before making a decision to pursue the
evaluation. Management’s complete
response to the draft report is included as Appendix V.
Copies of this
report are also being sent to IRS managers affected by the report
recommendation. Please contact me at
(202) 622-6510 if you have questions or Philip Shropshire, Acting Assistant
Inspector General for Audit (Small Business and Corporate Programs), at (215)
516-2341.
Impacts of
the Revised Risk Level Case Selection Criteria Are Not Adequately Measured
Appendix
I – Detailed Objective, Scope, and Methodology
Appendix
II – Major Contributors to This Report
Appendix
III – Report Distribution List
Appendix V –
Management’s Response to the Draft Report
The Small Business/Self-Employed (SB/SE) Division Collection function is responsible for promptly collecting the proper amount of Federal tax due from taxpayers. This includes securing tax returns that are not filed to determine the tax due. Between Fiscal Years (FY) 1996 and 2001, the Internal Revenue Service (IRS) Collection function’s business results declined, due in part to a 36 percent decrease in revenue officer staffing during the period. For example, the number of accounts closed decreased and the number of accounts not being worked increased. In an effort to improve operations, the Commissioner, SB/SE Division, established teams to review Collection function operations and suggest methods for improvement. These teams were commonly referred to as Collection Reengineering teams.
The initial Collection Reengineering team, called the Quick Hits Reengineering Team, was convened to identify high-impact, near-term opportunities to improve Collection function business results. Any recommendations from the Team were to be compatible with other modernization efforts, not require significant system changes, and be able to be implemented within 1 year. The Team concentrated on incremental process improvements that could be accomplished in a relatively short time period without redesigning the overall process.
One recommendation
from the Quick Hits Reengineering Team, which was implemented early in Calendar
Year 2002 and is the subject of our review, involved changes to the risk level
criteria used to identify workload for the Collection Field function (CFf). This new criteria places more emphasis on
assigning trust fund cases and assigning them earlier in the balance due
stream. Under the previous case
identification criteria, tax periods accumulated until they reached a
collective dollar threshold before being assigned to the CFf. Under the revised criteria, dollar threshold
is determined by tax period rather than by taxpayer entity (the total of all
periods due). In theory, this would
allow trust fund cases to be assigned sooner, and fewer tax periods would
accumulate (pyramid) before CFf intervention.
Addressing employment tax noncompliance is an operational priority
included in the SB/SE Division’s Business Action Plan.
This review was performed through the SB/SE Division National Headquarters Offices of Payment Compliance and Centralized Workload Selection and Delivery located in New Carrollton, Maryland, during the period October 2003 through June 2004. The audit was conducted in accordance with Government Auditing Standards. Detailed information on our audit objective, scope, and methodology is presented in Appendix I. Major contributors to the report are listed in Appendix II.
Our review of a sample of closed trust fund cases and analysis of the Collection Activity Reports (CAR) both showed improved business results since FY 2001 (our baseline period and the last full fiscal year prior to implementation of the new criteria) for the CFf. The number of trust fund Taxpayer Delinquent Accounts (TDA) closed increased by 42 percent. In addition, the percentage of trust fund TDAs closed as fully paid and the amount collected increased, while the percentage of those closed as not collectible decreased. While there were overall productivity improvements after implementation of the new risk level criteria for inventory selection, other reengineering recommendations were being implemented at the same time that could also have affected these productivity measures.
The Quick Hits Reengineering Team projected that the change in inventory selection criteria would result in an increase in case dispositions of between 4 and 25 percent and an increase in revenue of between 19 and 42 percent. It was expected that the results would occur in 1 to 1½ years after implementation of the recommendation. An analysis of the CARs for all types of delinquent accounts showed that CFf dispositions increased by 19 percent and the amount collected increased by 17 percent from FYs 2001 to 2003 (based on accomplishments for the last half of each fiscal year).
Our review of the CARs for trust fund cases overall (those closed by any Collection function) showed that the total number of trust fund TDA closures decreased slightly (by only 4 percent), but the number of accounts closed as fully paid and the amount collected both increased since FY 2001.
In addition, it does not appear that the shift in CFf workload to trust fund cases adversely affected overall TDA closures (for all types of taxes, including individual taxpayers). The TDAs closed by all components of the Collection function increased by 1 percent, accounts closed as fully paid increased by 9 percent, and the amount collected increased by 26 percent.
Case review of the TDAs closed by the CFf showed some improved results
We reviewed a sample of 100 closed TDAs for Employer’s Quarterly Federal Tax Returns (Forms 941) for each of 2 periods (1 period before and 1 period after the new risk level criteria were implemented) and determined that business results improved after the new criteria were implemented.
Our case review showed indications of performance improvements. Although not all were quantified, the following performance indicators were areas in which the Quick Hits Reengineering Team expected improvements. The figures may not be representative of all cases since our sample size was small (i.e., an abnormality could have a large impact on the figures due to the small sample size). However, results from our CAR analysis included in the following section also show trends in similar directions.
These improvements appear to indicate that the change in risk level criteria for inventory selection was successful.
Our case review showed mixed results for the expected improved performance results for pyramiding. While the average number of additional tax periods that became delinquent while a case was assigned in the CFf decreased by 17 percent, the number of taxpayers that pyramided increased by 8 percent. The decrease in the average number of additional tax periods indicates that the initiative was somewhat successful.
Analysis of the CARs showed some improved results in the CFf during FY 2002 but a decline in some results during FY 2003
We reviewed the CARs for the periods April through September 2001 (baseline period prior to implementation of the new criteria) and 2002 (after the new criteria were implemented) to determine the changes in productivity indicators. We also reviewed the CARs for the period April through September 2003 to get more current information. While there was overall improvement in business results for the period, some results showed a decline during FY 2003.
Figure 1 shows the year-to-year change for some business results for the CFf TDAs and Taxpayer Delinquency Investigations (TDI) from FYs 2001 to 2003 and the overall change from FYs 2001 to 2003, based on our analysis of the CARs. See Appendix IV for additional details and for productivity indicators on CFf trust fund cases.
Figure 1: Key SB/SE Division Productivity Indicators
for the CFf TDAs and TDIs, Shown As a Percentage Increase or Decrease
Figure
1 was removed due to its size. To see Figure
1, please go to the Adobe PDF version of the report on the TIGTA Public Web
Page.
Figure 1 shows that some productivity indicators for the TDAs improved each year from FYs 2001 to 2003, while other indicators showed a decline from FYs 2002 to 2003. For example, the percentage change for Total Cases Closed and Total Amount Collected improved each year. Other indicators such as Account Full Paid, Account Currently Not Collectible, Average Weeks in Function, and Average Tax Periods per Taxpayer (an indication of pyramiding) showed an improvement from FYs 2001 to 2002 but then declined during FY 2003. Likewise, productivity indicators for the TDIs, such as the percentage of cases closed with a Return Secured or Unable to Locate, showed an initial improvement, but then declined during FY 2003.
As shown in Figure 1, there were overall improvements in key SB/SE Division productivity indicators from FYs 2001 to 2003, but many of the improvements were entirely attributable to FY 2002. Some indicators showed a decline during FY 2003. It is unclear what caused these declines since the impacts of the revised risk level case selection criteria are not effectively measured. Without an effective process to measure the impact of individual reengineering projects, it is not known whether the overall improvements from FY 2001 and declines in FY 2003 were attributable to the risk level criteria change, additional reengineering recommendations that were being implemented, or other unrelated factors.
Other reengineering efforts were being implemented during FY 2003 that could also have affected some of the same productivity indicators. We recently reviewed one of these other reengineering efforts that involved the use of a model to predict the collectibility potential of incoming cases. In that review, we also reported that SB/SE Division management did not have an adequate method to measure the results of the Collection Reengineering team’s recommendation. Without an adequate method to measure the results of the recommendation, it is not known whether this reengineering effort contributed to some of the decline in productivity indicators noted above or if the decline was due to other factors.
In an overview report on Collection Reengineering, we advised that management would have difficulty determining whether any productivity gains were attributable uniquely and directly to recommendations from the various reengineering efforts, the increase in resources spent on collection activities, or other factors. It would also be possible that the overall results show an improvement in the measures, but one or more of the recommendations could actually have a negative impact. The decline in some productivity indicators during FY 2003 when other reengineering recommendations were being implemented demonstrates that this potential exists.
The Government Accountability Office’s (formerly the General Accounting Office) reengineering and process improvement guidelines provide for performance measurement of a new process to determine if it is achieving the desired results. In addition, agencies should use performance measurement as a feedback loop for continuously improving the process.
As stated previously, the Quick Hits Reengineering Team established expected impacts from implementation of the revised criteria but did not develop a means to measure them. The Collection Reengineering executive advised that a request was submitted for a study to measure the impact of the various reengineering efforts. However, the study was not funded. The following efforts, also requested by management, were attempts to measure the impact of the revised collection case selection criteria with existing resources:
· During FY 2003, the Office of Program Evaluation and Risk Analysis (OPERA) started to review the revised risk level impact. In an interim report showing the results of the CAR analysis, the OPERA concluded that Quick Hits Collection Reengineering resulted in some positive outcomes. It is currently conducting additional research in the CFf and intends to perform a review of case information from inventory systems to provide a more detailed review of the impact of the reengineering effort. The project will be closed with issuance of the final report.
While these efforts provide some indication of the impact of
the revised selection criteria, they do not isolate the impact of the revised
risk level criteria from other reengineering recommendations being implemented
almost simultaneously or provide for continuous feedback of the results. The efforts were primarily an analysis of the
CARs. In addition to not providing
results on individual reengineering efforts, the CARs do not include effective
measures for all of the expected benefits from the efforts. For example, information on pyramiding or the
amount of time a taxpayer’s account has been in balance due status, which were
expected to improve after the revised inventory selection criteria was
implemented, are not available through the CAR analysis. Limitations such as these inhibit
management’s ability to measure the success of reengineering efforts, thus hampering
decision‑making ability when considering the impact of the recommendations.
1. The Commissioner, SB/SE Division, should implement a process to ensure future reengineering and process improvement teams develop a means to measure the impact and report on the attainment of expected benefits of the recommendations being implemented. All proposed benefits of the recommendations, regardless of whether they are specifically quantified by the team, should be included in the measurement process. The process should include a feedback loop, providing effective information on the results for management’s use to adjust the recommendations for increased impact.
Management’s Response: SB/SE Division management will work with the SB/SE Division Research function to develop a methodology to quantify the attainment of benefits for future initiatives. Once they have the methodology, they will weigh the costs against the potential benefits of the knowledge or information before making a decision to pursue the evaluation.
Appendix I
Detailed
Objective, Scope, and Methodology
The overall objective was to determine whether new Collection Field function (CFf) case selection criteria helped the Small Business/Self-Employed (SB/SE) Division meet its goal of effectively resolving delinquent trust fund accounts. To accomplish this objective, we:
Based on the items being measured (see step I.B.), we used random sampling for variables, and the sample sizes were calculated based on universes of 112,475 tax periods for the baseline sample period and 205,091 tax periods for the new sample period. Based on a confidence level of 95 percent and variability of 75 percent, the sample sizes were calculated to be 858 and 861 tax periods, respectively. After reviewing the first 50 cases in each category, we recalculated the sample sizes. The updated sample sizes for the items being measured varied from 943 to 12,078 tax periods for the current period cases. We determined it would not be possible to review that volume of cases. Therefore, we decided to review another 50 cases in each category then stop, as long as the trends identified from the first 50 cases continued for the subsequent 50 cases. Our final sample size was 100 cases from each universe.
1.
The type of case closure (fully
paid, currently not collectible, installment agreement, bankruptcy, etc.).
2.
The number of days the case was
in balance due status until closure.
3.
Whether additional tax periods
became delinquent for the Taxpayer Delinquent Accounts (TDA) or the Taxpayer
Delinquency Investigations (TDI) after the initial case had been assigned to a revenue
officer (RO) for more than 90 days.
4.
Dollars collected while the case
was in TDA status.
Appendix II
Major Contributors to This
Report
Philip
Shropshire, Acting Assistant Inspector General for Audit (Small Business and
Corporate Programs)
Parker F. Pearson, Director
Amy L. Coleman, Audit
Manager
Joseph P. Snyder, Lead
Auditor
Timothy A. Chriest,
Senior Auditor
Appendix III
Commissioner C
Office of the Commissioner – Attn: Chief of Staff C
Deputy Commissioner for Services and Enforcement SE
Deputy Commissioner, Small Business/Self-Employed Division SE:S
Acting Director, Compliance, Small Business/Self-Employed Division SE:S:C
Acting Deputy Director, Compliance Policy, Small Business/Self-Employed Division SE:S:C:CP
Director, Strategy, Research, and Performance Management, Small Business/Self-Employed Division SE:S:SRM:SR
Director, Centralized Workload Selection and Delivery, Small Business/Self-Employed Division SE:S:C:CP:CW
Director, Filing and Campus Compliance, Small Business/Self-Employed Division SE:S:C:CP:FC
Director, Payment Compliance, Small Business/Self-Employed Division SE:S:C:CP:PC
Chief Counsel CC
National Taxpayer Advocate TA
Director, Office of Legislative Affairs CL:LA
Director, Office of Program Evaluation and Risk
Analysis RAS:O
Office of Management Controls OS:CFO:AR:M
Audit Liaison: Commissioner, Small Business/Self-Employed
Division SE:S
Appendix IV
The chart was removed due to its
size. To see the chart, please go to the
Adobe PDF version of the report on the TIGTA Public Web Page.
Appendix V
Management’s
Response to the Draft Report
The response was
removed due to its size. To see the
response, please go to the Adobe PDF version of the report on the TIGTA Public
Web Page.