The Revised Collection Case Selection Criteria That Expedites Trust Fund Workload to the Field Appears Effective

 

September 2004

 

Reference Number:  2004-30-173

 

 

This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.

 

September 17, 2004

 

 

MEMORANDUM FOR COMMISSIONER, SMALL BUSINESS/SELF-EMPLOYED DIVISION

 

FROM:     Gordon C. Milbourn III /s/ Gordon C. Milbourn III

                 Acting Deputy Inspector General for Audit

 

SUBJECT:     Final Audit Report - The Revised Collection Case Selection Criteria That Expedites Trust Fund Workload to the Field Appears Effective (Audit # 200330037)

 

This report presents the results of our review of the Internal Revenue Service’s (IRS) collection case selection criteria.  The overall objective of this review was to determine whether new Collection Field function (CFf) case selection criteria helped the Small Business/Self-Employed (SB/SE) Division meet its goal of effectively resolving delinquent trust fund accounts.

Between Fiscal Years (FY) 1996 and 2001, the IRS Collection function’s business results declined, due in part to the 36 percent decrease in revenue officer staffing during the period.  For example, the number of balance due tax accounts closed decreased and the number of accounts not being worked increased. 

In an effort to improve operations, the Commissioner, SB/SE Division, established teams to review Collection function operations and suggest methods for improvement.  One team recommended changing the risk levels used to identify workload for the CFf to place more emphasis on assigning trust fund cases and assigning the cases earlier in the balance due stream.  Addressing employment tax noncompliance is an operational priority included in the SB/SE Division’s Business Action Plan.

In summary, our review of a sample of closed trust fund cases and analysis of the Collection Activity Reports (CAR) both showed improved business results since FY 2001 (our baseline period and the last full fiscal year prior to implementation of the new criteria) for the CFf.  The number of trust fund cases closed increased by 42 percent, and the number of additional tax periods that accumulated (pyramided) while cases were assigned in the CFf decreased by 17 percent.  In addition, it does not appear that this shift in CFf workload caused an adverse impact on overall Taxpayer Delinquent Account (TDA) closures by the Collection function.  The number of TDAs closed, the percentage closed as fully paid, and the amount collected all increased.  These improvements appear to indicate that the change in risk level criteria for inventory selection was successful.

However, expected impacts of the revised risk level case selection criteria were not adequately measured.  Efforts to measure the impact of the revised risk level case selection criteria primarily involved an analysis of the CARs.  These reports do not provide results on individual reengineering efforts or effective measures for all of the expected benefits from the efforts, such as a decrease in pyramiding.  Limitations such as these inhibit management’s ability to measure the success of reengineering efforts, thus hampering decision-making ability when considering the impact of the implementation of recommendations.

While there was an overall improvement in business results from FYs 2001 to 2003, a year-to-year analysis showed that some results declined from FYs 2002 to 2003.  During FY 2003, other reengineering efforts were being implemented that could also have affected some of the same productivity indicators as the revised risk level case selection criteria.  Without an effective process to measure the impact of individual reengineering projects, it is not known whether the overall improvements from FY 2001 and declines during FY 2003 were attributable to the risk level criteria change, additional reengineering recommendations that were being implemented, or other unrelated factors.  In a prior overview report on Collection Reengineering, we advised management of the potential for this condition to exist.

We recommended the Commissioner, SB/SE Division, implement a process to ensure all future reengineering and process improvement teams develop a means to measure the impact of any recommendations.

Management’s Response:  SB/SE Division management agreed with our recommendation and plans to take corrective action.  They will work with the SB/SE Division Research function to develop a methodology to quantify the attainment of benefits for future initiatives.  Once they have the methodology, they will weigh the costs against the potential benefits of the knowledge or information before making a decision to pursue the evaluation.  Management’s complete response to the draft report is included as Appendix V.

Copies of this report are also being sent to IRS managers affected by the report recommendation.  Please contact me at (202) 622-6510 if you have questions or Philip Shropshire, Acting Assistant Inspector General for Audit (Small Business and Corporate Programs), at (215) 516-2341.

 

Table of Contents

Background

Productivity Results Showed Varying Degrees of Improvement After Implementation of the Revised Risk Level Criteria

Impacts of the Revised Risk Level Case Selection Criteria Are Not Adequately Measured

Recommendation 1:

Appendix I – Detailed Objective, Scope, and Methodology

Appendix II – Major Contributors to This Report

Appendix III – Report Distribution List

Appendix IV – Productivity Indicators for the Taxpayer Delinquent Accounts and the Taxpayer Delinquency Investigations in the Collection Field Function

Appendix V – Management’s Response to the Draft Report

 

Background

The Small Business/Self-Employed (SB/SE) Division Collection function is responsible for promptly collecting the proper amount of Federal tax due from taxpayers.  This includes securing tax returns that are not filed to determine the tax due.  Between Fiscal Years (FY) 1996 and 2001, the Internal Revenue Service (IRS) Collection function’s business results declined, due in part to a 36 percent decrease in revenue officer staffing during the period.  For example, the number of accounts closed decreased and the number of accounts not being worked increased.  In an effort to improve operations, the Commissioner, SB/SE Division, established teams to review Collection function operations and suggest methods for improvement.  These teams were commonly referred to as Collection Reengineering teams.

The initial Collection Reengineering team, called the Quick Hits Reengineering Team, was convened to identify high-impact, near-term opportunities to improve Collection function business results.  Any recommendations from the Team were to be compatible with other modernization efforts, not require significant system changes, and be able to be implemented within 1 year.  The Team concentrated on incremental process improvements that could be accomplished in a relatively short time period without redesigning the overall process.

One recommendation from the Quick Hits Reengineering Team, which was implemented early in Calendar Year 2002 and is the subject of our review, involved changes to the risk level criteria used to identify workload for the Collection Field function (CFf).  This new criteria places more emphasis on assigning trust fund cases and assigning them earlier in the balance due stream.  Under the previous case identification criteria, tax periods accumulated until they reached a collective dollar threshold before being assigned to the CFf.  Under the revised criteria, dollar threshold is determined by tax period rather than by taxpayer entity (the total of all periods due).  In theory, this would allow trust fund cases to be assigned sooner, and fewer tax periods would accumulate (pyramid) before CFf intervention.  Addressing employment tax noncompliance is an operational priority included in the SB/SE Division’s Business Action Plan.

This review was performed through the SB/SE Division National Headquarters Offices of Payment Compliance and Centralized Workload Selection and Delivery located in New Carrollton, Maryland, during the period October 2003 through June 2004.  The audit was conducted in accordance with Government Auditing Standards.  Detailed information on our audit objective, scope, and methodology is presented in Appendix I.  Major contributors to the report are listed in Appendix II.

Productivity Results Showed Varying Degrees of Improvement After Implementation of the Revised Risk Level Criteria

Our review of a sample of closed trust fund cases and analysis of the Collection Activity Reports (CAR) both showed improved business results since FY 2001 (our baseline period and the last full fiscal year prior to implementation of the new criteria) for the CFf.  The number of trust fund Taxpayer Delinquent Accounts (TDA) closed increased by 42 percent.  In addition, the percentage of trust fund TDAs closed as fully paid and the amount collected increased, while the percentage of those closed as not collectible decreased.  While there were overall productivity improvements after implementation of the new risk level criteria for inventory selection, other reengineering recommendations were being implemented at the same time that could also have affected these productivity measures.

The Quick Hits Reengineering Team projected that the change in inventory selection criteria would result in an increase in case dispositions of between 4 and 25 percent and an increase in revenue of between 19 and 42 percent.  It was expected that the results would occur in 1 to 1½ years after implementation of the recommendation.  An analysis of the CARs for all types of delinquent accounts showed that CFf dispositions increased by 19 percent and the amount collected increased by 17 percent from FYs 2001 to 2003 (based on accomplishments for the last half of each fiscal year).

Our review of the CARs for trust fund cases overall (those closed by any Collection function) showed that the total number of trust fund TDA closures decreased slightly (by only 4 percent), but the number of accounts closed as fully paid and the amount collected both increased since FY 2001.

In addition, it does not appear that the shift in CFf workload to trust fund cases adversely affected overall TDA closures (for all types of taxes, including individual taxpayers).  The TDAs closed by all components of the Collection function increased by 1 percent, accounts closed as fully paid increased by 9 percent, and the amount collected increased by 26 percent.

Case review of the TDAs closed by the CFf showed some improved results

We reviewed a sample of 100 closed TDAs for Employer’s Quarterly Federal Tax Returns (Forms 941) for each of 2 periods (1 period before and 1 period after the new risk level criteria were implemented) and determined that business results improved after the new criteria were implemented.

Our case review showed indications of performance improvements.  Although not all were quantified, the following performance indicators were areas in which the Quick Hits Reengineering Team expected improvements.  The figures may not be representative of all cases since our sample size was small (i.e., an abnormality could have a large impact on the figures due to the small sample size).  However, results from our CAR analysis included in the following section also show trends in similar directions.

These improvements appear to indicate that the change in risk level criteria for inventory selection was successful.

Our case review showed mixed results for the expected improved performance results for pyramiding.  While the average number of additional tax periods that became delinquent while a case was assigned in the CFf decreased by 17 percent, the number of taxpayers that pyramided increased by 8 percent.  The decrease in the average number of additional tax periods indicates that the initiative was somewhat successful.

Analysis of the CARs showed some improved results in the CFf during FY 2002 but a decline in some results during FY 2003

We reviewed the CARs for the periods April through September 2001 (baseline period prior to implementation of the new criteria) and 2002 (after the new criteria were implemented) to determine the changes in productivity indicators.  We also reviewed the CARs for the period April through September 2003 to get more current information.  While there was overall improvement in business results for the period, some results showed a decline during FY 2003.

Figure 1 shows the year-to-year change for some business results for the CFf TDAs and Taxpayer Delinquency Investigations (TDI) from FYs 2001 to 2003 and the overall change from FYs 2001 to 2003, based on our analysis of the CARs.  See Appendix IV for additional details and for productivity indicators on CFf trust fund cases.

Figure 1:  Key SB/SE Division Productivity Indicators for the CFf TDAs and TDIs, Shown As a Percentage Increase or Decrease

Figure 1 was removed due to its size.  To see Figure 1, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

Figure 1 shows that some productivity indicators for the TDAs improved each year from FYs 2001 to 2003, while other indicators showed a decline from FYs 2002 to 2003.  For example, the percentage change for Total Cases Closed and Total Amount Collected improved each year.  Other indicators such as Account Full Paid, Account Currently Not Collectible, Average Weeks in Function, and Average Tax Periods per Taxpayer (an indication of pyramiding) showed an improvement from FYs 2001 to 2002 but then declined during FY 2003.  Likewise, productivity indicators for the TDIs, such as the percentage of cases closed with a Return Secured or Unable to Locate, showed an initial improvement, but then declined during FY 2003.

Impacts of the Revised Risk Level Case Selection Criteria Are Not Adequately Measured

As shown in Figure 1, there were overall improvements in key SB/SE Division productivity indicators from FYs 2001 to 2003, but many of the improvements were entirely attributable to FY 2002.  Some indicators showed a decline during FY 2003.  It is unclear what caused these declines since the impacts of the revised risk level case selection criteria are not effectively measured.  Without an effective process to measure the impact of individual reengineering projects, it is not known whether the overall improvements from FY 2001 and declines in FY 2003 were attributable to the risk level criteria change, additional reengineering recommendations that were being implemented, or other unrelated factors.

Other reengineering efforts were being implemented during FY 2003 that could also have affected some of the same productivity indicators.  We recently reviewed one of these other reengineering efforts that involved the use of a model to predict the collectibility potential of incoming cases.  In that review, we also reported that SB/SE Division management did not have an adequate method to measure the results of the Collection Reengineering team’s recommendation.  Without an adequate method to measure the results of the recommendation, it is not known whether this reengineering effort contributed to some of the decline in productivity indicators noted above or if the decline was due to other factors.

In an overview report on Collection Reengineering, we advised that management would have difficulty determining whether any productivity gains were attributable uniquely and directly to recommendations from the various reengineering efforts, the increase in resources spent on collection activities, or other factors.  It would also be possible that the overall results show an improvement in the measures, but one or more of the recommendations could actually have a negative impact.  The decline in some productivity indicators during FY 2003 when other reengineering recommendations were being implemented demonstrates that this potential exists.

The Government Accountability Office’s (formerly the General Accounting Office) reengineering and process improvement guidelines provide for performance measurement of a new process to determine if it is achieving the desired results.  In addition, agencies should use performance measurement as a feedback loop for continuously improving the process.

As stated previously, the Quick Hits Reengineering Team established expected impacts from implementation of the revised criteria but did not develop a means to measure them.  The Collection Reengineering executive advised that a request was submitted for a study to measure the impact of the various reengineering efforts.  However, the study was not funded.  The following efforts, also requested by management, were attempts to measure the impact of the revised collection case selection criteria with existing resources:

·                  During FY 2003, the Office of Program Evaluation and Risk Analysis (OPERA) started to review the revised risk level impact.  In an interim report showing the results of the CAR analysis, the OPERA concluded that Quick Hits Collection Reengineering resulted in some positive outcomes.  It is currently conducting additional research in the CFf and intends to perform a review of case information from inventory systems to provide a more detailed review of the impact of the reengineering effort.  The project will be closed with issuance of the final report.

While these efforts provide some indication of the impact of the revised selection criteria, they do not isolate the impact of the revised risk level criteria from other reengineering recommendations being implemented almost simultaneously or provide for continuous feedback of the results.  The efforts were primarily an analysis of the CARs.  In addition to not providing results on individual reengineering efforts, the CARs do not include effective measures for all of the expected benefits from the efforts.  For example, information on pyramiding or the amount of time a taxpayer’s account has been in balance due status, which were expected to improve after the revised inventory selection criteria was implemented, are not available through the CAR analysis.  Limitations such as these inhibit management’s ability to measure the success of reengineering efforts, thus hampering decision‑making ability when considering the impact of the recommendations.

Recommendation

1.      The Commissioner, SB/SE Division, should implement a process to ensure future reengineering and process improvement teams develop a means to measure the impact and report on the attainment of expected benefits of the recommendations being implemented.  All proposed benefits of the recommendations, regardless of whether they are specifically quantified by the team, should be included in the measurement process.  The process should include a feedback loop, providing effective information on the results for management’s use to adjust the recommendations for increased impact.

Management’s Response:  SB/SE Division management will work with the SB/SE Division Research function to develop a methodology to quantify the attainment of benefits for future initiatives.  Once they have the methodology, they will weigh the costs against the potential benefits of the knowledge or information before making a decision to pursue the evaluation. 

 

Appendix I

 

Detailed Objective, Scope, and Methodology

 

The overall objective was to determine whether new Collection Field function (CFf) case selection criteria helped the Small Business/Self-Employed (SB/SE) Division meet its goal of effectively resolving delinquent trust fund accounts.  To accomplish this objective, we:

I.                   Determined if cases closed that were identified for assignment under the revised selection criteria resulted in improved business results over the baseline cases.

A.    Identified two random samples of closed cases – one for cases created prior to implementation of the revised case selection criteria (baseline sample) and another for cases created after the criteria were implemented (new sample).

The baseline sample consisted of taxpayer cases for which at least one tax period was created (assigned to the CFf) between April 1, 2001, and September 30, 2001 (inclusive), and the tax periods were closed by September 30, 2002.  We eliminated taxpayer cases purged during the inventory conversion process.  The new sample consisted of taxpayer cases for which at least one tax period was created between April 1, 2002, and September 30, 2002 (inclusive), and the tax periods were closed by September 30, 2003.

Based on the items being measured (see step I.B.), we used random sampling for variables, and the sample sizes were calculated based on universes of 112,475 tax periods for the baseline sample period and 205,091 tax periods for the new sample period.  Based on a confidence level of 95 percent and variability of 75 percent, the sample sizes were calculated to be 858 and 861 tax periods, respectively.  After reviewing the first 50 cases in each category, we recalculated the sample sizes.  The updated sample sizes for the items being measured varied from 943 to 12,078 tax periods for the current period cases.  We determined it would not be possible to review that volume of cases.  Therefore, we decided to review another 50 cases in each category then stop, as long as the trends identified from the first 50 cases continued for the subsequent 50 cases.  Our final sample size was 100 cases from each universe.

B.     Obtained Master File transcripts for the random samples to determine:

1.      The type of case closure (fully paid, currently not collectible, installment agreement, bankruptcy, etc.).

2.      The number of days the case was in balance due status until closure.

3.      Whether additional tax periods became delinquent for the Taxpayer Delinquent Accounts (TDA) or the Taxpayer Delinquency Investigations (TDI) after the initial case had been assigned to a revenue officer (RO) for more than 90 days.

4.      Dollars collected while the case was in TDA status.

C.     Determined if younger cases were being delivered to the field after the new criteria were implemented by stratifying the field cases by tax year and determining the number of tax periods per taxpayer.

D.    Determined the number of hours charged to the case while assigned to the RO.

II.    Determined if SB/SE Division information indicated that changes to business results occurred and if the changes could be attributed to the new case selection criteria.

A.    Determined how the SB/SE Division was measuring the impact of the change in case selection criteria.

B.     Obtained and compared the results being accumulated by internal reviews of the new case selection criteria to the results identified in our review.

C.     Analyzed the Collection Activity Reports 5000-2 and 4 for a baseline and current periods to determine the overall changes in business results.

 

Appendix II

 

Major Contributors to This Report

 

Philip Shropshire, Acting Assistant Inspector General for Audit (Small Business and Corporate Programs)

Parker F. Pearson, Director

Amy L. Coleman, Audit Manager

Joseph P. Snyder, Lead Auditor

Timothy A. Chriest, Senior Auditor

 

Appendix III

 

Report Distribution List

 

Commissioner  C

Office of the Commissioner – Attn:  Chief of Staff  C

Deputy Commissioner for Services and Enforcement  SE

Deputy Commissioner, Small Business/Self-Employed Division  SE:S

Acting Director, Compliance, Small Business/Self-Employed Division  SE:S:C

Acting Deputy Director, Compliance Policy, Small Business/Self-Employed Division  SE:S:C:CP

Director, Strategy, Research, and Performance Management, Small Business/Self-Employed Division  SE:S:SRM:SR

Director, Centralized Workload Selection and Delivery, Small Business/Self-Employed Division  SE:S:C:CP:CW

Director, Filing and Campus Compliance, Small Business/Self-Employed Division  SE:S:C:CP:FC

Director, Payment Compliance, Small Business/Self-Employed Division  SE:S:C:CP:PC

Chief Counsel  CC

National Taxpayer Advocate  TA

Director, Office of Legislative Affairs  CL:LA

Director, Office of Program Evaluation and Risk Analysis  RAS:O

Office of Management Controls  OS:CFO:AR:M

Audit Liaison:  Commissioner, Small Business/Self-Employed Division  SE:S

 

Appendix IV

 

Productivity Indicators for the Taxpayer Delinquent Accounts and the Taxpayer Delinquency Investigations in the Collection Field Function

 

The chart was removed due to its size.  To see the chart, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

 

Appendix V

 

Management’s Response to the Draft Report

 

The response was removed due to its size.  To see the response, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.