Executive Summary

Objective and Scope

Background

Results

Conclusion

Appendix I – Detailed Objective, Scope, and Methodology

Appendix II – Major Contributors to This Report

Appendix III – Report Distribution List

Appendix IV – Management’s Response to the Draft Report

Executive Summary

The Collection Partnership Redesign Team implemented the Mentor and Monitor (M&M) Prototype to help small business owners understand their tax responsibilities. The prototype was designed to inform new employers of education and assistance opportunities, assist them in complying with employment tax requirements, monitor their tax compliance, and identify and help resolve delinquencies in paying taxes and filing returns. The M&M Prototype project manager has estimated that, if fully implemented, the M&M Prototype would help 250,000 new employers annually at a cost of $51 million, or about $200 per employer.

The overall objective of this audit was to determine whether the M&M Prototype would help the Internal Revenue Service (IRS) improve service to each taxpayer by providing prompt, professional, and helpful treatment to new businesses.

The M&M Prototype appears to be helping certain new business taxpayers comply with their employment tax responsibilities. Small Business/Self-Employed (SB/SE) Division management used an innovative approach to determine the best method to help new employers comply with their tax responsibilities. The prototype should help reduce taxpayer burden, uncertainty and errors by clarifying tax law requirements, resolving issues early in the process, and offering effective communication and education assistance. However, we have concerns that there may not be sufficient information to evaluate the effectiveness of a nationwide rollout of this prototype.

SB/SE Division management used an innovative approach to determine the best method to help new employers comply with their employment tax responsibilities. They used a special mail-out with materials designed to assist employers in meeting their employment tax responsibilities, profiled employers to identify those most likely to need assistance, and surveyed employers to obtain feedback concerning the value of the prototype. They also partnered with the Small Business Administration (SBA) to secure 10,000 of the SBA’s Programs and Services Brochures. In addition, a database was established for tracking case progression, verifying data input, and monitoring telephone contacts. Conclusive results for the prototype are expected to be known in December 2001.

Management did not apply random sampling principles in selecting the samples for the M&M Prototype. Therefore, statistical inferences may not be made about the effectiveness of the prototype and the feasibility of a national rollout. However, the planned analysis of 20,000 new employers could provide valuable information about the behavior of small business taxpayers that would aid taxpayer education efforts.

The M&M Prototype is currently being developed as a program to help small business taxpayers. However, the mentoring aspect of the prototype may not be practical because it is very labor intensive, would require extensive hiring or reallocation of resources, and would help only a limited number of taxpayers. The M&M Prototype project manager estimated that if the prototype were implemented nationwide it would require approximately 1,100 staff years.

The IRS Headquarters’ National Public Liaison and Office of Tax Administration Coordination provided oversight during the early development of the M&M Prototype. However, they ceased their oversight relatively early, and there has been no oversight during most of the process. Greater management involvement may have identified the sampling problems in the initial planning stages, evaluated the feasibility of a national rollout earlier, and considered incorporating the prototype with other taxpayer education programs.

We recommended that SB/SE Division management ensure that statistically valid sampling techniques be employed where projections will be used and samples will be compared, consider incorporating the M&M Prototype with other taxpayer education programs, and provide oversight for the duration of the prototype.

Management’s Response: The Commissioner, SB/SE Division, agreed with two of the three recommendations presented in the report. The SB/SE Division will consider providing one-on-one taxpayer education service but only in those cases or markets involving high risks of non-compliance. The Commissioner stated that they are exploring the various channels and will include lessons learned and some of the fundamental principles from the prototype as they continue to define the role of the Taxpayer Education and Communication (TEC) organization. In addition, the Division will follow up with a business case analysis of the compliance results that includes the appropriate level of management oversight. The SB/SE Division’s Director, TEC, will be the recipient of the business case analysis.

The Commissioner, SB/SE Division, had a different opinion about the sampling methodology used in the prototype. He believed that the judgment sample used would provide sufficient information to evaluate the effectiveness of the prototype. The IRS Office of Research agreed that the judgment sample was an acceptable method that provided a number of benefits; however, they also stated that a valid statistical sampling technique would have been the preferred method.

Auditor’s Comment: While we believe that the SB/SE Division’s judgment sample will provide valuable information to evaluate the prototype, we concur with the Office of Research that a valid statistical sample would have been preferred. We continue to believe that the SB/SE Division cannot make statistical projections from its sample.

Objective and Scope

The overall objective of this audit was to determine whether the Mentor and Monitor (M&M) Prototype would help the Internal Revenue Service (IRS) improve service to each taxpayer by providing prompt, professional, and helpful treatment to new businesses. To achieve our objective, we interviewed appropriate managers in the National Headquarters offices of the National Public Liaison (NPL) and Office of Tax Administration Coordination (OTAC). We also interviewed managers in the Small Business/Self- Employed (SB/SE) Division Offices of the Director, Compliance, and the Director, Taxpayer Education and Communication (TEC). We determined the level of oversight by management, the validity of the sampling methodology, and the practicality of the prototype.

The audit work was performed between November 2000 and February 2001 at the National Headquarters, the SB/SE Division Compliance function’s North Texas Area Office, and the Brookhaven IRS Campus. We performed the audit in accordance with Government Auditing Standards.

The detailed objective, scope, and methodology are presented in Appendix I. Major contributors to this report are shown in Appendix II.

Background

The IRS Commissioner established the Taxpayer Treatment and Service Improvement (TSI) Program in November 1997 to plan, coordinate, and manage hundreds of commitments for service improvements, as well as proposals for new commitments. The TSI Program approved the implementation of the M&M Prototype to assist new employers.

This prototype is designed to help achieve the IRS’ goal of improving service to each taxpayer. In this case, the focus is on new businesses that will have employment tax deposits due. Also, this is part of the IRS’ near-term improvement efforts to identify systemic causes of taxpayers’ account problems and develop solutions.

SB/SE Division statistics show that, for 1997, there were 885,416 new employer firms. The project manager estimated that the M&M Prototype, if implemented nationwide, would help 250,000 of these new employers. The project manager also estimated that the cost to roll out this prototype would be approximately $51 million annually, or about $200 per employer.

The M&M Prototype began operations in August 1999. The IRS took a sample of approximately 5,600 new employers for its test group. These test cases were screened to determine which employers were eligible to be enrolled in the prototype. As of January 2001, approximately 620 employers were enrolled.

The Collection Partnership Redesign Team implemented the M&M Prototype to help small business owners understand their tax responsibilities. Through this prototype, the IRS plans to:

• Inform new employers of education and assistance opportunities at the time they receive their Employer Identification Numbers (EIN).
• Contact a test group of new employers by telephone to assist them in complying with employment tax laws.
• Monitor their tax compliance.
• Intervene by telephone to resolve identified delinquencies in making federal tax deposits and filing employment tax returns.

Results

The M&M Prototype appears to be helping certain new business taxpayers comply with their federal employment tax responsibilities. SB/SE Division management used an innovative approach to determine the best method to help new employers comply with their tax responsibilities. The prototype should help reduce taxpayer burden, uncertainty, and errors by clarifying tax law requirements, resolving issues early in the process, and offering effective communication and education assistance. However, we have concerns that there may not be sufficient information to evaluate the effectiveness of a nationwide rollout of this prototype.

The Mentor and Monitor Prototype Could Provide Valuable Information About the Behavior of Small Business Taxpayers

SB/SE Division management used an innovative approach to determine the best method to help new employers comply with their employment tax responsibilities. This approach used a special mail-out with materials designed to assist employers in meeting their employment tax responsibilities, profiled employers to identify those most likely to need assistance, and surveyed employers to obtain feedback concerning the value of the prototype. They also partnered with the Small Business Administration (SBA) to secure 10,000 of the SBA’s Programs and Services Brochures.

In addition, management did an effective job of establishing the test team and setting up the Management Information System (MIS). The MIS contains a historical audit trail of all telephone contacts between the IRS tax examiners and small business taxpayers. The MIS database allows for tracking case progression, verifying data input, and monitoring telephone contacts. The database also contains a record of undelivered mail and a series of useful managerial reports.

This approach could provide valuable information about the behavior of small business taxpayers that would aid taxpayer education efforts. Conclusive results for the prototype are expected to be known in December 2001.

The Sampling Methodology May Not Permit Valid Statistical Inferences About the Effectiveness of the Prototype

The techniques used to select the samples may not allow statistical inferences about the effectiveness of the prototype and the feasibility of a national rollout. However, the planned analysis of new employers could provide valuable information about the behavior of small business taxpayers that would aid taxpayer education efforts.

The two basic sampling methods are random and judgment sampling. Unless the sample is selected randomly, that is, by reliance on the laws of probability, most of the advantages of using sampling are lost. Furthermore, judgment sampling involves greater risk than random sampling when making decisions about the characteristics of the universe.

In order to draw valid statistical inferences, the sample must be selected randomly. However, management did not apply the following random sampling principles in selecting the samples for the M&M Prototype:

• The universe has to be known.
• Every item in the universe must have an equal chance of being selected as part of the sample.
• When comparing different sample groups, the comparison must be performed on similarly treated groups.

Without a statistically valid sample, it will be difficult to draw valid statistical inferences about the effectiveness of the prototype. In addition, sample results should not be projected nationwide to the universe.

The universe must be clearly defined. In selecting the sample for this prototype, it was essential to identify the universe, including the geographic location of the taxpayers and the time period to be covered by the prototype. In our discussions and analyses, we found that the geographic location was well defined, but there was no evidence that the time period or universe were identified.

The IRS’ Research Division assisted SB/SE Division management in determining the test and control group sample sizes, method of selection, and method for determining employers in need of mentoring and monitoring. However, we were unable to verify the sampling methodology because there was no documentation showing how the samples were selected.

The sampling process must give each item in the universe an equal chance of being selected. Unless each item in the sample is randomly selected, the sample will not accurately reflect the characteristics of the universe.

The test and control samples were selected from over 26,000 new employers that applied for an EIN from the Austin IRS Campus. There was no known universe at the time of selection. The first 10,000 new employers that met M&M-established criteria were selected for the test group. The next 10,000 cases were selected for the control group.

The test and control sample groups were not subjected to the same criteria. The test cases were run through 14 different criteria/filters that reduced the total number of taxpayers (test sample) by 88 percent. Some of the filters applied were: taxpayer already had professional tax advice, taxpayer declined assistance, and taxpayer already owed taxes and was in collection status. Removing these cases from the test sample could have both negative and positive effects on the results of the prototype. These filters were not applied to the control group.

1. In the future, the Commissioner, SB/SE Division, should ensure that statistically valid sampling techniques are employed where projections will be used and samples will be compared.

Management’s Response: The Commissioner, SB/SE Division, had a different opinion about the sampling methodology used in the prototype. He believed that the judgment sample used would provide sufficient information to evaluate the effectiveness of the prototype. The IRS Office of Research agreed that the judgment sample was an acceptable method that provided a number of benefits; however, they also stated that a valid statistical sampling technique would have been the preferred method.

Auditor’s Comment: While we believe that the SB/SE Division’s judgment sample will provide valuable information to evaluate the prototype, we concur with the Office of Research that a valid statistical sample would have been preferred. We continue to believe that the SB/SE Division cannot make statistical projections from its sample.

Incorporating the Prototype With Other Taxpayer Education Programs May Be More Cost Effective by Providing Greater Benefits to More Small Business Taxpayers

To be successful, the M&M Prototype needs to demonstrate its cost effectiveness. This includes issues such as the cost of the program when fully implemented, the types of benefits provided, and the number of taxpayers served.

The M&M Prototype was designed to help small business owners understand their federal payroll tax deposits and tax return filing responsibilities. However, the mentoring aspect of the prototype may not be practical because it is very labor intensive and would require extensive hiring or reallocation of resources from other IRS functions. The M&M Prototype project manager estimated that if the prototype were implemented nationwide, approximately 1,100 staff years would be required.

Starting in Fiscal Year 2001, the TEC function is to assume ownership of all initiatives intended to assist SB/SE Division taxpayers. After this transition, the TEC function should have several initiatives that will help small business owners comply with their federal tax responsibilities. However, the SB/SE Division’s TEC function is authorized only approximately 1,200 staff years to accomplish its entire taxpayer education and communication program.

Because the TEC function has not yet taken ownership of the prototype and management oversight has been limited, key decisions regarding a national rollout have yet to be made.

Without incorporating the M&M Prototype with other TEC function programs, the prototype may:

• Be too costly to roll out nationwide.
• Help only a limited number of new employers.
• Deal with employment taxes alone, which are only part of the total requirements that new business owners must comply with to meet their tax filing obligations.

1. The SB/SE Division’s Director, TEC, should consider incorporating the M&M Prototype with other taxpayer education programs.

Management’s Response: The Commissioner, SB/SE Division, agreed with this recommendation. The Division will consider providing one-on-one taxpayer education service but only in those cases or markets involving high risks of non-compliance. It is exploring the various channels and will include lessons learned and some of the fundamental principles from the prototype as it continues to define the role of the TEC organization.

Greater Management Oversight Would Help Ensure That the Prototype Will Achieve Its Goals and Objectives

The NPL and OTAC provided oversight during the early development of the M&M Prototype. However, they stopped providing oversight relatively early in the prototype, and there has been no oversight provided during most of the process. Oversight by management is needed throughout all project phases to ensure that the goals and objectives of the project are achieved within the specified budget and schedule.

Discussions with management indicated that the NPL no longer provided oversight after the SB/SE Division stood up in October 2000. We were also informed that the OTAC’s oversight was limited to a few taxpayer education initiatives because of staffing shortages.

In addition, the M&M Prototype was never allocated a budget with which to operate, and management was not tracking expenditures on a regular basis. Further, management did not routinely update estimated completion dates in the project management plan.

With additional oversight, management may have identified the sampling problems in the initial planning stages, evaluated the feasibility of a national rollout, and considered incorporating the prototype with other taxpayer education programs.

Recommendation

2. The SB/SE Division’s Director, TEC, should provide oversight for the duration of the prototype.

Management’s Response: The Commissioner, SB/SE Division, agreed with this recommendation. The prototype was completed and the Division will follow up with a business case analysis of the compliance results that includes the appropriate level of management oversight. The SB/SE Division’s Director, TEC, will be the recipient of the business case analysis.

Conclusion

The M&M Prototype appears to be helping certain new business taxpayers comply with their federal employment tax responsibilities. The prototype should reduce taxpayer burden, uncertainty, and errors by clarifying tax law requirements, resolving issues early in the process, and offering effective communication and education assistance. However, concerns with the prototype’s sampling methodology, cost effectiveness, and level of oversight raise questions about nationwide implementation.

Detailed Objective, Scope, and Methodology

The overall objective of this audit was to determine whether the Mentor and Monitor (M&M) Prototype would help the Internal Revenue Service (IRS) improve service to each taxpayer by providing prompt, professional, and helpful treatment to new business employers.

1. To determine whether the M&M Prototype will achieve its goals and objectives, we:
1. Reviewed the original goals, expected results, and objectives used to obtain approval for the prototype from the Executive Steering Committee to determine whether the prototype was on target with goals and objectives.
2. Evaluated the effectiveness of oversight provided by the Office of Tax Administration Coordination, National Public Liaison, and Collection Partnership Redesign Team.
3. Evaluated whether the taxpayer education portion of the prototype sufficiently covered taxpayers’ needs.
1. Determined if partnerships with the Small Business Administration and state and local agencies were considered and explored.
2. Determined if the education portion of the prototype was adequately designed to provide sufficient education for these newly created small businesses.
4. Evaluated whether the Management Information System was:
1. In place.
2. Adequate to keep management apprised of the status of the prototype.
3. Updated regularly.
5. Determined the effect that the IRS News Release (IR-2000-83) concerning simplification of the tax deposit rules for small business taxpayers would have on the prototype.
2. To determine whether the sampling methodology used to select the "test" and "control" databases was statistically sound and whether the sample was inclusive enough to project the test results nationwide, we:
1. Interviewed the project manager for the M&M Prototype to determine the methodology used to select the samples.
2. Consulted with the Chief, Statistical Support Section, Office of the Director, Research Analysis and Statistics of Income, to validate our findings.

Major Contributors to This Report

Gordon C. Milbourn III, Assistant Inspector General for Audit (Small Business and Corporate Programs)

Richard J. Dagliolo, Director

Robert K. Irish, Audit Manager

Philip W. Peyser, Senior Auditor

Paul R. Baker, Auditor

Stephen A. Wybaillie, Auditor

Report Distribution List

Commissioner N:C

Deputy Commissioner N:DC

Chief Counsel CC

Deputy Commissioner, Small Business/Self-Employed Division S

Director, Communications, Small Business/Self-Employed Division S:COM

Director, Compliance, Small Business/Self-Employed Division S:C

Director, Filing and Payment Compliance, Small Business/Self-Employed Division S:C:CP:FP

Director, Internal External Stakeholders, Small Business/Self-Employed Division S:C:CP:I

Director, Office of Program Evaluation and Risk Analysis N:ADC:R:O

Director, Taxpayer Education and Communication, Small Business/Self-Employed Division S:T

Office of Management Controls N:CFO:F:M

National Taxpayer Advocate TA

Director, Legislative Affairs CL:LA

Project Leader, Collection Reengineering, Small Business/Self-Employed Division S:C:CP:FP:CR

Audit Liaisons:

Small Business/Self-Employed Division, TEC - Liaison and Transition S:T

Management’s Response to the Draft Report

The response was removed due to its size. To see the response, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.