GPRA:Criminal Investigation Can Improve Its Performance Measures to Better Account for Its Results

 

 

January 2002

 

Reference Number:2002-10-009

 

 

This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.

 

January 28, 2002

 

 

MEMORANDUM FOR CHIEF, CRIMINAL INVESTIGATION

 

FROM:†††† Pamela J. Gardiner /s/ Pamela J. Gardiner

†††††††††††††††† Deputy Inspector General for Audit

 

SUBJECT:†††† Final Audit Report - GPRA:Criminal Investigation Can Improve Its Performance Measures to Better Account for Its Results (Audit # 200010032)

 

This report presents the results of our audit to evaluate the reliability and relevance of the performance measures used by Criminal Investigation (CI) in assessing its business results and customer satisfaction with CI products.This review is part of the Treasury Inspector General for Tax Administrationís (TIGTA) multi-year strategy to audit Internal Revenue Service (IRS) efforts to comply with the Government Performance and Results Act (GPRA).

In summary, we found that CI measures do not address mission accomplishment.CI developed outcome-neutral measures to report as its business results because of its concern about Regulation 26 CFR 801.As a result, CIís business results measures and its sole GPRA measure of cases initiated are not outcome oriented and do little to quantify the results of CIís activities.Additionally, CI does not have a measure to gauge the effectiveness of its refund fraud program.Refund fraud is one of the major management challenges the IRS has been facing for the last several years.CI measured its outreach efforts only by the staff years spent on those efforts which is also not results oriented.

CIís quality review measure includes only discontinued and non-prosecution cases with no grand jury involvement (about 27 percent of the CI workload), thus excluding the cases with the most impact on taxpayers from quality review.CI also restricted its customer satisfaction survey to U.S. Attorneys and did not include the Department of Justice (DOJ) Tax Division Attorneys.The Webster Report stated that these two agencies have conflicting interests about the types of cases CI pursues and the U.S. Attorneys had too much control over the CI caseload.Therefore, the surveys should obtain input from both parties.

We were not allowed access to the Criminal Investigation Management Information System (CIMIS) and thus could not determine whether CIMIS data were reliable.We could not validate the effectiveness of the CI process to ensure the accuracy of the CIMIS data and noted that CI had not conducted an audit of the database as had been recommended in the Webster Report.We also determined that system audit trails are not complete and were not effectively used, creating a potential security problem.

Managementís Response:The Chief, CI agreed with four of the nine recommendations included in the report. However, the Chief, CI did not agree that the CI business results measures did not address mission accomplishment, contending that their measures of the number of cases closed, the quality review of non-prosecution, non-grand jury cases and the number of education and outreach efforts reported CIís mission accomplishment.The Chief, CI also disagreed with the need to establish measures that reflect the effect of CIís efforts on non-compliance and the tax gap and a recommendation that a measure be developed that would reflect the effectiveness of CIís outreach efforts.The Chief, CI did not agree that limitations affecting the accuracy of CIMIS data should be disclosed when using the data.The Chief, CI also disagreed with a recommendation to have special agents in charge conduct the quality review at the time they reviewed the closing recommendation, which would allow grand jury prosecution cases to be included in the review process.

 

Managementís complete response to the draft report is included as Appendix VI.

 

Office of Audit Comment:Changing the GPRA measure from cases initiated to cases closed still does not show the effect CI has on tax compliance.CI has the ability to report meaningful data on its operation that, properly reported on a national basis only, will not violate the Internal Revenue Service Restructuring and Reform Act of 1998 provisions on the use of enforcement statistics.CIís quality review measure and process does not include significant numbers of cases, which impacts the usefulness of the data.Also, the ineffective use of audit trails, together with the potential conflict of interest arising from managersí review of their own data, highlights the need to disclose CIMIS data limitations. Where appropriate, we have included in the report our comments related to managementís response.

 

While we are not requiring the IRS Commissioner to respond to the Department of the Treasury concerning our disagreement on these matters, we are providing the Acting Assistant Secretary for Management/Chief Financial Officer an information copy of this report for use in any deliberations with Congressional staff concerning CI budget and performance issues.

 

Copies of this report are also being sent to the IRS managers who are affected by these recommendations.Please contact me at (202) 622-6510 if you have questions or Daniel Devlin, Assistant Inspector General for Audit (Headquarters Operations and Exempt Organizations Programs), at (202) 622-8500.

Table of Contents

Background

Criminal Investigation Business Results Measures Do Not Address Mission Accomplishment or Relate to All Significant Budget Line Items

Recommendation 1:

Recommendation 2:

Recommendation 3:

Recommendation 4:

Criminal Investigationís Management Information System Controls Could Not be Verified and System Audit Trails Are Not Complete

Recommendation 5:

Recommendations 6 and 7:

The Criminal Investigation Business Results Quality Measure Lacks Statistical Validity and Excludes Cases With the Most Impact on Taxpayers

Recommendation 8:

Criminal Investigation Is Not Including All External Customers in Its Customer Satisfaction Survey

Recommendation 9:

Appendix I Ė Detailed Objective, Scope, and Methodology

Appendix II Ė Major Contributors to This Report

Appendix III Ė Report Distribution List

Appendix IV Ė Federal Rules of Criminal Procedure

Appendix V Ė The Webster Report - Findings and Recommendations Pertinent to the Internal Revenue Service Criminal Investigation Balanced Performance Measurement Program

 

Appendix VI Ė Managementís Response to the Draft Report

Background

The Government Performance and Results Act (GPRA) requires federal agencies to establish standards for measuring their performance and effectiveness.Executive agencies are required to prepare multi-year strategic plans, annual performance plans, and performance reports on prior year accomplishments.

The overall goal of the GPRA was to improve agency performance and to provide objective information to congressional and executive branch decision-makers to assist them in appropriating and allocating federal funds. Therefore, it is essential that the data used for the performance measures are reliable and the results are valid and verifiable.

The Internal Revenue Service (IRS) Commissioner has established a Balanced Measurement System with three strategic goals that IRS staff and functions are directed to work toward:

        Top-quality service to each taxpayer in every interaction (Customer Satisfaction);

        Top-quality service to all taxpayers through fair and uniform application of the law (Business Results); and,

        Productivity through a quality work environment (Employee Satisfaction).

The IRS Balanced Measurement System will be used in each business unit or function to assess how well it is meeting its strategic goals.

In 2000, Criminal Investigation (CI) began reporting directly to the IRS Commissioner.CI has a budget of $395 million and a staff of approximately 3,700 special agents and support personnel.CIís mission is to serve the American public by investigating potential criminal violations of the Internal Revenue Code and related financial crimes in a manner intended to foster confidence in the tax system and compliance with the tax law.

CI delayed developing its measures while awaiting the results of a study of CI operations undertaken at the behest of the IRS Commissioner.The study started in July 1998 and the resulting Webster Report was issued on April 12, 1999.This report is discussed in greater detail in Appendix V.

On April 11, 2000, the IRS Executive Steering Committee approved the following measures for CI:

        Customer Satisfaction - measured through an annual satisfaction survey of U.S. Attorneys (USAs) and a transactional or periodic survey of the IRS operating divisions.

        Employee Satisfaction - measured through an annual employee satisfaction survey (we did not review the employee survey as the Organizational Performance Division administers an IRS wide survey of employees).

        Business Results - measured by the number of investigations completed by major program area (legal income tax crimes, illegal income financial crimes, and narcotics related financial crimes); a quality review of all non-prosecution cases; and, time and resources spent on outreach activities.

We conducted this review as part of the Treasury Inspector General for Tax Administration (TIGTA) multi-year strategy to audit IRS efforts to comply with the GPRA.Fieldwork was conducted in the Headquarters Office in Washington, D.C. from July 2000 to March 2001.We obtained background information on the CIMIS at the Office of the Special Agent-in-Charge (SAC), Dallas Field Office, and information on the Review and Program Evaluations process at the Director of Field Operations, Midstates Area.The audit was conducted in accordance with Government Auditing Standards, with the exception of the scope impairment detailed below.

We are unable to issue an opinion about the quality of the data used to report the CI business results due to a scope impairment.CI denied us access to the CIMIS database by stating that it contained information related to cases under grand jury investigation, access to which is restricted by the Federal Rules of Criminal Procedure (18 U.S.C. Appendix).The Federal Rules of Criminal Procedure, Rule 6(e), require that only those individuals approved by a government attorney may have access to grand jury material, and then only in furtherance of the criminal aspects of the case.Additional details about the Federal Rules of Criminal Procedure are provided in Appendix IV.Details of our audit objective, scope, and methodology are presented in Appendix I. Major contributors to the report are listed in Appendix II.

Criminal Investigation Business Results Measures Do Not Address Mission Accomplishment or Relate to All Significant Budget Line Items

Recently, the IRS decided on 65 GPRA measures (listed in the IRS FY 2002 Annual Performance Plan) that encompass all IRS operations.The only measure reported for CI activities is the number of cases initiated.CIís original balanced measures cover customer satisfaction, employee satisfaction, and business results.However, the business results measures donít assess the effect CI cases have on the tax gap and voluntary compliance as was recommended in the Webster Report.There is no specific measure for refund fraud, which is one of the IRSí major management challenges, and the number of staff years spent on outreach is CIís only measure of its outreach program.

CI chose these outcome neutral measures because the RRA 98 prohibits the use of Records of Tax Enforcement Results (ROTER) in performance evaluations and goal setting for individual employees.Additionally, the IRS issued Regulation 26 CFR 801, which prohibits using quantity measures to impose or suggest production goals for any organizational unit with employees responsible for exercising judgment with respect to tax enforcement results except in conjunction with an evaluation of customer satisfaction, employee satisfaction, and quality measurement.The regulation also forbade establishing goals for individual employees based on ROTERs.However, in our opinion, neither the Internal Revenue Restructuring and Reform Act of 1998 nor Regulation 26 CFR 801 prohibits reporting actual enforcement results at the national level.

CI is also reluctant to use ROTERs to measure operational results because of possible misperceptions by taxpayers and employees.CI believes that taxpayers might believe that CI is working to achieve its goals at taxpayer expense and that employees may believe that an expectation has been established for them to meet.

To avoid creating these perceptions, CI decided on the non-ROTER measures of quantity, quality, and outreach for its business results. However, these measures are not the statistics by which CI oversees its programs and not the statistics that will satisfy the GPRA by themselves.The OMB Primer on Performance Measurement states that statistics used to manage an organization are the statistics that should be used to report for the GPRA.

Business results balanced measures do not address mission accomplishment

CIís new measure of cases initiated and the original business results measures consisting of the number of cases closed by major program area, a quality review score of non-prosecution cases, and time and resources spent on outreach activities donít assess the effect of CIís investigative efforts.However, CI does have the ability to measure and record the results of its investigative efforts.The necessary information is reported in the quarterly Business Performance Review (BPR) and the National Operations Annual Report.

The BPR is a new process whereby CI reports quarterly to the Commissioner on its operations.The BPR contains measures such as the number of investigations initiated by program area, acceptance rates by the USAs and by the Department of Justice (DOJ) Tax Division Attorneys, and the number of convictions obtained.The report also contains information on the current business results measure of the number of cases closed.

Additionally, CIís National Operations Annual Report, (provided to the Congress and available to the public through CIís public web site) presents significant information on the major programs of CI and shows comparative statistics for a three-year period.Some of these statistics, such as the number of individuals indicted, convicted, and sentenced, were included in a January 23, 2001, USA Today article about CI.In our opinion, reporting these types of numbers will provide the Congress and others a better appreciation for what CI does and its results than just reporting the number of cases initiated.

Recommendation

1.      The Chief, CI should include the appropriate information from the National Operations Annual Report that presents the outcomes of CIís investigative efforts.

Managementís Response:The Chief, CI agreed that cases initiated was not the appropriate measure for CI for GPRA and changed the measure to cases closed.The Chief, CI also agreed with the need for more publicity and stated that the information in the National Operations Annual Report is put out on the CI web page for that purpose.

The Chief, CI disagreed with the overall condition that CI business measures do not address mission accomplishment.The Chief, CI contends that the measure of cases closed demonstrates that CI is investigating violations of the Internal Revenue Code and related financial crimes. A proposed business results quality measure of all non-grand jury cases submitted without a prosecution recommendation will ensure that all cases are conducted in accordance with the CI mission statement. The Chief, CI states that a proposed outreach measure of providing financial institutions and practitioners with education and outreach efforts will address the portion of the mission statement dealing with confidence in the tax system and compliance with the tax laws.Finally, the Chief, CI objects to the TIGTAís observation that the measures being reported for CI are not the statistics used to manage CI.As proof, the Chief, CI states that these statistics are included as part of the quarterly Business Performance Review furnished to the Commissioner.

The Chief, CI in a narrative preceding his response to the specific recommendations, stated that CI attempts to attain its mission through a general deterrence effect and therefore specific measures of accomplishment would not document how CI contributes to the IRS mission.Also, the Chief, CI believes that reporting specific results of cases sends the wrong message to the agents in the field.The Chief, CI stated that CI was forced by RRA 98, Section 1204, to develop outcome neutral measures.

Office of Audit Comment:Changing from cases initiated to cases closed still does not show the effect CI has on fostering confidence with the tax system and compliance with the law, as the measure (cases closed) by itself does not reflect any measurable impact on tax compliance.Combining that statistic with a quality review measurement of cases closed with a non-prosecution recommendation and no grand jury involvement will not show the effectiveness of CIís contribution to the IRS mission either.

A significant number of CIís cases involve either a grand jury or result in prosecution recommendations. Any CI performance measure that excludes these cases would not be representative of the total CI contribution to the IRS mission.The Chief, CI does not explain why cases with a prosecution recommendation and no grand jury involvement are not included in the quality review process.

The Chief, CIís proposed outreach measure also does not show how this effort will enhance confidence in the tax system and compliance with the law.The proposed measure is simply the number of outreach efforts, which again does not demonstrate the impact of those efforts on taxpayer attitudes and compliance.

While we acknowledge that the balanced measures are included in the BPR, CI also uses other statistics in the BPR, its web page, and the National Operations Annual Report to manage its operation.The CI measures taskforce identified many of these statistics as diagnostics to be used to identify areas in the CI operation needing improvement.TIGTA recommended using appropriate information from the National Operations Annual Report as part of the IRS GPRA submission to show the impact of CIís investigative efforts.The Chief, CIís reply does not address the use of this report for GPRA purposes.

TIGTA disagrees with the Chief, CIís assertion that using measures showing outcomes will violate RRA 98.While we are fully aware of the implications of this significant issue, RRA 98 only prohibits using records of tax enforcement results to evaluate individuals.Properly presented, enforcement results may be reported at a national level.

 

CIís actual effect on tax gap and non-compliance cannot be readily measured

The full impact of CI is difficult to measure.It is virtually impossible to tell how many people will decide not to break the law because of the conviction of someone to whom they can relate economically, personally, or professionally.However, CI can develop measures that will show its direct effect on non-compliance and the tax gap.This could be done, for example, by monitoring recidivism and the subsequent collection of taxes from individuals convicted of tax related crimes.

Recidivism can be measured through analysis of a convicted taxpayerís Masterfile records over a predetermined period of time to identify any instances of failure to file and/or pay the correct amount of tax.The direct effect on the tax gap could also be determined by the amount collected for the period included in the indictment.

OMB Circular A-11 recognizes that there can often be a significant lag in obtaining performance data for a particular period.This lag could be several years or more.Agencies are to report results when the data become available. Therefore, CI can include measures for recidivism and non-compliance even though the results may not be known until years after the investigation is completed.

Recommendation

2.      The Chief, CI should establish measures that show CIís effect on voluntary compliance and the tax gap.These measures should be included with the current balanced measures (customer satisfaction, employee satisfaction, and quantity and quality business results) to provide a more complete assessment of CI activities.

Managementís Response:The Chief, CI disagreed that CI should develop measures that would reflect the effect of CIís efforts on the tax gap and non-compliance.More specifically, the Chief, CI did not agree with the examples that TIGTA presented in the report to use recidivism, and taxes subsequently collected on periods included in the investigation for convicted tax felons, as measures of the actual effect CI has on reducing the tax gap and non-compliance. The Chief, CI stated that since 1988 the cumulative recidivism rate for CI has been 0.7 percent.The Chief, CI disagreed with using taxes collected as a measure since CI is in the business of general deterrence and taxes collected would be a measure of specific deterrence.He also felt that it would be wrong to motivate special agents with civil collections imposed, calling it a blatant ROTER violation.Finally, the Chief, CI objected to the burden of tracking this performance data with no tangible benefit.

Office of Audit Comment:TIGTA gave the use of the recidivism and taxes collected for convicted felons as examples of the type of measure CI needs to create to show the outcome of CIís efforts on taxpayer attitudes and non-compliance.While the Chief, CI may disagree with the suggestions for measures contained in the report, the obligation to develop measures that will show CIís effect on non-compliance and the tax gap remains.With regards to the 0.7 recidivism rate cited by the Chief, CI we understand that this was based on an analysis of repeat criminal tax behavior only.We believe recidivism should include both the civil and criminal aspects of subsequent compliance behavior to more comprehensively demonstrate the benefit of CIís work.

CI also disagreed with using tax collections as a measure, citing CIís primary objective of investigating criminal violations and the Webster reportís discussion that assessment and collection of taxes is a civil function to be conducted by other business units.We agree that assessing and collecting taxes is a civil function, but our recommendation strikes at the monetary impact of CIís efforts by evaluating those subsequent civil actions that most likely came about because of the criminal investigation and the subsequent adjudication.Using a Masterfile analysis, CI could identify the taxes collected for the periods emanating from the investigation, as it stands to reason that if there had been no investigation there most likely would have been no additional taxes.

As with any enforcement result, these measures should be reported on a national basis only. Also, the monetary results of the cases would most likely be compiled well after the criminal aspects are completed, which should further reduce any concern about agents misunderstanding their roles and expectations.Information of this nature is similar to the statistics on arrests, convictions, and sentencing placed on the public web page and the National Operations Annual Report.

While CI believes that there would be no tangible benefit to tracking this data, information of this or similar nature would help to bring CI into conformance with the GPRA, and also enhance the oversight of CI by providing the IRS Commissioner, OMB, Congressional committees and - most importantly - the American public with direct evidence of CIís success in achieving its mission.Analysis of subsequent compliance may also provide an added benefit of identifying repeat offenders for possible investigation.

CI does not currently include refund fraud as a separate measure even though specific resources are budgeted for this activity

CI has not developed a measure for its refund fraud program.For several years, the General Accounting Office (GAO) has identified refund fraud as a major management challenge for the IRS.The National Operations Annual Report does detail the results of the refund fraud programs such as the percentage and number of refunds deleted prior to issuance to the taxpayers.

One of the purposes of the GPRA was to improve congressional decision-making by providing more objective information on the relative effectiveness and efficiency of federal programs and spending.In the case of the refund program, it is very difficult to determine the effectiveness of the program since no measures have been established.To provide useful information, major programs and or significant resource allocations should have related performance measures.

Recommendation

3.      The Chief, CI should include measures that will report the results of major programs within CI.

Managementís Response:The Chief, CI agreed with this recommendation.CI will develop specific measures for refund fraud through the Fraud Detection Centers by October 1, 2002.

CI might better measure the effectiveness of its outreach efforts

One element of CIís business results is ďOutreach.ĒCI measures this by documenting the time and resources spent on outreach activities. We were informed that CI was considering evaluating the publicity obtained on significant cases to illustrate the effectiveness of its outreach program.In our discussion CI did not detail how the publicity was to be obtained or how its effect would be measured.However, CI has been including its National Operations Annual Report on its public web site.This report contains results of significant CI cases.

One means of measuring the effectiveness of the outreach efforts would be a survey of taxpayer attitudes and knowledge.Currently CI does not have any type of measure to assess how many taxpayers know about CI, what it does, or the possible consequences of not complying with tax laws.To gauge the publicís knowledge, CI might wish to consider surveying taxpayers and/or tax practitioners. Other federal agencies have faced similar circumstances and found the surveys to be useful.

For example, the Social Security Administration (SSA) used a similar approach on its strategic goal of strengthening public understanding of Social Security programs.The SSA wants 9 out of 10 Americans to know about important Social Security programs by 2005.To accomplish this, the SSA worked with the Gallup Organization to develop the SSA Public Understanding Measurement System (PUMS) Survey.The survey was first conducted in FY 1999 to establish the baseline of public knowledge and is now administered annually.

The SSA also developed public education products and presentations to reach the appropriate audiences, much as CI is attempting to publicize the successful prosecutions of persons convicted of tax fraud.CI could take a similar approach to better measure the effectiveness of its outreach efforts.We recognize that the SSA effort is directed toward educating the public about possible monetary benefits they may receive, while the CI effort is to reduce possible tax evasion through increased awareness of investigative results.Both, however, involve educating the public and measuring the effectiveness of the educational effort.

Recommendation

4.      The Chief, CI should consider an approach similar to the SSA survey of taxpayers or tax practitioners to determine if CIís outreach efforts are effective in educating the public about the consequences of tax evasion.

Managementís Response:The Chief, CI did not agree with the recommendation or the approach suggested by TIGTA.The Chief, CI stated that because of the results of two prior surveys, CI had formulated a communications strategy designed to make the public and practitioners more cognizant of CI successes.CI believes it can measure its obvious success by counting the number of accesses to its web site and the number of news stories published and speaking engagements conducted.

Office of Audit Comment:While CI has measures for the outreach efforts, these measures do not convey the impact of these efforts. A properly conducted survey could provide valuable information on whether, for example, CIís outreach program fosters confidence in the tax system and compliance with tax laws. The number of outreach efforts by itself will not provide that type of information.The number of news stories is at best an indirect means to assess public understanding because the decisions to publish stories are driven in part by editorial decisions regarding the newsworthiness of a particular case, and not necessarily a concern for the publicís understanding of IRS operations or tax compliance.

 

Criminal Investigationís Management Information System Controls Could Not be Verified and System Audit Trails Are Not Complete

CI uses the Criminal Investigation Management Information System (CIMIS) to control cases and as the source for the statistical information used to manage CI.The CIMIS also will provide the quantity measure for business results.The GPRA and OMB Circular A-11 both require agencies to identify the means they will use to verify and validate performance data.To date, CI has not provided that information.

As a part of regular operations, supervisory special agents are expected to review the monthly Measurement for Managers (MOM) reports of CIMIS information for their group.Internal Revenue Manual (IRM) 114.1 (Compliance and Customer Service Handbook) requires the supervisory special agents to conduct a quarterly open case review including a review of CIMIS updates.The handbook also requires special agents-in-charge (SACs) to conduct bi-annual operational reviews including an in-depth discussion of the CIMIS.Finally, the handbook requires that the national office Review and Program Evaluation teams conduct a review of CIMIS as part of its tri-annual reviews of the various SAC offices.While these actions should occur on a regular basis, we cannot assert that they are and we could not independently determine if CIMIS data are correct.

We requested a download of the CIMIS database to test the accuracy of these verification and validation processes.CI denied us access to the database, stating that the database contained grand jury information.Due to this scope impairment, we cannot comment on the validity of the data on the CIMIS or the accuracy of the compilation of information for the business results quantity measure. See Appendix IV for an explanation of the federal rules covering the confidentiality of grand jury information.

In 1998 concerns were raised by the Transactional Records Access Clearinghouse (TRAC) about the accuracy of CIMIS data when compared to DOJ records and the records of the Federal Courts.The Webster Commission attempted to address these concerns by conducting a review of CIMIS, but was not allowed access to grand jury information.The Commission was not provided a complete explanation of the alleged discrepancies and recommended in April 1999 that the IRS Commissioner conduct an audit of the CIMIS database to ensure its accuracy.This has not been done.

The CIMIS audit trail is not complete and has not been effectively used

We determined that there is no effectively managed audit trail for systems administrators.Five systems administrators can log on as root users of the operating system. Unlike regular system users, root users have access to change or view all information on CIMIS and other applications on the system.

According to the System Administration Manager guidance documentation, an audit trail has to be turned on for each individual as a user in order to record detailed system activities such as accessing data objects or files.CI personnel stated that audit trails were not turned on for system administrators, and the Director of Business System Planning (BSP) confirmed this.When the audit trail is turned on for system administrators, the audit log records each system call as well as actions taken by the administrator.

CI contends that the voluminous recordation of unrelated system actions makes the audit log useless and therefore does not run it.However, without an effective audit trail, the system administrators have virtually undetectable access to all information on the CIMIS including any grand jury information on the system.This control weakness means that the system could be accessed and data on the system could be changed or information could be obtained and used for unofficial purposes without detection.This could impact CIís ability to produce accurate and reliable data as required by the GPRA.Additionally, any sensitive information on the system is subject to unauthorized and undetected access.

To correct this weakness, one approach is to have a program developed that would filter out all actions but those of the systems administrators.This would minimize the number of transactions to record and overcome CIís concerns that too much information was tracked for the audit trail to be of use. CI is developing a new version of the CIMIS, and we recommend that an adequate system administrator audit trail be included in this new system.

Also, regular CIMIS user audit trails that are run are only reviewed when deemed necessary by CI.No one has been assigned responsibility to routinely perform audit trail reviews or to ensure that they are done.CI management agreed that corrective action would be taken and developed procedures assigning responsibility for review of audit trails.However, the procedures do not contain criteria for the review of the audit trail or how the audit trail reviews will be documented.

The Department of the Treasury Security Manual (TD-P 71Ė10) requires that information systems and networks which process, store, or transmit sensitive, but unclassified information, must meet the minimum baseline requirements for C2 level protection as evaluated by National Institute of Standards and Technology.The C2 level is defined by the Department of Defense in the ďOrange Book.ĒThis level of security requires that the system be able to record the actions taken by computer operators, systems administrators, and or/security officers and other security related events.For each recorded event, the audit record shall identify date and time of the event, user, type of event, and success or failure of the event.Also, the system administrator shall be able to selectively audit the actions of any one or more users based on individual identity.

Additionally, the C2 security level requires that the Information System Security Officer (ISSO) shall ďConduct security audits, verifications, and acceptance checks, and maintain documentation on the results.ĒFurther, it states that the ISSO shall ďExamine system audit logs regularly and report anomalies to the Senior Information Resource Management Official.Ē

CI contends that the first level managerís monthly review of CIMIS reports (as discussed on page 13 of this report) is sufficient to satisfy audit trail review requirements.However, this does not meet the minimum C2 security standard for regular, independent security reviews by the ISSO.Also, reviews of these reports cannot identify any unauthorized accesses to the system for potential illicit information gathering purposes.

Recommendations

5.      The Chief, CI should establish a process by which the CIMIS data will be verified and validated.

Managementís Response:The Chief, CI agrees with the premise underlying the recommendation, although he states that the recommendation may not be an issue once the new CIMIS III system is operating. CI expects that the new system will be functioning for the first quarter of FY 2003.In the interim, CI will rely on reviews of the CIMIS data by group managers for their own groups.The Chief, CI agrees that the CIMIS audit trails are incomplete and have not been used effectively, but states that incomplete audit trails do not necessarily equate to inaccurate data.The Chief, CI also stated that CI has discredited the allegations of the TRAC report on the inaccuracy of the CIMIS database.CI has requested that the Office of Performance Evaluation and Risk Analysis (OPERA) create an audit work plan for the current CIMIS.This is to be completed by September 30, 2002.

Office of Audit Comment:Since the new system is at least a year away, CI still needs to have an established process to verify and validate the data in the interim.While group managers currently validate the CIMIS data for their own groups, there is no process in place to ensure that this is done accurately. Also, reliance on this procedure places managers in the position of reviewing their own work.TIGTA had planned on reviewing this control, but could not because CI denied us access to any data on the CIMIS database.We thus cannot evaluate the effectiveness of this control or the accuracy of the database.

While the Chief, CI may be correct about inaccurate audit trails not necessarily equating to inaccurate data, the lack of complete and properly used audit trails is a severe security concern.There are five systems administrators that have virtually undetectable access to all of the information on the CIMIS, some of which is highly sensitive.

The Webster report stated that CIís response to the concerns of the TRAC report had provided an explanation of what might have caused the discrepancies between the TRAC data and the CI data.The Webster report also recommended that an empirical audit be done of the CIMIS database.In our opinion, this action along with our recommendation is necessary for CI to conclusively resolve the issue concerning the accuracy of CIMIS data.This seems to be what CI is now planning to undertake with its corrective action.

6.      The Chief, CI should ensure that audit trail requirements are met and documented, and that an ISSO is assigned to review audit trail data.These requirements should be incorporated into the new CIMIS program.

Managementís Response:The Chief, CI agreed with TIGTAís recommendation.The Chief, CI further stated that CI has established procedures to review CIMIS audit trail data and that a CI security officer has been assigned to review the data in accordance with the new procedures.The Chief, CI also stated that the new system (CIMIS III) would correct the problems with the audit trails under the old system.

7.      In the interim, fully disclose the CIMIS data limitations whenever reporting results based on it.

Managementís Response:The Chief, CI disagrees with the recommendation.The Chief, CI believes that the CIMIS data is valid and accurate.Therefore, CI does not have any basis for reporting any qualifications.The Chief, CI believes that CI has discredited the allegations in the TRAC report on the accuracy of the CIMIS database.

Office of Audit Comment:Since there is no effective audit trail for five systems administrators, undetected access and modification to the data could have occurred.Also, while CI depends on group managers to validate the accuracy of the database, there is no process to ensure that this work occurs and that it is done properly.The Webster report did not agree that CI had discredited the allegations in the TRAC report, and recommended an empirical audit of the database be done.At the time of our review no audit of the CIMIS database had been done, although CI indicated in its corrective action to recommendation number 6 that an audit will now be undertaken.

Finally there is no regular review of user audit trails.Based on these facts, CI should disclose that it cannot ensure the accuracy or security of the data.CI also should disclose that it currently relies on first level managers to validate their own groupís CIMIS data, creating a potential conflict of interest.

The Criminal Investigation Business Results Quality Measure Lacks Statistical Validity and Excludes Cases With the Most Impact on Taxpayers

The CI business results quality measure, which was approved by the IRS Executive Steering Committee, is a review of all discontinued and declined non-prosecution cases that did not involve grand jury material.

CI decided on these limitations for the following reasons:

        Prosecution cases are reviewed several times during the completion of the case;

        A review of the case prior to completion could be subject to discovery by defense attorneys, who could misdirect a jury using this information;

        The length of time to complete a prosecution case (sometimes several years) would be of no benefit to current year operations if the case was reviewed after completion; and,

        Federal Rules of Criminal Procedure Rule (6e) requires that access to grand jury information be granted only in furtherance of the criminal aspects of the case.

Only 27 percent of all CI cases had neither grand jury involvement nor a prosecution recommendation. A review of 27 percent of CI closed cases will not result in a true quality measure applicable to all CI closed cases.For any estimate of a population to be made, a valid statistical sample must be taken in which each case in the population has an equal chance of being selected.

During discussions with CI management and IRS Counsel in Washington in September 2000, we asked if a quality review of all closed cases could be performed at the same time the case closure recommendations were reviewed.CI subsequently obtained an opinion from IRS Counsel that supervisory special agents could do such a quality review on a case at the same time they were reviewing the closure recommendation.This would allow CI to do a 100 percent review of closed cases and remove concerns over the sample selection issue.

CI could have the SACs (the second-level managers) conduct the quality reviews when they do their reviews of the closure recommendations.While this creates a potential for a conflict of interest, it will provide much more accurate information on the quality of the agent casework.

Recommendation

8.      The Chief, CI should implement the SAC 100 percent quality review process, since this presents the best balance between the issues of sampling and objectivity.However, CI should fully disclose the effect of Rule 6(e) on the review process and the possibility of a conflict of interest on the part of the quality reviewers.

Managementís Response:The Chief, CI disagrees with the recommendation because of the following reasons:

        CI believes that the numerous levels of review that a case goes through ensure the quality of the work and adding another level of review is not necessary.This is shown by the elevated level of its acceptance of cases for prosecution and its conviction rates.

        CI does not believe that a 100 percent review is necessary.

        CI believes that the appearance of a potential conflict of interest in having the second level managers review the cases outweighs the effect of the limited sample size.

Office of Audit Comment:TIGTA concurs with the Chief, CI that a 100 percent review of all cases would not be necessary if CI performs the review using a random statistical sampling method and all cases are included in the universe.

While CI does not believe that another level of review is necessary, it has already added one for the non-grand jury cases closed without a prosecution recommendation.The basis of our recommendation was to enable CI to quality review all types of cases while following the Federal Rules of Criminal Procedures with regard to grand jury secrecy requirements.While CI is understandably concerned about the perception of a conflict of interest if the second level managers do the quality review, we believe that the perceptions that could arise by disclosing that no quality review is done on any prosecution case, grand jury involvement or not, is worse. CI needs to include at least a sample of all cases in the quality review process.

Criminal Investigation Is Not Including All External Customers in Its Customer Satisfaction Survey

One of CIís balanced measures is customer satisfaction.CI developed this measure even though the Webster Commission recommended that CI defer development until the other law enforcement agencies established a means of appropriately measuring customer satisfaction.CI believed that the USAs were the true end-users of its product and therefore the only customers who should be surveyed.

The Webster Commission reported that the USAs, and not CI itself, were determining CIís investigative agenda, resulting in the pursuit of other federal law enforcement initiatives at the expense of tax enforcement.The Commission also reported that the DOJ Tax Division stated that CI should be more involved in Income Tax (Title 26 of the U.S. Code) investigations and less involved in narcotics and money laundering cases. Surveying USAs only might result in a continued emphasis on cases with little impact on overall tax compliance.

Almost all cases involving tax law violations (Title 26) must be referred to the DOJ Tax Division for its approval of the prosecution recommendation. The following table reflects the number of Title 26 prosecution recommendations in FY 1998, 1999 and 2000, as opposed to the non-tax law related violations investigated by CI under U.S. Code Titles 18 and 31, most of which are direct referrals to the U.S. Attorneys.

Table 1: CI Prosecution Recommendations by FY and U.S. Code Violation

Year

Title 26

Titles 18 and 31

Total

Pct. Of Title 26 Referred To DOJ

2000

794

1640

2434

32.6%

1999

1068

2052

3120

34.2%

1998

1369

2158

3527

38.8%

Total

3231

5850

9081

35.6%

Source:FY 1998, 1999, and 2000 CI National Operations Annual Reports.

The above table shows that, at a minimum, over a third of CI prosecution recommendations are referred to the DOJ Tax Division.We believe that including the DOJ Tax Division attorneys in the customer satisfaction survey would provide a more balanced input from the two primary customers (USAs and DOJ Tax Division) with sometimes competing priorities for CI casework.

To date, CI has contracted with a private vendor to develop the questionnaires and the evaluation process. The development of the actual survey process is now underway, so we are not able to comment on its objectivity or effectiveness.

Recommendation

9.      The Chief, CI should include the DOJ Tax Division attorneys in the Customer Satisfaction Surveys.

Managementís Response:The Chief, CI agreed with the recommendation and stated that CI would include the DOJ Tax Division Attorneys in the survey by September 30, 2002.However, the Chief, CI also stated that he did not agree with TIGTAís rationale for making this recommendation.The Chief, CI believes that its survey was directed toward the quality of work sent to the DOJ and U.S. Attorneys and not to their satisfaction with the types of cases sent to them by CI.With the survey thus organized, the attorneys would have no input into the case selection.The Chief, CI also believes that CI is correcting the mission drift and getting back into tax casework.

Office of Audit Comment:The Webster Report stated that CI was pursuing overall law enforcement initiatives at the expense of tax enforcement.The report cited the increase in grand jury cases from 13 percent in 1980 to 78 percent in 1998, and interviews with special agents who said that federal attorneys and other law enforcement agencies were setting CIís caseload agenda as evidence of outside influence on the CI caseload.The agents also stated that CI almost always participated in grand jury investigations when requested to do so by the U.S. Attorneys.

The Webster report also expressed the desire of the DOJ Tax Division to see more CI involvement in tax cases, while the DOJ Criminal Division and the U.S. Attorneys were satisfied with the status quo.The Webster Report recommended that CI return to conducting tax investigations. Surveying the U.S Attorneys without surveying the DOJ Tax Division could lead to a mindset that U.S. Attorneys are the only customers and their wants should be addressed.We believe the issues presented and opinions expressed in the Webster report are still valid, and CI should take the necessary actions to achieve the reportís intent.

Appendix I

 

Detailed Objective, Scope, and Methodology

 

The overall objective of this review was to evaluate the reliability and relevance of the performance measures used by the Internal Revenue Service (IRS) Criminal Investigation (CI) in assessing its business results and customer satisfaction with CI casework.We addressed the overall objective through the following sub-objectives:

I.††††††††††† Determined if the CI business results measures accurately reflect CIís mission accomplishment and if the measurement is currently in place.

A.     Determined how CI decided on the current business results measures.

1.      Obtained documentation behind the development of the business results measures.

2.      Interviewed the officials responsible for the development of the measurements.

3.      Analyzed CI programs, staffing, and funding to determine if measures had been developed to report performance for each major program.

B.     Determined if CIís strategic plan contains specific, measurable goals by which the program could be accurately evaluated and the relative success or failure of the organization to meet its expectations could be readily determined.

C.     Determined if the new CI business results measures are in accordance with the recommended strategy contained in the Webster Report.

1.      Obtained current status of the recommendations in the Webster report on business results measures.Interviewed appropriate CI officials to determine the intent behind the specific actions taken.

2.      Compared the business results process with the Webster Report and the new CI mission statement. Determined if the process will measure business results in accordance with the recommendations and intent of the report and mission statement to develop a compliance strategy and apply its resources to effectively implement that strategy.

II.                 Determined if the IRS efforts to conform to the requirements of the IRS Restructuring and Reform Act of 1998 (RRA 98) governing the use of tax enforcement results were excessive and hindering accurate reporting for the Government Performance and Result Act (GPRA).

A.     Reviewed the GPRA and identified the requirements for measuring and reporting performance.

B.     Reviewed the RRA 98 and determined the limitations on the IRS in using tax enforcement data.

C.     Reviewed the Code of Federal Regulations (including Regulation 26 CFR 801) issued by the IRS and the Internal Revenue Manual Handbook 104.5 and identified the limitations the IRS has placed upon itself with regard to the use of tax enforcement measures.

D.     Compared the results of A and B with that of C and determined if CI could use enforcement statistics to satisfy the requirements of GPRA.

E.      Obtained an opinion from TIGTA Counsel to verify our position and recommendations on this issue.

III.               Determined if the Criminal Investigation Management Information System (CIMIS) database is accurate, secure, and validated properly.

A.     Determined if the IRS conducts evaluations of the accuracy of CIís CIMIS data to ensure the verification and validation of the information reported.

B.     Determined if CI has effectively addressed the concerns about the accuracy of the CIMIS database raised in the Webster Report by obtaining a copy of the IRS analysis of the Transactional Records Access Clearinghouse report and determining if it adequately addressed the issues mentioned in the report.

C.     Determined the effectiveness of the CIMIS controls through the following tests:

1.      Determined if an inventory validation process was in place, correctly followed, and if all necessary corrections were made.

2.      Determined if only authorized persons have access to the system.

3.      Determined who may input data to the system.

a.       Interviewed an operator to determine how they ensure that only valid, properly approved data is loaded onto the system.

b.      Determined how CI ensures the validity of the data.

c.       Determined if an audit trail exists and if it is adequate to ensure that undetected access to CIMIS data does not occur.

IV.              Determined if the CI customer satisfaction measures address the appropriate customers, and if the measurement is being effectively accomplished.

A.     Determined how CI decided on the current customer satisfaction measures.

1.      Obtained documentation behind the development of the customer satisfaction measures.

2.      Interviewed the officials responsible for the development of the measurements.

B.     Evaluated the methodology behind the survey of the U.S. Attorneys (USAs) and the operating divisions to determine if the customers included in the survey are appropriate and of the right mix.

C.     Determined if the new CI customer satisfaction measures are in accordance with the recommended strategy contained in the Webster Report.

1.      Obtained the specific corrective actions taken in response to the Webster report.

2.      Interviewed appropriate CI officials to determine the intent behind the specific actions taken.

D.     Determined if it is appropriate to have a customer satisfaction measure for CI.

1.      Reviewed the measures of other law enforcement agencies to determine if they attempt to measure customer satisfaction.

2.      Compared the efforts of the other agencies and CI and evaluate the feasibility of coming up with a valid and useful measurement for customer satisfaction.

E.      Determined if CI will report all of the limitations and qualifications of its data in the annual performance report.

Appendix II

 

Major Contributors to This Report

 

Maurice Moody, Former Assistant Inspector General for Audit (Headquarters Operations and Exempt Organizations Programs)

Dan Devlin, Assistant Inspector General for Audit (Headquarters Operations and Exempt Organizations Programs)

John Wright, Director

Kevin Riley, Audit Manager

David Cox, Senior Auditor

Ken Henderson, Senior Auditor

James Popelarski, Senior Auditor

Tom Burroughs, Auditor

Steve Holmes, Auditor

Bill Thompson, Auditor

Appendix III

 

Report Distribution List

 

Department of the Treasury

Acting Assistant Secretary for Management/Chief Financial OfficerM

Under Secretary for EnforcementUO

 

Internal Revenue Service

CommissionerN:C

Deputy CommissionerN:DC

Director, StrategyCI:S

Chief CounselCC

National Taxpayer AdvocateTA

Director, Legislative AffairsCL:LA

Director, Office of Program Evaluation and Risk Analysis N:ADC:R:O

Office of Management ControlsN:CFO:F:M

 

Appendix IV

 

FEDERAL RULES OF CRIMINAL PROCEDURE

18 U.S.C. Appendix

Rule 6(e) Recording and Disclosure of Proceedings

 

Federal Rules of Criminal Procedure govern the conduct of all criminal proceedings in the courts of the United States.They are also applicable whenever specifically provided for in one of the rules, to preliminary, supplementary, and special proceedings before United States magistrate judges and at proceedings before state and local judicial officers.These rules are intended to provide for the just determination of every criminal proceeding.They are intended to secure simplicity in procedure, fairness in administration, and the elimination of unjustifiable expense and delay.

Rule 6(e) of the Federal Rules of Criminal Procedure sets specific requirements with regard to the recordation and release of information obtained pursuant to grand jury investigations.Generally, any person to whom disclosure is made in grand jury matters may not disclose that information under the General Rule of Secrecy.Violation of the secrecy requirement may result in contempt of court charges to the individual violating the rule.

Additionally, any person to whom matters are disclosed shall not utilize grand jury material for any purpose other than assisting the attorney for the government in the performance of the attorneyís duty to enforce federal criminal law.An attorney for the government must promptly provide the district court that impaneled the grand jury whose material has been disclosed, with the names of the persons to whom such disclosure has been made.The attorney must also certify that the persons receiving the material were advised of their obligation of secrecy under the rule.

Several exceptions exist for the general rule of secrecy.Disclosure may only be made:

           As directed by a court preliminarily to or in connection with a judicial proceeding;

           By an attorney for the government for use in the performance of the attorneyís duty;

           To government personnel (including personnel of a state or subdivision of a state) deemed necessary by an attorney for the government to assist an attorney in the performance of the attorneyís duty to enforce federal criminal law;

           When permitted by a court at the request of the defendant, upon a showing that grounds may exist for a motion to dismiss the indictment because of matters occurring before the grand jury;

           When the disclosure is made by an attorney for the government to another federal grand jury; or,

           When permitted by a court at the request of an attorney for the government, upon a showing that such matters may disclose a violation of state criminal law, to an appropriate official of a state or subdivision of a state for the purpose of enforcing such law.

Appendix V

 

The Webster Report - Findings and Recommendations Pertinent to the Internal Revenue Service Criminal Investigation Balanced Performance Measurement Program

 

In July 1998 Internal Revenue Service (IRS) Commissioner Charles O. Rossotti requested that the Honorable William H. Webster conduct an independent review to assess Criminal Investigationís (CI) effectiveness in accomplishing its mission as the IRSí criminal enforcement arm.Mr. Webster, with the assistance of a former head of the Department of Justiceís (DOJ) Office of Professional Responsibility, formed a task force of experienced federal financial law enforcement investigators.The review team was supervised by a trial attorney from the Fraud Section of the Criminal Division of the DOJ.

The review focused on the make-up of CIís caseload, its investigative methods, organizational structure, and personnel policies and practices.The taskforce issued its report in April 1999.The report executive summary contained 25 overall recommendations for improvement, all of which were agreed to by the Commissioner. The report body provided 172 specific recommendations necessary to accomplish the overall recommendations.

The report emphasized that CI must recognize that its principal mission is to investigate criminal violations of the Internal Revenue Code.The report made the following recommendations that could have an impact on CIís balanced performance measures:

1.      CI, in conjunction with the other IRS compliance components, should develop a compliance strategy that will enable it to determine how best to allocate its resources in a manner consistent with its tax enforcement mission.As an initial step, rigorous empirical studies of noncompliance will enable CI to identify those cases that will deter noncompliance most effectively, and thereby reduce the tax gap.Concerns have been raised to the increasing percentages of CI investigations conducted through a grand jury and the corresponding decrease in administrative investigations.By fiscal year 1997, the percentage of CI prosecution recommendations that resulted from grand jury investigations had risen to 78 percent.Tax Division officials have expressed concern with the high percentage of investigations that CI is conducting through a grand jury because they believe it allows the U.S. Attorneys (USAs), and not the IRS, to control CIís caseload and agenda.In addition, they believe that the increase in grand jury investigations has resulted in a reduction in the investigation of traditional tax cases. They believe that CI is working on the cases that support law enforcement initiatives of the USAs instead of the mission of the IRS.

2.      CI should continue to exercise its authority to investigate violations of the money laundering and currency statutes in a way that is consistent with the compliance strategy.

3.      CI devotes considerable resources to narcotics investigations, for which it is not reimbursed.Narcotics investigations contribute only incidentally, if at all, to fostering tax compliance.Resources devoted to narcotics investigations in excess of reimbursed funds should be brought under control so as not to deplete resources devoted to tax compliance.

4.      CIís Criminal Investigation Management Information System (CIMIS) should track participation of other law enforcement agencies and the inclusion of non-IRS charges in an indictment or information in CI cases.The Commissioner should conduct a thorough, empirical audit of CIMIS to ensure that its data are accurate. CIís response to the concerns raised by the Transactional Records Access Clearinghouse is reasonable, but it only suggests what might cause some of the alleged inconsistencies in data between CI statistics and those maintained by the DOJ and the federal courts.The CI response does not provide a complete explanation for those alleged discrepancies.

5.      Like the other components of the IRS, CI should adopt several methods of measuring its performance as an organization.The proposed new measure of Resource Effectiveness Rate should be a good indicator of CIís business results.CI should also create and maintain a new measure to show the rate of its work in areas that IRS compliance research has identified specifically as being appropriate for criminal enforcement.CI should measure the ratio of agentsí investigative time on investigations in identified areas of noncompliance to the total investigative time of agents on all investigations.The Commissioner should not require CI, however, to assess ďCustomer SatisfactionĒ until the other federal law enforcement components develop an acceptable measure in this area.CI should measure ďEmployee SatisfactionĒ in the same way as the other IRS components.Finally, the Measurements Task Forceís recommendation to use a combination of measures is the best way to evaluate CI.

Appendix VI

 

Managementís Response to the Draft Report

 

The response was removed due to its size.To see the complete response, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.