Copies of this report are also being sent to the IRS managers who are affected by the report recommendations. Please contact me at (202) 622-6510 if you have questions, or your staff may call Gordon C. Milbourn III, Associate Inspector General for Audit (Small Business and Corporate Programs), at (202) 622-3837.

Executive Summary

Objective and Scope

Background

Results

Conclusion

Appendix I – Detailed Objective, Scope, and Methodology

Appendix II – Major Contributors to This Report

Appendix III – Report Distribution List

Appendix IV – Auditing Data Reliability – Guidance and Glossary of Terms

Executive Summary

The Enforcement Revenue Information System (ERIS) is an automated system developed to track Internal Revenue Service (IRS) enforcement revenues and associated costs. Enforcement revenues include the assessment and collection of all dollars resulting from IRS enforcement efforts.

The Department of the Treasury and the IRS use the ERIS for planning and budgeting. The ERIS also supplies data for the four IRS business units and is cited as a data source for measuring future IRS business results.

The ERIS works by compiling enforcement data from various IRS functional systems. These functions include Examination, Office of the Chief Counsel, Appeals, and Collection. Data from function-based "feeder" systems, such as the Examination Division’s Audit Information Management System (AIMS) and the Information Returns Program (IRP), are merged with Masterfile and Non-Masterfile data to create summary databases from which ERIS reports are generated.

The overall objective of this audit was to assess ERIS data reliability. To achieve this objective, we determined if the ERIS accomplishes its stated purpose of compiling statistics and dollars associated with IRS tax collection enforcement actions. The scope of the review also included reviews of controls within the ERIS and limited data testing. Unless otherwise noted, we relied on comments provided by IRS management and did not specifically test control weaknesses identified to determine their actual effect. The audit was not intended to identify specific errors in ERIS reports but rather to evaluate the system controls and processing which affect data reliability. We also did not review the accuracy of functional IRS systems that supply enforcement data to the ERIS.

Our assessment of data reliability is separated into opinions based upon the two types of data (revenues and costs) the ERIS was intended to compile and report. Regarding the revenue data, we cannot issue an unqualified opinion on data reliability because the ERIS has systemic design and control weaknesses that can affect the reliability of historical and future revenue data and because the reliability of systems that supply data to the ERIS has not been determined. However, we did identify many system controls working as designed, and the results of our system functionality tests identified no processing discrepancies.

Regarding cost data, we cannot express an opinion on data reliability because the ERIS does not compile or report complete cost data associated with enforcement actions. For example, in Fiscal Year (FY) 1997, the ERIS did not report on at least $2.1 billion in enforcement-related costs. The ERIS does report direct hours for the Examination, Office of the Chief Counsel, and Appeals Divisions; however, these hours are not converted to costs. The incomplete cost data are contrary to the original intent of the ERIS.

We identified problems with two specific control areas in the ERIS. First, some users and contractor employees had unlimited access to the ERIS, and computer audit trails are not effectively reviewed. Second, timing and processing differences between the ERIS and related systems are not routinely reconciled. While these differences do not materially affect revenue dollars reported by the ERIS, they do affect the way dollars are reported by tax year.

Computer system controls include general and application controls, which together provide reasonable assurance that computer-based data are complete, valid, and reliable. General controls are those that apply to an overall computer operation, such as management oversight of the system’s performance, assignment of responsibilities, and physical security. Application controls are those that apply to a specific application, such as testing software modifications before implementation and documenting program changes. While most of the ERIS general and application controls are adequate, we identified two specific control areas needing improvement.

Controls over ERIS computer user accountability can be further enhanced. Audit trails provide records of operator and system activity. These records provide invaluable day-to-day histories of system operations and account for actions both taken and omitted. At the time of our review, ERIS management did not see the need to routinely review ERIS-related audit trails, including database and system administrator activity. In addition, there was no audit trail of specific contractor activity at the Detroit Computing Center. ERIS management decided that some users needed unlimited system access for creating, updating, and/or deleting data within ERIS databases. Unlimited access, along with insufficient accountability and historical records of user actions, increases the risk of inappropriate access and data manipulation. It further creates an inability to determine the origin of errors.

Controls over ERIS error and functional data resolution are insufficient. One attribute of a successful, working system is a control to resolve processing errors. ERIS programming identifies errors and unmatched transactions, which it compiles to various registers. However, ERIS management stated that they do not resolve the problems identified on these registers because of the volume of transactions and the difficulty of performing cross-functional reconciliations. As a result, ERIS error and unmatched registers have grown exponentially since their creation. According to ERIS management, as of March 1999, the ERIS had 53,119 cases in error registers, out of approximately 236 million cases in the system. Further, as of March 1999, the ERIS had accumulated 1.05 million transactions (representing $25.5 billion) in "unmatched transaction" registers, out of about 1.9 billion transactions, since its creation. ERIS management advised us that they had identified the sources of 237,000 of these cases (representing $7.1 billion) as being AIMS- or IRP-related. Although the volumes in the error and unmatched transaction registers are not material in comparison to the overall volume of cases the ERIS processes for any given year, unresolved errors and unmatched transactions are a systemic processing weakness which can represent material, hidden discrepancies.

In addition to unmatched transactions, there are also unresolved discrepancies between the functional systems and the ERIS. According to ERIS management, the ERIS may report the correct revenue amount, tax period, and district office but may not be able to identify the program area within the functional office that conducted the enforcement effort. Except for periodic software upgrades, which resolve some errors and unmatched transactions, there is no structured, continuous reconciliation process between the ERIS project staff and the IRS functions to properly correct the misidentified program areas. Moreover, ERIS management has no plans to resolve unmatched and error transactions on a large scale. Instead, they will continue to address errors and unmatched transactions through ongoing means, such as making annual and periodic system programming upgrades and taking small, judgmental samples to detect additional errors. However, to help measure future business results, it is important to ensure the reliability of ERIS data.

The stated purpose of the ERIS is to compile and report revenues and costs associated with IRS tax enforcement actions. However, the ERIS has not fully accomplished this purpose because it was not provided data to calculate all applicable costs. Although the ERIS does report direct hours (time specifically spent on casework) for the Examination, Office of the Chief Counsel, and Appeals Divisions, the ERIS reports do not convert these hours into costs. For FY 1997, the excluded costs associated with enforcement actions were at least $2.1 billion, $582.5 million of which represented indirect costs. ERIS management acknowledged that the ERIS was designed to process all applicable costs and that the incompleteness is primarily caused by the various IRS functions not providing necessary cost data to the ERIS. When we brought the cost issue to ERIS management’s attention, they agreed that ERIS documents should not have references to cost data, except for direct hours. The ERIS information on the Chief Financial Officer’s (CFO) website, which the program office developed during the course of our audit, does not contain references to cost data, only to direct hours. However, there are other documents (e.g., the Internal Revenue Manual) that still show the ERIS’ purpose as tracking enforcement-related revenues and costs. Management indicated they plan to correct these other documents concerning the omission of cost data.

We recommend the CFO evaluate the cost-effectiveness of resolving the ERIS control weaknesses, including implementing reconciliation processes and statistical sampling. The CFO should also either incorporate all relevant cost data into the ERIS or alter all statements of purpose to accurately reflect the system’s actual use. In addition, the IRS should disclose on all applicable reports that cost data are incomplete.

Management’s Response: IRS management’s initial response did not adequately address the recommendations in the report. They are currently revising and reissuing their response.

Objective and Scope

The overall objective of this audit was to assess Enforcement Revenue Information System (ERIS) data reliability. To achieve this objective, we performed tests to determine if the ERIS accomplishes its intended purpose, which is to compile statistics and dollars associated with the Internal Revenue Service’s (IRS) tax collection enforcement efforts.

To develop our objective and tests, we used General Accounting Office (GAO) guidance for assessing data reliability. However, we limited our review to the ERIS system controls and did not review the IRS systems that provide data to the ERIS (i.e., "feeder" systems). The reliability of the ERIS data output is subject to the reliability of this data input. While the ERIS peripherally identifies weaknesses in these "feeder" systems, it does not validate them, nor does the ERIS ensure they provide accurate data. The scope of the review included reviews of controls within the ERIS and limited data testing. However, unless otherwise noted, we relied on comments provided by IRS management and did not specifically test control weaknesses identified to determine their actual effect.

Fieldwork was conducted between December 1998 and September 1999 in the National Headquarters, the Martinsburg Computing Center (MCC), and the Detroit Computing Center (DCC). We reviewed ERIS data as of March 1999, which was compiled and reported during September 1999, because the process to compile and generate reports can take several months to complete. The audit was performed in accordance with Government Auditing Standards.

Details of our audit objective, scope, and methodology are presented in Appendix I. Major contributors to the report are listed in Appendix II. Appendix IV provides a brief overview of principles for assessing data reliability and a glossary of terms.

Background

The ERIS is an automated system developed to track IRS enforcement revenues and associated costs. Enforcement revenues include the assessment and collection of all dollars resulting from IRS enforcement efforts. The ERIS works by compiling enforcement data from various IRS functional sources, including Examination, Office of the Chief Counsel, Appeals, Collection, and the Information Returns Program (IRP), with data from the Masterfile and Non-Masterfile into a comprehensive database. The data are merged to create summary databases from which ERIS reports are generated. This process can take several months to complete.

All information compiled by the ERIS (such as dollars collected for a specific tax year) is cumulative and subject to constant change as new information (such as additional dollars collected and previously pending or unpostable transactions) is submitted from IRS feeder systems. As such, the ERIS provides insight into enforcement activities for a given point in time. For example, as of December 31, 1997, the ERIS reported the following approximate numbers for Fiscal Year (FY) 1997 Examination function actions: closed 1.6 million cases; assessed $18.4 billion in additional taxes, penalties, and interest; and collected $9.8 billion from balances due. For the same FY period, the Collection function secured approximately 3.3 million returns and collected about $30 billion.

The Department of the Treasury and the IRS use the ERIS for planning and budgeting. The ERIS also supplies data for the four IRS business units and is cited as a data source for measuring future IRS business results.

The Office of Revenue Analysis (ORA), which is based in Washington, DC and is under the Office of the Chief Financial Officer (CFO), has primary responsibility for the ERIS. Since September 1998, ERIS processing has been performed at the DCC. Prior to that time, processing was performed at a Department of Justice computer center in Rockville, Maryland.

Results

The ERIS has systemic design and control weaknesses that can affect the reliability of historical and future revenue data and needs to be improved.

ERIS data reliability can be separated into opinions based upon the two types of data (revenues and costs) it was intended to compile and report. Regarding revenue data, we cannot issue an unqualified opinion on data reliability because the ERIS contains some control weaknesses and the reliability of systems that supply data to the ERIS has not been determined. This stipulation does not render ERIS revenue data unusable; however, it may affect the ERIS’ usefulness to interested parties.

Regarding cost data, we cannot express an opinion on data reliability because the ERIS does not compile or report complete cost data associated with enforcement actions. Total unrecognized costs for FY 1997 were at least $2.1 billion, $582.5 million of which represented indirect costs. The ERIS does report direct hours for the Examination, Office of the Chief Counsel, and Appeals Divisions; however, these hours are not converted to costs. The omission of complete cost data is contrary to the original intent of the ERIS.

Controls over computer system accesses and error resolution need improvement. For example, computer audit trails are not effectively reviewed. Also, timing and processing differences between the ERIS and related systems are not routinely reconciled. While these differences do not materially affect revenue dollars reported by the ERIS, they do affect the allocation of dollars reported by tax year and the functional offices shown as responsible for the enforcement actions.

Controls Over Enforcement Revenue Information System Data Can Be Strengthened

Computer system controls include general and application controls, which together provide reasonable assurance that computer-based data are complete, valid, and reliable. General controls are those that apply to an overall computer operation, such as management oversight of the system’s performance, assignment of responsibilities, and physical security. Application controls are those that apply to a specific application, such as testing software modifications before implementing them and documenting program changes.

We determined that most of the ERIS general and application controls are effective, as noted below.

• ERIS functional requirement and data collection requirement documents are up-to-date. This control indicates programming updates are properly documented when changes are made.
• ERIS computer system user and operating manuals are up-to-date and available to applicable parties. Adequate access to these documents decreases the possibilities of operator error.
• System documentation contains up-to-date reporting of system analysis testing. Periodic analysis testing decreases the possibility of undetected processing and programming errors.
• System documentation contains up-to-date system modification requests, as evidenced by Program Change Requests and ERIS Support Requests. A paper trail of programming changes is important because it identifies the reasons behind system error corrections and documents proper management authorizations.
• Periodic security risk assessments are conducted of contractor facilities in the ERIS processing environment. Periodic risk assessments are objective reviews of the system by its owners that identify problems before they become significant. For example, a risk assessment can identify users having inappropriate access capabilities.
• Limited testing of the ERIS’ processing of 25 judgmentally selected taxpayer modules containing revenue data identified no processing discrepancies. This test was limited to the processing functionality of the ERIS and did not assess the accuracy of revenue data.

While most of the ERIS general and application controls are adequate, we identified two specific control areas needing improvement. One control involves audit trails, which provide records of operator and system activity. These records provide invaluable day-to-day histories of system operations and account for actions both taken and omitted. The other involves a control to resolve processing errors.

ERIS management decided that some users in the Program Office and some contractor employees needed unlimited system access for creating, updating, and/or deleting data within ERIS databases. This capability violates the principle of separation of duties and creates a potential risk of unauthorized changes to ERIS data. Certain permissions should be limited and provided only to computer system users whose jobs require them to have these access rights.

In its system risk assessment report prepared in 1995, the IRS acknowledged system vulnerability and offered this corrective action:

ERIS application software is a proprietary, contractor developed application. The ERIS application does not incorporate security features such as user authentication, passwords, or provide an audit trail predicated upon those privileges. Protection of the application and its data is dependent upon the mainframe security package. The ERIS SSA [system security administrator] should work to get the monitoring of the security incident log decentralized to the department (ERIS) level to ensure proper analysis of these events is made.

The actions noted above, namely the various security features and proper analysis of the security incident log, were not completely taken. Consequently, the IRS still does not have an effective audit trail control. The ERIS is capable of tracking all operator actions. Paper audit trails can be and have been created in the past. However, ERIS management did not see the need to review audit trails on a routine or ongoing basis. Not establishing an effective audit trail control leaves ERIS management at risk for undetected, inappropriate system accesses and data manipulation.

Requests for ERIS system access are made via Automated Information System User Registration/Change Requests (Form 5081). Decisions involving access to the ERIS are made in the ERIS Program Office and implemented in the DCC. ERIS controls ensure only authorized persons can access the ERIS; however, we identified the following weakness.

DCC policy stipulates that non-IRS employee (e.g., contractor) keystrokes in the DCC computer environment be routinely tracked. An automated tool called U-Audit is used for this task. We reviewed all 31 applicable Forms 5081 at the DCC for ERIS users. Form 5081 information for contractor employees was improperly input by DCC personnel, resulting in these individuals not being correctly flagged for U-Audit. Because this control was not correctly used, no audit trail was captured at DCC for these contractors.

ERIS processing defines two types of data units: "cases" and "tax modules." "Cases" contain information, such as type of return, tax period, or location code. "Tax modules" contain accounting information expressed in terms of volumes and counts, such as dollars assessed.

Sometimes cases and tax modules coming from the ERIS feeder systems have conflicting and/or missing information. In these instances, the ERIS attempts to recognize dollars and rejects other incomplete information. These rejects are then compiled on the error or unmatched registers.

The ERIS classifies errors as "E," "I," and "W":

• "E" errors are true errors; cases are not processed.
• "I" errors are informational only; both cases and tax modules are processed.
• "W" errors are warnings; both cases and tax modules are processed.

Tax modules (e.g., revenue dollars) were processed for all error classifications except Non-Masterfile (NMF) Error Codes 113 and 114. At the time of our review, these two types of NMF error codes rejected both accounting and inventory information, which meant revenue dollars would not be included in ERIS reports. According to ERIS management, these 2 codes involved 7,598 cases, less that 1 percent of the cases in the system. During the course of our review, ERIS management stated they made programming changes that reduced the register inventory for one of these error codes. The inventory volume in the remaining error code is very small and will not materially affect ERIS revenue reports.

According to ERIS management, as of March 1999, the ERIS had 53,119 cases with "E" type errors, out of approximately 236 million cases in the system. ERIS error registers have been growing exponentially since the creation of the system because errors are not always reviewed or resolved. In addition, the ERIS had accumulated 1.05 million transactions (representing $25.5 billion) in "unmatched transaction" registers, out of about 1.9 billion transactions, since its creation. ERIS management advised us that they had identified the sources of 237,000 of these cases (representing $7.1 billion) as being Audit Information Management System (AIMS)- or IRP-related.

Another type of timing difference involves an assessment that is made after a tax module is closed off the AIMS. The Masterfile will show the additional assessment, but the AIMS will not. ERIS management stated that there will always be differences between the AIMS and the Masterfile for such transactions but did not provide us with the total number of the 1.05 million transactions that cannot be matched.

ERIS management does not work these error and unmatched transaction registers because of the volume of transactions and the difficulty of resolving cross-functional reconciliations. In addition, according to ERIS management, even though the 1.05 million transactions could not be matched, the revenue was reported. Although the volumes in these registers are not material in comparison to the overall volume of cases the ERIS processes for any given year, unresolved errors and unmatched transactions are a systemic processing weakness which can represent material, hidden discrepancies.

Currently, ERIS management resolves ERIS errors through software upgrades. The entire ERIS database is recompiled at every software upgrade. These upgrades may involve new processing changes which process data elements previously coded as errors (thereby resolving the errors). However, new error files are created each time the database is reprocessed. These files will include errors from previous periods still unresolved. Some unmatched transactions are never resolved because they cannot be corrected with system upgrades.

In addition to unmatched transactions, there are also unresolved discrepancies between the functional systems and the ERIS. According to ERIS management, the ERIS may report the correct revenue amount, tax period, and district office but may not be able to identify the program area within the functional office that conducted the enforcement effort.

In their Computer Security and Privacy Plan (prepared in 1995), ERIS management made the following assumption and claim regarding IRS feeder systems:

It is assumed that the feeder systems providing the information to ERIS have been properly tested and maintained to supply valid data to the system. ERIS is validated against its feeder systems and is itself validated to ensure proper processing of the data.

As part of a recommended corrective action from a previous audit, IRS enforcement functions were requested to validate applicable data on the ERIS. This reconciliation was conducted in 1996. However, differences between the ERIS and feeder systems were not resolved and a new reconciliation has not been attempted.

There is no structured, continuous reconciliation process between the ERIS project staff and the IRS functions to properly correct the misidentified program areas. Moreover, ERIS management has no plans to resolve unmatched and error transactions on a large scale. Instead, they will continue to address errors and unmatched transactions through ongoing means, such as making annual and periodic system programming upgrades and taking small, judgmental samples to detect additional errors.

1. The CFO should evaluate the cost-effectiveness of resolving the following control weaknesses or of accepting the risk that they could be causing unreliable data:

• Having the capability to read, write, and alter work of all other users and contractor employees.
• Not maintaining and reviewing audit logs of all user accesses.
• Not conducting periodic (e.g., annual) reconciliations of errors and unmatched transactions from the feeder systems. Such reconciliations should include statistically valid accuracy sampling of ERIS data.

Management’s Response: IRS management’s initial response did not adequately address the recommendations in the report. They are currently revising and reissuing their response.

The Purpose of the Enforcement Revenue Information System Should Be Reevaluated

The stated purpose of the ERIS is to compile and report revenues and costs associated with IRS tax enforcement actions. In its 1996 report, an IRS Investment Evaluation Review team reported the ERIS had accomplished its "objective." This team offered the following paraphrase of the ERIS objective, as presented in the original 1990 ERIS Requirements Application Package:

The primary objective of ERIS was the development of a comprehensive, accessible, interfunctional Management Information System, which eliminated double counting of enforcement revenue. ERIS’ goal was to be the [Internal Revenue] Service’s source of corporate information on revenue and costs associated with enforcement activities.

However, the ERIS has not fully accomplished its purpose because it was not provided data to calculate all applicable costs. Although the ERIS does report direct hours (time specifically spent on case work) for the Examination, Office of the Chief Counsel, and Appeals Divisions, the ERIS reports do not convert these hours into costs. ERIS management acknowledged that the ERIS was designed to process all applicable costs and that the incompleteness is primarily caused by the functions not providing necessary cost data to the ERIS. As a result, uninformed users may believe that the ERIS compiles and reports cost information, which it does not. For FY 1997, the missing (unreported) costs were at least $2.1 billion, $582.5 million of which represented indirect costs.

According to ERIS management, approximately two-thirds of enforcement-related work effort comes from the Examination function and one-third from the Collection function. Customer Service function efforts are already included in these above percentages because large portions of Customer Service work are either Examination- or Collection-oriented.

For the Examination function, the ERIS receives two types of cost information from the AIMS: Employee Grade and Total Hours Worked. While the ERIS is designed to handle additional costing elements and calculations, these features are not used because cost data are not received from the functional systems.

The Entity is a Collection-developed information system tool, similar to the AIMS, that provides Collection direct time (by tax module). The ERIS does not use Entity information and has no current plans to use it in the future.

When we brought the cost issue to ERIS management’s attention, they agreed that ERIS documents should not have references to cost data, except for direct hours. The ERIS information that is part of the CFO’s website, which the program office developed during the course of our audit, does not contain references to cost data, only to direct hours. However, there are other documents (e.g., the Internal Revenue Manual) that still show the ERIS’ purpose as tracking enforcement-related revenues and costs. Management indicated they plan to correct these other documents concerning the omission of cost data.

IRS Performance Measures have changed, reflecting a new organizational mission and culture, and they attempt to balance business results with customer and employee satisfaction. Because the IRS is using ERIS data for one of the new business results measures, it is important that the IRS contribute adequate resources and oversight to ensure the future reliability of ERIS data.

The CFO should:

2. Evaluate what level of oversight and resources are needed to ensure the ERIS can support these new measurement priorities.
3. Either incorporate cost information into the ERIS, in keeping with its stated purpose, or alter its stated purpose or mission to accurately reflect the ERIS’ actual use. These actions should also involve ensuring all instructional materials (e.g., the Internal Revenue Manual) reflect management’s decision on the cost information.

4. Appropriately qualify all applicable ERIS reporting documents to reflect that the system does not contain cost information. These disclosures should also refer users to the unmatched transactions and error registers, should the information in these registers affect the use of the ERIS data in making management decisions.

Conclusion

We cannot issue an unqualified opinion on the reliability of ERIS revenue data because the reliability of systems that supply data to the ERIS has not been determined and the ERIS contains some control weaknesses. This conclusion does not render ERIS revenue data unusable; however, it may affect how management uses ERIS data in making decisions. Additionally, ERIS cost data include only direct hours, not other cost data. Omitting complete cost data is contrary to the original intent of the ERIS.

The two specific control areas needing improvement involve audit trails and error resolution. Computer audit trails are not effectively reviewed, and timing and processing differences between the ERIS and related systems are not routinely reconciled. These control weaknesses do not materially affect revenue dollars reported, but they do affect the allocation of dollars reported by tax year and the identification of both methods of collection and functional offices responsible for the associated enforcement actions.

The CFO needs to evaluate the cost-effectiveness of resolving the ERIS control weaknesses. The CFO should also either incorporate all relevant cost data into the ERIS or alter all statements of purpose to accurately reflect the system’s actual use. In addition, the IRS should disclose on all applicable reports that cost data are incomplete.

Detailed Objective, Scope, and Methodology

Our overall objective was to assess Enforcement Revenue Information System (ERIS) data reliability. To achieve this objective, we developed tests based on the General Accounting Office (GAO) guide Assessing the Reliability of Computer-Processed Data (dated April 1991), which includes the following key steps for assessing data reliability:

• Determine how computer-processed data will be used and how they will affect the job objectives.
• Obtain a thorough understanding of relevant system controls.
• Test data for reliability.

Fieldwork was conducted in the National Headquarters, the Martinsburg Computing Center (MCC), and the Detroit Computing Center (DCC). We did not evaluate ERIS data received from other data systems.

To accomplish our overall objective, we conducted the following tests:

1. To determine how ERIS data are used and how reliance on the ERIS data is relevant, we:

1. Researched the following sources for ERIS-related information:

• The Internal Revenue Manual.
• The Treasury Inventory, Tracking, and Closure System.
• The Chief Financial Officer (CFO) Home Page (the Office of Revenue Analysis and the ERIS are included on the CFO home page).

1. Reviewed Internal Revenue Service (IRS) and GAO documents for ERIS-related information.

1. Made visitations to the National Headquarters, ERIS program office, the MCC, and the DCC and interviewed employees assigned ERIS-related work to determine the nature, purpose, effectiveness, and future plans of the ERIS.

2. To determine whether the general controls are adequate to ensure the accuracy, completeness, integrity, and authenticity of the data, we:

1. Secured, reviewed, and analyzed ERIS-related contracts executed between the IRS and applicable outside parties.
2. Analyzed the dates and types of ERIS-related documents received by and sent to the ERIS program office, including bi-weekly written progress reports and meeting minute notes (conducted both in-person and through teleconferencing) for the period October 1997 to March 1999.
3. Determined whether functional requirements and data collection requirements documents are up-to-date.

1. Determined whether computer system user and operating manuals are up-to-date and available to applicable parties.
2. Determined whether system documentation contains up-to-date reports of system analysis testing.
3. Determined whether system documentation contains up-to-date system modification requests, as evidenced by Program Change Requests and ERIS Support Requests.

1. Reviewed reports, memoranda, and other written communications originating in the IRS related to the review of contractor employees in the areas of privacy, disclosure, and security.
2. Determined whether periodic security risk assessments are performed and documented for offices in the ERIS processing environment.
3. Determined how and in what circumstances general controls are bypassed or overridden.
4. Made visitations to the MCC, the DCC, and the ERIS program office and interviewed employees assigned ERIS-related work on a variety of general control topics, such as separation of duty and security-related issues.
5. Assessed physical access to IRS work areas in the ERIS-related environment.
6. Determined whether electronic access to data is controlled, tracked, and monitored in the ERIS-related environment.

1. Reviewed audit trails.
2. Reviewed applicable Automated Information System User Registration/Change Requests (Form 5081).

1. Identified the status of actions taken to address general control-related ERIS recommendations previously made by audit organizations.

3. To determine whether application controls are sufficient to protect data accuracy, completeness, integrity, and authenticity, we:

1. Prepared high-level analyses of ERIS input, processing, and output for each of the following feeder systems/source offices:

• Audit Information Management System.
• Individual Masterfile.
• Business Masterfile.
• Office of the Chief Counsel.
• Appeals function.
• Collection function.
• Non-Masterfile (Information Returns Program – Underreporter).

1. Made visitations to the MCC, the DCC, and the ERIS program office and interviewed employees assigned ERIS-related work on a variety of application controls topics.
2. Judgmentally selected an extract of 25 taxpayer modules, which contained all 12 possible valid ERIS transaction codes.

1. Compared ERIS-prepared "detail record" information for the 25 selected sample modules to data housed on the Masterfile, as evidenced by hard-copy Masterfile transcripts, and reconciled discrepancies.
2. Used test-deck testing to assess the existence and effective operation of controls.

1. Determined the effectiveness and completeness of the ERIS quality system testing by reviewing documents of ERIS quality system testing to determine if documentation was up-to-date. We also assessed the extent and validity of testing performed on the initial ERIS and each subsequent upgrade by reviewing the system test reports and related documentation.

Major Contributors to This Report

Gordon C. Milbourn III, Associate Inspector General for Audit (Small Business and Corporate Programs)

M. Susan Boehmer, Director

Nancy A. Nakamura, Director

Gary L. Swilley, Audit Manager

Kenneth L. Carlson Jr., Senior Auditor

Anthony J. Choma, Senior Auditor

James S. Mills Jr., Senior Auditor

Lawrence R. Smith, Senior Auditor

Jeffrey E. Williams, Senior Auditor

Phillip H. Dearth, Auditor

Charlene Riley, Auditor

Rashme Sawhney, Auditor

Mildred R. Woody, Auditor

Report Distribution List

Deputy Commissioner Operations C:DO

Commissioner, Large and Mid-Size Business Division LM

Commissioner, Small Business/Self-Employed Division S

Chief Financial Officer C:DO:CFO

Director for Financial Analysis CFO:A

Director, Legislative Affairs CL:LA

Director, Office of Program Evaluation and Risk Analysis M:O

National Taxpayer Advocate C:TA

Office of the Chief Counsel CC

Office of Management Controls CFO:A:M

Audit Liaison: Office of Chief Financial Officer C:DO:CFO

Auditing Data Reliability - Guidance and Glossary of Terms

The General Accounting Office (GAO) provides guidance for assessing data reliability, including:

• Determining how computer-processed data will be used and how they will affect the job objectives.
• Obtaining a thorough understanding of relevant system controls.
• Testing data for reliability.

We used the GAO’s definitions for the following terms relating to data reliability.

Data reliability is a state that exists when data are sufficiently complete and error-free to be convincing for their purpose and context. It is a relative concept that recognizes that data may contain errors as long as they are not of a magnitude that would cause a reasonable person, aware of the errors, to doubt its validity, completeness, and accuracy.

Computer system controls are policies and procedures which provide reasonable assurance that computer-based data are complete, valid, and reliable, and include general and application controls.

General controls are the structure, methods, and procedures that apply to an overall computer operation. They include organization and management controls, security controls, and system software and hardware controls. Specific areas falling under the umbrella of general controls include:

• Management commitment to system design and operation: This control includes management’s methods for monitoring and following up on performance, including corrective action on internal audit recommendations and on user complaints.
• Organization of the system functions, including assignment of responsibilities and separation of duties: This control provides that key duties and responsibilities in authorizing, processing, recording, and reviewing transactions are assigned to different individuals.
• Physical security of the computer facility and its components, including restrictions on access: Restricting access helps ensure data reliability by reducing the risk of unauthorized data entry or modification.
• Supervision: Effective supervision requires a clear communication of duties and responsibilities, regular oversight – particularly at critical points – and periodic performance evaluations.

Application controls are methods and procedures designed for specific applications to ensure data origination authority, data input accuracy, processing integrity, and output verification/distribution. They address such issues as:

• Application software and subsequent modifications, authorizations, and testing before implementation.
• Frequency of system modification (and underlying reasons).
• Control and documentation of program changes.
• Review, approval, control, and editing of source transactions for completeness and error prevention.
• Sources and frequency of update for tables used in computer processing.
• Narrative system descriptions and flowcharts.
• Reconciliation of output records to input entries.
• Error detection and correction procedures.
• End-user opinions/assessments of data reliability.

We also used the following terms:

AIMS (Audit Information Management System) – An Internal Revenue Service (IRS) management information system which tracks the status and activity of taxpayer audits.

Application – Any computer software program or any manual operation which performs a specific routine, action, or function.

Audit Trial – A chronological record of system activities that is sufficient to permit reconstruction, review, and examination of transactions, from inception to final results.

Business Masterfile – The IRS database that consists of federal tax-related transactions and accounts for businesses. These include employment taxes, income taxes on businesses, and excise taxes.

Enforcement Efforts/Actions – Activities primarily undertaken by the Examination and/or Collection Divisions to legally enforce the tax laws. These actions are caused by non-voluntary tax reporting, non-compliance with tax laws, and non-payment of tax liabilities, including penalties and interest.

Entity – An IRS management information system which compiles and reports Collection Division direct costs.

Individual Masterfile – The IRS database that maintains transactions or records of individual tax accounts.

IRP (Information Returns Program) – An IRS computer system which compiles and reports on information returns, such as the Statement for Recipients of Dividends and Distributions (Form 1099-DIV).

Masterfile – The IRS database that stores various types of taxpayer account information. This database includes individual, business, and employee plans and exempt organizations data.

Non-Masterfile – Consists of transactions on tax accounts not included on the Masterfile.

Test-deck testing – Simulated tests of computer processing, which can be used to evaluate software programming code. These tests can involve using live or simulated data.

U-Audit – A software program used to capture all accesses (e.g., log-ons) and other actions taken by system users.