The Government Performance and Results Act of 1993 (GPRA) requires the Internal Revenue Service (IRS) to submit annual performance plans to the Office of Management and Budget (OMB) and the Congress. The plans include performance goals and prior performance reports that compare actual performance to annual goals. To implement the GPRA requirements, the IRS developed the Balanced Measurement System composed of three types of performance measures: employee satisfaction, customer satisfaction, and business results. The IRS has contracted with independent vendors to manage the employee satisfaction and customer satisfaction measures. Each IRS business unit and functional organization is responsible for establishing and compiling business results measures.

The objective of this audit was to determine if the Information Systems (IS) organization’s business results measures are adequately defined, meet legal standards, and support the IRS’ Balanced Measurement System. To accomplish this objective, we assessed the effectiveness of the procedures the IS organization developed to collect, monitor, and analyze business results measures information and set baselines for each business results measure.

In March 1999, the IRS began working with Booz-Allen & Hamilton (BAH) to develop a new set of organizational performance measures for the IS organization. BAH delivered its proposal for measuring the IS organization’s performance in October 1999. The IS organization’s Performance Assessment Office (PAO) further refined these measures and, in March 2000, the Chief Information Officer (CIO) and the Commissioner approved the IS organization’s measures.

After the IS balanced measures approval, the PAO and the IS line organizations (the actual IS operating divisions and offices) began gathering data and developing the data analysis to establish performance baselines for its 16 business results measures and 44 diagnostic indicators that support these measures. The PAO prepared the Information Systems Balanced Measures Data Dictionary and a Policies and Procedures document as references for the IS managers who will collect data and for PAO personnel who will monitor and report on the business results measures. The November 2, 2000, IS Business Performance Review (BPR) included the first presentation of the business results measures as an attribute of IS performance.

While much has been accomplished, additional work is needed to meet the business measurement needs of IS and IRS management. The PAO needs to work with the IS line organizations to complete development of the business results measures and diagnostic indicators, present business results measures accurately and completely in its business performance review, and consider developing business results measures to address identified high-risk management challenges.

The GPRA requires that the annual performance plans include a description of the means used to verify and validate business results data. The PAO and the IS line organizations have had difficulty accomplishing this activity because the systems the IS organization uses to collect data were not designed to calculate the business results measures.

Our analysis showed that 3 of the 16 business results measures did not include complete data for adequate reporting. Without completed business results measures, the PAO cannot provide data to IS managers that could be used to improve organizational performance. The business results measures will be used to determine if proposed service levels to the IRS’ business units and functional organizations are achieved.

The PAO is in the process of reviewing the 44 IS diagnostic indicators to evaluate which indicators will provide adequate support for the business results measures. At the time of our review, 26 of the 44 (59 percent) diagnostic indicators did not have data sources available to provide information to support the business results measures.

Also, there were 24 diagnostic indicators presented in BAH’s October 1999 proposal that the PAO removed from use by March 2000. The PAO removed these indicators because of inadequate data sources or a vague understanding of the indicator’s significance. Our analyses and conversations with the PAO managers and analysts indicated that 18 of these removed indicators have value and potential data sources to support the business results measures. Without good diagnostic indicators, managers will be unable to determine the reasons for sub-optimal performance identified in the business results measures.

The IS organization submits a quarterly BPR report to the Commissioner which includes the IS business results measures. The BPR provides the Commissioner and IS management with performance data so that they can make informed decisions about the organization’s activities.

The IS organization’s first quarterly BPR was presented to the Commissioner in November 2000. The IS organization intends to use the 16 business results measures in this BPR to set baselines for future years’ results. This BPR showed that all business results measures had data sources, baselines/targets, and available data. However, our analysis showed that one measure was not clearly reported, three other measures did not include all relevant data in the presentation, two measures did not have complete data available to calculate and meet the measure’s definition, and one measure was not presented at all. Without accurate and complete business results measures in its BPR, IS and IRS management could make inappropriate decisions.

The Fiscal Year 2001 Congressional Justification, which proposes agency budget initiatives, includes a performance plan that describes 13 management challenges/high-risk areas for the IRS. The IS organization shares responsibility for 3 of the 13 challenges identified. However, the IS business results measures do not specifically address these 3 challenges. The IS organization’s responsibilities are: accuracy of the inventory system for automated data processing equipment, information systems security controls, and updating programs for filing season readiness.

By not including specific performance measures in the performance plan, the IS organization is not reporting to the Congress what it is doing to address these challenges. Without specific measures, it is difficult, if not impossible, for the Congress to assess progress in addressing major management problems and to hold agencies accountable.

To improve the ability of IS managers to assess program performance and make more informed decisions in directing operations, the CIO should direct the PAO to further develop the IS business results measures and diagnostic indicators. To accomplish this, the PAO should work with the IS line organizations to research information systems data sources for use in compiling the measures and indicators. The PAO also needs to follow GPRA guidance and ensure that it documents the verification and validation of these data to provide assurance of its accuracy and completeness.

To ensure that the BPR provides relevant information for management decision-making purposes, the CIO should direct the PAO to complete efforts to clearly and fully define the IS business results measures and include all relevant data for measure calculation. It also needs to disclose the data sources and limitations of the business results measures in the BPR presentation.

To meet performance assessment requirements of the Congress, General Accounting Office, OMB, and Department of the Treasury, the CIO should develop business results measures that assess the IS organization’s performance in meeting the management challenges. The IS organization and the PAO should reassess the measures annually, developing measures, as necessary, that address new challenges as they arise.

Management’s Response: Management agreed that they need to perform additional work to further develop business results measures and improve their data to meet the business needs of the IS organization and IRS management. Management’s complete response to the draft report is included as Appendix VII.