Executive Summary

Objectives and Scope

Background

Results

Conclusion

Appendix I – Detailed Objectives, Scope, and Methodology

Appendix II – Major Contributors to This Report

Appendix III – Report Distribution List

Executive Summary

For more than a decade, the Internal Revenue Service (IRS) has been attempting to modernize its outdated, paper-intensive tax processing system. After 10 years and over $3 billion spent with minimal improvement and intense scrutiny from the Congress, the IRS agreed to use a contractor to help develop modernized systems. The IRS is currently in the early phases of its new systems modernization effort. This multi-billion dollar effort is projected to last up to 15 years.

Audits of the previous computer systems modernization initiatives identified serious management and technical weaknesses. This audit is the first in a series of audits to evaluate the IRS’ oversight of the new systems modernization effort. The objectives were to determine the adequacy of 1) the organizational structure developed to oversee the systems modernization effort, 2) organizational staffing, 3) performance monitoring capabilities, and 4) risk management capabilities.

The IRS has made progress in correcting organizational weaknesses from past systems modernization efforts by ensuring that top level IRS executives are heavily involved in the systems modernization and by recognizing the need to develop program management capabilities, risk management processes, and quality assurance policies and procedures.

However, we found that the oversight of the systems modernization effort has been hampered by the lack of a stable program management organization. Program management staffing needs have not been determined, roles and responsibilities inside the IRS and between the IRS and the contractor are not yet clearly defined, and key processes such as risk management and performance monitoring need to be improved.

A recent IRS review of two key systems modernization initiatives found that a significant number of the work products required during the planning phases of these projects had not been completed. As a result, the IRS had to scale back or delay delivery of several modernization initiatives which were intended to provide improved service to taxpayers in the 2001 Filing Season. Examples of initiatives that have been delayed include: 1) a telephone application that would allow taxpayers to determine whether their individual tax returns have been received, 2) an Internet application that would allow taxpayers to determine the status of their refunds, and 3) an application that would provide for automated delivery of taxpayer account information. Had an effective performance monitoring process been in place, these problems could have been identified much sooner and corrective actions taken without the substantial delays the projects are now facing.

To address these and other problems, the IRS is revising its role and the role of the contractor to assign clear lines of accountability. The IRS now sees its primary role as defining what its business needs are, and the contractor is accountable for delivering the systems to meet those needs. The IRS has other initiatives underway which it believes will help address these concerns. We have incorporated these actions throughout the report.

There have been several changes to the IRS Program Management Office (PMO) developed to oversee the information systems modernization effort and, as of the end of our audit, the IRS did not have an approved organizational structure in place. Without a stable PMO, the IRS has encountered difficulties in timely requesting the release of modernization funds from the Congress, conducting complete quality assurance reviews, fully implementing risk management capabilities, defining architecture standards, and effectively monitoring contractor performance.

Since a stable PMO has not been formed, the IRS has not determined the roles and responsibilities needed to oversee the modernization contractor. Without defined roles and responsibilities, the IRS has not been able to create a staffing plan for the PMO. In addition, a staff skills analysis and training plan have not been developed. Without skills analyses, the IRS cannot ensure that needed skills and abilities to oversee the contractor are available to the PMO.

The IRS has not created policies and procedures to provide an adequate framework for overseeing systems modernization. The IRS was conducting reviews of the modernization contractor ineffectively and did not have an automated method for collecting and disseminating contractor performance data. In addition, no formal assessment had been made of the modernization contractor to ensure that information provided to the IRS was accurate and complete. Due to the lack of a stable PMO, contractor performance has not been closely monitored and modernization projects have approached the end of planning without completing all the required work products.

While the IRS has developed policies and procedures, risk management has not been fully implemented and training has not occurred. Since the PMO has been unstable, a central office responsible for risk management has not been approved. Without a fully implemented risk management framework, there is no assurance that PMO (program-wide) and project level risks are being identified, addressed, and monitored.

We recommend that IRS management stabilize the PMO designed to oversee the systems modernization effort and develop plans to ensure the adequacy of PMO staffing. We also recommend establishing offices responsible for developing and enhancing performance monitoring and risk management capabilities. This should result in improved service to taxpayers through the implementation of quality modernization projects.

Management’s Response: We issued a draft of this report to IRS management on May 2, 2000, with a June 1, 2000, response period. An extension was granted until June 16, 2000. However, management’s response was not available as of the date this report was released.

Objectives and Scope

This audit is the first in a series of audits to evaluate the Internal Revenue Service’s (IRS) oversight of the new systems modernization effort. The objectives were to determine the adequacy of:

• The IRS’ organizational structure developed to oversee the systems modernization effort.
• IRS Program Management Office (PMO) staffing.
• Performance monitoring capabilities.
• Risk management capabilities.

The Treasury Inspector General for Tax Administration (TIGTA) is required by law to evaluate the adequacy and security of IRS technology. This audit was performed in accordance with Government Auditing Standards.

The audit was conducted in the Office of the Chief Information Officer in New Carrollton, Maryland from October 1999 through March 2000. At the IRS’ request, we reviewed actions being planned or taken on the issues we detail in this report. These actions are presented as Current Efforts Underway in the Results section of the report.

The scope of the audit included discussing the status of the organizational structure, staffing, risk management, and performance monitoring with key IRS and contractor officials and reviewing available documentation. Due to the in-progress nature of the areas reviewed, we will perform a future audit to assess the progress made on these topics.

Details of our audit objectives, scope, and methodology are presented in Appendix I. Major contributors to this report are listed in Appendix II.

Background

For more than a decade, the IRS has been attempting to modernize its outdated, paper-intensive tax processing system. Previous audits identified serious management and technical weaknesses in the Tax Systems Modernization effort. After 10 years and over $3 billion spent with minimal improvement and intense scrutiny from the Congress, the IRS agreed to use a contractor to help develop modernized systems.

The IRS is currently in the very early phases of a new systems modernization effort. Computer Sciences Corporation (CSC), also known as the PRIME contractor, was awarded the PRIME contract on December 9, 1998. The 15-year PRIME contract is potentially worth $5 billion.

As of the end of our audit, systems modernization initiatives had been funded for approximately $68 million. The IRS recently requested an additional $176 million from the Congress to continue building program management capabilities while allowing low risk modernization projects to progress.

Several different organizations with different roles are involved in the IRS’ systems modernization. Under the PRIME contract, CSC is responsible for designing new systems to meet IRS business needs, developing these systems, integrating them into the IRS, and ultimately transferring operation of these systems to the IRS.

The MITRE Corporation (MITRE) is also under contract to assist the IRS with the systems modernization. MITRE provides the IRS with specific expertise in establishing strategic priorities, making investment decisions, evaluating proposals, managing the systems modernization program, monitoring contracts, performing specific research, and conducting testing activities.

Several projects are currently underway as part of the IRS’ systems modernization. Two of the main projects, Customer Communications and e-Services, are expected to achieve customer service improvements in the next two filing seasons.

The focus of this project is to increase customer service by providing the capability to route taxpayer calls to any IRS employee at any location. In addition, this project will concentrate on improved self-service telephone and Internet services for taxpayers. This project is currently being evaluated to determine readiness to move out of the planning phases into the development phase.

The e-Services project is one of several initiatives designed to help the IRS meet its goal of 80 percent electronic interactions with taxpayers by 2007. The project is focusing on electronic filing, education and self-help applications, and increased use of secure e-mail to deliver requested information. This project is currently being evaluated to determine readiness to move out of planning phases into the development phase.

Specific funding, software development, and capability assessment processes are being followed or planned for the IRS’ systems modernization.

The Congress places funds for the IRS’ systems modernization activities in an Information Technology Investment Account (ITIA). The IRS must then submit a spending plan requesting that funds from the ITIA be withdrawn for use by the IRS. The spending plan is required to be reviewed by the General Accounting Office (GAO) and approved by the Department of the Treasury, the Office of Management and Budget (OMB), and the Congress.

The IRS requires the PRIME contractor to follow the Enterprise Life Cycle (ELC). The ELC is a structured business systems development method that requires specific work products to be developed during different phases of the development process.

The IRS plans to use the Software Engineering Institute’s Capability Maturity Model for evaluation of the IRS’ and PRIME’s ability to acquire and design the software needed to meet the modernization objectives. The Capability Maturity Model is a structured process that helps organizations improve their abilities to consistently and predictably acquire and develop high-quality information systems. Organizations that have implemented the Capability Maturity Model processes have seen dramatic improvements in their abilities to meet planned time frames, reduce errors, and increase value on dollars invested.

Results

The IRS has taken several significant steps to correct organizational weaknesses from previous systems modernization efforts. Some of these steps include:

• Ensuring that executives from the business units and the Information Systems organization are involved in the modernization process. In December 1998, the Commissioner created the Core Business Systems Executive Steering Committee. The Committee was established to provide strategic direction, make decisions on which projects should be funded to ensure projects deliver maximum value, and approve projects at critical milestones. The Committee has met monthly since April 1999, and the Commissioner and top level IRS executives are present at each meeting. The Commissioner also created Executive Steering Committees (Sub-ESCs) to oversee a project or set of projects. Several Sub-ESCs have been meeting regularly, reviewing projects at critical milestones, and ensuring that the IRS business (user) management team is involved in systems modernization.
• Recognizing the need to build program management capabilities. Recognition of unhealthy trends and learning from the past are fundamental to successfully managing a long-term modernization effort. In its recent request for additional funding from the ITIA, the IRS documented a set of lessons learned including the fact that modernization projects are ahead of the IRS’ ability to oversee them. To address this risk, the IRS has hired several executives with systems modernization experience to help with program management. The IRS recently performed reviews to determine if modernization projects complied with the ELC. The IRS also plans to conduct milestone readiness reviews before allowing modernization projects to proceed from planning into development. In addition, plans for modernization projects are being scaled back, in part because of the IRS’ inability to oversee them all.
• Drafting risk management and quality assurance policies and procedures. The IRS and MITRE have developed risk management policies and procedures that provide a good start to creating a risk management framework. The framework includes risk management procedures for IRS top management, the PMO, and project levels. Essential elements of risk management are defined. Quality assurance policies and procedures have also been drafted. The policies and procedures include documentation on how to conduct and document quality assurance reviews.

While the IRS has made progress in correcting organizational weaknesses from previous systems modernization efforts, significant risks must be addressed to ensure the success of the current systems modernization effort. We determined that:

• A stable organization structure has not been established to oversee the systems modernization effort.
• Adequate staffing levels for the PMO have not been determined.
• An adequate performance monitoring framework has not been established.
• A risk management framework has not been fully implemented.

A Stable Organization Structure Has Not Been Established to Oversee the Systems Modernization Effort

An approved program management organizational structure has not been implemented. Without a stable PMO, the IRS has encountered difficulties in timely requesting the release of ITIA funds, conducting complete quality assurance reviews, fully implementing risk management capabilities, and defining architecture standards.

The current modernization projects have not had the aid of program management oversight during planning stages. As a result, these modernization projects have approached the development stage without completing required work products, and the IRS had to delay delivery of some modernization initiatives. These initiatives were intended to provide improved service to taxpayers in the 2001 Filing Season.

Examples of initiatives that were delayed include: 1) a telephone application that would allow taxpayers to determine whether their individual tax returns have been received, 2) an Internet application that would allow taxpayers to determine the status of their refunds, and 3) an application that would provide for automated delivery of taxpayer account information.

During our audit, the PMO responsible for overseeing the systems modernization underwent several changes in its number of operating divisions and key personnel. At the end of our audit, the IRS did not have an approved organizational structure with mission statements, charters, and defined roles and responsibilities.

The OMB requires agencies to establish information system management oversight mechanisms that ensure major information systems projects proceed in a timely fashion towards agreed-upon milestones in the information systems life cycle, meet user requirements, and deliver intended benefits to the agency and affected public.

In December 1998, the Commissioner documented the need for a program management structure to oversee systems modernization. In October 1999, MITRE raised the lack of a PMO as one of the six key risks to systems modernization. At that time, the IRS was working toward implementing a program management organization known as the Enterprise Program Management Office.

In January 2000, the IRS decided that the roles of the IRS and PRIME contractor needed to be better defined. Until this point, the IRS and PRIME contractor were working jointly to design and deliver modernization projects. The IRS determined that it needed to focus on defining what its business needs and standards are and then acquiring the means to meet those needs through the PRIME contractor. In essence, the IRS would serve in an acquisition and oversight role, and the PRIME contractor would be accountable for designing and delivering modernized systems. The Enterprise Program Management Office was not an adequate organizational model to accomplish these revised roles, so the IRS began working toward a new program management organization known as the Business Systems Modernization Office (BSMO).

The IRS identified the lack of a stable PMO as a key systems modernization risk in February 2000. The IRS states that the consequence of this risk will be uncoordinated decision making and failure to provide consistent direction to the systems modernization effort.

Current Efforts Underway - The IRS is working on the following areas to correct identified weaknesses:

• The IRS has proposed a new organizational structure to oversee systems modernization.
• MITRE has developed a document describing the roles and relationships of organizations involved in the systems modernization.
• The IRS has planned a meeting of IRS executives to obtain consensus about the organizational structure and interrelationships among the organizations involved in the systems modernization effort.

To ensure the IRS has an organization in place to effectively oversee the systems modernization effort, the Commissioner should take the following actions:

1. Stabilize the PMO responsible for oversight of systems modernization.
1. Approve an organizational structure for the PMO.
2. Create mission statements and charters for each major group in the approved organizational structure.
3. Define roles and responsibilities for each group within the approved organizational structure and how they will interact with other organizations involved in the systems modernization.

Management’s Response: We issued a draft of this report to IRS management on May 2, 2000, with a June 1, 2000, response period. An extension was granted until June 16, 2000. However, management’s response was not available as of the date this report was released.

The proposed organizational structure does not include a Quality Assurance function with sufficient independence

Without an independent Quality Assurance function, top level IRS executives may not be informed of quality concerns. Since the BSMO is envisioned to be more involved in overseeing than implementing systems modernization, the Quality Assurance function within BSMO will play a significant role.

The proposed BSMO organizational structure includes a Quality Assurance function; however, the Quality Assurance function is depicted as reporting to an executive within the BSMO. Since the Quality Assurance function may be reviewing activities within the BSMO, this structure may not allow for sufficient independence. Sound management practices dictate that a Quality Assurance function should have sufficient independence to carry out its work freely and objectively.

Quality Assurance policies and procedures have not been implemented

The IRS has developed Quality Assurance policies and procedures, and has conducted some quality reviews, such as the review of compliance with the ELC. However, because of the instability of the PMO, the policies and procedures have not been fully implemented.

Recommendations

To ensure an effective Quality Assurance function, the Commissioner should take the following actions:

2. Provide for an independent Quality Assurance function within the approved BSMO organizational structure. The Quality Assurance function could be located administratively within the BSMO but should report directly to the Chief Information Officer/Business Systems Modernization Executive.
3. Finalize and fully implement Quality Assurance policies and procedures throughout the organization.

Adequate Staffing Levels for the Program Management Office Have Not Been Determined

Because a stable PMO has not been established, staffing levels and training plans for the PMO have not been developed. Without a staffing plan, skills analysis, and training plan, the IRS cannot ensure that the PMO will be adequately staffed with the needed mix of skills and abilities. Without a backup plan, the IRS cannot ensure consistent leadership and technical skills are available when needed.

The IRS has not developed a staffing plan to ensure adequate and competent oversight of the PRIME contract. Without roles and responsibilities defined, the key skills needed and the number of individuals needed to fill positions has not been determined. Since the key skills needed within the BSMO are not known because roles and responsibilities have not been defined, existing PMO personnel have not been evaluated to determine training needs. Without an analysis of skills present versus skills needed in the PMO, the IRS cannot determine key skill positions that need to be addressed in a backup plan.

Adequate staffing of the PMO with the proper experience and training for key processes is essential for the IRS to mature into an organization capable of overseeing major systems acquisition projects. Because the IRS has not established a stable PMO, staffing and training analyses have not been conducted.

Current Efforts Underway - The IRS is working on the following areas to correct identified weaknesses:

• PMO personnel state that they will be working to build an ELC-compliant staffing model to address staffing issues.
• An ELC Deployment Plan, that includes a strategy to train current staff and project members on ELC processes, has been developed.

To ensure that the PMO is adequately staffed and the personnel have proper training, the Commissioner should take the following actions:

1. Develop a staffing plan for the PMO that includes:
1. The roles and responsibilities needed within each of the PMO divisions.
2. The key skills needed to accomplish the PMO roles and responsibilities.
3. The number of staff with key skills needed to accomplish the PMO roles and responsibilities.
4. A plan for attaining and/or retaining needed PMO staff.
2. Identify any gaps in skills of the current PMO staff by performing a skills analysis using the key skills identified in the staffing plan.
3. Develop a training plan that addresses any skills deficiencies identified in the skills analysis.
4. Develop a backup plan for key personnel that identifies:
1. Leadership and technical skills that are in short supply within the PMO based on the skills analysis.
2. Methods for ensuring that these skills are always present when needed in the PMO. Methods can include cross-training of PMO personnel, memoranda of understanding with other organizations to provide needed skills, etc.
5. Develop policies and procedures to regularly update the staffing plan, skills analysis, training plan, and backup plan for key personnel.

An Adequate Performance Monitoring Framework Has Not Been Established

The IRS has recently taken significant steps to increase performance monitoring capabilities; however, because of the instability of the PMO, further improvements need to be made. The following risks need to be addressed:

• A formal performance monitoring framework has not been approved and implemented.
• Collection of performance monitoring data is not efficient.
• PRIME contractor quality assurance and risk management capabilities have not been fully assessed.

The IRS is currently developing a formal performance monitoring framework. Without a formal framework describing what performance information is needed for review and when it should be reviewed, the full spectrum of information needed to evaluate the contractor’s performance may not be obtained. This could allow sub-standard performance by the PRIME contractor to go unchecked. In addition, without a clear understanding of performance monitoring roles and responsibilities, PMO personnel may duplicate efforts to obtain and review project performance data.

The IRS recently conducted a review of modernization projects’ compliance with the ELC. The IRS determined that the Customer Communications and e-Services projects, initially planned to begin development in November 1999 and January 2000, respectively, had not completed many of the ELC work products required during the planning and design phases of the projects. Had an effective performance monitoring process been in place, these problems could have been identified much sooner and corrective actions taken without the substantial delays the projects are now facing.

The IRS documented in its recent request for additional funds that projects had approached the development stage without comprehensive plans. Based on this situation, the IRS is scaling back projects that were initially planned to increase service to taxpayers for the next filing season.

The IRS has documented the risk that there is not a defined structure with processes in place for effective PMO and project control. The stated consequence is continued ineffective PMO and project level oversight and lack of program control data on which to make decisions.

During the audit, the IRS was conducting a myriad of reviews to determine the status of modernization projects. Much of the information presented for these different reviews was similar, and the affected IRS managers and staff spent a large percentage of their time attending meetings and/or gathering and reviewing performance data. Although performance monitoring reviews are crucial to the success of systems modernization, reviews have not been efficient and have not always identified weaknesses. The reviews, the frequency, and the major topics covered for each of the current project level reviews are as follows:

Reviews could be more efficient if a framework existed that detailed the project performance data that are needed, the frequency with which the data need to be provided, and the responsibility for reviewing the data. Detailed policies and procedures depicting the information needed for each different performance monitoring review do not currently exist.

The OMB states that agencies are to establish information systems management oversight mechanisms that provide for periodic review of information systems. Because the PMO is unstable and does not have an approved central office responsible for performance monitoring, a formal performance monitoring framework has not been developed.

Current Efforts Underway - The IRS is working on the following areas to correct identified weaknesses:

• Milestone exit criteria have been developed to help ensure that ELC work products are completed at the appropriate time.
• The IRS has stated that it is developing templates for task orders to ensure that required ELC work products are delivered.
• The IRS has developed a performance monitoring Concept of Operations and Implementation Approach. This document identifies key business processes for measuring the progress of projects. The document also includes a time line for creating project reporting requirements, collecting automated project information, and creating detailed policies and procedures. The IRS stated that policies and procedures would be available in September or October 2000.
• "Quad Charts" have been developed to gather and present uniform information from projects for review.
• The IRS has developed a draft listing of project reporting requirements.

To ensure the IRS timely gathers and reviews needed performance monitoring information, the Commissioner should take the following action:

1. Ensure that the approved PMO includes a single office responsible for developing policies and procedures to create a formal performance

monitoring framework. The framework should define:

1. Performance monitoring roles and responsibilities.
2. Project data requirements and frequency.

Collection of performance monitoring data is not efficient

Inefficient performance monitoring practices require PMO and project personnel to collect and review similar project information for different audiences. Without implementing an automated method for collecting project performance data, PMO personnel cannot determine the status of modernization projects at any point in time.

The Paperwork Reduction Act requires agencies to use information resources to improve the efficiency and effectiveness of mission-related operations.

MITRE has recommended a concept for automated collection and distribution of performance monitoring information. According to the IRS, MITRE has not completed the development and installation of this process.

Recommendation

To improve the efficiency of the collection of performance monitoring data, the Commissioner should take the following action:

2. Ensure MITRE timely completes the development and installation of an automated method for collecting and distributing project performance data.

Without fully assessing the PRIME contractor’s capabilities, the IRS could make improper decisions regarding modernization efforts based on inaccurate information being provided by the PRIME contractor. Due to recent developments, the IRS has assigned a high-risk level to the Customer Communications and e-Services projects based on either the lack of or the consistency of information being provided by the PRIME contractor.

The IRS has not made an assessment of the adequacy of the PRIME contractor’s quality assurance and risk management capabilities. The IRS has stated that the BSMO will be responsible for oversight, and the PRIME contractor will be responsible for accomplishing modernization. Based on this relationship, the IRS must ensure that the PRIME contractor’s processes will ensure that quality information is provided to the IRS for review.

The success of any Information Technology program is contingent upon management’s ability to make sound decisions based on accurate information. Management decisions are only as good as the information being used to make those decisions. Because the IRS has not had a stable PMO with a fully staffed office to monitor contractor performance, the PRIME contractor’s processes have not been fully reviewed and assessed.

Current Efforts Underway - The IRS is working on the following area to correct identified weaknesses:

• The IRS has requested that the PRIME contractor develop a "get well" plan to deliver needed performance monitoring information to the PMO.

To ensure the adequacy of the PRIME contractor’s performance monitoring capabilities, the Commissioner should take the following action:

1. Assess the PRIME contractor’s processes to ensure that performance monitoring data being provided to the IRS are complete and accurate.

A Risk Management Framework Has Not Been Fully Implemented

The IRS has made progress toward implementing a risk management framework; however, full implementation and training have yet to occur. Also, the IRS has not fully established a central office responsible for overseeing PMO/project risk management activities.

Without full implementation of risk management policies and procedures, there is no assurance that PMO (program-wide) and project level risks are being identified, addressed, and monitored. Without training on the risk management framework, there is no assurance that PMO and project level policies and procedures will be adequately implemented.

The IRS documented in its recent request for additional funds that there had been limited focus on managing and reducing risks. Also, the IRS is concerned that the PRIME contractor has not been providing adequate risk documentation for modernization projects.

Until risk management policies and procedures are finalized, risk management training cannot be conducted. Risk management training is planned for the summer of 2000. Full implementation of the risk management framework is planned for September 2000.

To mature into an organization capable of managing the acquisition of large systems modernization projects, the IRS must have a written policy for the management of software acquisition risk. In addition, the IRS must provide individuals who have experience or have received required training to perform risk management activities.

The GAO’s May 1998 Executive Guide Information Security Management: Learning from Leading Organizations provides guidance to help federal managers better manage their information resources and implement good risk management practices. In the Guide, principles and practices include: implement appropriate policies and related controls, promote awareness, and continually educate users and others on risks and related policies.

Full implementation of the risk management framework and related training has not occurred due to the instability of the PMO.

Current Efforts Underway - The IRS is working on the following areas to correct identified weaknesses:

• The IRS stated that PMO and project level risk management policies and procedures would be implemented in the near future.
• The IRS identified the top risks to the current systems modernization effort in February 2000.
• The IRS has sent the PRIME contractor a "get well" plan to deliver needed risk management information to the PMO.

To ensure that risk management is fully implemented within the IRS, the Commissioner should take the following actions:

1. Implement and institutionalize PMO and project level risk management policies and procedures.
2. Enhance risk management capabilities, including providing training to personnel at the PMO and project levels, prior to initiating any high-risk projects.

Without a centralized office responsible for risk management, there is no assurance that PMO and project risks are being identified, addressed, and monitored. Without constant focus on risk management, the IRS has been working on risks as they begin to affect systems modernization instead of working on them before their impact is felt.

To progress to an organization capable of monitoring major projects, the IRS must designate responsibility for risk management activities and assign responsibility to a group to coordinate risk management activities. The GAO’s May 1998 Executive Guide Information Security Management: Learning from Leading Organizations includes the following principles and practices: manage risk on a continuing basis, establish a central management focal point, and monitor and evaluate policy and control effectiveness.

In December 1998, MITRE recommended in the Task 2.1: Organizational Readiness Final Report that the IRS assign responsibility for risk management processes to a single office with program-wide oversight authority. The IRS has begun implementing risk management processes but due to the instability of the PMO, the risk management office has not yet become fully functional.

Current Efforts Underway - The IRS is working on the following area to correct identified weaknesses:

• The proposed organizational structure for the BSMO includes an office responsible for risk management activities.

To ensure that adequate risk management processes are developed, implemented, and followed, the Commissioner should take the following action:

1. Ensure the final PMO organization includes a centralized office responsible for risk management.

Conclusion

The IRS has undertaken a multi-billion dollar effort to modernize its systems over a 15-year period. The complexity of this task requires a stringent system of controls to ensure that expected results are achieved. The addition of a contractor tasked to accomplish the systems modernization creates a need for strong program management and oversight on the IRS’ part.

To oversee the systems modernization, the IRS needs to establish an adequately staffed PMO with strong risk management and performance monitoring capabilities. The IRS is beginning to make progress toward implementing this structure, and many of the processes that fit under the PMO have been implemented and are functioning. However, some significant risks still need to be addressed to provide the oversight necessary to ensure the current systems modernization effort is successful.

Detailed Objectives, Scope, and Methodology

The overall objectives of this audit were to determine the adequacy of the Internal Revenue Service’s (IRS) organizational structure developed to oversee the systems modernization effort, IRS Program Management Office (IRS PMO) staffing, performance monitoring capabilities, and risk management capabilities. To accomplish this, we:

1. Determined if an adequate organizational structure is in place to oversee the PRIME contract.
1. Determined the role of senior management in overseeing the IRS PMO.
1. Interviewed top IRS PMO officials to determine the reporting mechanisms in place to ensure senior management is kept informed of progress made.
2. For the Core Business Systems Executive Steering Committee:
1. Reviewed the charter for roles and responsibilities.
2. Determined membership, if the members met regularly, and if meetings and decisions were documented. (Members should include senior IRS management, including Information Systems and User management.)
2. Determined if key IRS PMO personnel had sufficient authority to accomplish their roles and responsibilities.
1. Reviewed organizational charts to determine whether appropriate line authority had been established.
2. Determined if key unit mission charters had been developed.
3. Determined if key IRS PMO personnel understood levels of responsibility and authority.
3. Determined if adequate quality assurance existed in the IRS PMO.
1. Determined if there was a quality review policy and/or plan for programs/projects and if key performance indicators had been set for quality assurance.
2. Determined whether the Quality Assurance function had sufficient independence and staffing to perform its mission.
2. Evaluated the adequacy of IRS PMO staffing.
1. Determined if IRS management had a plan in place to ensure staffing adequately provided competent oversight of the PRIME contract.
2. Determined if IRS management performed an analysis to identify existing gaps in qualifications of current personnel and had accounted for and planned/taken actions on any gaps identified.
3. Determined if IRS management had a training plan in place to address any identified gaps in qualifications.
4. Determined if IRS management had a backup plan to cover unexpected vacancies in the key personnel structure.
3. Determined whether IRS management could effectively manage the PRIME contractor and whether roles and responsibilities were clearly defined.
1. Determined if IRS management had established partnerships to ensure the quality of the relationships with the PRIME contractor.
1. Interviewed IRS management to identify organizational relationships with the PRIME contractor and determined whether communication of contract issues existed.
2. Interviewed the PRIME contractor and determined if it had experienced any difficulties in communicating with the IRS.
3. Interviewed PRIME contractor, IRS PMO, IRS Procurement, and IRS Contract Management personnel to determine if a process was in place for issue resolution.
2. Determined whether IRS management had defined specific procedures to ensure contracts were defined and agreed upon.
1. Obtained a list of all goods/services that were under contract as of November 8, 1999.
2. Discussed with key PRIME contractor and IRS PMO personnel the types of goods/services being provided and determined if there were goods/services being provided which were not on the list obtained in step 1.
3. Interviewed IRS management and determined if contractors were actually performing services as defined in the contracts.
3. Determined whether IRS management had established a process to monitor and evaluate the PRIME contractor’s performance.
4. Evaluated the adequacy of program and project risk management activities.
1. Determined if IRS management had developed a systematic risk management framework and approach.
1. Interviewed IRS PMO staff to determine if there was a method of managing risks for the overall modernization program.
2. Obtained copies of risk management documentation.
1. Determined if a risk management methodology was documented.
2. Determined if significant exposures and corresponding risks had been identified.
3. Determined if a process was in place to regularly update risk assessments.
2. Determined if IRS management had developed a risk management approach that focused on the essential elements of risk identification.
1. Determined how the IRS PMO and projects identified all program and project risk factors (e.g., internal and external factors, results of audits, inspections, identified incidents, etc.).
2. Determined if program and organization-wide objectives were included as part of risk identification.
3. Determined if IRS management had developed a risk management approach that was measurable.
1. Determined if policies offered a formal framework for quantitative and/or qualitative measurement of risks.
2. Determined if a policy was in place to prioritize risks based on risk ratings.
4. Determined if IRS management had developed a risk management approach that included the development of a risk action/mitigation plan.

Major Contributors to This Report

Scott E. Wilson, Associate Inspector General for Audit (Information Systems Programs)

Scott A. Macfarlane, Director

Terry W. Black, Audit Manager

Troy D. Paterson, Audit Manager

Deadra M. English, Senior Auditor

Lynn Faulkner, Senior Auditor

Esther M. Wilson, Senior Auditor

Nelva U. Blassingame, Auditor

Charlene L. Elliston, Auditor

Perrin T. Gleaton, Auditor

Report Distribution List

Deputy Commissioner Modernization C:DM

Chief, Agency-Wide Shared Services A

Chief Information Officer IS

Business Systems Modernization Executive B

Deputy Business Systems Modernization Executive (Business) B:E

National Director for Legislative Affairs CL:LA

Director, Office of Program Evaluation and Risk Analysis M:O

Office of the National Taxpayer Advocate C:TA

Office of Chief Counsel CC

Office of Management Controls M:CFO:A:M

Audit Liaisons:

Chief Information Officer IS

Business Systems Modernization Executive B