Executive Summary

Objectives and Scope

Background

Results

Conclusion

Appendix I – Detailed Objectives, Scope, and Methodology

Appendix II – Major Contributors to This Report

Appendix III – Report Distribution List

Executive Summary

The Internal Revenue Service (IRS) is currently in the early phases of its latest effort to modernize its outdated, paper-intensive tax processing system. This multi-billion dollar effort, known as Business Systems Modernization, is projected to last up to 15 years. The IRS has set up the Business Systems Modernization Office (BSMO) to oversee this modernization effort.

Our audit objectives included determining the adequacy of the IRS’ 1) short-range planning efforts, 2) the systems modernization Program Management Plan, 3) significant controls to guide projects from initiation through post-implementation, 4) independent assurance process, and 5) performance monitoring capabilities.

The IRS is making solid progress in building program management capabilities. Many of the actions being taken by the IRS and the modernization contractor are significant undertakings and, if implemented correctly, should lead to future success for the systems modernization effort. We believe that improvements in short-range planning, funding, post-implementation review, independent assurance, and performance monitoring processes would increase the chances for success in the long-term systems modernization efforts.

We found that the IRS does not have a systems modernization short-range plan in the form of a single unique document, although most elements of a short-range plan exist or are being developed separately. Without a comprehensive short-range plan, the IRS may not achieve coordinated short-range goals such as timely implementing business process changes, modernized systems, and program management capabilities. However, the IRS’ draft Program Management Plan could be enhanced to create a short-range plan to guide systems modernization efforts.

The modernization contractor had to suspend work on 10 projects while the IRS waited for additional funding to be approved by the Congress. Work on the projects was delayed for almost 2 months, and the contractor had to replace several employees when the funding was restored. The BSMO has not documented its procedural controls over funding. Until this occurs, the funding process remains loosely defined and may continue to result in funding gaps, denial of funds, and projects that are over- or under-funded.

Under the current funding strategy, funding intervals may become even more frequent as more projects are initiated. As the systems modernization effort matures and management capacity increases, the current funding strategy will also need to mature by employing processes that will reduce the chances of future funding gaps.

Current systems modernization Quality Assurance documentation does not include a requirement to perform a post-implementation review of projects. Without this type of review, the IRS may not identify program management strengths and weaknesses. The BSMO is currently evaluating who would be responsible for these reviews.

The IRS has made strides to ensure that organizations are in place to provide independent assurance concerning the IRS’ and the modernization contractor’s compliance with security, legal, and effectiveness requirements. However, not all organizations have formal agreements documenting their commitment to the Business Systems Modernization program. In addition, the BSMO has not documented processes and procedures for conducting independent reviews.

In a previous audit, we recommended that the IRS develop an adequate framework for monitoring the performance of the modernization contractor. The BSMO has made strides in implementing this framework, including the execution of performance-based task orders. However, we found that positive and negative contractor incentives in the task orders were generally weak and had not been consistently implemented. For example, several task orders were written such that the IRS only withholds payment until the modernization contractor delivers, rather than reducing the payment to the modernization contractor to compensate for the delay in implementing systems that will benefit taxpayers.

The IRS is currently building program management capabilities. We recommend that, during this period of growth, the BSMO create a short-range plan to guide the new systems modernization effort. We also recommend that the BSMO create policies and procedures for ensuring consistent funding, conducting post-implementation reviews, obtaining independent viewpoints, and improving performance-based contracting processes. With a single document guiding short-range systems modernization efforts and improved policies and procedures, the IRS should have a greater chance for success in delivering quality modernization projects that result in improved service to taxpayers.

Management’s Response: Management’s response was due on January 11, 2001. As of January 18, 2001, management had not responded to the draft report.

Objectives and Scope

This audit is one of a series of audits to evaluate the Internal Revenue Service’s (IRS) efforts to modernize its tax processing systems. Our objectives were to determine the adequacy of the IRS’:

• Short-range planning efforts.
• Systems modernization Program Management Plan.
• Significant controls to guide projects from initiation through post-implementation.
• Independent assurance.
• Performance monitoring capabilities.

The audit was conducted in New Carrollton, Maryland, from June through August 2000. The audit scope included assessing the status of short-range planning, the systems modernization Program Management Plan, systems life cycle concepts, independent assurance, and performance monitoring with key IRS and contractor officials and reviewing available documentation. This audit was performed in accordance with Government Auditing Standards.

Details of our audit objectives, scope, and methodology are presented in Appendix I. Major contributors to this report are listed in Appendix II.

Background

The IRS is currently in the early phases of a new systems modernization effort known as Business Systems Modernization. The IRS is working with a private contractor on this multi-billion dollar effort, which is expected to last up to 15 years. This is our second audit of the IRS’ oversight of the systems modernization effort. Our first audit found that the oversight of the systems modernization effort had been hampered by the lack of a stable program management organization. Program management staffing needs had not been determined, roles and responsibilities were not yet clearly defined, and key processes such as performance monitoring and risk management needed to be improved.

Since our first audit, the IRS has entered a critical stage of building program management capabilities. For example, the IRS is deploying a systems life cycle methodology, updating the Modernization Blueprint, and performing a comparison of skills needed to those present within the IRS’ systems modernization program office. The goal of our audit was to look at the many actions being taken by the IRS and modernization contractor during this period of growth and provide recommendations to enhance the systems modernization effort.

Several different organizations with different roles are involved in the IRS’ systems modernization effort.

Computer Sciences Corporation serves as the Prime Systems Integration Services Contractor (PRIME) for the systems modernization program. Under the PRIME contract, Computer Sciences Corporation is responsible for designing new systems to meet IRS business needs, developing these systems, integrating them into the IRS, and ultimately transferring operation of these systems to the IRS.

The Business Systems Modernization Office (BSMO) was set up to oversee the systems modernization effort. The BSMO performs such functions as reviewing PRIME contractor project data, requesting funds, and ensuring that delivered systems satisfy contractual requirements.

The Core Business Systems Executive Steering Committee was established to provide strategic direction, make decisions on which projects should be funded to ensure projects deliver maximum value, and approve projects at critical milestones. Voting members include the Commissioner and executives from Information Systems and IRS operations.

The MITRE Corporation (MITRE) is also under contract to assist the IRS with systems modernization. MITRE provides the IRS with specific expertise in establishing strategic priorities, making investment decisions, evaluating proposals, managing the systems modernization program, monitoring contracts, performing specific research, and conducting testing activities.

The IRS and the PRIME contractor are following specific funding, systems development, and capability assessment processes for the IRS’ systems modernization.

The Congress places funds for the IRS’ systems modernization activities in an Information Technology Investment Account (ITIA). The IRS must submit a spending plan requesting that funds from the ITIA be withdrawn for use by the IRS. The spending plan must be reviewed by the General Accounting Office and approved by the Department of the Treasury, the Office of Management and Budget (OMB), and the Congress.

The IRS and the PRIME contractor are required to follow the Enterprise Life Cycle (ELC). The ELC is a structured business systems development method that requires specific work products to be developed during different phases of the life of a project.

The IRS plans to use the Software Engineering Institute’s Capability Maturity Model to evaluate the IRS’ and PRIME contractor’s abilities to acquire and design the software needed to meet systems modernization objectives. The Capability Maturity Model is a structured process that helps organizations improve their abilities to consistently and predictably acquire and develop high-quality information systems. Organizations that have implemented the Capability Maturity Model processes have seen dramatic improvements in their abilities to meet planned time frames, reduce errors, and increase value on dollars invested.

Results

The IRS is making solid progress in building program management capabilities. Many of the activities that are currently underway are significant undertakings and, if implemented correctly, should lead to future success for the systems modernization effort.

On May 1, 2000, the BSMO released the first version of its Program Management Plan (PMP). We found the PMP correctly defines the BSMO’s oversight role. The PMP also defines oversight of the BSMO by the Core Business Systems Executive Steering Committee. Finally, the PMP requires adherence to the ELC, describes how the IRS will select and manage projects, and ties the BSMO funding process to legal and regulatory requirements.

The PMP, dated May 1, 2000, is a solid draft document that describes the systems modernization program. The BSMO has agreed that the PMP is a solid draft document, but plans to make some improvements and has scheduled updates.

While the IRS and the modernization contractor have experienced some difficulty deploying the ELC, we believe the ELC should provide the general mechanisms to ensure that projects are controlled from their inception through implementation.

The ELC provides for the following:

• IRS operations and Information Systems must approve projects at various milestones.
• The nature and scope of a project must be clearly defined and feasibility studies conducted prior to developing a system.
• Project Management Plans are required to 1) define approvals for projects at each stage of completion, 2) define acceptance procedures for each project, 3) include a method for monitoring of time and costs created after a project is approved, and 4) identify project milestones, completion dates, and specific deliverables.
• Quality Assurance Plans are required to be developed by each project and approved by IRS operations and Information Systems management.

Independent assurance is a vital tool to ensure that independent viewpoints are being received during the life of the systems modernization effort. We determined that the following independent viewpoints are being provided to the BSMO:

• The IRS’ Office of Security and Privacy Oversight has assisted in the development of a draft Security and Privacy ELC Supplement that documents its role in providing an independent security certification for each computer system.
• MITRE acts as an independent verification and validation contractor.
• IRS operations executives, who are organizationally independent from the BSMO, have been approving ELC deliverables.
• The General Accounting Office, the OMB, and the Treasury Inspector General for Tax Administration have provided independent viewpoints to the BSMO on different topics.

In our first audit, we reported that an adequate performance monitoring framework had not been developed. The IRS agreed with our assessment and provided corrective actions with scheduled implementation dates that conclude in May 2001. The BSMO has taken some actions in response to the issues in our previous report:

• A concept of operations for obtaining and analyzing project performance data has been approved.
• Data reporting requirements for the PRIME contractor that describe the data needed and the frequency with which they are to be provided to the BSMO have been approved.
• Operation of the Management Information Center has begun.
• Policies and procedures describing the types of meetings that should occur and the purpose of each meeting have been approved.
• Program Management Reviews and Milestone Readiness Reviews are being conducted.
• Best practices in creating performance-based task orders are beginning to be implemented.

While significant activities are occurring within the BSMO, we believe the following program improvements will increase the chances for success in the long-term systems modernization effort:

• A comprehensive short-range plan should be created.
• The funding process should be documented and a more flexible funding strategy considered.
• Procedures for conducting post-implementation reviews should be documented.
• Processes to ensure independent views are considered should be defined.
• Performance-based contracting incentives should be strengthened.

A Comprehensive Short-Range Plan Should Be Created

We found that the BSMO does not have a systems modernization short-range plan in the form of a single unique document, although most elements of a short-range plan exist or are being developed separately in various BSMO documents. Without a comprehensive short-range plan, the IRS may not achieve coordinated short-range goals such as timely implementing business process changes, modernized systems, and program management capabilities . Lack of a short-range plan could also cause communication problems between the BSMO and the Core Business Systems Executive Steering Committee as to the overall direction and health of the modernization effort.

A comprehensive short-range plan should include 1) a complete list of project- and program-level efforts in progress in the short-range period; 2) a time schedule for all efforts, with dates for major checkpoints and delivery times; 3) a statement of where the systems modernization effort is at present and where it should be at the end of the short-range period; 4) reference to supporting plans for all short-range efforts; and 5) assurance that the short-range efforts are linked to the strategic plan.

The Clinger-Cohen Act of 1996 directs agencies to plan and implement long-term systems acquisition projects, such as systems modernization, in small, manageable segments. By definition, the segments will be of shorter duration than the strategic plan, which they support. We believe a short-range plan is needed to manage and control the short duration modernization segments recommended by the Clinger-Cohen Act of 1996.

While the PMP does not define a short-range planning period, it comes closest to having the elements necessary to be considered a short-range plan. The PMP contains the most complete list of projects and program-level efforts currently in progress. Although there are few dates included in the PMP for major checkpoints and delivery times for all efforts, the information from the Integrated Master Schedule and Government Program Master Schedule could be used to satisfy this goal. The PMP also presents project-level information on current and future activities; however, the draft PMP lacks a comprehensive statement of the current overall status of the systems modernization and the expected status at the end of the short-range period. Once completed, the Blueprint 2000 and the Tax Administration Vision and Strategy projects may be able to provide this information.

The PMP does not always cite supporting plans for developing and delivering each program/project effort, even though such plans exist or are being developed. The PMP also does not tie short-range projects to the long-range plan (Modernization Blueprint). However, once developed, the Architecture’s Enterprise Transition Strategycould be helpful in providing this information.

The BSMO has been evolving since inception. Initially, project work advanced ahead of the BSMO’s ability to manage the oversight function. Until recently, short-range planning has been fragmented; however, the creation of a PMP is a positive step toward pulling all the fragments together into a comprehensive short-range plan.

To ensure the systems modernization effort has a single document describing goals and direction for the short-range planning period, the BSMO should:

1. Incorporate into the PMP short-range plan elements that:
1. Define the time frame for the short-range planning period.
2. Include descriptive statements of where both the IRS and the systems modernization effort are at present and where they will be at the end of the short-range planning period.
3. Refer to any major effort that could have an effect on modernization, including all project- and program-level efforts. This might include finalizing charters, standing up fledgling units, funding programs, selecting projects, modifying legacy systems, and training personnel.
4. Ensure a link exists that relates projects in progress to the long-range plan (Modernization Blueprint).
5. Cite the supporting plans in place or under development, when applicable, for each project- or program-level effort addressed.

Include a time schedule for all efforts to show major checkpoints and delivery times that fall within the short-range planning period.

Management’s Response: Management’s response was due on January 11, 2001. As of January 18, 2001, management had not responded to the draft report.

The Funding Process Should Be Documented and a More Flexible Funding Strategy Considered

A key factor in the future success of systems modernization is maintaining ITIA funding on a consistent basis without interruption. In April 2000, the IRS had depleted its allocated ITIA funds and had not received a new allocation from the Congress. As a result of this funding gap, the PRIME contractor:

• Suspended work on 10 projects for almost 2 months. The delay was partially due to the length of time it took to create the spending request.
• Lost a project manager, a database expert, and a security expert during the work suspension. These employees had to be replaced when the funding resumed.
• Lost momentum and experienced inefficiencies associated with restarting postponed activities.

The March 7, 2000, Spending Plan stated that the IRS had developed a repeatable process for creating, submitting, and managing future spending plans. BSMO management advised us that they are following a consistent process; however, the BSMO has not documented ITIA funding procedures. Until documented, the process remains loosely defined and may continue to result in funding gaps, denial of funds, and projects that are over- or under-funded.

Sound business practices suggest that for a process to be repeatable, it should be documented so that others have guidance to follow the process. BSMO management has stated that it has not documented the process because resources were diverted to preparing spending plans. However, the BSMO states it has recently assigned responsibility for creating a flowchart of these procedures.

The ITIA was set up by a 1998 Congressional appropriation to fund the IRS systems modernization program. While the appropriation does not require the IRS to submit multiple incremental spending plans, the Clinger-Cohen Act of 1996 suggests the use of incremental plans.

The IRS’ current funding strategy pursues funding on a frequent basis based on project milestones. According to BSMO management, the ITIA funding process is labor-intensive and requires several months lead time to develop the spending plans.

Under the current funding strategy, funding intervals may become even more frequent as more projects are initiated. Since funding gaps have been experienced and long lead times (up to 6 months) are required to develop spending plans, the current funding strategy will need to mature as the systems modernization effort matures and management capacity increases.

While the BSMO recognizes the need to progress to a more flexible funding strategy, the IRS has not pursued this change, due in part to criticisms from the Congress. For instance, the IRS recently requested funding for multiple milestones for the Customer Relationship Management Exam project. While the Congress denied this request, its reasons were the IRS’ lack of an implemented life cycle and an updated Modernization Blueprint, not a disagreement with the concept of multiple milestone funding.

To ensure the systems modernization effort receives consistent funding and evolves to a more flexible funding strategy, the BSMO should:

1. Document the process of obtaining ITIA funds based on best practices and lessons learned. When developing this documentation, the BSMO should consider short- and long-term alternatives to the current funding strategy.

Procedures for Conducting Post-Implementation Reviews Should Be Documented

Current BSMO Quality Assurance documentation does not include a requirement to perform a post-implementation review of projects to identify program management strengths and weaknesses. Federal guidelines require performance of post-implementation reviews. Without this type of review, the BSMO may not identify program management strengths and weaknesses. As a result, the IRS’ and PRIME contractor’s processes may not improve based on lessons learned.

The IRS is currently evaluating roles and responsibilities within the BSMO. BSMO management agreed that post-implementation reviews needed to be conducted on modernization projects; however, it has not assigned responsibility for creating post-implementation review procedures because the BSMO has yet to determine who would be responsible for the reviews.

Recommendation

To ensure that post-implementation reviews are conducted to identify program management strengths and weaknesses, the BSMO should:

2. Create and document policies and procedures for conducting post-implementation reviews and assign responsibility for conducting the reviews.

Processes to Ensure Independent Views Are Considered Should Be Defined

The IRS has made strides to ensure that organizations are in place to provide independent assurance that the IRS and PRIME contractor meet security, legal, and effectiveness requirements. However, the IRS does not have a documented process to ensure all appropriate organizations have formal agreements documenting their commitment to perform the independent reviews.

Sound management principles dictate that agreements should be formally documented to ensure commitment by both parties. Without formal agreements, the risk is increased that the BSMO and PRIME contractor may not take full advantage of the independent organizations’ abilities to detect or foresee current and future barriers to the success of the systems modernization effort.

The IRS is still defining what constitutes an independent reviewer. The Core Business System Executive Steering Committee identified this issue as a risk at its May 30, 2000, meeting. As recommended in our previous report, the IRS is seeking to validate that the systems modernization Quality Assurance organization is independent. The IRS set a due date of September 30, 2000, to complete the validation.

The IRS has a contract with MITRE to provide independent assessments of systems modernization activity, and the role of the Office of Security and Privacy Oversight in the systems modernization effort is documented in an ELC supplement. However, other offices or organizations providing independent assurance do not have formal agreements detailing the scope of their work and continuing involvement in performing independent assessments. These offices and organizations include:

• The IRS Office of Program Evaluation and Risk Analysis. This office is involved in looking at the risk assessment components of business cases and is developing risk management guidelines that MITRE will incorporate into the Business Case Resource Guide.
• The Software Engineering Institute. This Institute has developed the Capability Maturity Model, which evaluates an organization’s ability to acquire and design systems software. The IRS’ goal is to reach a level II rating by September 2001. The PRIME contractor is required to reach a level III rating. While the IRS and PRIME contractor stated that several activities were upcoming to move both toward Capability Maturity Model evaluations, we did not find any formal agreements detailing who would conduct the evaluations or when they would be conducted.

Written procedures are needed to provide a defined process for conducting independent evaluations. Because the BSMO is still developing processes and procedures, written guidance has not developed for conducting independent evaluations. The BSMO should prepare written procedures to clarify what constitutes an independent organization, what aspects of the systems modernization program and project activities will be evaluated, and when these evaluations would be performed. While we found entities in place to perform independent reviews, we did not identify any processes or procedures to guide the following activities:

• Approving of ELC deliverables by IRS executives, independent of the BSMO.
• Coordinating with IRS Product Assurance. The BSMO has agreed to complete this documentation.
• Obtaining independent assurance on legal compliance. In a recent skills self-assessment, BSMO personnel consistently scored lower than their target level in identifying which government rules and regulations are applicable to the systems modernization program. To assist the BSMO in this area, MITRE has compared legal requirements to the Capability Maturity Model and Investment Decision Model process. However, the BSMO currently does not have plans to obtain further guidance concerning other policies and procedures and whether they comply with legal requirements.

To ensure that independent and objective reviews of the systems modernization process continue in areas where an independent viewpoint is needed, the BSMO should:

1. Create and document policies and procedures defining an independent organization, the program and project activities for evaluation, and the timing of these evaluations. The BSMO also should formalize agreements with the reviewing organizations to ensure that these independent viewpoints continue as necessary.

Performance-Based Contracting Incentives Should Be Strengthened

The BSMO has rewritten and better defined task orders issued to the PRIME contractor, but performance-based incentives have not been clearly defined. Without strong positive and negative incentives, the PRIME contractor may not work efficiently and may not provide timely products to the IRS. As a result, systems that are intended to provide benefits to taxpayers may not be provided within time, cost, and quality constraints. For example, several task orders were written such that the IRS only withholds payment until the modernization contractor delivers, rather than reducing the payment to the modernization contractor to compensate for the delay in implementing systems that will benefit taxpayers.

In March 2000, the IRS tasked Jefferson Solutions to teach IRS and PRIME contractor personnel how to properly implement performance-based contracting concepts. As of September 1, 2000, the IRS had issued 10 task orders that contain performance-based specifics. The IRS also intends to contract with Jefferson Solutions in Fiscal Year 2001 to review issued performance-based task orders. With recent training and experience, the IRS and Jefferson Solutions should look for ways to improve incentives contained within performance-based task orders.

Government policy states that contracts shall include incentive provisions to ensure that contractors are rewarded for good performance and quality assurance deduction schedules to discourage unsatisfactory performance. Since the IRS has not had a lot of experience in creating performance-based task orders, incentives have been treated differently per task order.

Positive Incentives

While two of the task orders that we reviewed contain a method for determining positive incentives, most task orders do not provide strong positive incentives for completing work beyond the acceptable quality level.

For instance, the Customer Communications task order includes a payment of performance incentive table that lists amounts to be paid if the contractor exceeds the acceptable quality level by 10 percent. However, most of the task orders we reviewed did not include this table.

Negative Incentives

We did not find strong negative incentives to address situations where the contractor’s performance falls below the acceptable quality level. While the performance-based task orders contain a myriad of expected performance and acceptable quality level statements, most task orders do not define any penalties for falling below the acceptable quality level.

For instance, the e-Services task order has a performance standard that requires complete, consistent status information to be provided weekly. The acceptable quality level is 0 percent deviation. We have noted that the e-Services project has not been providing timely, consistent status information. Our concern is that there is no penalty for not meeting the acceptable quality level.

Recommendation

To ensure that performance-based task order incentives are strengthened, the BSMO should:

2. Task Jefferson Solutions to review performance-based incentives and provide recommendations for improving incentives in future task orders. Accepted recommendations and suggestions received from Jefferson Solutions should be used to establish policies and procedures for creating performance-based task orders.

Conclusion

The IRS has undertaken a multi-billion dollar effort to modernize its tax processing systems over a 15-year period. Initially, oversight of the systems modernization effort was hampered by the lack of a stable program management organization. Program management staffing needs had not been determined, roles and responsibilities were not yet clearly defined, and key processes such as performance monitoring and risk management needed to be improved.

Since that time, the IRS has made important progress in developing program management capabilities within the BSMO, and significant work is underway to provide the building blocks for successful systems modernization. Once the IRS stabilizes program management capabilities, it can concentrate on ensuring compliance with policies and procedures, while increasing its software acquisition maturity level. These actions will increase the chances for success in the systems modernization effort.

Detailed Objectives, Scope, and Methodology

The objectives of this review were to determine the adequacy of 1) short-range planning, 2) the systems modernization Program Management Plan, 3) significant controls to guide projects from initiation through post-implementation, 4) independent assurance, and 5) performance monitoring capabilities. To accomplish these objectives, we:

1. Determined whether the Internal Revenue Service’s (IRS) systems modernization short-range planning efforts were adequate.
1. Reviewed documentation (Business Systems Modernization: Taking Modernization to the Next Level, 60 Architecture Work Products, selected sections of the May 1997 Modernization Blueprint, Draft Program Management Plan, current Information Technology Investment Account Spending Plan, Government Program Master Schedule, and any sub-plans identified during audit field work) and interviewed Business Systems Modernization Office (BSMO) personnel to determine whether the following elements of a short-range plan were present:
1. Comprehensive list of all program/project efforts to be completed in the short-range plan time frame.
2. Time schedule for all program/project efforts that specifies major developmental checkpoints and delivery times.
3. Statement of the current position of the systems modernization effort and where the IRS expects to be at the end of the current short-range plan.
4. Supporting plans that define how the current program/project efforts will be developed and delivered.
2. Interviewed BSMO personnel and obtained documentation to determine whether the short-range plan(s) directly supported the goals and objectives of the Modernization Blueprint (i.e., the strategic, long-range plan).
3. Determined whether the IRS has implemented controls to ensure that funding is obtained timely and funding gaps are prevented in the future.
1. Interviewed the BSMO staff and reviewed documentation to determine what steps have been implemented to prevent the funding shortages.
2. Obtained and reviewed the BSMO formal procedures for addressing the funding gaps, including details regarding the proposed annual funding process.
3. Determined how the 10 percent variance allowed by the Federal Acquisition Streamlining Act and Office of Management and Budget (OMB) Circular A-11 is being used and whether it has or will help prevent the funding shortages.
4. Measured the impact of the April 2000 funding shortage on systems modernization operations, including delivery of program initiatives (e.g., delay in implementing the PRIME Get Well Plan) and project activities (e.g., problems in completing deliverables for the 2001 Filing Season roll-out of Customer Communications).
1. Obtained records from the BSMO Budget Office to identify the expenditures made from December 1999 to April 2000 (e.g., burn-rate reports).
2. Analyzed the expenditure records and determined the dates the PRIME contractor’s personnel were/were not working on the program and project activities.
3. Interviewed BSMO personnel and PRIME contractor staff to identify and verify the program areas and projects that were affected by the April 2000 funding shortage.
2. Determined whether a comprehensive program management framework exists to guide and coordinate business systems modernization activities by reviewing the draft Program Management Plan to determine whether it:
1. Defines the scope and limitations of the BSMO.
2. Provides for executive oversight of the BSMO.
3. Specifies the basis for assigning staff and explains responsibilities and authorities.
4. Requires implementation and adherence to the Enterprise Life Cycle (ELC), including the ELC phases and milestones.
5. Describes how projects will be initiated and managed, in both the short-range and long-range.
6. Describes how systems modernization projects will be integrated with each other and the IRS architecture and blueprint.
7. Requires funding approaches that adhere to authoritative requirements of the Congress, the OMB, and federal government regulations.
3. Determined whether adequate controls are in place to guide projects from initiation through post-implementation.
1. Determined whether the project Vision and Strategy and Architecture phases are well controlled.
1. Determined whether users are required to be involved in these phases by approving required documents on the projects.
2. Determined whether the nature and scope of each project is required to be clearly defined (e.g., has an ELC path been selected?) prior to the initiation of task orders or projects for the Development phase.
3. Determined whether feasibility studies are required before projects are approved.
2. Determined whether IRS users and Information Systems management are required to approve a project at critical junctures.
3. Determined whether project management plans exist and are adequate.
1. Determined whether project management plans are required for each project that has completed the Architecture phase.
2. Determined whether project management plans are required to:
1. Define approvals for projects at each stage of completion.
2. Define final acceptance procedures for each project.
3. Include monitoring of time and costs created after a project has been approved.
4. Identify project milestones, completion dates, and specific deliverables.
4. Determined whether project quality assurance plans are required to be developed and whether users and Information Systems management personnel are required to approve project quality assurance plans.
5. Determined whether policies provide for a thorough post-implementation review of each project to identify program management strengths and weaknesses.
4. Determined whether processes exist to provide independent assurance that the IRS and the PRIME contractor are meeting security, legal, and effectiveness requirements.
1. Interviewed key BSMO personnel to determine whether a decision has been made as to whether the IRS will hire another contractor as the Independent Verification and Validation contractor in addition to the MITRE Corporation (MITRE). (Per the December 1999 Infrastructure Executive Steering Committee meeting minutes, the BSMO was assigned responsibility for this action item. The due date was shown as March 2000).
2. Determined who is responsible for evaluating the IRS’ and the PRIME contractor’s Capability Maturity Model level.
1. Interviewed key IRS personnel to determine how they plan to determine at which systems acquisition Capability Maturity Model level they are.
2. Interviewed PRIME contractor personnel to determine whether they have a contractor to perform an assessment to determine the software development and systems acquisition Capability Maturity Model level of the PRIME.
3. Reviewed the contract between the IRS and the PRIME contractor to determine whether there are any requirements for the PRIME contractor to be at a certain software development and systems acquisition Capability Maturity Model level.
4. Interviewed Software Engineering Institute personnel to determine what they are required to do for the IRS and the PRIME contractor.
5. Obtained and reviewed any schedules of planned reviews of the IRS’ and PRIME contractor’s Capability Maturity Model levels.
3. Interviewed key personnel in the IRS Office of Security and Privacy Oversight to determine whether they are required to conduct independent assurance reviews and the area(s) they are responsible for reviewing.
4. Interviewed key personnel in the IRS Office of Program Evaluation and Risk Analysis to determine whether any independent assurance reviews of risk assessments are planned/in process.
5. Interviewed key BSMO and MITRE personnel to determine whether any reviews have been performed to determine whether the IRS and the PRIME contractor are meeting legal requirements (e.g., the Clinger-Cohen Act of 1996, OMB Circulars, etc.).
5. Reviewed and evaluated current efforts in performance monitoring within the BSMO.
1. Determined whether the BSMO has implemented policies/procedures for preparing performance-based contracting task orders.
1. Interviewed key personnel to determine whether policies and procedures have been developed for preparing task orders for performance-based specifics. Obtained and reviewed available documents (e.g., performance-based templates, Project Reporting Requirements for PRIME Task Orders, etc.).
2. Obtained a list of the original task orders that were rescoped to reflect performance-based measures.
3. Obtained and reviewed project level performance measurements contained in Project Management Plans or Project Performance Measure Plans, which are needed to support performance-based task orders. Selected measurements from the Customer Communications and Security and Technology Infrastructure Release projects.
2. Determined whether the BSMO has developed procedures/policies for obtaining individual project plans and progress data.
1. Interviewed key personnel at the program level to determine reporting requirements.
2. Obtained and reviewed documentation (e.g., Cost, Schedule, Earned Value, and Quad Charts) which illustrates the program reporting requirements for projects.
3. Determined whether the Program Control Concept of Operations has been approved.
4. Determined whether the BSMO is monitoring the PRIME contractor’s reporting requirements.
1. Interviewed key BSMO personnel to determine whether the PRIME contractor’s reporting requirements have been implemented.
2. Obtained and reviewed weekly/monthly data items submitted by the PRIME contractor to determine whether the BSMO is monitoring the reporting.
3. Analyzed the weekly/monthly data items to determine whether the BSMO is monitoring and elevating the results it receives from the PRIME contractor to higher levels (e.g., Core Business Systems Executive Steering Committee, etc.) as appropriate.

Major Contributors to This Report

Scott E. Wilson, Associate Inspector General for Audit (Information Systems Programs)

Scott A. Macfarlane, Director

Edward A. Neuwirth, Acting Director

Troy D. Paterson, Audit Manager

Wallace C. Sims, Senior Auditor

Nelva U. Blassingame, Auditor

Yolanda D. Brown, Auditor

Charlene L. Elliston, Auditor

Perrin T. Gleaton, Auditor

Gwendolyn M. Green, Auditor

Report Distribution List

Commissioner N:C

Assistant Deputy Commissioner Modernization N:ADC:MOD

Chief, Agency-Wide Shared Services A

Director, Business Systems Modernization B

Deputy Director, Business Systems Modernization B:E

Director, Office of Program Evaluation and Risk Analysis N:ADC:R:O

Office of Security and Privacy Oversight IS:SPO

Office of Management Controls N:CFO:A:M

Office of Chief Counsel CC

National Taxpayer Advocate TA

Director, Legislative Affairs CL:LA

Audit Liaisons:

Chief Information Officer IS

Director, Business Systems Modernization B:E