The Service Center Mainframe Consolidation Project Has Made Significant Progress, But Project Execution and Administration
Risks Remain
September 1999
Reference Number: 199920068
September 28, 1999
MEMORANDUM FOR COMMISSIONER ROSSOTTI
FROM: Pamela J. Gardiner /s/Pamela J. Gardiner
Deputy Inspector General for Audit
SUBJECT: Final Audit Report – The Service Center Mainframe Consolidation Project Has Made Significant Progress, But Project Execution and Administration Risks Remain
This report presents the results of our review of the Internal Revenue Service’s (IRS) Service Center Mainframe Consolidation (SCMC) project. We conducted the review to determine whether SCMC project management controls were adequate to ensure that: lessons learned during the Kansas City Service Center consolidation were corrected before additional cutovers occurred; any negative impact on taxpayer rights was minimized during the consolidation; and that congressionally mandated budget reporting was accurate. We also evaluated corrective actions taken as a result of our prior audit.
In summary, we found that the SCMC project is making significant progress. However, there are several risks and controls that IRS management must continue to address. We also found that IRS management needs to improve controls over contract administration and budget accounting.
To strengthen the existing management control structure, we recommended improvements in the areas of standardization of operations, testing, training, budget accounting, and defining contract requirements and costs.
IRS management agreed with recommendations submitted as part of an interim memorandum we issued during the review. As of the date of this report, IRS management had not responded to the remaining issues within this report. IRS management’s comments have been incorporated into the report, where appropriate, and the full text of their response to the interim memorandum is included as an appendix.
Copies of this report are also being sent to the IRS managers who are affected by the report recommendations. Please contact me at (202) 622-6510 if you have questions, or your staff may call Scott E. Wilson, Associate Inspector General for Audit (Information Systems Programs), at (202) 622-8510.
Critical Risks Related to Service Center Mainframe Consolidation Remain Unresolved
Controls Over Technical Aspects of Contract Administration Remain Inadequate
Staffing Costs of Consolidation Are Not Accurately Budgeted, Captured, and Reported
Appendix I - Detailed Objective, Scope, and Methodology
Appendix II - Major Contributors to This Report
Appendix III - Report Distribution List
Appendix IV - Abbreviations Used in This Report
Appendix V - Memorandum #1 - Outstanding Risks Related to Service Center Mainframe Consolidation
Appendix VII -Memorandum #2 - Follow-up to Contracting Issues
The Internal Revenue Service (IRS) is in the process of consolidating the mainframe computer operations used to process tax return data. The consolidation involves moving mainframe processing, currently done at the IRS’ 10 service centers, to new mainframe computers located in 2 facilities referred to as computing centers—the Tennessee Computing Center (TCC) and the Martinsburg Computing Center (MCC).
This is our second audit of the Service Center Mainframe Consolidation (SCMC) Project. The first audit, Readiness for Service Center Mainframe Consolidation (Reference Number 085812, dated September 21, 1998) reported that: (1) testing plans were not adequate, (2) Project Office controls over critical tasks and communications to the field needed improvement, (3) controls over technical aspects of contract administration needed improvement, and (4) critical staffing shortages still existed at the computing centers.
The overall objective of this review was to determine whether SCMC project management controls were adequate to ensure that: (1) problems identified during the Kansas City Service Center (KCSC) consolidation were corrected before additional consolidations occurred, (2) consolidation minimized the impact on taxpayer rights, and (3) budget postings and reports to the Congress on the costs of this initiative were accurate. We also evaluated corrective actions taken as a result of the prior audit, and immediately raised significant issues to the attention of IRS Management and Finance (M&F) and Information Systems (IS) Division management.
Results
While the SCMC Project is making significant progress, several risks remain. IRS management also needs to ensure that the technical aspects of contract administration are improved and that the cost of the project is accurately budgeted, captured, and reported.
Critical Risks Related to Service Center Mainframe Consolidation Remain Unresolved
KCSC Mainframe Consolidation to the TCC
When the KCSC mainframe operations were consolidated to the TCC on September 19, 1998, significant problems were encountered. For example, the methods used to transfer data were not efficient, some programming changes were not effective, and detailed standard operating procedures had not been developed for the consolidated environment.
In a November 1998 memorandum, we reported the results of our assessment of the impact of mainframe consolidation on KCSC operations and taxpayers (see Appendix V). We also advised IRS IS management that several risks relating to testing, preparation for consolidation, and staffing were still present prior to the Brookhaven Service Center (BSC) consolidation.
BSC Mainframe Consolidation to the MCC
BSC mainframe computer operations were consolidated to the MCC on December 4, 1998, and the results were very encouraging for the SCMC Project. Several problem areas were improved by addressing lessons learned from the KCSC consolidation. As a result, the IRS was able to return to normal processing times at the BSC much faster than it did during the KCSC consolidation. Other reasons for the successful BSC consolidation included the continuous cooperation and communication between the MCC and the BSC, and the fact that the BSC provided on-site support to the MCC during the initial weeks of consolidation.
However, the IRS did experience some problems during the BSC consolidation. For example, some programs did not work, some telecommunication hardware failed, and automated scheduling was cumbersome.
There are still several issues that should be addressed before the remaining service center consolidations.
Controls Over Technical Aspects of Contract Administration Remain Inadequate
In the prior review, we recommended that IRS management provide additional resources to adequately define and validate SCMC contract requirements, and that the SCMC Project Office coordinate with the Procurement Division during price negotiations. IRS management agreed with the recommendations and established an August 1998 completion date for corrective actions. The IRS had not adequately corrected these conditions by October 1998. As a result, the IRS’ negotiating position has diminished.
The Project Office was not verifying all invoices for the vendor’s services and travel due to a lack of staffing. This could lead to the IRS paying for services not received or for unauthorized travel. In addition, inadequate contract administration has resulted in the contractor performing an estimated $7 million of work without following the proper procurement procedures.
Staffing Costs of Consolidation Are Not Accurately Budgeted, Captured, and Reported
Improvements are needed to ensure that the IRS accurately budgets, captures and reports all SCMC staffing costs to the Congress. Some service center employees who were working on the SCMC Project were not charging their time to the correct project. Without complete and accurate information on the costs of the SCMC, IRS management cannot accurately report to the Congress or estimate the future costs of consolidation. While we could not determine the actual amount unreported, we estimate that for the three sites tested (TCC, KCSC, BSC), there were approximately $1.07 million in unreported labor costs for Fiscal Year (FY) 1998.
Summary of Recommendations
The following summarizes the specific recommendations contained in the report:
Management’s Response: IRS management provided an adequate, detailed response to our first memorandum which addressed the first recommendation (see Appendix VI). Management’s response to the remainder of the issues in this report was not available at the time this final report was issued. We provided the IRS with a draft of this report on June 30, 1999, with a 30-day calendar comment period.
This review was initiated as a continuation of the initial Service Center Mainframe Consolidation (SCMC) review (Readiness for Service Center Mainframe Consolidation, Reference Number 085812, dated September 21, 1998).
The overall objective of the review was to determine whether SCMC project management controls were adequate to ensure that: (1) problems identified during the Kansas City Service Center (KCSC) consolidation were addressed before any additional consolidations occurred; (2) the Internal Revenue Service (IRS) minimized the impact on taxpayer rights; and (3) budget postings for work on this Century Date Change (CDC) initiative were accurate. We also evaluated corrective actions taken as a result of the prior audit.
We conducted the current review between September 1998 and January 1999 according to Government Auditing Standards. We interviewed key personnel and obtained and reviewed documentation at the following sites.
We issued two memoranda and held discussions with IRS Management and Finance (M&F) and Information Systems (IS) Division management to advise them of the most significant issues identified. The memoranda and the IRS’ response to the first memorandum are included as Appendices V, VI, and VII. We issued the second memorandum near the completion of fieldwork, and did not ask for an IRS management response.
Details of our audit objective, scope, and methodology are presented in Appendix I. Major contributors to this report are listed in Appendix II. Abbreviations used in this report are listed in Appendix IV.
The SCMC Project is one of four critical projects monitored monthly by the IRS Commissioner’s Combined Management Program for CDC and Filing Season Executive Steering Committee. The CDC initiative addresses computer problems regarding Year 2000 (Y2K).
In September 1997, the IRS formed the SCMC Project Office within the IS Division, with an objective to consolidate mainframe operations from 10 service centers to mainframes at the 2 computing centers. In June 1998, the Memphis Service Center (MSC) became the first service center to consolidate its mainframe operations into a computing center. During this audit, two other service centers’ mainframe operations were consolidated— the KCSC on September 19, 1998, and the BSC on December 4, 1998. The IRS is currently reassessing the project schedule for consolidating the remaining seven service centers.
The IRS plans to spend approximately $360 million on the SCMC during Fiscal Years (FY) 1997 through 1999. For FY 1998, the Congress set limitations on how the IRS spent funds appropriated for the SCMC, and directed the IRS to provide quarterly reports tracking its progress in meeting this CDC initiative.
Mainframes Targeted For Consolidation
The consolidation process involves moving applications that currently reside on mainframe computers in IRS service centers (facilities currently used to process tax returns) to mainframe computers at facilities referred to as computing centers. After consolidation, data input by the service centers will be transmitted to the computing centers for further processing, and the resulting data output will be sent back to the service centers.
The applications residing on four mainframe systems are being consolidated as part of the SCMC project.
The IRS is investing a significant amount of resources on the SCMC’s goal of ensuring that the consolidated service centers and computing centers timely and accurately complete the processing of tax data. The consolidation is a very complex task and requires extensive coordination and effort by several contractors and IRS functions. While the SCMC Project is making significant progress toward this goal, there are several risks that IRS management must continue to address.
The KCSC consolidation resulted in negative impact on taxpayers and IRS operations. IRS management has worked hard to minimize taxpayer impact and implement additional actions based on lessons learned from that consolidation.
As a result, the SCMC project achieved some success with the subsequent BSC consolidation and minimized the impact on taxpayers. However, there is still a need to improve future consolidations. During our review, we reported the following issues to IRS M&F and IS management.
As a result, IRS IS management has initiated several corrective actions based on our recommendations (see Appendix VI).
During the audit, we provided IRS M&F and IS management with detailed information on the issues that follow.
Critical Risks Related to Service Center Mainframe Consolidation Remain Unresolved
We issued a memorandum on November 10, 1998, advising IRS IS management that several risks relating to testing, preparation for consolidation (automation readiness), and staffing were still present prior to the BSC consolidation. These issues were originally reported to IRS management during the prior review.
We believe these risks contributed, in part, to problems experienced by the KCSC and the TCC after consolidation.
KCSC Mainframe Consolidation to the TCC
When the KCSC mainframe operations were consolidated to the TCC on September 19, 1998, significant problems were encountered. For example, the methods used to transfer data from site to site were not efficient, some programming changes were not effectively implemented, and detailed standard operating procedures had not been developed for the consolidated environment.
In our November 10, 1998, memorandum we reported our assessment of the impact of mainframe consolidation on KCSC operations and taxpayers (see Appendix V). We also assisted IRS management in identifying affected taxpayers.
Systemic and procedural problems were experienced during the KCSC consolidation. Systemic problems included:
Procedural problems included:
Computing Centers Continue to Experience Staffing Shortages
In the November 10, 1998, memorandum we also reported that the TCC and the MCC continued to report staffing shortages with few options remaining to correct the problem, including directed reassignments which must be negotiated with the National Treasury Employees Union (NTEU).
Furthermore, we determined that if the causes of the processing and staffing problems were not corrected before the 1999 filing season, there would be a very high risk of adverse impact on a significant number of taxpayers (see Appendix V).
Recommendations
We recommended that before consolidating any additional service centers to computing centers, IS management should:
This should include all known combinations of system-to-system and site-to-site movements of raw and processed electronic data in the consolidated and service center environments, and resulting final dispositions, whether distributed internally or externally, in printed or electronic form.
Management’s Response: IRS IS management agreed to initiate the following corrective actions.
BSC Mainframe Consolidation to the MCC
The BSC mainframe operations were consolidated to the MCC on December 4, 1998. While this consolidation was more successful than the KCSC consolidation, the IRS did experience some problems during the BSC consolidation. We reviewed both the preparation for consolidation and the actual transfer of processing from the BSC to the MCC (known as cutover).
Review of Preparation for Consolidation
We evaluated the implementation of lessons learned from the KCSC consolidation at both the BSC and the MCC and, in November 1998, we met separately with IRS IS management for both locations to discuss the results of those reviews.
BSC staff was working on several issues, including the following. These items were timely completed before the BSC consolidation to the MCC.
The MCC staff was also working on several issues. We identified the following problems during the review of their preparations.
Also, system-to-system balancing was not implemented for the KCSC consolidation. MCC personnel developed a process to help ensure they receive and balance all mid-level system inputs from the BSC prior to processing these files to the mainframes. This process is a good manual control for the BSC consolidation. However, this process will need to be automated prior to the MCC assuming the workload of additional service centers.
In addition to reviewing the IRS’ progress on addressing issues identified in the KCSC consolidation, we identified these additional risks, which we recommended be addressed before the BSC consolidation.
No backup Single Point of Operation (SPO) or Distributed Enterprise Print Controller (DEPCON) servers were present in the MCC configuration, potentially making those servers single points of failure.
Management’s Actions: After we verbally reported these issues, IRS IS management took the following actions prior to the BSC consolidation.
With the exception of another critical system backup (SPO server), the remaining items are being addressed by the SCMC Project Office, and were not considered crucial for resolution prior to the BSC consolidation (see Appendix VI).
Results of the Transfer of Computer Processing From the BSC to the MCC
The BSC consolidation was encouraging for the SCMC Project, and the return to normal processing times occurred twice as fast for this consolidation as it did for the KCSC consolidation. Some of the areas improved by addressing lessons learned from the KCSC consolidation include:
A significant contributing factor to the successful BSC consolidation was the amount of cooperation and communication between the MCC and the BSC. Also, the BSC provided on-site support to the MCC during the initial weeks of consolidation. This also appears to have had a positive impact on initial consolidated operations at the MCC.
Some problems occurred during initial processing after the BSC consolidation, including some programs that did not work and telecommunication hardware failures. IRS management took actions to alleviate some problems, such as using manual deposit procedures to ensure timely deposits. The IRS also temporarily suspended taxpayer notices for one week to minimize the impact on taxpayers. IRS management did not quantify taxpayer impact in its documentation detailing initial problems experienced after consolidation.
Start-up problems included server failures, unexpected system drop-off of some user sessions, and automated scheduling.
Additional lessons learned to improve future service center consolidations
While the BSC consolidation was very encouraging for the SCMC Project, there are still some lessons learned that can be taken forward for the remaining service center consolidations.
Augmentation uses the selected hardware, software, and procedures to position the service centers for consolidation into the two computing centers. Augmentation activities include reducing mainframe 9-track round reel tape output by converting to 36-track square tape cartridges or virtual tape, which are the common media used by the computing centers.
The number of round reels received from the BSC caused significant problems at the MCC on the first day of consolidation. The BSC could not convert some programs to create square tape cartridge output due to programming (internal assigns) present in Tier I applications. The contractor has created a program to help future consolidating service centers eliminate this problem. Also, IRS IS management is creating a National Virtual Tape Handler inclusion list to reduce the number of tape cartridges needed for service center consolidations.
4. Staffing – The BSC did not have adequate liaison staff trained and in place prior to consolidation.
Management’s Actions: The SCMC Project Office has begun discussing actions that need to be taken prior to consolidating another service center. We concur with management’s proposed actions.
Controls Over Technical Aspects of Contract Administration Remain Inadequate
In the prior review, we recommended that IRS management provide additional resources to adequately define and validate SCMC contract requirements, and that the Project Office coordinate with the Procurement Division during price negotiations. In its response, IRS management agreed with the recommendations and established an August 1998 completion date.
We followed up on the implementation of corrective actions and issued a memorandum on January 13, 1999, advising IRS M&F and IS management of the following concerns. As of October 1998, the IRS had not completed the corrective actions, and inadequate controls over the technical aspects of contract administration continue to be a problem for the SCMC project.
The Project Office is not coordinating with the Procurement Division to ensure effective contract administration. As a result, delivery orders (goods and services contracted for) were not adequately defined, and goods and services were provided by the contractor without authorization from the appropriate IRS official—the Contracting Officer (CO). Lack of defined contract requirements has diminished the IRS’ negotiating position. Inadequate contract administration has resulted in the contractor performing an estimated $7 million of work without authorization from the CO. In addition, the invoices for services and travel are not being adequately verified due to lack of resources.
Delivery Orders Under the Contract Used to Fund SCMC Have Not Been Adequately Defined
Contract requirements for the SCMC Project have not been adequately defined, which could result in additional contract costs. As of October 1998, 32 of 44 delivery orders (for about $175 million of $215 million) remained undefinitized.
Undefinitized delivery orders are those in which the contractor has been authorized to begin work, but the cost of the work has only been proposed and not formally negotiated. Although the funds for these tasks have been obligated, the Procurement Division cannot timely and effectively negotiate final prices with the contractor because of undefined requirements and incomplete technical evaluations by the Project Office.
This issue was initially reported in a May 1998 memorandum. The Project Office responded that by August 1, 1998, additional resources would be assigned to adequately define and validate delivery order requirements, and to coordinate with the Procurement Division until all requirements could be defined for negotiations. This has not occurred. A lack of resources in the Project Office and late proposals from the contractor were among the reasons cited. The SCMC Project Manager explained that, in many cases, by the time the proposal was complete, the IRS’ requirements had changed so the proposal was of no use.
The contract used to fund the SCMC is a Firm Fixed Price (FFP) Level of Effort contract. The hardware and software are FFP - the price is not subject to adjustment based on the contractor’s costs. The services are level of effort - the contractor is required to provide a specified level of effort, over a stated period of time, on work that can only be stated in general terms. When using a FFP contract, requirements need to be well defined to allow fair and reasonable prices to be negotiated.
Undefinitized delivery orders provide the IRS the flexibility to issue delivery orders quickly but, in a FFP environment, expose the IRS to possible liability for additional costs if the pricing elements are not quickly finalized. The IRS’ aggressive implementation schedule for consolidation adds a great deal of pressure to the Project Office and the contractor to complete the project timely. The schedule did not allow time for well thought out requirements, which resulted in the Procurement Division issuing undefinitized delivery orders to get the project started. The pressure to stay on schedule has further hindered defining requirements, thus preventing delivery orders from being timely definitized.
In response to the Project Office’s inability to define and validate the requirements, Procurement management has developed a plan to definitize the delivery orders by June 1999. The plan will be carried out in three steps which involve:
Since the requirements have not been defined and validated, the IRS is paying the contractor for its actual costs until the prices are negotiated. The earlier the prices are negotiated, the better chance the IRS has of negotiating a lower price. However, since the implementation is already under way, the IRS has lost significant leverage in negotiating lower prices.
Invoices for Services and Travel Are Not Being Adequately Verified
The Project Office has not provided the resources needed to verify invoices for services and travel, which could result in IRS paying for goods or services not received. On large, complex contracts, there should be IRS staff on-site, called Government Task Managers (GTMs), who report to the CO’s Technical Representative (COTR) in the SCMC Project Office. The GTMs verify invoiced items, such as travel and labor hours worked, and report acceptance of those items to the COTR.
No GTMs have been assigned on-site due to pressure to stay on schedule. However, the Project Office is currently working on assigning GTMs on-site. Without the information GTMs provide, the COTRs cannot adequately verify items invoiced. This could potentially lead to the IRS paying for services not received or for unauthorized travel.
Currently, the COTRs sign Surveillance Reports to verify the hours worked and travel taken by the contractor. These reports state that, to the best of the COTRs’ knowledge, the contractor worked the hours invoiced; however these hours are unverified and the Surveillance Reports do not provide for any visual inspection of goods or services.
Goods and Services are Being Procured Which Were Not Authorized by the Contracting Officer
The SCMC Project Office made decisions to order goods and services without obtaining approval by the CO due to an immediate need for the items. This resulted in several procurements, estimated at $7 million, which were made without following proper procedures.
According to the Federal Acquisition Regulations, only the CO has the authority to authorize work that binds the government. An unauthorized procurement is a procurement for which someone other than the CO instructs the contractor to perform work. In cases of unauthorized procurements, the CO then has to decide whether to ratify the procurement, i.e., approve the procurement and agree to pay for the actions.
Although the funds were available and the items were needed, the Project Office did not follow the proper procurement process. They did not submit a delivery order request to the CO for processing, approval, and issuance to the contractor authorizing work to begin. For the delivery orders in question, it appears a Project Office employee authorized the contractor to begin work without the issuance of a delivery order by the CO.
In September 1998, the CO and COTR became aware of the Project Office’s actions, ratified the procurement and issued undefined delivery orders and/or modifications to fund the procurements. However, the IRS will pay the contractor for only 50 percent of these procurements until the required proposals for the work are submitted and approved.
Recommendations
Management’s Action and Resulting Cost Savings: IRS IS management stated that delivery orders were definitized, resulting in $19 million in savings from prior requisitions.
Management’s Response: Management’s response was not available for inclusion in the report at the time this final report was issued. We provided the IRS with a draft of this report on June 30, 1999, with a 30-day calendar comment period.
Staffing Costs of Consolidation Are Not Accurately Budgeted, Captured, and Reported
Staffing costs of consolidation are not being accurately budgeted, captured, and reported to the Congress by the SCMC Project Office. As a result, we estimate that there was approximately $1.07 million in unreported staffing costs in FY 1998 for the three sites tested (TCC, KCSC, BSC). Without complete and accurate information on the staffing costs of SCMC, IRS management cannot accurately report to the Congress, as required, or estimate the future costs of consolidation. As a result, both the FY 1999 budget and the projected FY 2000 budget could be significantly understated.
A factor which affected the Project Office's ability to accurately budget, capture, and report staffing costs is the funding of staff by two different IRS functions in FY 1998. The IS Division, including the Project Office, is funded by the Chief Information Officer; IS personnel in service centers are funded by the Chief Operations Officer.
Public Law Requirement: Public Law 105-61 provided for the IRS’ 1998 appropriations. The related FY 1998 Conference Report stated, in part, "The conferees are concerned that the Century Date Change requirements are not yet finalized and projects and activities considered as part of the program may frequently change. The conferees direct the IRS to provide quarterly reports tracking its progress in meeting this strategy. The report should include expenditure of funds, application of full-time equivalents (FTEs)…."
An FTE is the equivalent of one person working eight hours per work day (including holidays and paid leave) for one year. SCMC staffing costs are estimated and budgeted in FTEs through individual office budgets. Actual expenses are then recorded using the Project Cost Accounting System (PCAS) code to allow for tracking of budget and actual expenditures by project on the Automated Financial System (AFS). The PCAS is the official source for reporting IS project costs, and the SCMC Project has been assigned a specific PCAS code.
Because IRS IS management did not issue comprehensive directions for budgeting and charging staffing costs, offices were not consistent in how they did so. The SCMC Project Office budgeted $3.7 million for its immediate staff. Other offices, such as the service centers and district offices, estimated and
budgeted their staffing needs through their individual budgets. AFS reports show that:
To confirm the time charges on the AFS report, we attempted to contact 146 of the 218 employees working on the SCMC at the three locations. Of the 78 employees who responded, 55 (71 percent) stated they worked on the SCMC during FY 1998. However, not all of the time charges for these individuals were reflected under the SCMC PCAS code on AFS. Additionally, other field personnel informed us that staffing costs associated with the SCMC were charged to a general PCAS code and not the SCMC PCAS code.
Because IRS IS management did not issue comprehensive directions regarding budgeting and charging staffing costs, SCMC staffing budgets were inaccurate. Also, the FY 1999 and projected FY 2000 budgets do not include FTEs and labor for the service centers and computing centers.
In February 1997, the Deputy Commissioner had issued a memorandum emphasizing the importance of using PCAS codes for IS expenditures. The SCMC Project Office documented the problem of improper usage of PCAS codes in April 1998. However, IRS management did not take timely action to identify all the offices incurring staffing charges, and did not ensure that all costs were reported.
The Quarterly Congressional Report on IRS’ Y2K Conversion Program (which includes SCMC) does not reflect information on expenditure of funds or FTE budgeting, as required by congressional mandate in the FY 1998 Conference Report. Further, the Project Office is not maintaining sufficient documentation to support the information reported to the Congress.
We identified the following report discrepancies.
The IS Budget Office was responsible for gathering data for the report, and did not require the SCMC Project Office to report its FTE expenditures.
Without a coordinated and comprehensive disclosure of the use of SCMC funds in the quarterly congressional reports showing obligations and expenditures, the IRS cannot ensure an effective accounting and control of appropriated funds, or meet congressional requirements.
Recommendations
Management’s Response: Management’s response was not available for inclusion in the report at the time this final report was issued. We provided the IRS with a draft of this report on June 30, 1999, with a 30-day calendar comment period.
Follow-up on IRS Management’s Corrective Actions for Previously Reported Deficiencies
In addition to the follow-up work on consolidation and contracting risks (see pages 5 and 14), we also followed up on four corrective actions related to the previous review that were scheduled for completion during the current audit period. Three of the four corrective actions were effectively implemented, but the fourth corrective action was only partially completed. Due to changes currently being considered by the IRS that may considerably reduce the staffing resources required of the service centers during consolidation (see Appendix VI), we are not currently recommending that this corrective action be fully completed. Our analysis follows.
Follow-up on Recommendation Number 1
We recommended that the Project Office obtain resources to provide accountability for the identification, documentation, assignment, and monitoring of all short and long-term actions required from the field offices and the national office to successfully implement the consolidation plan. Project management should also update its communications plan to include the necessary weekly meetings, conference calls, etc., and require participation of field offices, as appropriate, to monitor and communicate the status of critical tasks.
Proposed Corrective Actions: The SCMC Project Office will develop a "SCMC Web Page" and work with National Office Communications to have SCMC project information included in workforce transition issues and other appropriate communication channels. In addition, the SCMC Project Office will incorporate the "end user" in the Integrated Process Team process by appointing an "end user" representative to the Integrated Process Team Steering Committee.
Audit Work Performed: We verified that a SCMC Web Page was established. We also reviewed minutes of the Risk Oversight Committee (ROC) meetings and Executive Steering Committee minutes and noted that IRS Executives were present for oversight purposes.
Follow-up on Recommendation Number 2
We recommended that IRS management raise fall-back as a risk through the SCMC ROC and ensure that sub-risks, such as lack of a tested fall-back plan and effects on service level agreements, are considered by the ROC for elimination.
Proposed Corrective Actions: The SCMC Project Office stated that a plan would be created to provide for a three-day window for a service center fall-back to its former environment with minimal data loss. Problems occurring subsequent to this three-day period would be resolved by the IRS and vendor technicians in the new environment.
Audit Work Performed: We determined that fall-back was addressed by the ROC, and BSC’s network consolidation procedures included provisions for fall-back.
Follow-up on Recommendation Number 3
We recommended that the Project Office request written assurances from Service Center and National Office Directors that the necessary resources would be committed to accomplish near-term SCMC objectives.
Proposed Corrective Action: The SCMC Project Office stated they would obtain the formal assurance for resources from Service Center Directors.
Audit Work Performed: We determined that three service center directors had responded that they would provide adequate staffing for the upcoming consolidations. One service center director responded that she would not be able to provide staffing for the upcoming consolidations. After further review, we determined that changes forthcoming in the consolidation method may lead to less staffing needed from the service centers for the upcoming consolidation (see Appendix VI). Therefore, until the IRS determines the amount of service center support needed for the upcoming consolidations, we are not recommending that this action be completed.
Follow-up on Recommendation Number 4
We recommended that the Project Office assign a full-time COTR for the contract, thus separating the COTR’s and Program Manager's duties.
Proposed Corrective Action: The SCMC Project Office will assign a new COTR.
Audit Work Performed: We determined that a new full-time COTR had been assigned.
The SCMC Project goal of consolidating the mainframe processing at 10 service centers into 2 computing centers is a very complex task and requires extensive coordination and effort by several contractors and IRS functions. While the project is making significant progress towards its goal, the IRS needs to ensure that future service center consolidations, technical contract administration, and budget accounting are improved.
Appendix I
Detailed Objective, Scope, and MethodologyThe overall objective of this review was to determine whether the Internal Revenue Service’s (IRS) Service Center Mainframe Consolidation (SCMC) project management controls were adequate to ensure that: (1) problems identified during the Kansas City Service Center (KCSC) consolidation were corrected before additional consolidations occurred, (2) consolidation minimized the impact on taxpayer rights, and (3) budget postings were accurate. We also evaluated corrective actions taken as a result of the prior review, Readiness for Service Center Mainframe Consolidation, Reference Number 085812, September 21, 1998.
I. To evaluate key process controls regarding pre-consolidation readiness and post-consolidation operations at the KCSC and the Brookhaven Service Center (BSC), we:
A. Evaluated the process of run-to-run balancing and system-to-system balancing between the Tier I (mainframe) and Tier II (non-mainframe) systems in the consolidated environment.
B. Evaluated the impact of the KCSC consolidation on taxpayers by determining whether notices, liens, and lien releases were timely processed.
C. Determined the IRS’ readiness to consolidate the BSC’s mainframe operations by assessing actions taken to address lessons learned from the KCSC consolidation.
1. Determined the BSC’s readiness to consolidate its mainframe operations.
2. Determined the Martinsburg Computing Center’s (MCC) readiness to consolidate the BSC mainframe operations.
D. Determined the effectiveness of the IRS’ efforts to control and resolve any adverse effects of the BSC consolidation.
1. Determined the impact of the BSC consolidation on taxpayers.
2. Determined any lessons learned (positive or negative) from the BSC consolidation.
II. To determine if the IRS was effectively assessing and monitoring the costs associated with the SCMC, we:
A. Analyzed the SCMC Budget Summary included in The Quarterly Congressional Report on IRS’ Year 2000 Conversion Program to determine the following:
1. How the information was compiled.
2. If the information was comprehensive and accurate.
3. If the information was consistent with recorded financial events.
4. If the information in the budget summary was consistent with any internally generated reports.
B. Determined whether a Fiscal Year (FY) 1998 Project Cost and Accounting System (PCAS) code was being used to capture SCMC costs and, if it was, conducted the following tests:
1. Interviewed IRS personnel to determine if they charged SCMC full-time equivalent (FTE) time to the PCAS code.
2. Compared a list of personnel who were identified as working on the SCMC with a list of personnel who charged time to the SCMC PCAS code. Contacted any personnel who were not on the PCAS report to determine what PCAS code they charged their time to.
3. Selected 146 of the 218 IRS personnel who charged time to the SCMC PCAS code during FY 1998 at the BSC, the KCSC, and the Tennessee Computing Center (TCC). Attempted to contact the 146 people to determine if the work being performed was related to the SCMC.
4. Through an analysis of FY 1998 expenditures, determined the types of expenditures made related to the SCMC.
III. Conducted follow-up to recommendations/corrective actions taken as a result of the prior review.
A. Evaluated the status of staffing for computing centers.
1. Determined if written staff commitments from Service Center and National Office Directors had been obtained to accomplish near term SCMC objectives.
2. Determined the status of the Letter of Understanding regarding staffing.
3. Determined the adequacy of IRS management’s actions to ensure adequate staffing was available either on-site or remotely at the TCC and the MCC to effectively plan for filing season processing needs.
B. Followed up on contracting concerns.
1. Determined if the project had assigned additional resources to adequately define and validate delivery order requirements.
2. Determined if IRS Information Systems management was coordinating with Procurement until all delivery orders were definitized for negotiations.
3. Determined if a new contract administration structure had been implemented.
4. Determined if a new Contracting Officer’s Technical Representative was officially designated.
5. Determined if the baseline change request process and procedures were revised to ensure integration with acquisition, contracting, and budget processes.
C. Determined the status of corrective actions for all project control improvement recommendations.
1. Identified all project control improvement recommendations from the review.
2. Evaluated IRS management's response and proposed corrective action(s).
3. Identified the official(s) responsible for taking corrective action and the proposed completion date(s).
4. Obtained confirmation that corrective action(s) had been taken, completed, or initiated.
5. Evaluated the results of the corrective action(s) taken, completed, or initiated.
Appendix II
Major Contributors to This ReportScott Wilson, Associate Inspector General for Audit (Information Systems Programs)
Control Office
Susan Boehmer, Director
Danny Verneuille, Audit Manager
Joseph Wabnitz, Audit Manager
Vann Coe, Senior Auditor
Troy Paterson, Senior Auditor
Van Warmke, Senior Auditor
Catherine Cloudt, Auditor
Roy Evans, Auditor
Perrin Gleaton, Auditor
Support Offices
|
Baltimore, Maryland |
Kansas City Service Center |
|
Gary Swilley, Audit Manager |
Gary Manning, Audit Manager |
|
Anthony Choma, Senior Auditor |
Larry Gore, Senior Auditor |
|
Wallace Sims, Senior Auditor |
John Hawkins, Senior Auditor |
|
Mary Thomas, Senior Auditor |
Kenneth Bennett, Auditor |
|
John Baxter, Auditor |
Kathleen Hughes, Auditor |
|
Joseph Cooney, Auditor |
Ronald Stuckey, EDP Auditor |
|
Brookhaven Service Center |
Office of Audit Projects |
|
Robert Irish, Audit Manager |
Vincent Dell’Orto, Audit Manager |
|
Carol Gerkens, Auditor |
Theresa Haley, Senior Auditor |
|
Gail Schuljan, Auditor |
Andrew Harvey, Auditor |
Appendix III
Report Distribution ListCommissioner C
Deputy Commissioner Modernization C:DM
Deputy Commissioner Operations C:DO
Chief Information Officer IS
Deputy Chief Information Officer, Operations IS
Assistant Commissioner, National Operations IS:O
Director, Martinsburg Computing Center IS:O:M
Acting Director, Tennessee Computing Center IS:O:T
Deputy Chief Information Officer, Systems IS
Assistant Commissioner, Systems Development IS:S
Project Director, Mainframe Consolidation IS:S:MC
Assistant Commissioner, Service Center Operations IS:SC
Office of Information Resources Management IS:IR
Chief Management and Finance M
Assistant Commissioner (Program Evaluation and Risk Analysis) M:OP
Chief Financial Officer M:CFO
National Director for Budget M:CFO:B
Office of Management Controls M:CFO:A:M
Assistant Commissioner, Procurement M:P
Chief Operations Officer OP
Executive Officer for Service Center Operations OP:SC
Director, Kansas City Service Center D
Director, Brookhaven Service Center D
National Director for Legislative Affairs CL:LA
Audit Liaisons:
Service Center Mainframe Consolidation IS:O:O
Deputy Chief Information Officer, Systems Development IS:S
Deputy Chief Information Officer for Information Resources Management IS:IR
Chief, Audit Assessment and Control IS:I:IS:O:A
Management Controls Coordinator for Office of Chief, Management and
Finance M
Executive Assistant to the Assistant Commissioner (Procurement) M:P
Executive Officer for Service Center Operations OP:SC:SP:P
Appendix IV
Abbreviations Used in This ReportACS – Automated Collection System
AFS – Automated Financial System
BSC – Brookhaven Service Center
CDC – Century Date Change
CO – Contracting Officer
COTR – Contracting Officer’s Technical Representative
CRS – Communications Replacement System
DEPCON – Distributed Enterprise Print Controller
ECL – Executive Control Language
FFP – Firm Fixed Price
FTE – Full-Time Equivalents
FY – Fiscal Year
GTM – Government Task Managers
ICS – Integrated Collection System
IRM – Internal Revenue Manual
IRS – Internal Revenue Service
IS – Information Systems
JCL – Job Control Language
KCSC – Kansas City Service Center
M&F – Management and Finance
MCC – Martinsburg Computing Center
MSC – Memphis Service Center
NTEU – National Treasury Employees Union
PCAS – Project Cost Accounting System
PRINT – Printer Replacement to Integrate New Tools
ROC – Risk Oversight Committee
RIS – Request for Information Services
SCMC – Service Center Mainframe Consolidation
SCRS – Service Center Replacement System
SPO – Single Point of Operation
TCC – Tennessee Computing Center
TEBB – Taxpayer Service Electronic Bulletin Board
Y2K – Year 2000
Appendix V
Memorandum #1: Outstanding Risks Related to Service Center Mainframe ConsolidationDEPARTMENT OF THE TREASURY
INTERNAL REVENUE SERVICE
2900 WOODCOCK BLVD
CHAMBLEE, GA 30341
REGIONAL INSPECTOR
November 10, 1998
RESPONSE DUE: November 25, 1998
MEMORANDUM FOR CHIEF INFORMATION OFFICER IS
FROM: for Acting Regional Inspector /s/M. Susan Boehmer
Southeast Region
SUBJECT: Internal Audit Memorandum – Outstanding Risks Related to Service Center Mainframe Consolidation
This is the first memorandum issued to you as part of our second phase of auditing the Service Center Mainframe Consolidation (SCMC) project. Several risks we identified in our earlier review relating to testing, automation readiness and staffing are still present. We believe these risks may have contributed, in part, to post cutover problems experienced by the Kansas City Service Center KCSC) and Tennessee Computing Center (TCC) from September 21, 1998 to the present.
We are providing the attached information to advise you of our concerns regarding the continued transfer of service center mainframe processing operations to computing centers until all identified high-risk issues are satisfactorily resolved.
If you are in agreement with the facts presented, please follow the procedures for responding to audit memoranda outlined in Internal Revenue Manual (IRM) 1289. Due to the time sensitive nature of these issues, I would appreciate your response within ten workdays. If you disagree with any of the facts presented or would like to discuss these issues further, please contact acting Assistant Regional Inspector Susan Boehmer at 770-986-6910 or Audit Manager Danny Verneuille at 901-546-3111.
Attachment
CHIEF INFORMATION OFFICER -
AttachmentCritical Risks Related to Service Center Mainframe Consolidation Remain Unresolved
In our earlier review of SCMC, we issued several Internal Audit Memorandums between February and May of 1998 that identified issues which included, but were not limited to:
At present, these issues still prevail and continue to hinder the success of the SCMC project while exposing the Service to unnecessary risks. Further, if the causes of these problems are not corrected before the 1999 filing season, there is a very high risk of adverse impact on a significant number of taxpayers. In the seven weeks following the Kansas City Service Center (KCSC) mainframe cutover to TCC:
The following issues, impacting taxpayers, resulted from the KCSC consolidation:
Additional concerns:
If the computing centers combined are at 75% of authorized staffing at this stage of consolidation, it is not clear how they will operate increasing workloads and also be responsible for business resumption/disaster recovery for one another.
Recommendations
Before cutover of any additional service centers to computing centers, management should:
(1) Ensure all critical operational and technical aspects of consolidation at computing centers and each individual service center (including KCSC) are:
This should include all known combinations of system-to-system and site-to-site movements of raw and processed electronic data in the consolidated and service center environments and its resulting final disposition, whether distributed internally or externally in printed or electronic form.
Appendix VI
Management's Response to Memorandum #1Response has been removed due to its size. To see the complete Response, please go to the Adobe PDF version of this report.
Appendix VII
Memorandum #2: Follow-up to Contracting IssuesDEPARTMENT OF THE TREASURY
INTERNAL REVENUE SERVICE
WASHINGTON, D.C. 20224
January 13, 1999
Response Due: January 21, 1999
MEMORANDUM FOR CHIEF INFORMATION OFFICER IS, ASSISTANT COMMISSIONER (PROCUREMENT) M:P
(for)
/s/ Nancy A. NakamuraFROM: Acting Assistant Regional Inspector (Internal Audit)
Southeast Region
SUBJECT: Internal Audit Memorandum # 2: Follow-up to Contracting Issues
This is the second Internal Audit Memorandum (IAM) issued as part of our follow-up audit of the Service Center Mainframe Consolidation (SCMC) project. We have identified concerns in the subject area that warrant your immediate attention. Some of the issues in the IAM were previously reported in May 1998, but have not been corrected. We have discussed the issues in this IAM with the Contracting Officer and the Project Director for Mainframe Consolidation.
This IAM is being presented for your information and agreement to the facts. If you are in agreement, please follow the procedures for responding to audit memoranda outlined in Internal Revenue Manual (IRM) 1289. If you do not agree with the facts in this memorandum, please contact my office within five workdays.
Results
Inadequate controls over the contract administration and lack of defined requirements continue to be problems for the SCMC project. Management stated by August 1998, they would provide additional resources to adequately define and validate delivery order requirements. Also, they would coordinate with Procurement until all requirements could be definitized for negotiations to reduce its backlog of undefinitized delivery orders. As of October 1998, this has not occurred. This has diminished the Service’s negotiating position and resulted in the contractor performing work estimated to be approximately $7 million without the Contracting Officer’s knowledge. Specifically, the Program Office is not coordinating with Procurement to ensure effective administration of the Service Center Support Services (SCSS). This has resulted in the delivery orders under the SCSS contract not being adequately defined and goods and services provided without authorization from the Contracting Officer, which could be construed as unauthorized procurements. In addition, the invoices for services and travel were not being adequately verified due to lack of resources.
Delivery Orders under the SCSS Contract Have Not Been Adequately Defined
As of October 1998, 32 (totaling approximately $175 million) of 44 (totaling approximately $215 million) delivery orders obligated under the SCSS contract for SCMC remain undefinitized. Although the funds for these tasks have been obligated, Procurement has not been able to timely and effectively negotiate final prices with the contractor because of undefined requirements and incomplete technical evaluations.
The issue with undefinitized delivery orders was previously reported in a May 1998 Internal Audit Memorandum. The Program Office responded that by August 1, 1998, additional resources would be assigned to adequately define and validate delivery order requirements and to coordinate with Procurement until all requirements could be definitized for negotiations. This has not occurred. The Program Manager explained that a lack of resources in the Program Office and late proposals from the contractor were among the reasons that the requirements and technical evaluations were not completed. He explained that in many cases, by the time the proposal was complete, the Service’s requirements had changed so the proposal was of no use.
The SCSS contract is a Firm Fixed Price (FFP) Level of Effort contract. The hardware and software is FFP and the services are level of effort. In a FFP environment, requirements need to be well defined to allow fair and reasonable prices to be negotiated. Undefinitized orders are those in which the contractor has been authorized to begin work, but the cost of the work has only been proposed and not negotiated. This technique provides the Contracting Officer (CO) the flexibility to issue delivery orders quickly, but in a FFP environment, exposes the Service to possible liability for additional costs if the pricing elements are not quickly finalized.
Additionally, the extremely aggressive implementation schedule for consolidation adds a great deal of pressure to the Program Office and the contractor to complete the project by Year 2000. The schedule did not allow time for well-thought-out requirements that resulted in Procurement issuing undefinitized delivery orders to get the project started. The pressure to stay on schedule has further hindered defining requirements, thus prevented definitization of delivery orders.
In response to the Program Office’s inability to define and validate the requirements, Procurement has developed a plan to definitize the delivery orders. The plan will be carried out in three steps which involve obtaining information from the contractor regarding goods and services ordered. Procurement will validate the information for pricing accuracy and the Program Office will validate it for configuration accuracy, quantity, and delivery schedule and sites. The whole process is expected to be complete in June 1999.
This is not the best way to determine what goods and services have been provided. At this late date, the Service may be unable to verify the contractor claims or to question whether particular goods and services were actually delivered. However, considering that Procurement must have the delivery orders definitized to negotiate prices and the Program Office has not provided what is needed, this plan may be the only way to obtain the information Procurement needs.
Also, since the requirements have not been defined and validated, the Service is paying the contractor for their actual costs until the prices are negotiated. The earlier the prices are negotiated, the better chance the Service has of negotiating a lower price. However, since implementation is under way, the Service has lost its leverage in negotiating lower prices.
Furthermore, since the requirements have not been defined, the contractor’s performance cannot be measured and evaluated to determine if it is meeting contractual requirements.
Invoices for services and travel cannot be adequately verified
Because of the pressure to stay on schedule, the Program Office has not provided the resources needed to verify invoices for services and travel. On very large, complex contracts there should be staff on site, Government Task Managers (GTMs), who report to the Contracting Officer’s Technical Representatives (COTR). The GTMs verify equipment deliveries, labor hours worked, travel, etc. and report receipt and acceptance of these items to the COTR. On tasks as large as the SCMC, GTMs are necessary. The Program Office is currently working on assigning GTMs on site, but without the information they provide, the COTRs cannot adequately verify the invoices.
This could potentially lead to the Service paying for services not received or for unallowable travel. Currently, the COTRs sign Surveillance Reports to verify the hours worked and travel taken by the contractor. These reports state that, to the best of the COTRs’ knowledge, the contractor worked the hours invoiced, but they do not provide for any visual inspection of goods or services.
Goods and Services are being provided which were not authorized by the Contracting Officer (CO)
The aggressive implementation schedule influences decisions for the consolidation project. Decisions to order goods and services without following the proper procurement process were made due to the immediate need for these items. This resulted in several procurements with an estimated value of $7 million, which could be construed as unauthorized procurements. According to the Federal Acquisition Regulations, only the Contracting Officer (CO) has the authority to authorize work and bind the government. An unauthorized procurement is a procurement for which someone other than the CO instructs the contractor to perform work. Procurement then has to decide whether or not to ratify the procurement. Ratifying the procurement means the CO approves the procurement and agrees to pay for the actions.
Although the funds were available and the items were needed; the Program Office did not follow the proper procurement process. This requires the Program Office to submit a delivery order request to be processed by the CO and results in a delivery order being issued to the contractor. The delivery order authorizes the contractor to begin to perform work. For the delivery orders in question, it appears a Program Office employee authorized the contractor to begin work without the issuance of a delivery order by the CO.
In September 1998, the COTRs and COs became aware of these actions and Procurement issued delivery orders and/or modifications (undefinitized) to fund the procurements. Also, the Service will only pay the contractor for 50% of these procurements until the required proposals for the work are submitted and approved.
We recommend that:
IRS management ensure that proper procurement procedures are followed to obtain goods and services by taking the following actions: