TREASURY
INSPECTOR GENERAL FOR TAX ADMINISTRATION
Uninstalled Computer Security Patches Continue to Put Computer Systems at Risk
September 21, 2006
Reference Number: 2006-20-167
This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.
Redaction Legend:
2(b) = Law Enforcement Guideline(s)
7 = Predecisional Staff Recommendations or Suggestions to Agency Decision Makers
8 = Information Reflecting the Bureau’s Decision-making Processes
Phone Number | 202-927-7037
Email Address | Bonnie.Heald@tigta.treas.gov
Web Site |
http://www.tigta.gov
September 21, 2006
MEMORANDUM FOR ACTING CHIEF INFORMATION OFFICER
FROM: Michael R. Phillips /s/ Michael R. Phillips
Deputy Inspector General for Audit
SUBJECT: Final Audit Report – Uninstalled Computer Security Patches Continue to Put Computer Systems at Risk (Audit # 200520035)
This report presents the results of our review to assess the effectiveness of the Internal Revenue Service’s (IRS) practices for ensuring the identification and installation of security updates for computer systems and applications.
Impact on the Taxpayer
When vendors identify security flaws with their
systems, they make security patches[1] available to be
installed on their customers’ computers.
The IRS process for installing patches has not ensured all of its
100,000 computers have been adequately protected. As a result, sensitive taxpayer information
is more susceptible to unauthorized disclosure to hackers and unethical
employees and contractors, and computer systems are more vulnerable to
disruptions of operations that could jeopardize and waste taxpayer dollars.
Synopsis
In May 2004, the IRS suffered one of its most significant computer security incidents when the Sasser Worm[2] propagated itself throughout the entire IRS computer network. The incident could have been avoided if an available security patch had been installed on infected systems. Operational organizations within the IRS were notified numerous times by the Office of Mission Assurance and Security Services to install the patch from April 14, 2004, when the patch became available, through May 2, 2004, when the Worm first infected IRS systems. However, the patch was not applied to servers consistently and was not applied to any workstations. The Worm cost the IRS an estimated $3.6 million in lost salaries and $50.6 million in lost or delayed tax assessments and tax collections.[3]
Despite
recent improvements to patch management practices, the IRS continues to have
unpatched computers throughout its infrastructure.
From June 2001 through February 2006, we issued 11 reports containing patch management issues.[4] During our current and prior reviews, we found patch identification, testing, and monitoring efforts were generally adequate. The IRS has established a vulnerability and remediation group tasked with identifying software for improving the overall management of this process. Additionally, the IRS has implemented corrective actions related to patch management issues from our prior reports. Finally, during the aftermath of the Sasser Worm incident, the IRS conducted an internal review that identified breakdowns in procedures and recommended corrective actions to prevent such events from recurring.
Although the IRS has made commendable progress towards improving its patch management processes, controls over patch implementation continue to allow unpatched systems. For example:
The patches were not always installed for two primary reasons: the automated approach used to install patches on Windows-based systems did not always have valid connections to the systems requiring patching, and system administrators did not always install patches due to the impact they believed such patches would have on systems under their control or due to the labor-intensive process of manually installing patches on numerous systems.
As for its internal review of the Sasser Worm incident, the IRS either took no corrective actions or did not complete corrective actions for 3 of the 10 recommendations. While the IRS has formed a group to develop stronger patch management controls, the scope of the group’s work is limited and not designed to address the causes mentioned above. As of September 2005, the group estimated full implementation of the controls within its scope may not occur for an additional 12 months to 18 months.
Ineffective IRS patch management practices continue to put
the IRS network at risk. The IRS
continues to be exposed to network intrusions that could result in enormous
financial impact related to lost or delayed tax assessments and collections and
nonfinancial impact related to lost productivity, similar to the effects that
occurred when the Sasser Worm infiltrated the IRS.
Recommendation
Because we have included recommendations related to patch management issues in our prior audit reports and the IRS is taking actions to address patch management, we made no additional recommendations in this report. We will continue to monitor the IRS’ patch management strategy and report any actions taken to eliminate the risks or deficiencies identified in our future security-related reviews.
Response
IRS management agreed with the facts in our report and noted they continue to take aggressive approaches towards improving the patch management process. The IRS has developed a self-install script (computer program) that identifies and installs patches on workstations and laptops. A nationwide roll out of this script is scheduled to be completed by February 2007. The IRS has also taken steps to improve the success rate of patch distributions to workstations. These steps include aggressive management of Tivoli® endpoints and considering an approach that would not allow workstations onto the network until missing patches are updated. Management’s complete response to the draft report is included as Appendix V.
Patch Installation
Practices Continue to Result in Unpatched Computer Systems
Appendices
Appendix
I – Detailed Objective, Scope, and Methodology
Appendix
II – Major Contributors to This Report
Appendix
III – Report Distribution List
Appendix V
– Management’s Response to the Draft Report
Abbreviations
|
CIO |
Chief Information Officer |
|
COE |
Common Operating Environment |
|
CSIRC |
Computer Security Incident Response Center |
|
IRS |
Internal Revenue Service |
A patch is
a fix of a design flaw in a computer program.
When patches are not installed timely, hackers could exploit the
unpatched weakness and assume control of a computer.
A 2004 Computer Security Institute and Federal
Bureau of Investigation survey[7] showed that 91 percent
of the respondents believed their computer system intrusions could have been
prevented if system administrators had implemented patches for countering known
vulnerabilities. A patch is a fix of a
design flaw in a computer program.
Patches must be installed or applied to the appropriate computer for the
flaw to be corrected. While vendors try
to address known security flaws immediately, a time gap occurs from when the
problem becomes publicly known until the vendor prepares the update to correct
the flaw and users install the update. This
gap, which provides potential intruders an opportunity to take advantage of the
known flaws and mount attacks on vulnerable computers and networks, is becoming
increasingly shorter as technology increases and hackers get wiser.
For this reason, it is critical, particularly
for high-risk security vulnerabilities, that organizations apply security patches
as quickly as possible. The potential
risk of an unpatched weakness varies, depending on the nature of the
weakness. A hacker could exploit an
unpatched weakness and take over control of a computer to access its contents
(e.g., user accounts, password information), use the computer as a launching
point to attack other computers, or simply damage the computer so no one else
can use or access it.
The actual installation of a patch appears to be a simple task. However, two factors complicate and challenge this task. First, all computers to which the patch applies must be identified and patched. The larger the organization, the more computers are likely to exist and be affected by vulnerabilities. Second, there are thousands of vulnerabilities being identified each year. The CERT® Coordination Center[8] determined 5,990 security vulnerabilities were reported during 2005. Vulnerabilities are generally spread across different software products. The more types of software used within an organization, the more difficult the task of patching all affected software products becomes.
The Internal Revenue Service (IRS) is a large organization with almost 100,000 employees and is very reliant on automation and the use of computers to administer the nation’s tax system. It has over 100,000 computers containing various operating systems and applications. Consequently, the seemingly easy task of patching computers turns into a monumental effort.
Computer security patching can be segmented into four basic processes: identification, testing, distribution and installation, and monitoring and follow-up.
Various organizations within the IRS manage the patch process. The Computer Security Incident Response Center (CSIRC) within the Office of Mission Assurance and Security Services has primary responsibility for identifying and notifying the Chief Information Officer (CIO) and business unit organizations about the availability of patches. It also conducts patch monitoring and follow-up. Upon being notified of patches, system administrators from various functions under the CIO and business units conduct patch testing, installation, monitoring, and follow-up, depending on the type of computer and user. For example, the End User Equipment and Services organization under the CIO is responsible for end-user computers, so it is also responsible for testing, installing, and following up on patches for IRS employees’ computers.
This review was performed in the office of the CIO at the
IRS National Headquarters in Washington, D.C., and New Carrollton, Maryland,
during the period November 2005 through April 2006. This review also relied on results presented
in 11 of our security-related audit reports issued from June 2001 through
February 2006.[9] The audit was conducted in accordance with Government Auditing Standards.
Detailed information on our audit objective, scope, and methodology is
presented in Appendix I. Major
contributors to the report are listed in Appendix II.
In May 2004, the IRS suffered one of its most significant computer security incidents when the Sasser Worm[10] propagated itself throughout the entire IRS computer network. The incident could have been avoided if an available security patch had been installed on infected systems. Operational organizations within the IRS were notified numerous times to install the patch from April 14, 2004, when the patch became available, through May 2, 2004, when the Worm first infected IRS systems. However, the patch was not applied to servers consistently and was not applied to any workstations. The Worm cost the IRS an estimated $3.6 million in lost salaries and $50.6 million in lost or delayed tax assessments and tax collections.[11]
During our current and prior reviews, we found patch identification, testing, and monitoring efforts were generally adequate. For example:
During Fiscal Year 2005, the IRS established a vulnerability and remediation group tasked with identifying software for improving the overall management of the patching process. This group includes managers and technicians from IRS computer security and operations functions. During the first phase of this project, the group identified whether any vendor software existed that could improve the IRS patch management process. The second phase of this project would involve implementing any software procured.
Additionally, the IRS has implemented corrective actions from our prior reports containing patch management issues. For example, the IRS established procedures for identifying, testing, and monitoring security patches. Finally, the IRS conducted an internal review during the aftermath of the Sasser Worm incident. This review identified breakdowns in procedures and recommended corrective actions to prevent such an occurrence from happening again. These issues included breakdowns in the communication and implementation processes that have since been addressed.
Although the IRS has made commendable progress towards improving its patch management processes, controls over patch installation continue to require attention.
Patch Installation Practices Continue to Result in Unpatched Computer Systems
The IRS installs patches on its computers either through automated processes or by having system administrators manually apply the patches to specific servers or workstations under their control. However, these processes did not always ensure required patches were installed on all computers. We identified the following problems with patch installation:
Despite
recent improvements to patch management practices, the IRS continues to have
unpatched computers throughout its infrastructure.
Ø ****7,8,2(b)****
Ø ****7,8,2(b)****
Security patches were not always installed for two primary reasons:
The IRS responded to the aftermath of the Sasser Worm incident with a review that presented 10 findings and recommended corrective actions. ****7****
While the IRS has formed the aforementioned vulnerability and remediation group to develop stronger patch management controls, the scope of the group’s work is limited and not designed to address the causes previously discussed. As of September 2005, the group estimated full implementation of the controls within its scope may not occur for an additional 12 months to 18 months.
Ineffective IRS patch management practices continue to put the IRS network at risk. The IRS continues to be exposed to network intrusions that could result in enormous financial impact related to lost or delayed tax assessments and collections and nonfinancial impact related to lost productivity, similar to the effects that occurred when the Sasser Worm infiltrated the IRS.
Recommendation
Because we have included recommendations related to patch management issues in our prior audit reports and the IRS is taking actions to address patch management, we are making no additional recommendations at this time. We will continue to monitor the IRS’ patch management strategy and will report any actions taken to eliminate the risks or deficiencies identified in our future security-related reviews.
Management’s
Response: IRS management agreed with the facts in
our report and noted they continue to take aggressive approaches towards
improving the patch management process.
The IRS has developed a self-install script (computer program) that
identifies and installs patches on workstations and laptop computers. A nationwide roll out of this script is
scheduled to be completed by February 2007.
The IRS has also taken steps to improve the success rate of patch
distributions to workstations. These
steps include aggressive management of Tivoli® endpoints and
considering an approach that would not allow workstations onto the network until
missing patches are updated.
Appendix I
Detailed Objective, Scope, and Methodology
The overall objective of this review was to assess the effectiveness of the IRS’ practices for ensuring the identification and installation of security updates for computer systems and applications. To accomplish our objective, we:
I. Determined whether the IRS effectively distributed and installed patches by consulting with appropriate staff and reviewing documentation, including IRS scans of Windows-based servers and workstations.
II. Determined whether the IRS effectively tested patches prior to installation to applicable computing devices by identifying and evaluating the testing process.
III. Determined whether the IRS effectively followed up on patch installation and proactively identified unpatched computers by consulting with appropriate organizations and reviewing documentation.
IV. Determined the status and progress of IRS actions to address issues related to the Sasser Worm[19] by consulting with appropriate offices and reviewing relevant documentation.
V. Determined the status and progress of IRS actions to address recommendations from eight Treasury Inspector General for Tax Administration reports issued from Fiscal Years 2001 through 2004 that contained issues related to patch management.
VI. Reviewed the results from three Treasury Inspector General for Tax Administration reports issued after Fiscal Year 2004 that contained issues related to patch management. We did not review IRS actions to address the recommendations contained in these three reports because corrective actions had not been completed at the time of our review.
VII.
Determined
the status of the IRS vulnerability and remediation group for addressing
security patch issues.
Appendix II
Major Contributors to This Report
Margaret E. Begg,
Assistant Inspector General for Audit (Information Systems Programs)
Kent Sagara, Acting
Director
Joseph Cooney, Acting
Audit Manager
Bret Hunter, Lead
Auditor
Jody Kitazono, Senior
Auditor
Larry Reimer, Senior
Auditor
Appendix III
Commissioner C
Office of the
Commissioner – Attn: Chief of Staff C
Deputy Commissioner for Operations Support OS
Chief, Mission Assurance and Security Services OS:MA
Deputy Chief Information Officer OS:CIO
Associate Chief Information Officer, End User Equipment and Services OS:CIO:EU
Associate Chief Information Officer, Enterprise Networks OS:CIO:EN
Associate Chief Information Officer, Enterprise Operations OS:CIO:EO
Director, Information Security OS:CIO:IS
Director, Enterprise Systems Management OS:CIO:EU:ESM
Chief Counsel CC
National Taxpayer Advocate TA
Director, Office of Legislative Affairs CL:LA
Director, Office of
Program Evaluation and Risk Analysis
RAS:O
Office of Internal
Control OS:CFO:CPIC:IC
Audit
Liaison: Director, Program Oversight OS:CIO:SM:
Appendix IV
Prior Treasury Inspector
General for Tax Administration Audit Reports With Security Patch Management
Issues
The following Treasury Inspector General for Tax Administration audit reports contain patch management issues.
Appendix V
Management’s
Response to the Draft Report
The
response was removed due to its size. To
see the response, please go to the Adobe PDF version of the report on the TIGTA
Public Web Page.
[1] A patch is a fix of a design flaw in a computer program. Patches must be installed or applied to the appropriate computer for the flaw to be corrected.
[2] The Sasser Worm exploited a flaw in the Local Security Authority Subservice System on Microsoft Windows computers and transferred additional exploit code to the computers. It also probed for other computers to infect. This Worm rendered computers inoperable.
[3] The $50.6 million estimate was identified by the IRS in its post-Sasser Worm evaluation soon after the incident occurred. The IRS has since stated that tax assessments and tax collections would have been processed by the IRS in subsequent tax periods and, therefore, do not represent actual losses.
[4] See Appendix IV for a list of the audit reports included in this review.
[5] See Appendix IV, Report 1.
[6] See Appendix IV, Report 2. Tivoli® is a registered trademark owned by International Business Machines.
[7] The 2004 Computer Crime and Security Survey was conducted by the Computer Security Institute with participation of the San Francisco Federal Bureau of Investigation’s Computer Intrusion Squad. The 2004 survey results were based on the responses of 494 computer security practitioners across the United States.
[8] The CERT® Coordination Center is a center of Internet security expertise located at the Software Engineering Institute, a Federally funded research and development center operated by the Carnegie Mellon University.
[9] See Appendix IV for a list of the audit reports included in this review.
[10] The Sasser Worm exploited a flaw in the Local Security Authority Subservice System on Microsoft Windows computers and transferred additional exploit code to the computers. It also probed for other computers to infect. This Worm rendered computers inoperable.
[11] The $50.6 million estimate was identified by the IRS in its post-Sasser Worm evaluation soon after the incident occurred. The IRS has since stated that tax assessments and tax collections would have been processed by the IRS in subsequent tax periods and, therefore, do not represent actual losses.
[12] The Law Enforcement Manual Checker is an internally developed software suite designed to scan computer systems to ensure compliance with various computer security standards required by the IRS.
[13] See Appendix IV, Report 1.
[14] Tivoli® is a registered trademark owned by International Business Machines.
[15] See Appendix IV, Report 2.
[16] ****7,8,2(b)****
[17] ****7,8,2(b)****
[18] ****7,8,2(b)****
[19] The Sasser Worm exploited a flaw in the Local Security Authority Subservice System on Microsoft Windows computers and transferred additional exploit code to the computers. It also probed for other computers to infect. This Worm rendered computers inoperable.
[20] Tivoli® is a registered trademark owned by International Business Machines.