Virginia Releases Report On August 2010 State Computer Disruption
The McDonnell Administration and the Joint Legislative Audit and Review Committee (JLARC) released today the results of the external audit of the events surrounding the August state computer disruption.
On August 25, 2010, the failure of a key hardware component owned and operated by Northrop Grumman at the state data center (and subsequent delays in restoring operations) caused a significant impact to the operation of several state agencies.
These delays had an adverse effect on the citizens of the Commonwealth and resulted in unexpected expense to state agencies.
All told, outages related to the disruption impacted 13% of the Commonwealth’s executive branch file servers and 26 of 89 applicable executive branch agencies, including, most visibly, the Virginia Department of Motor Vehicles.
In the wake of the disruption, Governor McDonnell ordered that immediate steps be taken:
- First he directed Department of Motor Vehicle offices in every region of the state to hold unprecedented weekend open hours in order to accommodate Virginians inconvenienced by the outages.
- Second, he called for a comprehensive external audit of the situation, to be overseen jointly by the Executive and Legislative branches, and paid for by Northrop Grumman.
Working in conjunction with JLARC, the McDonnell Administration, represented by Secretary of Technology Jim Duffey, chose Agilysys, a provider of information technology services to Fortune 50, 500, and mid-tier customers, to conduct the audit.
The audit’s major findings are:
- The outage resulted from the failure of a key data storage system (EMC DMX-3) and subsequent human error during its repair.
- Loss of data, and the delay in restoring operations and data, resulted from the failure to follow two key industry best practices. Also, that there was an insufficient degree of self-governance towards continuous process improvement and the management of risk.
- Many components selected for the data center and its IT infrastructure meet or exceed industry best practices, but that implementation falls short in certain areas.
Governor McDonnell issued the following statement regarding the results of the audit:
“The disruption to our state computer system last August caused the Commonwealth to incur significant expenses. It impacted a broad array of state agencies, including some of the agencies our citizens most commonly interact with on a daily basis. Many Virginians were inconvenienced by this disruption and lost hours of their time in dealing with the outages. It was an unacceptable failure and one that cannot be allowed to reoccur.
In conjunction with Northrop Grumman, the Virginia Information Technologies Agency, led by Sam Nixon, worked around the clock to bring all agencies back on line and fix the situation as quickly as possible. When that was completed, we moved immediately to take every step necessary to assist impacted Virginians. We also called for a thorough external audit that would tell us exactly how this situation occurred and what after-action steps should take place to both address the recent outage and to prevent future disruptions of service.
The findings from the external audit conducted by Agilysys, and formally released today, confirm our basic understanding of the circumstances that led to the computer outage and subsequent delays in recovery and full restoration of agency operations. Further, the report that has been presented provides a path forward for the Commonwealth to recover losses from Northrop Grumman for the outage pursuant to the contract between the parties.
To this end, I have directed Secretary of Technology Jim Duffey to accelerate, and bring to a prompt conclusion, discussions with senior Northrop Grumman executives on compensation and corrective actions. We are committed to holding all state contractors accountable for the performance of their duties on behalf of the Commonwealth and its citizens.”
Topics: 2010, backlog, contract, data services, data storage system, Department of Motor Vehicles, dispute resolution, DMV, economic development, economic growth, Economy, EMC DMX-3, external audit, failure, Governance, government, Governor Bob McDonnell, help desk, human error, ICT, IT, JLARC, Joint Legislative Audit and Review Committee, monitor, Motor Vehicle offices, news, Northrup Grumman, outsourcing, private sector, rapid response team, technology, U.S., United States, Virginia, Virginia Information Technologies Agency, VITA
Print This Article in Plain Text
|
|
Latest Headlines
- Osama Bin Laden Dead?
- Florida To Share $1 Billion For Gulf Coast Restoration Efforts
- Texas Prays For Rain During Extreme Drought Conditions
- Governor Christie Takes Action To Reduce Diesel Emissions, Improve Air Quality
- Governor McDonnell Outlines Draft Transportation Improvement Program
- Alaska Energy Bill Passes Legislature
- Illinois State Building Receives ENERGY STAR Award
- Homeland Security Begins Using National Terrorism Advisory System
- U.S. Outlines New Health Insurance Options For Graduating College Students
- Electrolux Expands In North Carolina, To Create 200 Jobs In Charlotte
- Virginia Storm Victims To Receive Tax Extension
- U.S. Expands Broadband Investment For Rural Areas To Accelerate Job Creation
- U.S. To Invest $130 Million For Advanced Energy Research
- Michigan Passes “I’m Sorry” Bill
- Kentucky Sells Two State Aircraft To Save Taxpayer Dollars
