Smoke shuts down government provisioning at federal data centre in Canada
Fri 29 Jul 2016
Several government data services, including Blackberry email and the Phoenix payroll system, were interrupted when smoke set off alarms and caused an evacuation at the Canadian government’s core IT department, Shared Services Canada.
According to sources in the Ottawa data centre, no actual fire was detected inside the building, despite the smoke, which would seem to indicate either that some cooling facet failed in one of the components, short of actually igniting, or that someone was smoking in the data centre. The centre is investigating the cause of the incident.
After workers were evacuated, the shutdown affected 50,000 public service workers, mostly due to the downtime for Blackberry email servers.
The outage was particularly unfortunate for the already-damaged reputation of the IBM-designed Phoenix payroll system, problems with which have become so extensive that an emergency summer meeting of the House of Commons operations and estimates committee was called recently to address the grievances of the 80,000 civil staff who have experienced problems in receiving their salaries.
The Shared Services data centre is part of a government initiative to consolidate 485 data centres into just seven, taking the Canadian government’s DC footprint down from 600,000 square feet to 180,000. Projections for development also estimate that the current government tally of 23, 424 servers will be reduced to 14,369, but that the current storage capacity of 70 petabytes will grow to 482 petabytes.
Though it can’t account for furtive smokers, fire is a critical insurance issue for the data centre industry. In 2015 a fire in the connectivity room of a British Telecom data centre in Belfast caused major outages, while in the hot summer of 2012 two government data centres were hit by major fires on two continents, affecting government services at the IBM data centre in Calgary and the Unique ID Project data centre at Greater Noida in the Gautam Budh Nagar district of the northern state of Uttar Pradesh in India.
In Germany one build has decided to radically lower the oxygen content of its operation, significantly reducing fire risk – an approach which will be easier to implement once the already low-staffing levels at the latest builds reduce even further.