Officials from Rogers Communications Inc. and a slew of other stakeholders are set to appear at a parliamentary committee in Ottawa on Monday to explain how the company’s recent massive network outage happened, and outline the steps they are taking to make sure it won’t happen again.
On the morning of July 8, the entire Rogers network — from cable television, to internet and wireless telephone services — went down, and remained so for much of the day while the company scrambled to figure out what happened.
Late in the day, the company isolated and addressed the core problem, which they said was caused by a software update gone wrong, which then cascaded into a massive chain reaction that took down the entire system.
The outage triggered a slew of seemingly unrelated systems to fail beyond the company itself, as things as varied as debit payments, 911 services and government services were unavailable for much of the day, causing havoc.
Rogers customers were caught off guard by Friday’s massive outage involving both mobile and internet networks, which also caused widespread disruption for banks, businesses and some emergency services across Canada.
A wide cross section of Canadian life was touched by the outage, sparking anger from consumers and government officials who demanded compensation and assurances that the system will be remade with failsafes included to ensure there are functioning backups when things go wrong.
The Canadian Radio-television and Telecommunications Commission (CRTC), the country’s telecom regulator, was at the top of the pile of those seeking a satisfying explanation. The CRTC sent Rogers a letter on the following Tuesday demanding answers to dozens of questions.
The company was given a deadline of July 22 to reply to those concerns and apparently did so, as the regulator published a redacted version of Rogers’ explanations on the CRTC’s website on Friday evening.
On Sunday, Rogers released more details of what it plans to do, which includes:
- Ensuring that 911 calls switch automatically to other carriers in the event of an outage.
- Separating wireless and cable systems so that an outage in one should not impact the other.
- Spending $10 billion to beef up its networks.
“I know that it is only through these actions that we can begin to restore your confidence in Rogers and earn back your trust,” Rogers CEO Tony Staffieri said.
Questions remain, however, and politicians and the public will get a chance to prod the company for more details at hearings in Ottawa starting at 11 a.m. ET, when senior Rogers officials including Staffieri, chief regulatory officer Ted Woodhead and newly minted chief technology officer Ron McKenzie will appear in front of the Standing Committee on Technology and Industry.
Rogers has been trying to assuage anger since the outage, with Staffieri saying that the firm is going to do whatever it takes to makes things right. “It is clear that what matters most is that we ensure this doesn’t happen again,” Staffieri said in an open letter to Canadians. “You have my personal commitment that Rogers will make every change and investment needed to help ensure that it will not happen again.”
Comments from Ron McKenzie should be especially interesting to watch, since he has only been on the job for less than a week, after the previous CTO, Jorge Fernandes, was ousted on July 21.
“I don’t think he had any choice but to resign or depart,” said Patrick Horan, an analyst at Agilith Capital in Toronto, of Fernandes’ sudden ouster. “He had to fall on the sword in a very delicate situation.”
Horan said Rogers’ explanation — that a problem with a third party’s equipment managed to knock down all the company’s services — doesn’t sit right in the telecom world, where redundancies and backups are the norm.
He said it’s inconceivable to imagine a similar outage at a company whose network had been built for wireless technology exclusively, as opposed to a firm like Rogers which was a cable company first and foremost that has since moved into telecommunications yet seemingly has key parts of its infrastructure on a legacy system.
Cable companies “don’t have the robust standards that telecom networks have,” Horan said. “If it was a third party vendor or software vendor that had an upgrade … there should have been testing before any upgrade went on the system,” he said.
“This is part of the learning curve of a cable network becoming a communications company.”
Outage may yet knock out Shaw merger, too
The hearings come against the backdrop of Rogers trying to finalize its $27-billion takeover of Calgary-based Shaw in a deal that would make Canada’s concentrated telecom industry even more top-heavy.
Rogers pitched the merger more than a year ago and the marriage has already proven to be harder to get to the altar than anticipated.
About a month after the merger plan was announced last year, Rogers’ wireless network went down across most of the country for about a day, irking customers and raising questions about whether or not it is in Canadians’ best interest to allow Rogers to buy up Calgary-based Shaw, the owner of Freedom Mobile, which is a distant fourth in Canada’s wireless industry.
Last year, the CRTC signed off on allowing Rogers to buy Shaw’s broadcasting assets, but much of the focus has been on the wireless side of the business. Rogers seemingly solved that problem when it announced a deal to sell the wireless business to Quebec-based Quebecor, but hurdles remain.
The company set a self-imposed deadline of July 15 to finalize the merger, but that day came and went with no update from anyone involved. With regards to the entire Shaw deal, Rogers earlier this year set a revised date of July 31 to finalize the merger, and doubts are seemingly mounting as that day nears.
Investors certainly don’t seem to think it’s a done deal. Shaw shares were trading for about $35 a share on Friday — well below the more than $40 Rogers has pledged to pay for the company.
Competition concerns
Securing the OK for the merger from Canada’s competition watchdog, the Competition Bureau, is proving to be the highest hurdle of all.
In May, the Competition Bureau filed an injunction seeking to stop the merger until it can decide whether it would be bad for consumers. Rogers and Shaw agreed to talks with the regulator, but even before the outage, they expressed concerns that those talks were going nowhere.
Andrew Cameron, a small business owner in Amherst, N.S., is among those who want to see the Bureau start doing more to ensure competition and less rubber stamping of massive mergers.
“[Rogers and Shaw] say this merger will generate efficiencies,” the podcaster and co-founder of the Canadian Anti-Monopoly Project said in an interview. “How does that benefit me? Because those efficiencies just basically translate into increased margins [for them.]”
On the day of the outage, Cameron was like millions of Canadians who found himself without a working cellphone, although he said he maintained internet at home because he gets that from a different provider. But he couldn’t use many key services he normally relies on.
“Imagine running a small business and not being able to take Interac for a whole day,” he said.
Cameron is among those who hopes the focus in Ottawa stays trained on what’s best for local communities.
“If you want to merge, you should have to show us how it’s not anti-competitive,” he said.
