[UPDATE: Some customers reported full or partial restoration of service from around 1pm. At 1.19pm, Telecom's official Twitter account weighed in with: "Some sites are back up, this is not yet stable. Please bear with us." As of 6.17pm, the telco said all of the 80,000 to 90,000 customers affected should have services restored.
Some cell sites became congested as traffic returned in a rush.
For those still experiencing problems, Telecom recommends turning off your cellphone, removing its battery, waiting several seconds, then reinserting it and restarting.]
Since 4.30am this morning, Telecom XT customers from Taupo to the southern tip of the country have been without voice, txt or mobile broadband service.
Details have been scant through Telecom's website or through official press statements, but an email sent to corporate clients and obtained by NBR explains: "an unplanned restart of our Christchurch XT Mobile RNC switch which has resulted in the majority of XT cell sites south of Taupo not operating."
Restoration of service is expected "later today".
One switch, two switch ...
Regardless of issues of 2G fallback (discussed below), one industry insider told NBR the problem was that Telecom only had two RNC switches ("a large bit of kit that coordinates all the traffic coming in and out of the network from cellsites").
Telecom has two RNCs - one for Taupo and north (located in Auckland), the other for everything south of Taupo (located in Christchurch). Each has multiple redundant parts, but apparently not the ability to cover for the other. Vodafone, by contrast, has five "which cover each other" and a sixth coming on stream during summer.
Telecom had no immediate comment on the RNC theory.
The catastrophic outage reveals the new network’s Achilles’ heel: XT's sole reliance on 3G spectrum.
Telecom has a second nationwide network, of course - it’s older CDMA service- but that’s incompatible with its shiny new XT.
Vodafone customers, by contrast, should be able to trip down to its older 2.5G/2G (GPRS/GSM) service when 3G stuffs out - for their phones are compatible with the telco’s older network.
I’ve consistently found XT faster in more places, as advertised, but this is one area where Vodafone, at least on paper, has the edge (I say at least on paper since a couple of brief Auckland outages have seen Vodafone’s 2G network simply overhelmed after 3G has gone down, nullifying any benefit of the backup).
2degrees also has a 3G/2G hybrid network (albeit with the 3G bit still under construction) in main centres, giving it the same fall-back benefit (and elsewhere, it piggybacks on Vodafone's infrastructure).
Above: blogger Lance Wiggs promotes a freestyle reinterpretation of Telecom's XT coverage map.
A so-so effort keeping customers in the loop
A number of people have complained about the lack of information on Telecom’s home page, or its XT home page (certainly, I couldn’t find any; I’m sure it’s there somewhere, but it should be front and centre).
A number of customers questioned Telecom's assertion that emergency calls could still be made. on XT mobiles that seemed "as dead as a brick" as one Twitter use put it.
Telecom’s Twitter team deserves more kudos. Updates have been constantly flowing through the social network (follow Telecom on Twitter here).
And customers of Telecom’s corporate division, Gen-i got sent detailed messages from chief executive Chris Quin at 9:41am, 11:04am, 12:16pm and 2:15 pm.
This article is tagged with the following keywords. Find out more about MyNBR Tags
- Kevin Roberts, Rob Fyfe join Kiwi start-up's board
- Budget 2017: Joyce announces tougher debt target and $11b new infrastructure investment
- Impending house price falls will sideswipe economic growth
- Opinion: The real story on immigrant work visas
- NZ dairy expansion will hit limits as environmental impact grows, must chase value, Guy says
Most listened to
- Greg Thompson canvasses the case for tax cuts
- Budget 2017: Infometrics' Gareth Kiernan expects house prices in December of this year to be sitting below last year's
- Harbour's Oyvinn Rimer on A2 Milk's sales upgrade
- John Bowie on angry judges, Beverly Hills shop, retiring judges and Max Harris in this week's Briefcase
- The NBR Radio interviews most listened to from the third week of April, with Grant Walker