[UPDATE: Some customers reported full or partial restoration of service from around 1pm. At 1.19pm, Telecom's official Twitter account weighed in with: "Some sites are back up, this is not yet stable. Please bear with us." As of 6.17pm, the telco said all of the 80,000 to 90,000 customers affected should have services restored.
Some cell sites became congested as traffic returned in a rush.
For those still experiencing problems, Telecom recommends turning off your cellphone, removing its battery, waiting several seconds, then reinserting it and restarting.]
Since 4.30am this morning, Telecom XT customers from Taupo to the southern tip of the country have been without voice, txt or mobile broadband service.
Details have been scant through Telecom's website or through official press statements, but an email sent to corporate clients and obtained by NBR explains: "an unplanned restart of our Christchurch XT Mobile RNC switch which has resulted in the majority of XT cell sites south of Taupo not operating."
Restoration of service is expected "later today".
One switch, two switch ...
Regardless of issues of 2G fallback (discussed below), one industry insider told NBR the problem was that Telecom only had two RNC switches ("a large bit of kit that coordinates all the traffic coming in and out of the network from cellsites").
Telecom has two RNCs - one for Taupo and north (located in Auckland), the other for everything south of Taupo (located in Christchurch). Each has multiple redundant parts, but apparently not the ability to cover for the other. Vodafone, by contrast, has five "which cover each other" and a sixth coming on stream during summer.
Telecom had no immediate comment on the RNC theory.
The catastrophic outage reveals the new network’s Achilles’ heel: XT's sole reliance on 3G spectrum.
Telecom has a second nationwide network, of course - it’s older CDMA service- but that’s incompatible with its shiny new XT.
Vodafone customers, by contrast, should be able to trip down to its older 2.5G/2G (GPRS/GSM) service when 3G stuffs out - for their phones are compatible with the telco’s older network.
I’ve consistently found XT faster in more places, as advertised, but this is one area where Vodafone, at least on paper, has the edge (I say at least on paper since a couple of brief Auckland outages have seen Vodafone’s 2G network simply overhelmed after 3G has gone down, nullifying any benefit of the backup).
2degrees also has a 3G/2G hybrid network (albeit with the 3G bit still under construction) in main centres, giving it the same fall-back benefit (and elsewhere, it piggybacks on Vodafone's infrastructure).
Above: blogger Lance Wiggs promotes a freestyle reinterpretation of Telecom's XT coverage map.
A so-so effort keeping customers in the loop
A number of people have complained about the lack of information on Telecom’s home page, or its XT home page (certainly, I couldn’t find any; I’m sure it’s there somewhere, but it should be front and centre).
A number of customers questioned Telecom's assertion that emergency calls could still be made. on XT mobiles that seemed "as dead as a brick" as one Twitter use put it.
Telecom’s Twitter team deserves more kudos. Updates have been constantly flowing through the social network (follow Telecom on Twitter here).
And customers of Telecom’s corporate division, Gen-i got sent detailed messages from chief executive Chris Quin at 9:41am, 11:04am, 12:16pm and 2:15 pm.
This article is tagged with the following keywords. Find out more about MyNBR Tags
- Trump slams Amazon, again, wiping $US5b from its market cap
- Poll result: Should English stand down National’s Ohariu candidate?
- Labour tops $500K in post-Ardern donations, two big-note contributions revealed
- More questions emerge about Sam Ovens’ rags to riches story
- Powerhouse, Hydroworks shareholders remain in the dark
Most listened to
- Craigs Investment Partners' Mark Lister discusses whether Fletcher Building can recover from its troubled year
- NBR's Jenny Ruth on the end of a fractious era for Abano
- Richard Dellabarca discusses a new investment framework for investment framework for the Seed Co-investment Fund
- Democracy means politicians give people what they want, not what they need. Rodney Hide thinks that's a shame.
- Bill Ralston on first debate moderator's Mike Hosking's bias, and broader bias in the media
- NBR Radio: best of the week ended August 11, with Grant Walker