[UPDATE: Some customers reported full or partial restoration of service from around 1pm. At 1.19pm, Telecom's official Twitter account weighed in with: "Some sites are back up, this is not yet stable. Please bear with us." As of 6.17pm, the telco said all of the 80,000 to 90,000 customers affected should have services restored.
Some cell sites became congested as traffic returned in a rush.
For those still experiencing problems, Telecom recommends turning off your cellphone, removing its battery, waiting several seconds, then reinserting it and restarting.]
Since 4.30am this morning, Telecom XT customers from Taupo to the southern tip of the country have been without voice, txt or mobile broadband service.
Details have been scant through Telecom's website or through official press statements, but an email sent to corporate clients and obtained by NBR explains: "an unplanned restart of our Christchurch XT Mobile RNC switch which has resulted in the majority of XT cell sites south of Taupo not operating."
Restoration of service is expected "later today".
One switch, two switch ...
Regardless of issues of 2G fallback (discussed below), one industry insider told NBR the problem was that Telecom only had two RNC switches ("a large bit of kit that coordinates all the traffic coming in and out of the network from cellsites").
Telecom has two RNCs - one for Taupo and north (located in Auckland), the other for everything south of Taupo (located in Christchurch). Each has multiple redundant parts, but apparently not the ability to cover for the other. Vodafone, by contrast, has five "which cover each other" and a sixth coming on stream during summer.
Telecom had no immediate comment on the RNC theory.
The catastrophic outage reveals the new network’s Achilles’ heel: XT's sole reliance on 3G spectrum.
Telecom has a second nationwide network, of course - it’s older CDMA service- but that’s incompatible with its shiny new XT.
Vodafone customers, by contrast, should be able to trip down to its older 2.5G/2G (GPRS/GSM) service when 3G stuffs out - for their phones are compatible with the telco’s older network.
I’ve consistently found XT faster in more places, as advertised, but this is one area where Vodafone, at least on paper, has the edge (I say at least on paper since a couple of brief Auckland outages have seen Vodafone’s 2G network simply overhelmed after 3G has gone down, nullifying any benefit of the backup).
2degrees also has a 3G/2G hybrid network (albeit with the 3G bit still under construction) in main centres, giving it the same fall-back benefit (and elsewhere, it piggybacks on Vodafone's infrastructure).
Above: blogger Lance Wiggs promotes a freestyle reinterpretation of Telecom's XT coverage map.
A so-so effort keeping customers in the loop
A number of people have complained about the lack of information on Telecom’s home page, or its XT home page (certainly, I couldn’t find any; I’m sure it’s there somewhere, but it should be front and centre).
A number of customers questioned Telecom's assertion that emergency calls could still be made. on XT mobiles that seemed "as dead as a brick" as one Twitter use put it.
Telecom’s Twitter team deserves more kudos. Updates have been constantly flowing through the social network (follow Telecom on Twitter here).
And customers of Telecom’s corporate division, Gen-i got sent detailed messages from chief executive Chris Quin at 9:41am, 11:04am, 12:16pm and 2:15 pm.
This article is tagged with the following keywords. Find out more about MyNBR Tags
- $14 billion school property portfolio not up to scratch
- MARKET CLOSE: NZ shares mixed, A2 at record while Fletcher, Tower recover, Sky City drops
- Nosh debts still rebounding on Veritas
- Kiwi Property CEO Gudgeon to step down in 2018
- NBR Rich List backlash the canary in the coal mine, pointing to an ‘anti-wealth’ election: commentator
Most listened to
- Earl Gasparich discusses Oceania's first annual result since listing
- Trade Minister Todd McClay explains a new market access deal New Zealand officials have signed with Thailand
- AgriHQ dairy analyst Susan Kilsby says farmers won't go wild over their spending
- Immigration Minister Michael Woodhouse briefs media on the government's changes to its immigration policy
- Penny Pepperell looks at two reports just out on whistleblowing – one local and one Australasian
- NBR Radio: best of the week ended July 21, with Grant Walker