[UPDATE: Some customers reported full or partial restoration of service from around 1pm. At 1.19pm, Telecom's official Twitter account weighed in with: "Some sites are back up, this is not yet stable. Please bear with us." As of 6.17pm, the telco said all of the 80,000 to 90,000 customers affected should have services restored.
Some cell sites became congested as traffic returned in a rush.
For those still experiencing problems, Telecom recommends turning off your cellphone, removing its battery, waiting several seconds, then reinserting it and restarting.]
Since 4.30am this morning, Telecom XT customers from Taupo to the southern tip of the country have been without voice, txt or mobile broadband service.
Details have been scant through Telecom's website or through official press statements, but an email sent to corporate clients and obtained by NBR explains: "an unplanned restart of our Christchurch XT Mobile RNC switch which has resulted in the majority of XT cell sites south of Taupo not operating."
Restoration of service is expected "later today".
One switch, two switch ...
Regardless of issues of 2G fallback (discussed below), one industry insider told NBR the problem was that Telecom only had two RNC switches ("a large bit of kit that coordinates all the traffic coming in and out of the network from cellsites").
Telecom has two RNCs - one for Taupo and north (located in Auckland), the other for everything south of Taupo (located in Christchurch). Each has multiple redundant parts, but apparently not the ability to cover for the other. Vodafone, by contrast, has five "which cover each other" and a sixth coming on stream during summer.
Telecom had no immediate comment on the RNC theory.
The catastrophic outage reveals the new network’s Achilles’ heel: XT's sole reliance on 3G spectrum.
Telecom has a second nationwide network, of course - it’s older CDMA service- but that’s incompatible with its shiny new XT.
Vodafone customers, by contrast, should be able to trip down to its older 2.5G/2G (GPRS/GSM) service when 3G stuffs out - for their phones are compatible with the telco’s older network.
I’ve consistently found XT faster in more places, as advertised, but this is one area where Vodafone, at least on paper, has the edge (I say at least on paper since a couple of brief Auckland outages have seen Vodafone’s 2G network simply overhelmed after 3G has gone down, nullifying any benefit of the backup).
2degrees also has a 3G/2G hybrid network (albeit with the 3G bit still under construction) in main centres, giving it the same fall-back benefit (and elsewhere, it piggybacks on Vodafone's infrastructure).
Above: blogger Lance Wiggs promotes a freestyle reinterpretation of Telecom's XT coverage map.
A so-so effort keeping customers in the loop
A number of people have complained about the lack of information on Telecom’s home page, or its XT home page (certainly, I couldn’t find any; I’m sure it’s there somewhere, but it should be front and centre).
A number of customers questioned Telecom's assertion that emergency calls could still be made. on XT mobiles that seemed "as dead as a brick" as one Twitter use put it.
Telecom’s Twitter team deserves more kudos. Updates have been constantly flowing through the social network (follow Telecom on Twitter here).
And customers of Telecom’s corporate division, Gen-i got sent detailed messages from chief executive Chris Quin at 9:41am, 11:04am, 12:16pm and 2:15 pm.
This article is tagged with the following keywords. Find out more about MyNBR Tags
- Winston's choice: Jacinda Ardern poised to become prime minister
- NZ's top tech companies generate $10b in annual sales in 2017: TIN100 report
- Kraft Heinz buys ANZ Cerebos assets for $A290 million
- Super Fund CEO pay rise 'too large,' Joyce says
- Nats baulking over Winston Peters' demand for more ministers
Most listened to
- Rob Hosking on Winston's choice
- IDC's Chayse Gorton on Kiwis' online vs offline shopping preference - and how it's out of step with the rest of the world
- NZSA chief executive Michael Midgley on how he will vote undirected Fletcher proxies
- Restaurant Brands' Grant Ellis discusses progress at the fast food group
- Rob Hosking says politicians need to understand the effect their promises will have on what the Reserve Bank has to do
- AMP Capital investment manager Jonathan Armstrong discusses why an expansion is right for Tauranga's Bayfair shopping centre
- NBR Radio: The best interviews, with Grant Walker — updated daily