Senior Telecom manager reveals extent of XT outage

[UPDATE, 7.15pm: NBR just spoke to Gen-i chief executive Chris Quin, who said Telecom chief executive Paul Reynolds will issue a major statement this evening. The statement is expected to answer customers' questions about compensation, and what steps are being taken to avoid a future outage. Mr Quin said the outage had been traced to Christchurch, and that the technical cause had been confirmed - but that it was not the same fault as caused the December 14 problems.]

Read also'Angry' Reynolds orders independent review of XT

Following the massive mobile outage that hit Telecom XT customers from Taupo to bluff from 10am this morning, the company's spin doctors said the problem would be solved by 1pm.

But five hours later, multiple XT customers were telling NBR they were still experiencing problems.

Telecom's spin doctors also said they couldn't say how many customers were affected, or the extent of the voice, text and data problems.

But an email from Chris Quin, chief executive of Telecom's Gen-i Australasia IT and telco services division, obtained by NBR, tells corporate customers a fuller story.

His wording implies that all XT users Taupo south suffered problems, though emails to NBR indicate that not all were effected.

"Bizarrely, my cellphone kept working whilst those around me failed – all (including mine) on the XT network," emailed Festivé Ltd's Gavin Holley, of Christchurch.

Mr Quin's email reads:

Today we experienced issues with our Christchurch XT Mobile RNC switch which has resulted in the degradation of the XT Mobile Network from Taupo south between approximately 10.30am and 12pm.

Users of the XT service from Taupo south will have experienced approximately one hour of degraded mobile service, and approximately 30 minutes of inability to use the XT service in the area.

Since 12pm we begun restoring the XT service, and full service will resume for all clients from Taupo southwards by 1.30pm.

XT services north of Taupo, and CDMA services nationwide, were not affected.

We are establishing the root cause of the degradation of the network.

We continue to understand that mobile is critical for your business, and we apologise and deeply regret any inconvenience or problems you are experiencing due to the interruption of the XT service.

We are aware that this is not the first time that we have experienced service degradation. Please accept my personal assurance that this issue was unexpected and was immediately escalated to the Telecom Executive team and our global partner.

I will communicate with you further with relevant information as it is available, and confirm that full restoration is continuing.

If you wish to discuss this further with me, or any of the Gen-i team, please let us know.

Chris Quin
CEO Gen-i Australasia

One Gen-i corporate customer told NBR he gave Mr Quin credit for including his personal cellphone number at the end of his two messages (a third is expected later this evening). The Gen-i boss was also noteably more candid than his colleagues during XT's previous outage on December 14 - had included his email and cellphone number at the end of the message.

Telecom's Twitter account is advising those still suffering problems to remove their batteries, then reinsert them to reset their phones. 

However, as of 2.59pm, at least a dozen customers tell NBR they still have no service after following this advice.

Telecom blames network congestion as cell sites come back online, and advises people to "keep trying".

UPDATE (3.54pm)
Mr Quin has now sent a second message to customers. It reads:

This is the second email today to update you on the restoration of services for XT mobile clients operating between Taupo and the bottom of the South Island.

From approximately 10.30am this morning we have experienced issues with our Christchurch XT Mobile RNC switch which resulted in the degradation of the XT mobile network from Taupo south. As a result, many users in these regions have not been able to use XT services, including voice, SMS and mobile broadband.

Telecom has restored services for a significant amount of the XT mobile network that has been without service, however, due to the load on the network as services are restored and users commence using the network, we are seeing intermittent service for some clients.

XT services north of Taupo, and CDMA services nationwide, are unaffected.

For some XT clients still experiencing service issues, it may assist to try restarting their phone, by powering down then removing the battery for several seconds, then restarting the phone.

This issue has the highest level of executive escalation and our management and technical teams are working urgently to put a solution in place.

Once again, I apologise and deeply regret any inconvenience or problems you are experiencing due to the interruption of the XT service.

We are keeping our front line teams constantly updated, and will keep them informed on the resolution and next steps.

We will also keep you updated via email.

For further details, or to tell us how we can support your business immediately, please contact me or your Gen-i client manager.

This article is tagged with the following keywords. Find out more about My Tags

Post Comment

48 Comments & Questions

Commenter icon key: Subscriber Verified

"...approximately one hour of degraded mobile service, and approximately 30 minutes of inability to use the XT service in the area." well that's absolute crap. It was down since just after 10 and for a couple of hours at least not 30 mins.

Reply
Share

XT= XTRA TIME OUT OF SERVICE LOL

Reply
Share

My phone still isn't working. Screw XT, I'm changing to Vodafone.

Reply
Share

Yeah - most of us did that years ago !!

Reply
Share

4:07pm and still no service in Invercargill. Tried switching off and removing battery (twice), did not help. Luckily I'm not that popular so I don't think I will have missed to many calls........

Reply
Share

do we get refunds for the inconvenience?

Reply
Share

and this is why vodafone is a million times better :D

Reply
Share

XT = The most unrealiable Mobile Network in the World??

Reply
Share

More false advertising by Telecom? Surely not!!

Reply
Share

Telecom are a no service provider and have been for years. Try no cell coverage 60km north of Auckland for a start. Since 1991 the governance model has been to strip mine and minimise infrastructure to the point it is now worse than a 3rd world telco.

Vodafone are way in front but even they are restricted by Telecoms lack of infrastructure

Time JK et al put a blowtorch down Reynolds Y fronts

Reply
Share

As soon as Vodafone launched in NZ I signed up. Over the first year or so there were numerous outages and very weird problems with the text messaging service. Sometimes texts would take days to reach people I was sending to, or just as long for incoming ones to reach me. Calls were getting disconnected a lot, and calls not going through at all was a regular feature.

I'm not a telecom fanboy, but what is it that people don't get? It's new technology. Like Vodafone when their infrastructure was new, or the old CDMA network when it was new, most of the internet services when they were new, it will have teething problems.

To expect something as huge as a nationwide mobile phone network to be flawless from the start is unreasonoble. This is why I'm still on CDMA. I will switch to XT, but once it's stable.

Reply
Share

Well the big difference here is Vodafone have a back up 2G Network and Telecom chose not to have a back up and are now paying the price.
Telecom NZ should know better as they are on the worlds top 10 bad business decisions. Which was to go CDMA or GSM - Hello! A decade later they go GSM only to make another bad business decision no back up network.
Vodafone - Global Business
Telecom - Local Business

I choose Vodafone all the way!

Reply
Share

Surely all the talk about "best network" and "faster in more places" and "no problems here, Richard" mean we can take action against them? Under investing in the roll out to this degree is pathetic and a huge loss of face.

Reply
Share

But declan, that's exactly what Telecom said. And when its network was interfering with Two Degrees and Vodaphones they still said it was everyone else's fault.

Telecom doesn't know how to run a GSM network and they hired Alcatel, which has never built one, to build one! Talk about the blind leading the blind.

Reply
Share

Heh. Yeah, I'll give you that. But, it'll probably get sorted over time, just like everything else that's big and technical and new. LHC springs to mind for some reason.

Standard procedures for consumers is to jump up and down and act like children when any service has problems though.

Reply
Share

Vodafone never had two major outages affecting half the country in one month Declan.

Reply
Share

yeah. they did. but only in the first couple of months of operation.

Reply
Share

XT absolute rubbish, in one their tv promos featuring a user on a shipping container in the middle of the ocean i fail 2 c how she could've had recep if it doesn't even work properly on land highly frustrating when travelling.

Reply
Share

Are you giving sexual favours to Paul Reynolds?

Reply
Share

Anyone want to swap my xt phone for string and 2 empty baked bean cans....please call me

Reply
Share

Telecom and Transpower must share the same PR company. Might be better to just tell the truth guys or don't you remember what that is?

Reply
Share

XT Customers have every right to "jump up & down" when they are the ones PAYING for a "service" that regularly breaks down - South of Taupo"!!

This stupid XT Network was promoted as the best thing since sliced bread "TO THE HILT" - during every ad break for months, they went on & on & on about the stupid thing!!

Fact is - Alcatel Lucent are the worst choice of partner they could of chosen & Telecom got the cheapest possible deal - which has now landed them in a rather poor position & they WILL lose customers because of this.

Furthermore - this WILL happen again & their share price & customer numbers will most likely fall, to the advantage of Vodafone (who are doing just fine atm).

Reply
Share

Agreed, ALU - have virtually no 3G experience - this is a direct result of Telecom's one network vendor policy which they claim is so successful. Successful for their bank balance maybe, but not for the end users. I bet Ericsson and NokiaSiemens (who should have built this network) are enjoying ALU's pain. Even now Telecom don't mention ALU. Compare this to Rod Fife and the IBM debarcle, he went for them all gun blazing. Makes you wonder what ALU are holding to TNZ's head

Reply
Share

I don't really care as long as we get reimbursed like last time.

Reply
Share

Don't bother with Vodafone - save money your money and go to 2degrees!

Reply
Share

You do realise that 2 Degrees roam on Vodafone NZ's Network, outside of Auckland, Wellington, Christchurch, Dunedin & Queenstown don't you??

Furthermore - 2 Degree's have no TXT Plans or On Account Plans - 2 Degree's is cheaper for calling per min, but with no TXT plans yet, it makes it a rather limited service & they also don't have 3G yet!

Reply
Share

Actualy 2d is also roming here in Duedin on vodas network

Reply
Share

Do you realise that in two months Telecom are withdrawing the $10 TXT offer on the CDMA network. This is of course a ploy to force customers on to their XT network, or should I say their lucky dip XT network when it works.

Reply
Share

Yeah mate. I agree with you and think that this is disgusting. Telecom forcing the big text users to a network that they have proved is unreliable. Surely this must be against law. Fair Trading or something.

Reply
Share

I just read Vodafone NZ have 6 live RNCs and Telecom NZ XT has 2.............Good one TELESCUM

Reply
Share

Who gives a rats arse? Shi* just happens. Go without your phone for a few hours and ENJOY THE BREAK!

Reply
Share

Cheers SCM - thx for the correction. :)

Reply
Share

No worries ;)
Im not sure but I beleave that Queenstown 2d also roams on vodas network

Reply
Share

No - 2 Degree's do have their own coverage in Queenstown

Reply
Share

True, good to know.
Cant wait for 2d to install their own sites here. Last time I spoke with them they were unsure of when if ever they would be setting up sites in Dunedin

Reply
Share

It's been what, 10 and half hours? Still nothing! FIX IT TELECOM.

Reply
Share

New network name is Monica as it goes down so often without warming and screws you!!

Reply
Share

I note Telecom have not told their customers that the entry level R100 phone is only capable of receiving a WCDMA 850MHZ signal. As you know Telecom uses WCDMA 850 and WCDMA 2100 MHZ on their XT network. So in 2100MHZ coverage areas this phone will not work.

Reply
Share

XT still no go for me in Invercargill!!! Beginning to wonder whether it will be up at any time tonite???

Reply
Share

1984 is the year AT&T in USA had their monolpoy ended by the government. It's been a free and competitive market since. 2009 in New Zealand and Telecom are clutching still in the time bubble holding on to their old infrastructure. "High availability" is yet to get in their vocabulary. Did you know Telecom have been the #1 spenders in advertising in the country, only second to Loreal/Garnier ? Telecom have to have huge ad campaigns, in order to present a good image because their product is still years behind.

Reply
Share

It is now 1:45pm and it is still down in Timaru

Reply
Share

I t is incredible that since Gatting and the board decided that they were not running a telco but were setting up a phillipine scam, this country will never have a decent comms structure, our scottish cousin has been brought onto the board to deflect any government interference!!!

At least we do not pay tooooo much for a crap sevice.

Whe they broght Alcatel they would have known that they stuffed up up the Ozz.

but then again, how did air kiwi buy into ansett

Reply
Share

Senior Telecom manager reveals extent of XT outage | The National Business Review - New Zealand - business, markets, finance, politics, property, technology and more adley@gigemail.net

Reply
Share

Senior Telecom manager reveals extent of XT outage | The National Business Review - New Zealand - business, markets, finance, politics, property, technology and more adley@gigemail.net

Reply
Share

Senior Telecom manager reveals extent of XT outage | The National Business Review - New Zealand - business, markets, finance, politics, property, technology and more adley@gigemail.net

Reply
Share

Senior Telecom manager reveals extent of XT outage | The National Business Review - New Zealand - business, markets, finance, politics, property, technology and more guy@gigemail.net

Reply
Share

Senior Telecom manager reveals extent of XT outage | The National Business Review - New Zealand - business, markets, finance, politics, property, technology and more guy@gigemail.net

Reply
Share

Post New comment or question

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.

NZ Market Snapshot

Forex

Sym Price Change
USD 0.7895 0.0010 0.13%
AUD 0.9088 0.0004 0.04%
EUR 0.6375 0.0019 0.30%
GBP 0.5045 0.0003 0.06%
HKD 6.1243 0.0046 0.08%
JPY 92.8860 0.0450 0.05%

Commods

Commodity Price Change Time
Gold Index 1201.1 7.690 2014-11-21T00:
Oil Brent 78.6 -0.120 2014-11-21T00:
Oil Nymex 76.5 0.710 2014-11-21T00:
Silver Index 16.4 0.260 2014-11-21T00:

Indices

Symbol Open High Last %
NZX 50 5526.9 5526.9 5526.9 -0.56%
NASDAQ 4751.0 4751.6 4701.9 0.24%
DAX 9521.2 9736.1 9484.0 2.62%
DJI 17721.0 17894.8 17719.0 0.51%
FTSE 6678.9 6773.1 6678.9 1.08%
HKSE 23353.7 23508.0 23349.6 0.37%
NI225 17285.7 17381.6 17300.9 0.33%