Member log in

Senior Telecom manager reveals extent of XT outage

[UPDATE, 7.15pm: NBR just spoke to Gen-i chief executive Chris Quin, who said Telecom chief executive Paul Reynolds will issue a major statement this evening. The statement is expected to answer customers' questions about compensation, and what steps are being taken to avoid a future outage. Mr Quin said the outage had been traced to Christchurch, and that the technical cause had been confirmed - but that it was not the same fault as caused the December 14 problems.]

Read also'Angry' Reynolds orders independent review of XT

Following the massive mobile outage that hit Telecom XT customers from Taupo to bluff from 10am this morning, the company's spin doctors said the problem would be solved by 1pm.

But five hours later, multiple XT customers were telling NBR they were still experiencing problems.

Telecom's spin doctors also said they couldn't say how many customers were affected, or the extent of the voice, text and data problems.

But an email from Chris Quin, chief executive of Telecom's Gen-i Australasia IT and telco services division, obtained by NBR, tells corporate customers a fuller story.

His wording implies that all XT users Taupo south suffered problems, though emails to NBR indicate that not all were effected.

"Bizarrely, my cellphone kept working whilst those around me failed – all (including mine) on the XT network," emailed Festivé Ltd's Gavin Holley, of Christchurch.

Mr Quin's email reads:

Today we experienced issues with our Christchurch XT Mobile RNC switch which has resulted in the degradation of the XT Mobile Network from Taupo south between approximately 10.30am and 12pm.

Users of the XT service from Taupo south will have experienced approximately one hour of degraded mobile service, and approximately 30 minutes of inability to use the XT service in the area.

Since 12pm we begun restoring the XT service, and full service will resume for all clients from Taupo southwards by 1.30pm.

XT services north of Taupo, and CDMA services nationwide, were not affected.

We are establishing the root cause of the degradation of the network.

We continue to understand that mobile is critical for your business, and we apologise and deeply regret any inconvenience or problems you are experiencing due to the interruption of the XT service.

We are aware that this is not the first time that we have experienced service degradation. Please accept my personal assurance that this issue was unexpected and was immediately escalated to the Telecom Executive team and our global partner.

I will communicate with you further with relevant information as it is available, and confirm that full restoration is continuing.

If you wish to discuss this further with me, or any of the Gen-i team, please let us know.

Chris Quin
CEO Gen-i Australasia

One Gen-i corporate customer told NBR he gave Mr Quin credit for including his personal cellphone number at the end of his two messages (a third is expected later this evening). The Gen-i boss was also noteably more candid than his colleagues during XT's previous outage on December 14 - had included his email and cellphone number at the end of the message.

Telecom's Twitter account is advising those still suffering problems to remove their batteries, then reinsert them to reset their phones. 

However, as of 2.59pm, at least a dozen customers tell NBR they still have no service after following this advice.

Telecom blames network congestion as cell sites come back online, and advises people to "keep trying".

UPDATE (3.54pm)
Mr Quin has now sent a second message to customers. It reads:

This is the second email today to update you on the restoration of services for XT mobile clients operating between Taupo and the bottom of the South Island.

From approximately 10.30am this morning we have experienced issues with our Christchurch XT Mobile RNC switch which resulted in the degradation of the XT mobile network from Taupo south. As a result, many users in these regions have not been able to use XT services, including voice, SMS and mobile broadband.

Telecom has restored services for a significant amount of the XT mobile network that has been without service, however, due to the load on the network as services are restored and users commence using the network, we are seeing intermittent service for some clients.

XT services north of Taupo, and CDMA services nationwide, are unaffected.

For some XT clients still experiencing service issues, it may assist to try restarting their phone, by powering down then removing the battery for several seconds, then restarting the phone.

This issue has the highest level of executive escalation and our management and technical teams are working urgently to put a solution in place.

Once again, I apologise and deeply regret any inconvenience or problems you are experiencing due to the interruption of the XT service.

We are keeping our front line teams constantly updated, and will keep them informed on the resolution and next steps.

We will also keep you updated via email.

For further details, or to tell us how we can support your business immediately, please contact me or your Gen-i client manager.

More by Chris Keall

Comments and questions
48

"...approximately one hour of degraded mobile service, and approximately 30 minutes of inability to use the XT service in the area." well that's absolute crap. It was down since just after 10 and for a couple of hours at least not 30 mins.

XT= XTRA TIME OUT OF SERVICE LOL

My phone still isn't working. Screw XT, I'm changing to Vodafone.

Yeah - most of us did that years ago !!

4:07pm and still no service in Invercargill. Tried switching off and removing battery (twice), did not help. Luckily I'm not that popular so I don't think I will have missed to many calls........

do we get refunds for the inconvenience?

and this is why vodafone is a million times better :D

XT = The most unrealiable Mobile Network in the World??

More false advertising by Telecom? Surely not!!

Telecom are a no service provider and have been for years. Try no cell coverage 60km north of Auckland for a start. Since 1991 the governance model has been to strip mine and minimise infrastructure to the point it is now worse than a 3rd world telco.

Vodafone are way in front but even they are restricted by Telecoms lack of infrastructure

Time JK et al put a blowtorch down Reynolds Y fronts

As soon as Vodafone launched in NZ I signed up. Over the first year or so there were numerous outages and very weird problems with the text messaging service. Sometimes texts would take days to reach people I was sending to, or just as long for incoming ones to reach me. Calls were getting disconnected a lot, and calls not going through at all was a regular feature.

I'm not a telecom fanboy, but what is it that people don't get? It's new technology. Like Vodafone when their infrastructure was new, or the old CDMA network when it was new, most of the internet services when they were new, it will have teething problems.

To expect something as huge as a nationwide mobile phone network to be flawless from the start is unreasonoble. This is why I'm still on CDMA. I will switch to XT, but once it's stable.

Well the big difference here is Vodafone have a back up 2G Network and Telecom chose not to have a back up and are now paying the price.
Telecom NZ should know better as they are on the worlds top 10 bad business decisions. Which was to go CDMA or GSM - Hello! A decade later they go GSM only to make another bad business decision no back up network.
Vodafone - Global Business
Telecom - Local Business

I choose Vodafone all the way!

Surely all the talk about "best network" and "faster in more places" and "no problems here, Richard" mean we can take action against them? Under investing in the roll out to this degree is pathetic and a huge loss of face.

But declan, that's exactly what Telecom said. And when its network was interfering with Two Degrees and Vodaphones they still said it was everyone else's fault.

Telecom doesn't know how to run a GSM network and they hired Alcatel, which has never built one, to build one! Talk about the blind leading the blind.

Heh. Yeah, I'll give you that. But, it'll probably get sorted over time, just like everything else that's big and technical and new. LHC springs to mind for some reason.

Standard procedures for consumers is to jump up and down and act like children when any service has problems though.

Vodafone never had two major outages affecting half the country in one month Declan.

yeah. they did. but only in the first couple of months of operation.

XT absolute rubbish, in one their tv promos featuring a user on a shipping container in the middle of the ocean i fail 2 c how she could've had recep if it doesn't even work properly on land highly frustrating when travelling.

Are you giving sexual favours to Paul Reynolds?

Anyone want to swap my xt phone for string and 2 empty baked bean cans....please call me

Telecom and Transpower must share the same PR company. Might be better to just tell the truth guys or don't you remember what that is?

XT Customers have every right to "jump up & down" when they are the ones PAYING for a "service" that regularly breaks down - South of Taupo"!!

This stupid XT Network was promoted as the best thing since sliced bread "TO THE HILT" - during every ad break for months, they went on & on & on about the stupid thing!!

Fact is - Alcatel Lucent are the worst choice of partner they could of chosen & Telecom got the cheapest possible deal - which has now landed them in a rather poor position & they WILL lose customers because of this.

Furthermore - this WILL happen again & their share price & customer numbers will most likely fall, to the advantage of Vodafone (who are doing just fine atm).

Agreed, ALU - have virtually no 3G experience - this is a direct result of Telecom's one network vendor policy which they claim is so successful. Successful for their bank balance maybe, but not for the end users. I bet Ericsson and NokiaSiemens (who should have built this network) are enjoying ALU's pain. Even now Telecom don't mention ALU. Compare this to Rod Fife and the IBM debarcle, he went for them all gun blazing. Makes you wonder what ALU are holding to TNZ's head

I don't really care as long as we get reimbursed like last time.

Don't bother with Vodafone - save money your money and go to 2degrees!

You do realise that 2 Degrees roam on Vodafone NZ's Network, outside of Auckland, Wellington, Christchurch, Dunedin & Queenstown don't you??

Furthermore - 2 Degree's have no TXT Plans or On Account Plans - 2 Degree's is cheaper for calling per min, but with no TXT plans yet, it makes it a rather limited service & they also don't have 3G yet!

Actualy 2d is also roming here in Duedin on vodas network

Do you realise that in two months Telecom are withdrawing the $10 TXT offer on the CDMA network. This is of course a ploy to force customers on to their XT network, or should I say their lucky dip XT network when it works.

Yeah mate. I agree with you and think that this is disgusting. Telecom forcing the big text users to a network that they have proved is unreliable. Surely this must be against law. Fair Trading or something.

I just read Vodafone NZ have 6 live RNCs and Telecom NZ XT has 2.............Good one TELESCUM

Who gives a rats arse? Shi* just happens. Go without your phone for a few hours and ENJOY THE BREAK!

Cheers SCM - thx for the correction. :)

No worries ;)
Im not sure but I beleave that Queenstown 2d also roams on vodas network

No - 2 Degree's do have their own coverage in Queenstown

True, good to know.
Cant wait for 2d to install their own sites here. Last time I spoke with them they were unsure of when if ever they would be setting up sites in Dunedin

It's been what, 10 and half hours? Still nothing! FIX IT TELECOM.

http://www.telecom-media.co.nz/releases_detail.asp?id=3648&page=index

New network name is Monica as it goes down so often without warming and screws you!!

I note Telecom have not told their customers that the entry level R100 phone is only capable of receiving a WCDMA 850MHZ signal. As you know Telecom uses WCDMA 850 and WCDMA 2100 MHZ on their XT network. So in 2100MHZ coverage areas this phone will not work.

XT still no go for me in Invercargill!!! Beginning to wonder whether it will be up at any time tonite???

1984 is the year AT&T in USA had their monolpoy ended by the government. It's been a free and competitive market since. 2009 in New Zealand and Telecom are clutching still in the time bubble holding on to their old infrastructure. "High availability" is yet to get in their vocabulary. Did you know Telecom have been the #1 spenders in advertising in the country, only second to Loreal/Garnier ? Telecom have to have huge ad campaigns, in order to present a good image because their product is still years behind.

It is now 1:45pm and it is still down in Timaru

I t is incredible that since Gatting and the board decided that they were not running a telco but were setting up a phillipine scam, this country will never have a decent comms structure, our scottish cousin has been brought onto the board to deflect any government interference!!!

At least we do not pay tooooo much for a crap sevice.

Whe they broght Alcatel they would have known that they stuffed up up the Ozz.

but then again, how did air kiwi buy into ansett

Senior Telecom manager reveals extent of XT outage | The National Business Review - New Zealand - business, markets, finance, politics, property, technology and more adley@gigemail.net

Senior Telecom manager reveals extent of XT outage | The National Business Review - New Zealand - business, markets, finance, politics, property, technology and more adley@gigemail.net

Senior Telecom manager reveals extent of XT outage | The National Business Review - New Zealand - business, markets, finance, politics, property, technology and more adley@gigemail.net

Senior Telecom manager reveals extent of XT outage | The National Business Review - New Zealand - business, markets, finance, politics, property, technology and more guy@gigemail.net

Senior Telecom manager reveals extent of XT outage | The National Business Review - New Zealand - business, markets, finance, politics, property, technology and more guy@gigemail.net