500,000 customers hit by Telecom broadband blackout - but no word on compo
Telecom says a power fault at its central Auckland exchange caused the weekend's catastrophic national broadband outage. But the telco has yet to work out the cause of the fault, and there is so far no word from its retail division on customer compensation [a compo deal, of sorts, has now been announced; details here].
The company says its broadband blackout began at 9pm Friday night, swamping its help lines. Basic internet connectivity was restored for all customers between 4am and 5am, but problems with some services, such as email, persisted into Saturday evening.
Telecom spokeswoman Katherine Murphy says all of Telecom Broadband (formerly Telecom Xtra)'s half million customers were affected by the outage.
Ms Murphy says Telecom Retail will handle the customer compensation issue, but she has yet to hear the division's compo intentions. Telecom Broadband's standard terms and conditions call for customer compensation when an outage lasts more than 24 hours, Ms Murphy notes, while the weekend's blackout lasted eight hours. However, some readers have told NBR it was into Saturday night before their full service, including email, was restored.
In a statement, Telecom says it "apologises for any inconvenience caused by this fault, and thanks customers for their patience while the restoration work took place and normal service was restored". The company says some of its internal systems were also taken down by the fault but that no emergency, fixed line or mobile services were affected by the outage.
Other ISPs contacted by NBR said their customers were not affected, beyond access to websites directly hosted by Telecom.
Signup to free NBR email alerts here
Back to home
Back to top
Print
Email
Share
Delicious
Digg
StumbleUpon
Reddit
Google
Yahoo
Technorati
Scoopit
NBR SERVICES
SUBSCRIPTIONS
Sign up for NBR subscriber content or email alerts
Latest
Comments and questions34
I rely heavily on broadband to conduct my business and the outage although was at night had a major impact and resulted in me loosing quite a bit of revenue...!!!
Contrary to the telecom spokesman's comment, the disruption lasted well into the afternoon, as the overloaded circuitry tried to play catchup.
I know, because I endeavoured to send e-mails at 4.00 pm; they finally went sometime later in the evening.
[Telecom's spokeswoman did say it was Saturday evening before all services were restored. Just how far into Saturday evening is an interesting point, since after 9pm you could argue that 24 hours had passed, and customer compensation would automatically kick in as per Telecom's T&Cs - CK]
It would have been REALLY helpful if Telecom could have put a message on their telephone-answering apparatus telling us what the problem was. All I got, after doing all of the "If you want xxx press x" rigmarole, was a "number unavailable" signal.
Very frustrating and time wasting. With the technology available to them, you would think they could have a message on their help line saying they were having problems. Instead they just keep you waiting...and waiting...till I hung up !!
It's symptomatic of trying to deal with any problem with telecom. Perhaps I should invoice them for the time spent fruitlessly trying to get thru to an actual human. Then when they call me to complain I can put them on MY call management system. Enjoy the muzak, telecom accounts payable!
Down here in Taranaki i had my internet drop out on Friday night, but i noticed it was just the DNS and Mail servers down. I used a free public DNS server which not only got my internet working again (sans email) but my page loads easily halved in time compared to normal. Maybe there was a silver lining to the outage.
For the curious these are the DNS servers i used which came from the www.opendns.com/ website:
Primary: 208.67.222.222 Secondary: 208.67.220.220
The only way to get improvements from telecom is if we, the customer leave them for another ISP, which I did two years ago and I've had no problem since that day.
Try World Exchange, they good and no contract.
whats 8 hours come on guys if so important go find a wifi link
these things happen to everybody cut them some slack
The outage was an inconvenience because of the nature of our business. But whats more annoying is Telecoms broadband help desk staff saying only a "few" customers were affected, and at the same time, telecom faults admitting it was nationwide.
Anytime there's a network problem, their help desk staff try their best to deny it, or make silly suggestions to check your configurations or reboot the router.
A bit of honesty might go someway to in these situations so we dont waste our time chasing ghosts.
The lesson we learned this time is the 2 telecom broadband lines we have must now become one, and the other we will use a different provider.
Telecom's internal systems were impacted as well ( the article states ) Have you concidered that maybe they could'nt put up a message ?
What about the electricity co. giving the compo?
Funny how everybody complains when something goes wrong....
Maybe I should bill Transport NZ next time I get stuck in traffic b/c of the inconvenience... or the next time a lightning strike hits a power pylon and my alarm clock doesn't work b/c the power went out... shall I charge Genesis for the cost of a new 9v battery ?
Should I ask the insurance companies to refund me b/c I haven't had to use them...
Expect things to go worng people... it happens... did it threaten your lives ? No....
Was it an inconvenience b/c you couldn't surf the net, or maybe buy your favourite lollies online for a couple of hours... yeah it was... but your still alive... and if you are reading this or typing here... then I guess you can access the net again huh....
Don't take life so seriously !
It's within Telecom's power to have the system running, it's a legitimate expectation that given the nature of their business that they have redundant systems. You forget that some people rely on the internet to conduct commercial activity and it can cost significant sums of money when the system fails to respond.
It's not the same as power because outages are typically acts of god, i.e. trees on the lines, bad weather - and it's not within the companies PHYSICAL control to ensure a working system. You pay an insurance company to take the risk of insuring you, not for the claims you can make.
Don't be such a dolt.
Ah, so when a rat chews through the backup cable, after an act of stupidity (digger drilling through main cable) took out the main line, it's still Telecom's fault, as evidenced by the howls of anguish then. It seems there are rules for other infrastructure companies, including other Telecommunication providers, and then there's Telecom. When WILL Telstra's network be unbundled??
I work for telecom, in the building that was the cause of the problem.
I was working at the time the outage happened, and i tell you, it sucked.
before you all complain about "i can't get through to a human", think about the humans that are working when stuff like this happens.
the article state's that "some of the internal systems went down". It was pretty much EVERY internal system we use.
Every single telecom broadband customer was affected. So were all of telecom's broadband staff. we are people too.
Bfeore you all tell me to quit my job because telecom sucks, I was on the unemployment benefit before i started working for telecom, would you all rather me go back to bludging off your tax instead? I like working here, and I like fixing customer problems when I have the power to do so.
Friday night unfortunately I didn't have the power to do so.
Deal with it, and go back to complaining about the All Blacks, or whatever your thing is.
www.nationalblog.co.nz: Granted. But I guess, in this situation, the question would be: "Did Telecom have the 'power' to keep the system running in this case?" I don't know the details, but it would really depend on the details wouldn't it?
ok so you running a business over the internet.. how bout some common sense??? you rely so heavily on your internet connection.. so how about make sure you have alternatives incase your single residential dsl line goes down?? smart business yes? if you are dumb enough to put all your eggs in one basket then good luck to you and your business...
Regarding the post saying you need a back-up plan in New Zealand if you rely on the Internet to do business. That is a fallacy! That would work in a developed country, but this is NZ with "developing developed world services". We aren't third world but a quick trip up to USA, Canada or Singapore shows you how far we've fallen behind. Any decent metropolitan area there (espeically main business centre) had duality (minimum) in all services. Perhaps the person who made this post should travel overseas once in a while.
As far as this outage goes it was Telecom who didn't have the backups in place. They remain the most incompetent of all. The outage was over 24h until full service was restored. Telecom's own customer support people didn't know what was happening (when you finally got through!). No doubt we will see Telecom try and weasel themselves out of this one (they are about as competent as their old friends in the Post Office!).
But when service was restored we still don't have ADSL2 or fibre to the kerb. We still suffer with absolutely atrocious broadband.
In order to lift up the NZ economy to punch above its weight we need to recognise that a good broadband infrastructure - from the backbone to the front door - is key. Are you listening Mr. Key?
We didn't move in ten years of Labour. It will be interesting to see if National-ACT are any different.
Broadband went down? On Telecom, never noticed. Definately used it over the weekend......hmmm
If you want to run your business via the web, and require a connection at all times (apparently people posting here do a lot of web busiiness friday after 9pm that cannot wait until saturday night. Who knew?), how about having a datacard to go with your DSL connection to act as backup? Seems pretty sensible to me, and not that expensive at around $30/month.
Well worth it when you know that the nature of DSL is that it can go down at an unfortunate moment - that applies regardless of ISP. All ISPs go down at one time or another.
Strange that someone would hound Telecom for not having a backup generator when a) you know almost no details of the problem yourself - maybe the backup was taken out at the same time?
and
b) you have no backup either. Pot calling kettle black I beleive.
If as Mark says above it was a problem with Telecom's DNS servers, why on earth did they both got down at the same time. The whole point of having primary and alternate DNS servers is so if one goes down, the other takes over.
If so, then this problem could have been avoided if the two DNS servers were housed in different buildings preferably in different cities ?
If you are running a business and you depend on an internet connection. Have a backup plan if you loose your main connection. I lived in Japan and its pretty much the same there situation there, most places dont have a choice of who they get their internet connection from, it all goes through NTT. The places where there is a choice, you have 100,000s of people per square km. If your unhappy with infrastructure in NZ, YOU bank roll it and take losses until we have a bigger population to support to pay for it.
Oh dear, good to see the ole com bashers back in force.
Yes it was an outage, no it is not good,
yes there are multiple backups generators, it is a good question why they failed to come up, suggests the issue may not the external power supply but internal somewhere.
Life is a question of economics, for enough $ you can provide soemthing approaching complete redunancy in any system, however hands up anywhere as to people who are prepared to pay for it?
And as to broadband rankings, great, yes a country with this poulation density? the com is a company, unless people want the govt to buy it back? it needs to be able to operate commercially not as a charity like every other company in the country.
Surely if it is critical to your business you should have a back up plan if it fails? Relying on an unmanaged non guaranteed service that probably cost you less thatn $100 a month seems like risky business to me
"You forget that some people rely on the internet to conduct commercial activity and it can cost significant sums of money when the system fails to respond"
If its crucial to your business make sure you have some diversity, probably cost less than $1000 a year to have a back up connection with another provider you dolt.
back the drawing board for you and your business buddy, im a kiwi, and when I have a problem i find a resoution to my problem, if my problem is my business is small and relys heavily on the internet.. then I as a smart kiwi ensure I have internet provided on multiple mediums. Further to this.. I have ADSL 2... dont tell me your one of these people in the middle of no where who expect 50mbit+ ... AND just becuase I can... If you had any computer smarts.. you would have realised that you probably only lost DNS last weekend... how about changing your anternate DNS server to one of the other ISPs?(not a computer smarty? PAY ONE TO FIX IT FOR YOU)
Your very quick to condemn people when they're systems go down. Calling someone a "dolt" because they still have faith in a New Zealand born & financed company. Where was your quick wit when Telstra Clear (A company with a much larger financial backing) started swaping their customers emails with other customers emails?
Like you say, power outages are "typically acts of God"
From what I understand, if there is a power surge anywhere, that could affect Telecom's systems too. I'd imagine that they have quite a few systems in place to protect what ever system gets the internet running, BUT, like most forms of protection (like the one's that should have prevented some people from being born... hint... hint) nothing is 100%
I know that if I was to form a business that RELY on internet, I'd be getting a mobile broadband connection for every computer as a backup. I would say if anyone was a dolt, it would be those that never considered the possibility that wires under the ground may too be affected by factors that the company does not control. Like some person in a digger, landslides, other utility's pipes bursting. Use your imagination!
I don't think this would have been a problem emanating from the power company, as I heard reports of a transformer blowing up on Telecoms backup power system last week, which resulted in a staff evacuation. Telecom do spend a large amount of money on redundancy and fault tolerance, but as we know, even the best laid plans of mice and men may go awry. I say spare a thought for the servicemen and technicians spending their evening resolving this issue as expediantly as possible.
UPS. Uninterruptible Power Supply. This comes in the form of a backup generator. Telecom does not have a backup generator?
http://en.wikipedia.org/wiki/Uninterruptible_power_supply
This flimsy "power outage" falls into the same category as the 502-T routers Xtra used to give out to their clients. Both the 502-T routers given to me by Xtra were pre-loaded with firmware set to operate in Texas, USA... Power outage?
Internet went down for one day & all people can do is complain how they were not able to browse the net. There are other important things in life than sitting infront of the computer on a weekend.
Telecom provide a great service 99.9% of the time...Leave them alone!
Realistically speaking, sure if they gave compensation, it would be a pro-rata thing, so you would only be getting around 15 hours worth, which equates to around ohh i duno, $1??
Now, if i was a business, i wouldnt be moaning about compensation. This like this happen with every business. Get over it you whingers. Get a life.
Telecom for life
Well all it has happened again. Wellington has been affected 24-26 Jan 2009. I wonder if it's the same issue just repeating itself in the Wellington exchange. We are now SERIOUSLY considering Telstra's cable offerings......
Post new comment or question
To share this article, click on a service below