Google: think of outage as an air traffic control blunder

Google has issued an explanation for its 90-minute brown-out, which hit around 14% of users of its search engine, Gmail and other services worldwide.

An official blog post reads:

This is your pilot speaking. Now, about that holding pattern...
Imagine if you were trying to fly from New York to San Francisco, but your plane was routed through an airport in Asia. And a bunch of other planes were sent that way too, so your flight was backed up and your journey took much longer than expected. That's basically what happened to some of our users today for about an hour, starting at 7:48 am Pacific time.

An error in one of our systems caused us to direct some of our web traffic through Asia, which created a traffic jam. As a result, about 14% of our users experienced slow services or even interruptions. We've been working hard to make our services ultrafast and "always on," so it's especially embarrassing when a glitch like this one happens. We're very sorry that it happened, and you can be sure that we'll be working even harder to make sure that a similar problem won't happen again. All planes are back on schedule now."

The outage - which quickly became known as #googlefail - as Twitter users relayed the news - lasted from about to about 2.45am to 4.20am NZ time this morning - or 10:45am to about 12:20pm EDT in the US, smack in the middle of people’s internet day.

Users as far afield as North America, China and Australia complained they could not access any Google services.

With its own site down for many users, and very slow for others, Google used its corporate Twitter account to relay service updates (see screen grab below).

During February Google extended Google Apps’ users 12-month licenses with 15 bonus days after an outage violated the company’s new uptime guarantee.