The volume of web traffic after Michael Jackson’s death provided a costly lesson for some of the world’s biggest web sites in ensuring that their software and coding is up to the job in future.

So intense was the interest around the globe that Google News (http://news.google.com) believed it was under automated attack. Its servers returned error messages for more than 20 minutes until it became clear what was happening.

You could argue that Google’s system worked the way it was intended to in terms of safety measures — but that was of little comfort to people searching for the biggest news story of the moment.

It wasn’t the only embarrassment for the search engine. Users looking for information were also told that Michael Jackson died in 2007. A writer called Michael Jackson did die in 2007, but Google’s algorithms failed to take account of the fact that people searching for the name might be more interested in the more recent death.

After the celebrity news source TMZ (www.tmz.com) broke the news of the King of Pop’s passing, web sites fell like dominoes.

First to go was TMZ itself, as it suffered several outages under the weight of traffic. When that happened, the traffic shifted to Perez Hilton’s gossip blog (http://perezhilton.com), which also struggled to cope.

The more mainstream news sites also took a battering but managed to stay online.

CNN (www.cnn.com) reported 20 million page views in one hour, five times its normal traffic. And ABC (www.abc.com) survived a five-hour deluge of visitors. Twitter, however, stopped tweeting as people posted messages at www.twitter.com.

The site has the added challenge of users being able to post information by mobile phone, thus multiplying its traffic.

Wikipedia (www.wikipedia.org) even saw its protection measures collapse. A cached server, designed to take the pressure off its main servers when millions of people view the same page, gave up under the pressure. In the days since Jackson’s death there has been much exaggeration about the effect on the web.

Some media coverage suggested that the internet was on the verge of collapse. That was never the case. But individual sites had a painful reminder of the need to test their software and infrastructure to prepare for surges in traffic.

Some of the sites that continued to function experienced sluggish performance because of the way they were built. For example, some slowed down because they needed to fetch third-party content, such as advertisements, before serving up the page to the reader.

The Jackson effect took its toll on sites which attract visitors from around the globe, so it’s not likely to be a worry for you. Or is it?

Many web sites experience spikes in traffic. Software is put under strain for all sorts of reasons.

For a retail site, it may be the Christmas rush. For a government department, it might be large numbers of businesses looking for information after a change in the law (the reduction of VAT to 15% being just one example). And the web site, let’s remember, is often linked to back office systems which may be under pressure themselves because of the volume of business.

It’s my experience that in many companies and organisations, testing of software begins too late in a project’s life cycle, when defects and shortcomings cost much more to fix.

Testing can simulate load levels much higher than anything you would experience even at your busiest times. Yet companies regard it as an optional part of the process, and the first part to be abandoned if money is tight or costs have escalated. This is a big mistake.

In fact, we in Northern Ireland should have even more interest in this issue than other countries.

Thanks to our booming software development sector, we have the opportunity to become a centre of excellence for quality and testing.

Rob McConnell is regional director for SQS NI. (www.sqs-uk.com). Email: rob.mcconnell@sqs-uk.com