Online surveys don't tick all the boxes

Readers may not be surprised to know that this is one of the most market-researched countries in the world on the basis of interviews per capita.

The growth of online interviewing has only added to this. Online interviews are generally cheaper than telephone ones, so budgets can be used to gain much larger, more robust samples than those gained from other methodologies.

So far, so good. But this rapid growth is also raising concerns. Some have been raised by a fellow Synovate researcher, Emiel van Wegen in the Netherlands, as they point to a future where the very nature of market research could be fundamentally altered (see box.)

First, though, are some issues local research buyers need to be aware of when investing in online research. When these surveys are to be conducted among the general population (that is, not a customer base or similar), then such people are accessed via panels, which are essentially groups who have agreed to be emailed invitations to participate.

Millions belong to such panels and there is a wide range of specialist panels available - for those in certain occupations, age groups, or even with certain ailments.

However, because most online panels use incentives such as entries into prize draws or incremental participation-based payments, there is a constant risk of people manipulating their panel membership to gain income.

Websites abound commenting on which market research companies are the best to earn money from, and these people are referred to as professional respondents. In addition, the question must always be asked, "what sort of people actively join up to panels?"

This question can and should be asked by marketers who are being recommended an online survey conducted through a panel - the three largest being managed by Consumer Link (owned by Colmar Brunton), SmileCity (owned by PermissionCorp) and GMI.

In answering, all panel managers must be able to demonstrate they reduce the presence of professional respondents by ensuring they are not recruited by promoting rewards, nor through partners that recruit for multiple marketing panels. Potential new panellists need to be rigorously checked for duplication.

Recruitment advertisements should be run in other media other than just online - print advertisements are recommended as well as recruiting respondents during any telephone surveys.

Any overtures to potential respondents should emphasise sharing opinions, not monetary rewards, and panellists must accept membership terms and conditions that include protection of confidentiality, the need for accurate and engaged response, and the automatic termination of the membership due to fraud.

Online panel managers also need to monitor the size and composition of the panels, and where particular demographic groups are too small to provide nationally representative samples then special recruitment should be implemented.

The final area of panel management concerns the observation of panellists' activities while completing online questionnaires. For example, survey results from basic demographic information in the database should be compared with those gained within the questionnaires, and inconsistent data removed.

Other checks can include the insertion of three questions involving a polar-opposite response to check consistency and engagement in the questions, and applying tests that identify "straightliners" (those ticking the same column across all statements.)

Tracking your digital shadow

The Dutch online researcher, Mr van Wegen, says "over the next 12 years we will move away from measuring 'claimed behaviour' (the whats, whens, wheres and how much) toward analysing 'real behaviour.' The only thing left in the near future that we may still need to ask for is the why. All the other Ws can be measured simply because (research) data will become a commodity now that all new technologies today are based upon the concept of at least having huge databases."

He has a point. A study by the highly regarded market research theorist Dr Jan Hofmeyr found that the relationship between the FMCG items people said they bought and what was actually in their pantries was surprisingly low. The growth of new technologies that can track a phenomenal amount of behavioural data (one's digital shadow) is therefore exciting.

The following is just a taste of what is now on offer as the market research industry shifts from asking to observing:

n A UK company can set up cellphone receivers throughout a shopping mall, which track the constant pulses that cellphones emit in order to provide accurate data on how people move through malls.

n RFID (Radio Frequency Identification) tags are being used to track merchandise travel and will eventually become as ubiquitous as bar codes.

If Mr van Wegen's predictions come true, much market research data could indeed become commoditised, recorded and traded just like today's Eftpos, Census or MOSAIC data.

Obviously this stands to hurt much of the market research data collection supply-chain. Time for researchers to reinvent research itself?