|Benefits understanding the customer journey||Marketing benefits|
|Reveals customer’s decision making thoughts, feelings, and needs at each stage||Reveals under or over performance and thus where and when to optimise or smooth the customer journey|
|Clarifies decision making influences, i. e. what are the relationship drivers or barriers (pain points) at each stage||Informs how to better communicate to engage, attract, retain customers and secure referrals, for example, through what medium, and with what messages|
Also how to optimise the customer or brand experience experience, e.g. through better processes.
In 1961, Lavidge and Steiner created ‘The Hierarchy of Effects’ model to explain customer buying behaviour, and also help advertisers make better advertisements (1). The concept of the customer journey is essentially the flip side of the same coin.
Lavidge and Steiner’s ‘hirarchy of effects’ model comprises three stages; each has one, two or three steps. Thus six steps overall, as illustrated in Figure 1, and summarised from top to bottom, below:
Lavidge and Steiner’s model is effectively a linear series of steps:
However, while most relevant in a world of mass advertising, in today’s online world, customer journeys are much more complex.
Understanding the customer’s journey allows marketers to determine the sequence, nature and importance of the steps, also the triggers, drivers and barriers to sales of a particular service or product. Thus, where and how to promote and add value to an offer. Also specifically how to improve trial, retention and endorsement.
It is akin to a powerful ‘sat nav’ to marketing success.
However, shopping no longer takes place just in the High Street but anywhere, anytime, with the result that nearly 20% purchases were made online in 2019 (and significantly more during the lock-down). Further almost all UK adults aged 16-44 years, and 95% adults aged 16-74 years use the Internet daily (2). 79% also own a smartphone making it easy to go online on the go (3). Thus the ability to buy food, clothes, music, films, sports equipment, holidays, cars etc. online has never been easier.
With the growth of the online world, the customer journey has become a more complex, stepping stone process (Figure 3). Sometimes purposeful, and fast, sometimes serendipitous, looping, and seemingly never – ending. Recent research by Google Australia/New Zealand, describes this as the ‘messy middle’ (4).
Different media present different environments, and thus attract different demographics and psychographics. Further media entry points and occasions vary, and technology increasingly links from one medium to another. Of course, markets, competitive contexts and customer needs also vary. Consequently, influences differ, resulting in different customer behaviour.
The threat to marketers is to only see digital media and only to rely on website analytics data. This only sees the effects of customer behaviour and fails to explain why they do things and what their needs are.
The opportunity is to see and understand the big picture. In particular to understand customers, in context of the entire media and competitive landscape. Where, when, how and why they behave as they do, and then to plan marketing accordingly.
First, use qualitative research to understand the big picture, what customers do, and why? The benefit of a qualitative approach is to fully explore and understand what customers consider important, what engages, fails to engage and most importantly why? Here are some start-point (though generic) questions to ask (Figure 4).
In our experience, there are also two tracks to any customer journey; first the journey into the category, and secondly, the journey to discover, and choose a particular brand. Thus original customer research is valuable to understand the twin tracks, and also the relationships between the tracks.
It is almost impossible to predict a journey from the outside looking in, and thus research will inevitably unearth new insights.
Google’s own research suggests consumers are in sequential exploration and evaluation modes. With continuous offer selection, short-listing, and deselection until a final purchasing decision is made. They also highlight the importance of brand awareness, and the power of benefits, recommendations, and incentives to shift demand from a familiar to lesser known brand (4).
There are many ways to map customer journeys, and what to do should follow from your business objectives and needs. A grid matrix may best inform a new IT customer contact system, but risks the ‘wood being hidden by the trees’. A more visual representation may better help colleagues understand and act on research findings. Figure 5 (below) summarises on one page, a customer journey to research family history or family trees. It highlights key events, and drivers and barriers on the journey. It is useful in bringing to life rational and emotional factors. And thus opportunities to optimise the journey and better market brands.
Use quantitative research to rank ‘influencing’ factors, and thus prioritise marketing effort. Though conduct qualitative research first to scope your quant survey, and finesse the questions. You’ll need to devise a long list of answers to the following:
Figure 6 (below) summarises some of the results from a survey to assess key reasons (and barriers) to buying telecoms services (fixed line, mobile, broadband and TV). This shows how issues and attitudes differ across two countries. It also reveals a marketing opportunity simply to promote additional services!
Stimuli is the plural of stimulus. In psychology, stimuli is any object or event that elicits a sensory or behavioural response in human (consumer research) studies. So what are the benefits of using creative stimuli in market research, particularly in qualitative research such as interviews and group discussions?
Forms of market research stimuli include;
In summary, using creative stimuli in market research helps:
Stimuli is very useful in the innovation and marketing process. Firstly, to understand and verify needs both now and in the future. Second, to develop concepts, products, services, as well as communications. Third, to future-proof ideas and strategies, and improve chances of marketing success. It is also useful in all markets, especially in service markets (to bring-to-life or make the intangible tangible). And in technology markets (where products are often a collection of features searching for a need).
This is in order to avoid ‘bias’ such as the ‘framing effect’; whereby the means of presentation, for example, quality of presentation, may affect perceptions. Thus all stimuli should be equally ‘rough’ or ‘polished’ etc. Further, when researching overseas, use the same images, and translate copy into natural language. This enables ‘fair’ comparison across countries.
In our experience it is impossible to prejudge which ideas will resonate. Thus creating ideas to cover all variables and more rather than less helps increase the chance of finding real winners. Do this by involving a diverse group of your colleagues, and agencies, and then working together to refine ideas.
Overly complex stimuli risks confusing, and thus will be discounted. So keep it simple; first summarise your idea, and then give a little more detail. Remember too that consumers buy benefits rather than features. So make sure they are crystal clear. Also that the features support the benefits.
Combined word and image stimuli work better than words or images alone. This is partly because they engage different parts of the brain. Including photos “mines deeper shafts into a different part of human consciousness than do words-alone” (1). Combined word and image stimuli also mimic real-life brand encounters, such as an advert or leaflet. Then, as consumers are familiar with adverts, and leaflets, this reduces misunderstanding, and helps them react to the stimuli in an every-day way. Equally, real-life stimuli is more powerful than mood-boards or written ‘concepts’. The latter are intermediary to communication development, rather than a direct form of communication, and thus alien to consumers.
This is in order to avoid ‘groupthink’ or herd behaviour. We do this in several ways, either by encouraging respondents to stand up and tour a ‘gallery of stimuli’ or by remaining seated and reviewing stimuli on a single page. But always in silence and without conferring. We then typically ask respondents to write down their views and collect in responses to check what is said and written are the same.
This helps consumers see the ‘wood from the trees’ and helps address potential key issues first, rather than get bogged down in tactical detail early on. This also aids analysis. However note that some stimuli covers both strategic and executional elements, and that consumers really don’t judge ideas this way. Interpretation, of course, remains down to the skill of the analyst!
There are two main ways to co-opt consumers to create and use creative stimuli in market research interviews or focus groups. Firstly, prior to research, whereby respondents are pre-tasked to reflect on topics before attending research. For example, to select or draw images, or take photographs, to illustrate their thoughts and feelings on particular subjects or brands. (Figure 1). Secondly, to share ideas in the group forum itself to inspire further discussion and idea building.
The second way is to create stimuli within the group discussion. For example, by writing thoughts and feelings on a flip chart, for example, in simple columns, or as a mind-map. The process of graphically recording discussion points, provides a common focus for discussion and theorising (2). Thus helping respondents to easily recall, assimilate, and process more complex thoughts. As a result, respondents feel listened to. And thus more engaged, and willing to fill in gaps, and build or challenge ideas. For example, to determine potential implications or opportunities for themselves or a brand.
Products may ‘rough’ visuals, more highly finished ideas, or even samples. This type of stimuli (Figure 2) aids comprehension, and thus provokes richer understanding on what appeals, is different, and could be improved. Rough stimuli encourages respondents to fill gaps, imagine benefits and provide insights of a more strategic nature. Whereas more highly finished stimuli lends itself to discussing more detailed elements such as packaging and visual identity. And product samples also elicit reactions to usage issues, performance, as well as propensity to purchase and reuse.
Service stimuli or visualisations (Figure 3) bring-to-life intangible ideas, and thus again aid consumer comprehension. In so doing they also allow respondents to assess the relative appeal of ideas, and test the logic behind service propositions i.e. what works, or doesn’t, requires improvement and why.
Simple lists (Figure 4) help determine the relative appeal, and importance of, features and benefits, within a product or service. This is a qualitative-quantitative technique. It helps to construct bundles, to target particular segments. Also to ‘sort the wheat from the chaff’ and spot potential category killing features.
Mini adverts with words and images are akin to press or poster ads. Thus they are familiar to consumers, evoke memories, forgotten thoughts, and are easily understood. They are also powerful to assess the importance of needs, and to uncover new needs. In turn they help reveal what are the most appealing and distinctive benefit combinations, and the most salient and credible features. Also the most engaging way of communicating. Increasing, we find that there are a very small number of salient needs in particular markets. Thus that personality traits are important to set brnds apart. Using mini ads dramatises all of these elements, and helps us understand how to best position products, services and brands. Also to inspire effective marketing communication.
Brand stories invest content and character into brand positionings. While we originally applied this technique to media brands, through the power of archetypes, stories are relevant to all brands and categories. Figure 6 shows an example, used to develop a well known media brand; The Famous Five. Translated into multiple languages, it helped explore appeal and stand-out, across different countries.
1. The human brain comprises around 100 billion neurons. This makes it the most complex structure on the planet. To aid research and marketing efforts use consumers’ brains for all their worth!
2. While consumers can’t tell you everything, using creative stimuli in research begets creativity, which begets innovation, and thus profit. In other words, they can help you create more ideas, and then turn ideas into more practical solutions.
3. Using real-life stimuli elicits more detailed responses. Thus revealing richer and more nuanced content for brand or communication models. In markets where there are just a few benefits, this helps to build more distinctive and appealing selling propositions.
4. Despite many years of using communication stimuli in market research neither we, nor our clients, are yet to correctly anticipate which ideas are most appealing! Sometimes the text is most attractive, though sometimes the image. So push your creative boundaries. Then trust in consumers to help you develop more successful advertising!
5. With any innovation research, just bear in mind that research typically covers a very large number of target variables, as well as ideas. Thus more than type, and wave, of market research service is often needed. Both to focus the target ‘sweet-spot’ and then build ideas into fully fledged solutions
(1) Harper. D. Talking about Pictures: A Case for Photo Elicitation, Visual Studies 17(1): 13–26 (2002)
(2) Crilly. N, Blackwell. AF, Clarkson. PJ. Graphic Elicitation: Using Research Diagrams as Interview Stimuli (2006)
There has been much in the press in recent years about market research losing its place in the boardroom. Most notably from Unilever who say that their senior managers are unwilling to invest time in research debriefs. An ESOMAR survey also adds that most CEOs consider market research less useful than finance, marketing, information services and human resources (1). A further BCG survey suggests that even market research professionals seem in denial about their lack of relevance (2). Yet criticism is also made by major research agencies (3). The problems appears to result from less than robust data collection, and also flimsy market research analysis and strategic interpretation.
Issues also trace to the research methods used and the skills of the people involved. Some say researchers lack the ability to integrate information, fail to connect research results with business outcomes, and also fail to turn complex data into clear narratives (3). Of course, concise presentations and explanations are important. But not if they result in more questions than answers. In particular ‘so what does this mean?’.
Triangulation is a mainstay market research method. The idea is that using two or more methods in a study gives more confidence in the results. Denzin defines four basic types of triangulation. Firstly, methodological triangulation. This involves using multiple research methods to gather information, such as interviews, observations, and documents. Secondly, data triangulation which involves multiple time periods and respondents. Thirdly, investigator triangulation which involves multiple researchers. And finally, theory triangulation which involves using multiple analytical methods or models (4).
Bricolage is a term used to describe multiple or multi-perspectival research methods; also a way to learn, and solve problems, by trying, testing and playing around. It avoids the reductionism in any single method (monological) and also mimetic research approaches (5 and 6). It also enables more deductive reasoning (in which a conclusion is based on the concordance of multiple premises). Lastly, it produces more comprehensive and specific insights.
Qualitative research data is usually unstructured so the challenge is to manage, shape and make sense it. The most common qualitative market research analysis method is observer impression. Computers and software also classify, sort, and arrange information. Though computers and software fail to think; leaving human skill to spot themes, patterns, and thus uncover insights.
While skills and knowledge lie with the observer and analyst, for life stage and economic reasons, fieldwork and analysis tasks often fall to younger, less experienced researchers. While many are also graduates, experience is acquired mainly on the job. Thus explaining why ‘business savvy’ may be lacking.
Every marketer knows that customers have needs and seek products and services that offer benefits that match their needs. So to design products and services, researchers must first understand needs, and the drivers behind those needs. Only then can product benefits be matched to meet those needs. This simple marketing logic therefore helps challenge and analyse market research findings. It is therefore vital that researchers understand basic marketing principles both to uncover, analyse and interpret findings. A broad and deep know how on a businesses’ aims, as well as marketing and brand concepts, also allows broader and more penetrating enquiry. Thus inspiring more insightful, relevant, and actionable findings and conclusions.
Probing and testing cause and effect relationships also ensures more robust analysis. In particular, the ‘Manchester Map’ is useful technique learned in management consulting days. This involves systematically reviewing findings and then asking ‘so what does this mean?’ or ‘why does this happen?’. It also helps sort and delineate information. Thus helping understand and express findings and conclusions.
Within qualitative research, employing simple numerical scoring (or semi-quantitative) techniques helps give weight to findings. Thus sorting the ‘wheat from the chaff’. We call this quali-quant research. For example, we ask respondents to choose the most appealing comms idea from a gallery. Or to rate a product concept on a scale from ‘will definitely buy’ to ‘will definitely not buy’. This reduces reliance on subjectivity (interpretivism) (7). Equally it adds scientific rigour to qualitative research i.e. objectivity (empiricism, positivism). Thus helping spot differences in meaning and relative customer appeal. In turn, spotlighting key issues and thus opportunities and ‘outliers’ (8) that demand further investigation.
1. Esomar Research World / ARF (2005)
2. Boston Consulting Group (2009)
3. Does Market Research Need inventing? www.InspectorInsight.com (2014)
4. Denzin, N. Sociological Methods: A Sourcebook. Aldine Transaction (2006)
5. Kincheloe, Joe. L. Berry, Kathleen, Rigour and Complexity in Educational Research (2005)
6. What is Mimetic Theory? www.woodybelangia.com
7. Interpretivism (or antipositivism) is a view that social research should not be subject to the same methods of investigation as the natural world. Gerber, John J. Macionis, Linda M. Sociology (7th Canadian ed.) page 32 (2010)
8. An ‘outlier’ or outlying observation deviates markedly from other members of the sample in which it occurs. Grubbs, F. E. “Procedures for detecting outlying observations in samples”, Technometrics 11 (1): 1–21 (February 1969)
Digital media continues to inspire lots of new customer research methods. However, there are lots of myths and misconceptions surrounding digital vs traditional research methods. So what’s available and what factors affect the choice between digital vs traditional research methods? For sure, the booming range of options are a boon for market researchers everywhere.
More traditional research methods involve either face-to-face or verbal conversations in real-time such as :
Traditional face-to-face or telephone research methods enable the moderator to follow the natural flow of the discussion and understand what’s really important to interviewees. Also to flex the discussion, intervene, probe and challenge at any point in the proceedings.
Face-to-face methods allow observation of non-verbal indicators, such as facial expressions, body language, general behaviour and voice intonation. What’s not said is sometimes as important as what’s said. Albert H. Mehrabian found that body language accounts for 55% of received communication, while tone of voice accounts for 38% and words only account for 7% (1). Non-verbal communication provides extra richness and texture to information and gives deeper insight.
Costs not only include research moderation and analysis but also travel and respondent recruitment and research incentives. Research incentives typically cover undertaking pre-tasks, travel as well as time for attending research.
The massive growth in Internet use, both at home and on-the-go and specifically via social media, provides more direct consumer research options and ways to better understand the digital world. New digital functionality such as wikis, video filming and uploading and messaging also gives researchers new ways to capture information. All helps researchers and customers work together to explore and develop ideas.
Some groups love the digital world and are easier to engage e.g. the youth market. Groups such as early technology adopters also help pressure-test new ideas and anticipate the future. And allowing anonymous responses encourages participation and openness too.
Some digital media offer an almost ‘instant’ sample. For example, polls on Facebook, Twitter or blogs. Though lots of followers are needed to generate useful insights.
The growing range communications, for example via smartphones, make it easier to reach a wide geographic target. In-built cameras also make it easier to collect visual or audio insights. Thus avoiding travel and some communication costs.
However, some technology, for example, as used in online qualitative research, is more difficult to master. So allow time for set-up, to help respondents, as well as to moderate and analyse research. All adds to costs making online discussions more expensive than face-to-face.
Online moderation is more difficult too. The process is often more linear and mechanical, thus limiting ability to pursue all avenues of exploration. There are also visual limitations. Zoomed in head shots or screen size room views, make it difficult to see the big picture, and non-verbal responses. Qualitative responses vary between the superficial and detailed. Thus, superficial responses require probing. Conversely, verbose responses, take time to follow and interpret.
(1) Mehrabian Albert H, ‘Silent Messages; Implicit Communication of Emotions and Attitudes’ 2nd edition 1981.
If you have some new or digital research methods we’ve not covered then please let us know!
Focus groups are a tried and tested qualitative research staple, having risen to prominence in the 1950s (1). Yet in today’s highly competitive environment relying on simple focus groups alone is limiting. If everyone just uses focus groups how can anyone possibly unearth new insights (2)?
As insights can come from anywhere the recipe for success is to use mixed methods. Both within focus groups, as well as other qualitative methods, to explore respondents from different angles, and in different ways. When designing research we employ four strategies to unearth new insights; we call these the four Cs: Context, Challenge, Collaboration and Calculation.
To understand the context in which consumers make choices requires getting up close. For example, through observation and recording daily life. To understand who consumers are, their needs, behavioural influences and the processes involved in choosing to buy or consume a product. They are seldom what you think. For example, by exploring the customer journey (say in food) from discovery through to choosing, buying, storing, preparing, eating and using the left-overs and packaging helps reveal what’s important and nice-to-have at each stage. Including success factors such as convenience, ease of use and sustainability. Observing meal preparation also helps reveal product misconceptions or packaging inadequacies. And observing eating occasions helps explore social drivers and barriers. All pinpoints previously unconsidered product, positioning and promotion issues, and opportunities.
What consumers think and feel is based on their own frame of reference i.e. experiences, prejudices, and memory. Stimulating with new experiences helps uncover new, hidden or forgotten thoughts. Do this by taking them out of their comfort zones and giving new experiences. For example, giving consumers a new or different product to try, helps reveal new or unmet needs, or barriers to overcome. Combining loyal and lapsed consumers in a ‘conflict situation’ to debate what’s good, bad or plain ok about a product or service helps reveal barriers to usage. It can also shed light on the strength of views and whether, and if so, how these can be overcome.
No-one has a monopoly on good ideas and we live in a society with increasing free-flow of information and collaboration. Consumers’ familiarity with advertising and brands means that they are more ‘savvy’ and able to converse in ‘technical’ terms. This is a boon for researchers and marketers as it allows consumers to evaluate and create new marketing solutions. The concept of collaboration applies to both who, how to and what to research. The only limiting factor is our imagination! Involving technical experts or opinion leaders in research brings even more critical and creative thinking. It also brings the future closer.
While qualitative research is good at answering ‘what’ and ‘why’ questions it is less effective at answering ‘how’ many or ‘how much’ questions. Using simple creative stimuli and scoring techniques overcomes the problem by providing a more substantive response. These allow people to think for themselves and mitigate the group ‘herd’ effect, and also distinguish the ‘really good’ or ‘poor’ from the ‘ok’ or ‘indifferent’. Thus spotlighting winning ideas and making sure research really helps marketing colleagues drive the business forward.
1. While focus groups are a qualitative research ‘default’, embrace the 4 Cs, to view your market research challenge in a new light and unearth more insights.
2. While quantitative and qualitative research were once different disciplines, and worlds apart, they are now blurring and overlapping. This presents new creative opportunities for researchers.
3. It is a myth that mixed research methods cost you more. When writing your next market research brief be clear about your aims, needs and guidelines – especially budgetary. This encourages agency ‘creativity’ and leads to more effective research ;-). For an inspiring response to your brief, get in touch.
(1) University of Columbia, History of Focus Group Research
(2) An insight is a ‘consumer need, want or belief that points to a new opportunity. Perhaps giving extra importance to something that has previously been ignored, forgotten or dismissed. Net the insight should shed new light and aid the business or brand. Source : Chapter 13, Managing Market Research, The Marketing Director’s Handbook.
The qualitative vs quantitative research debate started in the 1970s. It’s all about epistemology (1), a branch of philosophy concerned with the theory of knowledge. Qualitative research is described as ‘interpretivism’ i.e. non-scientific and subjective. Whereas quantitative research is ‘positivism’ i.e. scientific and objective.
But there is an academic argument that the two methods cannot and should not work together.
“The chief worry is that the capitulation to “what works” ignores the incompatibility of the competing positivistic and interpretivist epistemological paradigms that purportedly undergird quantitative and qualitative methods, respectively”. Blah, blah, blah…Prof. Kenneth R. Howe (2)
The blurring of lines between qualitative and quantitative research has gone on for some time. Though how many times have you attended focus groups and a done a quick ‘tally’ of responses to gain some quantitative guidance? Or, within an omnibus, included a few open-ended questions to add a little more colour? Superficial instances perhaps, but evidence of ‘blurring’ nonetheless.
A possible reason overlap is not fully acknowledged is because many believe the disciplines still run separately? Another is because qualitative and quantitative researchers are defined at birth. And thus never the twain shall meet? However, many researchers train under one discipline and most large research organisations run separate quantitative and qualitative departments.
Nevertheless from hard-won experience it is possible to marry both approaches and gain extra benefits. Thus there is room for a new model; a qualitative and quantitative research hybrid. Here are some examples:
Qualitative research discussions often solicit a few ‘subjective’ answers to questions where it is difficult to discern differences in meaning. For example, whether there are differences in meaning are between ’like’ and ‘love’ or ‘great’ and ‘good’ etc. However, when two people say they ‘like’ something, they may not mean the same thing. Though seeking numeric measures, using a simple likert scale (3) better distinguishes the ‘wheat from the ‘chaff’.
So rather than asking consumers who ‘likes’ what, asking them to say who ‘would definitely try or buy’ product ideas clarifies product purchasing intent. This is a particularly useful ‘gate’ in a typical NPD process. It helps better assess market potential and marketing implications. Thus, when developing new products this can help save you barking up the wrong tree. And also help you save thousands of hours and pounds!
Quantitative data uses open-ended questions to explain the numbers. However, in many cases it doesn’t explain anything because respondents fail to fill in the boxes or just write two or three words. Data is also costly to code and cumbersome to analyse.
However, combined qualitative and quantitative research can assess and improve products and more. For example, in a recent study, respondents tasted and critiqued a number of competitive food products. Research was conducted in a high traffic place so people could be recruited off the street into a hall. With some support from a moderator, consumers completed a simple survey to assess relative product appeal and brand fit. Also opportunities for product improvement as well as reasons why.
The same techniques can assess service ideas, communications and packaging. For example, at the pack refinement stage, to give a clear read on shelf stand-out, and reasoning. Firstly, by co-opting a minimum of 100 consumers to check a mocked-up retail fixture. Then by identifying the appealing packs and critiquing them within the visual noise of a fixture provides a numerical assessment of stand-out. Finally, adding in a group discussion to deconstruct and reconstruct the pack elements adds understanding and guides improvement.
1. The debate does not have to be about qualitative vs quantitative research as there are also many other types of market research services. Yet each has a different role, application and benefits.
2. Combined qualitative-quantitative research offers the benefits of both qual and quant research methods. So dial either up or down to answer ‘why’ questions as well as gain meaningful numbers. Within this it is also possible to establish quotas for consumer types, and save time and money too. So do you need understanding or numbers? Or both? Choose a creative research agency to help you get the most for your money.
1. What is Epistemology? https://en.wikipedia.org/wiki/Epistemology
2. Howe Kenneth R. PhD – Professor of Philosophy at University of Colorado, Boulder. Against the qualitative-quantitative incompatibility thesis (or dogmas die-hard), Educational Researcher 17(8) 10-16 1988
3. What is a Likert scale? https://en.wikipedia.org/wiki/Likert_scale
Recent OFCOM Research highlighted that 71% of the UK receive 9 nuisance calls a month, and that telephone research is the #4 culprit (1). So has telephone research had its day? At the same time online grows apace. We’ve looked closely at the merits of telephone, online and face-to-face (ftf). So if you commission quantitative research surveys, this article summarises some insights and ideas to help you make the most of your research investment.
Quantitative research costs are sensitive to sample size, ease of reaching an audience or ‘incidence’, the length of survey, mode and complexity of fieldwork and analysis. Compared with online (index =100) fieldwork costs are typically higher for face-to-face (index 350-450) than telephone (index 250-300) due to the greater human time involved. Other costs such as coding for online research, computer aided telephone interviewing (CATI) and computer assisted personal interviewing (CAPI) are similar.
97% of the UK are online though many online surveys use panels which cover just 5% of the population. There are some geographic gaps and respondents are more ‘Internet experienced’. Thus some sample bias is possible. Nearly all homes have access to at least one phone though telephone databases cover just 60% UK homes (though even fewer will have agreed to take part in research!). Fixed line telephone reaches 79% homes (and proportionately more of the elderly) while mobiles reach 96% (and proportionately more of the young) (1). Face-to-face can reach most places (though with extra travel costs).
Online response depends on the nature of the panel, and how responsive and interested respondents are. Expect between 5-30%. Response from links on websites or emails will similarly depend on the nature of the source. Telephone responses have fallen over the last decade and responses are now around 10-15%. Face-to-face response is also around 15-20%.
The self-selection nature of online panels means there is a greater risk of respondents only participating in surveys that interest them. So-called avidy bias. Typically online respondents are younger, more familiar with the online world and spend more time on it. They are also more informed, more opinionated and more politically activist. (2) Panels also contain more early technology adopters though it remains possible to discern other types on the ‘diffusion of innovation’ spectrum.
Telephone research respondents present more socially desirable responses more often than face-to-face (3,4). This is particularly the case with those with lower intellectual ability or fewer years of formal education (i.e. C2DEs). Research has also shown that respondents are more comfortable discussing sensitive subjects face-to-face as they can see, and thus have greater trust in, the interviewer. Conversely, face-to-face interviews conducted in the respondent’s home minimises anonymity, making socially desirable responses more pronounced. Overall however, interpersonal trust between the interviewer and the respondent has a greater influence resulting in more honest responses. Face-to-face shows similar results to online (where there is no interviewer effect). However, some research (5) has observed higher valuation responses to some ‘willingness to pay’ questions (e.g. when there is a perceived ‘civic virtue’ in being seen to add to a common good).
Satisficing (a combination of the words satisfy and sacrifice) involves short-cutting the response process, settling on a solution that is ‘good enough’ but could be ‘optimised’.
Telephone poses an increased cognitive burden. It increases difficulty to comprehend questions, thus reducing the effort to cooperate, search the memory and process information. Perceived time pressure also fatigues and demotivates. This results in questions being less considered, giving rise to higher acquiescence (answering affirmatively regardless of the question), and reducing disclosure. Also choosing mid-points or only extremes in rating scales, easier to defend answers and having no opinions. This is most evident with those with lower intellectual ability. Research (3,4,5) suggests face-to-face researchers are better able to judge confusion, waning motivation, distraction (via watching a tv, eating etc.) and thus motivate and make it easier for the respondent to understand the questionnaire and improve cooperation on complex tasks. While online respondents go at their own pace.
1. Useful quantitative research surveys start with a clear market research brief. So decide your objectives, target market, what you need to know and any guidelines. Also beyond feasibility and answers to questions, what’s the relative importance of cost, speed, ‘reliability’ etc? Be clear too about expected sample sizes. All helps recommend the best market research service for your needs.
2. Understand the pitfalls in conducting quantitative research. Larger samples give greater reliability. Thus a sample over 1000 gives more reliability than a sample of 500, i.e. if repeating a survey 100 times, in 95 instances a confidence interval i.e. variance of responses will be within +/- 1%. So prefer shorter surveys to cut the risk of satisficing.
3. Make sure samples are not biased. Nationally representative samples are key to measure awareness, usage and market share. Anything else builds in bias and risks misleading. Thus ensure your sample removes any demographic, subject affinity, usage or other bias.
4. There are even more pitfalls in repeating a quantitative research survey or a brand tracker. So take extra care to make sure the pool of respondents delivers a sufficient and matched sample for each survey wave. Then findings will be comparable.
5. Beware spurious analysis. Remember the Whiskas advert that told us that ‘8 out of 10 cats prefer Whiskas’. This eventually changed to ‘8 out of 10 owners that expressed a preference said their cats preferred Whiskas’. However, what we still don’t know is how many said ‘don’t know’, how many expressed a preference, and the sample size. So in all quantitative research surveys, be clear about the sample size. Also what is statistically significant or merely directional to make the context clear. Ensuring clear and fair analysis gives more useful insights and thus leads to better decision-making!
1. OFCOM Telephone Nuisance Research (2014).
2. Duffy Bobby, Smith Kate, Terhanian George, Bremer John. Comparing Data from Online and Face-to-face Surveys. International Journal of Market Research Vol 47 Issue 6. (2005)
3. Holbrook Allyson L, Green Melanie C, Krosnick Jon A. Telephone versus Face-to-face interviewing of National Probability Samples with Long Questionnaires. Public Opinion Quarterly, Volume 67:79–125 (2003).
4. Szolnoki G, Hoffman D. Online, face-to-face and telephone surveys – Comparing different sampling methods in wine consumer research. Wine Economics and Policy 2 (2013) 57-66.
5. Lindhjema Henrik, Navrudb Ståle. Are Internet surveys an alternative to face-to-face interviews in contingent valuation? Ecological Economics 70(9): 1628-1637 (2011).