Does your organization need a market research audit?

Is your market research (MR) driving your bottom line? Do you have a clear strategy and expected ROI for your MR efforts?  If not, can you really expect that the research you are doing will be an effective use of resources? As a practioner, I often look back at the research conducted and realize that [...]

Should political polls be held to a higher standard?

Anyone looking for a good glimpse into the state of the market research industry need look no further than the Open Letter penned by two senior members of Ipsos Reid and the coverage of how other professionals have responded. It shows that (a) the accuracy of political polls continues to be central to both our [...]

The “Magic” of the First Question

In the market research, the first question has incredible magic. Like the first sentence in a paragraph or the first line of a movie, that first question can grab the attention or put the respondent off. In addition it can give us more and better information.

The magic of the first question is derived from the fact that at no other time will the respondent be more focused and unaffected by the survey topic.

The answers to ALL other questions are affected by that first intervention. The effect on future questions may be subtle or profound but that first question changes the answers to all other questions. That first question does so many things:

• Tells the respondent more about what the survey is about. This could spur interest or disinterest.
• Gets the respondent to think about the issue more deeply for future questions. He or she may try to be consistent with that earlier answer or to rationalize it.
• Shifts the interview from a “conversation,” where respondents are just engaging with the introduction/ interviewer, to a “test” where respondents must fit what they think into the survey instrument.

The effects tend to increase over the length of the survey as respondents learn and adapt to the survey instrument. Their motivation can also change (for better or worse).

Ideally the first question should be easy to answer, engaging for the respondent, and address a core research objective.

We often see awareness, and unfortunately, screening questions at the beginning of surveys and while there is good reason for this practice greater attention needs to be paid to developing those first questions that take advantage of the unique opportunity that we have at the beginning of surveys.

You only get once chance to ask the first question. So for your next survey, make sure you get that first question right! It will set the tone for the rest of the interview.

Why a Crackdown on Rogue Market Research in Canada is Hard to Imagine

While the industry remains largely healthy in Canada, it is under considerable pressure to define and enforce standards. What is now playing out (partially, in the press) reflects the fundamental tension evident in new ways of doing things as well as the difficulty of self-regulating industry associations to deal with change.

Yet another news article appeared this week to question the integrity of the market research industry in Canada. The article is a natural reaction to the controversy around Campaign Research practices in the last federal election (more information on this topic is available here and here).  The MRIA (the national association) is now considering disciplinary procedures after receiving complaints about the activities of the firm, which is a gold seal member.

Susan Delacourt’s latest article is significant not because a firm may have done something wrong, because ultimately that does happen. The real issue is that the article  links the issue to a broader problem in the industry. There is a rehash of the infamous letter from Bricker and Wright criticizing some polling practices (see my discussion of it here) and reference to a recent book chapter that addresses the “troubled state of polling in Canada.”

The MRIA can, and probably will, act to address specific incidents of violations of the Code of Conduct but disciplinary actions will not change the frame through which polling is now being viewed. A frame that is not conducive to positive stories or positive industry development.

The discipline available is, however, quite modest. This reflects the fact that any self-regulating industry is only as good as the strength of the consensus that members share (in spite of the competition between them).

I would argue that the latest incident is in part a reflection not of a rogue violation but of a fundamental undermining of the consensus of what polling and market research is all about. In this sense, the letter by Bricker and Wright is a reflection of that breakdown. it also means that the solution — better and stricter enforcement — will be difficult to accomplish unless a new and clear consensus emerges in response to the controversy. A crackdown is only possible with such as consensus.

Collaboration and the Art of Business

Business is ruthless. Business is ultra competitive. Business is about winning. It is all of these things until it is not.

Sometimes it is about collaboration, about working outside of the box, about doing something together.

There is always risk involved in collaboration and it not the right approach for everyone. That said, small companies can elevate themselves and their partners if they embrace relationships that are more than sub-contracts to suppliers.

In the consulting and business services areas (where market research is firmly positioned), big companies all compete over the same claims usually rooted around the trifecta of (1) full-service; (2) quality; and, (3) experience. Price becomes the final consideration and because the claims for 1 to 3 are all similiar, the result is a commodification of the product/ service.

Small companies tend to stake out territories based, for example, on an approach to the insight or on  a set of experiences (e.g. industry-specific expertise). By definition they are rarely comprehensive and while they compete they are a natural place for collaboration.

When I launched my own company I had very little interest in just building another version of companies I had previously worked for and have embraced opportunities to partner with other organizations. By doing so I get rewarding experiences — the client often gets better results.

The latest strategic partner is Experience Renewal. Experience Renewal is an innovative company that thinks about research differently. Up front is storytelling not traditional research reports. Weaving a story from multiple sources of information allows for a richer understanding of the customer experience. It is a great opportunity to partner because I support the philosophy and bring expertise to the table around the quantitative insights part of the story.

Collaboration is clearly different from outsourcing or contracting out. When it works, there is a seamless integration of the companies. The client will feel that the company is working as one. It is particularly important in areas where the work cannot be easily portioned. You can sub-contract data collection but how do you sub-contract the executive summary?

Do you embrace collaboration? Is there a way for you to improve your product or service through working with someone else? What’s holding you back?

We Overvalue Norms… in Market Research and in Life

As a society we value norms — how is something compared with the “normal”. School starts it off but throughout our lives we seem to find ways to comparing ourselves to some normal. Even online we have developed measures of how we are doing relative to some standard (see klout).

When we reduce ourselves to a comparison with the normal we are inevitably losing the essence of who we are.

In market research norms are particularly banal. The norms, by definition, are an average of a disparate number of, probably unlike, observations. A great score or a really bad score moves the needle of a norm only perceptively with a lot of observations.  The norm is not special; its just there.

Particularly used to benchmark customer satisfaction or ad success, the norm is an interpretive crutch used to avoid the more difficult work of unpacking a research findings.

• “You are doing better than average”… so no problem.
• A good score is anything above…”; … this is your target.

Norms may be here to stay but as Seth Godin has pointed out, We Are All Weird, the world is getting weirder. Our preoccupation with normal is a cultural phenomenon. “Marketers have made normal a moral and cultural standard, not just a statistical one” (Seth Godin, We Are All Weird).

As a proponent of evidence-based decision-making, I appreciate the desire to measure and compare. But why use an amorphous norm? How about a “best-in-class” metric? Or, how about develop your metrics with your business in mind? Who wants to be average?

Science and Public Health: When Messages Cross

Public health authorities are in a significant bind when it comes to persuasion. Encouraging people to adopt healthier lifestyles is key to better individual and social outcomes (from less disease to lower costs). Changing behaviour is, however, not easy as evidence from growing obesity rates to continued smoking . A further challenge emerges from the fact that science continues to evolve and, sometimes, challenge previously held beliefs.

Unfortunately, one of the consequences of social marketing campaigns is that while they struggle to convince some, they also create deeply held beliefs in many about the “righteousness” of various courses of action.

Just this week, Canadian women are being challenged to choose science when it comes to breast cancer screening (a good article about the issues is here). After years of promoting regular screenings and self-exams for all women, Canada (as well as other countries) has adopted a revised set of recommendations that significantly reduce the role of screening. For example, those women under 50 are not recommended to have mammograms anymore and the clinical and self-examinations are no longer recommended at all.

One of the reasons why the science in this case will not easily translate into change is that advocates of screening (doctors, survivors, other Canadians) have built up the case (rationally and emotionally) for screening to the point that many could probably resist arguments against screening (which new guidelines will just appear as).

The issue is important because the revised guidelines are not just about unnecessary tests — they are a response to analysis that indicates that the unnecessary screening has negative consequences for those who undergo them. And, in this, the argument bumps up against the dreaded “common sense.” How can a test produce a negative outcome?

When we move to probability and risk, the public is significantly challenged to understand how science works. And, when faced with this potential confusion may be less likely to accept the science by suggesting that it might be wrong.

The issue of breast cancer screening, is, however only one of many issues where science and health promotion are coming up against a skeptical or resistant public. Consider the following examples:

1. PSA test for prostate cancer. The U.S. Preventive Services Task Force has recommended that healthy men no longer be tested because testing had no impact on mortality and was associated with significant negative effects resulting from false positive.
2. The HPV vaccine which is supported by science and advocated by health promoters has run into significant resistance on the part of various groups. This reflects the significant resistance to vaccination in general, which at its worse has led some parents to intentionally expose their children to disease so that they build “natural” immunity.

Public opinion on these issues and the challenges of communicating advice based on science are illuminating. The question is why does science have such a hard time? Is it a function of the natural skepticism that is at the heart of all scientific inquiry? Or, it is all about our misguided “common sense”?

Voting Early is Not a Sign of Electoral Health

During the last Canadian federal election campaign, there was considerable speculation about the large increase in turnout at the advance polls. At the time I speculated that the media and other commentators were wrong in suggesting the increase was a product of the campaign (e.g. greater interest). I thought I would revise the issue and see if we could better understand the dynamics of voting early.

First, now that the official numbers are out we can actually get a better sense of voting in the 2011 federal election. Turnout (votes cast divided by registered voters (not actual eligible voters) was officially 61.1% in Canada. This represents a modest rise from 58.8 and does not match the 64.7 in 2006.

In 1997 only 5.4% of all votes cast were at the advance poll (which are held on the 10^th, 9^th and 7^th day before Election Day) compared to 14.3 per cent in the last election.  The trends of turnout and voting in advance polls are not following the same path. Given this, it is hard to see that the surge in advance poll voting represented rising public interest.

With each election campaign, more and more voters decide to cast their votes at an advance poll and Elections Canada is making it easier by expanding the number of these polling stations.

Second, I went back and revisited data from the 2008 Canadian Election Study. Since the study included questions about the advanced polls, it was possible to see who was more likely to vote in the advance polls. The full is available on my site or here but the evidence is that:

• Older Canadians are the most likely to vote and to take advantage of the advance poll when they do so. It is not reaching people who otherwise would not have voted.
• Voting early is associated with having made up one mind before the campaign started, with higher interest, and with feeling an obligation to vote.

So, clearly the combined evidence suggests that voting early is just a new phenomena in Canadian elections.

The trend toward early voting is likely to further reduce the impact of the campaign on the outcome and therefore lead to greater importance for the non-election period — when, ironically, fewer people are paying attention to  politics.

While giving people more opportunities to vote is a plus, we should not take evidence of voting in this way as a sign of electoral health. In fact, it may be a sign of quite the opposite.