Yet another news article appeared this week to question the integrity of the market research industry in Canada. The article is a natural reaction to the controversy around Campaign Research practices in the last federal election (more information on this topic is available here and here).  The MRIA (the national association) is now considering disciplinary procedures after receiving complaints about the activities of the firm, which is a gold seal member.

Susan Delacourt’s latest article is significant not because a firm may have done something wrong, because ultimately that does happen. The real issue is that the article  links the issue to a broader problem in the industry. There is a rehash of the infamous letter from Bricker and Wright criticizing some polling practices (see my discussion of it here) and reference to a recent book chapter that addresses the “troubled state of polling in Canada.”

The MRIA can, and probably will, act to address specific incidents of violations of the Code of Conduct but disciplinary actions will not change the frame through which polling is now being viewed. A frame that is not conducive to positive stories or positive industry development.

The discipline available is, however, quite modest. This reflects the fact that any self-regulating industry is only as good as the strength of the consensus that members share (in spite of the competition between them).

I would argue that the latest incident is in part a reflection not of a rogue violation but of a fundamental undermining of the consensus of what polling and market research is all about. In this sense, the letter by Bricker and Wright is a reflection of that breakdown. it also means that the solution — better and stricter enforcement — will be difficult to accomplish unless a new and clear consensus emerges in response to the controversy. A crackdown is only possible with such as consensus.

Sometimes it is about collaboration, about working outside of the box, about doing something together.

There is always risk involved in collaboration and it not the right approach for everyone. That said, small companies can elevate themselves and their partners if they embrace relationships that are more than sub-contracts to suppliers.

In the consulting and business services areas (where market research is firmly positioned), big companies all compete over the same claims usually rooted around the trifecta of (1) full-service; (2) quality; and, (3) experience. Price becomes the final consideration and because the claims for 1 to 3 are all similiar, the result is a commodification of the product/ service.

Small companies tend to stake out territories based, for example, on an approach to the insight or on  a set of experiences (e.g. industry-specific expertise). By definition they are rarely comprehensive and while they compete they are a natural place for collaboration.

When I launched my own company I had very little interest in just building another version of companies I had previously worked for and have embraced opportunities to partner with other organizations. By doing so I get rewarding experiences — the client often gets better results.

The latest strategic partner is Experience Renewal. Experience Renewal is an innovative company that thinks about research differently. Up front is storytelling not traditional research reports. Weaving a story from multiple sources of information allows for a richer understanding of the customer experience. It is a great opportunity to partner because I support the philosophy and bring expertise to the table around the quantitative insights part of the story.

Collaboration is clearly different from outsourcing or contracting out. When it works, there is a seamless integration of the companies. The client will feel that the company is working as one. It is particularly important in areas where the work cannot be easily portioned. You can sub-contract data collection but how do you sub-contract the executive summary?

Do you embrace collaboration? Is there a way for you to improve your product or service through working with someone else? What’s holding you back?

When we reduce ourselves to a comparison with the normal we are inevitably losing the essence of who we are.

In market research norms are particularly banal. The norms, by definition, are an average of a disparate number of, probably unlike, observations. A great score or a really bad score moves the needle of a norm only perceptively with a lot of observations.  The norm is not special; its just there.

Particularly used to benchmark customer satisfaction or ad success, the norm is an interpretive crutch used to avoid the more difficult work of unpacking a research findings.

• “You are doing better than average”… so no problem.
• A good score is anything above…”; … this is your target.

Norms may be here to stay but as Seth Godin has pointed out, We Are All Weird, the world is getting weirder. Our preoccupation with normal is a cultural phenomenon. “Marketers have made normal a moral and cultural standard, not just a statistical one” (Seth Godin, We Are All Weird).

As a proponent of evidence-based decision-making, I appreciate the desire to measure and compare. But why use an amorphous norm? How about a “best-in-class” metric? Or, how about develop your metrics with your business in mind? Who wants to be average?

Unfortunately, one of the consequences of social marketing campaigns is that while they struggle to convince some, they also create deeply held beliefs in many about the “righteousness” of various courses of action.

Just this week, Canadian women are being challenged to choose science when it comes to breast cancer screening (a good article about the issues is here). After years of promoting regular screenings and self-exams for all women, Canada (as well as other countries) has adopted a revised set of recommendations that significantly reduce the role of screening. For example, those women under 50 are not recommended to have mammograms anymore and the clinical and self-examinations are no longer recommended at all.

One of the reasons why the science in this case will not easily translate into change is that advocates of screening (doctors, survivors, other Canadians) have built up the case (rationally and emotionally) for screening to the point that many could probably resist arguments against screening (which new guidelines will just appear as).

The issue is important because the revised guidelines are not just about unnecessary tests — they are a response to analysis that indicates that the unnecessary screening has negative consequences for those who undergo them. And, in this, the argument bumps up against the dreaded “common sense.” How can a test produce a negative outcome?

When we move to probability and risk, the public is significantly challenged to understand how science works. And, when faced with this potential confusion may be less likely to accept the science by suggesting that it might be wrong.

The issue of breast cancer screening, is, however only one of many issues where science and health promotion are coming up against a skeptical or resistant public. Consider the following examples:

1. PSA test for prostate cancer. The U.S. Preventive Services Task Force has recommended that healthy men no longer be tested because testing had no impact on mortality and was associated with significant negative effects resulting from false positive.
2. The HPV vaccine which is supported by science and advocated by health promoters has run into significant resistance on the part of various groups. This reflects the significant resistance to vaccination in general, which at its worse has led some parents to intentionally expose their children to disease so that they build “natural” immunity.

Public opinion on these issues and the challenges of communicating advice based on science are illuminating. The question is why does science have such a hard time? Is it a function of the natural skepticism that is at the heart of all scientific inquiry? Or, it is all about our misguided “common sense”?

First, now that the official numbers are out we can actually get a better sense of voting in the 2011 federal election. Turnout (votes cast divided by registered voters (not actual eligible voters) was officially 61.1% in Canada. This represents a modest rise from 58.8 and does not match the 64.7 in 2006.

In 1997 only 5.4% of all votes cast were at the advance poll (which are held on the 10^th, 9^th and 7^th day before Election Day) compared to 14.3 per cent in the last election.  The trends of turnout and voting in advance polls are not following the same path. Given this, it is hard to see that the surge in advance poll voting represented rising public interest.

With each election campaign, more and more voters decide to cast their votes at an advance poll and Elections Canada is making it easier by expanding the number of these polling stations.

Second, I went back and revisited data from the 2008 Canadian Election Study. Since the study included questions about the advanced polls, it was possible to see who was more likely to vote in the advance polls. The full is available on my site or here but the evidence is that:

• Older Canadians are the most likely to vote and to take advantage of the advance poll when they do so. It is not reaching people who otherwise would not have voted.
• Voting early is associated with having made up one mind before the campaign started, with higher interest, and with feeling an obligation to vote.

So, clearly the combined evidence suggests that voting early is just a new phenomena in Canadian elections.

The trend toward early voting is likely to further reduce the impact of the campaign on the outcome and therefore lead to greater importance for the non-election period — when, ironically, fewer people are paying attention to  politics.

While giving people more opportunities to vote is a plus, we should not take evidence of voting in this way as a sign of electoral health. In fact, it may be a sign of quite the opposite.

My relationship with brands that I use is a relationship but it is not one in which the brand can call on me to do it favours and completing a survey is a favour.

Expecting free participation is not new in the research industry. Telephone surveys began and are largely still based on the concept of a willingness on the part of respondents to participate without an incentive. And while response rates to telephone surveys are dreadfully low at the moment, for a long time large proportions of the population were willing to give their time with no personal benefit, except the feeling of participation.

When the survey is about public affairs, the giving of my time makes sense as a public service, but when my credit card company wants to know my satisfaction with their call centre I fail to see the tangible benefit to me.

Surveys — at least some types — may improve products or services (for which we benefit) but as a public good I do not need to participate in order to benefit from the improvements companies make.

Incentives certainly increase an organizations costs but the investment is fairly small because a good incentive is one that sends a signal to the respondent about how much the survey response is valued. More importantly they seem in-line with social trends (consider the number of offers that are tied to liking the brand on social networking sites). People seem to expect to get something in return for their efforts.

So why are companies not using incentives? Three possibilities come to mind:

1. There is enough sample that they do not care about response rates. People not motivated to participate are just ignored.
2. Incentives are no longer proving effective. Seems unlikely.
3. Brands feel that incentives are in conflict with their identity and engagement strategy with consumers. If I pay them for this, is it reminding them that it is a commercial relationship?

Have you noticed more and more of the businesses you are dealing with expecting you to give them feedback for nothing in return? Does an incentive work for you?

The magic of the first question is derived from the fact that at no other time will the respondent be more focused and unaffected by the survey topic.

The answers to ALL other questions are affected by that first intervention. The effect on future questions may be subtle or profound but that first question changes the answers to all other questions. That first question does so many things:

• Tells the respondent more about what the survey is about. This could spur interest or disinterest.
• Gets the respondent to think about the issue more deeply for future questions. He or she may try to be consistent with that earlier answer or to rationalize it.
• Shifts the interview from a “conversation,” where respondents are just engaging with the introduction/ interviewer, to a “test” where respondents must fit what they think into the survey instrument.

The effects tend to increase over the length of the survey as respondents learn and adapt to the survey instrument. Their motivation can also change (for better or worse).

Ideally the first question should be easy to answer, engaging for the respondent, and address a core research objective.

We often see awareness, and unfortunately, screening questions at the beginning of surveys and while there is good reason for this practice greater attention needs to be paid to developing those first questions that take advantage of the unique opportunity that we have at the beginning of surveys.

You only get once chance to ask the first question. So for your next survey, make sure you get that first question right! It will set the tone for the rest of the interview.

Many, including the Governor of the Bank of Canada, have supported the move to get public approval.

The depth of the problem and solution, make a referendum a powerful way for Greece to demonstrate its commitment to Europe and the policy pain they are being asked to endure. If they reject the compromise and turn away from Europe, one wonders if the European experiment will fail.

The alternative is probably getting out of the Euro 

At the moment, the referendum looks like a vote of acceptance or rejection of the austerity package that will no doubt have a severe impact on the standard of living of Greek citizens.

Given the pain people will  suffer, getting their buy-in through a vote is a good way to legitimize the policy direction, especially given the fractured and divided nature of political elites in Greece.

But what is the alternative to accepting the austerity measures? Surely, it is not a better deal. In fact, the referendum will effectively become a vote on the future of the Euro and Greeks participation in the Euro-zone. The question really needs to make this clear.

This is not just a problem for Greece?

The crisis may be centered in Greece but it is a bigger problem and the vote will have profound impacts on other members of the European Union (especially the Euro zone). While adopting a democratic course is commendable, the unspoken element of this is that citizens in other jurisdictions (who will also share the burden of the pain regardless of the direction Greece takes) do not get a say. This is especially important if the Greek voters reject the compromise and this has a cascade effect that destroys the Euro.

Given the urgency, there is another way

The referendum is an excellent tool for making long-term decisions about fundamental values and directions for a country. We expect these to be preceded by a lengthy and informed debate. In fact, the referenda that occurred to join the EU in many countries reflected this use of the tool. In the middle of a crisis (when emotions such as anger and fear may dominate), it is not clear that a referendum is the right course.

Representative public opinion polling can provide a very good idea of where Greek voters stand on all aspects of the choice — not just a make or break question. This should be the way forward combined with a real, sustained public dialogue about the future, which can include a discussion of the future of the Euro.

Both surveys contained insight and one provided a potential tonic to internal disagreements based on assumptions and beliefs about the organization.

Neither survey neared perfection but perfection is rarely attainable and is usually divorced from the economics of business decisions anyway. The errors or omissions in these DIY surveys rarely threaten the value of the data and are usually very affordable — even if you then hire someone like me to analyze the results.

The biggest hint that it is not really research is the phone call you receive after the survey seeking to get more information about your poor rating of some aspect of the experience you had.

Now I get it, how else is management to understand your experience with their company in a world in which only a small percentage of people complain to the manager. People are too often going to exercise their displeasure with their feet and/ or their mouths — telling everyone they can about their bad experience. Companies want to rescue these relationships and the survey is the guise they use to follow up.

From a consumer point of view there is value in providing feedback to companies since this enables them to improve their offering and service but giving up your anonymity should be something that is done consciously though a specific disclaimer and or option at the end of the survey.

In fact, the minimum standard for customer satisfaction research should be that you must give permission to have someone call you to discuss your findings. I have seen this done well in surveys where the survey has identified an issue and the respondent actually wants an opportunity to tell someone in management what was wrong.

The gold standard is independent and anonymity. It can get all of the information an organization needs to improve its relationships without being overly focused on the customer. Lets face it…. if you are going to call me to explain my survey results (with  no compensation for the call or the survey), then I am going to ignore your surveys.

Things to look out for:

1. A promise of anonymity at the beginning. This is a sure sign that the survey is primarily research and a lack of such promise means that your responses (even your name) might be shared internally and may be used to follow up with you.
2. The company running the survey. Does the survey look and feel like a survey from the company you did business with rather than an independent company? if yes, this is a sign that the survey is probably not motivated by research interests.

How do you feel about completing customer satisfaction surveys?