Customer Surveys Are No Substitute for Actually Talking to Customers

I’ll never forget the questionnaire handed to me midway through a flight from Los Angeles to Sydney. It was massive. Page after page of detailed tick-the-box or circle-the-response questions – it seemed to me it would take the full 13-hour flight to complete. I started, but it was too much work and I abandoned it halfway through. I thought to myself: does management really believe they get valid and reliable data from these surveys?

For many organizations, surveys like this qualify as “talking to the customer.” They’re ubiquitous – appearing in hotel rooms, after online purchases, and in hospital emergency departments. But do they really qualify as customer consultation? Or are they a symptom of an isolated management just putting on a show of interest? What can be done instead?

The obvious answer is to talk with customers directly. But executives are often put off by the idea of interviewing customers individually, believing that it involves many hours and massive expense. Instead they get together in a group and guess what the customer — or any stakeholder — wants, with only the flimsy, half-hearted responses of customer surveys to guide them. It usually results in the wrong answers and the wrong strategies.

If only they knew just how simple and straightforward a customer interview process can be, and how rich the rewards, if you know how to ask the right questions.

At the beginning of my public workshops on strategic planning I conduct an exercise which has a profound impact on my audience. I choose a convenience store as my business example, since everyone has been a customer of one, and I ask what seems like a benign question: “How do you decide to shop at one convenience store versus another?”

This is my audience’s first step on the path to strategic thinking and strategic planning.

The responses come quickly and they always yield the same six criteria: location, hours of operation, range of goods sold, store presentation, customer service, and price. I label these the particular “strategic factors” of running a competitive convenience store. I explain how every business can define a list like this as the basis for developing customer strategies, for differentiating a business and for achieving competitive advantage.

What my audience does not appreciate at the start of the workshop is that it took a group of people to come up with all six factors. No one individual would have articulated the complete set. That’s because each person’s recent experiences at a convenience store has pushed his or her recall towards one factor or another. If someone chose the store due to its easy-to-navigate design, presentation springs to mind. If someone else sought help getting the groceries into the car, customer service was front and centre. If price was the main factor, that’s what leapt out. None of this neutralizes the relevance of the total final list of six factors. My audience always signs off on its validity.

But how does this translate into gaining insights from customer interviews instead of surveys?

Consider a company I worked with recently that provides a range of civil engineering and planning services to clients nation-wide. I’ll call it Command. Its clients are large organizations involved in massive mining projects, multi-story buildings and big residential developments. In preparation for a strategic-planning exercise, which I was to facilitate, the CEO at Command asked me to interview a dozen of their key clients.

If you’re like a lot of people, your initial response might be: “Twelve clients? The sample is too small. It’s not enough to tell you anything useful.”

But in conversations with clients, you’re after quality not quantity. You want to know how they think about issues and how they make decisions. You want to get inside their minds. You want to get a feel for their needs, wants and pain. You can’t get that from a questionnaire.

In the interviews, some of which were conducted in person and others by phone, I asked this key question among others: “What criteria do you weigh up in deciding to engage Command or the competition?” Notice that the question is open-ended. Note also the similarity between this question and the one I asked my audience about the convenience store.

However, there was a key difference in the process. In my public workshops on strategic planning, I had a group of convenience-store customers all in the room at the one time. For Command, I was talking to one client at a time. The full picture was bound to emerge more slowly – drip-fed, if you wish. Like the convenience store exercise, though, I didn’t expect a single client to provide me with the full list of criteria/strategic factors. Their responses depended on their individual experiences. For example, one client made “safety” paramount, because this factor was front and centre in his company’s policies.

Overall, the clients nominated seven factors: capability (ability to execute the work required), client service (personal and tailored), quality (meeting professional standards), image (reputation in the industry), cost, location (of offices across the nation), and safety (in terms of prior record and systems in place).

Apart from this valuable list, several other important outcomes emerged from the interviews. One concerned the list of strategic factors itself. The clients came up with a far crisper set than the management team had been able to. The clients’ focus was “outside-in” and driven by needs, whereas management’s focus was “inside-out”, muddied by internal politics and functional biases.

A particularly useful insight related to Command’s assumptions about future development. The company had grown recently by acquiring several other businesses, both in its core area and in related industries. This broadened Command’s range of services and took the company nation-wide. The company’s management and board believed that Command would obtain a competitive edge through this.

But this was not borne out by the customer interviews. Clients explained that they saw no special connection across the expanded service range. Further, they pointed out that their offices made decisions locally, rather than nationally from head office. While the expansion “felt” like a natural fit to management, it didn’t to clients. Command was in the process of being structured based on its false premise.

This presented Command with a reality check and an unanticipated challenge: they would need to convince clients that they would benefit from the nationwide plan that Command had designed for them.

Clearly, there’s no way that traditional surveys could deliver this kind of deep strategic analysis. But what about the expense of conducting interviews with a large sample of customers?  To get meaningful customer input, would you not need to conduct a great many interviews?

The short answer is: you need enough interviews to get to the point at which you hear nothing new and material is being repeated – so called “saturation”.  You can, it turns out, reach this point surprisingly quickly.

Let me give you some examples. My company was asked to undertake a series of customer interviews for an importer of gift products. These are sold in boutique outlets in shopping malls. We quoted to undertake 16 interviews. After 12 the picture was clear, saturation had been achieved. The further four interviews simply confirmed the result. That doesn’t seem particularly daunting, does it?

At Command each client also nominated several factors but as before no one client provided the full set. It took, once again, just 12 interviews to obtain the full set of strategic factors – with definitions. I could have continued with the interviews till I reached, say, 25. But experience told me that I wouldn’t have captured any additional knowledge or insight beyond the initial 12 respondents.

When it comes to obtaining customer input, executives often think a multiple-choice survey will be the most cost-effective option. They have their place, of course, such as if you want to know the percentage of people who liked or disliked something. But these instruments are shallow and derivative at best, and at their worst they can be annoying and counterproductive. So don’t let them become an excuse for not talking to the customer.

Author: Graham Kenny

Source: www.hbr.com