Most business leaders know the value of customer input and feedback. But if some recent developments are any indication, the right way to get that customer insight is less well understood and practiced.

Take, for example, the recent letter J.C. Penney CEO Ron Johnson issued to his company’s customers. After explaining the intention behind recent changes in the store experience, he wrote:

“I hope you’ll let me know how we’re doing, and share any ideas that could help us do better. Just click the link below to send me a note.”

Any feedback he received was likely lacking in reliable, actionable insights. Non-response bias was surely a factor, since those who chose to provide feedback were probably more engaged with the brand (either positively or negatively) than those who didn’t. Also by referencing the company’s commitment to its founder’s values and describing the various tactics it has undertaken in the first part of the letter, surely readers were biased. This is an effect known in psychology as priming.

Plus, Johnson’s general call for feedback and ideas probably prompted responses that contained generalities too broad and varied to inform any specific strategies. So his solicitation for customer input can be characterized only as a goodwill gesture—not a concerted effort to truly understand his customers.

Social Media Research

The purpose of another customer relations development is also questionable:  Companies such as Wal-Mart, Frito-Lay and Samuel Adams are posting surveys on Facebook, Twitter, etc., turning these social media sites into extensions of their research departments. This growing practice of using social networks to gather ideas and input is an effective customer engagement and trend-monitoring effort, but it’s a poor substitute for rigorous consumer research and market testing. 

Social networks and tools are fraught with an even more significant non-response bias than that of the letter from Penney’s CEO. Those who participate in social networks in general, and on a brand’s page specifically, are not likely to represent all of its customers.  And their opinions certainly don’t represent prospects.

Shady Statistics

Response bias (not the opposite of non-response bias) is another problem inherent in such social surveying tactics: People often say one thing and do another. Countless studies show people are generally poor predictors of how they will actually behave. The gap between claimed and actual desires and behavior is further exaggerated when people know they’re being watched, something psychologists call “observer-expectancy effect.” So asking Facebook followers to vote on a new product is an unreliable method of assessing demand, since they’re likely to be considering how their friends might perceive their preferences.

There are other examples of questionable customer inquiry. Take the practice of checkout attendants at the local grocery store who mindlessly mutter “Did you find everything you needed?” as they scan customers’ items. Do they really want to know the answer? How about those register receipts promising entry into a drawing in exchange for participation in a survey?  Do they produce any semblance of true insight into the drivers of customer satisfaction or dissatisfaction?

Research Done Right

No doubt, all of these efforts are well-intentioned and they may serve important marketing or quality control purposes. If you have limited resources, they may be the only means for any gleaning customer insight. But they should not be considered adequate surrogates for thorough and thoughtful research and analyses.

Rich, actionable customer insights are generated from research that is:

• Specific. Studies should be focused on discrete objectives and questions within a survey or discussion should be clear and exacting. That way, respondents understand what you’re asking and analysts are certain that people’s responses are what they’re looking for.
• Targeted. Research that is designed to understand a particular group of people is far more revealing than research that surveys a general population because you can dig into the most relevant issues or ideas—and implement the findings more directly.
• Methodologically rigorous.  Studies should be designed to avoid the previously mentioned pitfalls like non-response bias and priming effect. They should also employ statistically significant sample sizes, carefully worded questions and robust sources of participants like nationally-representative survey panels.
• Inclusive of multiple sources and methods. Reliable research draws upon multiple methods to acquire data and doesn’t rely on any one single method. By integrating qualitative and quantitative methodologies and varied analytical approaches, you can verify your results and ensure you thoroughly understand the issues and potential solutions.
• Proactive and anticipatory. Research should be used to inform decisions before they’re made, not to justify them afterwards. Likewise, it shouldn’t be used to understand only past occurrences, but also to predict future changes in consumer behavior.

When it comes to understanding your customers, a little knowledge can be a dangerous thing.