Skip to Content
  • Bain.com Home
  • Subscribe
  • Connect
  • About NPS
    Main menu

    About NPS

    • Understanding the System
    • The History
    • The Benefits
    • Three Types of Scores
    • Companies Using NPS
    • NPS in B2B
    Recommended reading: A collection of our best thinking
    Important Concepts
    • Measuring Your Score
    • Employee NPS
    • NPS and Growth
    • Predictive NPS
  • Resources
    Main menu

    Resources

    • NPS Prism® Benchmarks
    • NPS Loyalty Forum
    • Additional Vendors
    • Recommended Videos
    • CX Standards
    Listen to The Customer Confidential Podcast
    Interactive tool
    • Customer Love Quiz
  • Book
  • Insights
  • Contact
    Main menu

    Contact

    • Workshops and Speaking
    • Trademarks and Licenses
  • Bain.com Home
  • Subscribe
  • Connect
  • About NPS
    • About NPS

      • Understanding the System
      • The History
      • The Benefits
      • Three Types of Scores
      • Companies Using NPS
      • NPS in B2B
      Recommended reading: A collection of our best thinking
      Important Concepts
      • Measuring Your Score
      • Employee NPS
      • NPS and Growth
      • Predictive NPS
  • Resources
    • Resources

      • NPS Prism® Benchmarks
      • NPS Loyalty Forum
      • Additional Vendors
      • Recommended Videos
      • CX Standards
      Listen to The Customer Confidential Podcast
      Interactive tool
      • Customer Love Quiz
  • Book
  • Insights
  • Contact
    • Contact Us

      • Workshops and Speaking
      • Trademarks and Licenses
    Popular Searches
    • Net Promoter Score
    • Employee NPS
    • Benchmarks
    Your Previous Searches
      Recently Visited Pages

      Content added to Red Folder

      Red Folder (0)

      Removed from Red Folder

      Red Folder (0)

      Forbes.com

      Survey Sabotage: Is Your Customer Data Misleading You?

      Survey Sabotage: Is Your Customer Data Misleading You?

      Does your data match what you see at your business? If not, bias may to be blame.

      By Fred Reichheld and Rob Markey

      • min read

      Article

      Survey Sabotage: Is Your Customer Data Misleading You?
      en

      This article originally appeared on Forbes.com.

      In this age of Yelp, Facebook and Twitter, consumer opinions have never been so powerful. While it’s easier than ever to find reviews and ratings, companies still grapple with how to best gauge their relationships with their clients.

      Many top companies are using or have adopted a Net Promoter System® to learn more about the quality of their relationships. A closed-loop feedback process can help businesses identify key clients, product features customers like or dislike, service opportunities and problems that need immediate attention. But these efforts can only go so far if the survey data is flawed by biased answers.

      Like a virus that yields no obvious symptoms, bias can creep into survey data all sorts of ways. Here are some of the most common factors that can contaminate your data, and how to avoid them:

      Risk: Fear and embarrassment. Consumers may avoid giving negative scores out of embarrassment, particularly when surveyed face to face or on the phone. In business-to-business (B2B) situations, smaller customers may avoid negative reviews when surveyed by a large or critically important vendor, for fear of being cut off.

      Prevention: To coax out candid feedback, leading NPS® practitioners find ways to offer each customer appropriate levels of confidentiality. In B2B settings, for instance, companies can maintain transparency by reporting average scores for an account while keeping individual ratings confidential.

      Risk: Employees who try to game the system. Sales and service representatives at auto dealerships are notorious for begging customers for top survey scores. Employees of other companies may engage in shenanigans, such as altering the phone numbers of unsatisfied customers so surveyors can’t reach them.

      Prevention: Companies with robust Net Promoter Systems pay close attention to potential gaming of this sort. At the car rental firm Enterprise, which uses a customer loyalty system called ESQi, gaming the system is called “speeding” and is grounds for dismissal.

      Risk: Responder bias. This occurs when the individuals who respond to a survey offer a different view than the overall population being surveyed. Imagine that a company administers a survey with a 20% response rate. It indicates that it’s Net Promoter Score® is 50% (60% promoters minus 10% detractors). Now imagine that the firm studies the behavior of nonresponders by looking at whether these customers made repeat purchases or spent more over time. The additional research determines that the mix of that group is 10% promoter, 40% passive and 50% detractor. That means 80% of customers who ignored the survey had an NPS of negative 40%. So the true NPS for the company—the weighted average of the two groups—is negative 22%, not 50%. The company’s relationship with its customers isn’t as rosy as it seemed.

      Prevention: If you can’t study nonresponders, consider scoring all of them as detractors (probably not too far off in business-to-business settings) or as a 50-50 mix of passives and detractors (a reasonable estimate for many consumer businesses). Naturally, a high response rate reduces the risk of responder bias.

      There is only one sure way to check whether your system has effectively defused the land mines of low response rates and bias: You must regularly validate the link between individual customers’ scores and those customers’ behavior over time.

      Does your research match what you see at your business? If not, bias may to be blame.

      Fred Reichheld is a Fellow and Rob Markey a partner at Bain & Company. They are authors of the bestseller The Ultimate Question 2.0: How Net Promoter Companies Thrive in a Customer-Driven World.

      Net Promoter®, Net Promoter System®, Net Promoter Score® and NPS® are registered trademarks of Bain & Company, Inc., Fred Reichheld and Satmetrix Systems, Inc.

      Authors
      • Headshot of Fred Reichheld
        Fred Reichheld
        Bain Fellow, Boston
      • Headshot of Rob Markey
        Rob Markey
        Advisory Partner, Boston
      Contact us
      Survey design
      Why I’m Skeptical of Surveys (and What to Use Instead)

      The best companies look beyond customer feedback to actual behavior, focusing on repeat purchases and referrals to understand what really earns growth.

      More
      Survey design
      How to Get Input from Survey-Saturated Customers

      Astute companies are deploying creative front-end customer research and effectively structured customer interaction to get the feedback they need.

      More
      Survey design
      Sleuthing Out Worthwhile Data

      Data makes it possible to learn more about many things—including customers—but not all data is equally good.

      More
      Survey design
      Covid-19: Why Feedback Is Critical in a Crisis

      Some companies have halted customer and employee surveys, but crises are when it’s most important for leaders to listen to their two key constituencies.

      More
      Survey design
      What Do You Want to Stand For When This Is All Over?

      Rob Markey talks with Bain partner and customer experience expert Maureen Burns about why companies that empathize with customers and listen to employee feedback will emerge stronger from the Covid-19 pandemic.

      More
      October 26, 2012
      Tags
      • Survey design

      Want to continue the conversation?

      We offer unparalleled analytic and organizational tools for the Net Promoter System. Together, we can create an enduring customer-centric culture.

      Get the latest on loyalty in your inbox. Our quarterly Loyalty Insights newsletter offers our best thinking and tips on running the Net Promoter System.

      *I have read the Privacy Policy and agree to its terms.

      Please read and agree to the Privacy Policy.
      Bain & Company
      Contact us Connect Subscribe Terms of use Privacy Environmental Policy Sustainable Procurement Policy Sitemap

      Net Promoter®, NPS®, NPS Prism®, Net Promoter System®, and the NPS-related emoticons are registered trademarks of Bain & Company, Inc., NICE Systems, Inc., and Fred Reichheld. Net Promoter Score℠ is a service mark of Bain & Company, Inc., NICE Systems, Inc., and Fred Reichheld.