There's a multimedia presentation in the online Harvard Business Review about how to get a better understanding of what customers may truly want.
I think this research method is a small step in the right direction. It shows how most quantitative evaluations that ask consumer to rate the importance of different features can lead to average ratings of each feature against the others. The new method they propose forces trade-offs of one feature over others in a variety of groupings. This results in a better understanding of what the consumer truly would prefer.
While this method is an improvement, I can't help feeling a bit uncomfortable with the process of having consumers choose discrete features from a list that was created based on aggregated focus group input. What's uncomfortable is that a consumer's ultimate opinion is the result of how they perceive experiences holistically. In the presentation the example was from a restaurant chain. The features were things like "food served hot and on time", or "updated decor". What this does is force consumers to decide which single attributes are most responsible for creating the experience they would like to have.
Asking consumers to do this is unfair. They are not restaurant space designers, and they are not chefs. It takes a lot of work to understand what experience the consumer would like to have, and then even more work to figure out how to combine the design and food elements to holistically deliver this experience. It also leaves designers and chefs with ambiguous instructions for how to proceed. Should the chefs deliver hot hamburgers quickly while the designers put white linen on the tables? Yes, this is an extreme, but we've all seen it happen.
Next time you're evaluating features in your next product, think about what you're really asking consumers to do. Are you evaluating products holistically, or are you shirking your responsibility and asking consumers to translate features to experiences for you?
Hmmm – This Bain stuff looks like good ol’ choice modelling to me. I think your criticisms are absolutely valid. From my limited understanding, choice modelling works well when you are presenting consumers with choices that are easy to understand and assess and the usual caveats around any kind of survey instrument apply.
Matt, that’s a great way to describe the right time to use choice modelling. It’s striking to me that we see articles like this in HBR and Strategy+Business, and they are all labeled as ways to fully understand consumer desires. That’s misleading and could be dangerous.
What will it take for these publications to accept articles that discuss new ways to mitigate risk while embracing ambiguity through holistic consumer understanding?
Any choice based task is a lot better than asking straight preference or ranking, but I agree that the choices consumers need to given have to be quite specific. (and I still chuckle that this is seen NOW as the way to measure utilities of choice in product drivers…this sort of stuff has been happening for a LONG time)
Kelly I share your chuckle regarding the timing. All the more reason for us to spread our methods as broadly as we can. There is a lot of well-intended, yet misleading work out there, and we need to do something about it.