Yesterday I had two conversations about metrics for innovation; what should be measured, how do you measure it, and one question about whether it can or should be measured at all. As my definition of innovation is doing something new that adds value to the business, I believe innovation should be measured in order to capture that value. So, in the macro sense innovation can, and should, be measured.
However, I do understand the basis of the question of whether innovation can be measured at all. If you're dealing with a truly disruptive innovation, how can you measure when there are no existing benchmarks? Ultimately, you may find that innovation cannot be measured by using existing metrics. It then becomes necessary to develop new metrics as part of the innovation process. Here are some of my thought processes for measuring whether a disruptive innovation with no existing benchmarks will be a success. I'm being careful to call them thought processes rather than methods because the resulting process is different based on each individual context.
Benchmarks Typically, improvements to existing products or processes are measured against the existing offering. A truly new process or product or process cannot be compared directly, and a different context needs to be developed. I often use scenarios in this situation. For example, I may show a scenario in which a consumer is engaged in a specific task. I then ask evaluators two questions. "Describe what is going on in this scenario – what is the person trying to do?" and then "What does XYZ product need to do to help accomplish this task?" After they describe the attributes of a successful offering, I present the options (including the current and competitive offerings) and ask which one is most likely to achieve what they described. What happens is that the benchmark shifts from a current embodiment to a goal to be achieved.
Consumer involvement Never underestimate the consumer's bias toward what is familiar. In the evaluation method described above, consumers are not asked their preference of one product over another. When you are evaluating a truly disruptive innovation the offering may look so foreign or exotic, that consumers may not readily prefer it. It's necessary to create a context where they are evaluating based on how well a new offering will accomplish the intended criteria. That said, the offering shouldn't be offensive, but keep in mind that successful innovations often fail initial preference tests.
The numbers When evaluating a truly disruptive innovation, it's important to quantify the value provided before defining the cost structure. This is especially helpful when developing an offering that has no existing benchmarks. In the example above, a second step would be to refine different iterations of the embodiment that performed the best. An evaluation could then be performed to figure out the perceived value of the new offering. (These evaluations can get very involved, so I won't try to define it in detail here. If people are interested I can show an example in another post.) Once the value is determined, a new cost structure can be defined as part of the development process.
Overall success At the end of the day, we need to know whether our efforts have been successful. It is a fairly straightforward exercise to measure the holistic success after the new offering has been in the market. I've often been asked about the success of a specific component, such as the pricing, marketing campaigns, or internal processes. At this point, I have not found a good (reliable) way to measure the success of any specific component of the process. I'm hoping that if we can start working with the ideas above with greater frequency and rigor, then the value of specific components will start to emerge. I did do one project where we were able to quantify some of the consumer perceived elements of the design, so I know it can be done. It was a fairly involved process, but if the stakes are high in terms of investment or brand equity it is well worth the effort.
I'm interested to know how others view innovation metrics, and whether there are other ways to look at the areas I've mentioned.
Ellen-
I agree that measurement of innovation is essential, especially during today’s economic downturn. Business experts are consistently telling us to innovate through the downturn. The best way to know you’re getting what you paid for from innovation is to measure.
And I agree that measurement of innovation is difficult because innovation efforts are often new and/or intangible.
One suggestion I give to my colleagues and clients is to map tangible and intangible value on a grid based on whom value is important to inside and outside of the company. And then come up with some metrics for each quadrant. I included some possible metrics in my blog in October. If you’re interested, I hope you’ll check out my post at http://innovation.fleishmanhillard.com/?p=957.
Thanks for writing this post. Hopefully it inspires people who may be unsure how to measure their innovation efforts to try!
Kathie
FH Innovation
What an excellent definition of innovation in the opening paragraph. And using this definition, a key metric becomes obvious — value added (VA). Since VA is a financial term it can be calculated from annual reports or internal accounting. For the past six years the UK government has published a ‘Value added scoreboard’ which caluclates VA for the top 800UK and 600 EU companies as well as other parameters based on VA. See http://www.innovation.gov.uk/value_added/ This gives a benchmark at the company level but also the basis for evaluating individual project proposals. So basically any company with a set of accounts can compare themselves with sector averages. This applies to large or small companies and is equally relevant for manufacturing and service companies. In our consulting work at the Business Innovation Group we generally use this measure as the starting point for discussions with CEOs on growth strategies through innovation.
I agree with the importance in measuring the innovation. I worked in a brazillian company thats focus on management innovation. Here in Brazil, we realized that all the questions and comments above are so important.
When we see a situation as incremental innovation, we think should use the same measuares of product/process development. However, when we deal with radical innovation the measure isnt a easy thing.
Traditionally, in Brazil its common replicate the same things that is using, mainly, in USA and Europe.
Last year we started a internal trainning for all people that work with us. We called MBI (Master Bussiness in Innovation). Its our definition for this kind of thing. The first class of our MBI we discuss metrics. In synthesis, we learned a lot about metrics and other factors in management innovation. If someone whant more details about the MBI class, please contact me: rafael.augusto@institutoinovacao.com.br
@Kathie – Thanks for pointing me to your post. Lots of great ideas for areas that can be measured, and the most important part is the advice to keep iterating until the right combination is found. There is no one size fits all silver bullet. Thanks!
@David – Thank you for the link to the Value Added Scoreboard. I will review it in more detail, but I like that it captures less traditional sources of value. These are the types of tools we need to start working with, modifying for specific circumstances. I would love to learn more about how you have applied it.
@Rafael – I would love to know more about the MBI, and how the ideas play out in practice. I will contact you to learn more. Thank you for bringing your work to our attention!
I Ma very interested in whether any of you have worked with the quality of innovation processes- the interactionss and relations, in thee different steps of an innovation process, and to which extent it is possible to define some guiding metrics-
If so I am very interested in hearing from you- worked on these aspects in relation to education systems- but want to think about it in other policy contexts
RD projects…