Fuzzy Metrics For Quantifying Innovation
Innovation, as with every initiative or process that companies want to track, needs to be measurable. If you can’t measure it, break it into measurable pieces. But innovation can be a messy process. While we are good with tracking the value and depreciation of hard assets, measuring the value of patents, soft skills, and innovation can become a challenge. Not least do balance sheets and accounting only reflect hard assets. Which makes it so difficult to measure innovation in companies and evaluate startups. Innovation projects and startups don’t follow the usual rules.
Companies have come up with a number of metrics that measure hard asset. But most of them can be deceiving. The sheer budget for the R&D department can be misleading and does not reflect how innovative a company is. Volkswagen is ranked as the top R&D spender, but where has this led the company?
But what about the more fuzzy ones? Like the Net Promoter Score that asks visitors of a site how likely they were to recommend the site to others. The willingness to take on or come up with risky projects is crucial as we have discussed in Micro-scale versus Macro-scale Innovation Evaluations. Innovation requires a smarter way of measuring and evaluating its success.
The objective to be more innovative can be broken down into how many ideas are generated, how many of them are executed on, and how many of them make it into a product. That requires a definition what counts as an idea, and where do we track the number of ideas.
To measure the monetary impact of innovation, new questions can be formulated in form of key results, such as how much money do we spend on the overall idea generation process, how much money and time is spent for ideas that we execute on in every stage, and what did product development cost in total. Once we know that this can be mapped into how much profit do we make from new products? And what counts as new product (came to the market or on the price list within the last 12 months)?
Innovation Metric Matrix
Innovation doesn’t just happen on a product or service level, but also on a number of other dimensions. They include brand, channel, business/profitability model, network, structure, process, and/or customer engagement. We are looking at three types of metrics: some basic metrics, economic metrics, and then some fuzzy metrics that are less tangible and go to new shores.
Basic Metrics (Type 1)
When we start from the basics, we need to consider input metrics such as
- Number of ideas (submitted to an innovation management system)
- Number of ideas entered into hackathons and innovation contests
- Number of ideas pitched to internal idea scouts
- Number/ratio of ideas that move upwards to the next innovation stages
- Number of ideas per department
Those can be easily gamed and do not more than form a basis for the next set of metrics, where they are being enriched and put into relation to other factors.
Economic Metrics (Type 2)
Going beyond absolutes, economic metrics put them in relation to other factors. Not one single metric is *the* most valid one, over time it will change on which metric companies need to focus. In the following list we look at 2 dimensions: do they measure external or internal metrics, and are they past/present or future oriented?
- External / Past focused
- Innovation hit rate for initiatives across the portfolio (# of them that return their cost of capital)
- Economic value created by innovation
- Firm’s Net Promoter Score (Wikipedia) (NPS)
- Customer Satisfaction (Wikipedia)
- Brand perception – for both the individual initiatives and the overall portfolio
- # of positive media and analyst mentions the firm receives regarding innovation
- External / Future focused
- Economic value estimates (EVE) of individual initiatives and the overall portfolio
- Percentage of initiatives in the portfolio that are clearly motivated by compelling customer insights
- Percentage of initiatives in the portfolio that involve co-creation with suppliers, customers, or partners
- Percentage of innovation costs borne by you versus co-creators – for individual initiatives and the innovation portfolio
- Percentage of revenue from new products
- Innovation Premium
- Internal / Past focused
- Net Present Value (Wikipedia) (NPV) created by innovation
- Growth in NPV across the innovation portfolio
- Percentage of innovation initiatives taken to market (aka idea transfer rate)
- Percentage of innovations that have survived three or more years in the market versus the number of initiatives in the portfolio
- Revenue generated by innovation initiatives
- Time to profit
- Internal / Future focused
- NPV estimates across initiatives and portfolio
- Growth in project NPV estimates across the portfolio
- Size, speed, and efficiency of the pipeline across the portfolio
- Degree of alignment of the innovation portfolio with other strategic growth platforms
- Ratio of incremental to game-changing innovation in the portfolio, measured in the number of initiatives and/or expenditures
Fuzzy Metrics (Type 3)
Still, those numbers do still focus too much on hard assets. That’s why this third type of metrics plays with the soft and barely tangible ones. The tricky thing is that the underlying factors for building those metrics are shifting.
Those metrics build on some of the criteria that scientific literature has identified as creative and innovative behaviors.
Fuzzy Metric 1: Willingness to take risky projects
What is the percentage of people willing to take on a risky project given a predicted success rate?
Fuzzy Metric 2: Willingness to promote to take risky projects
Similar to the NPS, would employees promote taking risky projects to their colleagues?
Fuzzy Metric 3: Rate of covering the adjacent possible
The adjacent possible is a concept that says that once certain building blocks are here, it’s just a matter of time that somebody combines them to build something new. That’s also a reason why unrelated innovators are often coming up with the same idea within the same time period.
This measure would take all relevant building blocks and combine them in a directed way until one of them sticks. When Thomas Edison tried a combination of thousand materials to find the first working light bulb, he proceeded exactly like that. By calculating all combination possibilities and working the diligently through them can be an indication of progress.
Fuzzy Metric 4: Network-Nodes / Degrees of Separation / Power-nodes vs Intrapreneurs
Based on who in an organization is in contact with whom and identifying nodes that are part of many innovation projects, and how far the departments are from each other can indicate where the most novel and original ideas can come from. Network-nodes that do not allow the connection between disparate departments are less likely to lead to original ideas.
Fuzzy Metric 5: Cross-Team Collaboration
How much communication and collaboration happens across team boundaries? Is this number low or high? The higher, the more likely that original ideas are generated.
Fuzzy Metric 6: Returns on Collision
This concept is based on how likely it is that people from different group bump into each other. Pixar uses a building design where all facilities such as cafeteria, bathrooms, snacks, seating spaces etc. are arranged in a central lobby. This allows collisions (in a positive sense) of teams and people, that increases the chance of an exchange of ideas that may lead to an innovative idea.
These are just a few of new and fuzzy innovation metrics that allow to get a clearer picture of innovative organizations. Innovation metrics cannot be just based on the hard skills and hard assets, but must include the factors that science has identified as required elements for creativity and innovation.