Measuring the effectiveness of R&D can be a challenge. Larry Schwartz, Founder of IP Business-Tech Solutions LLC, led an investigation by the Industrial Research Institute’s research-on-research working group to looked at how managers define R&D effectiveness and what metrics they use to measure it.  We asked about this work.

Q&A with Larry Schwartz

From your work could you recommend a tool or technique that is useful for improving the effectiveness of R&D management? 

I would suggest Patent Landscaping tools, of which a good example is Patent Inspiration www.patentinspiration.com. Before starting a project, you need to understand the breadth of a technology, who are the key players, key inventors, how fast the landscape is changing, what are the key patents (most cited)?

Are there areas of the landscape less populated that would allow an easier entry point? If the landscape is too crowded what are options (license technologies, hire key inventors, partner, invent around)?

What would you recommend someone new to this subject reads? 

It is important not just to read technical articles, but also articles on techniques of how to improve the R&D process, for example how can you improve your cycle time, integration of new product development phase gate process, metrics, continuous improvement (a good source is Research-Technology Management)

Looking back on your career what advice would you give your younger self about R&D?

No matter how good an idea sounds, it is essential that you understand what it will take to make it successful. This includes time, money, buy-in from the organization, people, skills necessary (not just scientists) but also marketing, sales, support staff. You must get buy-in from many areas.

Be realistic on your estimates, it always takes at least twice as long to complete a project than you think!

Review of Larry Schwartz’s paper “Measuring the effectiveness of R&D”

Measuring the effectiveness of R&D can be a challenge. This article outlines the results of an investigation by the Industrial Research Institute’s research-on-research working group Measuring the Effectiveness of R&D which looked at how managers define R&D effectiveness and what metrics they use to measure it.

The investigation revealed that whilst the top three metrics had remained the same for a number of years there were significant differences in metrics used depending on the industry type. It also revealed issues with metrics in general and the need for new metrics to meet the challenging R&D environment.

The IRI’s working group reviewed the literature which was available on R&D metrics and then polled members on the metrics they used. The group then then attempted to evaluate whether metric choices were dependent on the type or industry of the organisation. Using a modernised version of a model created by a previous research-on-research group, ‘The Technology Value Pyramid’, the group:

• Classified and organised existing metrics;
• Provided a guide for choosing relevant, useful metrics;
• Identified gaps in available metrics.

The authors report that enhanced importance of reliable metrics is being driven by several forces: the need to justify the investment in R&D to senior management, the desire to improve efficiency in the use of R&D resources and the need for a means to estimate the value of the R&D investment for the future growth of the company.

As the R&D model is changing from exclusively internal development to an open innovation perspective, metrics depicting these changes, and measuring their outcomes, will be required. Since R&D will continue to evolve, identifying and integrating new metrics is a continuous process.

Taken from Measuring the effectiveness of R&D by Lawrence Schwartz, Roger Miller, Daniel Plummer and Alan R Fusfeld, published in Research Technology Management September-October 2011.

Biography

Larry Schwartz is the Founder of IP Business-Tech Solutions LLC which uses IP and business analysis to understand the landscape around a business area or technology to support informed business decisions. He has strong domain experience in technical management, business development and intellectual property.

Larry specialises in business development, R&D link to business strategic direction, competitive analysis including determining competitors’ business/IP strategy, technical and business consulting and materials science. He is a Subject Matter Expert for the Industrial Research Institute (best practices for R&D organisations) and NSF SBIR panel reviewer. He has authored multiple articles on sustainability and new product development.