Skip to content
Generic filters
Exact matches only
Search in title
Search in content
Search in excerpt
Filter by Article Type
Papers
Events
Tools
Funding Articles
Case Studies
Resources
Opportunities
Theme Editor Blogs
Filter by Categories
Business model innovation
Ideation and creativity in R&D
Latest news
Managing international R&D
Managing technology platforms
Managing the R&D pipeline
Open innovation
Outsourcing R&D
Project valuation and selection
R&D strategy
Roadmapping
Stage gate processes
Technology intelligence
  • Home
  • About
    • About R&D Today
    • Contributors
    • R&D Publications
    • R&D Today newsletter archive
  • Themes
    • R&D Management
      • Rationale for key themes
      • Ideation and creativity in R&D
      • Managing international R&D
      • Managing the R&D pipeline
      • Open Innovation
      • Roadmapping
      • Technology Strategy
    • Special Features
      • Innovation for a Sustainable Future
      • How to measure the value created by innovation
      • Dynamic capabilities for strategic innovation
      • Would a ‘Strategy Lab’ provide sustainable renewal of competitive advantage?
      • Design Thinking
      • China’s new model for Open Innovation
      • Penetrating the fog of Agile
      • The resurgence of frugal innovation
      • Impact of digital technologies
    • Key Conference Tracks
      • Business model innovation
      • Entrepreneurial Ecosystems, Innovation Ecosystems and platforms
      • Intellectual Property Rights
      • Sustainable Innovation
    • Innovation Leadership
  • RADMA
    • About RADMA
    • R&D Project Exchange
    • Celebrating 40 Years of RADMA
    • RADMA Scholars
    • R&D Management Journal
  • The Pentathlon Framework
    • Strategy
    • Ideas
    • Selection & Prioritisation
    • Implementation
    • People & Organisations
  • Knowledge Hub
  • R&D Management Conference
  • Events
    • Upcoming events
    • Events Archive
  • Contact
  • Home
  • About
    • About R&D Today
    • Contributors
    • R&D Publications
    • R&D Today newsletter archive
  • Themes
    • R&D Management
      • Rationale for key themes
      • Ideation and creativity in R&D
      • Managing international R&D
      • Managing the R&D pipeline
      • Open Innovation
      • Roadmapping
      • Technology Strategy
    • Special Features
      • Innovation for a Sustainable Future
      • How to measure the value created by innovation
      • Dynamic capabilities for strategic innovation
      • Would a ‘Strategy Lab’ provide sustainable renewal of competitive advantage?
      • Design Thinking
      • China’s new model for Open Innovation
      • Penetrating the fog of Agile
      • The resurgence of frugal innovation
      • Impact of digital technologies
    • Key Conference Tracks
      • Business model innovation
      • Entrepreneurial Ecosystems, Innovation Ecosystems and platforms
      • Intellectual Property Rights
      • Sustainable Innovation
    • Innovation Leadership
  • RADMA
    • About RADMA
    • R&D Project Exchange
    • Celebrating 40 Years of RADMA
    • RADMA Scholars
    • R&D Management Journal
  • The Pentathlon Framework
    • Strategy
    • Ideas
    • Selection & Prioritisation
    • Implementation
    • People & Organisations
  • Knowledge Hub
  • R&D Management Conference
  • Events
    • Upcoming events
    • Events Archive
  • Contact

Menu

Assessing innovation and technology management capabilities in an organisation

To extract best value from investments in technology and innovation, managers need to address a wide range of aspects. Tools to assist this process can also provide insights into performance trends and identify issues commonly faced across the sector.

The emerging ISO standards for innovation management, provide an opportunity for developing these types of tools.  In this study, researchers developed an Innovation Technology Management (ITM) process framework based on a set of 87 question. The survey covers three themes and ten areas of significance to technology-intensive organisations and was successfully applied to the oil and gas sector.

Assessing Innovation Technology Management performance

There are a number of innovation assessment approaches that are based on self-assessment, some of which are briefly described below as an illustration of the range available.

  • IMP3rove was established by the European Commission with the aim to improve innovation management performance of SMEs in Europe with sustainable impact. It covers innovation strategy, organisation, culture and innovation processes, including enabling factors. The online self-assessment tool supports and drives the process by generating reports on SME performance, developing insights into strengths and weaknesses, and monitoring progress and performance.
  • Innovate! New developed by CAU in Germany and other academic partners, is a benchmarking methodology based on an innovation excellence model, which has three central areas leadership & governance, innovation performance, process and projects in the context of six areas: strategy, organisation, launch, development, front end and culture.
  • The Innovation Management Maturity Assessment offered by CIMS, North Carolina State University, is a tool to assess a company’s level of innovation management proficiency. The assessment looks at self-assessed strengths and weaknesses across the organisation. By department, or throughout the entire enterprise, individuals rate their organisation’s performance across five core competences (strategy, organisation & culture, processes, tools & techniques, metrics) and five management dimensions of innovation (ideas, market, portfolio, platform and project management).

 ITM tools and toolkits

Understanding which tools to use, in what combination, requires an understanding of the particular organisational context, and the effectiveness of management practices in order to focus improvement efforts. So, the process of developing ITM tools and toolkits requires active collaboration with industry and working with organisations on real management problems, due to the deeply context-specific situations they present.

The technology management assessment procedure (TMAP) was developed to assess technology management processes in manufacturing firms, it is based on Gregory’s (1995) process model for technology management, which consists of five generic process areas.

Figure 1 - Five-process technology management framework, showing example activities
Figure 1 – Five-process technology management framework, showing example activities

The technology management framework consists of five generic process areas:

  1. Identification of technologies which are (or may be) of importance to the business. Example processes include scanning, monitoring, benchmarking and data collection.
  2. Selection of technologies that should be supported by the organisation. For instance, forecasting, portfolio analysis and scenario analysis are associated with selection of technology.
  3. Acquisition and assimilation of selected technologies. Example processes include technology transfer, research and development, corporate mergers and acquisitions.
  4. Exploitation of technologies to generate profit, or other benefits. Example processes include licensing, new product development, incremental developments, process improvements, and supply chain management.
  5. Protection of knowledge and expertise embedded in products and manufacturing systems. These processes include patenting, risk assessment, security management and staff retention.

The aims of the TMAP assessment were to 1) provide a framework for linking technology with business needs, identify critical technology management issues in the firm; 2) map and evaluate important technology management processes in the area of interest; 3) identify strengths and weaknesses, and hence areas of best practice for transfer, and areas for improvement; and 4) lead to recommendations for action plans which have cross-functional support and can be clearly justified.

Innovation and technology management assessment (ITM)

The paper presents a robust approach for assessing the innovation and technology management (ITM) capabilities within technology-based organisations so that focused improvements can be made, and appropriate interventions or solutions can be developed.

Technology management

Phaal et al.’s (2004) technology management framework (Figure below), is an advancement on Gregory’s (1995) because makes it clear that technology management decisions are often made in business areas such as strategy formulation, product development and in operational process, rather than discrete technology management processes.

Figure 2 - Technology Management framework (Phaal et al, 2004)
Figure 2 – Technology Management framework (Phaal et al, 2004)

 

Innovation Management

In addition, Goffin and Mitchell, identified the main elements of innovation management succinctly as follows:

  • Innovation strategy: which, in addition to other things, should capture the role of technology and the opportunities it can open up.
  • The development funnel: made up of ideation, prioritisation and implementation – a process model reflecting Wheelwright & Clarke’s innovation funnel.
  • People and organisation: which relates to creation of a culture of innovation. Looking back into literature compiled by Adams et al. (2006), one would identify organisational culture, management commitment and leadership as being part of people and organisation.

Thus, the inextricable nature of technology and innovation justifies the need for a framework that explicitly links and combines elements of Phaal et al.’s (2004) technology management framework and Goffin and Mitchell’s (2004) innovation management models.

ITM Framework has three themes

In summary, the innovation and technology management framework can be described as covering three themes and ten areas of significance to technology-based organisations:

Strategy: investigating the innovation and technology strategy and importantly, how it links and supports the organisation’s overall business strategy

  • Business strategy: is it sufficiently defined to enable ITM strategy (and other functional strategies) to be developed?
  • Innovation and Technology strategy: is there a clear statement in terms of competitive positioning and strategic direction in terms of what is required from the innovation and technology system?

ITM System: examining the fundamental technology and innovation management processes of technology identification, selection, acquisition, exploitation and protection;

  • Identification: how do we identify product or process technologies that are or may be of importance to the business?
  • Selection: how do we know which of these technologies are most appropriate to deliver business and ITM strategies?
  • Acquisition: how do we successfully acquire, develop and assimilate selected technologies?
  • Exploitation: how do we generate benefits (e.g. licensing, new products, process improvements etc.) from the technologies?
  • Protection: what strategies and processes are in place to ensure that our technology knowledge and capabilities, embedded in our products, processes and systems, are appropriately protected and denied to competitors?

People and Organisation: focusing on the organisation’s leadership and management style, organisational culture and the management of competences, and their appropriateness for desired ITM and, ultimately, business performance.

  • Leadership: how do we set and communicate clear objectives, manage interfaces and structures within the organisation and empower our personnel?
  • Organisational culture: does ‘the way we do things’ help or hinder the achievement of our business and ITM objectives?
  • Competences: do we have the skills and capabilities to deliver projects and manage our technology and innovation activities?

Development of ITM assessment

The 97 questions that made up the first version of the ITM were compiled by rationalisation of a total of over 400 questions/statements compiled from the various assessments, followed by selection of those considered most appropriate.

Following initial tests with organisations, the ITM assessment was revised to improve its usefulness, feasibility and usability and the resulting 87 questions spread across the ten areas of the ITM framework and correlated to a five-scale maturity model.

Maturity model 

The maturity model is based on Bessant et al.’s (2005) levels of organisation absorptive capacity. This gives an indication of the knowledge or capability level of the organisation with respect to the issue being explored by the respective question.

Table 1 - ITM Assessment maturity levels
Table 1 – ITM Assessment maturity levels

Application: ITM Assessment survey of oil and gas sector

ITM assessment was applied to a selected group of oil and gas exploration and service companies, to provide an industry-wide assessment of a sector. It was found to give a preliminary indication of performance trends and issues which are commonly faced across the energy sector.

Figure 3 - Roadmapping-based workshop format for TIM Assessment deployment
Roadmapping-based workshop format for TIM Assessment deployment

Key trends

An arithmetic mean of the responses revealed some key trends in the oil and gas industry.

The highest variation in the average maturity levels is in the ITM System – Protection category with average scores ranging from 1.2 (Company 2) to 5.0 (Company 7). At the sector-level, the average maturity level is similar for nine of the ten ITM categories. The one notable exception is the People & Organisation – Competences category with the arithmetic mean average of 2.6 for the 15 organisations that completed the assessment. The lower maturity score for the competences area is perhaps unsurprising given the recent decline of the industry with rock bottom oil prices triggering significant staffing cuts worldwide starting in 2015.

Table 2 - Summary of selected findings in oil & gas survey
Summary of selected findings in oil & gas survey

The analysis of responses to date has revealed some key trends in the oil and gas industry and these average maturity levels are summarised for each assessment category. It also offers the potential for benchmarking against peers, higher scoring average maturity levels reveal promising trends while lower average levels reflect issues that may need attention.

Table 4 - Selected findings in oil & gas survey - Higher scoring questions
Table 4 – Selected findings in oil & gas survey – Higher scoring questions

Positive feedback from the questionnaire shows a focus on project execution, particularly with respect to new product development, the emphasis on supply chain performance and awareness of the value of intellectual property.

Existing successful products form the backbone of all their engineering development. Building on these allows the industry to keep their existing customer base happy with upgrades and satisfies new customers coming in, in terms of incremental improvement in products. Moreover, it appears there are well defined processes and procedures in place to protect company’s intellectual property as most companies seek to maintain and dominate a market.

The data shows that the surveyed organisations have very good customer relationships. Field personnel not only talk to customers about their needs, but also use their knowledge to predict what the customer will need. Combining these insights with good business intelligence and analysis provides these companies with a solid basis for creating products with superior value.

Key learning points 

There are several key observations from this work:

  • In complex areas such as innovation and technology management, time efficient and tangible approaches to support managers in formulating their own action plans are welcomed by industry
  • Easy to access survey interfaces can enable companies to select where they wish to go into greater depth although they do not replace workshop discussions and face to face interactions
  • Companies are interested in comparing themselves to others in the same industry as a driver for performance improvement
  • Feedback from industry is a crucial input for the development and refinement of management tools such as the ITM survey
  • Building up a wider view of survey outputs could guide future research into building effective toolkits for industry

Paper submitted to: R&D Management Conference 2018 “R&Designing Innovation: Transformational Challenges for Organizations and Society” Imoh Ilevbare, Clare Farrukh, John Saiz, Rob Phaal and Nii Ahele Nuno

Download the paper from ifm.eng.cam.ac.uk

Summary of other innovation and technology management-related assessments:

Speeding New Products to Market (Gardiner et al. 1998),

Technology Management Assessment Procedure (Farrukh et al. 2000),

Innovation Management (Pentathlon Framework) assessment (Goffin & Mitchell, 2010)

Design Audit tool (Moultrie et al. 2007) which highlighted multiple facets of technology and innovation management.

Related reading

Bessant, J., Phelps, B., & Adams, R. 2005. External Knowledge: A Review of the Literature Addressing the Role of External Knowledge and Expertise at Key Stages of Business Growth and Development: Final Report. Advanced Institute of Management Research.

Griffin, A. and A.L.Page (1993), An interim report on measuring product development success and failure, Journal of Product Innovation Management 10(4): 291-308.

Goffin, Keith, and Rick Mitchell. Innovation Management: Effective Strategy and Implementation. Third ed., Palgrave Macmillan, 2017.

Ilevbare I.M., Dusch B. & Templeton P.W. 2016. “A framework and methodology for developing decision support tools for business and policy.” Institute for Manufacturing, University of Cambridge.

ISO/CD 50501 (Under development): Innovation management System – Guidance.

ISO/NP TR 50502 (Under development): Innovation management – Assessments – Guidance.

Kerr, C., Farrukh, C., Phaal, R. and Probert, D. 2013. “Key principles for developing industrially relevant strategic technology management toolkits.” Technological Forecasting and Social Change, 80:1050-1070.

Mitchell, R., Phaal, R., Athanassopoulou, N., 2014. “Scoring Methods for prioritizing and selecting innovation projects”, Proceedings of Portland International Conference in Management of Engineering & Technology (PICMET)’14: Infrastructure and Service Integration.

Mortara, L., Phaal, R., Kerr, C., Farrukh, C. and Probert, D. 2014. “Tool Finger printing: Characterising Management Tools.” Proceedings of the Portland International Conference in Management of Engineering & Technology (PICMET), Kanazawa, 27-31 July (907-920). IEEE.

  • 29 September 2020
View our newsletter archive
  • Related posts

    • Improving the assessment of innovation and technology management capabilities
    • Must read papers on aspects of R&D Management
    • Direct coaching supports SMEs with inbound knowledge transfer from university
    • Portfolio management tools and approaches
    • Failure Mode and Effects Analysis (FMEA)
    • Business Appraisal of Technology Potentials – development of a practical approach to valuing technology
    • A review of TRIZ its benefits and challenges in practice
    • Sustaining roadmapping in commercial organisations – Subsea 7 case-study
    • How to design an Innovation Challenge – learning points and a great Canvas tool
    • Product technology roadmapping – a visual framework for ‘sense making’ in a changing world
    • Tool Fingerprinting: Characterising Management Tools
    • Integrating sustainable value thinking into technology forecasting: a configurable toolset
  • Have your say

    Have your say / Follow us

    Linkedin Soundcloud Twitter Youtube Linkedin-in

    R&D Today is the outreach site for the Research and Development Management Association, a charitable organisation that supports research, best practice and innovation.  www.radma.net

    Click here to sign up to our newsletter, and
     click here to view our newsletter archive.

    © Copyright R&D Today

    2025.

    All rights reserved.