Improving objectiveness of idea selection

With the increase in use of online ideation platforms to harness creativity, it is important that ideas are fairly judged, without bias, and that the truly best ones bubble up to the top, comments Phil Kennedy who has reviewed the following paper for R&D Today.

Persuasion in Corporate Idea Contests: The Moderating Role of Content Scarcity on Decision-Making Tobias Kruft , Christoph Tilsner, Andreas Schindler, and Alexander Kock

The authors contend that, “Finding indicators for idea success is a major goal of idea evaluation research. In order to do so, an understanding about how issue-irrelevant information affects an evaluation team’s decision-making process is of high importance since persuasion might lead evaluators to select certain ideas over others.”

This paper explores the phenomenon of ‘content scarcity’ for ideas – submitters not giving enough information about the idea in the submission – and how that gives rise to those ideas being unfairly judged as assessors fall back on ‘issue-irrelevant’ information around which to make their decision. ‘Issue-irrelevant’ refers to information is such as the originator of the idea, personal distance to authors, the idea message’s sentiments, the writing style, the community’s attention to an idea in the form of comments or votes, and the source credibility.

In such cases, evaluators may make a more intuitive, rather than reflective, choice. In a nutshell, if evaluation teams do not have sufficient issue-relevant information, they seek issue-irrelevant information to close the gap and make a decision. But even if issue-relevant information is available, issue-irrelevant information can bias evaluation teams to make an intuitive decision rather than a reflective one.

What are take-aways for practitioners?

Idea descriptions should not be too short; otherwise evaluators tend to supplement the scarce content with even more intuitive information.
Ideators may omit information as they think it is obvious. Encourage them to more fully explain their ideas and not assume the evaluators will correctly fill in the information gaps.
Instead of providing too much text-based content, ideators should consider providing visualisations of the content, which enable evaluators to more quickly and effectively assimilate the gist of the idea.
The persuasive issue-irrelevant aspects of the ideator, message, and community – the three dimensions – increase the probability that an idea may be selected with bias (see Yale attitude change approach). Content scarcity strengthens the relationships between the persuasive aspects and idea selection. This may be further compounded by having too many ideas (crowding), and/or lack of evaluator resource or time and/or evaluator expertise.
Reflective decisions from the evaluators are what are needed. These can be promoted by suppressing the issue-irrelevant information. A central strategy could, for example, be to deliberately prevent the evaluators from seeing issue-irrelevant information. This strategy works particularly well with the ideator dimension. If evaluators do not know who the ideators are, they cannot derive information about their reputation and dedication.
Managers should try to give all the ideators the same opportunity to persuade the evaluator through the message dimension (the idea). For example, campaign managers should define the campaign search fields more clearly.
Evaluators should transparently and in advance communicate their decision criteria in order to align the ideators to the same goals.
Platform managers could also provide ideators with predefined templates that allow them to address specific aspects of their idea, thus enhancing the ideas’ assessment and explicitness. However, such templates could be an obstacle to ideators posting ideas. Consequently, this work suggests that the initial idea should be posted form-free on the platform, but that the final template fields must be filled out at the end of the campaign. Ideators should thereafter be encouraged to improve their idea in exchange with the community and fill out the templates consecutively.
Platform managers should consider incentivising comments, which contribute to the idea with the help of gamification elements, while also incentivising ideators to actively improve the idea accordingly. Then, however, the comments and votes should not be visible to the evaluators, as they distract from the actual idea’s content.
Managers should offer ideators training that explains the importance of issue-relevant information.
Each ideator should write an idea description of at least 170 well-considered words, or perhaps longer. However, it is important for ideators to understand that more content is not always better; content needs to add to the idea, not submerge it, and evaluators may have limited time to review ideas.

In conclusion

Overall, this work presents certain simple strategies for reducing bias from ‘issue-irrelevant’ content when evaluating ideas posted on an online ideation platform. These are derived from a detailed analysis, based on a data set of 3025 ideas from 227 campaigns on a globally operated corporate online ideation platform, as well as consideration of much past work in the field of decision analysis and persuasion.

The paper: Persuasion in Corporate Idea Contests: The Moderating Role of Content Scarcity on Decision-Making Tobias Kruft , Christoph Tilsner, Andreas Schindler, and Alexander Kock J PROD INNOV MANAG 2019;36(5):560–585  © 2019 Product Development & Management Association DOI: 10.1111/jpim.12502