Selection Regimes and Selection Errors

doi:10.1287/orsc.2023.17482

Selection Regimes and Selection Errors

2025-05-14

0

PRE

AI

DOI：10.1287/orsc.2023.17482

Original

Original request for help

Save

Abstract

En 中文

How can the selection of innovation projects be designed to reduce false positives and false negatives? Prior research has provided theoretical insights into organizing to reduce errors, yet we know little about how organizations adapt selection over time and the effects of this on selection outcomes. Drawing from qualitative data from 126 interviews conducted over several years, we explore how an accelerator evolved through three selection regimes for high-stakes funding decisions, focusing on the organizational changes and their underlying reasons. We then analyze quantitative data from all 3,580 submissions they received, assessing false positives and false negatives across these regimes. Our findings reveal a persistent occurrence of both types of errors, with relatively small differences across the regimes despite deliberate efforts to enhance the process. In the final regime, which increased submission quality by emphasizing applicant track record and adding additional layers of screening, evaluators surprisingly became more prone to making selection errors. This finding stands net of accounting for (1) differences in the pool of submissions, (2) differences in treatment effects through training and resources provided, (3) learning, and (4) market evolution. By combining qualitative and quantitative data, we explain this through two mechanisms: (1) mean reversion in combination with increased emphasis on applicant track record and (2) within-type adverse selection enabled by a more stringent selection process. The study reveals that evolving an organization's selection regime may require adjustments across multiple aspects, resulting in unintended consequences.

Keywords:

organization design

ideas

innovation

evaluation and selection of innovation projects

screening

selection error false positives and false negatives

mixed methods

longitudinal research design

accelerator

app

Selection Regimes and Selection Errors

Abstract

Journal

Organization

Cited Papers