Why indirect contributions matter for science and scientists

Leo Tiokhin, Karthik Panchanathan, Paul Smaldino, Daniel Lakens

The beauty and the tragedy of the modern world is that it eliminates many situations that require people to demonstrate a commitment to the collective good.

- Sebastian Junger¹

Imagine two scientists, Kotrina and Amber, who have just obtained their PhDs and are entering the job market.

Is choosing the best scientist that simple?

Now imagine that you talk to colleagues and learn a bit more about each candidate.

Typical evaluation criteria ignore indirect effects

Of course, Kotrina and Amber are caricatures—real differences between candidates are rarely so clear cut. Yet, their tale is useful because it illustrates the two pathways by which scientists contribute to science: directly and indirectly.

The repercussions of ignoring indirect effects

The thing is, ignoring indirect effects does have serious consequences. Consider three:

Ignoring indirect effects fails to reward scientists who help others and fails to penalize scientists who harm others.

Consider an extreme case in which a scientist generates little direct output, such as not having any first-authored publications. Given current evaluation criteria, such a scientist would struggle to find a research position, get grants, and receive awards. Would this be justified?

Ignoring indirect effects increases the intensity of competition between individual scientists

Ignoring indirect effects increases the intensity of individual-level competition by reducing the “stake” that scientists have in the outcomes of other scientists. In biology, this is well established: evolutionary mechanisms that cause individuals to have a stake in each other’s outcomes (such as relatedness) result in a “shared fate,” reducing individual-level competition and promoting cooperation in many cases.⁷ ⁸

Ignoring indirect effects reduces the incentive to specialize in unique skills that complement others.

A focus on direct, individual contributions using a narrow set of metrics, such as first-authored papers, creates an additional problem: scientists have fewer incentives to specialize in roles that are not rewarded by the prevailing regime, even if these roles are essential for science.

Accounting for indirect effects: lessons from animal husbandry and professional sports

How should we approach the problem of accounting for indirect effects in scientific evaluation?

Animal husbandry

Just as we seek to select scientists to improve scientific outcomes, in domesticated livestock, breeders seek to select animals in a way that maximizes the amount of some commodity.

Professional sports

Sports team managers must evaluate which players have the largest positive effect on team performance. Superstar players with impressive individual performances (such as scoring many points) might seem like the natural choice.²¹ But superstars aren’t always the players that have the largest positive impact on a team, which is why evaluations of professional athletes rely on metrics that capture indirect effects.²²

Changing the level of selection: an overarching principle to account for indirect effects

Animal husbandry and professional sports illustrate an overarching principle for improving group-level outcomes: accounting for indirect effects by changing the level of selection, from lower levels (individuals) to higher ones (teams).

Just change the level of selection. Easy, right?

Nothing is easy in war.

- Dwight D. Eisenhower²⁷

Changing the level of selection has significant potential to improve science. It works in professional sports and animal husbandry; and throughout evolutionary history, in cases where natural selection at higher levels has dominated selection at lower ones, the resulting “superorganisms” (eukaryotic cells, eusocial insects) became so ecologically dominant that many other species had little hope.²⁸

Postdoc at Eindhoven University of Technology, researching how to make science more efficient and reliable. Metascience | Incentives | Evolutionary Theory.