Final post of the 6-part series, Modeling for metascientists (and other interesting people).
Poor old psychology: looking silly again.
It wanted to be seen as a real science, hypothetico-deductive model and all. First it looked a bit silly when most of the ‘effects’ that constituted the alleged support for its theories turned out not to replicate. And now it’s looking silly for a second time, as people point out that those theories themselves are remarkably weak (actually this problem has been known from much longer, since before the beginning of the current replication crisis).¹
It’s often not clear what, precisely…
Part 5 of the 6-part series, Modeling for metascientists (and other interesting people).
Hopefully you are now convinced that formal models are useful tools to help scientists understand the world. As an anthropologist with training in evolutionary biology who started her career as a modeler, I chose to do mathematical modeling because models have the following three advantages:
Part 4 of the 6-part series, Modeling for metascientists (and other interesting people).
Many social scientists do not consider formal models a useful tool in theory construction. They might defend this position by arguing that “formal models make assumptions that are too simplistic, too unrealistic, and too arbitrary.” Instead, they rely on verbal models.
In turn, formal modelers complain about the ambiguities associated with verbal models, including imprecise constructs, implicit assumptions, and predictions based on intuitive reasoning rather than deduction.
In disciplines where verbal models dominate, theoretical ambiguity is widespread. This raises some uncomfortable questions: Don’t scientists notice these ambiguities…
In Part 1, I covered the misconception that models are mainly useful if you want to generate precise predictions. In fact, models have many uses, a key one being that models promote transparency.
In Part 2, I covered the misconception that models must be realistic to be useful. In fact, models are unrealistic by design: just like maps, models leave out details to provide a clearer picture of the things that really matter, and different types of maps (models) are useful for different purposes.¹ ²
In part 3…
Part 2 of the 6-part series, Modeling for metascientists (and other interesting people).
The simple observation that most models are oversimplified, approximate, incomplete, and in other ways false gives little reason for using them. Their widespread use suggests that there must be other reasons.
- William Wimsatt¹
A common criticism of models is that they are unrealistic. I’ve had more than a handful of conversations where a model was criticized for this reason. A typical conversation might go something like:
Hey — did you see the recent PNAS model about how rewarding replications doesn’t actually incentivize better research?
I often hear the argument that theories in psychology are immature because they don’t make “risky” predictions (that is, predictions that are likely, given the theory, but unlikely, absent the theory). So, the argument goes, we need formal models to help generate precise predictions, such as a range of values or a smallest effect size of interest.¹
But formal models have many uses besides prediction.² Off the top of his head, Joshua Epstein lists 16 reasons (other than prediction) to build models, including developing causal explanations…
This is a series about modeling. It is mainly meant for metascientists in the social sciences. With a bit of luck, it will also be useful to other interesting people.
Part 1: Models are for transparency. (Leo Tiokhin)
Part 2: Models are unrealistic by design. (Leo Tiokhin)
Part 3: Five modeling misconceptions. (Leo Tiokhin)
Metascientists are becoming increasingly interested in developing theory, particularly…
The beauty and the tragedy of the modern world is that it eliminates many situations that require people to demonstrate a commitment to the collective good.
- Sebastian Junger¹
Imagine two scientists, Kotrina and Amber, who have just obtained their PhDs and are entering the job market.
Kotrina has four empirical papers. She is first author on two, including a publication in a prominent psychology journal, Journal of Experimental Psychology: General. Kotrina has 75 citations, with two papers cited 25 times each—not bad for a newly-minted PhD. …
New blog post:
Provides some reflections on our recent paper:
Tiokhin, L., Yan, M. & Morgan, T.J.H. Competition for priority harms the reliability of science, but reforms can help. Nat Hum Behav (2021). https://doi.org/10.1038/s41562-020-01040-1
Update 2021: This post now has a DOI: http://dx.doi.org/10.13140/RG.2.2.22893.51682
Co-written with Noah van Dongen.
Our field is meta-research: the study of the research process itself. When we first got into this field, we both remember reading Munafo et al.’s “A manifesto for reproducible science” and Chambers’ “The seven deadly sins of psychology: a manifesto for reforming the culture of scientific practice” and feeling inspired. “Yes, goddamnit — these people get it!”
A few years later, while browsing Twitter, one of us saw the preprint, “A Manifesto for Team Science”. This time though, the feeling was different. “Really? Another manifesto?” …