Why the crisis of academic credibility may not have a solution

Academia faces a deepening crisis of credibility. Problems emerged at the time of the replication crisis – a crisis, beginning in psychology, in which large amounts of academic papers could not be replicated – yet recent developments have confirmed this trend. Single datasets seem to show ambiguous findings. In a 2018 paper, 29 expert teams used a single dataset to analyze whether football referees were more likely to red card dark-skin-toned players, the teams coming to radically different results, none of which could be proved incorrect. Recently, other studies have suggested that researchers overestimate the effect of the variables which they investigate. In short, we have fewer reasons to trust academic output, the social sciences being particularly affected.

For as long as these problems have existed, people have identified solutions. Writing in the Guardian last week, Stuart Ritchie proposed getting rid of scientific papers, replacing them with interactive notebooks on which scholars report findings interactively and eliminating the problem of ‘cherry picking’ of interesting findings. This argument seems excessively bold, but others propose more reasonable steps. Such measures include the registration of hypotheses prior to data analysis, an increasingly common practice among quantitative social scientists which reduces cherry picking, and the publication of null hypotheses and replications, these practices remaining rarer. More broadly, there have been calls for greater collaboration at field-level, consultative exercises potentially identifying areas for improvement. Ironically, the football study was an excellent example of collaboration.

Many of these steps are sensible, potentially improving the quality of research. Pace certain social constructivists, most do not think that empirical research is beyond saving. Some measures are comparatively easy to implement. For example, should top journals require preregistration, this standard is likely to diffuse. Yet intriguingly, the institutional profile of academia may be the biggest barrier to reform. Within academia, there are multiple entrenched interests, precluding solutions to many issues.

Over publication is a notorious example. In certain fields, the focus of researchers can be too narrow, entailing studies which have questionable relevance and validity. I have written some of these, including in ‘top’ journals! Relatedly, pressures to publish encourage the publication of dubious results, early-career researchers being particularly exposed. Ostensibly, these problems have solutions: the reduction of journals and revision of publication targets, researchers developing aims which genuinely advance fields and have greater relevance to society. Yet over publication has been part of the path dependency of academia for many decades, making such measures difficult to implement. There are cultural issues, researchers being expert in narrow topics and accustomed to heavy publication targets. Many jobs depend upon such output. Aside from the jobs of academics, much administrative activity is geared towards monitoring and supervising academic output, meaning that administrators also have an entrenched interest in publication culture. Publishers profit from the system, firms such as Sage and Taylor & Francis owning multiple journals. In short, entrenched interests may forestall meaningful reform of problems, entailing the endurance of the status quo; it is a classic problem of institutions being too big to reform.

Such developments herald a future in which studies continue to proliferate yet few have confidence in them. This is a bleak scenario, but reflects the logic of a sector which is so difficult to reform. Maybe we are here already, repeated blows denting confidence in academic findings. In certain fields, conditions are particularly desperate.

Perhaps this is a grave prescription. I do not argue that there will be no good research again or that academia is incapable of delivering this. Clearly, this is untrue; ironically, many of those studies which flag problems are examples of careful, important scholarship. Administrative support is essential to the production of such work. But pessimism does not take place in a vacuum, academia facing problems on several fronts. Academic working conditions are deteriorating, lowering morale and making the sector less attractive to new entrants. Cancel culture threatens freedom of expression. Administrative positions are expanding disproportionately, making management less sensitive to the concerns of academics. Increasingly, academia seems bloated and unreformable, giving rise to grave concerns about the future.

---

If you enjoyed reading this, do think about subscribing! Subscription is free – all it means is that you’ll receive a weekly email. But every new subscriber makes me very happy 😊 😊 😊