In The Manifesto for Trial and Error in Science we have laid out our vision about scientific practice and scientific publication, and the intimate relation between the two. You can also view or download the pdf version.
We state that …
Trial and Error is the elementary process in Science by which knowledge is acquired. We differentiate between two types of scientific Trial and Error processes:
- Methodological errors in a practical sense, driving improvement in the understanding and application of techniques. These errors are here understood in a broad sense, those that go beyond the learning of the individual researcher and have an impact at the scale of the scientific community.
- Conceptual flaws, arising from hypothesis being confronted with conflicting observations. When the initial hypotheses are inappropriate in the face of empirical evidence, scientists improve or reject theoretical frameworks by developing alternative theses aimed at increasing empirical adequacy. Not only hits (positive results), but also misses (negative results) are key to scientific progress.
We identify three core problems in today’s Science. Namely, …
- … a public image of Science based on breakthrough discoveries, fascinating images, and clear results. This reputation comes at a cost. Both scientists themselves, as well as philosophers, sociologists and historians of science have increasingly been highlighting the importance of science in the making. A more faithful picture of Science, the one of practices and fine-tuning methodologies, seems to be at odds with the unrealistic public image of big-discovery Science.
- … a gap between what is published and what is researched. We know positive publication bias pressures scientists to conceal methodological mistakes and discard research containing negative findings, threatening proper interpretation. In the face of failed research —outcomes of Science that do not meet the initial aim of the individual researchers— scientists have two options at hand: not publishing or framing the results as productive by, for example, adding ad-hoc hypotheses in a potentially inadequate manner. This point is a consequence of the expectations of big-discovery Science and the publish-or-perish politics of this Science.
- … a replication crisis. Since scientists validate their results in terms of replicability, the present-day situation of unreplicable experiments is a serious problem. Debate on this replication crisis has focused on the misuse of statistics by scientists, on methodological carelessness, or theoretical inappropriateness. Only a few venues are attentive to the potential harm.
We stand in the context of …
- … a call for democratizing Science. Society rightfully demands that results are made accessible to both the public and fellow scientists. What is even more concerning is that individual researchers or citizens have to pay large amounts of public money to get access to mostly publicly funded research results. We need to rethink how Science is communicated by means of traditional publishing channels.
- … a need for dialogue. We identify a highly specialized academic community, aiming to tackle and reflect on social and intellectual challenges in a frequently unproductive way. Because of the scattered organisation of university departments and faculties, a constructive dialogue between different tribes of cutting-edge Science is missing. In the context of the earlier mentioned problems of a harmful public image of Science, the publication bias, the replication crisis and inaccessible Science, the lack of communication has to be addressed even more urgently. In the face of these multifaceted problems, we need useful solutions for the future of Science.
Therefore, we propose
A journal serving as a platform for Trial and Error in Science. We want to publish (1) methodological errors which have productive conclusions for the scientific community at large, and (2) conceptual errors in the form of negative results. In addition, our initiative aims to create a platform to openly talk about failure. That does not mean that we want to publish sloppy science. Rather, we believe that in talking about errors, scientists can learn about the do’s and don’ts of their methods and concepts. As well, because negative results are highly informative, this would help alleviate the issue of publication bias, and reframe the replication crisis. Young researchers are the hope for a change in Science, therefore we do take their work and ideas seriously. We aim to publish high quality work of early-career scientists, peer-reviewed and edited by more senior scholars. On every published article, a subject specialist, or a philosopher, historian, anthropologist or sociologist of science will be invited to reflect, thereby answering and problematizing the question “what went wrong?”. This combination aims to ensure novelty and quality in our journal.
So that …
- … both society and members of the scientific community appreciate scientific endeavour in a more realistic and productive way. By establishing a forum for failure, we aim to do justice to the difficulties of empiricism. We, at the Journal of Trial and Error acknowledge Science’s struggles in the practice as crucial elements in the generation and dissemination of knowledge.
- … the gap between what is researched and what is published will be closed. As a response to the false dichotomy between publishing breakthroughs or publishing nothing, we aim at giving a platform for publishing mistakes without fear nor shame. We claim that it is compatible to err in the experiment and be a contributing scientist, if we rethink what failure means. We already know that Trial and Error is productive in scientific practice; we are now exploring what productive means in scientific publishing
- … the replication crisis is understood in its complexity. Our project aims to provide a common ground for the reflection on one of the landmarks of Science: replicability. Both empirical scientists and humanities scholars of science have long thought about what it means to show (in)comparable, (in)compatible or (un)identical results. Our journal offers a place to exchange such varied views.
- … users of scientific results get unrestricted access to relevant scientific content. In the age of Open Science, we share the optimism of freely sharing articles and results, and wish to extend it to sharing data, methods and errors.
- … methodological pluralism is concrete and constructively focused, thereby helping scholars to err in a productive way, so they can trial enriching solutions for social and intellectual challenges.
Coauthored by the founders of JOTE
Martijn van der Meer
Max Bautista Perpinyà
Upheld by all its current members and lead editors.