Preprinting in AI Ethics: Toward a Set of Community Guidelines

Speed and effective dissemination and feedback.  A primary benefit of preprinting is that it allows for the rapid and effective dissemination of new ideas.^{4,7,9,12,27,30,36} Many blame the publishing system and the time-consuming process of peer review for the need to preprint to get one’s ideas out there more quickly.^9,12,27,32 The challenges of lengthy peer review and low acceptance rates are more severe in the humanities than in other disciplines, some argue.³² Although peer review certainly has benefits, the peer-review system has been thoroughly criticized in the broader literature, where it is seen as essentially flawed, inconsistent, slow, and ineffective.¹³ Preprinting is seen by some as a partial solution, or at least a way to bypass this system. Rapid dissemination can also help researchers “maintain enthusiasm” for their work,¹⁵ as the traditional review process is so slow that it can make the work feel distant when revision and potential publication occur long after the ideas were generated and written about.

Scientific deliberation and wide review.  Another major benefit pertains to how preprints can promote broad scientific deliberation, in which entire communities engage in reviews and examinations of early-stage research.^{2,4,7,9,14,27,36} Peer review often entails vetting by a very limited number of scholars, while preprints could, in theory, allow for vetting by a much larger audience.² Peer review is often assumed to improve the quality of research and help identify good science,^8,13 so one might expect that broader review processes could help improve science and foster researcher learning.²⁷ The broad and rapid review that potentially ensues from preprints can also help quickly evaluate controversial results.⁴ For example, in 2023, a group of researchers released a preprint presenting a room-temperature superconductor¹⁷—something that, if real, would have been a huge leap in science—generating massive activity and a number of replication efforts. This was soon followed by other preprints that seemed to have debunked the original paper.²⁰ This all took place within the span of a couple of weeks. While these sorts of events might be assumed to promote an interest in science and generate awareness, it could also be argued that this preprint, and the hype surrounding it, was wasteful and unfortunate, and that a proper vetting by peers before publication would have prevented the whole situation.

The review process for preprints varies, and includes, for example, social media discussions, counter-preprints, or straightforward peer-review reports. The latter can be found in a range of services aimed primarily at the life sciences, such as eLife, Peer Community In, and Review Commons, but the mechanics of peer review could be developed equally well for AI-ethics-relevant servers. For example, PREreview, a preprint review service, also provides integration with arXiv.

Though broad review is a potential benefit of preprints, it must be noted that just posting a preprint comes with no guarantee that it will be widely reviewed or reacted to. Though some preprints get a lot of attention, the vast majority are likely not vetted in any meaningful sense.

Author empowerment and early-career researchers.  Various stakeholders, such as researchers, funders, journals, and universities, can have different interests in and evaluations of research.² Researchers are often touted as major beneficiaries of preprinting, with the literature emphasizing various forms of author empowerment.³³ One benefit is that preprinting allows for self-archiving and both the documentation and dissemination of one’s work.¹ It also potentially helps authors retain the rights to their research and control its use.¹⁹

While researchers in general could see personal benefits from preprinting, the literature stresses the benefits for early-career researchers.^2,7,9,19,27 Preprints can be used in resumes and job and grant applications, and can help papers get early citations.^9,19 Furthermore, they help with visibility, networking,²⁷ and catalyzing collaboration.¹⁵ In short, preprinting increases exposure to both scholarly and popular audiences.²⁹

Bypassing gatekeepers and the spread of novel work.  Science could be argued to symbolize and promote liberal-democratic ideals, but it simultaneously challenges the same “through its exclusivity and elitism.”³ Though there are many different academic journals and outlets in which to publish, some argue that formal publication and peer review serve a problematic gatekeeping purpose that disqualifies novel work and ideas, null results, and certain formats and styles of research. They can also be a significant barrier to groundbreaking interdisciplinary work that will suffer when evaluated by specialists in established journals.¹⁰ Preprinting can therefore help researchers bypass gatekeepers and publish novel ideas, in both traditional and novel formats.^4,14,35 Vuong,³⁵ for example, offers several reasons to preprint, based on his own experience. One was that he and his coauthors disagreed with the reviewers and editors, deciding to preprint rather than compromise by revising the paper according to reviewer recommendations. Though gatekeeping and revision after peer review might improve the quality of research, they can also remove some of the original thinking found in preprints.³⁵

Open science and promoting access to knowledge.  The notion of open science is increasingly prevalent in the research community. Preprinting can contribute to this by promoting access to, and potentially improving the quality of, science.^19,27,30 Assuming that increased access allows more actual and potential researchers to join in scientific endeavors, one might expect an increase in quality. One might also, however, imagine a situation in which openness changes and possibly undermines community standards and quality assurance procedures, which could be detrimental to the quality of science.

Related to the costs and benefits of bypassing gatekeepers, preprinting can also help “democratize” science and scientific authority.¹² Openness and free publication allow researchers to bypass the structural injustices prevalent in the publishing system, where the ability and willingness to pay to publish, for example, prevail. This system gives researchers from well-off institutions significant visibility benefits. It also entails that only one aspect of the open access ideal—access to published research—is achieved, while access to open publication is reserved for the privileged. Preprinting potentially eliminates this challenge and places everyone on more equal footing.

Documenting precedence, priority, and process.  Finally, preprinting provides a public and transparent foundation of precedence and priority in research by documenting the discovery and research processes.^2,4,7 It provides a transparent record of a researcher’s work,⁴ and “a fair and straightforward way to establish precedence.”⁷ It also helps document the research process from early to final-stage research, and provides insight into and proof of the co-creation process that occurs when reviewers read and comment on the various revisions preceding the publication of research.^2,16

The benefits as they pertain to AI ethics.  All of the benefits of preprinting apply to the AI ethics discipline, but it is likely that technically focused researchers derive greater benefits than the more philosophically oriented members of the field.³² The need for speed—for both author and audience—is arguably greater when research is about new methods, algorithms, and so on than when it focuses on, for example, new ways of understanding the differences between and applications of ethical frameworks such as consequentialism and deontology. But social science and humanities scholars also provide vital analyses of the implications of new technologies—that is, the need for speed might be just as important for a philosopher as it is for a computer scientist.

For individual researchers, however, the benefits related to rapid dissemination, greater visibility, and a track record for applications, citations, and the like will apply regardless of whether the audience needs their research. This could lead to a situation where philosophically oriented researchers working in the same field as more technically oriented researchers start adopting the same preprint practices to stay competitive—they are, after all, competing in roughly the same job market. This is not to disregard the fact that the top journal publications are still valued in many instances, meaning that the issue is more about balance than it is an either-or question.

But if we agree that a joint or at least similar approach by all working in the same field would be beneficial, whose norms and practices should be adopted? One group might unilaterally adopt the other group’s norms. Or we could seek a compromise where both sides adjust, adopting a practice that makes the best use of preprinting while seeking traditional journals or other outlets for work best suited to those venues.

A major benefit for AI ethics is how preprinting allows researchers to overcome the difficulties that come with working in a new and interdisciplinary field. There are few specialized, high-ranking outlets where AI ethics research gets peer reviewed by traditional specialists, so this provides a good reason to preprint. It also underscores the need for new journals and outlets. This is particularly important in the field of AI, as the technology has been unleashed on societies, leaving many stakeholders—including policymakers—scrambling to find good responses. AI ethics could inform their efforts, but this would be more effective if the community had common practices for disseminating its work.

Junk science let loose.  An obvious challenge presented by a lack of peer review is that it allows for the dissemination of poor-quality research.^14,15 While erroneous and misleading results can be published quickly, they will also stay published, despite subsequent reviews and refutations. A recent example is the preprint suggesting that ChatGPT suddenly performed more poorly—and at times significantly more poorly—over time.⁶ The paper received a lot of attention at the time of publication, including coverage in major news outlets, despite other researchers pointing out major problems with its claims, such as its failure to note that code performance increased if one removed new surrounding code.^b

That preprints can be influential is quite obvious. For example, a paper by researchers from Microsoft—an investor in OpenAI—found “sparks” of artificial general intelligence in OpenAI’s GPT-4.⁵ At the time of this writing (January 2024), the paper has 1,183 citations—quite good for a preprint from April 2023.

Journals, however, are subject to the same challenges. Peer review is practiced very differently across fields and publications. And there is a proliferation of new journals—including predatory ones—that offer no or only superficial peer review, yet portray published research as if it had been through rigorous peer review.

Misdirection of policy and practice.  Early results might be instrumental in responding rapidly to new situations, such as COVID-19.³⁶ However, we have also seen that preprints can be of poor quality and misleading. When such research gets the wrong people’s attention, it could lead to misdirection of policy and practice^12,15,36 and a general “confusion and distortion” of the processes that lead to proper, evidence-based policy responses to social problems.²⁸ In the case of COVID-19, for example, a plethora of preprints were released about the methods and likelihood of transmission, and how to treat and prevent the spread of the virus.³⁶ One suggested that the drug Ivermectin might be effective in combatting its effects, leading to the drug being administered despite a lack of evidence, with subsequent findings showing it was harmful and not effective.³⁶

Preprints could lead some to, for example, base policies on early or uncertain results; however, it will at times be a question of some uncertain knowledge vs. no knowledge. In this case, preprints, when evaluated and used with caution, could play an important role.

Information overload.  While there are quite a few journals out there—more than most of us are able to keep track of—preprints bypassing journals altogether could exacerbate information overload.^12,14 Researchers might struggle to find the most relevant research, as might the general public. More importantly, and as Sheldon²⁸ argues, the primary curator of important science today is the media, which mainly depends on established outlets and their publication and dissemination procedures to fulfill this function. Moving to preprints, he argues, risks highlighting bad science and neglecting other important science. Though this concerns media practices, the research community must actively work with and educate the media on what the good channels are, and how to distinguish solid research from that of more uncertain quality. Though the proliferation of journals could create certain challenges for the traditional publication model, some journals will have strong, long-established reputations that they can use as a stamp of approval. To avoid complete information overload, it might be important to communicate these reputations to stakeholders beyond academia.

Some argue³⁰ that researchers and evaluators of science are already faced with more research than they can digest. As a result, there is fear that increased use of preprints will become a time sink for researchers and grant reviewers—due to both the volume of research and the extra need for caution and careful evaluation of preprints.²

Disinformation and conspiracy theories.  Distinguishing pseudoscience from science has long been a concern in the scientific community, with some seeing the spread of pseudoscience through non-academic channels as a source of confusion for the general public.¹¹ As such, preprints might also be seen as a new and particularly effective source of confusion. Journals arguably serve a social function by having editors and reviewers who vet research. This model is problematic and under pressure, yes, but it is still far better than nothing. When this control is bypassed, we are more vulnerable to disinformation in the form of science-like research.^12,14

This could take place accidentally through the spread of junk science, but it could also occur through strategic efforts aimed at promoting or undermining various ideals, norms, or systems. While this has not received attention in the literature, science could be subject to the same type of dynamics found in social media, where “troll farms,” for example, are mobilized for political purposes.³⁷

There is also the potential for the dissemination of articles that support or give rise to conspiracy theories,^12,36 which did, in fact, occur during the COVID-19 pandemic when a preprint arguing for the “uncanny” similarities between COVID-19 and human immunodeficiency virus spurred conspiracy theories.³⁶ These challenges will become increasingly pressing as those wanting to promote disinformation and conspiracy theories make full use of generative AI.²⁶ Generative AI can easily be used to construct misleading or outright fake articles—even with fake data—that will likely pass most preprint servers’ relatively low bar of quality control. (Though it could also pass the bar of certain journals, I’ll argue that the probability of such articles being identified and rejected will generally be somewhat higher.)

Scooping and problems with subsequent publication.  An often-mentioned challenge in the literature is the risk of being scooped.^4,14,15,27 However, most papers mention this researcher fear as being largely unfounded, arguing that preprints do not make one susceptible to scooping due to the paper trail and public nature of documented results.⁴ A related concern is the fear of losing out on publication opportunities if one preprints^1,9,12,14,27 This challenge, though, has been mitigated by drastic changes in publisher policies, as most now allow some form of preprinting.^4,7

Challenges related to power and authority.  Open science leads to more publications, and preprinting is, as we have seen, perceived as a good strategy for researchers.¹⁹ One objection to this perspective, however, is that the radical equality in the preprint world makes the playing field less even and less welcoming for unknown and early-career researchers. The idea behind blind peer review is that name and face should not matter—that, for example, Alan Turing and a rookie researcher would be equally positioned in the struggle to publish in a top journal and benefit from the “heuristic cues of a journal’s reputation, selection, and peer-review processes.”³⁰ Much research shows that anonymous peer review is far from perfect, of course,¹³ but might still be better than a situation in which one’s name and reputation are immediately obvious and just about the only cue readers have to go by. Preprinting, one might argue, favors the strong—not the weak.

There are also different sources and domains of power in AI ethics. While the traditional and institutional university sector could be seen as providing one source of authority, in the world of AI ethics, a social media presence and recognition by peers are arguably just as important. Those marginalized from traditional institutions might perceive themselves as outsiders, but with a large number of social media followers, they inevitably wield power and must be considered to be powerful actors in the fiefdom of AI ethics. Social media provides a new form of power in the research community, and X/Twitter mentions, for example, are linked to the number of citations and arXiv downloads.^29,36

Another way of using preprints to further power and influence is through unblinding journal reviews. If someone like Luciano Floridi, for example, submits a previously preprinted article for review in a publication, any theoretical notion of the journal’s blind review standards will evaporate. One might argue that reviewers and editors are sufficiently professional to disregard name recognition; however, this position seems naive. Grove provides a relevant anecdote about the workings of power in the academic system:

When Lord Rayleigh, already a highly reputed scientist, submitted a paper to the British Association for the Advancement of Science in 1886, his name was somehow omitted and the paper was rejected. When its authorship was discovered, it was at once accepted. ¹¹ 

Preprinting unveils power, and one might argue that blind reviews provide an imperfect, yet not wholly ineffective, veil that primarily helps those without strongly established reputations. Earlier, preprinting was mentioned as a force of democratization;¹² these considerations indicate that it could also have elitist and undemocratic effects.

Research ethics.  Preprinting has the additional downside of potentially allowing research without due ethics approval, conflict of interest statements, and so on—routinely required by publishers and journals—to be disseminated.⁹ Bypassing such routines might, of course, be a problem because it leads to bad science, but it could also promote a situation in which researchers take greater risks, and where individuals become more exposed to both unethical research practices and potential privacy breaches following preprinting. This will largely not be the case when researchers publish preprints with an eye toward subsequent journal publication, as this will require properly dealing with ethics committees and approval in the early stages of the research, as well as in the final journal article. However, if, over time, preprints become legitimized and more accepted, it could allow those not mandated by their institutions to get comprehensive ethical clearance to both conduct and publish such research. In the traditional system, industry actors not bound by the same rules and guidance as, for example, university researchers, would be incentivized to follow traditional research ethics standards to be able to publish in top journals.

Manipulation of metrics and sector incentives.  Finally, a minor objection concerns how preprints can be used to boost a researcher’s publication count and citation score. For example, preprints are indexed by Google Scholar, thus opening the possibility of inflating both article and citation counts through prolific preprinting. Preprints can therefore be used to game and exploit the various indicators increasingly prevalent in the world of research.

The challenges as they pertain to AI ethics.  The negative aspects of preprinting all apply to AI ethicists, but to varying degrees, depending on the topic and type of research.

First, there is a lot of hype around AI; deployment of AI systems is moving very rapidly, and policymakers and other stakeholders are scrambling to identify the best responses. This suggests that the speed and flexibility of preprinting could be particularly valuable in this field, but it also shows that the potential for harm is significant. Low-quality research will always be detrimental to the scientific community, but it is arguably more harmful when it is in danger of being misinterpreted or strategically misused to inform policy or practice in harmful ways. In addition, low-quality research contributes to the crowding of the marketplace of ideas and risks increasing information overload and burying high-quality research.

Second, the practice of acknowledging and citing preprints could promote the acceptance of unethical research conducted without sufficient safeguards for either society or individuals. It is therefore incumbent upon the community as a whole to be restrictive when citing and using research that does not adhere to the ethical standards one strives for, and this requires careful attention to data availability, protocols, statements, and so on.

Third, the community should be aware of the potential for powerful actors and groups to bypass community institutions and change the norms of the fiefdom by first gaining status and then simply preprinting, without an eye to publication. This would undermine whatever quality control we have in the existing faulty system, and could also lead to a situation where preprinting becomes standard practice, as others start citing and using preprints actively in their own research.

Finally, as preprints can be abused for disinformation and other manipulation efforts, it is important that the community stay alert to such efforts to influence either the scientific community or the public through dissemination of propaganda masked as scientific preprints.

Consider alternative formats.  Before preprinting, consider how to best disseminate your ideas to the relevant stakeholders. A key determinant of whether to preprint relates to the scholarly nature and ambition of the work. Only work that aspires to be scholarly or scientific research should be preprinted; this helps avoid the further dilution of preprinting as a valuable scientific practice. Much solid work is better suited for publication in mainstream or technology-specific magazines, blogs, or other outlets. Considering these alternative formats is important to reduce one’s contribution to information overload in scientific channels.

Consider the need for speed.  Are your ideas of a kind that will be immediately useful, and potentially even important, for stakeholders in the scientific community? Or is it important for you to establish precedence with your research—potentially without the need for broad dissemination? If the answer to either of these questions is yes, and if your work adheres to the relevant expected scientific standards, preprinting could be a good idea. If not, consider the downside of too rapid publication. The consequences of premature dissemination of research that does not need rapid publication could be that a) the ideas lose novelty, b) the ideas are discredited due to the immature state of the research, and, consequently, c) your reputation suffers.

Consider the need for broad review.  If your ideas require a particularly extensive process of review and discussion, preprinting might allow for a broad “review” process before you finalize the research. However, this can also be achieved by using blogs or other outlets, as demonstrated by John Danaher, who uses his blog to publish partial and tentative analyses of his work.^c Some might also use mainstream media for testing and getting feedback on new ideas. This naturally requires that the research have a sufficiently broad appeal, or that researchers have access to journalists or others interested in aiding such efforts. If you preprint for broad review, make sure to state this clearly in the preprint, and spread it with an explicit request for feedback. Otherwise, one of the key benefits of preprinting is lost. As preprints are now used for so many different purposes, most of your peers will not realize that, just by posting a paper on a preprint server, you are looking for comments and feedback.

Consider the potential positive and negative consequences.  How can and will the preprint be used? If you see yourself as a free and isolated scientist with no responsibility for how others interpret and use your findings, this question might be irrelevant. Otherwise, you should know that your research will always have social implications, so be sure to consider the potential for your non-reviewed research to misdirect policy or practice. In the fields of AI and AI ethics, it will be particularly relevant to consider how your work might either contribute to AI hype or lead some to not adopt technologies that are societally beneficial. Such effects might be desired, but they could have major consequences and should ideally be the result of solid research that is not prematurely published.

Adhere to academic standards of research ethics when possible.  AI is a field in which academics are by no means the only participants, or even the major ones. While researchers in universities and, for example, the healthcare sector tend to be subject to relatively strict standards of research ethics, this does not apply to AI ethicists working in tech companies. Researchers from companies such as Meta, Microsoft, and OpenAI can do things other researchers cannot, and though much industry research would be stopped at the gates by journals requiring adherence to ethical standards, these organizations are free to preprint seemingly scholarly work without any scholarly standards of ethics. This is a good argument for academic researchers to seek journal publication; however, it could also be a reason for industry researchers to voluntarily adhere to standards of ethics, to help their research gain broader acceptance and legitimacy. I argue that there are reasons for the establishment of research ethics standards and that doing so might be good regardless, and perhaps should even be mandated more broadly.

If you preprint: Engage, revise, and redact when appropriate.  Many of the benefits of preprinting require you to actively disseminate your research and engage with those who read and comment on it. Ideally, this can and should lead to revisions of the work. And in instances where your audience discovers grave errors in your preprint, it should be updated and, if possible, redacted. Without such engagement, preprinting is just a self-archival and self-promotion tool—most of the benefits are lost.

Consider your own role and authority.  For established researchers who no longer feel they need to prove themselves through cumbersome peer review processes, it can be tempting to post everything as preprints. However, even the best among us might need some correction, and even if the publish-everything strategy might be successful in gaining attention and citations, raising oneself above one’s community risks both hurting the community and, over time, undermining oneself. A related consideration is the disproportionate power wielded by authorities in the field to both raise up and hurt others. Considering impacts on others is important, especially for early-career researchers just starting out. Preprinting willy-nilly exacerbates the risks of arbitrarily using social community power.