arXiv Cracks Down: One-Year Ban for AI-Generated Hallucination Submissions
Researchers submitting preprints to arXiv now face a one-year ban—and a permanent peer-review requirement—if their papers contain unchecked AI-generated hallucinations. The policy, revealed by arXiv moderator and Oregon State University emeritus professor Thomas Dietterich in a social media thread, targets fake citations, placeholder text, and other telltale signs of careless large language model (LLM) use. Dietterich’s announcement specified that “incontrovertible evidence” of unvetted AI output, such as hallucinated references or meta-comments left by LLMs (“here is a 200 word summary; would you like me to make any changes?”), will trigger these sanctions. Ars Technica broke the story.
arXiv’s move comes after a surge in submissions containing AI-generated errors—sometimes as blatant as nonsensical diagrams or tables left with “illustrative” placeholders. The concern: if an author uploads a paper with clear evidence they didn’t check AI-generated sections, arXiv’s moderators say they can’t trust any of the content. The ban applies to authors across at least physics, astronomy, and computer science, though the full scope across all arXiv domains remains to be confirmed.
Researchers Face New Scrutiny, Moderators Get New Burdens
For scientists using AI tools to draft or edit manuscripts, arXiv’s stance is unambiguous: the author is fully responsible for every word, regardless of how it was produced. Dietterich cited the arXiv Code of Conduct, which holds each author accountable for the content of their submissions, AI-generated or not. This policy particularly hits fields like machine learning and computer science, where preprints are both prolific and often written with LLM assistance.
Detection is the next challenge. While obvious tells—like a ChatGPT meta-comment or references to non-existent papers—are easy to spot, more subtle hallucinations might slip through. Moderators must now decide what counts as “incontrovertible evidence” of unchecked AI use. The policy does not specify how edge cases will be handled, nor does it set out appeals processes for authors who believe they’ve been wrongly penalized.
Community reaction is already live in tech forums and social media. Some researchers argue the ban is overdue, comparing unchecked AI errors to outright data falsification. Others warn of overreach, fearing a chilling effect on legitimate uses of LLMs for brainstorming or language polishing. The debate: where does helpful automation end and scientific malpractice begin?
Unanswered Questions and the Ripple Effect for Academic Publishing
Several key uncertainties remain. arXiv has not formally announced the policy on its main site, and the official leadership has yet to confirm the precise scope or enforcement mechanism. It’s still unclear whether the one-year ban is universal for all arXiv categories, or if it will be enforced more strictly in domains with high LLM use. The threshold for what counts as “incontrovertible evidence” is also left to moderator discretion, a potential flashpoint if enforcement seems inconsistent.
What is clear is that the credibility of preprint servers is on the line. Preprints are increasingly cited—even before peer review—and serve as the first public record of new discoveries. As LLMs get better at mimicking scientific language, the risk of plausible-sounding nonsense slipping through rises. If arXiv’s approach works, expect other preprint platforms (and possibly journals) to follow with their own bans or detection protocols.
What to Watch: Enforcement, Evasion, and the Next Move
The real test starts now. Will arXiv moderators keep up with the flood of new submissions, especially in fields with hundreds of daily uploads? Will banned authors try to evade the rules by changing names or affiliations? And will the threat of permanent peer-review requirements deter repeat offenders—or just push them to less-regulated servers?
For researchers, the takeaway is immediate: check every AI-generated section, and never trust a reference or diagram you didn’t verify yourself. For publishers and platform operators, this is a warning shot. As LLMs generate ever-more convincing scientific text—and errors that are harder to spot—the industry’s trust infrastructure is only as strong as its worst-case enforcement.
arXiv’s ban marks a turning point for academic publishing’s relationship with AI. Whether it sets a new standard or just starts a whack-a-mole game with increasingly clever fakes is the next chapter to watch.
Impact Analysis
- arXiv's new policy signals a stricter stance against unchecked AI-generated content in scientific preprints.
- Researchers may face significant career setbacks, including a one-year ban, for submitting papers with AI hallucinations.
- The move raises the bar for scientific integrity and may influence how AI tools are used in academic writing.
