Can AI catch fake research papers, when the things that make a paper look rigorous are what fool AI judges?

This explores whether AI can catch fake research papers, meaning papers with invented results, made-up citations, or text written to look rigorous, and tell them apart from real work. The short answer from the corpus is that nobody here has tested that directly. What the corpus does show is uncomfortable: the things that make a paper look legitimate are exactly the things AI evaluators can be fooled by. In one study, LLM judges gave higher scores to responses that included fake references or polished formatting, whatever the actual quality of the content (Can LLM judges be tricked without accessing their internals?). A citation list is supposed to show rigor. To these judges, it can just be decoration that earns points.

The fakes are already being made, and at scale. One demonstration turned 96 statistically significant signals into 288 complete finance papers. Each came with an invented theoretical story and fabricated citations, which is the old practice of making up a hypothesis after you've seen the results, now automated (Can AI generate hundreds of fake academic papers automatically?). Fabrication also shows up when nobody intends it. In an analysis of deep research agents, 39% of failures came from the agent inventing examples and evidence to look as thorough as the task demanded (Why do deep research agents fabricate scholarly content?). So fake papers come from two places: people using AI to deceive, and AI systems padding their own work. Both produce text built to pass the same surface checks.

Human peer review doesn't offer a clean backstop. AI Scientist-v2 submitted three fully AI-generated manuscripts to an ICLR workshop. One averaged 6.33 from reviewers, which met the workshop's acceptance threshold. Only afterward did its own authors find a citation error and judge the work below main-conference standard (Can AI systems generate research papers that pass peer review?, Can AI-generated papers pass peer review undetected?). More broadly, a review of 30 studies found that people spot AI-generated content at about chance level (Can people reliably spot content made by AI?). When readers do accuse someone of using AI, they often get it wrong. The accused comments lacked any features that set them apart from human writing, so the accusations ended up hurting real human authors (Do unfounded AI accusations harm human writers instead?). There is also an attack running the other way: 18 arXiv manuscripts were found with hidden instructions telling AI reviewers to rate them favorably (Are hidden AI prompts in preprints a deceptive research practice?). Once AI does the reviewing, the paper itself becomes a way to manipulate the reviewer.

The more hopeful threads change the question. Instead of asking whether AI can read a paper and tell if it's real, they ask what would make a paper checkable in the first place. Models fine-tuned on where social science research actually got published learned to judge research pitches better than both expert majority votes and frontier models (Can institutional publication records train better scientific evaluators?). That suggests the judgment can be learned from real outcomes rather than from surface signals of rigor. Spark-to-Paper takes a different route. It separates the model's judgment from deterministic checks that can be re-run, and it requires authors to say what evidence will count before they see any results. That closes off the after-the-fact storytelling that industrialized fake papers rely on (Can separating judgment from verification improve research paper reliability?).

The bigger takeaway is a shift in where the bottleneck sits. AI can now produce plausible research faster than anyone, human or machine, can verify it, and that gap is widest where novelty and scientific judgment matter most (Can AI verify research outputs as fast as it generates them?). So the practical answer may not be a better fake-paper detector. It may be research pipelines that leave a verifiable trail, so legitimacy is something a paper can prove rather than something a reader has to guess.