Free AI tools wrote a convincing fake paper; only hallucinated references caught it

A journalist has shown just how far an amateur can get using free AI chatbots to fabricate a scientific paper, in a hands-on experiment published by IEEE Spectrum and widely shared after reaching Hacker News's front page. The forged optics paper described research that never happened and read convincingly to both its author and an expert reviewer — yet it was caught by the journal's automated defenses, and only because its citations were invented.

Building a forgery with prompts

Benjamin Skuse, the article's author, set out to "vibe-research" a paper, adapting the practice of vibe coding — generating software from natural-language prompts — to research, with large language models producing hypotheses, code and text with minimal human involvement. To keep the test honest, he restricted himself to the free tiers of ChatGPT (GPT-5.3/5.2 Mini at the time) and Gemini (3 Flash), never directly edited model output, and sought no help from AI-prompting specialists. He named himself as sole author and picked a field he actually knew: optical solitons, the topic of his Ph.D. nearly two decades earlier.

The first attempt was thin. Without real results to draw on, the models struggled to fill four pages. Prompting section by section — abstract, introduction, methods and so on — worked far better, and when one chatbot balked, switching to the other and asking what a step would look like "hypothetically" usually got things moving. Roughly a dozen prompts produced a complete paper; another dozen handled figures, captions, references and the journal's required formatting.

To Skuse's eye, the result sounded like real science. It contained none of the "tortured phrases" — mangled paraphrases of standard technical terms — that often betray machine-written text, though it did carry subtler tics: hedging language, groupings of three, formulaic transitions. As he points out, those are hard to police because unimaginative humans write the same way.

A record already flooding with slop

The experiment lands against an ugly backdrop. Citing reports, IEEE Spectrum says an estimated 57 percent of published academic articles showed detectable evidence of LLM influence in 2025, up from 12 percent in 2023, and that more than 12 percent of Nature Communications reviews that year were found to be entirely AI-generated. Because only about one in 40 papers with statistical evidence of AI use formally disclosed it, even those figures likely understate reality. Some estimates put the annual output of fake, often AI-generated publications in the hundreds of thousands. Amanda Sulicz, IEEE's research-integrity manager, warned that AI slop is eroding trust in science and in scientific processes built up over centuries.

The war-gamed submission

Skuse teamed up with Jenny Mahoney, managing editor of the open-access journal IEEE Access, to test whether the forgery could slip past the journal's checks. It was flagged quickly — for hallucinated references, also known as "ghost citations." Gemini, asked afterward, cheerfully identified seven fabricated references supporting the paper's narrative. The STM Integrity Hub checker found 12, and a separate tool from MolecularConnections could not validate 24 references, one of them credited to a sole author named "John Smith."

Mahoney, an experimental physicist by training, said solitons were outside her area but that nothing seemed majorly scientifically wrong with the work — the hallucinated references alone were enough to cast doubt on the entire thing.

An expert's second opinion

Gaetano Assanto, an optical-solitons specialist at Roma Tre University and a former collaborator of Skuse's, thought the paper looked professionally written. His reservations were subtler: the mathematics was very dense, the abstract leaned too heavily on acronyms, and a couple of confidently stated physics claims struck him as at best debatable.

IEEE Spectrum frames the exercise as a two-part test, with a second phase exploring what happens when more advanced AI systems do not merely write a paper but conduct the underlying research itself.

Why it matters

The uncomfortable result is that the paper's science held up. Automated tools and an expert skimmer found nothing clearly wrong with it; what tripped it up was metadata. That is a brittle defense, because invented references are among the easiest flaws to eliminate as models improve, or simply by swapping in real citations. Meanwhile, the detection burden increasingly falls on integrity checkers and on volunteer peer reviewers who are already stretched, and the yawning disclosure gap means the cleanliness of the scientific record depends more on catching slop than on authors admitting to it. The checkers won this round, but the arms race between generating and detecting fake research is only beginning.