Data Colada alleges data tampering in influential 2002 procrastination study

A failed replication led to a fraud conclusion

The paper at the center of the story, "Procrastination, Deadlines, and Performance: Self-Control by Precommitment," was published in Psychological Science in 2002 by Dan Ariely and Klaus Wertenbroch. A new replication by Kyle Hyndman and Alberto Bisin, also appearing in Psychological Science, failed to reproduce the result of its second study. That failure prompted the researchers behind the Data Colada blog, who had obtained the original data files years earlier, to analyze them in depth. In a newly published post, they conclude that the data in both of the paper's main studies were tampered with. According to the post, when Data Colada shared its findings with Ariely and Wertenbroch, the two authors asked Psychological Science to retract the 2002 article, and that process is still ongoing.

What the 2002 study claimed

Ariely and Wertenbroch ran an incentivized proofreading experiment. Sixty participants, exactly twenty per condition, each received three ten-page documents seeded with 100 grammatical and spelling errors apiece, and had to find and correct them. One group faced evenly spaced deadlines, with one document due every seven days over three weeks. A second group chose its own deadlines within a 21-day window. A third group had to submit everything on day 21. The reported outcome was dramatic: evenly spaced deadlines produced markedly better performance, less delay and higher earnings than either alternative. The paper became a fixture of the field — assigned reading in many economics and psychology courses, with more than 2,100 citations on Google Scholar, according to Data Colada.

How the original data resurfaced

The provenance of the data files, as Data Colada recounts it, is unusual. In April 2006, Kyle Hyndman received three Excel files — covering a pilot, Study 1 and Study 2 — in an email from an address associated with the original research. File properties listed Dan Ariely as the last person to save them. Hyndman passed the files to Data Colada in August 2023, a week after Francesca Gino filed a $25 million lawsuit against the blog's authors. Crucially, Data Colada reports being able to reproduce all nine means and nine standard errors in the original article's main figure, plus six additional means reported in the text — strong evidence that these were the files behind the published results.

The evidence of tampering

Data Colada organizes its case around four red flags, beginning with the sheer size of the effect. Participants with evenly spaced deadlines corrected an average of 136.1 errors, against 71.1 for those facing a single end-of-period deadline. That is a Cohen's d of 2.5, with a correlation of r = .79 between condition and performance. For context, the post compares this with the effect of gender on height (roughly d = 1.8) and with a classic manipulation check in which people exposed to nine arguments rather than three noticed the difference at only d = 1.49. The idea that deadline structure moves proofreading performance more powerfully than people notice how many arguments they just read, the authors argue, is not credible. The distributions barely overlap: no participant in the last-day condition exceeded 100 corrections, while 90 percent of the evenly spaced group did.

The second red flag is duplicated observations. Several participants recorded not only identical totals but identical correction counts on each of the three separate tasks. The analysis flags 18 of the 20 participants in the last-day condition within this pattern of duplication. A follow-up post, the authors say, will examine Study 1.

A dispute over the data

There is also an unresolved dispute about the files. Footnote 14 of the replication paper, quoted by Data Colada, states that in October 2024 the replicators shared their analysis with Ariely at the editors' request and asked permission to summarize it in the paper. Ariely refused, arguing the files might not be the actual data, and did not provide any alternative dataset. Data Colada adds that, to its knowledge, co-author Klaus Wertenbroch never had access to any version of the data, and that it was his 2006 reply to Hyndman's inquiry that put the files into circulation. The blog notes its conclusion rests entirely on its own analyses, and has published the data and code in its ResearchBox so readers can verify the results.

Why it matters

A study cited more than 2,100 times and taught in university courses for over two decades now stands accused of resting on fabricated data, and a retraction would ripple through the literature on precommitment, self-control and deadline design that builds on it. The case is also a demonstration of why access to original data matters: the warning signs here — implausibly large effects and repeated response patterns — went undetected for twenty years until a failed replication sent researchers back to the raw files. That the co-author who handed over the data apparently never worked with it himself underscores how rarely the underlying numbers of even highly influential studies are ever examined.