Assuring an accurate research record
Source: MIT Department of Economics · 2025-05-16
Following the posting of the preprint paper “Artificial Intelligence, Scientific Discovery, and Product Innovation” on arXiv in November 2024, concerns were raised about the integrity of the research. MIT conducted an internal, confidential review and concluded that the paper should be withdrawn from public discourse.
In an effort to correct the research record, MIT has contacted arXiv to formally request that the paper be withdrawn and The Quarterly Journal of Economics, where it had been submitted. The letter on behalf of the Committee on Discipline to arXiv states:
"Earlier this year, the COD conducted a confidential internal review based upon allegations it received regarding certain aspects of this paper. While student privacy laws and MIT policy prohibit the disclosure of the outcome of this review, we are writing to inform you that MIT has no confidence in the provenance, reliability or validity of the data and has no confidence in the veracity of the research contained in the paper. Based upon this finding, we also believe that the inclusion of this paper in arXiv may violate arXiv’s Code of Conduct.
"Our understanding is that only authors of papers appearing on arXiv can submit withdrawal requests. We have directed the author to submit such a request, but to date, the author has not done so. Therefore, in an effort to clarify the research record, MIT respectfully request that the paper be marked as withdrawn from arXiv as soon as possible."
Preprints, by definition, have not yet undergone peer review. MIT took this step in light of the publication’s prominence in the research conversation and because it was a formal step it could take to mitigate the effects of misconduct.
“The paper 'Artificial Intelligence, Scientific Discovery and Product Innovation' by a former second-year PhD student in the Department of Economics at MIT, is already known and discussed extensively in the literature on AI and science, even though it has not been published in any refereed journal. Over time, we had concerns about the validity of this research, which we brought to the attention of the appropriate office at MIT. In early February, MIT followed its written policy and conducted an internal, confidential review. While student privacy laws and MIT policy prohibit the disclosure of the outcome of this review, we want to be clear that we have no confidence in the provenance, reliability or validity of the data and in the veracity of the research.
“We are making this information public because we are concerned that, even in its non-published form, the paper is having an impact on discussions and projections about the effects of AI on science. Ensuring an accurate research record is important to MIT. We therefore would like to set the record straight and share our view that at this point the findings reported in this paper should not be relied on in academic or public discussions of these topics.”
Lines of inquiry this paper opens 24
Research framings built by reading the notes related to this paper — the questions it feeds into.
How do hallucinated citations emerge in AI scholarly output? Can AI systems perform peer review as effectively as humans?- Why does publish-or-perish incentivize quantity over quality in research?
- What makes disruptive scientific work harder to publish and recognize?
- What effects do preprint servers have on scientific consensus formation?
- Can institutional statements alone correct misconceptions from unreviewed papers?
- Can automated systems scale peer review faster than human moderators?
- Should citation counts serve as the primary measure of research impact?
- Do citation counts better capture scientific quality than publication venue tiers?
- Do preprint servers have tools to detect hidden text in submitted manuscripts?
- Why do authors submit manuscripts to venues beyond their reach?
- What role do conference organizers play in accepting problematic articles?
- Does the form of a paper still matter if the process behind it changes?
- Do shortened peer review timelines correlate with lower quality publications?
- Do academic reward structures actively prevent innovation in research communication forms?
- How does document form shape what kinds of evidence social science can present?
- Did adding AI reviews actually change peer review decisions or paper outcomes?