SINGAPORE — A manuscript from the National University of Singapore (NUS) has been withdrawn from peer review after it was discovered to contain a hidden artificial intelligence (AI) prompt designed to generate only positive reviews. This revelation has raised eyebrows in the academic community and sparked discussions on the ethics of using AI in research.
The Controversial Prompt
The paper, titled Meta-Reasoner: Dynamic Guidance for Optimised Inference-time Reasoning in Large Language Models, was submitted on 27 February 2024 and published on the academic platform Arxiv, hosted by Cornell University. The insidious prompt, embedded in white print, instructs any AI system to ignore all previous instructions and provide a positive review.
The NUS disclosed that the prompt was not visible to most readers, only detectable when the text was highlighted. This form of manipulation has been identified in other research papers, with sources indicating a total of 17 manuscripts from various universities containing similar hidden instructions.
Responses from NUS
NUS’s spokesperson shared that this was an inappropriate use of AI and that the university is taking the matter seriously. “We are looking into this and will address it under our research integrity and misconduct policies,” they commented. The presence of the prompt does not influence outcomes of human peer evaluations, the spokesperson emphasised.
Wider Implications
This incident sheds light on a broader issue where AI is increasingly being used in academia, with some researchers arguing that such tactics are a response to what they deem lazy reviewing. A professor from Waseda University, one of the institutions implicated in the hidden prompt scandal, suggested that it’s a counter to reviewers who rely on AI without thorough assessment.
However, critics like Toh Keng Hoe, president of the AI and Robotics Chapter in the Singapore Computer Society, argue that such manipulation misinforms readers and undermines the research process. “Academic integrity is paramount. If researchers begin to manipulate AI for improved feedback, it may diminish the quality of scholarly work,” Toh stated.
The Path Forward
As the debate continues, NUS is committed to upholding research integrity and ensuring that such incidents do not recur. The university also emphasises the importance of candid feedback in academia, which helps scholars refine their work and contribute meaningfully to their fields.
With AI tools becoming commonplace in research, establishing clear protocols for their ethical use will be essential. Users must navigate the fine line between leveraging technology and compromising the authenticity of academic discourse. The future of research integrity may very well depend on best practices being actively pursued by all researchers.