The Australian Research Council (ARC) has issued a warning to academics about the potential breach of confidentiality when using generative AI models, such as ChatGPT, to write peer reviews of grant applications. The ARC’s statement comes after reports surfaced that some peer reviews for ARC discovery projects were generated with ChatGPT.
ARC_Tracker, a Twitter account that monitors ARC activities, received information indicating that certain assessment reports provided to research teams were generic summaries of their grant applications, lacking critique or assessment. One report even included the telltale phrase “regenerate response,” indicating it was likely generated by ChatGPT.
The account owner, who is a researcher at an Australian university, shared that the phrases used in these AI-generated assessments were simple rearrangements of the original grant proposals. Researchers themselves had tested ChatGPT using their own proposals and found strikingly similar summaries to those in the assessments they received.
Following these developments, the ARC cautioned peer reviewers against incorporating AI tools into their assessments. The council emphasized that releasing materials, including using generative AI tools, outside of the closed research management system may breach confidentiality. The ARC plans to update its guidance on AI use in the near future.
While the ARC did not comment on the prevalence of ChatGPT use in peer reviews or the adoption of AI detection models to identify its use, they acknowledged the confidentiality and security challenges posed by generative AI tools.