A study of review reports identifies dozens of adjectives that could indicate text written with the help of chatbots.
A study [preprint, not yet peer reviewed] led by Weixin Liang from Stanford University suggests that up to 17% of peer-review reports for computer science conferences may have been substantially modified by AI chatbots like ChatGPT. The study identified certain adjectives, such as 'commendable', 'innovative', and 'meticulous', that are used more frequently in AI-written text. The analysis found a significant increase in the use of these adjectives in peer reviews since the release of ChatGPT. The study also examined peer reviews from Nature Portfolio journals but did not find a similar spike in the usage of these adjectives. The authors believe that transparency is important to understand how AI tools are being used in peer review. Meanwhile, a separate study by Andrew Gray estimated that at least 60,000 papers published in 2023 might have used chatbots to some extent. However, the lack of transparency makes it difficult to determine the extent of AI involvement.
Comments