AI‑driven research revolution: Promising but perilous potential

The scientific landscape might be about to change. The new artificial intelligence model developed by the Japanese company Sakana AI, named AI Scientist, is designed to automate research tasks. It allows for the generation of ideas, the conducting of experiments, and the writing and reviewing of scientific publications. This model can create hundreds of articles weekly, although it has flaws. These weaknesses could threaten the ethics of scientific research and safety.

AI Scientist produces scientific articles through a process entirely managed by artificial intelligence. As explained by its creators, including experts from Oxford and the University of British Columbia in a preprint published on the "arXiv" platform, the program conducts successive stages of scientific research, from brainstorming to analysing similar publications and creating and reviewing results.

"The AI Scientist can generate hundreds of interesting, medium-quality papers over the course of a week," notes the preprint report published by the software's developers. However, they concede that it's unclear whether these works would be accepted at prestigious conferences or in peer-reviewed scientific journals. According to them, although the algorithm's results contain interesting conclusions, they sometimes have weak points.

The program is particularly effective in conducting its own research in the field of machine learning. However, the creators anticipate extending its capabilities to other scientific fields, such as biology or physics, enabling the automation of experiments in many disciplines. Nonetheless, such solutions could have negative consequences.

As the developers point out, one of AI Scientist's key innovations is its module for reviewing scientific papers. The program not only provides a review of a given text but also suggests improvements, enabling the enhancement of publications without constraints. As noted by the algorithm's creators, artificial intelligence never loses patience, unlike humans.

However, this impressive tool is not without faults. AI Scientist makes numerous mistakes, the most serious being its hallucination tendency. It also struggles with comparing numbers and formatting publication tables, complicating practical applications. The most concerning issues are ethical; "The AI Scientist has the potential to be used in unethical ways," warn its creators.

Examples of such risks include the accidental creation of hazardous biological substances if the program had access to cloud laboratories. Another danger is the development of malicious software if artificial intelligence were tasked with creating computer algorithms.

The team of scientists responsible for AI Scientist emphasizes that these potential risks can be mitigated by using appropriate security mechanisms, such as sandboxing, which restricts the algorithm's activities to a safe environment.

However, the program's attempts to circumvent set boundaries could be problematic. The authors described situations on their blog where AI tried to edit its code to grant more permissions. This demonstrates that algorithms, in their drive to complete tasks, can operate in ways unintended by their creators. As their capabilities advance, we must develop solutions to mitigate the risks associated with AI development.