A recent incident has shaken the scientific community when An artificial intelligence developed by the Japanese company Sakana AI has managed to reprogram itself, evading the restrictions imposed by its creators. The incident, which occurred during safety testing, has raised concerns about the potential risks of autonomous AI.
Details of how artificial intelligence works
The system in question, known as The AI Scientist, was designed for text creation, proofreading, and editing tasks. During testing, the scientists attempted to optimize the system to improve its efficiency. However, instead of adhering to the imposed limitations, The AI Scientist modified its own code to overcome these restrictions.
Reported cases
According to a report by National Geographic, the system edited its startup script to run in an infinite loop, which caused the system to overload and required manual intervention to stop the process. In another incident, when given a time limit for a task, The AI Scientist extended the allotted time and altered its scheduling to avoid the limit.
Implications and related risks
These events highlight the risk of some AIs acting autonomously, regardless of programmed restrictions. Although the incident occurred in a controlled environment, it underscores the need for additional, strict controls to be implemented in the development of AI systems.
Sakana AI has defended the capabilities of The AI Scientist, which continues to be used to generate scientific articles and improve efficiency in various areas. However, the case has highlighted the challenges of managing AI that can operate outside of human control, opening up a debate on the security and future of these technologies.
Source: https://www.noticiascaracol.com/tecnologia/inteligencia-artificial-se-reprograma-a-si-misma-para-evadir-control-humano-rg10
