news 
AI Researcher François Chollet Co-Founds Nonprofit to Develop Benchmarks for AGI
François Chollet, a renowned AI researcher and former Google engineer, has announced the establishment of a new nonprofit organization focused on creating benchmarks for Artificial General Intelligence (AGI). The ARC Prize Foundation will serve as a hub for developing evaluation metrics that can assess whether an AI system has achieved human-level intelligence.
Chollet, who introduced the ARC-AGI benchmark in 2019, emphasized the importance of this effort. “We’re growing into a proper nonprofit foundation to act as a useful north star toward artificial general intelligence,” he explained on the organization’s website.
The newly formed entity will also oversee the development and launch of new benchmarks, including the second-generation ARC-AGI benchmark, which is expected to be released this year alongside a fresh competition. Additionally, the organization aims to design the third edition of ARC-AGI, further pushing the boundaries of AI evaluation.
It remains unclear how the ARC Prize Foundation will address the criticism Chollet has faced for overpromising the capabilities of the ARC-AGI benchmark in its quest for AGI. Some experts have questioned whether the metric truly captures the essence of human intelligence or merely evaluates specific skills, potentially leading to inflated claims about AI achievements.
Notably, OpenAI CEO Sam Altman recently expressed his intentions to partner with Chollet’s team to build future benchmarks. However, no update on potential collaborations was provided by Chollet in today’s announcement.
The establishment of the ARC Prize Foundation marks a significant milestone in the ongoing pursuit of AGI. As AI continues to advance at an unprecedented rate, it is crucial that we develop robust evaluation methods to ensure accountability and transparency in this field.
Source: techcrunch.com
