Is a FINRA-ish group the answer for frontier AI standards?

Dennis Hassabis, CEO of Google DeepMind, argued in a post on X that frontier AI models need a standard for new releases and FINRA (Financial Industry Regulatory Authority) serves as a template.

FINRA has been mentioned as a model for AI standards before, but Hassabis' take is likely to put a lot more weight behind the idea. For the uninitiated, FINRA creates and enforces rules that govern Wall Street, monitors daily market events for manipulation and insider trading, licenses securities pros and resolves disputes. FINRA falls under the Securities and Exchange Commission, but is a private non-profit funded by member fees.

Hassabis argues that FINRA can serve as a template to resolve what's a big issue: Standards for frontier AI releases aren't solidified. Frankly, the frontier model release process is a bit of a mess. Just ask Anthropic and OpenAI.

According to Hassabis, the industry has to navigate AI's development closely. Hassabis note lands a day after nearly 200 wonks urged further study of the economic impacts of AI on jobs and the broader economy. Hassabis said:

Urgent action is needed to address risks that might arise as we get closer to AGI. We’ve already seen the challenges frontier models pose for cybersecurity, and other threats including nuclear and bio risks may soon emerge as capabilities continue to advance. On the horizon, we will need robust safeguards to maintain control of increasingly agentic, recursively self-improving systems - and tackle unknown issues that will only become clearer over time.

Hassabis proposes the following:

• A standards body would be responsible for developing assessment protocols and working with appropriate federal agencies and the US National Labs to conduct testing in areas relevant to national security.
• Models would qualified as frontier class by this standards body based on benchmarks.
• The group would popularize best practices for model cards and technical details including security.
• AI labs would voluntarily share models with the standards body for review up to 30 days before release. Frontier labs would work with the standards group to resolve issues before release.
• Model assessments would focus on cybersecurity, biological threats and elevated risk areas.
• Evaluations would be regularly updated and include an ecosystem of industry, US government and third-party auditors.