Anthropic CEO Dario Amodei has reiterated his concerns about the risks associated with open-weight models, emphasizing their potential for misuse. In a recent blog post, he clarified that the company has never supported a ban on open-weight models, calling them a public good when they lack dangerous capabilities. Amodei outlined two 'nightmare scenarios': authoritarian states, particularly China, could develop more powerful models than the U.S. and use them for military or surveillance purposes, and open models could be exploited for cyber or biological attacks. He pointed out that such misuse is already occurring with closed models, as evidenced by AI labs' usage data. Amodei argued that open models are riskier due to their lack of safeguards and the inability to recall released models. While they empower defenders, he highlighted an imbalance in biological threats, where models could weaponize viruses quickly, while building defenses would take years even in the best-case scenario.
A study by the UK AI Security Institute found that open-weight models lag behind frontier models by a few months in cybersecurity capabilities. Amodei has previously warned about his own models' capabilities, which contributed to a U.S. ban on certain models. He is advocating for stricter chip export controls to prevent China from training more powerful models than the U.S. without U.S. chips. He also wants to address industrial distillation, a method China reportedly uses to bypass chip restrictions and benefit from U.S. AI advancements. Critics argue that restricting chip sales to China may accelerate the growth of China's chip industry and that distillation is a legitimate technique. Amodei also supports mandatory safety tests for all powerful models, regardless of their origin or licensing.
Amodei's comments come after reports that U.S. authorities are considering measures to limit the use of Chinese open-weight models by U.S. companies. A letter signed by major tech companies opposes such measures. Critics accuse Anthropic of downplaying open-weight models to protect its business from cheaper competition. OpenAI is also reportedly lobbying against open weights, despite signing the letter.
Source: thedecoder
