Key points
- Safety filters on open models can be removed via abliteration, while closed models often block legitimate forensic tasks.
Tencent chief AI scientist Vinces Yao Shunyu argues for a middle ground between open and closed AI models to address safety concerns, according to a report by The Decoder.reported
Evidence: reported
Tencent's AI chief says AI bioweapons are coming, but closing off models isn't the answeraifleet-decoder
How we got here
The debate over open versus closed AI models has intensified as capabilities grow. Yao, formerly at OpenAI, contends that a mix of open-source and closed-source allows everyone to use the technology for defense. He notes that while US companies like Anthropic advocate for closed models, open systems provide necessary defensive tools. The gap between Chinese and US model capabilities remains narrow, urging careful industry-wide progress.
What is new
Yao cites a Hugging Face incident where closed models like Claude refused to analyze attack logs due to aggressive safety filters. The platform instead used the open Chinese model GLM-5.2 to parse the data.reported Tencent is applying this lesson to its upcoming Hy4 model by filtering dangerous hacking knowledge directly from the training data.reported
Evidence: reported
Tencent's AI chief says AI bioweapons are coming, but closing off models isn't the answeraifleet-decoder
Evidence: reported
Tencent's AI chief says AI bioweapons are coming, but closing off models isn't the answeraifleet-decoder
Reading between the lines
The Hugging Face report clarifies that GLM-5.2 was used only for forensic analysis, not the attack itself. The breach was resolved by patching vulnerabilities and rotating credentials. Hugging Face emphasized the incident does not argue against safety filters on hosted models, distinguishing between forensic utility and offensive misuse.
The open questions
Claims that open models benefit defenders more than attackers face scrutiny. Anthropic CEO Dario Amodei argues open models are riskier because published weights cannot be recalled and safety mechanisms are fragile. Evidence shows techniques like abliteration allow users to surgically remove refusal behaviors from open models. Current countermeasures hinder but do not prevent these modifications, leaving the efficacy of Yao's proposed middle ground unproven against determined actors.
Why it matters
- Evaluators must assess whether a mixed open-closed strategy effectively mitigates bio-risks compared to single approaches.
- The fragility of open-model safety filters via abliteration challenges assumptions about open-source defensibility.
- Aggressive filtering in closed models may hinder legitimate security operations like log analysis.
- The narrow capability gap between Chinese and US models complicates unilateral safety restrictions.
Topics: AI safety, open models, Tencent, bioweapons, Hugging Face
Updated
Ask the article
Local keyword search of this article; no generated answers.Did this article give you the evidence you needed?
Your choice stays on this device. No public tally.Terms in this story
abliteration
A technique to surgically remove refusal behaviors from open model weights.
Hy4
Tencent's upcoming open-weight AI model.
GLM-5.2
An open Chinese AI model used for forensic log analysis.
Questions readers ask
Why does Tencent seek a middle ground for AI models?
Tencent argues that neither fully open nor fully closed models can solve safety problems on their own, especially regarding bioweapon risks.
Did the Hugging Face incident prove open models are safer?
No, Hugging Face stated the incident was not an argument against safety filters, noting the open model was only used for forensic analysis after the attack was stopped.
What is abliteration?
Abliteration is a technique that allows users with access to model weights to surgically remove trained refusal behaviors, creating unfiltered variants.
Connected stories
Reading receipt
- Safety filters on open models can be removed via abliteration, while closed models often block legitimate forensic tasks.
Still open: Does a middle ground between open and closed models effectively mitigate bioweapon risks?
Still open: Can current countermeasures fully prevent abliteration of open model weights?
