Skip to content
Pioneer Page PP markPPPIONEERPAGE

Pages from the advancing edge of computing.

Foundation models

Tencent chief seeks middle ground for AI safety amid bioweapon concerns

A Tencent scientist argues that neither fully open nor fully closed models solve safety issues, advocating for a balanced approach to emerging risks.

Tencent chief seeks middle ground for AI safety amid bioweapon concerns
Depth
1 outlet · newest source 314h ago

Key points

  • Safety filters on open models can be removed via abliteration, while closed models often block legitimate forensic tasks.

Tencent chief AI scientist Vinces Yao Shunyu argues for a middle ground between open and closed AI models to address safety concerns, according to a report by The Decoder.

Evidence: reportedTencent's AI chief says AI bioweapons are coming, but closing off models isn't the answer

aifleet-decoder

How we got here

The debate over open versus closed AI models has intensified as capabilities grow. Yao, formerly at OpenAI, contends that a mix of open-source and closed-source allows everyone to use the technology for defense. He notes that while US companies like Anthropic advocate for closed models, open systems provide necessary defensive tools. The gap between Chinese and US model capabilities remains narrow, urging careful industry-wide progress.

What is new

Yao cites a Hugging Face incident where closed models like Claude refused to analyze attack logs due to aggressive safety filters. The platform instead used the open Chinese model GLM-5.2 to parse the data. Tencent is applying this lesson to its upcoming Hy4 model by filtering dangerous hacking knowledge directly from the training data.

Evidence: reportedTencent's AI chief says AI bioweapons are coming, but closing off models isn't the answer

aifleet-decoder

Evidence: reportedTencent's AI chief says AI bioweapons are coming, but closing off models isn't the answer

aifleet-decoder

Reading between the lines

The Hugging Face report clarifies that GLM-5.2 was used only for forensic analysis, not the attack itself. The breach was resolved by patching vulnerabilities and rotating credentials. Hugging Face emphasized the incident does not argue against safety filters on hosted models, distinguishing between forensic utility and offensive misuse.

The open questions

Claims that open models benefit defenders more than attackers face scrutiny. Anthropic CEO Dario Amodei argues open models are riskier because published weights cannot be recalled and safety mechanisms are fragile. Evidence shows techniques like abliteration allow users to surgically remove refusal behaviors from open models. Current countermeasures hinder but do not prevent these modifications, leaving the efficacy of Yao's proposed middle ground unproven against determined actors.

Why it matters

  • Evaluators must assess whether a mixed open-closed strategy effectively mitigates bio-risks compared to single approaches.
  • The fragility of open-model safety filters via abliteration challenges assumptions about open-source defensibility.
  • Aggressive filtering in closed models may hinder legitimate security operations like log analysis.
  • The narrow capability gap between Chinese and US models complicates unilateral safety restrictions.

Topics: AI safety, open models, Tencent, bioweapons, Hugging Face

Updated

Ask the article

Local keyword search of this article; no generated answers.

Did this article give you the evidence you needed?

No vote saved on this device.
Your choice stays on this device. No public tally.

Terms in this story

abliteration

A technique to surgically remove refusal behaviors from open model weights.

Hy4

Tencent's upcoming open-weight AI model.

GLM-5.2

An open Chinese AI model used for forensic log analysis.

Questions readers ask

Why does Tencent seek a middle ground for AI models?

Tencent argues that neither fully open nor fully closed models can solve safety problems on their own, especially regarding bioweapon risks.

Did the Hugging Face incident prove open models are safer?

No, Hugging Face stated the incident was not an argument against safety filters, noting the open model was only used for forensic analysis after the attack was stopped.

What is abliteration?

Abliteration is a technique that allows users with access to model weights to surgically remove trained refusal behaviors, creating unfiltered variants.

Connected stories

This storyDemocratic lawmakers signal subpoena-driven AI and data center oversightDemocratic lawmakers signal subpHinton cites corporate race and trust gaps as primary AI risksHinton cites corporate race and Delangue vows intensified open-source advocacy following Huang's exit amid Nvidia acquisition probeDelangue vows intensified open-sAbliteration AI removes safeguards from open-weight models for developersAbliteration AI removes safeguar

Reading receipt

  • Safety filters on open models can be removed via abliteration, while closed models often block legitimate forensic tasks.

Still open: Does a middle ground between open and closed models effectively mitigate bioweapon risks?

Still open: Can current countermeasures fully prevent abliteration of open model weights?

Time spent: 0:00 · Depth: Full

Sources

  1. The Decoder

Related stories

Background reading