Skip to content
Pioneer Page PP markPPPIONEERPAGE

Pages from the advancing edge of computing.

Foundation models

Anthropic's FlashProve generates cryptographic proofs at 110 tokens per second on H100 GPUs

A new system from Anthropic accelerates zkSNARK generation for large models, enabling verifiable inference outputs while maintaining model weight secrecy.

Anthropic's FlashProve generates cryptographic proofs at 110 tokens per second on H100 GPUs
Depth
110 tokensFlashProve achieves proof generation speeds of 110 tokens per second on a single H100 GPU for a 400B-parameter model · reported
400BFlashProve achieves proof generation speeds of 110 tokens per second on a single H100 GPU for a 400B-parameter model · reported
1 outlet · newest source 433h ago

Key points

  • FlashProve achieves proof generation speeds of 110 tokens per second on a single H100 GPU for a 400B-parameter model.
  • The system produces proofs approximately 70 to 95 times slower than standard model inference using vLLM.
  • Generated proofs hide model weights but do not conceal the underlying architecture of the AI system.

Anthropic researchers published details on FlashProve, a system that generates cryptographic proofs for AI inference at 110 tokens per second on a single H100 GPU. This performance figure applies to Llama 4 Maverick, a 400B-parameter mixture of experts model, using 4-bit quantized weights and integer arithmetic during prefill.Anthropic created a system that proves an AI response came from a specific model, though it runs much slower than normal.FlashProve leverages zkSNARKs to generate 19 MB proofs at 110 tokens/second on H100s, incurring a 70-95x latency penalty over vLLM inference. Each proof covers a 1,024-token request and results in a file size of approximately 19 megabytes. The authors note this speed is substantially faster than prior open-source systems, though it remains 70 to 95 times slower in GPU time than running the model directly via vLLM.

Evidence: reportedProgress on provable inference for large language models

aifleet-hn-front

How we got here

Current AI interactions lack reliable verification methods for users to confirm they are interacting with the specific model claimed by a developer. Responses could originate from a substituted model used by a routing service for cost reduction or from an attacker who has tampered with the token stream. Judging model authenticity solely by output quality is unreliable, even when responses appear high-quality.

What is new

FlashProve utilizes zkSNARKs, a type of cryptographic proof chosen for its strong hiding properties and efficiency. The system allows a model provider to commit to a specific model and then prove that each response was produced by that exact model. This approach enables applications ranging from consumer products to internal security checks and could support verification of compliance with AI pacing requirements.

Reading between the lines

The researchers applied novel methods to integrate state-of-the-art tools from cryptography and security literature. They relied on well-established design principles for cryptographic proof systems to build this infrastructure. The system specifically hides the weights of the model within the commitment and subsequent inference proofs, ensuring proprietary parameters remain secure during verification.

The open questions

The claim that FlashProve is "substantially faster" rests on comparisons with prior open-source systems, which the authors state are orders of magnitude slower. However, the evidence shows proof generation is still 70 to 95 times slower than standard inference. The source acknowledges that hiding the model architecture is not a goal of this system, limiting the scope of what is concealed. Uncertainty remains regarding real-world deployment latency impacts.

Evidence: reportedProgress on provable inference for large language models

aifleet-hn-front

Why it matters

  • Model evaluators can now verify that a response originated from a specific committed model, addressing trust issues in routing services.
  • The 70-95x overhead in GPU time suggests provable inference may currently be viable only for high-assurance use cases rather than general chat.
  • The ability to hide weights while proving inference supports proprietary model protection without sacrificing output verifiability.
  • Proof sizes of 19 MB per 1,024-token request indicate significant bandwidth considerations for integrating this into production pipelines.

Topics: Anthropic, cryptographic proofs, zkSNARKs, model verification, AI security

Updated

Then / now / next

Then

Users could not cryptographically verify that an AI response came from the claimed model.

Now

FlashProve enables verifiable inference with proof generation at 110 tokens per second on H100 hardware.

Next

Researchers will likely address the significant latency overhead to make provable inference more practical for broader use.

Story timeline

  1. Then Users could not cryptographically verify that an AI response came from the claimed model.
  2. 04 Nov 2026 Progress on provable inference for large language models
  3. Now FlashProve enables verifiable inference with proof generation at 110 tokens per second on H100 hardware.
  4. Next Researchers will likely address the significant latency overhead to make provable inference more practical for broader use.

The strongest doubt

Counterpoint from the sources

The proof generation process is 70 to 95 times slower than standard inference, which may limit immediate practical adoption for latency-sensitive applications.

For readers

Researchers

  • FlashProve demonstrates novel applications of zkSNARKs for AI inference verification.
  • The system hides model weights but not the architecture, defining clear privacy boundaries.

Developers

  • Proof sizes of 19 MB per 1,024 tokens require consideration for bandwidth and storage.
  • Integration with vLLM-based workflows will need to account for significant GPU time overhead.

Policy

  • Provable inference could provide a technical mechanism for verifying compliance with AI pacing requirements.

Ask the article

Local keyword search of this article; no generated answers.

Did this article give you the evidence you needed?

No vote saved on this device.
Your choice stays on this device. No public tally.

Questions readers ask

What model was used to benchmark FlashProve's performance?

The benchmarks were conducted using Llama 4 Maverick, a 400B-parameter mixture of experts model.

Does the system hide the model's architecture?

No, the system hides the model weights but does not conceal the architecture.

How does the proof generation speed compare to standard inference?

Proof generation is 70 to 95 times slower in GPU time than running the model on vLLM.

Connected stories

This storyDOD Contractor Linked Personal AI Agent to Secure Pentagon System for Two MonthsDOD Contractor Linked Personal ACrowdStrike buys Fiddler AI and Dreadnode for nearly $500M to secure agentsCrowdStrike buys Fiddler AI and Claude Code v2.1.325 release notes detail zstd decoding fix and subagent model routing correctionsClaude Code v2.1.325 release notBlackstone Predicts High Junk Rating for Anthropic Post-IPO, Contradicting Prior ForecastsBlackstone Predicts High Junk Ra

Reading receipt

  • FlashProve achieves proof generation speeds of 110 tokens per second on a single H100 GPU for a 400B-parameter model.
  • The system produces proofs approximately 70 to 95 times slower than standard model inference using vLLM.
  • Generated proofs hide model weights but do not conceal the underlying architecture of the AI system.

Still open: How will the 70-95x latency overhead impact user experience in real-time applications?

Still open: Will future updates aim to hide the model architecture in addition to the weights?

Time spent: 0:00 · Depth: Full

Sources

  1. Hacker News front page (trending)

Related stories

Background reading