Anthropic builds breakthrough system to help users trust AI
US AI giant Anthropic has developed a system that will help people make sure they are talking to the right AI, and that their responses have not been tampered with.
In a post published on its research blog on Tuesday 3 November, Anthropic announced that it has developed a provable inference system called FlashProve, built on powerful cryptographic proofs much more effective than previous solutions.
Currently, it can be relatively easy for platforms to serve users a different AI model than the one they are requesting. For example, an AI app might claim to serve a Claude Fable 5.1 session to the user but instead be using a cheaper but less capable model such as Claude Sonnet.
The system allows users to prove that the model they are using is the one they are promised, and it also helps to protect against attackers who might try to tamper with the token stream between the model and the user.
It does this by using zkSNARKs (zero-knowledge succint non-interactive arguments of knowledge), a type of cryptographic proof which can be used to efficiently prove the information from a model is legitimate without revealing its underlying weights or processing all token data.
Anthropic said it has used Claude to build a system for these types of proofs that is an order of magnitude faster than existing open-source solutions.
FlashProve has two phases – offline and online. In the first offline phases, the prover publishes a ‘fingerprint’ of the model weights. In the second online phases, the prover receives requests from the verifier and produces a proof file verifying that the model produces the exact expected output tokens when evaluated.
This system will allow everyone from independent evaluators to end-users to effectively determine whether they are being served the right model. This is easier to do with open-weight models but is still expensive, as users can check the results by running the model themselves.
Anthropic noted that servers running AI models are becoming prime targets for attackers as adoption increases. This system can help companies and end-users be sure that the inference they are receiving are generated by a specific, approved set of model weights, not a manipulated set designed to cause harm.
In addition to preventing platforms from serving their customers cheaper models, FlashProve could also be used to verify claims made by companies that their text was generated, or their code was audited, by a specific model.
Anthropic’s FlashProve and high-efficiency provable inference could also be used to help regulators keep tabs on frontier AI companies, requiring them to submit proof of inference if they require firms to focus on serving models over secretly training newer, more powerful models.
In the future, regulators might require companies like Anthropic to serve only approved safe models. This provable inference system will help them to easily verify the models being served by firms are compliant.
While this new provable inference system seems powerful, it does have limitations. It does not provide formal guarantees on fast executable code and can only prove inference over contexts of up to 1,024 tokens.
Anthropic said that its aims in releasing FlashProve, together with the formalisation of its Lean proofs, are to accelerate the development of provable inference and demonstrate the zkSNARKs are practical for provable inference.
“We believe that this progress has meaningful implications for security, in pacing AI development, in providing AI customers with integrity guarantees, in facilitating tooling for auditors and regulators, and more,” Anthropic said.
