Cohere opens confidential-computing inference service to public beta

Cohere announced a public beta of Confidential Computing on Model Vault on October 22, describing it as a managed AI inference service designed to protect data while models process it. The deployment runs on Microsoft Azure confidential-computing infrastructure with NVIDIA H100 GPUs. The launch adds a product option for companies seeking to use hosted models with prompts, records or code they consider sensitive.

Cohere says the protection boundary covers both CPU and GPU execution. It describes hardware-backed isolation for workload memory and encryption for communication between the processor and accelerator. The company also says Model Vault encrypts application request contents to the confidential environment, so intermediary infrastructure can route encrypted payloads without reading their contents. Customers use Cohere's API and familiar request formats, according to the release.

A separate engineering article published the same day explains the role of remote attestation: the environment provides signed measurements of hardware, firmware and software configuration, which a customer-side verifier checks against its own approved policy before releasing a protected request. Cohere says the client refuses to send data if those requirements are not met. The explainer notes that the design still depends on trusted hardware, firmware, measured software and attestation infrastructure.

The company also identifies limits. Confidential computing does not prevent denial-of-service, every physical or side-channel attack, or application-level vulnerabilities such as prompt injection and authorization errors. Some metadata, including request timing, message size, routing information, API credentials and model identifiers, remains visible. Cohere says the system does not determine data residency or replace contractual and compliance controls. The release establishes a beta product and a vendor-described security architecture; it is not an independent security audit or evidence that all risks are eliminated.

The architecture description also sets out what remains observable or trusted. Cohere says the encrypted application payload travels using Oblivious HTTP to a key associated with the attested environment, while routing and authentication information, including API credential and model identifier, remain outside that protected payload. The vendor identifies hardware, firmware, measured software and attestation services as part of the trust base. It says timing, message size and traffic volume may remain visible and that denial of service, side channels, prompt injection and other application flaws are not prevented by confidential computing. The launch is a public beta on Azure with NVIDIA H100 GPUs; Cohere’s own technical descriptions are not independent security certification.