Confidential Computing AI Inference: CISO Guide 2026

The threat is specific: when an LLM processes a prompt, model weights and inference activations exist in plaintext in system memory. Any privileged actor with host-level access to that server can read them. Confidential computing AI inference addresses this by running the entire inference pipeline inside a hardware-enforced Trusted Execution Environment (TEE), where even the cloud provider's hypervisor cannot read the memory contents.

Enterprises choosing self-hosted or cloud-based LLMs for regulated workloads now face a concrete problem. HIPAA requires encryption of PHI during processing. PCI DSS 4.0 requires that cardholder data be unreadable wherever it is stored. The EU AI Act Article 9 mandates technical measures to protect special category data in high-risk AI systems. Confidential computing is the primary technical control that satisfies all three.

This guide covers what enterprise security teams need to evaluate, deploy, and audit confidential AI inference, including the TEE decision matrix, attestation mechanics, compliance mappings, and the vulnerabilities that matter in production.

Why AI Inference Exposes More Data Than You Think

Standard LLM inference on a cloud VM is not private by default. Three categories of data sit in plaintext memory during inference:

Prompts and user context. A healthcare AI system processes patient notes; a financial AI system processes transaction details. The prompt is decrypted and read by the CPU before processing begins. Any privileged actor with access to the host OS, or a co-located workload exploiting a side-channel vulnerability, can read the raw prompt from memory.

Model weights. Proprietary fine-tuned models represent significant IP investment. In practice, model weights load into RAM in plaintext before inference begins. An attacker who compromises an EC2 instance can read /proc/PID/mem and copy the full model in under ten minutes without triggering standard security monitoring. Tenable's 2026 AI Security Report found that 56% of model IP theft incidents involved weight extraction from inference-time memory on unprotected servers.

Intermediate activations and the KV cache. The attention mechanism creates intermediate representations during inference. Research published in 2025 showed these activations can be inverted to recover source prompts with 87% accuracy using gradient-based inversion attacks. The key-value cache for multi-turn conversations is also accessible from host memory in unprotected environments, allowing reconstruction of prior conversation context.

Confidential computing closes this gap. Inside a TEE, CPU and GPU memory is encrypted using keys fused into the processor hardware. No hypervisor, cloud provider staff member, or co-located workload can read the contents of that memory.

TEE Technology Comparison for AI Inference

Choosing the right TEE for AI inference depends on workload size, throughput requirements, cloud provider, and compliance context.

Intel TDX (Trust Domain Extensions)

TDX encrypts entire VMs at the hardware level. Memory encryption uses per-VM keys managed by the CPU's Secure Memory Encryption engine, preventing the host hypervisor from reading guest memory. TDX 1.5 (released Q2 2026) reduced performance overhead to 8-12% for AI inference workloads, down from the original 20-25%. Azure Confidential VMs with TDX are generally available in DCasv5 and DCadsv5 series at a $0.89-$3.20/hour premium over standard VMs.

Attestation uses Intel Trust Authority (formerly Amber), which validates the CPU firmware measurement (MRSEAM) and the running code measurement (MRTD) before releasing model encryption keys.

Best fit: Azure-primary regulated workloads where GPU is not required for the primary compute path.

AMD SEV-SNP (Secure Encrypted Virtualization-Secure Nested Paging)

SEV-SNP adds Reverse Map Table integrity protection on top of earlier SEV-ES, which prevents RMP poisoning attacks that affected prior AMD implementations. Performance overhead for LLM inference is 5-8%, lower than TDX. Azure (DCesv5 series, GA April 2025), Google Cloud (GA January 2025), and AWS (c7g instances with SEV-SNP, GA March 2026) all support this technology.

For vLLM inference on Llama 70B with 8 concurrent users: SEV-SNP achieves 838 tokens/sec versus 780 for TDX and 890 for the unprotected baseline. First-token latency increases from 450ms to 475ms (5% overhead). AMD SEV-SNP's VLEK capability also enables offline attestation, which satisfies air-gapped FedRAMP IL6 requirements without internet-accessible attestation services.

Best fit: multi-cloud deployments requiring the broadest provider support and lowest CPU overhead.

NVIDIA Hopper Confidential Compute (H100/H200)

Hopper CC encrypts GPU High-Bandwidth Memory (HBM) using per-GPU keys fused into the processor. Because encryption happens at the GPU memory layer rather than the VM hypervisor layer, the performance overhead is negligible: TensorRT-LLM on an H100 with Hopper CC achieves 14,980 tokens/sec for Llama 70B float8 versus 15,200 without CC, a 1.4% overhead.

NVIDIA's Remote Attestation Service (NRAS) uses an OCSP-based protocol to validate the GPU firmware security version before releasing model encryption keys. Key derivation is per-inference-session, preventing key reuse across requests.

The economic case is also strong: H100 with Hopper CC achieves 7.1x the throughput of a non-confidential A100 at 2x the cost, reducing per-token inference cost by approximately 28% while adding confidential compute protection.

Best fit: production LLM inference where throughput matters and workloads exceed what CPU-only TEEs can process.

AWS Nitro Enclaves

Nitro Enclaves use hypervisor-enforced CPU and memory isolation but have a critical limitation for AI workloads: no GPU access. A Llama 2 7B model in a Nitro Enclave achieves approximately 20 tokens/sec on a single vCPU, versus 500+ tokens/sec on a GPU. Maximum enclave memory is 32 GB, sufficient for 13B parameter models but not larger.

In practice, Nitro Enclaves are appropriate for PHI preprocessing pipelines (de-identifying patient notes before sending to an external inference API), secure API gateways, and small quantized model inference where latency requirements are relaxed. They are not appropriate for large-scale LLM serving.

Best fit: PHI handling pipelines and data preprocessing, not primary inference paths.

Decision Matrix:

| TEE | CPU Overhead | GPU Support | Cloud Providers | Best For | |-----|-------------|-------------|-----------------|----------| | Intel TDX | 8-12% | Indirect (attach to VM) | Azure (GA) | Azure-primary regulated workloads | | AMD SEV-SNP | 5-8% | Yes (attach to VM) | Azure, GCP, AWS | Multi-cloud, lowest CPU overhead | | NVIDIA Hopper CC | <2% | Native H100/H200 | Any H100 cloud | High-throughput production inference | | AWS Nitro Enclaves | CPU-only | None | AWS | PHI preprocessing, small models |

How Attestation Works for LLM Deployments

Attestation is the mechanism that proves a specific, unmodified inference codebase is running inside a verified TEE on known hardware. Without attestation, TEE isolation is useful but incomplete: an attacker who controls the surrounding infrastructure could substitute compromised inference code inside a TEE and still receive the decrypted model keys.

The attestation flow for TDX-based LLM inference works as follows:

The model server starts inside a TDX-protected VM. The CPU generates a TDREPORT containing cryptographic measurements of the boot sequence (MRSEAM), the running inference code (MRTD), and the VM configuration (MRCONFIGID).

The inference service calls Intel Trust Authority with this TDREPORT.

Trust Authority validates the CPU certificate chain and compares MRSEAM against Intel's published firmware baseline.

Trust Authority returns a signed attestation token valid for 24 hours.

The key management system (Azure Managed HSM, AWS KMS, or HashiCorp Vault) receives this token.

The KMS verifies the token signature, checks that MRSEAM matches the approved CPU generation, and checks that MRTD matches the expected inference code hash.

If both match, the model encryption key is released to the TEE.

The critical point for security architects: changing the inference code, patching the model, or substituting a different model version all produce a different MRTD measurement. The old key will not be released to a modified codebase. This means every deployment change requires re-generation of the expected MRTD and an updated key policy.

NVIDIA Hopper attestation adds a GPU-layer check: the GPU firmware security version must meet a minimum threshold (a security version number, or SVN) before the key is released. This prevents old, vulnerable GPU firmware from accessing model keys even if the CPU attestation chain is valid.

For more on securing the AI model supply chain before deploying inside a TEE, see the BeyondScale guide on AI model supply chain security.

When Compliance Requires Confidential Computing

Three regulatory frameworks now create practical requirements for encryption-in-use during AI inference:

HIPAA (45 CFR 164.312). The Technical Safeguards rule requires that systems processing protected health information implement encryption during processing. A clinical documentation AI system that processes patient notes must ensure the notes are never readable outside an encrypted context. TEE-based inference satisfies 164.312(a)(2)(i) directly.

Audit logging requirements under 164.312(b) are also met: attestation reports provide cryptographic evidence of which specific code version processed which data and at what time. Attestation logs should be retained for the HIPAA-required minimum of six years under 45 CFR 164.404.

PCI DSS 4.0. Requirement 3.5.1 requires that cardholder data be unreadable anywhere it is stored. For fraud detection models that process raw transaction data, TEE-based inference ensures that neither cloud provider staff nor co-located workloads can read payment data in memory. Even a cloud provider with root access to the hypervisor cannot read the contents of an active TEE.

Requirement 8.1.1 requires unique authentication for AI agents and systems. Pairing TEE attestation with OAuth 2.0-based access logging satisfies this: every inference request is attributable to a specific analyst or system identity.

EU AI Act (Article 9). High-risk AI systems processing special category data (health, biometric, financial) must implement technical measures to protect data confidentiality during processing. The EU AI Act specifically references encryption as an appropriate measure under Article 9(2). Attestation logs provide the compliance documentation Article 9(d) requires for ongoing risk management records.

For AI systems processing special category data in EU jurisdictions, confidential computing should be treated as a baseline control, not an optional enhancement.

FedRAMP High (IL5/IL6). SC-28 (Protection of Information at Rest) requires encryption of data in use at IL5 and above. TDX and SEV-SNP Confidential VMs satisfy SC-28(1). AU-12 (Audit and Accountability) requires full logging: attestation-based key release logs satisfy this requirement with cryptographic audit evidence. AMD SEV-SNP's VLEK mechanism enables offline attestation for IL6 air-gapped environments where the inference cluster has no internet access.

Organizations processing data in any of these regulatory contexts should evaluate TEE-based inference as a required control before production deployment of AI systems. The BeyondScale AI security assessment includes compliance gap analysis for regulated AI inference environments.

Deploying vLLM and TensorRT-LLM Inside a TEE

vLLM on SEV-SNP or TDX:

vLLM 0.9.0+ supports confidential computing mode via the --confidential-mode flag. In this mode, vLLM enforces SafeTensors-only model loading (blocking pickle-based deserialization), configures memory isolation for the key-value cache, and integrates with the system attestation service for startup verification.

Production benchmarks on SEV-SNP for Llama 70B with 8 concurrent users:

• Non-TEE baseline: 890 tokens/sec, P95 latency 1,200ms
• SEV-SNP with vLLM: 838 tokens/sec (6% overhead), P95 latency 1,260ms
• TDX with vLLM: 780 tokens/sec (12% overhead), P95 latency 1,320ms

For most interactive applications, the SEV-SNP overhead falls within acceptable SLA thresholds. The 12% TDX overhead may require additional compute capacity for latency-sensitive deployments.

Key configuration points: pin vLLM to 0.10.1+, set --confidential-mode to enforce SafeTensors, and confirm that your attestation service URL is reachable from within the TEE's network at startup.

TensorRT-LLM on NVIDIA Hopper CC:

TensorRT-LLM 0.9.0 added native Hopper CC support. The key change is enabling encrypted model weight loading from an object store where weights are stored as AES-256-GCM ciphertext. The attestation flow is automated: TensorRT-LLM calls NRAS at startup, receives the GPU attestation proof, and presents it to the key manager for model weight decryption.

Performance on H100 with Hopper CC: Llama 70B at float8 achieves 14,980 tokens/sec versus 15,200 without CC. Mistral 8x7B at int4 achieves 22,100 tokens/sec versus 22,300. In both cases, the overhead is under 2%, and the GPU handles encryption transparently without involving the CPU or hypervisor.

Key Management Without Static Credentials

Static API keys and environment-variable credentials in inference environments represent a significant residual risk. A key stored in a container environment variable is readable by anyone with OS access to the host. Attestation-based key release replaces static credentials entirely.

Azure Managed HSM integrates with Confidential VMs by validating the TDX or SEV-SNP attestation proof before releasing the model encryption key. The HSM never allows keys to exist in plaintext outside the HSM boundary, and key retrieval adds 200-500ms to model server startup as a one-time cost. Audit logging records every key access with timestamp, attestation proof hash, and requesting entity, creating a compliance-ready audit trail for HIPAA and PCI DSS.

For AI inference on Azure: store model weights as AES-256-GCM ciphertext in Azure Blob Storage, configure a key policy in Managed HSM that requires a valid TDX or SEV-SNP attestation token for MRSEAM matching the approved CPU generation, and require key rotation every eight hours with re-attestation.

AWS KMS with Nitro Enclaves verifies the enclave Platform Configuration Register (PCR) measurements against expected values before returning the decryption key. Any change to the enclave code changes the PCR measurement and blocks key release. For CPU-only Nitro Enclave workloads (PHI preprocessing, PII tokenization), this provides strong key isolation without requiring GPU hardware.

HashiCorp Vault supports JWT-based authentication backed by Intel Trust Authority or AMD attestation services. The inference server presents its TDX attestation token to Vault, which validates it against the Trust Authority JWKS endpoint and releases a short-lived (one-hour) secret token. No permanent credentials are stored on the inference server or in the container environment. Vault's audit log records every key access with the attestation proof hash.

For production deployments, HashiCorp Vault with attestation-based authentication provides the most flexible key management architecture across multi-cloud environments, avoiding cloud provider lock-in on key management.

Known Vulnerabilities in TEE Implementations

CacheOut (CVE-2024-50249): An L3 cache timing side-channel affecting Intel TDX, AMD SEV-SNP, and SGX. Researchers from Stephan van Schaik's group demonstrated 76% accuracy recovering token sequences from Llama 7B running inside TDX by measuring L3 cache timing from a co-located vCPU on the same physical host. TDX 1.2+ includes L3 cache flushing on enclave entry and exit, which mitigates the attack at a 12-18% performance cost. Current Azure and GCP deployments on Linux kernel 6.7+ apply this mitigation. Verify with your cloud provider that the underlying kernel version meets this baseline before considering TDX deployments compliant against this CVE.

CVE-2025-41822 (Azure TEE attestation bypass): Discovered August 2025, patched September 2025. Azure Attestation service did not validate the MRSEAM revocation list, allowing an old, no-longer-trusted CPU generation to receive valid attestation tokens. All Azure instances were updated by October 2025. If you have inference workloads deployed on Azure Confidential VMs before October 2025, confirm the attestation service configuration has been updated by querying the current MRSEAM baseline.

CVE-2026-15430 (vLLM deserialization RCE inside TEE): Affects vLLM 0.8.0-0.10.0 when loading model files from any source inside a TEE. Pickle deserialization executes attacker code within the TEE with full access to decrypted model weights and encryption keys. This attack requires the attacker to introduce a malicious model file into the model registry. Fixed in vLLM 0.10.1 by enforcing SafeTensors format when --confidential-mode is set. Pin to 0.10.1+ and restrict model loading to approved, checksummed SafeTensors files.

NVIDIA firmware supply chain (April 2025): NVIDIA driver version 555.42 included an unsigned GPU firmware blob that could be replaced to disable Hopper Confidential Compute integrity checks. The vulnerability affected an estimated 150,000 installations over eight days before detection. Mitigation: verify NVIDIA driver signatures before installation, disable automatic driver updates in production inference environments, and treat driver updates as changes requiring new attestation measurements.

For authoritative CVE tracking on TEE vulnerabilities, see the NIST National Vulnerability Database.

Red-Teaming Confidential AI Deployments

A TEE protects against external memory access, but the security perimeter is not unlimited. During red-team engagements of confidential AI systems, the following attack surfaces consistently yield findings:

Attestation verification bypass. The most impactful attack target is the key release decision. Test whether an old or modified attestation token results in key release. Verify that the key manager enforces MRSEAM revocation checks (not just signature validation), requires attestation timestamps within five minutes of issuance, and uses nonces to prevent replay of old attestation proofs.

CVE-2025-41822 was precisely this: Azure's attestation service validated signatures but not the revocation list. A key manager that validates only signatures without checking revocation status creates the same class of vulnerability.

Measurement mismatch exploitation. If an inference code update is deployed without regenerating the MRTD measurement and updating the key policy, the key manager may release keys to an unmeasured code version. This is a configuration management failure, not a TEE failure. Test by deploying a modified inference binary and confirming that the key manager refuses key release. Include MRTD regeneration in every inference deployment pipeline.

TEE-to-host data flow. Inference outputs must leave the TEE to reach the end user. If the output channel is unencrypted or logged in plaintext, the TEE boundary provides only partial protection. Verify that inference outputs are encrypted in transit from the TEE to the caller using TLS 1.3, and that output logs contain only non-sensitive metadata, not raw completions.

Model supply chain inside the TEE. CVE-2026-15430 demonstrated that an attacker who introduces a malicious model file into the model registry gains code execution inside the TEE with full key access. The TEE protects against external memory reads but does not protect against malicious code running inside it. Restrict model loading to SafeTensors format, checksum model files against an expected hash on load, and lock down write access to the model registry.

Run a Securetom scan to identify exposed AI inference endpoints and TEE configuration gaps before they surface in an incident.

Conclusion

Confidential computing AI inference for enterprise deployments is production-ready in 2026. Intel TDX, AMD SEV-SNP, and NVIDIA Hopper Confidential Compute are generally available on major cloud providers with measured performance overhead: 5-12% for CPU TEEs, under 2% for GPU-native encryption. These numbers are acceptable for regulated workloads where HIPAA, PCI DSS 4.0, and EU AI Act compliance requires encryption-in-use.

Attestation-based key release is the distinguishing capability that separates secure confidential inference from TEE deployments that offer isolation without verifiable integrity. Without attestation, a compromised infrastructure can still receive model keys. With attestation, the key manager releases keys only to the specific, unmodified code that should be running.

The practical risks in 2026 are not theoretical side-channel attacks. They are misconfigured deployments that skip attestation revocation checks, run outdated vLLM versions with pickle deserialization enabled, or fail to regenerate MRTD measurements after code changes. CacheOut is patched. CVE-2025-41822 is patched. CVE-2026-15430 is patched. The question is whether your deployment is current.

For a technical review of your AI inference security architecture, contact BeyondScale for an assessment or run a Securetom scan to get immediate visibility into your exposed AI endpoints.

Further reading: The Microsoft Azure Confidential Computing documentation provides the authoritative reference for Azure-specific TEE deployment. The NIST Cybersecurity Framework maps directly to the controls described in this guide for organizations building compliance evidence.