Confidential Containers on Kubernetes: CoCo, AKS, GKE (2026)

Q: What is the difference between confidential nodes and confidential containers on Kubernetes?

Where the trust boundary sits. Confidential nodes, such as Google's Confidential GKE Nodes with AMD SEV, SEV-SNP or Intel TDX, put the entire worker virtual machine inside a trusted execution environment: the cloud provider and hypervisor are outside the boundary, but the Kubelet, every DaemonSet, every other pod on the node and your own cluster administrators are inside it. Confidential containers, as defined by the CNCF Confidential Containers project and implemented in Azure's AKS preview and Red Hat's OpenShift Sandboxed Containers, put a single pod inside its own confidential VM: the node, the Kubelet, the container runtime, the Kubernetes control plane and your administrators are all outside. The first protects you from the infrastructure provider with almost no change to platform operations. The second protects the workload from everyone who is not the workload owner, including the platform team, and changes how exec, logs, sidecars, secrets and resource management work. Most organisations that ask for confidential Kubernetes have a control phrased as protection from the cloud operator and need confidential nodes.

Q: What does the Confidential Containers trust model say is untrusted?

The project's trust model documentation names the Kubelet, CRI runtimes such as containerd, the host kernel, the Kata shim and the Kubernetes control plane as untrusted, and states that orchestration is untrusted, so the system provides few guarantees about where, when or in what order workloads run. What is trusted is the measured guest: the guest firmware, guest kernel, kernel command line and root filesystem, plus the hardware vendor's TEE implementation. The API between the Kata agent inside the guest and the Kata shim on the host is protected by an OPA policy running inside the guest that can block requests from the host, which is why kubectl exec, log streaming and stdin can be refused. The documentation also lists what confidential computing does not protect against: hardware side channels, denial of service, because the host controls scheduling, and certain hardware-dependent vulnerabilities. A design that assumes the enclave guarantees liveness or ordering has misread the model.

Q: Why does kubectl exec stop working on a confidential container, and should I re-enable it?

Because exec is a request from the Kubelet, which the trust model treats as untrusted, and the in-guest agent policy refuses host requests that are not explicitly permitted. Microsoft's AKS documentation says directly that to use exec, container log output or stdio you must modify the security policy to enable the ExecProcessRequest, ReadStreamRequest, WriteStreamRequest and CloseStdinRequest request types and redeploy. The project's own v0.23.0 release notes note that secure Kata agent policies may restrict Kubernetes commands such as exec. Whether to re-enable it is the central operational decision: allowing exec in production reopens the channel through which a compromised host could read the workload, which is the exact threat the architecture exists to close. The workable pattern is to permit exec and stream requests in development policies, forbid them in production policies, and build a debugging story based on telemetry the workload emits itself over an authenticated channel, because node agents cannot see into the pod VM either.

Q: How does attestation and secret release work for confidential containers?

The pod VM boots a measured guest image. The attestation agent inside the guest obtains hardware evidence, an Intel TDX quote or an AMD SEV-SNP report carrying the measurements of the guest firmware, kernel, command line and root filesystem. Through the confidential data hub, the guest presents that evidence to Trustee, the project's attestation service and key broker service. The attestation service verifies the evidence against reference values and a policy; if it passes, the key broker releases the secrets the workload needs: the image decryption key, database credentials, or the LUKS passphrase for an encrypted volume. The secret never exists on the host or in a Kubernetes Secret mounted by the untrusted Kubelet. Two consequences follow. Trustee must run on separate trusted infrastructure rather than on the cluster it attests, which is how Red Hat's documentation deploys it. And reference values are a supply-chain artefact that your own guest-image build pipeline must produce and promote; a policy that accepts whatever the latest image measures proves only that the pod is running something recent.

Q: What is the state of confidential containers on AKS, GKE and OpenShift in 2026?

Azure's Confidential Containers on AKS remain in preview as of the documentation page updated in September 2026. They use AMD SEV-SNP, support only Azure Linux hosts, cannot share a node pool with confidential VMs, pod sandboxing, Trusted Launch, FIPS or Arm64, and are unsupported on AKS Automatic, virtual nodes and node auto-provisioning. The documentation records that pod startup is slower than runc, that the Kata-CC handler allocates 2 GB to the utility VM plus the pod's limits, that resource requests are not passed to the shim, that host networking is unavailable and that Defender for Containers does not assess Kata pods. Google offers Confidential GKE Nodes as a generally available node-level feature with AMD SEV, SEV-SNP and Intel TDX selectable at cluster, node-pool or workload level, with GPUs on confidential nodes documented separately; pod-level isolation on GKE means bringing the CoCo peer-pods stack yourself. Red Hat productises the project in OpenShift Sandboxed Containers 1.13 with deployment guides for bare metal, Azure, Azure Red Hat OpenShift and IBM Z and LinuxONE, and a Red Hat build of Trustee that runs on a separate trusted cluster.

Q: What are peer pods and why do they change the economics of confidential containers?

Peer pods, implemented by the cloud API adaptor, are how confidential containers run on cloud worker nodes that cannot start a confidential VM locally because they lack bare-metal TDX or SEV-SNP hosts with nested virtualisation. Instead of a local hypervisor, the Kata runtime calls the cloud provider's API to create a confidential VM that becomes the pod; the design overview describes this as integrating confidential containers with cloud confidential VM offerings on virtualised worker nodes. It works on AWS, Azure, GCP and others, and it changes the cost model: every pod is a cloud VM with a cloud VM's startup time, billing granularity and networking. It also breaks assumptions about storage, because the pod is not on the node and there is no local block device to mount. The project's August 2026 CSI block driver addresses this by provisioning a cloud disk and performing LUKS encryption inside the pod VM with a key released by the key broker only after attestation, currently for AWS EBS and Azure Managed Disks, with GCP pending and rollback protection explicitly out of scope.

Q: When should a team not use confidential containers?

When the adversary is the cloud operator rather than anyone with cluster access, because confidential nodes satisfy that control at a fraction of the operational cost. When the data the enclave would protect is already readable by cluster administrators through another path, such as a database they can query, because the enclave then protects a copy of what is exposed elsewhere. When the workload is a latency-sensitive autoscaled service, because VM-per-pod startup, in-guest image pull with documented performance concerns and double memory allocation break the scaling model. When the team cannot name who is outside the boundary and what evidence a verifier will demand, because without those two answers the policy, the reference values and the Trustee deployment have no requirements to meet. And when the goal is isolating untrusted code from the platform rather than isolating trusted code from the platform; that is the microVM sandboxing problem, which uses the same virtualisation technology with the opposite trust direction and far fewer operational constraints.

Q: What does a mature confidential containers deployment look like?

Five stages. Stage 0 is encryption at rest and in transit only, with the cloud operator and cluster administrators trusted by default. Stage 1 is confidential nodes: confidential VM node pools with SEV-SNP or TDX, attested node images, platform operations unchanged, which satisfies infrastructure-provider exclusion controls. Stage 2 is a confidential containers pilot on one workload with a named adversary, a generated agent policy reviewed by the workload owner, exec disabled, and an honest inventory of the platform conveniences lost. Stage 3 is attestation-gated secrets and storage: Trustee on separate trusted infrastructure, reference values produced by the guest-image build pipeline and promoted through environments, image keys, credentials and LUKS keys released only after attestation, and workload telemetry emitted over an authenticated channel because the node cannot see in. Stage 4 is verifiable to a third party: attestation tokens and policy versions retained as audit evidence, an external relying-party verifier able to validate measurements against published reference values, re-attestation and key rotation on reference-value change, and a documented answer for every exclusion the trust model names.

Satyam Kumar

返回博客

ai-architecture

Confidential Containers on Kubernetes: The Pod Treats Its Own Cluster as the Adversary, So Every Platform Convenience Becomes a Policy Decision (2026)

By Satyam KumarOctober 7, 202621 min read

confidential containers kubernetes confidential containers coco kata containers confidential coco cncf incubating aks confidential containers preview confidential gke nodes sev-snp tdx openshift sandboxed containers confidential peer pods cloud api adaptor trustee key broker service attestation remote attestation kubernetes pods kata agent policy opa amd sev-snp kubernetes intel tdx kubernetes confidential computing kubernetes architecture in-use encryption containers regulated workloads kubernetes isolation

Confidential Containers on Kubernetes: The Pod Treats Its Own Cluster as the Adversary, So Every Platform Convenience Becomes a Policy Decision (2026)

Frequently Asked Questions

分享这篇文章

Twitter LinkedIn WhatsApp

Satyam Kumar

Founder & AI Architect, AppScale LLP

人工智能和云架构师。帮助团队构建可扩展到数百万的系统。

LinkedIn GitHub

Confidential Containers on Kubernetes: The Pod Treats Its Own Cluster as the Adversary, So Every Platform Convenience Becomes a Policy Decision (2026)

Frequently Asked Questions

分享这篇文章

Comments

Leave a comment