Certificate Expiration

What exactly happens when Kubernetes certificates expire, and how do you manually renew and rotate them using kubeadm?

When a Kubernetes cluster's certificates expire, the fundamental trust and cryptographic identity mechanism that secures the control plane and data plane breaks down. Because Kubernetes architecture mandates strict mutual TLS (mTLS) for communication between components, expired certificates lead to immediate and catastrophic connectivity failures.

If the kube-apiserver certificates expire, external clients (like kubectl or CI/CD pipelines) and internal controllers will receive x509: certificate has expired errors and fail to connect. If the kubelet client certificates expire and fail to automatically rotate, the API server will reject the node's heartbeat and status updates, causing the control plane to eventually mark the node as NotReady and evict its workloads.

To prevent this, kubeadm automatically renews certificates during control plane upgrades. However, if you do not upgrade your cluster within a year, you must manually renew them.

Here is the deep dive into the types of certificates, cryptographic algorithms, and the exact operational workflow to manually renew them and force the API server to reload.

Types of Certificates and Encryption Algorithms

A Kubernetes cluster operates using a decentralized Public Key Infrastructure (PKI) with multiple Certificate Authorities (CAs) and leaf certificates.

Certificate Types:

• Server Certificates: Used by endpoints to prove their identity to connecting clients. This includes the API server endpoint (apiserver.crt), the etcd server (etcd/server.crt), and the kubelet server certificates on every node.
• Client Certificates: Used by components to securely authenticate to servers. This includes the kubelet authenticating to the API server, the API server authenticating to etcd (apiserver-etcd-client.crt), the API server communicating with the kubelet (apiserver-kubelet-client.crt), and core controllers like the kube-scheduler and kube-controller-manager communicating with the API server.
• User/Admin Certificates: Client certificates embedded in kubeconfig files (such as admin.conf and super-admin.conf) used by human operators or automation to gain administrative access.

Algorithms and Validity Periods: By default, kubeadm generates these client and server certificates with a 1-year validity period, while the root CA certificates are valid for 10 years. When generating the public and private keys, kubeadm defaults to using the RSA-2048 algorithm. However, architects can customize the encryptionAlgorithm field in the ClusterConfiguration to enforce stronger cryptographic standards, selecting between RSA-2048, RSA-3072, RSA-4096, or ECDSA-P256.

---

Operational Workflow: Manual Certificate Renewal

To renew your certificates manually without performing a full Kubernetes version upgrade, follow this strict operational workflow on all control plane nodes.

Step 1: Audit Current Expiration Status

Before making changes, investigate the current state of the cluster's PKI by running:

kubeadm certs check-expiration

This command displays the expiration and residual time for all client certificates in the /etc/kubernetes/pki folder and the certificates embedded in admin.conf, controller-manager.conf, and scheduler.conf.

(Note: The kubelet.conf certificate is intentionally omitted from this list because kubeadm configures the kubelet to handle its own automatic certificate rotation in the background).

Step 2: Execute the Renewal

To renew all certificates managed by kubeadm, execute the following command. If you are running a highly-available cluster with a replicated control plane, you must execute this command on every single control-plane node:

sudo kubeadm certs renew all

kubeadm will seamlessly use the existing CA certificates and private keys stored in /etc/kubernetes/pki to mint and sign the new leaf certificates, extending their validity for another year.

Step 3: Reload the Control Plane Components

Generating new certificates on the disk does not magically inject them into the running control plane processes. Because dynamic certificate reloading is not supported for all components, the API server, controller manager, and scheduler must be restarted to pick up the new cryptographic material.

Because kubeadm deploys these components as Static Pods, they are managed directly by the local kubelet, not by the API Server. Therefore, you cannot use standard kubectl commands to restart them.

To forcefully restart a Static Pod, you manipulate the filesystem:

1. Temporarily move the component's manifest file out of the /etc/kubernetes/manifests/ directory.
2. Wait for approximately 20 seconds. The kubelet continuously monitors this directory (governed by the fileCheckFrequency configuration). When it notices the file is gone, it will forcefully terminate the running Pod.
3. Move the manifest file back into /etc/kubernetes/manifests/. After another fileCheckFrequency interval, the kubelet will detect the file and recreate the Pod, successfully completing the certificate renewal for that component.

Step 4: Update Administrator Kubeconfigs

Because the admin.conf file was updated with a new client certificate during the renewal, your local kubectl configuration (which typically relies on this certificate) is now out of sync.

To restore your administrator access, overwrite your local user configuration with the newly renewed admin.conf:

sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

This ensures your kubectl commands utilize the fresh, valid certificate when contacting the newly restarted API server.