Kubernetes Notes

Appearance

How does Kubernetes handle desired state management, and what role do controllers play in maintaining it?

Kubernetes handles desired state management by fundamentally rejecting the traditional "imperative" orchestration model—where a system executes a defined workflow of A, then B, then C. Instead, it utilizes a declarative configuration model based on a concept known as a "record of intent".

In this architecture, you do not tell Kubernetes how to achieve a state; you simply define what the result should look like. The system then assumes the responsibility of driving the actual environment toward that desired state.

The Mechanism: Objects as Records of Intent ​

The foundation of state management lies in the Kubernetes Object model. When you create an object (like a Pod or Deployment), you are essentially creating a persistent entity in the API that represents your desired workload.

Most Kubernetes objects are structured with two distinct nested fields that facilitate this comparison:

  • .spec (The Desired State): This section is provided by you (the engineer). It describes the characteristics you want the resource to have, such as the container image, the number of replicas, or the ports to expose.
  • .status (The Current State): This section is supplied and updated by the Kubernetes system. It describes the actual, observed state of the object at any given moment.

The Kubernetes control plane continually manages every object to match the actual state (.status) to the desired state (.spec) you supplied.

The Engine: Controllers and Control Loops ​

The active agents responsible for maintaining desired state are Controllers. From an engineering perspective, a controller is an implementation of a control loop—a non-terminating loop that regulates the state of a system, analogous to a thermostat in a room.

While a thermostat monitors temperature to turn equipment on or off, a Kubernetes controller monitors the state of your cluster to make or request changes.

How Controllers Maintain State ​

Controllers operate on a continuous reconciliation cycle:

  1. Watch: The controller tracks at least one Kubernetes resource type. For example, the Job controller watches for Job objects.
  2. Compare: It constantly compares the current state against the desired state defined in the object's .spec.
  3. Reconcile: If the states differ, the controller takes action to bring the current state closer to the desired state.

The Deployment Example: If you create a Deployment and set the .spec.replicas to 3, this is your record of intent.

  • Observation: The system reads the spec and observes the status.
  • Action: It starts three instances of the application.
  • Correction: If an instance crashes (a change in .status), the system detects the divergence from the .spec and automatically starts a replacement instance to restore the desired count to 3.

Architecture of Control ​

Crucially, Kubernetes does not rely on a monolithic control process. Instead, it comprises a set of independent, composable control processes.

  • Decoupled Logic: Controllers often carry out actions by sending messages to the API server rather than executing workloads directly. For example, when the Job controller sees a new task, it does not run the container; it tells the API server to create Pods,. The kubelet (a node-level agent) then sees the new Pod assignment and runs the container.
  • Resilience: Because controllers are simple and independent, if one fails, it does not bring down the entire system. Kubernetes is designed to allow for controller failure without losing the overall state definition.

Built-in vs. Custom Controllers ​

Kubernetes ships with a set of built-in controllers that run inside the kube-controller-manager binary. These handle core behaviors like:

  • Node Controller: Responds when nodes go down.
  • Job Controller: Ensures Pods are created to run one-off tasks.
  • ServiceAccount Controller: Creates default accounts for namespaces.

However, the architecture is extensible. You can write your own controllers to manage custom hardware or application logic. When you combine a custom controller with a Custom Resource, this is referred to as the Operator pattern. This allows you to automate the management of complex, stateful applications (like databases) using the same declarative principles as native Kubernetes objects.