Deployments

Deployments are the standard way to run stateless applications in Kubernetes. By wrapping your Pods in a Deployment, you gain self-healing, on-demand scaling, zero-downtime rolling updates, and versioned rollbacks — all managed automatically by the control plane.

---

The Three-Tier Hierarchy

Kubernetes separates concerns across three layers of objects:

Layer	Object	Responsibility
Top	Deployment	Governs rollouts, rollbacks, and update strategy
Middle	ReplicaSet	Ensures the correct number of Pods are running (self-healing + scaling)
Bottom	Pod	Runs the actual containerised application

When you kubectl apply a Deployment manifest, a cascading creation occurs: the Deployment creates a ReplicaSet, which creates and manages the individual Pods.

Caution

Always manage your workload through the Deployment. Never directly edit or delete the underlying ReplicaSets or Pods — the controllers will reconcile against your changes and may overwrite or undo them.

---

ReplicaSets

A ReplicaSet is the object that provides self-healing and scaling. It replaces the older, now-deprecated ReplicationController. Even for a single Pod, deploying via a ReplicaSet (or Deployment) is strongly preferred over creating a standalone Pod — if the node fails, the ReplicaSet reschedules the Pod on a healthy node automatically; a standalone Pod simply disappears.

The Reconciliation Loop

The ReplicaSet controller runs a continuous observe → compare → act loop:

1. Observe — watch the ReplicaSet and its dependent Pods
2. Compare — count Pods matching the selector against spec.replicas
3. Act — create missing Pods or delete excess ones to restore balance

This loop handles failures automatically with no human intervention.

Trigger	Controller reaction
Pod crashes or is deleted	Immediately creates a replacement
Extra Pod matching the selector appears	Terminates the excess to restore the desired count
Node goes NotReady	Marks Pods for deletion, spins up replacements on healthy nodes

Note

A ReplicaSet guarantees the number of Pods, not their health. If a Pod’s containers continuously crash or fail probes, the controller will not replace it — it uses exponential backoff and expects the Pod to recover. See the isolation technique below for how to handle this.

ReplicaSet Spec

ReplicaSets live in the apps/v1 API group and require three fields in spec:

apiVersion: apps/v1
kind: ReplicaSet
metadata:
  name: my-app
spec:
  replicas: 3                        # desired Pod count (default: 1)
  selector:
    matchLabels:
      app: my-app                    # which Pods this RS manages
  template:                          # blueprint for new Pods
    metadata:
      labels:
        app: my-app                  # must be a superset of selector labels
    spec:
      containers:
        - name: app
          image: my-app:1.0

Critical rule: the labels in template.metadata.labels must be a superset of the labels in selector.matchLabels. If they don’t match, the API server rejects the object — the ReplicaSet would create Pods it cannot see.

Immutability Constraints

Field	Mutable?	Notes
spec.selector	❌ No	Delete and recreate to change
spec.replicas	✅ Yes	Immediate effect
spec.template	✅ Yes	Only affects new Pods — existing Pods are not updated

Pod Naming

Pods in a ReplicaSet are fungible — there is no concept of order. Kubernetes uses the generateName field to produce names like my-app-x7k2p (ReplicaSet name + 5 random characters).

Scale-Down Priority

When you reduce replicas, Kubernetes doesn’t terminate Pods randomly. It follows this ordered priority:

1. Pods not yet assigned to a node
2. Pods with an unknown phase
3. Pods that are not ready
4. Pods with a lower controller.kubernetes.io/pod-deletion-cost annotation
5. Pods on nodes with more replicas of this ReplicaSet (promotes even distribution)
6. Pods ready for a shorter time
7. Pods with more container restarts
8. Most recently created Pods

Pod Ownership and Garbage Collection

When a ReplicaSet creates Pods, it becomes their owner, recorded in each Pod’s metadata.ownerReferences field. Deleting the ReplicaSet triggers cascading deletion of all its Pods via the garbage collector.

To delete the ReplicaSet while keeping Pods running (e.g., to recreate it with a changed selector):

kubectl delete rs <name> --cascade=orphan

The surviving Pods become independent orphans. If you later create a new ReplicaSet whose selector matches those orphaned Pods, it will adopt them automatically.

Debugging: Isolating a Failing Pod

If a Pod is continuously failing but you need to keep your service up and debug without destroying evidence:

1. Change one of the failing Pod’s labels so it no longer matches the selector:

   kubectl label pod <failing-pod> rel=debug --overwrite

2. The ReplicaSet detects a missing replica and immediately spins up a healthy replacement.

3. The isolated Pod continues running independently for inspection.

4. After debugging, delete it manually — the garbage collector won’t touch it since it’s now an orphan.

---

Deployments

A Deployment sits above the ReplicaSet and adds the machinery for rollouts and rollbacks. In practice, you should always use Deployments rather than bare ReplicaSets — they handle version management automatically.

The Declarative Model

The declarative model is the foundation of Deployment management:

Approach	How	Kubernetes can self-heal?
Declarative	Describe the desired end state in YAML; Kubernetes figures out how to get there	✅ Yes
Imperative	Issue step-by-step commands	❌ No — no concept of desired state

Deployment Spec

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app                        # valid DNS name (alphanumerics, dots, dashes)
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app                     # must match template labels exactly
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
        - name: app
          image: my-app:1.0
          ports:
            - containerPort: 8080
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1               # max Pods offline during update
      maxSurge: 1                     # max extra Pods during update
  revisionHistoryLimit: 5             # how many old ReplicaSets to retain
  minReadySeconds: 10                 # wait before marking a replica available
  progressDeadlineSeconds: 300        # timeout before marking rollout as stalled

Golden rule: spec.selector.matchLabels and spec.template.metadata.labels must match exactly. If they don’t, Kubernetes rejects the object. The top-level metadata.labels (the Deployment’s own labels) are irrelevant to this mapping.

Labels in Three Places

When inspecting a Deployment YAML, you’ll see labels appear three times:

Location	Purpose
metadata.labels	Labels on the Deployment object itself
spec.selector.matchLabels	How the Deployment finds its Pods
spec.template.metadata.labels	Labels stamped onto each new Pod

The pod-template-hash Safeguard

When a Deployment creates its ReplicaSet, Kubernetes automatically injects a pod-template-hash label (a cryptographic hash of the Pod template) into the ReplicaSet’s selector and into every Pod it creates. This prevents a ReplicaSet from accidentally adopting unrelated Pods that happen to share a primary label.

Caution

Never manually modify the pod-template-hash label — it’s a system-generated safeguard for ownership integrity.

---

Creating Deployments

Imperatively

# Minimum required: name + image
kubectl create deployment my-app --image=my-app:1.0

# With replicas and port
kubectl create deployment my-app --image=my-app:1.0 --replicas=3 --port=8080

# Generate YAML without applying
kubectl create deployment my-app --image=my-app:1.0 --dry-run=client -o yaml > deployment.yaml

Declaratively

kubectl apply -f deployment.yaml

Once applied, the Deployment and ReplicaSet controllers start their reconciliation loops immediately, scheduling Pods onto healthy worker nodes.

---

Inspecting Deployments

# High-level status (READY, UP-TO-DATE, AVAILABLE columns)
kubectl get deploy <name>

# Full configuration + events (use when troubleshooting label mismatches)
kubectl describe deploy <name>

# List ReplicaSets — names are prefixed with the Deployment name + a hash
kubectl get rs

# List Pods with their labels
kubectl get pods --show-labels

# List Pods belonging to a ReplicaSet by selector
kubectl get pods -l app=my-app

ReplicaSet naming: the Deployment name + a crypto-hash of the Pod template, e.g., my-app-54f5d46964. If you update the Pod template, a brand-new ReplicaSet with a new hash is created.

# View logs from all Pods in a ReplicaSet at once
kubectl logs rs/<name>
kubectl logs rs/<name> --all-pods --all-containers

---

Scaling in Practice

Delegation Chain

When you scale a Deployment, the Deployment controller does not create or delete Pods directly — it only updates the replicas count on the underlying ReplicaSet. The ReplicaSet controller then executes the actual Pod additions or removals. Any manual changes to a Deployment-owned ReplicaSet’s replica count are immediately overwritten.

Caution

Never scale a ReplicaSet that is owned by a Deployment. The Deployment controller will detect the discrepancy and revert it within seconds.

The Replicas Field Trap

A common production mishap when mixing imperative scaling with declarative manifests:

1. You imperatively scale to handle traffic: kubectl scale deployment my-app --replicas=50
2. Later, you apply an updated manifest that still has replicas: 3 hardcoded
3. The manifest overwrites the live scale — 47 Pods terminate immediately

The fix: omit replicas from your manifest entirely. Kubernetes defaults to 1 on creation; scale via kubectl scale or HPA afterwards. Future kubectl apply runs will never overwrite the live count.

# Repair an existing Deployment whose last-applied annotation already contains replicas
kubectl apply edit-last-applied deploy <name>
# Delete the replicas field from the annotation and save — future applies will skip it

Note

Removing replicas from the manifest does not reset the running count to 1. It only means the field is no longer tracked by kubectl apply, so the live replica count stays unchanged.

---

Troubleshooting Deployments

When Pods fail to appear after creating or updating a Deployment, the cause is almost always in the underlying ReplicaSet, not the Deployment itself.

Where to look	What you find
kubectl describe deploy <name> → Conditions	ReplicaFailure: True with reason FailedCreate — tells you something is wrong
kubectl describe rs <name> → Events	The exact error: “forbidden”, “service account not found”, “insufficient quota”, etc.

# Step 1: check Deployment conditions
kubectl get deploy <name> -o yaml | grep -A 10 conditions

# Step 2: find and inspect the failing ReplicaSet
kubectl get rs
kubectl describe rs <rs-name>

Tip

The Deployment’s Events section often shows nothing useful when Pods fail to start. Always follow the hierarchy down to the ReplicaSet for the root cause.

---

Rolling Updates

A rolling update replaces Pods incrementally — old Pods are terminated and new ones created until all replicas are running the new version. In Kubernetes, all updates are replacement operations — Pods are immutable, so “updating” a Pod means deleting it and creating a new one.

Prerequisites for Zero Downtime

For rolling updates to be truly zero-downtime, your application should be:

• Loosely coupled — services communicate via well-defined APIs
• Backward and forward compatible — during a rollout, old and new versions run simultaneously; clients hitting either version must get a valid response

Mechanism

1. You update the Pod template (e.g., change the image tag) and kubectl apply
2. The Deployment controller creates a new ReplicaSet for the new version
3. The controller incrementally scales up the new RS while scaling down the old one
4. The rollout completes when the old RS reaches 0 Pods and the new RS reaches the full replica count

Recreate Strategy

All existing Pods are deleted simultaneously, and only after they are fully terminated does Kubernetes start creating the new Pods — guaranteeing a period of zero availability between versions.

spec:
  strategy:
    type: Recreate   # no rollingUpdate block — no configuration options

Aspect	Detail
Configuration options	None
Client experience during update	503 Service Temporarily Unavailable via Ingress; connection rejected via ClusterIP
When to use	Applications that cannot run two versions simultaneously (e.g., database schema migrations that are not backward compatible)

Note

Changing strategy.type and applying the manifest does not immediately trigger a replacement — the update only begins when you modify the Pod template (e.g., change the image tag).

RollingUpdate Strategy

The default strategy. Gradually replaces old Pods with new ones, keeping the application continuously accessible throughout the update.

spec:
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1         # absolute number or percentage (default: 25%)
      maxUnavailable: 1   # absolute number or percentage (default: 25%)

Parameter	What it controls
maxSurge	Maximum extra Pods allowed above the desired count during the update
maxUnavailable	Maximum Pods allowed below the desired count (unavailable) during the update

Constraint: both cannot be 0 simultaneously — the controller would be unable to add new Pods or remove old ones.

Concrete parameter combinations (assuming 3 desired replicas):

maxSurge	maxUnavailable	Behaviour
0	1	Delete first, then create. Never exceeds 3 Pods; at least 2 available at any time.
1	0	Create first, then delete. Always ≥ 3 Pods available; temporary 4-Pod state.
1	1	Create and delete in parallel. Fastest; 2–4 Pods simultaneously.

minReadySeconds — The Airbag

By default a new Pod is marked available the instant its readiness probe passes, and the rollout immediately continues to the next Pod. minReadySeconds enforces a mandatory wait between “ready” and “available”:

spec:
  minReadySeconds: 60   # pod must stay ready for 60 s before it counts as available

If the Pod’s containers crash or fail their readiness probe at any point during the wait window, the timer resets and the Pod never becomes available — the rollout halts and no further old Pods are replaced.

Note

With the default minReadySeconds: 0, a Pod that passes its readiness probe for even one second is immediately marked available. If it then crashes, the rollout has already moved on — potentially replacing all healthy Pods with a broken version before anyone notices.

progressDeadlineSeconds — Stall Detection

If a rolling update makes no progress for longer than progressDeadlineSeconds (default: 600 s), the Progressing condition changes to False with reason ProgressDeadlineExceeded. Kubernetes takes no automated action — the rollout simply stops.

kubectl rollout status deployment <name>   # shows "error: deployment exceeded its progress deadline"

Caution

If you set minReadySeconds to a value greater than 600 s, you must also increase progressDeadlineSeconds to avoid false stall detections.

Monitoring, Pausing, and Resuming

# Watch rollout progress in real time
kubectl rollout status deployment <name>

# Wait until the Deployment is fully available (useful in CI/CD scripts)
kubectl wait --for condition=Available deployment/<name>

# Pause mid-rollout (e.g., to observe the first new Pod before continuing)
kubectl rollout pause deployment <name>

# Resume
kubectl rollout resume deployment <name>

Use cases for pausing:

Use case	How
Manual canary check	Trigger update → pause → verify the first new Pod → resume
Batch multiple changes	Pause before editing → apply several changes → resume once (single rollout)

Caveats when paused:

• Split traffic — client requests are served by both old and new versions simultaneously. A user’s browser may receive HTML from one version and CSS from another, causing rendering issues unless backward compatibility is strict.
• Rollback is blocked — kubectl rollout undo does nothing while a Deployment is paused. Resume first, then undo.
• Autoscaler distribution — if the HPA requests more Pods while paused, they are split across both ReplicaSets proportionally until the rollout resumes.

Updating Methods

# Declarative (preferred for production)
kubectl apply -f deployment.yaml

# Interactive edit
kubectl edit deployment <name>

# Quick image-only update
kubectl set image deployment <name> <container-name>=<new-image>

# Force replace (deletes and recreates)
kubectl replace -f deployment.yaml --force

---

Rollbacks

Revision History

Every change to the Pod template creates a new revision. Old ReplicaSets are kept (at zero replicas) as a documented history. The number of retained revisions is controlled by spec.revisionHistoryLimit (default: 10).

# View revision history
kubectl rollout history deployment <name>

# Inspect a specific revision
kubectl rollout history deployment <name> --revision=2

Documenting change causes — the CHANGE-CAUSE column in rollout history is blank unless you annotate manually:

kubectl annotate deployment <name> kubernetes.io/change-cause="Image updated to 1.2.0"

Executing a Rollback

# Roll back to the immediately previous revision
kubectl rollout undo deployment <name>

# Roll back to a specific revision
kubectl rollout undo deployment <name> --to-revision=2

# Monitor the rollback (it follows the same pace as a rollout)
kubectl rollout status deployment <name>

Caution

rollout undo is an imperative command — it causes your live cluster to drift from your YAML files. After an emergency rollback, always update your source manifests to reflect the reverted image version, otherwise your next kubectl apply will redeploy the broken version.

What a Rollback Does and Does Not Change

Reverted	Not reverted
spec.template (container image, env, mounts)	spec.replicas
—	Persistent data

History reordering: when you roll back to revision 1 while on revision 2, Kubernetes promotes the revision 1 configuration to become the newest revision (revision 3) and removes the original revision 1 entry.

rollout undo vs. Reapplying an Old Manifest

These two rollback approaches are not equivalent:

Method	What it reverts	What it preserves
kubectl rollout undo	Pod template only (spec.template)	Replica count, strategy, all other settings
kubectl apply with old manifest	Everything in the file	Nothing — blindly overwrites all live settings

Use rollout undo for surgical version rollbacks. Avoid applying old manifests as a rollback mechanism — you risk overwriting live operational changes (e.g., a scaled-up replica count) with outdated values.

---

Deployment Strategies

Beyond the built-in RollingUpdate and Recreate modes, more advanced release patterns can be implemented using combinations of standard Kubernetes objects.

Canary

Route a small slice of traffic to a new version before committing to a full rollout.

Partial canary (single Deployment):

• Set a high minReadySeconds to slow the rollout — gives time to observe stability at each step
• Or: trigger an update → immediately kubectl rollout pause after the first new Pod starts → inspect → resume

True canary (dual Deployments):

• Maintain two separate Deployments: stable (e.g., 9 replicas) and canary (e.g., 1 replica)
• Configure a single Service whose selector matches Pods from both — traffic is split ~90/10
• Once confident, perform a rolling update on the stable Deployment and delete the canary Deployment

A/B Testing

Route specific user segments to a different version based on request attributes (headers, cookies, location, user-agent).

Component	Role
Two Deployments	Version A (stable) + Version B (new)
Two Services	One targeting each Deployment
Ingress with conditional routing	Evaluates request attributes and directs traffic to Service-A or Service-B

• A single user is always routed to the same version (session consistency required)
• Requires a Layer-7 Ingress controller that supports header/cookie-based routing — standard Kubernetes Services route randomly across all matching Pods

Blue/Green

Deploy a complete parallel environment, test it, then switch all traffic at once with zero incremental rollout.

# Switch traffic from Blue to Green by patching the Service selector
kubectl patch service my-app -p '{"spec":{"selector":{"col":"green"}}}'

Aspect	Detail
Setup	Two Deployments with distinct labels (e.g., col: blue, col: green) + one Service
Switch	Update the Service selector — atomic, instant cutover
Rollback	Revert the Service selector to Blue
Native support	✅ No third-party tools required — works with standard Deployments and Services

Traffic Shadowing (Dark Launch)

Test a new version under real production load without affecting any users.

• Deploy the new version as a separate Deployment with labels that do not match the primary Service selector
• A proxy/Ingress mirrors each incoming request to both the stable and shadow Pods simultaneously
• Only the stable response is returned to the user — the shadow response is silently discarded
• Requires an Ingress controller or service mesh that supports traffic mirroring (not available natively in Kubernetes)

---

Network Access

A Deployment manages Pods but does not expose them to network traffic. To route requests to your Pods, create a separate Service object:

apiVersion: v1
kind: Service
metadata:
  name: my-app
spec:
  type: LoadBalancer
  selector:
    app: my-app           # must match Deployment's Pod template labels
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080

kubectl apply -f service.yaml

# Find the assigned external IP
kubectl get svc my-app

The Service discovers Pods via label selector — it has no direct reference to the Deployment itself. Changing the Service’s selector is how blue/green cutover works.

---

Deleting Deployments

kubectl delete deployment <name>

Deletion cascades: the Deployment, its ReplicaSets, and all managed Pods are removed automatically by the garbage collector.

# Verify full cleanup
kubectl get deployments,replicasets,pods