Application health
Argo CD reports two independent statuses for every Application. Sync status says whether the live cluster matches Git. Health status says whether what is running is working. An Application can be Synced and Degraded, or OutOfSync and Healthy, and sync waves depend on health, not on sync, to decide when to move on.
Sync status and health status
Section titled “Sync status and health status”| Status | Question it answers | Source |
|---|---|---|
Sync (Synced, OutOfSync) | Does the live state match the desired state? | A comparison of rendered manifests with the cluster |
| Health | Is the application operating correctly? | The status that Kubernetes reports for each resource |
The Application tile in the UI shows both, and an Application is ready only when it is Healthy and Synced.
Health uses these states:
| State | Meaning |
|---|---|
| Healthy | All resources are operating as expected |
| Progressing | Not healthy yet, but still changing, for example a rollout in progress |
| Degraded | A resource has failed or cannot reach its desired state |
| Suspended | A resource is paused or suspended, such as a suspended Job |
| Missing | The resource does not exist in the cluster |
| Unknown | Argo CD cannot assess the health of the resource |
How health is assessed
Section titled “How health is assessed”Argo CD has built-in checks for the standard Kubernetes types and rolls them up into the Application’s health.
| Resource | Healthy when |
|---|---|
| Deployment, ReplicaSet, StatefulSet, DaemonSet | The observed generation equals the desired generation, and the number of updated replicas equals the number desired |
Service of type LoadBalancer | status.loadBalancer.ingress has at least one hostname or IP |
| Ingress | status.loadBalancer.ingress has at least one hostname or IP |
| PersistentVolumeClaim | status.phase is Bound |
| CronJob | The last scheduled Job did not fail. Progressing while one is running, Degraded if it failed |
| Job | Not suspended. A suspended Job marks the Application Suspended |
Argo CD does not run probes itself. The kubelet runs them and Kubernetes reflects the result in the resource’s status, which Argo CD reads.
Probes and wave progression
Section titled “Probes and wave progression”Within a sync, Argo CD applies a wave and waits for every resource in it to become healthy before starting the next one. Probes are what make that wait meaningful.
- Readiness probe - decides when a container can receive traffic. Until it passes, the Pod is not ready.
- Liveness probe - decides when a container is unresponsive and must be restarted.
A Pod without a readiness probe counts as ready the moment its containers start. Argo CD then sees a healthy Deployment and begins the next wave while the application inside is still starting. Dependent resources can then fail or hooks can fire early. See Pods for how probes work.
A database and a web application show both probe styles:
# database Deployment: ready when MySQL accepts TCP connectionslivenessProbe: tcpSocket: port: 3306 initialDelaySeconds: 12 periodSeconds: 10readinessProbe: tcpSocket: port: 3306 initialDelaySeconds: 12 # give the server time to initialize periodSeconds: 10# web Deployment: ready only when the app answers HTTP 200readinessProbe: httpGet: path: / port: 8080 initialDelaySeconds: 5 periodSeconds: 2livenessProbe: tcpSocket: port: 8080 initialDelaySeconds: 5 periodSeconds: 2With these probes, a wave that holds the database does not complete until port 3306 answers, and the Application does not turn Healthy until the web application serves a 200. Both the readiness and liveness probes across all workloads must pass before the tile shows Healthy and Synced.
The same gate applies to hooks. A PostSync hook starts only after every resource in the sync is Healthy, so the quality of the probes decides whether a post-deployment job runs against a working application.
Custom health checks
Section titled “Custom health checks”Argo CD ships checks for the standard Kubernetes types and for many popular CRDs. A custom resource without a check reports Unknown, and Argo CD cannot tell whether the operator behind it has finished its work. A custom health check is a Lua script that reads the resource’s status and returns a health state and message. Add it to argocd-cm under resource.customizations.health.<group>_<kind>:
data: resource.customizations.health.db.example.com_PostgresCluster: | hs = {} if obj.status ~= nil and obj.status.conditions ~= nil then for _, c in ipairs(obj.status.conditions) do if c.type == "Ready" and c.status == "True" then hs.status = "Healthy" hs.message = c.message return hs end if c.type == "Ready" and c.status == "False" then hs.status = "Degraded" hs.message = c.message return hs end end end hs.status = "Progressing" # no verdict yet, keep waiting hs.message = "Waiting for the cluster to become ready" return hsThe script receives the live resource as obj and returns a table with status (Healthy, Progressing, Degraded, Suspended, Missing, or Unknown) and an optional message. Write the check from the status fields the operator documents. If the status carries observedGeneration, compare it to metadata.generation so the health does not flap while the operator is still processing a change. A wildcard in the group or kind ("*.aws.crossplane.io/*" under resource.customizations) covers a family of resources with one script.
Custom checks matter for waves. A wave advances when its resources are healthy, so an operator-managed resource with no check either blocks the wave forever or never gates it at all.
Health of Application resources
Section titled “Health of Application resources”An Application is itself a Kubernetes resource, and a parent Application that manages child Applications (the app-of-apps pattern) needs their health to order them with sync waves. Argo CD removed the built-in health assessment for the argoproj.io/Application kind in version 1.8, so with a default install the parent treats a child as healthy as soon as it exists. Restore the assessment by copying the child’s own health status:
data: resource.customizations.health.argoproj.io_Application: | hs = {} hs.status = "Progressing" hs.message = "" if obj.status ~= nil then if obj.status.health ~= nil then hs.status = obj.status.health.status if obj.status.health.message ~= nil then hs.message = obj.status.health.message end end end return hsCheck that the setting registered:
kubectl get cm argocd-cm -n argocd \ -o jsonpath='{.data.resource\.customizations\.health\.argoproj\.io_Application}'The same check lets ApplicationSet progressive syncs see when an Application is healthy. Both patterns are covered in app-of-apps and ApplicationSets.
Troubleshooting
Section titled “Troubleshooting”| Symptom | Likely cause | Fix |
|---|---|---|
| A parent Application starts the next wave before a child is ready | The Application kind has no health check, so every child counts as healthy | Add the argoproj.io_Application health customization |
Application stays Progressing | A readiness probe is failing, a rollout is incomplete, or a LoadBalancer Service has no address yet | Check the Pod’s events and probe results with kubectl describe pod, and the Service’s status.loadBalancer |
Application is Synced but Degraded | Resources match Git but one is failing, such as a crash-looping Pod | Fix the workload. Syncing again does not help |
| A later wave starts before the previous workload is working | The previous workload has no readiness probe | Add a readiness probe that reflects real readiness |
| Sync never leaves an early wave | A resource in that wave never becomes healthy | Find the unhealthy resource in the UI and fix it |
Health shows Unknown for a custom resource | Argo CD has no health check for that kind | Add a custom health check for the resource type |