Cluster management
One Argo CD installation can deploy to the cluster it runs in and to any number of others. Every Application names exactly one destination, and Argo CD treats the local cluster and a remote one as the same thing: a Kubernetes API server it has credentials for. Cluster management is the work of registering those credentials and deciding how much each one is allowed to do.
Hub and spoke
Section titled “Hub and spoke”Argo CD uses a push model. The control plane (the hub) renders manifests, caches the result, and connects out to the API server of each target cluster (the spokes) to apply them. Nothing needs to run on the spokes.
flowchart LR
G["Git repository"] --> H["Argo CD hub"]
H -->|"API calls"| L["Local cluster (in-cluster)"]
H -->|"API calls"| R1["Remote cluster A"]
H -->|"API calls"| R2["Remote cluster B"]
This shapes the operational requirements:
- Network - the hub needs a route and firewall rules to every spoke’s API server, and the spokes never need to reach the hub
- Credentials - the hub stores a credential for each spoke as a Secret in its own namespace, so those Secrets are among the most sensitive objects in the cluster
- Blast radius - an attacker who controls the hub controls every spoke whose credential has write access, which is why sync impersonation and narrow credentials matter
A pull model, where an agent on each spoke fetches from Git, avoids inbound access to the spokes. Argo CD does not work that way, so a hub that reaches clusters it cannot route to needs a network design (VPN, private connectivity, or one Argo CD instance per network zone).
Choosing a destination
Section titled “Choosing a destination”An Application sets spec.destination to one cluster, identified by its API server URL (server) or by the name it was registered under (name). Use one of them, not both.
spec: destination: name: prod-eu # or: server: https://prod-eu.example.com:6443 namespace: orders| Field | Matches | Notes |
|---|---|---|
server | The URL of the cluster’s API server | https://kubernetes.default.svc is the local cluster |
name | The name in the cluster Secret | in-cluster is the local cluster. Names survive an endpoint change, so they keep manifests portable |
namespace | The namespace for namespaced resources | Applied only to manifests that do not set their own metadata.namespace |
A single Application has a single destination, even when it has several sources. Deploying the same application to many clusters therefore means one Application per cluster, which is what ApplicationSets generate, or what a Helm chart renders from a values file per cluster.
The local cluster
Section titled “The local cluster”Argo CD can manage the cluster it runs on without any registration. It uses the in-cluster API address, the cluster’s CA, and the token of the application controller’s service account. kubectl get secrets -n argocd -l argocd.argoproj.io/secret-type=cluster returns nothing on a fresh install.
To refine the local cluster, create a cluster Secret named in-cluster. A common use is reserving the control plane for administrators by scoping it to one project:
apiVersion: v1kind: Secretmetadata: name: in-cluster namespace: argocd labels: argocd.argoproj.io/secret-type: clustertype: OpaquestringData: name: in-cluster server: https://kubernetes.default.svc project: sysadmin # only Applications in this project may deploy here config: | { "tlsClientConfig": { "insecure": false } }argocd cluster get in-cluster -o json | jq -r .project# Expected output: sysadminThe cluster then also appears with a settings page in the UI under Settings, Clusters.
Cluster Secrets
Section titled “Cluster Secrets”Each remote cluster is a Secret in the Argo CD namespace with the label argocd.argoproj.io/secret-type: cluster. There is one Secret per cluster, so plan the scope of each credential before creating it.
| Field | Purpose |
|---|---|
name | Name used by destination.name |
server | API server URL, used by destination.server |
namespaces | Optional comma-separated list of namespaces Argo CD may manage. When set, cluster-scoped resources are ignored unless clusterResources is "true" |
clusterResources | Optional "true" or "false". Read only when namespaces is set |
project | Optional. Makes the cluster available only to Applications in that project |
config | JSON with the connection details |
The config payload holds the credentials and TLS settings:
| Key | Contents |
|---|---|
bearerToken | A service account token, in plain text (not base64 encoded) |
tlsClientConfig.caData | The cluster CA certificate, base64 encoded |
tlsClientConfig.certData, keyData | A client certificate and key, base64 encoded |
tlsClientConfig.insecure | true skips server certificate verification. Avoid it outside tests |
awsAuthConfig, execProviderConfig | Cloud IAM and external-command authentication, used for EKS, GKE, and AKS |
proxyUrl | A proxy for the connection to the API server |
apiVersion: v1kind: Secretmetadata: name: prod-eu namespace: argocd labels: argocd.argoproj.io/secret-type: cluster env: prod # labels are selectable by the ApplicationSet cluster generator region: eutype: OpaquestringData: name: prod-eu server: https://prod-eu.example.com:6443 namespaces: orders,payments # optional restriction clusterResources: "false" config: | { "bearerToken": "<service account token>", "tlsClientConfig": { "insecure": false, "caData": "<base64 encoded CA certificate>" } }Labels on the Secret have a second job: ApplicationSets read them to decide which clusters receive which applications. Setting env, region, and similar labels at registration is the basis of label-driven fleet management.
Registering a cluster
Section titled “Registering a cluster”With the CLI
Section titled “With the CLI”argocd cluster add takes a context name from a kubeconfig file and does the setup for you:
argocd cluster add kind-remote \ --kubeconfig ~/remote-cluster.config \ --name prod-eu \ --label env=prod --label region=eu \ --project sysadmin \ --yesUsing your kubeconfig credentials, the command creates these objects on the target cluster:
- A service account
argocd-managerin thekube-systemnamespace (--system-namespacechanges the namespace) - A ClusterRole
argocd-manager-rolegranting every verb on every resource - A ClusterRoleBinding
argocd-manager-role-binding - A long-lived token for the service account
It then writes the cluster Secret to the Argo CD namespace with that token and the CA data. The effect is cluster-admin on the target. To narrow it, pass --namespace (repeatable) to restrict the cluster to specific namespaces, which creates namespaced Roles instead of a ClusterRole, and --cluster-resources to also allow cluster-scoped resources.
The other subcommands manage what exists:
argocd cluster list # server, name, version, status, projectargocd cluster get prod-eu -o yamlargocd cluster set prod-eu --project sysadmin # change settings without editing the Secretargocd cluster rm prod-euA new cluster shows Unknown status and the message Cluster has no applications and is not being monitored until an Application targets it. Running the commands requires clusters permissions in Argo CD RBAC, which lets platform teams delegate cluster work without handing out kubectl access to the hub.
Declaratively
Section titled “Declaratively”The CLI is convenient, but a GitOps team wants clusters described as manifests. Repeat what the CLI does by hand, then apply the Secret.
# on the remote clusterkubectl create sa argocd-manager -n kube-systemkubectl create clusterrolebinding argocd-manager-role-binding \ --clusterrole=cluster-admin --serviceaccount=kube-system:argocd-managerSince Kubernetes 1.24, service accounts no longer get a token Secret automatically. Request one explicitly:
apiVersion: v1kind: Secretmetadata: name: argocd-manager-token namespace: kube-system annotations: kubernetes.io/service-account.name: argocd-managertype: kubernetes.io/service-account-tokenTOKEN=$(kubectl get secret argocd-manager-token -n kube-system \ -o jsonpath='{.data.token}' | base64 -d)Put the token and the cluster CA in a cluster Secret like the one above and apply it to the hub. A Role scoped to the namespaces the tenant needs is safer than cluster-admin here, since you choose the permissions.
Keeping credentials out of Git
Section titled “Keeping credentials out of Git”A cluster Secret holds a bearer token, so a plain-text copy in a repository is a credential leak. Create cluster Secrets from a secret store instead, with a tool such as the External Secrets Operator or Sealed Secrets, and commit only the reference. Note that sealing a Secret removes its labels, so re-add argocd.argoproj.io/secret-type: cluster through the template.
Updating and rotating
Section titled “Updating and rotating”Because a cluster is a Secret, any Kubernetes tool updates it: kubectl edit, kubectl patch, a re-applied manifest, or argocd cluster set. All of them change the same object. Use argocd cluster set when the person making the change has Argo CD RBAC but no access to the hub cluster.
Typical changes are rotating a service account token, renewing a CA certificate, changing an endpoint, and moving a cluster to a different project. In config, bearerToken stays plain text and caData stays base64 encoded. A mistake in this encoding produces authentication or certificate errors.
Annotating a cluster Secret with argocd.argoproj.io/skip-reconcile: "true" stops the application controller from reconciling every Application on that cluster while keeping it listed. It suits maintenance windows and clusters being decommissioned.
Troubleshooting
Section titled “Troubleshooting”| Symptom | Likely cause | Fix |
|---|---|---|
Cluster status stays Unknown | No Application targets it yet | Deploy an Application, this status is normal for an unused cluster |
argocd cluster add fails to connect | The context’s API server address is not reachable from where the command runs (for example 127.0.0.1) | Use a kubeconfig whose server address is routable from the hub and from your machine |
Applications show Unknown and the controller logs x509 errors | The CA in caData is wrong, or not base64 encoded | Replace caData with the base64 of the cluster CA |
Unauthorized or Forbidden on the target | The token expired or was revoked, or the service account lost its binding | Issue a new token, update bearerToken, and verify the ClusterRoleBinding |
| An Application cannot deploy to a registered cluster | The cluster has a project set to a different project | Use the right project, or change the cluster’s project |
| Cluster-scoped resources are skipped | namespaces is set and clusterResources is not "true" | Set clusterResources: "true" or remove the namespace restriction |
| The destination is rejected as not permitted | The project does not allow that server and namespace | Add it to the AppProject destinations |
| The Application reports an unknown destination cluster | destination.name or server matches no cluster Secret | Check the spelling, and use name or server, not both |