Docker Compose

• Docker Compose is a tool for defining and running multi-container applications using a single docker-compose.yml file.
• Instead of running multiple docker run commands with flags, you declare your entire application stack (services, networks, volumes) in YAML and manage it with a single docker compose up.
• Compose is now part of Docker CLI (docker compose - v2). The standalone docker-compose CLI (v1) is deprecated.

Core Concepts

• Service: A container definition. Each service maps to one Docker image and runs one or more container instances.
• Network: Compose creates a default bridge network for the project. All services on it can reach each other by service name (built-in DNS).
• Volume: Named storage attached to one or more services for data persistence.
• Project: The logical grouping of everything in a docker-compose.yml. Project name defaults to the directory name. Resources are named <project>_<service>_<counter> (e.g. myapp_web_1) - the counter enables multiple replicas of the same service.
• Client-side tool: Docker Engine has no concept of a multi-container “application”. It only runs individual containers. Compose is the layer that reads the YAML, groups resources under a project name, and manages their lifecycle together.

docker-compose.yml Structure

# docker-compose.yml - a full application stack example
services:
  web:
    image: nginx:alpine
    ports:
      - "80:80" # host:container
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro # bind mount, read-only
    depends_on:
      - app # starts after app is healthy

  app:
    build:
      context: . # build from Dockerfile in current dir
      dockerfile: Dockerfile.prod
    environment:
      - DATABASE_URL=postgres://user:pass@db:5432/mydb # ⚠️ Don't hardcode creds - use env_file or secrets
    env_file:
      - .env # load additional env vars from file (add to .gitignore!)
    restart: unless-stopped

  db:
    image: postgres:16-alpine
    volumes:
      - db-data:/var/lib/postgresql/data # named volume for persistence
    environment:
      POSTGRES_USER: user
      POSTGRES_PASSWORD: pass
      POSTGRES_DB: mydb
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U user -d mydb"]
      interval: 5s
      timeout: 5s
      retries: 5

volumes:
  db-data: # declares the named volume

Top-level keys: services (required), networks, volumes, configs, secrets. Use configs for non-sensitive environment-specific files and secrets for credentials - both are injected into containers without ever landing in image layers or environment variables.

Service-level keys not shown above:

• command - overrides the default command from the image (same as CMD in a Dockerfile)
• deploy - orchestration rules: replicas, update_config (rolling update strategy), and restart_policy. Primarily used when deploying to Docker Swarm via docker stack deploy

Key Commands

# Start all services (detached)
docker compose up -d

# Start and rebuild images before starting
docker compose up -d --build

# Stop services (containers remain, volumes intact)
docker compose stop

# Stop and remove containers and networks (volumes preserved by default)
docker compose down

# Stop, remove containers, networks, AND named volumes
docker compose down -v

# View running services
docker compose ps

# Tail logs from all services
docker compose logs -f

# Tail logs from a specific service
docker compose logs -f app

# Execute a command in a running service
docker compose exec app bash
docker compose exec db psql -U user -d mydb

# Scale a service to N instances
docker compose up -d --scale app=3

# Pull latest images for all services
docker compose pull

# Validate and view the merged config
docker compose config

# Show running processes inside each container (PIDs as seen from host)
docker compose top

# Restart all containers (re-reads config only if you re-run `up`)
docker compose restart

# Complete teardown - removes containers, networks, volumes, AND images
docker compose down --volumes --rmi all

Caution

docker compose restart does not pick up changes to compose.yaml. If you’ve modified the file while the app was stopped, run docker compose up -d instead - that re-reads the config and applies the new desired state.

Networking in Compose

• Compose creates a default bridge network: <project-name>_default.
• Services can reach each other by service name - no IP addresses needed.

# In the app service, connect to the database using the service name "db"
DATABASE_URL=postgres://user:pass@db:5432/mydb # "db" resolves to the db container's IP

• Define custom networks for isolation between service groups:

services:
  frontend:
    networks: [public]
  api:
    networks: [public, private]
  db:
    networks: [private] # db not reachable from frontend

networks:
  public:
  private:

Compose File Format Versions

The version: key at the top of the file specifies the Compose file format.

Note

The version field is deprecated as of Compose Specification (2021). Modern Docker Compose (v2) uses the Compose Spec and ignores this field. Omit it from new files.

Format	Notable Addition
v1	No version key. Services linked with links:.
v2	Added version: key, depends_on:, health checks, named volumes.
v3	Added deploy: key for Swarm mode (replicas, resource limits).
Compose Spec	Unified spec replacing versioned formats. version field optional/ignored.

# Modern Compose file - no version key needed
services:
  web:
    image: nginx

Practical Patterns

Environment-specific overrides

# docker-compose.yml          - base config
# docker-compose.override.yml - auto-loaded on top (dev defaults)
# docker-compose.prod.yml     - explicit for production

docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d

Health checks with depends_on

Without health checks, depends_on only waits for the container to start, not to be ready:

depends_on:
  db:
    condition: service_healthy # waits for healthcheck to pass

One-off commands

# Run database migrations before starting the app
docker compose run --rm app python manage.py migrate

# If migrations need to connect to an exposed port, add --service-ports
# (docker compose run doesn't publish ports by default)
docker compose run --rm --service-ports app python manage.py migrate

Watch mode (hot reload)

Added in Docker Compose 2.22 - automatically syncs or rebuilds when files change:

docker-compose.yml

services:
  app:
    build: .
    develop:
      watch:
        - action: sync # copy changed files into the container without rebuild
          path: ./src
          target: /app/src
        - action: rebuild # rebuild image when these files change
          path: package.json

# Start services with file watching enabled
docker compose watch

Useful alternative to bind mounts for development - more explicit about what triggers a sync vs a full rebuild.

Variable substitution

Compose files can be parameterized using shell-style variable substitution:

services:
  app:
    image: my-app:${APP_VERSION:-latest} # default to "latest" if not set
    environment:
      - DEPLOY_ENV=${DEPLOY_ENV:-production}

A .env file in the same directory as compose.yaml auto-populates these variables - no export required:

.env

APP_VERSION=1.4.2
DEPLOY_ENV=staging

Caution

.env is not the same as env_file:. The .env file sets Compose variable substitution (used in the YAML itself). env_file: injects variables into the container’s runtime environment. Add .env to .gitignore to keep secrets out of version control.

Extension fields (DRY config with YAML anchors)

Repeated config blocks (logging, deploy settings, labels) can be defined once and reused across services using x- extension fields and YAML anchors:

x-common-logging: &logging
  driver: json-file
  options:
    max-size: "10m"
    max-file: "3"

services:
  api:
    image: my-api
    logging: *logging # reuse the anchor

  worker:
    image: my-worker
    logging: *logging # same block, no duplication

The x- prefix is ignored by Compose but valid YAML - blocks prefixed with it won’t cause unknown-key errors.

Deploying to a cluster (Docker Swarm)

The same compose.yaml used locally can deploy to a multi-node Swarm cluster using docker stack deploy:

# Deploy to Swarm as a named stack
docker stack deploy -c compose.yaml my-stack

# Re-apply after changes (compares desired vs current state, updates only what changed)
docker stack deploy -c compose.yaml my-stack

# Also remove services deleted from the file
docker stack deploy -c compose.yaml --prune my-stack

Note

Swarm verifies that cluster-level resources (external networks, secrets) exist before deploying. Missing dependencies cause a fast-fail so you catch configuration errors before any containers start.

Compose Best Practices

• Declarative changes only: Make all scaling and configuration changes inside the Compose file - never via imperative one-off commands like docker service scale. This keeps the file as the single source of truth and ensures the actual cluster state always matches it.
• Version-control the file: Treat compose.yaml as code. Store it alongside your application source in Git so that infrastructure changes have the same review and history as code changes.
• Use named volumes for persistence: Never rely on a container’s writable layer for data. Containers are ephemeral; named volumes survive docker compose down and are re-attached on the next up.
• Set a custom project name for shared machines: Use docker compose -p <name> up when running multiple stacks on the same host to prevent resource name collisions.
• Prefer condition: service_healthy in depends_on: The plain depends_on only waits for a container to start, not to be ready. Pair it with a healthcheck to prevent race conditions at startup.