Upgrade the Orchestrator

Upgrade a self-hosted Orchestrator to a new release, and roll back if you need to.

Upgrading a self-hosted Orchestrator means taking a backup, switching the image tag, applying the new schema script and restarting. The database schema only ever changes additively, which is what makes a rollback to the previous image possible.

Before you start#

  • Read the release notes for the new version. They say when a release needs anything beyond these steps.
  • Plan a short maintenance window. The API restarts, and during the restart users cannot sign in and Robots cannot claim new work. Robots keep running the jobs they already have and deliver results when the Orchestrator is back.
  • Upgrade the Orchestrator before the Robots. See Updating the Robot.

Never run two versions against one database

Stop the old API container before the new one starts. docker compose up -d does this for you. Do not start a second stack against the same database.

Upgrade with Docker Compose#

Prerequisites#

  • Shell access to the Docker host and the deployment directory (for example /srv/velophex).
  • The new release's files: at least core-schema.sql, and the new docker-compose.core.yml and deploy/caddy/Caddyfile if they changed.
  • Your escrowed .env values.

Steps#

  1. Take a cold backup. This is your restore point if the upgrade has to be undone.

    PowerShellCopy
    pwsh /path/to/release/orchestrator/backup.ps1 -ProjectDirectory /srv/velophex -Destination /backup/velophex
  2. Set the new image tag in .env:

    .envINICopy
    VELOPHEX_IMAGE=velophex/orchestrator:<version>
  3. Pull the image:

    BashCopy
    cd /srv/velophex
    docker compose -f docker-compose.core.yml pull

    If you received the image as a file, load it with docker load -i velophex-orchestrator-<version>.tar instead.

  4. Apply the schema. The API never migrates the database on start. The release's core-schema.sql is idempotent: it applies only the migrations the database does not have yet.

    BashCopy
    set -a; . ./.env; set +a
    docker compose -f docker-compose.core.yml exec -T postgres \
      psql -U "$VELOPHEX_POSTGRES_USER" -d "$VELOPHEX_POSTGRES_DB" -v < core-schema.sql
  5. Restart the stack:

    BashCopy
    docker compose -f docker-compose.core.yml up -d

Verify#

  1. Readiness returns 200:

    BashCopy
    curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8080/health/ready

    You should see 200.

  2. Sign in to the web app. You should see your tenants and workspaces as before.

  3. Start one job. You should see it claimed by a Robot and succeed.

If readiness returns 503 after an upgrade

If the log says a table or column does not exist, the schema script was not applied. Run step 4 and restart the API.

Roll back#

Because migrations are additive, the previous image normally runs on the newer schema:

  1. Set VELOPHEX_IMAGE back to the previous tag in .env.
  2. Run docker compose -f docker-compose.core.yml up -d.
  3. Check /health/ready, sign in and run a job.

Only if the release notes say the new migrations changed something the old version cannot handle, restore the backup you took in step 1 (see Backup and restore). Never try to roll the schema itself back by hand.

Upgrading a stack from before the bundled Caddy#

Newer compose files give the stack's network a fixed subnet, so the API can trust forwarded headers from the bundled Caddy. Docker cannot change the subnet of a network that containers are attached to, so for this one upgrade run:

BashCopy
docker compose -f docker-compose.core.yml down
docker compose -f docker-compose.core.yml up -d

down keeps your volumes.

After the upgrade#

  • Upgrade your Robots and Studio installations to the matching release. See Updating the Robot.
  • Run a restore drill against the new image when convenient, as recommended in Backup and restore.

Next steps#

Before you start

Upgrade with Docker Compose

Prerequisites

Steps

Verify

Roll back

Upgrading a stack from before the bundled Caddy

After the upgrade

Next steps