Retention

How long the Orchestrator keeps jobs, logs, queue items, audit events and notifications, and how to change it.

The Orchestrator deletes old operational data on a schedule so the database does not grow without bound. Each tenant has a retention window per data category. A background sweep runs every hour and removes rows older than the window, oldest first, in bounded batches.

Windows and their allowed ranges#

Category Default Allowed range Tenant setting
Execution logs 30 days 1 to 365 days jobLogDays
Jobs 90 days 7 to 730 days jobDays
Queue items 90 days 1 to 730 days queueItemDays
Audit events 365 days 90 to 2555 days auditDays
Notifications 90 days 1 to 365 days notificationDays

A tenant value outside the allowed range is refused. Setting enabled to false in the retention group pauses the sweep for that scope. When no tenant value is set, the platform value applies, and when neither is set, the default applies. A platform administrator can also set maxDays in the platform's retention group as a ceiling for every tenant window (1 to 3650 days).

Other data has fixed, platform-level retention:

Data Kept for Configured by
Webhook events and delivery records 7 days Webhooks:RetentionDays
Sent emails, ended sessions, used or expired tokens 30 days Retention:HousekeepingDays
Deleted workspace files and their blobs 30 days Retention:DeletedFileDays

The high-volume tables for Robot commands, domain events and webhook deliveries are partitioned by UTC day. Retention drops a whole day's partition once every row in it is past its window, which is much cheaper than deleting rows.

Change a tenant's windows#

Prerequisites: settings.edit in the tenant.

  1. Open the tenant's Settings page and choose Retention.
  2. Enter a number of days for each category you want to change. Leave a field empty to inherit the platform value.
  3. Save.

Or with the API:

HTTPCopy
PUT /api/v1/tenants/{tenantId}/settings/retention
Content-Type: application/json

{ "jobLogDays": 14, "jobDays": 180, "auditDays": 730 }

Verify: reload the Retention settings. You should see the new values as effective. After the next hourly sweep, the API log contains a line such as Retention removed N row(s) for categories that had data past the new window.

Shortening a window deletes data

Lowering a window makes the next sweep delete everything older than the new value. If you need the data for audits or investigations, take a backup first. Deleted rows can only be recovered by restoring a backup.

How the sweep works#

The RetentionWorker runs every Retention:IntervalMinutes (60 by default). For each tenant and category it deletes up to Retention:MaxBatchesPerRun (20) batches of Retention:BatchSize (5000) rows per run. A large backlog is therefore cleared over several runs, without one long transaction. The sweep deletes a blob before its row and child rows before their parents, which keeps the database consistent.

Tuning the sweep#

You rarely need to change these. They are deployment configuration in velophex.prod.env:

velophex.prod.envINICopy
Retention__IntervalMinutes=60
Retention__BatchSize=5000
Retention__MaxBatchesPerRun=20
  • If the log shows Retention run failed with a statement or command timeout, lower Retention__BatchSize to 1000 and recreate the API container. Raise it again once the backlog is gone.
  • To clear a very large backlog off-hours, temporarily set Retention__IntervalMinutes=1 and Retention__MaxBatchesPerRun=200, then restore the defaults.

Do not delete rows by hand

Manual DELETE statements can break the order the sweep relies on and leave blobs without rows or rows without blobs. Let the sweep do the work.

Monitor retention#

The sweep logs at Error level when a run fails and carries on at the next interval. A sweep that fails every run lets the database grow silently, so alert on its log category:

BashCopy
docker compose -f docker-compose.core.yml logs --since 2h api | grep '"Category":"VeloPhex.Orchestrator.Infrastructure.Operations.RetentionWorker"'

See Monitoring and health for the other background workers.

Next steps#

Windows and their allowed ranges

Change a tenant's windows

How the sweep works

Tuning the sweep

Monitor retention

Next steps