Databases

Stream Trace stores its own state - message indexes, throughput samples, DLQ cases, users, roles, and the audit log - in a database. Your NATS message data is read live from NATS and is not copied wholesale into this database.

SQLite (default)

SQLite needs no external service and is ideal for evaluation and single-instance deployments. The default DSN stores a file in the working directory:

ST_DB_TYPE=sqlite
ST_DB_URL="file:stream-trace.db?_foreign_keys=on&_journal_mode=WAL&_busy_timeout=5000"

Mount a volume and point the file at it to persist data across container restarts (see Installation). Full-text search is compiled in via SQLite FTS5.

SQLite runs as a single process. It cannot be shared by multiple Stream Trace instances, so SQLite deployments must run the api and indexer roles together in one process (the default ST_RUNTIME_MODE=api,indexer). For multiple instances or split roles, use PostgreSQL.

PostgreSQL

Use PostgreSQL when you need high availability, multiple API instances, or a separate indexer deployment:

ST_DB_TYPE=postgres
ST_DB_URL="postgres://streamtrace:streamtrace@db:5432/stream_trace?sslmode=disable"

Beta validation currently uses PostgreSQL 16. Create an empty database; Stream Trace creates its schema on first start. Older supported PostgreSQL releases have not yet been included in the beta test matrix.

Migrations

Schema migrations run automatically during normal startup by default:

Variable	Default	Description
ST_DB_AUTO_MIGRATE	true	Apply pending migrations at startup. Set false to manage migrations externally.

To apply migrations without starting the API, indexer, NATS connection, or auth modules, run the image’s migration-only command:

./stream-trace migrate

The explicit command always applies pending migrations and then exits. It ignores ST_DB_AUTO_MIGRATE; that setting controls normal application startup only. The migration command needs ST_DB_TYPE, ST_DB_URL, and any non-default ST_DB_WRITE_* pool settings. It does not need ST_NATS_URL or authentication settings.

Run migrations as a Kubernetes Job

Use PostgreSQL when migrations run separately from the application. Create one Job per release, wait for it to complete, and set ST_DB_AUTO_MIGRATE=false on the API and indexer Deployments:

apiVersion: batch/v1
kind: Job
metadata:
  name: stream-trace-migrate-RELEASE_ID
spec:
  backoffLimit: 1
  activeDeadlineSeconds: 600
  template:
    spec:
      restartPolicy: Never
      containers:
        - name: migrate
          image: git.prod.antomic.net/antomic/stream-trace:RELEASE_TAG
          command: ["./stream-trace", "migrate"]
          env:
            - name: ST_DB_TYPE
              value: postgres
            - name: ST_DB_URL
              valueFrom:
                secretKeyRef:
                  name: stream-trace-db
                  key: url

Wait before rolling the application:

kubectl wait --for=condition=complete \
  job/stream-trace-migrate-RELEASE_ID \
  --timeout=10m

Then set this on every application pod:

- name: ST_DB_AUTO_MIGRATE
  value: "false"

A Job does not order a Deployment by itself. Put the wait step in your release pipeline, or use the equivalent Helm pre-install/pre-upgrade hook or Argo CD PreSync hook. Do not run multiple migration Jobs for the same database at once. Pre-deploy migrations must remain compatible with any old pods that are still running; use an expand/contract rollout when a schema change is not backward compatible.

SQLite: keep automatic startup migrations for the single Stream Trace process. A separate Job is safe only if it mounts the same database file and has exclusive access while the application is stopped. A ReadWriteOnce volume alone does not guarantee single-process access.

Connection pools

Stream Trace uses separate read and write connection pools, each independently tunable (ST_DB_READ_* and ST_DB_WRITE_*). The defaults are reasonable for most deployments - see the Configuration reference to adjust pool sizes and connection lifetimes.