Command reference

sluice engines #

List the database engines built into this binary and their bulk-load / CDC capabilities.

sluice engines

14 engines are registered today: mysql (binlog CDC), planetscale and self-hosted vitess (both VStream CDC), mariadb (a MySQL-family flavor — bulk migrate source and target, backup/restore/verify, and continuous CDC sync since v0.99.271: sluice parses MariaDB's domain-based GTIDs and resumes off them; native uuid/inet columns are the one CDC-refused shape, steered to bulk migrate), postgres (logical-replication CDC), sqlite and d1 (migrate sources — sqlite is also a target — no CDC), the trigger-CDC engines postgres-trigger (slot-less Postgres), sqlite-trigger (local SQLite file), and d1-trigger (live Cloudflare D1), and the flat-file migrate sources csv, tsv, ndjson (ADR-0163) and mydumper (a mydumper / pscale database dump directory, ADR-0161). The vitess flavor shares the PlanetScale engine code with a self-hosted-vtgate capability set, and warm-resumes since v0.99.44.

sluice migrate #

Run a one-time schema + data migration: translate the schema, create tables, bulk-copy rows, then build indexes and constraints.

Filtered dry run, then apply:

sluice migrate --source-driver mysql --source ... --target-driver postgres --target ... \
    --include-table 'app_*' --exclude-table 'app_audit' --dry-run

Redact PII as it copies:

sluice migrate --source-driver mysql --source ... --target-driver postgres --target ... \
    --redact users.email=hash:sha256 \
    --redact users.ssn=mask:ssn

Import a SQLite file / .sql dump, or a live Cloudflare D1: point --source-driver at sqlite (a .db file or an auto-detected wrangler d1 export .sql dump) or d1 (a d1:// DSN; token via CLOUDFLARE_API_TOKEN). Big integers above 2⁵³ round-trip exactly; declared DATE / DATETIME columns are decoded per --sqlite-date-encoding. SQLite is also a target (--target-driver sqlite).

sluice migrate --source-driver sqlite --source ./app.db \
    --target-driver postgres --target 'postgres://...?sslmode=disable'
sluice migrate --source-driver d1 --source 'd1://<account_id>/<database_id>' \
    --target-driver mysql --target 'user:pass@tcp(host:3306)/app'

sluice sync start #

Start (or resume) a continuous-sync stream: consistent snapshot → bulk copy → ongoing CDC. Identified by --stream-id for clean restart.

Run as a service with metrics + idle-source heartbeat:

sluice sync start --source-driver postgres --source ... --target-driver mysql --target ... \
    --stream-id reporting \
    --metrics-listen :9090 \
    --source-heartbeat-interval 30s

With PlanetScale target-health telemetry + a storage alert (tokens via env, control-plane credential distinct from --target):

export PLANETSCALE_METRICS_TOKEN_ID=...   # the read_metrics_endpoints service token
export PLANETSCALE_METRICS_TOKEN=...
export SLUICE_NOTIFY_SLACK=https://hooks.slack.com/services/...
sluice sync start --source-driver mysql --source ... --target-driver planetscale --target ... \
    --stream-id app-prod \
    --planetscale-org acme --planetscale-metrics-db app \
    --notify-storage-util 0.85 --notify-slack "$SLUICE_NOTIFY_SLACK"

sluice sync status · stop · health · decommission #

Inspect, gracefully stop, health-check, and retire a stream. All take --stream-id plus the target connection (decommission takes the source too).

• sync status — show the stream's persisted position and phase.
• sync stop — request the stream to drain in-flight changes and exit cleanly. By default it just files the stop request and returns; pass --wait / -w to block until the running streamer drains and clears its stop signal (with --timeout, default 5m; on timeout the CLI exits non-zero and the stop request remains in place). Use --wait to coordinate ALTER windows or scripted teardowns.
• sync health — probe freshness against thresholds and return a cron-friendly exit code (non-zero when stale).
• sync decommission — retire a finished stream's durable footprint in one gated command (v0.99.291): drops its replication slot on the source (a leftover slot pins WAL and, on Postgres, blocks later differently-scoped cold starts under the scope guard), drops its recorded per-stream publication (never the shared sluice_pub), then clears its control row last — so a partially-failed run keeps the record and an idempotent re-run finishes the job. Takes the cross-DSN sync start shape (--source-driver/--source --target-driver/--target --stream-id); refuses on a live stream (SLUICE-E-DECOMMISSION-STREAM-ACTIVE — run sync stop --wait first) and without --yes; --dry-run previews without touching anything. MySQL-family sources have no source-side objects (the binlog is the stream), so decommission clears the control row and says so; trigger-CDC change-log tables belong to trigger prune/teardown.

sluice sync stop   --stream-id app-prod --target-driver postgres --target ... --wait --timeout 10m
sluice sync health --stream-id app-prod --target-driver postgres --target ... \
    --max-stale-seconds 300   # exit non-zero if the last apply was more than 5 minutes ago
sluice sync decommission --stream-id wave-1 --source-driver postgres --source ... \
    --target-driver postgres --target ... --yes   # drained wave: drop slot + publication, clear control row

sync health's freshness check is --max-stale-seconds N (target-side wall-clock seconds since the last apply; 0 = informational only). When you also pass --source-driver + --source the probe reads the source position too and, on a PG→PG pair, exposes --max-lag-bytes N (source LSN bytes ahead of target; MySQL GTID sets aren't byte-distance comparable). Both exit 1 when breached — cron-friendly.

sluice sync run --config syncs.yaml #

Supervise many syncs from one process (ADR-0122): each sync is failure-isolated with bounded-backoff restart, and a bad neighbor never takes the fleet down.

The process blocks until every sync exits; Ctrl-C / SIGTERM stops them all cleanly. Live reload without a restart: edit syncs.yaml and send the process SIGHUP — sluice re-reads and re-validates the file, then reconciles the live fleet (starts added syncs, drains removed ones, restarts changed ones, leaves unchanged ones untouched). A reload that fails to parse or validate is refused loudly and the running fleet keeps going on the old config. SIGHUP is POSIX-only; on Windows, restart the process to change the fleet. The full walkthrough is in Operate a sync fleet.

# validate + print the plan, start nothing
sluice sync run --config syncs.yaml --dry-run

# run the fleet with a read-only dashboard API on :9300
sluice sync run --config syncs.yaml --dashboard-listen :9300

# reload the running fleet after editing syncs.yaml (POSIX)
kill -HUP "$(pgrep -f 'sluice sync run')"

sluice sync tui --connect ADDR #

A full-screen terminal dashboard for a running fleet (ADR-0125) — it polls a 'sync run --dashboard-listen' server's /api/fleet endpoint, so it works locally or over an SSH tunnel without disturbing the fleet process.

The TUI keeps the last-known fleet on screen with an "unreachable" banner if a poll fails, instead of blanking.

# terminal 1: run the fleet with the dashboard API exposed
sluice sync run --config syncs.yaml --dashboard-listen :9300

# terminal 2 (local or over an SSH tunnel): live terminal view
sluice sync tui --connect :9300 --refresh 2s

sluice schema add-table <table> #

Bring a new source table into an active stream's scope without a destructive --reset-target-data cycle. Drain the stream first via 'sluice sync stop --wait'.

# drain first, add the table, then resume
sluice sync stop --stream-id app-prod --target-driver postgres --target ... --wait
sluice schema add-table new_events \
    --source-driver mysql --source ... --target-driver postgres --target ... \
    --stream-id app-prod
sluice sync start --stream-id app-prod --source-driver mysql --source ... --target-driver postgres --target ...

sluice sync from-backup run · stop #

Replay a backup chain into a target as a long-running broker — polls a chain root (S3/GCS/Azure/local) for new incrementals and applies them. No direct source↔target connectivity required.

The full walkthrough — producing the chain, cold-start vs warm-resume, stopping — is in the backup-chain sync guide.

sluice sync from-backup run \
    --backup-target s3://my-bucket/app-chain \
    --target-driver postgres --target ... \
    --stream-id app-broker --apply-concurrency 4 --poll-interval 30s

sluice sync from-backup stop --backup-target s3://my-bucket/app-chain

sluice cutover #

Two-phase sequence priming at cutover: re-read source sequence / AUTO_INCREMENT state and apply it to the target with a safety margin, so the first post-cutover INSERT can't collide on the primary key.

sluice cutover --config sluice.yaml --cutover-sequence-margin 1000

Run after the snapshot has caught up and just before switching application traffic to the target.

sluice backup #

Take and verify logical backups — full snapshots and incremental chains, optionally encrypted, to local FS or object storage.

sluice backup full --source-driver postgres --source ... --target s3://my-bucket/app-chain --chain-slot
sluice backup incremental --source-driver postgres --source ... --target s3://my-bucket/app-chain

# signed chain: generate an Ed25519 pair once, sign on write, verify on read
sluice backup keygen --out-dir ~/.sluice/keys
sluice backup full --source-driver postgres --source ... --target s3://my-bucket/app-chain \
    --sign-key ~/.sluice/keys/sluice-sign-key.pem
sluice restore --from s3://my-bucket/app-chain --target-driver postgres --target ... \
    --verify-key ~/.sluice/keys/sluice-verify-key.pem --require-signature

sluice backup export-as-parquet #

One-shot, read-only transcode of an existing backup's row chunks into one zstd-compressed Parquet file per table plus a parquet_index.json export manifest — the analytics exit surface over the chain sluice already captured (ADR-0164).

The export represents one snapshot — the latest full by default, or the full named by --backup-id. Incremental change-windows after that full are not folded in (a loud WARN names the count); operators who need point-in-time state restore the chain and re-export. Exit-only: sluice never reads its Parquet output back — sluice restore keeps the JSON-Lines path. The Parquet files themselves are written plaintext even from an encrypted chain — the analytics destination's encryption posture is a separate operator choice.

sluice backup export-as-parquet --from s3://my-bucket/app-chain \
    --output-dir ./warehouse-drop \
    --exclude-table 'audit_*'

Never a silent narrow: a column type or value with no faithful Parquet representation — a multi-dimensional array, a TIME outside a calendar day (MySQL TIME reaches ±838h), a PG NUMERIC NaN/Infinity, a sub-microsecond timestamp — is refused loudly with SLUICE-E-EXPORT-UNREPRESENTABLE (exclude the table and export the rest, or query that table's JSON-Lines chunks directly — DuckDB reads them natively). The documented string downgrades (unbounded NUMERIC, TIMETZ) carry the exact value text and are WARNs, not refusals.

sluice restore #

Restore a logical backup chain (full + every incremental up to the tail) into a target database.

sluice restore --from s3://my-bucket/app-chain \
    --target-driver postgres --target ...

Pair with sync start --position-from-manifest URL — point it at the chain URL whose terminal manifest's EndPosition becomes the stream's resume position, so CDC picks up from the chain's tail without re-bulking. (PG soft preflight warnings — wal_keep_size sufficiency, Patroni-managed source — fire here; --strict-preflight promotes them to refusals.)

Drive both restore parallelism axes (tables × within-table chunks, product bounded by the target budget):

sluice restore --from s3://my-bucket/app-chain \
    --target-driver postgres --target ... \
    --table-parallelism 4 --bulk-parallelism 4

Cross-engine restore (a MySQL backup into a Postgres target): --target-driver accepts any registered engine — the backup's source engine and the restore target need not match.

sluice restore --from s3://my-bucket/mysql-chain \
    --target-driver postgres --target 'postgres://user:pass@host:5432/app'

sluice backfill #

Backfill or transform a column in place — a same-database, keyset-chunked, resumable, online-safe UPDATE. The 'migrate' step of the expand-contract pattern (ADR-0159).

Backfill is single-endpoint — it runs INSIDE one database (no source/target pair), walking the table's primary key in bounded batches and issuing one UPDATE per batch, so no statement ever locks (or hits the statement-time wall of a managed provider on) more than --batch-size rows. The cursor persists in the same database's sluice_migrate_state control tables, so a killed run resumes where it left off — the crash-replay window is at most one chunk, and the replayed chunk is a no-op under a self-describing --where guard. The --set expressions and the --where predicate are native SQL for the --driver engine, emitted verbatim (same-database, so there is no cross-dialect translation to do).

# expand step done (new column exists); backfill it online, then gate the contract step
sluice backfill --driver planetscale --dsn 'user:pass@tcp(host)/app' --table users \
    --set "full_name = CONCAT(first_name, ' ', last_name)" \
    --where 'full_name IS NULL' \
    --verify

sluice expand-contract #

Drive the full expand → migrate → contract schema-change pattern on a PlanetScale database: deploy-request the ADD COLUMN, run the online backfill, verify, and (only with --yes) deploy-request the DROP COLUMN (ADR-0162).

This command mutates a production branch. It is PlanetScale-specific by design: it needs the control-plane service token on top of the data-plane DSN, and the production branch must have safe migrations enabled — deploy requests are the mechanism the expand and contract legs ship through. sluice never flips that toggle for you: with safe migrations off it refuses with SLUICE-E-PS-SAFE-MIGRATIONS-DISABLED. The legs: expand creates a sluice dev branch, applies --expand-ddl, and deploys it via a deploy request (sluice's control tables ride inside the same deploy); migrate runs the backfill against the production data with your --set/--where; verify re-counts the --where guard; contract — a destructive DROP COLUMN — runs only after a clean verify and --yes. Without --yes (or without --contract-ddl) the run stops after verify and prints the exact resume command, so the destructive leg is always an explicit second decision.

export PLANETSCALE_SERVICE_TOKEN_ID=...
export PLANETSCALE_SERVICE_TOKEN=...
sluice expand-contract --org acme --database app \
    --dsn 'user:pass@tcp(aws.connect.psdb.cloud)/app?tls=true' --table users \
    --expand-ddl 'ALTER TABLE users ADD COLUMN full_name VARCHAR(255)' \
    --set "full_name = CONCAT(first_name, ' ', last_name)" \
    --where 'full_name IS NULL' \
    --contract-ddl 'ALTER TABLE users DROP COLUMN first_name, DROP COLUMN last_name' \
    --yes

sluice deploy-ddl #

Ship ONE verbatim DDL statement to a PlanetScale production branch safely, as one command: dev branch (with the stale-base freshness gate), apply the DDL, deploy request, deploy, finalize, cleanup (ADR-0165).

This command mutates a production branch (through PlanetScale's governed deploy-request channel). It replaces five hand-driven pscale commands plus a hazard the operator can't see — a fresh PlanetScale dev branch can silently propose reverting recent production schema (the SLUICE-E-PS-BRANCH-STALE-BASE gate above catches it). It requires safe migrations ON the branch (the deploy-request prerequisite; without safe migrations, direct DDL works and this command is unnecessary). There is no data-plane DSN: the DDL runs on the dev branch via a just-minted branch password. The named consumer is the one-time control-table bootstrap on a safe-migrations branch — control-tables ddl prints the statements to ship.

# bootstrap sluice's control tables on a safe-migrations branch:
sluice control-tables ddl            # prints the exact CREATE statements
sluice deploy-ddl --org acme --database app \
    --ddl 'CREATE TABLE IF NOT EXISTS sluice_migrate_state (...)'   # one statement per run

sluice control-tables ddl #

Print the exact CREATE statements for sluice's own control tables (migrate-state + cdc-state), single-sourced from the engine's definitions — for bootstrapping a target that refuses direct DDL (ADR-0165).

Read-only, needs no credentials and no org/database — output is pure SQL plus -- comment lines, so it pastes or pipes into any governed channel: deploy-ddl (one statement per run), the PlanetScale UI, or a reviewed migration file. On a PlanetScale branch with safe migrations enabled, direct DDL is refused (Error 1105, surfaced as SLUICE-E-PS-DIRECT-DDL-BLOCKED) — sluice's own ensure paths are detect-first, so pre-creating the control tables this way lets migrate / sync / backfill run against the branch without ever needing a direct CREATE.

sluice control-tables ddl                       # planetscale dialect (default)
sluice control-tables ddl --engine mysql        # same dialect, spelled for vanilla MySQL

sluice trigger setup #

Install a trigger-CDC engine's source-side state — slot-less continuous CDC for managed Postgres that blocks logical replication, a local SQLite file, or a live Cloudflare D1.

sluice trigger setup --dsn 'postgres://user:pass@host:5432/app' \
    --tables=orders,customers --allow-polled-fingerprint
# then stream with the trigger engine:
sluice sync start --source-driver postgres-trigger --source ... --target-driver mysql --target ... --stream-id app

sluice trigger teardown #

Remove every trace of the trigger engine from the source Postgres database — the counterpart to trigger setup. Run it once the stream is finished to leave the source clean.

sluice trigger teardown --dsn 'postgres://user:pass@host:5432/app' --yes

sluice trigger prune #

Reap durably-applied rows from a trigger-CDC source's sluice_change_log while a sync is live — the capture path never removes consumed rows, so the change-log grows unbounded for the life of a continuous sync (ADR-0137).

The correctness crux: a change-log row is pruned only if its id is at or below the watermark the applier has persisted to the target. The exactly-once contract advances that watermark only on durable apply, so the target's persisted position is the durably-applied frontier — pruning on the source's MAX(id), the read cursor, or a TTL would delete not-yet-applied rows and cause silent permanent loss on the next warm-resume. Run it periodically against a live trigger-CDC sync (especially d1-trigger, where change-log growth and per-write billing both matter):

# preview the bound, delete nothing
sluice trigger prune --source-driver sqlite-trigger --source ./app.db \
    --target-driver postgres --target 'postgres://user:pass@host:5432/app' \
    --stream-id app --dry-run

# reap durably-applied rows, keeping a 1000-id margin, then reclaim space
sluice trigger prune --source-driver sqlite-trigger --source ./app.db \
    --target-driver postgres --target 'postgres://user:pass@host:5432/app' \
    --stream-id app --keep 1000 --vacuum

sluice schema preview · diff #

Inspect translation without moving data: print the target DDL sluice would emit, or diff a live target against what sluice would produce.

sluice schema preview --source-driver mysql --source ... --target-driver postgres
sluice schema diff    --source-driver mysql --source ... --target-driver postgres --target ...

sluice verify #

Compare data integrity between source and target — row counts by default, escalating to sampled or full per-row hashing.

sluice verify --source-driver mysql --source ... --target-driver postgres --target ... --depth count
sluice verify --source-driver mysql --source ... --target-driver postgres --target ... --depth sample

sluice matview refresh #

Refresh PostgreSQL materialized views on the target (PG-only). Handy as a scheduled job after a sync catches up.

sluice matview refresh --target-driver postgres --target ... \
    --matview daily_totals --target-schema reporting

--matview takes bare matview names (comma-separated, repeatable) that match pg_matviews.matviewname case-sensitively; the schema is named separately with --target-schema (default public). Omit --matview to refresh every matview in the schema. Add --concurrently to emit REFRESH MATERIALIZED VIEW CONCURRENTLY (requires a unique index on the matview; readers stay live).

sluice slot list · drop #

Manage source-side Postgres replication slots — list sluice-created slots, or drop an orphaned one left by an interrupted stream.

sluice slot list --source-driver postgres --source ...
sluice slot drop --source-driver postgres --source ... --slot-name sluice_slot

sluice diagnose #

Assemble an operator bundle (source/target capability + role state, debug-zip shape) to attach when filing an issue.

sluice diagnose --source-driver mysql --source ... --target-driver postgres --target ... --out ./sluice-diagnose.zip

Supply the five PlanetScale telemetry flags — --planetscale-org, --planetscale-metrics-token-id / --planetscale-metrics-token (env), --planetscale-metrics-db (defaults to the --target DSN's database), --planetscale-metrics-branch (default main) — to add a target-health metrics snapshot (CPU/mem/storage/lag) to the bundle. Control-plane credential, distinct from --target. See sync start for the same flag semantics.

sluice metrics-watch #

Standalone PlanetScale control-plane metrics daemon — poll a database's CPU/mem/storage/lag on an interval and fire threshold alerts, with no migration or sync attached. Opens NO connection to the database itself; reads only the PlanetScale metrics API.

Run as an alert-only daemon (tokens via env; fire on 85% storage):

export PLANETSCALE_METRICS_TOKEN_ID=...
export PLANETSCALE_METRICS_TOKEN=...
sluice metrics-watch --engine planetscale --planetscale-org acme --planetscale-metrics-db app \
    --notify-storage-util 0.85 --notify-slack "$SLACK_URL" --quiet