Skip to content
Sign in
← Documentation

Backup and restore

FeltDB supports verified offline bundles, write-consistent online bundles, and point-in-time selection

Backup and restore

FeltDB supports verified offline bundles, write-consistent online bundles, and point-in-time selection at completed backup boundaries. It does not synthesize a recovery point between backups; backup frequency therefore determines the recovery-point objective.

Create and verify

feltdb-server backup create \
  --data /var/lib/feltdb/state.log \
  --keys /var/lib/feltdb/api-keys.json \
  --audit /var/lib/feltdb/state.audit.log \
  --output /backups/feltdb-2026-08-11

feltdb-server backup verify /backups/feltdb-2026-08-11

Paths may be absolute, ./-relative, or bare relative (feltdb-server backup create --output backup); all resolve against the working directory and behave identically, and missing parent directories are created. feltdb-server backup <path> is shorthand for backup create --output <path>, and backup verify <path> for backup verify --archive <path>.

create reports success only after the bundle verifies independently of the code that wrote it; the exit status is the outcome. With --json, both commands print the result:

{
  "operation": "create",
  "status": "ok",
  "path": "/backups/feltdb-2026-08-11",
  "backup_id": "b239c69f…",
  "source": { "data": "/var/lib/feltdb/state.log", "state_sha256": "d735f346…", "server_version": "0.2.0", "durable_format": 2 },
  "format": 1,
  "created_ms": 1790339429136,
  "files": 3,
  "bytes_written": 1642,
  "verification": "verified",
  "duration_ms": 4
}

backup_id is the SHA-256 of the manifest and identifies the bundle. Verification checks that the manifest is readable and in a supported format, that every entry's length and SHA-256 match, that no file absent from the manifest is present, and that the bundle restores into a database this build can open (on a private copy). A bundle that fails any of these is reported failed with exit status 1 and is never described as valid. feltdb-server doctor --data <log> --backup <bundle> runs the same checks as part of a health report.

Creation writes to a .partial sibling and atomically renames only after every file and the manifest are durable. Existing output and stale partial directories cause failure. The format includes the canonical state log, checkpoint, replication acknowledgements, membership and recovery proposal, lease promises, API-key hashes, audit log, and content-addressed object tree. Missing optional files are omitted explicitly from the manifest.

Online backup

Create an API key with the cluster:write scope and request a backup from the running server:

curl -fsS -X POST http://127.0.0.1:7700/admin/backups \
  -H "Authorization: Bearer $FELTDB_ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"output":"/backups/feltdb-2026-08-11-live"}'

The server blocks mutations while it exports a hash-verified state snapshot with its causal version vector and copies membership, lease, authentication, audit, and content artifacts. Reads remain available. The output path is resolved on the server and must not already exist. A successful response is the completed manifest; verify the bundle before transferring or restoring it.

Online bundles contain database.snapshot.json in place of a live state log. Restore verifies the bundle and installs this snapshot into a pristine native state log, including its causal metadata.

Restore to a point in time

Keep completed bundles as direct children of a catalog directory, then supply a Unix timestamp in milliseconds:

feltdb-server backup restore \
  --archive /backups/feltdb-catalog \
  --at-ms 1786442400000 \
  --output /var/lib/feltdb-restored

FeltDB selects the newest completed bundle whose manifest timestamp is at or before the requested time. It verifies every manifest-bearing bundle in the catalog before selection. A corrupt candidate or the absence of an eligible recovery point fails the restore; FeltDB never silently falls back to older customer state. The selected snapshot boundary, rather than individual row wall-clock values, defines the recovery point and preserves causal metadata after compaction.

Restore

feltdb-server backup restore \
  --archive /backups/feltdb-2026-08-11 \
  --output /var/lib/feltdb-restored

feltdb-server --data /var/lib/feltdb-restored/files/state.log \
  --keys /var/lib/feltdb-restored/files/api-keys.json \
  --audit /var/lib/feltdb-restored/files/audit.log

Restore verifies the format, path safety, byte lengths, and SHA-256 of every entry before staging any installation. It refuses to overwrite an existing target. Corrupt, partial, traversal-bearing, and unsupported-format bundles are never installed.

Recovery objectives

Recovery point objective (RPO): your backup interval. FeltDB restores to the most recent backup and does not create recovery points between backups. Backing up every hour bounds the loss to an hour of writes; daily backups bound it to a day. Schedule feltdb-server backup create, or the online /admin/backups API (cluster:write scope), at the interval you can afford to lose.

Recovery time objective (RTO): not characterized beyond the test configuration.

On test configurations (single-node, moderate state size), measured RTO is < 1 second including verification. Production RTO scales with:

  • Total state size
  • Disk I/O throughput
  • Network transfer time (for multi-region setups)

Benchmark your environment:

# Create a full backup
time feltdb-server backup create --data state.log --keys api-keys.json --audit state.audit.log --output test-backup

# Verify integrity
time feltdb-server backup verify --archive test-backup

# Measure restore
rm -rf test-restore
time feltdb-server backup restore --archive test-backup --output test-restore

Document and alert on RTO regressions. If measured RTO exceeds 5 minutes, review state size, disk configuration, and network capacity.

Drill policy

Run a restore drill at least monthly and before every production release. Record bundle size, creation time, verification time, restore time, restored state count, content hash results, authentication check, subscription replay, and workflow/agent recovery.

.github/workflows/disaster-recovery.yml runs the authenticated offline and online clean-restore drill monthly and supports an operator-triggered run; it retains the test report for 90 days.