Blob reclamation
Several metadata records can refer to one digest. Distributed reclamation accounts for replicas and backups that have not applied the current reference snapshot. It records a durable decision before a storage executor removes bytes.
The selector, ledger, and scheduled job ship as HA components. The job requires a nonzero ownership term, so it performs
no work in dc, which has no ownership consensus.
Retained references
A selector builds the retained set from these sources:
- Owner-provided references to immutable content.
- Restorable trash entries.
- Verified placements that can serve the digest.
Each content owner implements the shared reference-inventory trait. The availability layer receives only a set of content digests and does not import owner metadata types.
An inventory spans several owner reads that cannot share one metadata transaction, so the selector reads an internal reference revision on both sides of the scan and keeps the result only when the revision did not move. Every driver-row write advances that revision, including one that appends no replication journal entry, so a reference committed part way through the scan retires the inventory rather than leaving a digest that an earlier read already reported as unreferenced.
Selection and readiness each prove their own inventory, and each reclamation write carries the revision its verdict came from. A reference committed after the proof moves the revision, the compare-and-put refuses the write, and the tombstone keeps the state it had. A digest outside the retained set is a candidate. A returned reference abandons an existing candidate.
Frontier gate
Candidate selection writes a reclamation tombstone with the current authoritative metadata serial as its required frontier. Deletion waits until each live replica and configured backup has applied that serial. One lagging plane keeps the candidate pending.
Replication planes publish applied frontiers through cluster liveness. Until a source reports a frontier, readiness uses zero. A candidate with a nonzero requirement cannot advance on missing evidence.
Fencing
One cluster worker selects and advances tombstones. Its singleton lease uses the ownership group's monotonic term. Each tombstone transition records that term and rejects a stale term. A process without an ownership group uses term zero and cannot reclaim content.
The first reclamation write creates its ledger; reads return an empty result before that write. Peryx advances a tombstone when the ownership term and reference scan still match, using one atomic update.
Reference guard
A delete and a reference commit race, so the collector that purges orphaned blobs arms a per-digest reference guard around its delete. Arming carries the reference revision the collector's unreferenced verdict came from, and the store re-reads that revision inside the arming transaction. A reference committed while the inventory was being scanned moves the revision, the arm is refused, and the collector re-scans rather than guarding a digest that is no longer orphaned.
While a guard is armed, a commit that names the digest as a blob reference is rejected, whether or not it appends a replication journal entry. A replica applying the primary's journal is admitted, because the primary fenced that write against its own guards before journaling it. The guard is released once the bytes are gone.
Tombstone states
| State | Meaning |
|---|---|
Pending | No current reference or serveable placement exists; required frontiers have not cleared. |
Ready | Frontiers cleared and final reference checks passed. Bytes stay until a collector claims the digest. |
Skipped | A reference or serveable placement returned. |
Each fenced transition increments attempts. Selecting the digest again raises its required frontier and returns it to
pending, including after it reached Ready. Selection runs in bounded batches outside request handling.
Each pass reads at most one batch of stored digests and one batch of tombstones, both in digest order, and records where each scan stopped. The next pass resumes from those positions and wraps to the first row once a scan reaches the end, so the configured batch bounds the rows a pass reads rather than only the rows it changes.
Backups
A completed backup captures the reference set at its recovery point. It does not retain future content. Reclamation waits for the applied frontier of each configured backup so the backup can capture every digest referenced through the candidate serial.
Recovery and metrics
Durable tombstones and attempt counts survive restart, snapshot, and restore. A resumed pass continues from the recorded scan positions; a crash before a position advances repeats a page rather than skipping one. A bounded prune removes terminal tombstones.
Metrics expose low-cardinality counts for pending, ready, and skipped tombstones.