Durable transaction recovery after a lost response
A Rust reference study of durable receipt recovery when a transfer commits but its response is lost. Inspect the failure experiments and operating limits.
The response can disappear after the state changes.
The caller still has a decision to make
After an uncertain transfer, retrying with a new identity can repeat an operation that already succeeded. Abandoning it can leave a completed operation unreconciled. The system needs a way to resolve that earlier outcome before the caller authorizes another one.
Atom’s reference system makes that uncertainty inspectable. It moves synthetic units between two fixed accounts, deliberately withholds an acknowledgement after storage completes, then accepts the same command in a later request. This is an engineering reference built by Atom, with no customer results or real funds involved.
Keep the operation recoverable under one owner
We kept the Rust Worker and added one SQLite-backed Durable Object. A stored, ordered command log is the authority for each session. Replaying that log reconstructs balances and original receipts together. An exact retry can recover its first receipt across requests and execution restarts, while the session remains valid and request capacity is available.
The decision fits a small public study: one shared coordinator, bounded sessions, and no external effects. It accepts a shared serialization point and dependence on Cloudflare’s runtime and storage. Independent capacity, longer retention, or a service that must deliver an external effect would reopen the design.
The stored log owns the answer
The caller keeps a session identity, a bearer capability, and the command it may need to repeat. The Rust Worker parses the request and routes it to a fixed TransferCoordinator. Inside that coordinator, the capability hash must match the requested session before its request allowance is spent. The capability authorizes only that session’s synthetic accounts.
The pure Rust ledger binds each command identity to its source account, destination, and amount. Equal identity and equal content return the original receipt. Changed content returns a conflict. Successful commands are stored in commit order; balances are reconstructed instead of written as a second copy that could drift from the receipts.
The runtime’s block_concurrency_while gate covers read, validation, mutation, and persistence. One storage put replaces the versioned application value, and the adapter awaits put and sync before returning success or the deliberate acknowledgement-withheld response. Another request must pass through the same owner and read the committed history.
Retain the command
Session, capability, identity, and unchanged content
Authorize, validate, commit
Storage completion precedes the response.
The alternatives change the operating burden
Keeping the original request-local study would preserve its simpler setup and sequential demonstrations. Its in-memory state ends with the request, so it cannot answer a retry arriving in a later execution. The original API and article remain valid within that narrower scope.
One Durable Object per anonymous visitor would give sessions independent coordination capacity. It would also permit unbounded object creation without an additional admission and lifecycle mechanism. A fixed coordinator gives this reference a finite shared boundary that is easy to inspect.
A transactional SQL service could enforce command uniqueness and atomically commit balances and receipts. It would add a service to operate, credentials, and migration work while providing more flexible querying and growth. A VPS would add host supervision and recovery. Neither was necessary for the study’s bounded synthetic sessions; no workload benchmark justified that expansion.
Read the original request-local noteOne public session, two committed transfers
The deployed API check completed on 2 October 2026. It exercised one bounded synthetic session at the public origin. A separate final read returned 650 operating units, 350 reserve units, two committed transfers, and 22 remaining authorized requests.
First, a 250-unit transfer completed its durable write. The deliberate failure returned HTTP 503 with acknowledgement_withheld and no receipt. Repeating the same command recovered the original receipt in a separate request, with 750 operating units and 250 reserve units.
Six concurrent requests submitted the same new 100-unit transfer. One returned applied and five returned replayed. All six identified receipt 2. The additional effect was one 100-unit movement.
A request that changed the original command’s content was rejected. A wrong capability was also denied without consuming the legitimate session’s allowance. These checks observed behavior at the tested origin and time; they establish neither production throughput nor regional failover.
Break the execution, then inspect what survived
In the local compiled-WASM workerd suite, the test committed another transfer with its acknowledgement withheld, killed the runtime with SIGKILL, and restarted it against the same storage. Retries recovered both that latest receipt and an earlier original receipt. A separate test discarded a real HTTP response and then recovered the operation on retry.
An independent review found a more subtle failure: the SDK’s optional string read treated a stored JavaScript null as though the key were absent. That could initialize a fresh history over an invalid stored value. The loader now checks key presence before it requires a string and validates the versioned log. Unknown or corrupt state returns an error and remains intact.
The released source passed 22 native tests and 22 isolated local runtime checks. Runtime regressions covered present null, undefined, numbers, booleans, arrays, and objects, plus real expiry alarms and capacity rejection. An independent compiled-WASM probe exercised 16 corrupt representations using in-memory storage. Process termination was a local experiment; SDK panic-wrapper recovery remains disabled.
Retention and capacity are part of the promise
A session expires 15 minutes after creation, with at most eight committed commands and 32 authorized requests. Reads and retries consume that allowance. Expiry ends replay authority: a new session cannot recover an expired session’s receipt. Request cleanup and an alarm release expired capacity, including while new v2 requests are paused.
All visitors share 32 live sessions, 60 admitted requests per minute, and eight creations per minute. Overload rejects work without evicting live receipts. One visitor can consume availability for others. These bounds are appropriate to a public demonstration; they do not provide a hard spending cap or protection against shared-capacity abuse.
Request bodies are limited to 512 bytes and encoded application state to 64 KiB. That state bound excludes SQLite metadata, its write-ahead log, platform backups, and logs. Rejected ingress can still incur platform requests. Larger workloads and longer recovery windows need their own measured capacity and retention decision.
A code restore cannot undo a stored transfer
The initial Durable Object class migration makes the pre-migration Worker an invalid rollback target. Recovery must preserve the class, binding, migration history, and records. The operator can set STUDY_ENABLED=false to pause v2 before object access, while v1 and existing expiry alarms continue, then deploy a compatible repair. Later code rollback requires a reader that understands the stored state.
The reference commits only its own synthetic ledger. It does not settle an external payment, promise exactly-once delivery to another service, or demonstrate recovery from a cloud-region outage. A production design with external effects must establish that provider’s identity, acknowledgement, and reconciliation contract. The browser cannot infer that an effect failed from a missing response.
Bring the uncertain operation to an assessment
A paid Critical Systems Assessment starts with the decision your team needs to make. Bring the write path, its callers, storage and migration history, and an example of an uncertain outcome. We identify who owns the command and state, reproduce the material failure, and compare a bounded repair with the alternatives your system can support.
The deliverable is a decision brief with evidence, operating limits, acceptance checks, and a staged execution plan your team can use. That work can lead into a Core Systems Engineering program when the decision calls for implementation. This reference supplies a concrete method; your system’s constraints determine the recommendation.