Define the service that must recover
A restore rehearsal needs a final decision: can the recovered service perform its required work, using the intended data, before traffic is admitted? A copied backup and a started database are intermediate observations. For this example, consider a PostgreSQL 18 cluster restored from a full physical base backup and archived write-ahead logs, or WAL. The exercise takes place in an isolated environment and includes the application that uses the recovered database.
State the failure being rehearsed and the recovery objectives before starting. AWS defines recovery time objective, or RTO, around the interval from service interruption to service restoration. Recovery point objective, or RPO, bounds the acceptable age of the recovery point. Set both from business needs and dependency capabilities. A database-only stopwatch leaves out work that may still prevent the application from serving a request.
Establish the backup and recovery chain
PostgreSQL documents pg_basebackup as a backup of the entire cluster. Identify the source cluster, selected backup, tablespaces, and intended recovery point in the rehearsal record. Continuous archive recovery requires the relevant unbroken WAL sequence from the start of the base backup through the intended recovery point. Confirm the intended timeline as well. A recent-looking archive filename alone is insufficient evidence that this backup can reach this target.
pg_verifybackup checks a pg_basebackup backup against its manifest. Its documentation warns that this does not reproduce every check made by a running server; test restores remain necessary. It also says extra WAL beyond what is needed to recover the backup is not checked. A verification pass for the base backup therefore leaves a separate question about the archive chain to the intended target. Retain the verification result and the actual recovery result as different evidence.
Restore the dependencies that make the data usable
Record the application build expected to work with the recovered schema, its required database extensions, and the configuration needed in the recovery environment. Treat these as retrievable release artifacts with owners. PostgreSQL explicitly states that WAL replay does not restore changes to its configuration files. The database backup cannot be the sole authority for reconstructing the service configuration around it.
Use the rehearsal to demonstrate that the selected application can start, authenticate through its intended path, and execute representative business reads and controlled writes. Keep outbound effects contained in the isolated environment. For data held outside PostgreSQL, such as documents referenced by recovered rows, identify the separate recovery dependency and test the required access. A consistent PostgreSQL recovery point does not establish a common recovery point across independent systems.
Choose verification cases before the outage exercise. For an order service, those might include a known order that should exist at the target point, expected relationships between that order and its lines, and a controlled new order using the intended schema. Define the expected result for each case and retain failures. Passing a few selected cases supports those cases; it does not certify every recovered row or every application path.
- Recovery inputsBind the backup to the target
Identify the source cluster, full backup, required WAL chain, timeline, and intended recovery point.
- Database recoveryObserve the recovered state
Retain backup verification and recovery results; confirm the database reached the intended state.
- Application readinessExercise the required service paths
Use the selected application build and recovery configuration to check the declared business cases.
- Operator decisionAccept or hold traffic readiness
Compare data age and elapsed recovery time with the objectives, then record remaining dependencies and the owner.
Time the whole path in the worked scenario
Assume an interruption at 12:00 UTC, an RPO of ten minutes, and an RTO of 90 minutes. Assume the chosen backup and WAL chain can recover to 11:54 UTC. Its six-minute gap is within the assumed RPO, leaving four minutes of allowance. This says nothing about the number or business significance of operations in the missing interval; that requires separate evidence.
For a planning calculation, allocate ten minutes to detection and preparation, 30 to restoring files, 20 to WAL recovery, 15 to application dependencies, 20 to verification, and five to the rehearsed traffic handoff and service confirmation. Assume these phases run sequentially. Database recovery ends after 60 minutes, but the complete path takes 100 minutes. The plan exceeds the 90-minute RTO by ten minutes even though the recovery-point requirement is met.
These durations are invented inputs, not a restore result or a sizing recommendation. A real rehearsal should record phase boundaries and overlaps so elapsed time follows the actual critical path. If a dependency is prepared in parallel, measure that overlap. If operator access or artifact retrieval delays the start, retain that delay rather than beginning the clock when the file copy finally starts.
Make the operator verification repeatable
AWS guidance calls for periodic restores to verify data integrity and recovery objectives. Use the operational runbook during the exercise and have the recovery owner retain the target, timestamps, checks, and unresolved failures. An instruction that depends on an unavailable author is a finding. Assign an owner to correct it and identify which part of the rehearsal needs to run again.
Traffic readiness also depends on the recovered environment having the capacity required for the intended service. AWS warns that a secondary environment can outgrow capacity demonstrated in an earlier test. A functional restore with a small sample establishes limited evidence. Keep capacity evidence separate and current for the proposed traffic level; this article provides no workload test or capacity result.
Record what the rehearsal actually permits
The handoff should identify the recovered database and application versions, the proven recovery point, elapsed time through service confirmation, and the acceptance cases that passed or failed. Name the person responsible for the traffic decision and any unresolved external-data dependency. In the worked scenario, the recovery point fits the requirement while the planned service restoration misses it. The next review should address the slow phases or the recovery design, with the verification steps still included.
