Backup & recovery
A backup you have not restored is a hope
Backup jobs report success for years and then fail the one time it matters — the wrong data, an unusable format, or an encryption key nobody kept.
In short
Backup is the copy; recovery is the capability, and only one of them matters. The two numbers that define it are RPO — how much data you can afford to lose, which sets backup frequency — and RTO — how long you can afford to be down, which sets how recovery is architected. Neither is a technical decision; both are business decisions about acceptable loss. A backup that has never been restored into a working system should be treated as untested, because in practice that is what it is.
When it applies
Signals this is the work
If more than one of these is true today, the problem is usually further along than it looks from the outside.
Restores have never been tested
The job is green. Nobody has proved it produces a working system.
Nobody can state RPO or RTO
There is no agreed answer to how much data may be lost or how long recovery may take.
Backups sit beside the thing they protect
Same account, same region, same credentials. A compromise that reaches one reaches both.
Only the database is covered
Configuration, secrets, uploaded files and infrastructure definitions are not, so a restore produces a database and no system.
Ransomware would reach the backups
Nothing is immutable or offline, so encrypted production means encrypted backups.
How we do it
The sequence
- 01
Set RPO and RTO with the business
Ask what an hour of lost data costs and what a day of downtime costs. The architecture follows from the answers rather than preceding them.
- 02
Back up the whole system
Data, configuration, secrets, files and the infrastructure definition. A restore has to produce something that runs.
- 03
Separate and make immutable
Different account, different region, write-once retention. Backups must survive the event that destroys production.
- 04
Restore on a schedule
Into a real environment, timed against the RTO. This is the only evidence that any of it works.
- 05
Write the runbook
Step by step, followable by someone who did not design it, under pressure, possibly at night.
- 06
Rehearse the whole failure
Not just the database restore — the full path including DNS, certificates, integrations and the decision to declare an incident.
What we will not do
The refusals
- Report backup success without a restore test behind it.
- Set an RTO the architecture cannot meet, because it sounds reassuring.
- Store backups where the same compromise reaches both copies.
- Write a recovery plan nobody has ever executed.
Questions
Before you ask
Tell us what isn't working.
One process, one system, one decision you are stuck on. We will come back with how we would approach it, what it would take, and whether it needs building at all.