In partnership with
THE PROBLEM
Every backup product sends a success email. Almost nobody reads them, and the ones who do are reading a claim rather than a fact.
The failure modes here are specific and they’re all quiet:
The job succeeded but wrote nothing. Source path changed, a volume got remounted elsewhere, a folder got renamed during a migration. The job ran, found nothing to copy, exited zero, and reported success. The backup file is 4KB.
The job succeeded on Monday and hasn’t run since. The scheduled task got disabled during troubleshooting, or the service account password expired. The last email you have says success — it just arrived eleven days ago and nobody noticed the silence.
The backup target filled up. Retention stopped pruning, the volume hit 100%, and writes started failing. Some products handle this gracefully and alert. Some write a truncated file and call it done.
The files are there but shrinking. A database backup that was 40GB last month and is 2GB today didn’t get more efficient. Something changed about what’s being included.
The common thread is that the backup software is reporting on its own behaviour, and you’re taking its word for it. This week we check the actual files on the actual target and compare them against what we expect to be there — independently of whatever the backup product claims.
THE SCRIPT
Save the script below to:
C:\Scripts\BackupVerify.ps1powershell
# Backup File Verification
# Automate & Operate — automateandoperate.com
# Email settings — update these
$from = "[email protected]"
$to = "[email protected]"
$smtpServer = "your.smtp.server"
$smtpPort = 587
$username = "[email protected]"
$password = ConvertTo-SecureString "yourpassword" -AsPlainText -Force
$credential = New-Object System.Management.Automation.PSCredential($username, $password)
$logFile = "C:\Logs\BackupVerify.log"
$reportFile = "C:\Logs\BackupVerifyReport.csv"
$historyFile = "C:\Logs\BackupVerifyHistory.csv"
# ---- DEFINE WHAT YOU EXPECT TO EXIST ----
# One entry per backup set you want verified.
# Path = folder where the backup files land (local or UNC)
# Filter = filename pattern to match
# MaxAgeHours = newest file must be younger than this
# MinSizeMB = newest file must be at least this big
# Recurse = search subfolders
$backupSets = @(
@{
Name = "SQL Full Backups"
Path = "\\BACKUPSRV\Backups$\SQL"
Filter = "*.bak"
MaxAgeHours = 26
MinSizeMB = 500
Recurse = $true
},
@{
Name = "File Server Nightly"
Path = "\\BACKUPSRV\Backups$\FileServer"
Filter = "*.vbk"
MaxAgeHours = 26
MinSizeMB = 1000
Recurse = $false
},
@{
Name = "Exchange / Mail"
Path = "\\BACKUPSRV\Backups$\Mail"
Filter = "*.bkf"
MaxAgeHours = 26
MinSizeMB = 200
Recurse = $false
},
@{
Name = "GPO Backups"
Path = "C:\GPOBackups"
Filter = "manifest.xml"
MaxAgeHours = 26
MinSizeMB = 0
Recurse = $true
}
)
# Alert if the backup target volume drops below this % free
$targetFreeSpaceWarning = 15
# Flag if newest file shrank by more than this % versus the 7-day average
$shrinkWarningPercent = 40
# Create log folder if missing
if (-not (Test-Path "C:\Logs")) {
New-Item -ItemType Directory -Path "C:\Logs" -Force | Out-Null
}
$timestamp = Get-Date -Format "yyyy-MM-dd HH:mm:ss"
$dateStamp = Get-Date -Format "yyyy-MM-dd"
$report = @()
$failures = @()
$warnings = @()
$spaceIssues = @()
foreach ($set in $backupSets) {
$name = $set.Name
$issues = @()
# ---- Does the path even exist? ----
if (-not (Test-Path $set.Path)) {
$entry = [PSCustomObject]@{
Date = $dateStamp
Name = $name
Path = $set.Path
NewestFile = "N/A"
FileAgeHours = "N/A"
SizeMB = 0
FileCount = 0
TotalSetSizeMB = 0
Status = "FAIL"
Issues = "BACKUP PATH UNREACHABLE OR MISSING"
}
$report += $entry
$failures += $entry
Add-Content -Path $logFile -Value "$timestamp — $name — FAIL — path unreachable: $($set.Path)"
continue
}
try {
$files = Get-ChildItem -Path $set.Path -Filter $set.Filter -File `
-Recurse:$set.Recurse -ErrorAction Stop
if ($files.Count -eq 0) {
$entry = [PSCustomObject]@{
Date = $dateStamp
Name = $name
Path = $set.Path
NewestFile = "NONE FOUND"
FileAgeHours = "N/A"
SizeMB = 0
FileCount = 0
TotalSetSizeMB = 0
Status = "FAIL"
Issues = "NO FILES MATCHING '$($set.Filter)'"
}
$report += $entry
$failures += $entry
Add-Content -Path $logFile -Value "$timestamp — $name — FAIL — no files matching $($set.Filter)"
continue
}
$newest = $files | Sort-Object LastWriteTime -Descending | Select-Object -First 1
$ageHours = [math]::Round((New-TimeSpan -Start $newest.LastWriteTime -End (Get-Date)).TotalHours, 1)
$sizeMB = [math]::Round($newest.Length / 1MB, 2)
$totalSizeMB = [math]::Round((($files | Measure-Object -Property Length -Sum).Sum) / 1MB, 2)
# ---- Age check ----
if ($ageHours -gt $set.MaxAgeHours) {
$issues += "STALE: newest file is $ageHours h old (limit $($set.MaxAgeHours) h)"
}
# ---- Size check ----
if ($sizeMB -lt $set.MinSizeMB) {
$issues += "TOO SMALL: $sizeMB MB (minimum $($set.MinSizeMB) MB)"
}
# ---- Zero-byte check ----
if ($newest.Length -eq 0) {
$issues += "ZERO BYTE FILE"
}
# ---- Still being written? ----
try {
$stream = [System.IO.File]::Open($newest.FullName, 'Open', 'Read', 'None')
$stream.Close()
$stream.Dispose()
} catch {
$issues += "FILE LOCKED (may still be writing)"
}
# ---- Shrink check against 7-day history ----
if (Test-Path $historyFile) {
try {
$history = Import-Csv -Path $historyFile |
Where-Object { $_.Name -eq $name -and $_.SizeMB -gt 0 } |
Sort-Object Date -Descending |
Select-Object -First 7
if ($history.Count -ge 3) {
$avgSize = ($history | Measure-Object -Property SizeMB -Average).Average
if ($avgSize -gt 0) {
$shrinkPercent = [math]::Round((($avgSize - $sizeMB) / $avgSize) * 100, 1)
if ($shrinkPercent -gt $shrinkWarningPercent) {
$issues += "SHRANK $shrinkPercent% vs 7-day avg ($([math]::Round($avgSize,0)) MB)"
}
}
}
} catch { }
}
$status = if ($issues.Count -eq 0) {
"OK"
} elseif ($issues -match "STALE|ZERO BYTE|TOO SMALL") {
"FAIL"
} else {
"WARN"
}
$entry = [PSCustomObject]@{
Date = $dateStamp
Name = $name
Path = $set.Path
NewestFile = $newest.Name
FileAgeHours = $ageHours
SizeMB = $sizeMB
FileCount = $files.Count
TotalSetSizeMB = $totalSizeMB
Status = $status
Issues = if ($issues.Count -gt 0) { $issues -join "; " } else { "None" }
}
$report += $entry
if ($status -eq "FAIL") { $failures += $entry }
if ($status -eq "WARN") { $warnings += $entry }
Add-Content -Path $logFile -Value "$timestamp — $name — $status — newest $($newest.Name) ${sizeMB}MB ${ageHours}h old — $($entry.Issues)"
# ---- Free space on the target ----
try {
if ($set.Path -like "\\*") {
$uncRoot = ($set.Path -split '\\')[0..3] -join '\'
$drive = Get-PSDrive -PSProvider FileSystem | Where-Object { $_.Root -eq $set.Path } | Select-Object -First 1
if (-not $drive) {
# Use WMI against the host for UNC paths
$hostName = ($set.Path -split '\\')[2]
$shareName = ($set.Path -split '\\')[3]
$vol = Get-CimInstance -ClassName Win32_LogicalDisk -ComputerName $hostName -ErrorAction SilentlyContinue |
Sort-Object FreeSpace | Select-Object -First 1
if ($vol) {
$pctFree = [math]::Round(($vol.FreeSpace / $vol.Size) * 100, 1)
if ($pctFree -lt $targetFreeSpaceWarning) {
$spaceIssues += "$name — $hostName drive $($vol.DeviceID) is $pctFree% free"
}
}
}
} else {
$driveLetter = (Split-Path -Qualifier $set.Path).TrimEnd(':')
$vol = Get-CimInstance -ClassName Win32_LogicalDisk -Filter "DeviceID='${driveLetter}:'" -ErrorAction SilentlyContinue
if ($vol) {
$pctFree = [math]::Round(($vol.FreeSpace / $vol.Size) * 100, 1)
if ($pctFree -lt $targetFreeSpaceWarning) {
$spaceIssues += "$name — local drive ${driveLetter}: is $pctFree% free"
}
}
}
} catch { }
} catch {
$entry = [PSCustomObject]@{
Date = $dateStamp
Name = $name
Path = $set.Path
NewestFile = "ERROR"
FileAgeHours = "N/A"
SizeMB = 0
FileCount = 0
TotalSetSizeMB = 0
Status = "FAIL"
Issues = "ENUMERATION ERROR: $($_.Exception.Message)"
}
$report += $entry
$failures += $entry
Add-Content -Path $logFile -Value "$timestamp — $name — ERROR: $_"
}
}
# ---------- EXPORT ----------
$report | Export-Csv -Path $reportFile -NoTypeInformation
if (Test-Path $historyFile) {
$report | Export-Csv -Path $historyFile -NoTypeInformation -Append
} else {
$report | Export-Csv -Path $historyFile -NoTypeInformation
}
# ---------- BUILD EMAIL ----------
$emailBody = "Backup Verification Report — $timestamp`n"
$emailBody += "Backup sets checked: $($backupSets.Count)`n"
$emailBody += "FAIL: $($failures.Count) | WARN: $($warnings.Count) | OK: $(($report | Where-Object { $_.Status -eq 'OK' }).Count)`n`n"
if ($failures.Count -gt 0) {
$emailBody += "*** FAILED VERIFICATION — YOU MAY NOT HAVE A BACKUP ***`n"
$emailBody += "=" * 60 + "`n"
foreach ($f in $failures) {
$emailBody += "Set: $($f.Name)`n"
$emailBody += "Path: $($f.Path)`n"
$emailBody += "Newest file: $($f.NewestFile)`n"
$emailBody += "Age: $($f.FileAgeHours) hours`n"
$emailBody += "Size: $($f.SizeMB) MB`n"
$emailBody += "Problem: $($f.Issues)`n"
$emailBody += "-" * 40 + "`n"
}
$emailBody += "`n"
}
if ($warnings.Count -gt 0) {
$emailBody += "WARNINGS — INVESTIGATE:`n"
$emailBody += "=" * 60 + "`n"
foreach ($w in $warnings) {
$emailBody += "$($w.Name) — $($w.NewestFile) — $($w.SizeMB) MB — $($w.Issues)`n"
}
$emailBody += "`n"
}
if ($spaceIssues.Count -gt 0) {
$emailBody += "BACKUP TARGET RUNNING LOW ON SPACE:`n"
$emailBody += "=" * 60 + "`n"
$spaceIssues | ForEach-Object { $emailBody += " !! $_`n" }
$emailBody += "`n"
}
$emailBody += "ALL BACKUP SETS:`n"
$emailBody += "=" * 60 + "`n"
foreach ($r in $report) {
$emailBody += "[$($r.Status)] $($r.Name)`n"
$emailBody += " Newest: $($r.NewestFile) ($($r.SizeMB) MB, $($r.FileAgeHours)h old)`n"
$emailBody += " Files in set: $($r.FileCount) | Total: $($r.TotalSetSizeMB) MB`n"
}
if ($failures.Count -eq 0 -and $warnings.Count -eq 0) {
$emailBody += "`nAll backup sets verified present, recent, and plausibly sized.`n"
}
$emailBody += "`nFull report: $reportFile`n"
$emailBody += "Size history: $historyFile`n`n"
$emailBody += "NOTE: This verifies files exist, are recent, and are a sane size.`n"
$emailBody += "It does NOT verify restorability. Test restores separately.`n`n"
$emailBody += "Automate & Operate — automateandoperate.com"
$subject = if ($failures.Count -gt 0) {
"BACKUP FAILURE: $($failures.Count) backup set(s) failed verification"
} elseif ($spaceIssues.Count -gt 0) {
"Backup Warning — target volume low on space"
} elseif ($warnings.Count -gt 0) {
"Backup Verification — $($warnings.Count) warning(s)"
} else {
"Backup Verification — All $($backupSets.Count) set(s) OK"
}
Send-MailMessage `
-From $from -To $to `
-Subject $subject `
-Body $emailBody `
-SmtpServer $smtpServer -Port $smtpPort `
-UseSsl -Credential $credentialUpdate these lines:
$from/$to/$username/$password— your email details$smtpServer— your SMTP server (e.g.smtp.office365.com)$backupSets— this is the whole script, see below$targetFreeSpaceWarning— 15% is a reasonable floor
Requirements: Read access to every backup path. For UNC paths, the account running the task needs share and NTFS read. Free space checks on remote hosts need WMI access — if that’s blocked, the rest still works.
THE CONFIG IS THE WORK
Everything else in this script is plumbing. The part that determines whether it’s useful is $backupSets, and it needs to describe what you expect rather than what happens to be there.
MaxAgeHours — set this to your backup interval plus a buffer. Nightly job? 26 hours. That gives you two hours of slack for a job that starts late without false alarms, while still catching a job that didn’t run at all.
MinSizeMB — this is the one people get wrong by setting it to zero. Look at what your backups actually weigh, then set the floor at roughly half. If your SQL full is reliably 40GB, a 500MB minimum catches the catastrophic “wrote nothing” case. Set it too tight and you’ll get noise from legitimate variation; set it to zero and you’ve disabled the most valuable check in the script.
Filter — be specific. *.bak not *. If you point it at a folder with logs and temp files alongside the backups, the newest-file logic will happily report on a 2KB log and tell you everything is fine.
Start with your three most important backup sets. Not all of them. Get those three reporting accurately for a week, then expand. A script covering three sets you trust beats one covering fifteen you’ve stopped reading.
THE SHRINK DETECTION
The script keeps a size history and compares today’s newest file against the 7-day average. A drop of more than 40% gets flagged.
This catches the subtle version of backup failure — the one where everything technically works but something changed about scope. A database got dropped from the backup plan. A folder exclusion got added. A VM got removed from the job during troubleshooting and never added back.
None of those produce an error. They produce a smaller file, and nobody is looking at file sizes.
It needs three days of history before it starts evaluating, so the first few runs won’t flag anything. That’s expected.
Worth knowing: a legitimate shrink will trigger this once — after a big data cleanup, or when you move from full to incremental. Confirm it’s intentional and it’ll fold into the new average within a week.
WHAT THIS DOES NOT DO
This needs saying plainly, because the gap matters.
This script verifies that a file exists, is recent, and is a plausible size. That’s it. It does not verify the backup is restorable.
A 40GB .bak file written yesterday can still be corrupt. It can be encrypted with a key nobody has. It can be a valid backup of a database that was already broken when it was taken.
The only thing that proves a backup works is restoring it. Nothing else counts, and no amount of file verification substitutes for that.
What this script does is close the much larger gap that exists before you get to that question — the gap where you don’t have a file at all, or you have a 4KB one, or the newest one is from eleven days ago. Most backup failures in practice are at that level, not at the corruption level.
So treat this as the daily check that catches the obvious failures, and keep a quarterly test restore on the calendar for the real answer. The report’s footer says exactly this, deliberately, so nobody reads a green email and concludes they’re covered.
SETTING UP TASK SCHEDULER
Press Windows key → type Task Scheduler → open it
Click “Create Basic Task” → name it
Backup VerificationTrigger: Daily → 8:00am
Action: Start a program
Program:
powershell.exeArguments:
-ExecutionPolicy Bypass -File "C:\Scripts\BackupVerify.ps1"Click Next → Finish
Under Properties → General, check “Run with highest privileges” and use an account with read access to every backup path.
Timing matters more here than in most of these scripts. Run it after your backup window closes, not during. If your nightly jobs finish around 5am, 8am is fine. If something runs until 7am, move this later — otherwise you’ll get FILE LOCKED warnings on a job that’s working perfectly.
Don’t run this on the backup server itself if you can avoid it. A verification script that lives on the machine it’s verifying goes down with it. Run it from a management box or a DC, reaching the target over UNC.
TEST IT
Run it as administrator:
powershell
powershell.exe -ExecutionPolicy Bypass -File "C:\Scripts\BackupVerify.ps1"Before you trust it, verify the paths resolve and the filters match what you think:
powershell
Get-ChildItem -Path "\\BACKUPSRV\Backups$\SQL" -Filter "*.bak" -Recurse |
Sort-Object LastWriteTime -Descending |
Select-Object Name, @{N='SizeMB';E={[math]::Round($_.Length/1MB,0)}}, LastWriteTime -First 5If that returns nothing, your filter or path is wrong and the script will report a FAIL that isn’t real.
To test the failure path, temporarily set MaxAgeHours = 1 on one set. It should come back FAIL with a STALE message. Set it back after.
THE FIRST RUN IS THE INTERESTING ONE
Most people run this for the first time expecting all green and get at least one surprise.
The usual candidates: a backup set pointing at a path that was renamed during a migration months ago. A job whose newest file is three weeks old because the scheduled task got disabled. A set where the file count is 400 because retention stopped pruning and the volume is now at 94%.
If everything comes back OK on the first run, raise your MinSizeMB values to something closer to reality and run it again. A floor of zero means the size check isn’t doing anything.
WHY THIS MATTERS
Backups are the one system where the cost of being wrong is unbounded, and also the one system where almost everyone relies on self-reported status from the thing being measured.
The asymmetry is brutal. You find out your monitoring was wrong during a normal Tuesday. You find out your backups were wrong on the worst day of your career, with people standing behind you asking how long the restore will take.
An independent check that looks at the actual files, from a different machine, costs you fifteen minutes of setup. It won’t tell you your backups are restorable. It will tell you whether they exist, which is a question a surprising number of environments cannot currently answer with confidence.
THIS WEEK’S ACTION
Pick your three most important backup sets. Open the backup target and note the real file sizes, then set MinSizeMB to half of each. Run it once. If anything comes back FAIL, that’s your Saturday sorted — and better to find out now than in the middle of an incident.
Reply to this email if you hit any issues — I read every reply.
Automate & Operate — automateandoperate.com
Yes, that Arnold Schwarzenegger.
He has a newsletter on beehiiv. So do Codie Sanchez, Jay Shetty, David Begnaud, Colin & Samir, and Joanna Stern.
They could publish just about anywhere. They chose beehiiv to build a direct connection with the people who want to hear from them most. And you can do the same, whether you’re world-famous or just at the start of your plan for world domination.