Skip to article
Biron Family / Behind the Build
Behind the Build / When free space lies

Engineering note / Capacity incident

When free space lies.

A backup filesystem still had free space, but the shared thin pool beneath it was approaching exhaustion. The useful metric lived one layer lower.

Last reviewed

On this page

The symptom

In September 2026, a local backup repository contributed to severe pressure on a Proxmox host’s thin-provisioned storage pool. The pool also held household guests. Its data allocation reached 97.7%.

The repository’s own filesystem could still show free space. That made the usual filesystem-level view an incomplete explanation of the risk.

The diagnosis

The repository occupied a thin logical volume, not a dedicated disk. Its growth consumed capacity in the same underlying pool used by virtual machines and containers.

A large, mostly archival directory was included in a frequently backed-up set. A new phone archive added substantial data to a nightly run. The backup policy and storage placement were individually understandable, but their interaction was not sustainable.

The response

Immediate mitigation recovered discarded blocks and added bounded pool headroom. The durable change separated the large archive into its own encrypted off-site repository and removed it from the local high-priority backup set after verification.

An approved repository rewrite/prune then reclaimed space. The documented post-cleanup figures were 85 GB to 14 GB of repository filesystem use, with thin-pool data allocation falling to 34.8%. These are historical measurements from that incident, not today’s utilization.

Preventing a repeat

A new monitor checks both thin-pool data and metadata allocation. It reports severity changes, uses hysteresis for recovery, and limits repeat notifications. The thresholds observe the storage layer that can stall the guests, rather than relying only on free space inside the repository filesystem.

The lesson

Capacity is a dependency graph. Filesystem free space, logical-volume size, thin-pool allocation, and physical storage describe different layers.

The lasting fix was not “buy more disk” or “delete some files.” It was to change the protection policy and placement, then monitor the failure domain that actually mattered.

Related: Backups & recovery · Monitoring