vps-backups

This is a harness for VPS-wide backups of Docker-based applications, including various per-app backup approaches, but with the ability easily determine how frequently backups are done and how long they're kept.

General idea

Periodic static backup

On a given Magnificent VPS we can have a number of Docker Compose-driven services which need backing up. We want each backup, e.g. a database dump, which is a static archive file with a common name that can be incrementally backed up by our remote backup utility (which stores 'diffs' between versions).

The specific tools for which we need backups include

• postgresql (different versions)
• mysql and mariadb (similar but not entirely compatible in more recent versions)
• sqlite3 (yes these are static files)
• mongodb (possibly, needed with Rocket.Chat and WeKan)
• Gitlab (uses Gitlab's in-built script)

some apps manage their own backups, like

• Mailcow (command line script, run by cron)
• Discourse (in-built website functionality)

So we'll treat those a special case.

Rate of backups

For each application, we want to determine how often backups are made (hourly? daily? weekly?) and for how long they are held. This should have a default configuration, but also be able to be adjusted on a per application basis.

Retention of older backups

Once we have individual backups archives for each relevant web application on the VPS we can, in turn, retain hourly, daily, weekly, and monthly archives of the backups for convenience. These can be stored on the VPS on a different filesystem to the active application to distinguish between the data dedicated to the VPS' users and that used for backups and other system-level data.

Managing secrets

Similarly, configuration for each application should not have to hold secrets, e.g. database credentials, for the containerised applications being backed up. These should be retrieved from the docker-compose.yml or other secrets (e.g. .env) files related to each app. This file, and the means of extracting values from it, should be specified in the configuration for each application.

Notifications

For this system to be really useful, in addition to performing the backups in a timely manner and managing retention of older backups, this system needs to notify the managers of the VPS about things like

• when the backups (over all and for each app) start and finish
• if any backup files fail to work, or end up with 0 size
• if any older backup rotation fails
• if the system is in danger of running out of disk space

How it works in practise

This application is a set of Bash scripts. It's intended to work on the basis of minimal handling and intended for automated deployment, usually by dropping one or more files into an appropriate directory (or removing them from it) to add support for additional systems.