The different storages following page details the process and files stored for ZWE MOH machines:
| Table of Contents | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
...
Backup Files and Process
The different databases/files for the e-Learning, VMMC and Analytic systems are backed up every 24hrs, at 1am local time. Resulting files are stored in the folder root/backups. Details about what is backed up is detailed on the table below.
Each server contains their own backups stored in the following folder:
| Code Block |
|---|
/opt/backups/ |
The process to backup files are triggered by multiple bash files inside the /opt/backups/ path of each app/service. The following table details the order that each script file is triggered in order to be backed up:
System | What is backup | Backup procedure | ||||||
|---|---|---|---|---|---|---|---|---|
VMMC DB | Mongo DB | Mongo dump → compress compressed fileMoodle | ||||||
eLearning | Resource files |
| ||||||
Postgres SQL database |
Files folders | SQL dump → compress .sql file files folder | Superset Configuraiton (User, Dashboards, Connections, etc) | Dashbaord definitons |
...
| ||
Superset | Config files |
|
Dashboard |
| |
Database |
| |
NiFi | Configuration folder (includes nifi flows, user credentials, etc) |
|
Datawarehouse | Postgres SQL Database |
|
Docker analytics volume | Volumes folder |
|
Docker elearning volume | Volumes folder |
|
...
File Naming Conventions
We use the following convention:
YYYY-MM-DD - the date that the backup was completed, local time
HH-MM: the time that the backup was completed, local time
[dwm]: one of these characters, depending if the backup is daily, weekly or monthly
“MOH_ZWE”+system: the name of the system backed up (MONGO, ELEARNING…)
databaseName: in the vmmc case, we also need to add the database name, we have to: db1 and dbOthers.
.X.X - The version of the associated product, to ensure restorability“restorability”
.XXX the file extension
Resulting file names for MOH ZWE:
Moodle db: YYYYyyyy-mm-MMdd-DDhh-HHmm-MMmoh-MOHzwe-ZWEmoodle-MOODLEd-4.5.SQL.GIZ.sql.giz
Moodle files: yyyy-mm-dd-hh-mm-moh-zwe-moodle—m-files.giz
VMMC db: YYYY-MM-DD-HH-MM-MOH-ZWE-MONGO.7.GIZ
Superset: YYYY-MM-DD-HH-MM-MOH-ZWE-SUPERSET.3.GIZ
...
yyyy-mm-dd-hh-mm-moh-zwe-d-mongo-dbOthers.gz
Superset: yyyy-mm-dd-hh-mm-moh-zwe-d-superset.3.giz
...
Retention Policies
In the absence of any mandate on backup frequency and retention, the following backup procedures procedure was implemented for the various systems' databases, as of December 2024:
Daily backups, kept for 30 31 days
Weekly backups (Sunday), kept for 1 year
Monthly backups (last day of the month), kept for 7 years
Storage needs
Total disk capacity required (cumulative)
...
Year 1
...
Year 2
...
Year 3
...
Year 4
...
Year 5
...
Year 6
...
Year 7
...
For the Moodle folder, we implemented the following backup policy:
Daily backups, 31 days
Weekly backups, kept for 6 months
Monthly backups, kept for 7 years
The moodle files folder contains:
Language customizations
Files & images uploaded to Moodle
Cache storage
User session’s data
...
Storage needs
Average backup size & transmisison time
Mooodle DB
Moodle files
Backup | Year-to-date | Year 2-7 |
|---|---|---|
Retention for databases | Last 30 days Last 52 weeks (Sunday) Last 12 month (Last day of month) = 94 backups | Last day of the month Year 2-7 = 72 backups |
Retention for Moodle files | Last 7 days | n/a |
Retention for Moodle files | Daily backups, 14 days Weekly backups, 6 months Monthly backup for 6 month | Monthly backups, kept for 7 years |
Moodle/ SQL | avg backup size: 1.0 Mb Transmission time: 5s | + 20% annual increase |
Moodle files | ||
VMMC Mongo | avg backup 15 .0 MbGB | |
Superset backup | avg backup 1.0 Mb |