/
Backups
  • In progress
  • Backups

    The following page details the process and files stored for ZWE MOH machines:


    Backup Files and Process

    The different databases/files for the e-Learning, VMMC and Analytic systems are backed up every 24hrs, at 1am local time. Resulting files are stored in the folder root/backups. Details about what is backed up is detailed on the table below.

    Each server contains their own backups stored in the following folder:

    /opt/backups/

    The process to backup files are triggered by multiple bash files inside the /opt/backups/ path of each app/service. The following table details the order that each script file is triggered in order to be backed up:

    System

    What is backup

    Backup procedure

    System

    What is backup

    Backup procedure

    VMMC DB

    Mongo DB

    Mongo dump → compressed file

    eLearning

    Resource files

    MISSING

    Postgres SQL database

    1. Dump postgres database.

    2. Extract database dumped from docker container.

    3. Transfer dump file to backup machine.

    Superset

    Config files

    1. Extract configuration file from docker container.

    2. Transfer configuration file to backup machine.

    Dashboard

    1. Rename existing backup files

    2. Remove previous backup files.

    3. Create new dashboards backup.

    4. Extract dashboards backup.

    5. Transfer dashboard to backup machine.

    Database

    1. Dump postgres database.

    2. Extract database dumped from docker container.

    3. Transfer dump file to backup machine.

    NiFi

    Configuration folder (includes nifi flows, user credentials, etc)

    1. Compress configuration directory.

    2. Extract configuration directory from docker container.

    3. Transfer configuration directory to backup machine.

    Datawarehouse

    Postgres SQL Database

    1. Dump postgres database.

    2. Extract database dumped from docker container.

    3. Transfer dump file to backup machine.

    Docker analytics volume

    Volumes folder

    1. Compress volume directory

    2. Transfer compressed directory to backup machine.

    Docker elearning volume

    Volumes folder

    1. Compress volume directory

    2. Transfer compressed directory to backup machine.


    File Naming Conventions

    We use the following convention:

    • YYYY-MM-DD - the date that the backup was completed, local time

    • HH-MM: the time that the backup was completed, local time

    • “MOH_ZWE”+system: the name of the system backed up

    • .X.X - The version of the associated product, to ensure “restorability”

    • .XXX the file extension

    Resulting file names for MOH ZWE:

    • Moodle db: yyyy-mm-dd-hh-mm-moh-zwe-moodle-4.5.sql.giz

    • Moodle files: yyyy-mm-dd-hh-mm-moh-zwe-moodle-files.giz

    • VMMC db: yyyy-mm-dd-hh-mm-moh-zwe-mongo.7.giz

    • Superset: yyyy-mm-dd-hh-mm-moh-zwe-superset.3.giz


    Retention Policies

    In the absence of any mandate on backup frequency and retention, the following backup procedure was implemented for the various systems' databases, as of December 2024:

    • Daily backups, kept for 31 days

    • Weekly backups (Sunday), kept for 1 year

    • Monthly backups (last day of the month), kept for 7 years

    For the Moodle folder, we implemented the following backup policy:

    • Daily backups, 31 days

    • Weekly backups, kept for 6 months

    • Monthly backups, kept for 7 years

    The moodle files folder contains:

    • Language customizations

    • Files & images uploaded to Moodle

    • Cache storage

    • User session’s data


    Storage needs

    Average backup size & transmisison time

    • Mooodle DB

    • Moodle files

    Backup

    Year-to-date

    Year 2-7

    Backup

    Year-to-date

    Year 2-7

    Retention for databases
    remote disk (PSH)
    local disk (MOH)

    Last 30 days

    Last 52 weeks (Sunday)

    Last 12 month (Last day of month)

    = 94 backups

    Last day of the month Year 2-7

    = 12 backups per year

    = 72 backups

    Retention for Moodle files
    Local disk (MOH)

    Last 7 days

    n/a

    Retention for Moodle files
    Remote disk

    Daily backups, 14 days

    Weekly backups, 6 months

    Monthly backup for 6 month

    Monthly backups, kept for 7 years

    Moodle/ SQL

    avg backup size: 1.0 Mb

    Transmission time: 5s

    + 20% annual increase

    Moodle files

     

     

    VMMC Mongo

    avg backup

    1.0 Mb

     

    Superset backup

    avg backup

    1.0 Mb

     

    Related content