JetBrains Space Help

Backup and Restore for Docker Compose Installation

Even though Docker Compose installation automatically saves Space data in persistent Docker volumes, it is still recommended to back up your data regularly. This way, you can restore your Space installation in case of unexpected issues.

What data is worth backing up

Docker Compose installation uses four services that are involved in data storage. Each service runs in a separate Docker container:

  • PostgreSQL database – stores Space user data; we strongly recommend that you back up this database. Learn more

  • MinIO object storage – stores Space files; we strongly recommend that you back up this storage. Learn more

  • Redis – stores Space cache; no need to back up.

  • Elasticsearch – stores Space search index; for small companies with a limited amount of data, backing up Elasticsearch may not be necessary since rebuilding the indexes should not take too long.

Back up and restore PostgreSQL database

To back up the PostgreSQL database

  • On the host machine, run:

    docker exec -t $DB_CONTAINER_ID pg_dump -U $DB_USERNAME -d $DB_NAME > spacedb_dump.sql

    Here:

    • DB_CONTAINER_ID is the ID of the postgres container. You can get the ID with the docker ps command.

    • DB_USERNAME is the services.postgres.environment.POSTGRES_USER value from the docker-compose.yml file used for the installation. By default, space.

    • DB_NAME is the services.postgres.environment.POSTGRES_DB value from the docker-compose.yml file used for the installation. By default, spacedb.

    This command runs the pg_dump command-line tool inside the PostgreSQL container to perform a database dump. The spacedb_dump.sql dump file is saved to the current directory.

To restore the PostgreSQL database

  1. Stop your Space instance if it is running:

    docker-compose -p space-on-premises down docker-compose -p space-on-premises rm -f

  2. As the PostrgreSQL container is also stopped, you should start it again with:

    docker compose up postgres
  3. Run:

    docker exec -i $DB_CONTAINER_ID psql -U $DB_USERNAME -d $DB_NAME < spacedb_dump.sql

    This command imports the previously created spacedb_dump.sql dump to an empty database in the PostgreSQL container.

  4. Start your Space instance:

    docker compose up

To automate backup and restore of the PostgreSQL database

  • You can automate the process of backup and restore with a Bash script. For example:

    #!/bin/bash DB_CONTAINER_ID="$2" DB_USERNAME="$3" DB_NAME="$4" function show_usage() { echo "Usage:" echo "$0 backup [DB_CONTAINER_ID] [DB_USERNAME] [DB_NAME] : Back up Space DB as spacedb_dump.sql to PWD" echo "$0 restore [DB_CONTAINER_ID] [DB_USERNAME] [DB_NAME] : Take spacedb_dump.sql from PWD and restore Space DB" } if [ $# -ne 4 ]; then # incorrect number of arguments echo "Error: Not all arguments are specified" show_usage exit 1 fi case "$1" in backup) echo "Backing up..." docker exec -t $DB_CONTAINER_ID pg_dump -U $DB_USERNAME -d $DB_NAME > spacedb_dump.sql echo "Backup created" ;; restore) echo "Restoring..." docker exec -i $DB_CONTAINER_ID psql -U $DB_USERNAME -d $DB_NAME < spacedb_dump.sql echo "Restore completed" ;; *) # invalid argument echo "Invalid command" show_usage exit 1 ;; esac

Back up and restore MinIO storage

To back up and restore the MinIO object storage, you will need the MinIO Client. You can run it either on the host machine that runs Space, or on the host machine that runs the MinIO container (if you run it separately).

To back up the MinIO storage

  1. Make sure you performed the backup of the PostgreSQL database.

  2. Add the MinIO host machine to the MinIO Client configuration:

    mc alias set <ALIAS> <URL> <ACCESSKEY> <SECRETKEY>

    For example, for the default installation (Space runs as localhost):

    mc alias set space-minio http://localhost:9000 space-access-key space-secret-key

  3. Mirror the data stored in the MinIO buckets to the backup directory:

    mc mirror <ALIAS> <BACKUP/DIRECTORY>

    After the backup is complete, the backup directory must contain the following subdirectories: automation-dsl-local, automation-fileshare-local, automation-logs-local, packages-local, space-local, and vcs-local.

To restore the MinIO storage

  1. If the restore process is performed on a separate host machine, make sure that the MinIO Client is installed and the MinIO host is added as an alias to the configuration (step 2 of the backup process).

  2. Stop your Space instance if it is running:

    docker-compose -p space-on-premises down docker-compose -p space-on-premises rm -f

  3. As the MinIO container is also stopped, you should start it again with:

    docker compose up minio
  4. Create the MinIO buckets (the command will ignore the existing buckets):

    mc mb --ignore-existing <ALIAS>/automation-dsl-local <ALIAS>/automation-fileshare-local <ALIAS>/automation-logs-local <ALIAS>/packages-local <ALIAS>/space-local <ALIAS>/vcs-local
  5. Restore the MinIO data:

    mc mirror <BACKUP/DIRECTORY> <ALIAS>
  6. Start your Space instance:

    docker compose up

To automate backup and restore of the MinIO storage

  • You can automate the process of backup and restore with a Bash script. For example:

    #!/bin/bash # Bash script configuring MinIO host and mirror data # Display usage instructions function show_usage() { echo "Usage:" echo "$0 add-host <minio_alias> <minio_url> <minio_access_key> <minio_secret_key> : Configure MinIO host" echo "$0 backup <minio_alias> <backup_dir> : Mirror data from MinIO to backup directory" echo "$0 restore <backup_dir> <minio_alias> : Restore data from backup directory to MinIO" } # Check the number of arguments if [ $# -lt 2 ]; then echo "Error: Invalid number of arguments" show_usage exit 1 fi # Execute add-host, backup, and restore commands case "$1" in add-host) if [ $# -ne 5 ]; then echo "Error: Invalid number of arguments for 'configure-host' command" show_usage exit 1 fi alias="$2" url="$3" access_key="$4" secret_key="$5" mc alias set $alias $url $access_key $secret_key ;; backup) if [ $# -ne 3 ]; then echo "Error: Invalid number of arguments for 'backup' command" show_usage exit 1 fi backup_dir="$3" echo "Mirroring data from MinIO to backup directory..." mc mirror $alias $backup_dir ;; restore) if [ $# -ne 3 ]; then echo "Error: Invalid number of arguments for 'restore' command" show_usage exit 1 fi backup_dir="$2" alias="$3" echo "Restoring data from backup directory to MinIO..." mc mb --ignore-existing $alias/automation-dsl-local $alias/automation-fileshare-local $alias/automation-logs-local $alias/packages-local $alias/space-local $alias/vcs-local mc mirror $backup_dir $alias ;; *) echo "Error: Unknown command '$command'" show_usage exit 1 ;; esac
Last modified: 05 July 2023