anynines Backup Process

Creating a New Backup

By creating a new backup of an active service instance using the a9s service dashboard, the endpoint /backup_agent/backup of the a9s backup manager will be triggered.

Afterwards, the backup manager starts the backup process of the selected service instance by inserting the backup job into the backup agent queue. The backup agent, co-located with the deployment of the service instance itself, start to create the backup by connecting to the virtual machine(s) of the service instance.

Subsequently, the created backup will be first compressed and encrypted using the encryption key, defined by the user using the a9s service dashboard. Finally, the backup will be uploaded to the configured backup store (e.g. AWS S3).

backup-manager

Restore a Backup

By restoring a service instance using the a9s service dashboard, the endpoint /backup_agent/restores of the a9s backup manager will be triggered. The backup manager handles the incoming request and triggers the restore of the service instance by inserting the restore job into the backup agent queue.

Once the restore job has been started, the backup agent downloads the selected backup from the configured backup store and prepares the backup by decrypting and unpacking the downloaded archive file.

Finally, the backup agent imports the backup data into the running service instance by connecting to the virtual machine(s) of the service instance.

Configure a Backup's maximum file size

Backups are done using the splitted files method, which allows large files to be handled more easily. Depending on the backup's size, this creates multiple files with a maximum expected size, which can be configured with the property file_size_limit, which is set in bytes, and a maximum size for the chunks to be uploaded, set in chunk_size, also set in bytes.

The properties file_size_limit and chunk_size can only be changed using the template-uploader-errand. The best way is to add the following content to the template-custom-ops section in the template-uploader-errand job.

For example it looks like this for the a9s PostgreSQL templates:

template-uploader:
  ...
  template-custom-ops:
    - type: replace
      path: /instance_groups/name=pg/jobs/name=backup-agent/properties/backup-agent/plugins/output?/fog?
      value:
        local:
          file_size_limit: 104857600 # 100 MB in Bytes
          chunk_size: 8388608 # 8 MB in Bytes
  ...

From now on, each Backup Agent has the file_size_limit and chunk_size properties for each newly created Service Instance.

These properties can be configured in the backup agent job, more specifically in the service instance manifest as shown below.

properties:
  backup-agent:
  ...
    plugins:
    ...
      output:
      ...
        fog:
          local:
            file_size_limit: 104857600 # 100 MB in Bytes
            chunk_size: 8388608 # 8 MB in Bytes
         ...

If the values are not set, then the properties are assigned default values of 1 GB for file_size_limit and 5 MB for the chunk_size.

Retention Policy for Deleted Service Instances

To ensure that backups are getting deleted when a Service Instance is deleted the retention_time_for_backups_of_a_deleted_instance property can be set. This property defines how many days to wait until all backups of this Service Instance should be deleted. Before that date is reached the normal retention policy is used. If it is set to 0 the backups of a deleted Service Instance will be deleted with the next run of the "delete old backups" job.

Example:

properties:
  anynines-backup-manager:
    retention_time_for_backups_of_a_deleted_instance: 30