a9s-pg Backup

The a9s-pg backup can be handled with the a9s Backup Manager in the same way the backup of service instances can be handled.

This provides an easy way to control the retention of all backups and unifies the way an operator triggers a backup and restore.

Limitations

Because of the reason that a9s-pg is the main database for the a9s Backup Manager and all services around anynines-deployment the consistency of the backup information of a9s-pg is not guaranteed:

  • Restores of a9s-pg can not be stored in the database because at the time the restore is running the database where the restore information is stored is gone and will be restored to a previous point in time.
  • All backups that have been triggered after the backup which is restored are not longer known in the database.

Important drawback

Be aware that restoring the a9s-pg to a previous point in time means you possibly loose information about deployed instances which results in ghost instances that can't not any longer be handled by the a9s Service Broker and a9s Deployer.

Usage

Prerequisites

  • Get Credentials for the backup-service from CredHub

    $ credhub get -n /backup_manager_password
    id: bb6af4e4-ab35-45d5-8180-4c03ac82164e
    name: /backup_manager_password
    type: password
    value: <PASSWORD>
    version_created_at: "2017-09-04T15:35:04Z"
    
  • Ensure you have the URL of the backup-manager VM. In our example it is backup-manager.service.dc1.consul:3000. You can also use the IP address of the VM given by bosh vms -d backup-service.

Trigger a backup

Curl the backup_agent/backup API endpoint of the a9s Backup Manager.

$ curl http://admin:<PASSWORD>@<URL>:3000/backup_agent/backup -d "instance_guid=a9s-pg"
{"message":"job to backup is queued"}

You can get the state of the backup by curling the instances/a9s-pg endpoint of the backup manager.

$ curl admin:<PASSWORD>@<URL>:3000/instances/a9s-pg
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  9906    0  9906    0     0   3691      0 --:--:--  0:00:02 --:--:--  3690
{
  "restores": [],
  "backups": [
    {
      "backup_agent_task": {
        "msg": null,
        "updated_at": "2018-11-22T04:01:28.853Z",
        "created_at": "2018-11-22T04:00:15.718Z",
        "status": "done",
        "task_id": 148,
        "id": 156
      },
      "is_downloadable": false,
      "size": 44985152,
      "backup_id": "a9s-pg-1542859215559",
      "instance_id": 38,
      "id": 148
    }
  ],
  "instance_id": "a9s-pg",
  "id": 38
}

The status flag "backups.backup_agent_task.status" can have the values

  • "done" when the backup is finished
  • "running" when the backup is running
  • "queued" while the backup is only queued
  • "failed" once the backup failed
  • "deleted" when the backup has been deleted.

Restore a backup

Use the backup_agent/restores endpoint to restore a backup. You need the instance_id and the backup_id (given as id) from the backup you wish to restore. To get these information you need to curl the instances/a9s-pg endpoint before, see Trigger a backup.

      "instance_id": 38,
      "id": 148

Then trigger the restore:

$curl admin:<PASSWORD>@<URL>:3000/backup_agent/restores -d "backup_id=148" -d "instance_id=38"
{"restore_id":5}

During the restore of the a9s-pg the a9s Backup Manager is not available and can't be used to see the state of the restore process. See Limitations. You can watch the backup-agent logs of the a9s-pg master node to see the state. On the VM itself you can use the following command:

$ tail -f /var/vcap/sys/log/backup-agent/production.log

Sample output:

172.28.19.56 - oy6Xh45byt4IMhMmlLSV9s_WCdKx9-gT [20/Nov/2018:12:12:30 +0000] "POST /v1/restore HTTP/1.1" 200 21 0.0023
172.28.19.51 - oy6Xh45byt4IMhMmlLSV9s_WCdKx9-gT [20/Nov/2018:12:12:31 +0000] "POST /v1/order HTTP/1.1" 200 13 0.7541
172.28.19.51 - oy6Xh45byt4IMhMmlLSV9s_WCdKx9-gT [20/Nov/2018:12:12:45 +0000] "POST /v1/order HTTP/1.1" 200 19 0.6944
I, [2018-11-20T12:13:00.939338 #31554]  INFO -- : success
172.28.19.51 - oy6Xh45byt4IMhMmlLSV9s_WCdKx9-gT [20/Nov/2018:12:13:02 +0000] "POST /v1/order HTTP/1.1" 200 13 0.7154