Skip to main content
Version: 31.1.1

Migrate Data from an a9s Elasticsearch to an a9s Search Service Instance

This section will guide you through the process of migrating your data from an a9s Elasticsearch Service Instance to an a9s Search Service Instance.

Prerequisites

  • An existing a9s Elasticsearch and a9s Search Service Instance that were created with, and are still using, a similar plan.

Limitation

  • The migration from a9s Elasticsearch to a9s Search is only supported for Service Instances with Elasticsearch version 7.
  • a9s LogMe Service Instances are not supported for migrations.
  • It can't be guaranteed that all Elasticsearch applications will work seamlessly with a9s Search. You need to ensure this before migrating it. In most cases, it will work, but we can't cover all edge cases.

Prepare the a9s Elasticsearch Service Instance

To migrate the data from an a9s Elasticsearch Service Instance to an a9s Search Service Instance we're going to use the backup and restore feature. To make this work there are some manual updates to be done inside the a9s Backup Manager database.

Set a New Encryption Key (Optional)

The backups encryption key of the a9s Elasticsearch and a9s Search Service Instance need to be the same. If the encryption key is not known it can be set at the a9s Service Dashboard. The a9s Service Dashboard URL can be found with cf service <a9s_elasticsearch_service_name>. For further information on how to set an encryption key, please refer to the a9s Service Dashboard documentation.

Unbind All Applications

Migration is not possible without a small downtime. All data added to the Service Instance after the backup was triggered will be lost. Therefore we recommend unbinding all applications from the Service Instance before triggering the backup.

Create a New Backup

To get the latest data, a new backup needs to be created. For further information on how to trigger a backup please refer to the a9s Service Dashboard documentation

Get the Backup Name

To get the backup name, the Cloud Foundry GUID of the Service Instance needs to be known.

cf service <a9s_elasticsearch_service_name> --guid

You need to BOSH ssh to the a9s Backup Manager VM. Change to the superuser with sudo -i. Start the rails console with /var/vcap/jobs/anynines-backup-manager/bin/rails_c

elasticsearch_service_instance_id = '<your elasticsearch cf guid>'
elasticsearch_instance = Instance.where(instance_id: elasticsearch_service_instance_id).first
elasticsearch_backup = elasticsearch_instance.backups.last
elasticsearch_backup_name = elasticsearch_backup.backup_id

elasticsearch_backup_name is the name of the backup. This is needed for the next steps.

Prepare the a9s Search Service Instance

Set a New Encryption Key

To perform a restore of the a9s Elasticsearch Service Instance the encryption key needs to be the same as the one from the backup. Therefore you need to update the one from the a9s Search Service Instance. How this can be done is mentioned at the a9s Service Dashboard documentation.

Create a New Backup

To perform the restore with the backup from the a9s Elasticsearch Service Instance you need to add its backup to the a9s Search Service Instance. For this reason you need to create a new backup for the a9s Search Service Instance. The a9s Service Dashboard URL can be found with cf service <a9s_search_service_name>. How a backup is trigged is mentioned at the a9s Service Dashboard documentation. Wait until the backup is done or failed. It does not matter if the backup is successful or not. It only needs to be in a final state.

Manipulate the Backup Entry at the a9s Backup Manager

To manipulate the backup, you need the Cloud Foundry GUID of the Service Instance.

cf service <a9s_search_service_name> --guid

You need to BOSH ssh to the a9s Backup Manager VM. Change to the superuser with sudo -i. Start the rails console with /var/vcap/jobs/anynines-backup-manager/bin/rails_c

# load the backup
search_service_instance_id = '<your search cf guid>'
search_instance = Instance.where(instance_id: search_service_instance_id).first
search_backup = search_instance.backups.last

# update the backup status to 'done'
backup_agent_task = search_backup.backup_agent_task
backup_agent_task.status = :done
backup_agent_task.save!

# set the backup_id to the a9s Elasticsearch one
search_backup.backup_id = '<the elasticsearch backup name>'
search_backup.save!

Restore the Backup

Go back to the a9s Service Dashboard of the a9s Search Service Instance and trigger the restore. For further information on how to trigger a restore please refer to the a9s Service Dashboard documentation

Bind the Applications

After the restore is done and the data was added, all applications can be bound to the Service Instance again.