a9s Messaging Migration

This document describes how to do data migration between two a9s Messaging Service Instances. We present two approaches:

1. the first one uses the Disaster Recovery feature to create a fork of an existing Service Instance,
2. the second one uses different commands via the RabbitMQ CLI to migrate the data.

The first approach has been tested for the following cases:

• from 3.12.x to 4.x
• from 3.13.x to 4.x

The second approach, and its migration steps, have been tested for the following cases:

• from 3.10.x to 3.12.x
• from 3.10.x to 3.13.x
• from 3.12.x to 3.13.x
• from 3.12.x to 4.x
• from 3.13.x to 4.x

Known Limitations

These methods only migrate the server queues and exchanges structures, not the messages contained on it.

Migrate a9s Messaging Using the Disaster Recovery Feature

Follow the steps described in Disaster Recovery to create a fork of an existing a9s Messaging Service Instance.

Migrate a9s Messaging Manually

In this scenario, we will use an SSH tunnel to get the RabbitMQ server definitions from an a9s Messaging Service Instance and then restore them on a fresh a9s Messaging Service Instance, using a similar process.

Migration Steps

Create New a9s Messaging Service Instance

Create a new and empty a9s Messaging Service Instance that will be the target for the data migration from an existing a9s Messaging Service Instance.

Note

• Do not insert any data on the new Service Instance before the migration and verification are done.
• Create a new Service Instance with the same capabilities or greater.

Create an SSH Tunnel

Create a reverse tunnel

The goal is to give the application developer access to the a9s Messaging Service Instance. And for this, it is necessary to make a SSH tunnels to the a9s Messaging origin Service Instance.

In the end, we want to achieve the following scenario below:

           /----> * RabbitMQ origin *                     /--------> * RabbitMQ destination *
          /                                              /
* CF Application * (via SSH Tunnel)                 * CF Application * (via SSH Tunnel)        [Infrastructure network]
--------/----------------------------------------------/--------------------------------------------------------------
       /                                              /                                           [Developer network]
      /                                              /
* Developer Machine *                     * Developer Machine *

It is possible to access any of the a9s Data Services locally. That means you can connect with a local client to the service for any purpose such as debugging. Cloud Foundry (CF) provides a smart way to create SSH forward tunnels via a pushed application. For more information about this feature, see the Accessing Apps with SSH section of the CF documentation.

First of all you must have an application bound to the service. How to do this see Bind an application to a Service Instance.

note

cf ssh support must be enabled in the platform. Ask your platform operator if you are not sure.

Follow the section Create a Tunnel to The Service to create the reverse tunnel.

Create RabbitMQ Administrator Credentials

You will need a user with administrative privileges to do the tasks required for the migration. In both Service Instances, create a rabbitmq user with the administrator role with the following command:

cf create-service-key my-messaging-service my-key -c '{"roles": ["administrator", "management"]}'

You should use these credentials to run all the commands on RabbitMQ instances described in the next sections.

Export Definitions From Old Instance

To export and import RabbitMQ definitions files on a9s Messaging services, one can use two methods: getting the files directly through the HTTP API, or using Backup Manager to create a backup that contains the Service Instance definitions and then restore this backup on the new instance.

To get the definition file through the HTTP API, make sure the SSH tunnel to the old instance is set up as described on the previous section, and run the following command from your local environment:

cf ssh <APP_NAME> -L 127.0.0.1:15672:<OLD_SERVICE_HOSTNAME>:15672

Then you can use curl to download the definitions file, passing the administrator credentials created in the previous step:

$ curl -o definitions.defs -u <SERVICE_ADMIN_USERNAME>:<SERVICE_ADMIN_PASSWORD> -X GET http://localhost:15672/api/definitions

# for ssl instances
$ curl -k -o definitions.defs -u <SERVICE_ADMIN_USERNAME>:<SERVICE_ADMIN_PASSWORD> -X GET https://localhost:15672/api/definitions

If, for some reason, you cannot access the HTTP API, you can use the Backup Manager to get the definitions file. Access the Service Dashboard of the Service Instance you want to migrate from, you can find the dashboard URL with this command:

$ cf service <SERVICE_NAME> | grep dashboard

Make sure you:

• set a encryption password for the backups using the Service Instance Dashboard,
• create a backup using the Dashboard,
• download the backup to your local machine.

Use the password set up before to decrypt the backup and write its contents to a file:

cat <BACKUP_FILE> | openssl enc -aes256 -md md5 -d -pass 'pass:<BACKUP_PASSWORD>' | gunzip -c > definitions.defs

danger

This definitions file does NOT contain any message data, only queues and exchanges definitions.

Prepare Definitions for Migration to a9s Messaging 4

Starting with RabbitMQ 4, mirrored classic queues have been removed, leaving only classic queues without the High Availability feature. Thus, High Availability queues with a9s Messaging 4.X are only possible via the quorum queues.

Therefore, the definitions file must be adapted if the migration is done from a9s Messaging 3.X to 4.X in order to remove unsupported policies. The unsupported policies ha and noha can either be manually removed from the definitions file's field policies or they can be automatically removed using the following command:

jq 'del(.policies | .[] | select(.name == "ha" or .name == "noha"))' definitions.defs > definitions_migrated.defs

More informations about converting the mirrored classic queues to quorum queues can be found in the RabbitMQ migration documentation.

Disk and Memory Usage on a9s Messaging 4.0

When migrating to a9s Messaging 4.0 and replacing the mirrored classic queues with quorum queues, the increased memory and disk space usage must be considered. Quorum queues can use up to 50% more memory and many times more disk space than classic queues. Therefore, the Service Instance should be upgraded to the next larger service plan depending on current usage.

More information about converting the mirrored classic queues to quorum queues can be found in the RabbitMQ migration documentation.

Import Definitions to New Instance

To import the queue definitions to the new instance, set up the SSH tunnel to the new instance:

cf ssh <APP_NAME> -L 127.0.0.1:15672:<NEW_SERVICE_HOSTNAME>:15672

Then you can use curl to import the definitions to the new instance like this:

$ curl --header 'Content-Type: application/json' -u <SERVICE_ADMIN_USERNAME>:<SERVICE_ADMIN_PASSWORD> -X POST -T definitions.defs http://localhost:15672/api/definitions

# for ssl instances
$ curl -k --header 'Content-Type: application/json' -u <SERVICE_ADMIN_USERNAME>:<SERVICE_ADMIN_PASSWORD> -X POST -T definitions.defs https://localhost:15672/api/definitions

With this step done, we will have both Service Instances set up with the same structure, so we can proceed to the application migration.

Migrate Producer Applications

You can now switch your producers to use the new Service Instance. This step changes depending on the application setup, reconfigure your load balancer or your consumer applications.

Migrate Consumer Applications

Once the queues in the old instance are almost empty, you can stop consumers. If message ordering is important to you, you can still wait a bit more so that the consumers finish draining the queues on the old instance. When they are empty, reconfigure them as you did for the producers and restart them. At this point everything is migrated to the new instance.

Decommission Old Instance

The last step is to stop the old Service Instance. Your migration is now complete.