a9s Data Services Installation

System Overview

The system landscape is divided into different BOSH deployments:

BOSH DeploymentDescription
consul-dns/consul-dns.ymlAn internal domain name system to resolve hostnames provided and used by the a9s Data Service Framework. While installing the a9s Data Services, you have to make these name servers available in your platform environment either via DNS delegation or by configuring these name servers into your platform directly.
a9s-pg/a9s-pg.ymlA PostgreSQL cluster of three nodes based on the a9s PostgreSQL BOSH Release. The PostgreSQL cluster will be used as a data store by all a9s Data Service Framework components (e.g. the a9s service brokers, a9s deployers, a9s service guard, etc.).

If you are going to deploy the anynines-PaaS-deployment/cf/cf.yml the Cloud Foundry setup will also use the a9s-pg cluster (Cloud Controller and UAA will store its data there).
service-guard/service-guard.ymlThe a9s Service Guard observes the Consul catalog and updates Cloud Foundry application security groups as soon as something changes, so that only applications deployed into Cloud Foundry space A can access service instances provisioned in space A.

It is an optional component that you don't need to deploy.
backup-service/backup-service.ymlThe a9s Backup Framework takes care of the backup of all service instances.

It is an optional component that you don't need to deploy.
mongodb-service/mongodb-service.ymlAll components required to provide the a9s MongoDB Data Service.
postgresql-service/postgresql-service.ymlAll components required to provide the a9s PostgreSQL Data Service.
elasticsearch-service/elasticsearch-service.ymlAll components required to provide the a9s Elasticsearch Data Service.
rabbitmq-service/rabbitmq-service.ymlAll components required to provide the a9s Messaging Data Service.
redis-service/redis-service.ymlAll components required to provide the a9s Redis Data Service.
mysql-service/mysql-service.ymlAll components required to provide the a9s MySQL Data Service.
logme-service/logme-service.ymlAll components required to provide the a9s LogMe Data Service.
a9s-environment-info/a9s-environment-info.ymlAll components required to provide the a9s Environment Info Service.
a9s-billing/a9s-billing.ymlAll components required to provide the a9s Billing System.

More conceptional readings can be found here:

  • Service Framework Architecture Introduction - A detailed description of each feature, component and the communication flows (only available for anynines employees and customers with contract)
    https://docs.google.com/document/d/1KHSFi-6sIwxTT53ZB9EzNhdG7Bssrno0c_MSfjoDqTE/edit
  • Why We Use the Consul DNS System

Hint: for a full list of Ports needed for each Data Service please take a look at the Additional Docs below or at the list of ports.

a9s Recommended Network Layout

a9s Data Services Setup

If you have a Cloud Foundry setup running and you want to install the a9s Data Services follow the steps below.

0. Prerequisites

General

IMPORTANT: minimum required version of BOSH CLI is v2.0.36

This guides assumes that you have a BOSH director up and running and that it is configured with a BOSH 2.0 Cloud Config. In this Cloud Config, the following entities should be defined:

Cloud Config Entry TypeEntry NamesComment
azsz1, z2, z3We strongly suggest to use three different Availability Zones. If you only have two or one zone, you can configure three zones in the BOSH Cloud Config anyway, and let them point to the same zones in your IaaS. We do not recommend to do this.
network*dynamic, staticIf you want to use different network names, you have to replace the names in the deployment manifests and in the deployer-templates. The dynamic network should be configured either as a dynamic or a manual network in the BOSH Cloud Config. Note that dynamic is only available on IaaS layers that provide a DHCP server. On OpenStack setups, manual network might cause trouble. On vSphere setups, dynamic networks might not be available.
vm_typesnano, small, largeIf you want to configure custom service plan, you must add more VM types and refer them in config/iaas-config.yml.
disk_typessmall,medium, largeIf you want to configure custom service plan, you must add more VM types and refer them in config/iaas-config.yml.

For information how to set up the BOSH director, please see the Setup BOSH Director Guide.

* Please note that the default network layout does not match the recommended network layout described in chapter Recommended Network Layout. To change the default network layout according your requirements, you have to add the required networks to the BOSH cloud config, and also change the network names in the deployment manifests. The default network layout will deploy the consul-dns deployment into the static network and everything else in the dynamic network.

You can use the a9s Cloud Config Wizard to create a valid BOSH Cloud Config for your IaaS.

BOSH Director

The following BOSH Director properties must be configured accordingly.

enable_post_deploy: true

In a standard setup the BOSH Director does not run post deploy scripts by default. Use the director.enable_post_deploy property to enable running post deploy scripts. See https://bosh.io/docs/post-deploy/ for more information.

Minimum system requirements overview

The deployments are heavily tested with the following VM and disk type configurations:

Disk small: 4 GB
Disk medium: 10 GB
Disk large: 30 GB
VM nano: 1 CPU, 1 GB RAM, 4 GB Disk
VM small: 1 CPU, 4 GB RAM, 4 GB Disk
VM large: 4 CPU, 8 GB RAM, 30 GB Disk

To provide all a9s Data Services the following footprint is required (this does not include service instances):

DeploymentsIP countMin Required RAMMin Required CPUDisk (persistent & ephemeral)
Inception VM1 public
1 internal
4 GB220 GB
BOSH director14 GB240 GB
consul-dns520 GB532 GB
a9s-pg314 GB330 GB
backup-manager12 GB14 GB
service-guard416 GB416 GB
elasticsearch-service44 GB312 GB
mongodb-service44 GB312 GB
potsgresql-service44 GB312 GB
rabbitmq-service44 GB312 GB
redis-service44 GB312 GB
sum1 public
35 internal
78 GB32196 GB

To provision an a9s service instance, the following resources are required for the corresponding service plan:

Service PlanIP countMin Required RAMMin Required CPUDisk (persistent & ephemeral)
single-small12 GB18 GB
single-big18 GB460 GB
cluster-small312 GB324 GB
cluster-big324 GB12180 GB

1. Checkout anynines-deployment

If not already done, check out this repository and change into the directory.

$ git clone git@github.com:anynines/anynines-deployment.git
$ cd anynines-deployment

2. Configure your Environment

Before you can deploy anything, you have to provide a proper IaaS configuration and also a service configuration for each service you want to deploy.

2.1 IaaS config

The IaaS config contains all information that can differ from environment to environment. The following properties are stored in the IaaS config:

  • a list of supported stemcells,
  • Consul configuration like static IP addresses and the Consul domain
  • Cloud Foundry configuration like system/app domain and static IP addresses
  • a9s-pg configuration like the used stemcell, the max connections and monitoring configurations
  • for each service the used stemcell and network

Have a look at config/iaas-config.yml.example for an example and create your own IaaS config that matches your specific environment and save it as config/iaas-config.yml. In this guide, we are going to use the config/iaas-config.yml in all following commands.

2.2 Service config

The service config is specific for each service and it contains all information about the services and plans that are provided by that specific service. There is an example config file for each service available in the config folder. Have a look at config/postgresql.yml for an example.

You can change the following parts here:

  • the name of each service plan
  • the GUID of each service plan
  • the VM and persistent disk type of each service plan
  • the migration paths for each service plan

You are also able to

  • delete service plans that you don't want to provide in your setup
  • add new service plans that you want to provide in your setup

Have a look at service_plans to understand how to configure service plans in the service config.

Note: Currently it is not possible to change the configuration of the services.

3. Ensure Stemcell is Uploaded

Please see stemcells/ to learn more about the use of stemcells in anynines-deployment.

Ensure there is a stemcell uploaded to your BOSH director. Note that there is a different stemcell for each IaaS.

Have a look at https://bosh.io/ to find a link for your desired stemcell version.

You can then upload the stemcell version using the link provided in the previous step:

$ bosh upload-stemcell https://bosh.io/d/stemcells/...

4. Deploy the a9s Consul Domain Name System

The a9s Consul DNS is a system used for service discovery within the a9s Data Services. It must be in place before deploying any a9s Data Services. To deploy the a9s Consul DNS, run:

$ bosh deploy -d consul-dns consul-dns/consul-dns.yml -l config/iaas-config.yml

Please verify your consul-dns deployment before moving to the next steps. To do so, run:

$ bosh vms -d consul-dns

This command should produce an output similar to the following:

Using environment '192.168.50.6' as client 'admin'

Task 1113. Done

Deployment 'consul-dns'

Instance                                      Process State  AZ  IPs          VM CID                                VM Type
consul/23e80f11-3663-49b1-b08e-c3246fbea2e8   running        z1  10.244.5.15  32492afa-ed2e-448a-6c73-d8691e96d3c2  small
consul/3e9a5950-b146-4b98-9daf-b6c69b99d2ea   running        z2  10.244.6.15  b9cfbbde-4f1f-44cb-59b2-b5f7c965a2cf  small
consul/664a2a40-4a0c-4a16-a46f-4ecdf1e01f27   running        z3  10.244.7.15  06f1e485-b702-4e17-6fd0-4c151a7c36aa  small
dnsmasq/55bed760-e149-4afc-b1a4-dd0029e86145  running        z1  10.244.5.16  64ea1c82-9896-4097-6fa7-0f6e177f1f56  small
dnsmasq/8bc0f00a-609c-4df5-bba5-ac57c905cf87  running        z2  10.244.6.16  3f14f873-b576-4c2a-7149-34364230cb70  small

5 vms

Succeeded

Please ensure that each instance has the process state running. Further, pick the IP address of one of the dnsmasq servers and try to resolve a consul hostname:

$ dig @10.244.5.16 consul.service.consul +short
10.244.5.15
10.244.6.15
10.244.7.15

If the setup works correctly, the command above should return the IP addresses of the consul nodes.

5. Make the Consul DNS Available in your platform

Ensure the applications deployed to Cloud Foundry can resolve the hostnames of consul. You can set the suffix/TLD of the hostnames in your IaaS config under iaas.consul.domain.

To configure a DNS delegation inside your existing name server, you have to setup your name servers to delegate all DNS queries which end with your selected suffix to the IP addresses of the dnsmasq jobs in the consul-dns deployment.

If setting up a DNS delegation inside your name server is not an option, you can also configure the IP addresses of the dnsmasq servers in your elastic runtime so that each DNS query is first redirected to the consul-dns. The consul-dns will redirect all queries that don't end with your selected suffix to upstream name servers.

IMPORTANT: Once the Consul DNS is available in your platform, please double check that it is possible to resolve internal (e.g. consul.service.consul) hostnames as well as external hostnames. Perform this check in each AZ and each isolation segment of your platform.

6. Deploy a9s-pg

All a9s Data Service Framework components require this PostgreSQL Cluster to store their state.

Before deploying the a9s-pg you must provide the configuration for the backup.

Variable nameDescription
a9s_pg_backup_scheduleA Cron expression to schedule a9s-pg backups
a9s_pg_backup_id_prefixPrefix for backup files in the AWS bucket

Using AWS to store backups:

Variable nameDescription
a9s_pg_backup_aws_access_key_idThe AWS access key
a9s_pg_backup_aws_secret_access_keyThe AWS secret key
a9s_pg_backup_aws_bucketThe AWS bucket name
a9s_pg_backup_aws_regionThe AWS region where the bucket is

Using Azure to store backups:

Variable nameDescription
a9s_pg_backup_azure_storage_access_keyThe Azure storage access key
a9s_pg_backup_azure_storage_account_nameThe Azure storage account name
a9s_pg_backup_azure_containerThe Azure container name

Using Alibaba Cloud to store backups:

Variable nameDescription
/a9s_pg_aliyun_accesskey_idThe AliCloud access key
/a9s_pg_aliyun_accesskey_secretThe AliCloud secret key
/a9s_pg_aliyun_oss_bucketThe AliCloud bucket name
/a9s_pg_aliyun_oss_endpointThe AliCloud endpoint
/a9s_pg_aliyun_oss_locationThe AliCloud region name

In case you are using CredHub you can execute the following commands to define these variables:

$ credhub set --name '<BOSH-Director-name>/a9s-pg/a9s_pg_backup_schedule' --type value --value 'XXX'
$ credhub set --name '<BOSH-Director-name>/a9s-pg/a9s_pg_backup_id_prefix' --type value --value 'XXX'

For AWS, execute the following commands:

$ credhub set --name '<BOSH-Director-name>/a9s-pg/a9s_pg_backup_aws_access_key_id' --type value --value 'XXX'
$ credhub set --name '<BOSH-Director-name>/a9s-pg/a9s_pg_backup_aws_secret_access_key' --type password --password 'XXX'
$ credhub set --name '<BOSH-Director-name>/a9s-pg/a9s_pg_backup_aws_bucket' --type value --value 'XXX'
$ credhub set --name '<BOSH-Director-name>/a9s-pg/a9s_pg_backup_aws_region' --type value --value 'XXX'

For Azure, execute the following commands:

$ credhub set --name '<BOSH-Director-name>/a9s-pg/a9s_pg_backup_azure_storage_access_key' --type value --value 'XXX'
$ credhub set --name '<BOSH-Director-name>/a9s-pg/a9s_pg_backup_azure_storage_account_name' --type password --password 'XXX'
$ credhub set --name '<BOSH-Director-name>/a9s-pg/a9s_pg_backup_azure_container' --type value --value 'XXX'

For Alibaba Cloud, execute the following commands:

$ credhub set --name '/a9s_pg_aliyun_accesskey_id' --type value --value 'XXX'
$ credhub set --name '/a9s_pg_aliyun_accesskey_secret' --type password --password 'XXX'
$ credhub set --name '/a9s_pg_aliyun_oss_bucket' --type value --value 'XXX'
$ credhub set --name '/a9s_pg_aliyun_oss_endpoint' --type value --value 'XXX'
$ credhub set --name '/a9s_pg_aliyun_oss_location' --type value --value 'XXX'

<BOSH-Director-name>: You can find the BOSH-Director-name by running bosh env.

As alternative to CredHub you can use a creds file (e.g. secrets/a9s-pg-backup-secrets.yml):

For AWS:

a9s_pg_backup_aws_access_key_id:
a9s_pg_backup_aws_secret_access_key:
a9s_pg_backup_aws_bucket:
a9s_pg_backup_aws_region:
a9s_pg_backup_schedule:

For Azure:

a9s_pg_backup_azure_storage_access_key:
a9s_pg_backup_azure_storage_account_name:
a9s_pg_backup_azure_container:

For Alibaba Cloud:

/a9s_pg_aliyun_accesskey_id:
/a9s_pg_aliyun_accesskey_secret:
/a9s_pg_aliyun_oss_bucket:
/a9s_pg_aliyun_oss_endpoint:
/a9s_pg_aliyun_oss_location:

You must also pay attention to the a9s-pg max_connections configuration, by default the a9s PostgreSQL BOSH release calculates max_connections based on the size of shared_buffers and work_mem, but when deploying a9s-pg you might wanna use a higher value, that better reflects the usage of the services. To configure this parameter, use the iaas.a9s_pg.max_connections property in your IaaS File.

To deploy it, simply execute:

$ bosh deploy -d a9s-pg a9s-pg/a9s-pg.yml -l config/iaas-config.yml

If using Azure you need to add an ops file:

$ bosh deploy -d a9s-pg a9s-pg/a9s-pg.yml -l config/iaas-config.yml -o ops/a9s-pg-backup-on-azure.yml

This is also the case for Alibaba Cloud:

$ bosh deploy -d a9s-pg a9s-pg/a9s-pg.yml -l config/iaas-config.yml -o ops/a9s-pg-backup-on-alicloud.yml

If you are using a generic S3 API, you might want to take a look at the a9s-pg-backup-on-generic-s3.yml ops file.

Or (in case you are using an external creds file):

$ bosh deploy -d a9s-pg a9s-pg/a9s-pg.yml -l config/iaas-config.yml --vars-store secrets/creds.yml -l secrets/a9s-pg-backup-secrets.yml

You can also use the a9s dfs-helpers scripts to trigger backups and restore, to know more about this see the documentation here.

7. Deploy a9s Data Services

This step demonstrates how to deploy the a9s PostgreSQL Data Service. You can repeat this step for each service you want to install.

7a. Update BOSH director configuration

It is required that the a9s Data Service Components (the a9s Deployer to put it more specifically) can access a BOSH director in order to provision service instances. You have to configure the BOSH director credentials into the deployment. This file is also used for the configuration of the backup service and, if a s3 blobstore is used for the cf. This is done like by executing the following steps:

$ cp secrets/external_secrets.yml.example secrets/external_secrets.yml

and insert your BOSH credentials into secrets/external_secrets.yml.

7b. Deploy Components

Deploy via:

$ bosh deploy -d postgresql-service postgresql-service/postgresql-service.yml -l config/iaas-config.yml --vars-store secrets/creds.yml -l secrets/external_secrets.yml -l config/postgresql.yml

NOTE: Don't forgot to add -l secrets/external_secrets.yml and -l config/postgresql.yml in the command above.

NOTE2: When storing Elasticsearch backups on Azure, you must also use the ops/elasticsearch-service-backup-on-azure.yml ops file.

7c. Upload a9s Deployer Templates

The a9s Deployer is the component that talks with the BOSH director and creates BOSH deployments. In order to do this, this components needs templates of BOSH deployment manifests. There are templates for each a9s Data Service available.

Each service provides an errand called templates-uploader, this errand encapsulates this templates and uploads them to a9s Deployer.

After deploying the service, you can execute the errand with the following command:

$ bosh -d <deployment-name> run-errand templates-uploader

E.g.:

$ bosh -d postgresql-service run-errand templates-uploader

7d. Create Service Broker in Cloud Foundry

In order to make the a9s-postgresql service offering available in a Cloud Foundry marketplace execute the following commands as a Cloud Foundry admin:

$ cf create-service-broker a9s-postgresql-broker admin [password] http://postgresql-service-broker.service.dc1.consul:3000
$ cf enable-service-access a9s-postgresql94

For PostgreSQL the key for getting the password looks like this: /postgresql_service_broker_password. You can get it with bosh int secrets/creds.yml --path /postgresql_service_broker_password when using secret file or credhub get -n /postgresql_service_broker_password when using CredHub.

7e. Run Service Smoke Tests to Validate your Setup

For each a9s Data Service, there is a smoke-test errand included. This errand verifies that the main a9s Data Service features work in your environment:

  • Ensures service instances can be created
  • Ensures service instances can be bound to cf application
  • Ensures application can use the service (e.g. insert data to PostgreSQL)
  • Ensures service instances can be updated to a bigger plan (vertical scaling)
  • Ensures data is still in the database after vertical scaling
  • Ensures data can be backed up and restore (not activated yet)
$ bosh -d postgresql-service run-errand smoke-tests

IMPORTANT: Depending on how you have configured your Cloud Foundry Application Security Groups, you first must deploy the Service Guard (See below) otherwise the smoke tests will fail.

IMPORTANT: The a9s Kubernetes smoke tests do not work with a9s Service Guard. You need to manually create a Cloud Foundry App Security Group bound to the space kubernetes-smoke-tests that allows traffic to those instances.

8. Deploy a9s Service Guard (optional)

Deploying the a9s Service Guard is optional, but if you don't deploy the a9s Service Guard you have to create a global Cloud Foundry application security groups which whitelists the whole network where the service instances are deployed to.

Deploying the a9s Service Guard

To deploy the a9s Service Guard side by side with the a9s PostgreSQL service run

$ bosh deploy -d service-guard service-guard/service-guard.yml -l config/iaas-config.yml --vars-store secrets/creds.yml -o ops/add-postgresql-broker-to-service-guard.yml

IMPORTANT: If you are using also other services than the a9s PostgreSQL Service you have to add the corresponding Ops file to the command above. For more information have a look at the Ops files README.

Once the deployment of the a9s Service Guard is finished, you should see a Cloud Foundry application security group for each service instance existing in the corresponding Cloud Foundry space.

$ cf security-groups
Getting security groups as admin
OK

     Name                                         Organization   Space
#0   services
#1   public_networks
#2   dns
#3   guard_34f22bce-d540-46c0-9efd-8bf293ad0a95   anynines       test
#4   guard_933c47ba-e3fc-4b00-9549-6fd8f323f3ca   anynines       test

The security group are prefixed with guard_ and suffixed with the guid of the service instance.

Whitelist the Services Network

In case you don't use the service-guard you have to whitelist the whole services network. Otherwise cf pushed apps won't be able to access service instances. While this is useful for development and testing purposes it is not recommended for production systems.

Have a look at your IaaS config (e.g. config/iaas-config.yml) and note down the subnets where service instances will be created.

Check the currently active security groups via cf security-groups. Check the currently active services security group via cf security-group services. If what you see doesn't fit the subnets from the IaaS config you have to adjust them.

The following example refers to the BOSH-lite IaaS settings.

Create a file (security-group-services.json) with the following content:

[
  {
    "destination": "10.244.10.1-10.244.10.254",
    "protocol": "all"
  },
  {
    "destination": "10.244.11.1-10.244.11.254",
    "protocol": "all"
  },
  {
    "destination": "10.244.12.1-10.244.12.254",
    "protocol": "all"
  }
]

Update the existing security group via

cf update-security-group services security-group-services.json

Check whether the security group services is active via cf running-security-groups. If it is not listed there you have to bind it via

cf bind-running-security-group services

9. Deploy a9s Backup Manager (optional)

To deploy the a9s Backup Manager you must first provide credentials to a S3 bucket. Fill out the variables under the backup key of secrets/external_secrets.yml.

/backup_access_key:
/backup_secret_access_key:
/backup_bucket:
/backup_region:
backup_manager_encryption_key: Ggpe6P-=b2rEvrUyVGyabZ3YMRMAThBB5
/global_graphite_endpoint:

In case you are using CredHub you can execute the following commands:

$ credhub set --name '/backup_access_key' --type value --value 'XXX'
$ credhub set --name '/backup_secret_access_key' --type password --password 'XXX'
$ credhub set --name '/backup_bucket' --type value --value 'XXX'
$ credhub set --name '/backup_region' --type value --value 'XXX'
$ credhub set --name '/global_graphite_endpoint' --type value --value 'XXX'

IMPORTANT: /backup_endpoint needs to be set if you are not using AWS S3, but a generic S3 API compliant service. Set /backup_endpoint in your CredHub or creds file if you use only a AWS compatible service instead of AWS itself. (Hint: /backup_host could be set, but is ignored)

If you are using a generic S3 API, you might want to take a look at the ops/backup-on-generic-s3.yml Ops file.

Backup Monitoring

To monitor the backups in the backup store you need to configure all service brokers at backup-monit.config.broker. It is also possible to monitor backups that are not associated to a service broker. This is useful if you want to backup also services like the a9s-pg. Use the backup-monit.config.external_backups properties to configure these backups.

Monitoring backups on AWS store:

Variable nameDescription
/backup_access_keyThe AWS access key
/backup_secret_access_keyThe AWS secret key
/backup_bucketThe AWS bucket name
/backup_regionThe AWS region where the bucket is

Monitoring backups on Azure store:

In order to use Azure as backup store, you have to deploy with the backup-on-azure.yml ops file and configure the following variables:

Variable nameDescription
/azure_storage_access_keyThe Azure storage access key
/azure_storage_account_nameThe Azure storage account name
/azure_containerThe Azure container name

Monitoring backups on Alibaba Cloud store:

This is not supported at the moment.

IMPORTANT: backup_manager_encryption_key must be a string with minimum 32 characters (when using CredHub, it will be generated the right way)

After changing the secrets/creds.yml respectively add the variables to CredHub, run:

$ bosh deploy -d backup-service backup-service/backup-service.yml -l config/iaas-config.yml --vars-store secrets/creds.yml

IMPORTANT: Elasticsearch backup is not encrypted

The Elasticsearch backup is currently limited to AWS S3 and Azure as storage backend. We cannot use the default filter plugins. This service do not support generic S3 API services.

Deploy without the backup monitoring

There is an ops file which removes the backup monit plugin backup-service-without-backup-monit.yml.

$ bosh deploy -d backup-service backup-service/backup-service.yml -o ops/backup-service-without-backup-monit.yml -l config/iaas-config.yml --vars-store secrets/creds.yml
Metrics

In case a Graphite endpoint has been configured (/global_graphite_endpoint), the following metrics are generated for each known instance and streamed to that endpoint:

MetricDescription
backup.<instance_guid>.backup_monit.<deployment_name>.minutes_since_last_backupThe age of the last backup in minutes for this instance, or -1 if there is no backup yet.
backup.<instance_guid>.backup_monit.<deployment_name>.content_lengthThe size of the last backup for this instance.
  • <instance_guid>: The GUID of the instance.
  • <deployment_name>: The deployment name of the instance.

Any prefix a CF user creates for a service instance will only apply to metrics sent to the CF user's specified graphite endpoints. Metrics sent to the /global_graphite_endpoint will not be prefixed with any CF user defined prefixes.

9a. Azure Blob Store (optional)

To stream the backup to Azure Blob Store, apply the ops file ops/backup-on-azure.yml during the deployment of the Backup Manager.

$ bosh deploy -d backup-service backup-service/backup-service.yml -o ops/backup-on-azure.yml -l config/iaas-config.yml --vars-store secrets/creds.yml

Additionaly fill out the variables /azure_storage_access_key, /azure_storage_account_name, /azure_container. Follow the instructions in the Backup Manager Section how to do that.

9a. Alicloud Blob Store (optional)

To stream the backup to Alicloud Blob Store, apply the ops file ops/backup-on-alicloud.yml during the deployment of the Backup Manager.

$ bosh deploy -d backup-service backup-service/backup-service.yml -o ops/backup-on-alicloud.yml -l config/iaas-config.yml --vars-store secrets/creds.yml

Additionaly fill out the variables /aliyun_accesskey_id, /aliyun_accesskey_secret, /aliyun_oss_bucket, /aliyun_oss_location and /aliyun_oss_endpoint. Follow the instructions in the Backup Manager Section how to do that.

10. Deploy a9s Environment Info (optional)

With the a9s Environment Info service, a9s deployments provides a way to serve static information. The endpoints can be configured in your IaaS config (e.g. config/iaas-config.yml).

$ bosh deploy -d a9s-environment-info a9s-environment-info/a9s-environment-info.yml -l config/iaas-config.yml --vars-store secrets/creds.yml -l secrets/external_secrets.yml
  • Once the a9s Environment Info service is deployed you can use it to get a list of all service brokers, see example below (This API does not require any authorization).
  • You can modify the output of this service by manipulating the IaaS-config accordingly.
  • Mind to adapt the Service Broker URLs in the IaaS-config according to your chosen consul domain.
$ curl <a9s environment info host/ip>:3000/service_brokers | jq '.'
[
  {
    "service_description": "This is a service broker creating and managing dedicated MongoDB instances, powered by the anynines Service Framework",
    "service_name": "a9s-mongodb",
    "url": "http://mongodb-service-broker.service.dc1.consul:3000"
  },
  {
    "service_description": "This is a service broker creating and managing dedicated PostgreSQL service instances and clusters, powered by the anynines Service Framework",
    "service_name": "a9s-postgresql",
    "url": "http://postgresql-service-broker.service.dc1.consul:3000"
  },
  {
    "service_description": "This is a service broker creating and managing dedicated Redis service instances, powered by the anynines Service Framework",
    "service_name": "a9s-redis",
    "url": "http://redis-service-broker.service.dc1.consul:3000"
  },
  {
    "service_description": "This is a service creating and managing dedicated RabbitMQ service instances, powered by the anynines Service Framework'",
    "service_name": "a9s-messaging",
    "url": "http://rabbitmq-service-broker.service.dc1.consul:3000"
  },
  {
    "service_description": "This is a service broker creating and managing dedicated Elasticsearch service instances and clusters, powered by the anynines Service Framework",
    "service_name": "a9s-elasticsearch",
    "url": "http://elasticsearch-service-broker.service.dc1.consul:3000"
  },
  {
    "service_description": "This is a service broker creating and managing dedicated MySQL service instances and clusters, powered by the anynines Service Framework",
    "service_name": "a9s-mysql",
    "url": "http://mysql-service-broker.service.dc1.consul:3000"
  },
  {
    "service_description": "This is a service broker creating and managing dedicated ELK stacks to monitor applications and service instances, powered by the anynines Service Framework",
    "service_name": "a9s-logme",
    "url": "http://logme-service-broker.service.dc1.consul:3000"
  }
]

11. Deploy a9s Billing (optional)

The a9s Billing system can be used for two use cases:

  • Observe the service instance usage via timeseries
  • Create invoices based on price plans and platform usage data

To deploy the a9s Billing system for the first use case, run:

$ bosh deploy -d a9s-billing a9s-billing/a9s-billing.yml -o a9s-billing/ops/add_grafana.yml -o a9s-billing/ops/grafana_dashboard_data_services.yml -o a9s-billing/ops/add_grafana_route.yml -o a9s-billing/ops/without_invoices.yml -l config/iaas-config.yml --vars-store secrets/creds.yml
  • The a9s-billing/ops/add_grafana.yml opsfile adds an instance group which runs grafana.
  • The a9s-billing/ops/grafana_dashboard_data_services.yml opsfile configures grafana with a default dashboard to visualize Data Service usage.
  • The a9s-billing/ops/add_grafana_route.yml opsfile, adds a route to the gorouter so that grafana can be accessed the same way the CF API can be accessed.
  • The a9s-billing/ops/without_invoices.yml disables the invoice subsystem.

You can access the statistics using the following credentials:

SYSTEM_DOMAIN=`bosh int config/iaas-config.yml --path /iaas/cf/system_domain `
GRAFANA_URL="https://a9s-billing-dashboard.${SYSTEM_DOMAIN}"
GRAFANA_USER=admin
GRAFANA_PASSWORD=`credhub get -n /bosh-lite/a9s-billing/grafana_password`

Open the GRAFANA_URL in your browser and access the value from GRAFANA_USER and GRAFANA_PASSWORD in the browser.

NOTE: It can take up to one day before you see the usage of Data Service instances in grafana.

12. Enable Production Ready Services Only

By default each a9s Data Service manifest contains all a9s Data Service versions. If you want to enable production ready services only, you need to apply the ops/enable-production-ready-services-only.yml ops file.

13. Plan Updates

The a9s Data Service instances support updates from smaller to bigger plans.

Examples:

  • postgresql-single-small to postgresql-single-big
  • postgresql-replica-small to postgresql-replica-big
  • postgresql-single-small to postgresql-replica-small
  • postgresql-single-small to postgresql-replica-big

The following a9s Data Services do not support plan updates from single to cluster:

  • a9s LogMe in general, since there are no plan updates
  • a9s MongoDB 3.2
  • a9s PostgreSQL 9.4

Known Issues

  • At the moment you cannot deploy the Data Service Framework without having Cloud Foundry deployed before.
  • The data center component of a Consul DNS name must not be empty or contain a "." (dot). It should not be changed from the default "dc1". We are working on eliminating these requirements.

Additional Docs