Using a9s Prometheus

This topic describes how to use a9s Prometheus for PCF after it has been successfully installed.

Stream service metrics to a9s Prometheus

To use a9s Prometheus with an application, create a service instance and bind the service instance to your application. For more information on managing service instances, see Managing Service Instances with the cf CLI.

View the Service Offering

After the tile is installed, run cf marketplace to see a9s-prometheus service offering and its service plans:

$ cf marketplace
Getting services from marketplace in org test / space test as admin...
OK
service    plans                                                                            description
a9s-prometheus  prometheus-single-small, prometheus-single-medium, prometheus-single-big, prometheus-platform     This is the a9s Prometheus service.

Create a Service Instance

To provision a Prometheus instance, run cf create-service SERVICE-NAME PLAN-NAME INSTANCE-NAME where INSTANCE-NAME is any name you want to give the instance you create.

$ cf create-service a9s-prometheus prometheus-single-small my-prometheus-service

Depending on your infrastructure and service broker utilization, it may take several minutes to create the service instance.

Run the cf services command to view the creation status. This command displays a list of all your service instances. To view the status of a specific service instance, run cf service NAME-OF-YOUR-SERVICE.

Create a Service Key to get Information about your deployment

Create a service key to get URLs of your Prometheus deployment:

$ cf create-service-key my-prometheus-service key1
$ cf service-key my-prometheus-service key1

{
"graphite_exporters": ["d37f7da-prometheus-0.node.dc1.consul"],
"graphite_exporter_port": 9109,
"prometheus_urls": "http://d37f7da-prometheus-0.node.dc1.consul:9090/",
"grafana_urls": "http://d37f7da-grafana-0.node.dc1.consul:3000/",
"alertmanager_urls": "http://d37f7da-alertmanager-0.node.dc1.consul:9093/",
"username": "username",
"password": "password"
}

Here you get all Informations about how to reach the components of your Prometheus deployment.

Configure a service instances to stream the logs

That your service instance streams it's logs to your Prometheus instance, make the following steps:

  1. Create a service key to get URL of the graphite exporter (see above)

  2. Create or update your service instance and add the graphite custom parameter, i.e.:

$ cf create-service a9s-redis redis-single-small my-redis-service -c '{ "graphite": [ "d37f7da-prometheus-0.node.dc1.consul:9109" ] }'
$ cf update-service my-redis-service -c '{ "graphite": [ "d37f7da-prometheus-0.node.dc1.consul:9109" ] }'

As soon as your instance in deployed/updated, it will stream its metrics to the graphite exporter of your prometheus.

See your metrics

To access the dashboards of you a9s Prometheus instance, execute the following command:

cf service SERVICE_NAME

you will get an output similar to this:

Showing info of service SERVICE_NAME in org ORG / space SPACE as admin...

name:             SERVICE_NAME
service:          a9s-prometheus
tags:
plan:             single-small
description:      This is a service creating and managing dedicated Prometheus to monitor applications and service instances, powered by the anynines Service Framework
documentation:    https://www.anynines.com
dashboard:        https://a9s-dashboard.system.a9s-ds-concourse.a9s-ops.de/service-instances/2fe38eeb-e3a0-4d18-86bc-1c0c0b0e303f
service broker:   prometheus-service-broker

Showing status of last operation from service SERVICE_NAME...

status:    update succeeded
message:
started:   2019-04-26T08:48:11Z
updated:   2019-04-26T08:49:18Z

There are no bound apps for this service.

Extract the dashboard URL from here, i.e.:

https://a9s-dashboard.system.a9s-ds-concourse.a9s-ops.de/service-instances/2fe38eeb-e3a0-4d18-86bc-1c0c0b0e303f

To access Grafana add "grafana" to the URL:

https://a9s-dashboard.system.a9s-ds-concourse.a9s-ops.de/service-instances/2fe38eeb-e3a0-4d18-86bc-1c0c0b0e303f/grafana

To access Prometheus add "prometheus" to the URL:

https://a9s-dashboard.system.a9s-ds-concourse.a9s-ops.de/service-instances/2fe38eeb-e3a0-4d18-86bc-1c0c0b0e303f/prometheus

To access Alertmanager add "alertmanager" to the URL:

https://a9s-dashboard.system.a9s-ds-concourse.a9s-ops.de/service-instances/2fe38eeb-e3a0-4d18-86bc-1c0c0b0e303f/alertmanager

We also plan to provide an easier to access dashboard for the final release

Best Practices

There are some best practices for using service binding information in apps in a separate document.

Stop streaming service metrics to a9s Prometheus

You have 2 possibilities to stop streaming metrics:

  1. you decide to stop using Prometheus at all, in this case you can delete the Prometheus instance directly
  2. you decide to stop streaming of a spacific instance, in this case you need to reconfigure your instance

Also in case 1 it is strongly recommended to also update your service instances!

List Available Services

Run cf services to list available service instances and get the name of the service instance you want to delete.

$ cf services

Getting services in org test / space test as admin...
OK

name                 service       plan                   bound apps        last operation
my-prometheus-service     a9s-prometheus     prometheus-single-small     -     create succeeded
my-redis-service     a9s-redis     redis-single-small     a9s-redis-app     create succeeded

This example shows that my-prometheus-service is not bound to an app. To stop the streaming of metrics, we could simply delete the my-prometheus-service instance.

Delete the Prometheus Service Instance

Run cf delete-service INSTANCE-NAME to delete the service instance:

$ cf delete-service my-prometheus-service

It may take several minutes to delete the service. Deleting a service deprovisions the corresponding infrastructure resources. Run the cf services command to view the deletion status.

To come back to a clean setup, your should also reconfigure your service instance.

Overwrite Your Service configuration

If you want to stop streaming your service instance metrics to your Prometheus instance, you can overwrite the graphite key of your service configuration.

Run cf update-service INSTANCE-NAME -c PARAMETERS-AS-JSON to update the graphite endpoint of the instance you don't want to monitor anymore. For this, you need to overwrite the value of the graphite key with an empty string.

$ cf update-service my-redis-service -c '{"graphite": [""]}'

Delete the Service Key and Instance

If you are not using it anymore, you may want to delete the service key and the service instance itself. Run cf delete-service-key INSTANCE-NAME SERVICE-KEY-NAME and cf delete-service INSTANCE-NAME to do so.

$ cf delete-service-key my-prometheus-service key1
$ cf delete-service my-prometheus-service

It may take several minutes to delete the service. Deleting a service deprovisions the corresponding infrastructure resources. Run the cf services command to view the deletion status.