Troubleshooting
danger
This feature is not fully released yet!
We strongly recommend to test this feature in testing environments and do not deploy to production environments yet.
General
Unless you are using your companies CA, we are using self-signed certificates to protect the endpoints provided by the different a9s Framework components.
Those certificates are created by CredHub. They are included in the deployment manifest under the key variables.
More information can be found on the configuration page.
Analyzing TLS Certificates
When working with TLS certificates, it's not obvious what the content of the certificate is, as the certificates are Base64 encoded.
There are several online tools that decodes certificates and displays informations about it, such as Common Names, Subject Alternative Names, Validity, and so forth. One tool is the Certificate Decoder. You can simply paste the certificate in PEM format in the textbox, and it will show you those informations.
If you have the openssl CLI available, you can also get the certificate informations by running:
openssl x509 -in <Certificate_Path> -text -noout
Exchange <Certificate_Path> with the path on your filesystem, where the certificate is located.
Certificate Chain
As you can see in the Certificate Authority Overview table, we're using a chain of certificate authorities to create the certificates for our components. In order to communicate with the TLS secured endpoints, you need to provide a valid certificate chain.
A certificate chain is the concatenation of multiple certificate authority certificates in PEM format.
A very basic certificate chain could look like this:
-----BEGIN CERTIFICATE-----
[Content of certificate 1]
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
[Content of certificate 2]
-----END CERTIFICATE-----
Important: A private key is never part of a certificate chain!
You also need to have the correct order of the certificates. It always ends with the Root CA and starts with the last CA, which signed the components certificate, followed by the intermediate CA(s).
Example: In our setup, you can see the following BOSH property in the deployment manifest:
((/a9s_deployer_ca.certificate))((/a9s_components_ca.certificate))
This is a concatenation of two certificates, starting with the last CA which signed the components certificate, and ending with the Root CA.
Hint: You can check the correctness of a certificate chain with this tool: Certificate Chain Checker
Manual cURL Requests
If you want to test an endpoint of a component, you can use curl for this.
Example: Testing the a9s Service Broker for a9s Messaging HTTPS endpoint with curl:
curl -u "Broker_Username":"Broker_Password" https://rabbitmq-service-broker.service.dc1.consul.example:3001/instances --cacert [CA_Certificate]
Keep in mind that you need to provide a valid certificate chain with the curl request!
Last Resort
If nothing seems to work it can help to remove all necessary certificates from CredHub and deploy the Data Service manifest again. The missing certificates will be (re-)created on deployment.
If you have trouble to find the correct certificates, keep in mind that they have the following structure:
/<BOSH director name>/<BOSH Deployment name>/<Components Certificate name>
More information on CredHub paths can be found on the configuration page
You can also find all certificates for a specific deployment with:
credhub find | grep <BOSH Deployment name>
Example:
credhub find | grep rabbitmq-service