Using a9s Search

This topic describes how developers use a9s Search.

Use a9s Search with an Application

To use a9s Search with an application, create a service instance and bind the service instance to your application. For more information on managing service instances, see Managing Service Instances with the cf CLI.

View the a9s Search Service

After the service is installed, you can see the a9s-search and its service plans appear in your CF marketplace. Run cf marketplace to see the service listing:

$ cf marketplace
Getting services from marketplace in org test / space test as admin...
OK
service               plans                                                     description
a9s-search2           search-single-small, search-cluster-small,                This is the a9s Search 2 service.
                      search-single-big, search-cluster-big

Create a Service Instance

To provision an OpenSearch database, run cf create-service. For example:

cf create-service a9s-search2 search-single-small my-search-service

Depending on your infrastructure and service broker utilization, it may take several minutes to create the service instance.

Run the cf services command to view the creation status. This command displays a list of all your service instances. To view the status of a specific service instance, run cf service NAME-OF-YOUR-SERVICE.

Bind an Application to a Service Instance

After your database is created, run cf bind-service to bind the service to your application:

cf bind-service a9s-search-app my-search-service

Restage or Restart Your Application

To enable your application to access the service instance, run cf restage or cf restart to restage or restart your application.

Obtain Service Instance Access Credentials

After a service instance is bound to an application, the credentials of your OpenSearch database are stored in the environment variables of the application. Run cf env APP-NAME to display the environment variables.

You can find the credentials in the VCAP_SERVICES key.

$ cf env a9s-search-app
Getting env variables for app a9s-search-app in org test / space test as admin...
OK

System-Provided:
{
  "VCAP_SERVICES": {
   "a9s-search2": [
    {
      "credentials": {
       "host": [
        "EXAMPLE_HOST"
       ],
       "hosts": [
        "EXAMPLE_HOST"
       ],
       "password": "EXAMPLE_USER",
       "username": "EXAMPLE_PASSWORD",
       "scheme": "http",
       "port": 9200
     },
     "label": "a9s-search2",
     "name": "my-search-service",
     "plan": "search-cluster-small",
     "tags": [
      "searchengine"
     ]
    }
   ]
  }
}
...

You can use the host, username and password values to connect to your database with an OpenSearch client.

Best Practices

There are some best practices for using service binding information in apps in a separate document.

Delete an a9s Search Service Instance

WARNING: Before deleting a service instance, you must backup data stored in your database. This operation cannot be undone and all the data is lost when the service is deleted.

Before you can delete a service instance, you must unbind it from all apps.

List Available Services

Run cf services to list your available services.

$ cf services

Getting services in org test / space test as admin...
OK

name                       service              plan                         bound apps              last operation
my-search-service          a9s-search2   search-single-small          a9s-search-app          create succeeded

This example shows that my-search-service is bound to the a9s-search-app application.

Unbind a Service Instance

Run cf unbind-service to unbind the service from your application.

cf unbind-service a9s-search-app my-search-service

Delete a Service Instance

After unbinding the service, it is no longer bound to an application. Run cf delete-service to delete the service:

cf delete-service my-search-service

It may take several minutes to delete the service. Deleting a service deprovisions the corresponding infrastructure resources. Run the cf services command to view the deletion status.

Upgrade the Service Instance to another Service Plan

Once created, you can upgrade your service instance to another, larger service plan. A larger service plan provides more CPU, RAM and storage. For more information, see the Update a Service Instance of the Managing Service Instances with the cf CLI topic.

cf update-service my-search-service -p a-bigger-plan

Add a Graphite Endpoint

Important: Streaming of logs and metrics might not be available for your plan! If unsure, please check your plan description.

If you want to monitor your service with Graphite, you can set the custom parameter graphite. It expects the host and port where the Graphite metrics should be sent to.

For example, in order to send Graphite metrics to an endpoint yourspace.your-graphite-endpoint.com:12345, you can use the following command:

cf update-service my-instance -c '{ "graphite": "yourspace.your-graphite-endpoint.com:12345" }'

The endpoint would then receive metrics in the format:

<service_guid>.<service_type>.<host>.<metric> <metric value> <metric timestamp>

Metrics Frequency

By default, metrics will be emitted every 10 seconds. You can change the interval via the custom parameter metrics_frequency.

For example, in order to send Graphite metrics to an endpoint every minute, you would set the custom parameter metrics_frequency to 60 using the following command:

cf update-service my-instance -c '{  "metrics_frequency": 60 }'

Metrics Prefix

Depending on your graphite provider, you might need to prefix the metrics with a certain value, like an API key for example. In this case you can leverage the custom parameter metrics_prefix.

cf update-service my-instance -c '{  "metrics_prefix": "my-api-key-for-a-certain-provider" }'

The resulting metric path would have the format:

<metrics_prefix>.<service_guid>.<service_type>.<host>.<metric>

Metrics

These are the all metrics provided by the a9s Search.

Metric Id	type	Description
metric_fetch_status	Integer (1 = true / 0 = false)	When the last metric update is successful the result is true.
cluster_health.active_primary_shards	Integer	The number of active primary shards. documentation
cluster_health.active_shards	Integer	The total number of active primary and replica shards. documentation
cluster_health.active_shards_percent_as_number	Float	Percent of shards active documentation
cluster_health.delayed_unassigned_shards	Integer	The number of shards whose allocation has been delayed by this timeout. documentation
cluster_health.initializing_shards	Integer	The number of shards that are under initialization. documentation
cluster_health.number_of_data_nodes	Integer	The number of nodes that are dedicated data nodes. documentation
cluster_health.number_of_in_flight_fetch	Integer	The number of unfinished fetches. documentation
cluster_health.number_of_nodes	Integer	The number of nodes within the cluster. documentation
cluster_health.number_of_pending_tasks	Integer	The number of cluster-level changes that have not yet been executed. documentation
cluster_health.relocating_shards	Integer	The number of shards that are under relocation. documentation
cluster_health.status	Integer (0 = green / 1 = yellow / 2 = red / 3 = unknown)	Health status of the cluster, based on the state of its primary and replica shards. documentation
cluster_health.task_max_waiting_in_queue_millis	Integer	The time expressed in milliseconds since the earliest initiated task is waiting for being performed. documentation
cluster_health.timed_out	Integer (1 = true / 0 = false / 2 = unknown)	If false the response returned within the period of time that is specified by the timeout parameter (30s by default). documentation
cluster_health.unassigned_shards	Integer	The number of shards that are not allocated documentation
discovered_master	Integer (1 = true / 0 = false)	Specifies whether the cluster master is discovered.
discovered_cluster_manager	Integer (1 = true / 0 = false)	Specifies whether the cluster manager is discovered.

Examples

*.opensearch.*.*.*.*.metric_fetch_status
*.opensearch.*.*.*.*.cluster_health.active_primary_shards
*.opensearch.*.*.*.*.cluster_health.active_shards
*.opensearch.*.*.*.*.cluster_health.active_shards_percent_as_number
*.opensearch.*.*.*.*.cluster_health.delayed_unassigned_shards
*.opensearch.*.*.*.*.cluster_health.initializing_shards
*.opensearch.*.*.*.*.cluster_health.number_of_data_nodes
*.opensearch.*.*.*.*.cluster_health.number_of_in_flight_fetch
*.opensearch.*.*.*.*.cluster_health.number_of_nodes
*.opensearch.*.*.*.*.cluster_health.number_of_pending_tasks
*.opensearch.*.*.*.*.cluster_health.relocating_shards
*.opensearch.*.*.*.*.cluster_health.status
*.opensearch.*.*.*.*.cluster_health.task_max_waiting_in_queue_millis
*.opensearch.*.*.*.*.cluster_health.timed_out
*.opensearch.*.*.*.*.cluster_health.unassigned_shards
*.opensearch.*.*.*.*.cluster_health.discovered_master
*.opensearch.*.*.*.*.cluster_health.discovered_cluster_manager

System Metrics

Aside from the metrics shown above, this Data Service also provides system metrics, common to all a9s Data Services. They can be seen in the Graphite metrics documentation.

Cloud Foundry Org and Space Guid

The platform operators can enable support on a global level to prefix the Graphite metrics with the CloudFoundry organization and space. This means that all metrics of all service instances (not only yours!) contain that information.

In this case the Graphite metric paths have the following format:

<organization_guid>.<space_guid>.<service_guid>.<service_type>.<host>.<metric>

When you enable in addition the metrics_prefix for your instance, you will end up with the metric path format:

<metrics_prefix>.<organization_guid>.<space_guid>.<service_guid>.<service_type>.<host>.<metric>

Add a Syslog Endpoint

The cf update-service command used with the -c flag can let you stream your syslog to a third-party service. In this case, the command expects a JSON string containing the syslog key. A single endpoint item for syslog can be provided, as well as an array of endpoints. host and port are mandatory parameters.

The protocol parameter is optional. The accepted values are:

• udp sets rfc by default to rfc3164.
• tcp sets rfc by default to rfc3164.
• ssl-tcp sets rfc by default to rfc6587.
  • For ssl-tcp; ssl_cert and ssl_key are required parameters.
  • If ssl_verify is set to true, then ssl_cacert must also be provided.

To remove all endpoints, simply set syslog to [].

rfc values

Optionally, the rfc parameter can be explicitly overwritten. Valid values: rfc3164, rfc5424 and rfc6587.

cf update-service my-service \
 -c '{
  "syslog": [
    {
      "host": "logs4.your-syslog-endpoint.com",
      "port": "6514",
      "protocol": "ssl-tcp",
      "rfc": "rfc6587",
      "ssl_cert": "-----BEGIN CERTIFICATE----- ... -----END CERTIFICATE-----",
      "ssl_key": "-----BEGIN CERTIFICATE----- ... -----END CERTIFICATE-----",
      "ssl_cacert": "-----BEGIN PRIVATE KEY----- ... -----END PRIVATE KEY-----",
      "ssl_verify": "false"
    }
  ]
}'

Syslog Endpoints' Legacy Format

The legacy format to specify syslog endpoints (["hostname1:port1", "hostname2:port2"]) is still supported.

Enable Custom Plugins

You can use the cf update-service command with the -c flag to enable plugins on demand such as the analysis-phonetic plugin which convert tokens to their phonetic representation using Soundex, Metaphone, and a variety of other algorithms.

To enable the analysis phonetic plugin you would use the following command:

cf update-service my-search-service \
  -c '{"plugins": ["analysis-phonetic"]}'

The currently available plugins are:

• repository-s3 (enabled by default, cannot be disabled)
• repository-azure (enabled by default, cannot be disabled)
• analysis-phonetic

Cloud Foundry Application Security Groups

This topic describes how to check whether a security group was created.

Each a9s Data Service will automatically create and update Cloud Foundry security groups in order to protect service instances to be accessed by applications not running in the same Cloud Foundry applications space. To get a better understanding about Security Groups you can have a look on the Understanding Application Security Groups topic.

Get Service Instance GUID

Run cf service INSTANCE_NAME --guid to get the guid of the service instance.

$ cf service my-search --guid
ca16f111-5073-40b7-973a-156c75dd3028

Check available Security Groups

To see all available security groups use cf security-groups.

$ cf security-groups
Getting security groups as demo@anynines.com
OK

     Name                                         Organization     Space
#0   public_networks
#1   dns
#2   tcp_open
#3   guard_432fb752-876d-443b-a311-a075f4df2237   demonstrations   demo
#4   guard_ca16f111-5073-40b7-973a-156c75dd3028   demonstrations   demo

There you can see a security group named guard_ca16f111-5073-40b7-973a-156c75dd3028 was successfully created.

NOTE: in some circumstances the connection between the application and the service instance is not possible, in this case check if a security group was created.

Backup and Restore Service Instances

a9s Search provides an easy way to create backups and restore if needed. For a more detailed description, please see the a9s Service Dashboard documentation.

Known Limitations of a9s Search's Backups

• The backup process is based on OpenSearch's Snapshot functionality, this means that:
  • Backups are not encrypted.
    • Due to this, the backup_manager_encryption_key property cannot be set.
  • Backups cannot be downloaded.
  • The storage backend is limited to AWS S3 and Azure; generic S3 API services are not supported.
• OpenSearch's default filter plugins are not available.

Make a Service Instance Locally Available

It is possible to access any of the a9s Data Services locally. That means you can connect with a local client to the service for any purpose such as debugging. CF provides a smart way to create SSH forward tunnels via a pushed application. For more information about this feature see the Accessing Apps with SSH section of the CF documentation.

First of all you must have an application bound to the service. How to do this see Bind an Application to a Service Instance.

NOTE: cf ssh support must be enabled in the platform. Ask your administrator if you are not sure.

Get The Service Url and Credentials

When you follow this instructions Obtain Service Instance Access Credentials you will get the hostname of the service and the user credentials.

$ cf env a9s-search-app
Getting env variables for app a9s-search-app in org test / space test as admin...
OK

System-Provided:
{
  "VCAP_SERVICES": {
   "a9s-search2": [
    {
      "credentials": {
       "host": [
        "d67901c.service.dc1.a9svs"
       ],
       "password": "a9s-brk-usr",
       "username": "a9s-password"
     },
     "label": "a9s-search2",
     "name": "my-search-service",
     "plan": "search-cluster-small"
    }
   ]
  }
}
...

Notice the host d67901c.service.dc1.a9svs, the username a9s-brk-usr and the password a9s-password. You will need this in the next step.

Create a Tunnel to The Service

With the cf ssh as mentioned before you can create a ssh forward tunnel to the management dashboard. Use port 9200 to connect to the a9s Search Instance.

$ cf ssh a9s-search-app -L 9200:d67901c.service.dc1.a9svs:9200
vcap@956aaf4e-6da9-4f69-4b1d-8e631a403312:~$

When the ssh tunnel is open you can access the instance over the address localhost:9200.

NOTE: Don't forget to close the session with exit.

Setup Disk Usage Alerts

Each service comes with the a9s Parachute. This component monitors ephemeral and persistent disk usage. See the a9s Parachute documentation how to configure the component.

Custom Parameters

As an end user you have the opportunity to customize your service instance by using custom parameters. Custom parameters are passed on to a service instance by using the -c switch of the cf CLI commands cf create-service and cf update-service. For example

cf update-service my-instance -c '{"my_custom_parameter": "value"}'

would set the custom parameter named my_custom_parameter to the value value for the service instance my-instance.

You don't have to utilize those settings. There are sane defaults in place that fit your service plan well.

Tuning

As usage of an a9s Search instance grows you may need to tune the OpenSearch to meet demand.

The following custom parameters are available for tuning the used memory:

Name	Description	Min	Max	Default
java_heapspace	The amount of memory (in MB) allocated as heap by the JVM for OpenSearch	256	%50 of VM memory (check your plan details)	not set, 46% of available memory will be used
java_maxmetaspace	The amount of memory (in MB) used by the JVM to store metadata for OpenSearch	256	1024	512

Additionally there is a custom parameter available to set the used Garbage Collector:

Name	Description	Default	Available Options
java_garbage_collector	The JVM Garbage Collector to be used for OpenSearch.	UseG1GC	UseSerialGC, UseParallelGC, UseParallelOldGC, UseG1GC

TLS

tls-protocols

You can specify the allowed TLS protocols via the custom parameter tls-protocols. The custom parameter tls-protocols correlates with OpenSearch's configuration parameters plugins.security.ssl.http.enabled_protocols and plugins.security.ssl.transport.enabled_protocols, see Limiting TLS Protocols Used by the Server.

An array with protocol versions is expected. Only Java format is supported. The allowed protocol version values are TLSv1.3 and TLSv1.2.

tls-ciphers

You can limit the TLS ciphers via the custom parameter tls-ciphers. The custom parameter tls-ciphers correlates with OpenSearch's configuration parameters plugins.security.ssl.http.enabled_ciphers and plugins.security.ssl.transport.enabled_ciphers, see Configuring Cipher Suites.

An array with cipher names is expected. Only Java format is supported.

WARNING: There is no validation enabled for the user provided value and therefore existing instances can break when applying this parameter.