Primary Alias Monitor

This document describes how the primary alias monitor works and how to configure this feature.

Summary

Primary Alias

All data services provided by anynines support a DNS entry that points to one node in the deployment. This DNS entry name changes which node is pointing according to the cluster state.

a9s PostgreSQL offer two options for deployments: single and cluster (sometimes called replica). On single instances the DNS entry always points to the node of the deployment. On cluster deployments this alias always points to the primary of the cluster (the node that handles write operations).

This alias is updated when the cluster goes through changes. For example, when the current primary goes down and a new node is promoted to primary, the alias starts pointing to the new primary after the promotion.

a9s PostgreSQL Primary Alias Monitor

a9s PostgresSQL collocates a posgresql-info-webservice process on each node along side PostgreSQL. The postgresql-info-webservice continuously monitors the state of a cluster. When the state of the cluster changes the postgresql-info-webservice will updated the primary alias if necessary.

When a change in the address the alias is pointing to is detected, the primary alias is marked as unstable and fixed to point to the current primary. This update follows two rules:

  • Only the primary of the cluster runs the monitor: Standby nodes just skip the check to avoid resource over usage.
  • Only the primary of the majority can update the alias: this means that nodes in a split partition can not update the alias. This means that the primary of the majority of the cluster retrieves the status for all nodes in order to detect who is the correct primary (primary of the majority).

Monitoring Frequency

When the alias is marked as unstable, checks starts happening more frequently, every min_primary_dns_check_interval (default 10s). When the alias remains correct and for primary_dns_stable_age (default 10m), the alias is marked as stable and checks starts happening less frequently, every max_primary_dns_check_interval (default 5m). Currently it is not possible to change the values. Let us know if you need this values to be configurable.

Configuring Primary Alias Monitor

Primary alias monitor is currently active by default, it is possible to disable by removing ops/postgresql-enable-alias-monitor.yml file from properties.template-uploader.template-ops-files. Note that after this change, the templates-uploader has to bee executed so the configuration can be applied to new instances. On existing instances, after running the templates-uploader errand it also necessary to execute the deployment-updater errand, so the new configuration is applied. This configuration must be applied on the a9s PostgreSQL service manifest.