a9s Router Architecture

This document describes the a9s Router component.

Overview

The a9s Router component routes traffic to appropriate targets.

The idea is to make it possible for multiple stackholders to register routes at one a9s Router setup, to make the corresponding applications available over the internet.

The a9s Router consists currently of the a9s Route Registry and the HAProxy backend.

a9s Route Registry

The a9s Route Registry manages routes. There is a HTTP API available for the a9s Route Registry to add, delete and get routes. There's a component called a9s Route Registrar that leverages this HTTP API to announce routes.

The a9s Route Registry syncs periodically the routes with the HAProxy backend.

The a9s Kubernetes service instances for example use the a9s Route Registrar to announce their route, ip and port combination to the a9s Route Registry. The routes get synced with the HAProxy backend.

HAProxy Backend

The HAProxy backend is a basic HAProxy 3 noded cluster. This setup includes a Data Plane API which is used to configure the HAProxy cluster dynamically.

This HAProxy cluster is able to route HTTP/2 and SPDY traffic for multiple domains based on the Host HTTP header. It was originally developed to fulfill the role of a single HA entry point for multiple Kubernetes deployments in one platform.

A route in the HAProxy backend context is a HAProxy configuration consisting of a backend switching rule and a backend. A backend switching rule is a rule within a frontend. In the frontend we define which domain should be routed to which backend. And a backend is a definition of one or multiple endpoints to route the traffic to (target endpoint).

BOSH Release

The a9s-router BOSH Release has multiple jobs:

• haproxy - starts HAProxy with Data Plane API
• a9s-route-registry - starts the a9s Route Registry: collects, maintains routes and syncs routes to HAProxy
• a9s-route-registrar - starts the a9s Route Registrar: announces periodically routes to the a9s Route Registry

Route Registering

Every worker node has an a9s Route Registrar process that announces its route every a9s-route-registrar.registry.publish_route_interval (default 10s), it connects to the a9s Route Registry using the a9s Node Guard. At this point, the connection is TLS encrypted.

The a9s Node Guard forwards the connection to the a9s Route Registry, which creates an entry for the route in its internal registry. This registry is kept in memory only.

Every a9s-route-registry.update_interval (default 10s), the a9s Route Registry synchronizes the internal state of its registry with all HAProxy instances using the Data Plane API. This connection is also TLS encrypted. The Data Plane API synchronizes with the HAProxy process every ha_proxy.dataplaneapi.reload_delay: (default 5s). Check here to know more about it.

Route Unregistering

If a node fails to send heartbeats after a9s-route-registry.update_interval + a9s-route-registry.ttl_interval (default: 10s), the a9s Route Registry will understand that this node is gone, and it will trigger the removal of the route to the failing node.

The a9s Route Registry uses again the Data Plane API to request the HAProxy to remove the route of the shutdown or failing node. If all nodes of the deployment fails to send the heartbeats (e.g.: Deployment deleted), the whole configuration for the target instance is removed from HAProxy.

Routing

When a request reaches the HAProxy it discovers the target by matching the domain with the route, and fowarding the request to the target. Any target can be specified.

We recommend a load balancer in front of th HAProxy cluster. The request is routed to a HAProxy node, that matches the domain of the request with the configured route, after that the connection is routed to the correct target (address and port). To know more about how a9s Kubernetes handles this connection, check the a9s Kubernetes documentation.

Known Issues

• If for any reason a route is configured on the HAProxy, but not on the a9s Router Registry, this route will not be removed by the a9s Router Registry. This should be fixed on latter versions.