a9s Router Architecture
This document describes the a9s Router component.
The a9s Router component routes traffic to appropriate targets.
The idea is to make it possible for multiple stackholders to register routes at one a9s Router setup, to make the corresponding applications available over the internet.
The a9s Router consists currently of the a9s Route Registry and the HAProxy backend.
a9s Route Registry
The a9s Route Registry manages routes. There is a HTTP API available for the a9s Route Registry to add, delete and get routes. There's a component called a9s Route Registrar that leverages this HTTP API to announce routes.
The a9s Route Registry syncs periodically the routes with the HAProxy backend.
The a9s Kubernetes service instances for example use the a9s Route Registrar to announce their route, ip and port combination to the a9s Route Registry. The routes get synced with the HAProxy backend.
The HAProxy backend is a basic HAProxy 3 noded cluster. This setup includes a Data Plane API which is used to configure the HAProxy cluster dynamically.
This HAProxy cluster is able to route HTTP/2 and SPDY traffic for multiple domains based on the Host HTTP header. It was originally developed to fulfill the role of a single HA entry point for multiple Kubernetes deployments in one platform.
A route in the HAProxy backend context is a HAProxy configuration consisting of a backend switching rule and a backend. A backend switching rule is a rule within a frontend. In the frontend we define which domain should be routed to which backend. And a backend is a definition of one or multiple endpoints to route the traffic to (target endpoint).
a9s-router BOSH Release has multiple jobs:
haproxy- starts HAProxy with Data Plane API
a9s-route-registry- starts the a9s Route Registry: collects, maintains routes and syncs routes to HAProxy
a9s-route-registrar- starts the a9s Route Registrar: announces periodically routes to the a9s Route Registry
Every worker node has an a9s Route Registrar process that announces its route every
a9s-route-registrar.registry.publish_route_interval (default 10s), it connects to the
a9s Route Registry using the a9s Node Guard. At this point, the connection is TLS encrypted.
The a9s Node Guard forwards the connection to the a9s Route Registry, which creates an entry for the route in its internal registry. This registry is kept in memory only.
a9s-route-registry.update_interval (default 10s), the a9s Route Registry synchronizes
the internal state of its registry with all HAProxy instances using the Data Plane API. This
connection is also TLS encrypted. The Data Plane API synchronizes with the HAProxy process
ha_proxy.dataplaneapi.reload_delay: (default 5s).
Check here to know more about it.
If a node fails to send heartbeats after
a9s-route-registry.update_interval + a9s-route-registry.ttl_interval (default: 10s),
the a9s Route Registry will understand that this node is gone, and it will trigger the removal
of the route to the failing node.
The a9s Route Registry uses again the Data Plane API to request the HAProxy to remove the route of the shutdown or failing node. If all nodes of the deployment fails to send the heartbeats (e.g.: Deployment deleted), the whole configuration for the target instance is removed from HAProxy.
When a request reaches the HAProxy it discovers the target by matching the domain with the route, and fowarding the request to the target. Any target can be specified.
We recommend a load balancer in front of th HAProxy cluster. The request is routed to a HAProxy node, that matches the domain of the request with the configured route, after that the connection is routed to the correct target (address and port). To know more about how a9s Kubernetes handles this connection, check the a9s Kubernetes documentation.
- If for any reason a route is configured on the HAProxy, but not on the a9s Router Registry, this route will not be removed by the a9s Router Registry. This should be fixed on latter versions.