Idea: Client side load balancing using metrics

[gedw99]

I am planning to implement Client side load balancing.

The way it works is very simple.. Each Application Server exports telemetry to a central telemetry server ( or cluster). The telemetry server aggregates the data and does real time PUSH to all connected clients. The client can use whatever logic they want to choose what server to connect to and when.

The advantage of this approach is that:

• the Load Balancer is no longer in the data plane for every request, and so it allow the system to scale out.
• by using Net Data you get automated metrics and alerts for everything you deploy anyway. They are real time metrics :)

Topology / Approach to do it

1. Setup Net Data Or use their free cloud offering ( which is free and unlimited )

https://www.netdata.cloud/pricing/

https://github.com/netdata/netdata

GUI DEMO: https://london.my-netdata.io/#after=-540;before=0;=undefined;theme=slate;help=true;utc=Europe%2FBerlin

2. Add the Netapp plugin to the Server so that it can export metrics to Net Data Server

https://github.com/netdata/netdata/blob/master/packaging/docker/README.md

3. Netapp exports the data in realtime to a Postgresql Timeseries DB.

This doc and code shows a simple example.
Doc: https://www.timescale.com/blog/writing-it-metrics-from-netdata-to-timescaledb/amp/
Code: https://github.com/mahlonsmith/netdata-timescale-relay

https://github.com/timescale/timescaledb

4. Postgresql Timeseries DB is exporting an aggregation to the Client over WS, SSE, WenTransport or other.

https://github.com/timescale/promscale gives you aggregation as a prometheus feed...

5. The client then implements whatever Load balancing logic it wants

Scale out / HA / Global load balancing of the metrics system itself.

You can scale out Postgresql Timeseries DB using the standard approaches for any Postgresql DB.
https://fly.io/blog/globally-distributed-postgres/
https://fly.io/docs/flyctl/postgres-db/

I am not saying that Fly.io has to be used, but just as an example. With fly you get Global Load Balancing for free and so then this proposed Client side load balancing would be globally load balanced. So clients can get their metrics feed from fly to connect to wherever the go between servers are located.

The other thing is that it is better to host your metrics on a different cloud or network than what your own Servers or Cloud is running on. Otherwise when things go down, you can't see that they have gone down.