Load balancing the load balancer (dockerized?)

[markschmid]

Hey

I'm struggling getting good information on this, so I'm trying to at least get your opinion (or even some advice):

It seems pretty standard to load balance containerized services, aka to "put an nginx/haproxy/etc in front of your docker (swarm)". Yet, in order to achieve high availability on the load balancing layer as well, there might be the requirement to run multiple instances of that reverse proxy / load balancer.

In my case, I'm lacking the option to make use of ready-made cloud loadbalancer-instances (e.g. AWS). Given this, I can run my nginx/haproxy/etc:
1 "natively" (as their own nginx/haproxy/etc process) on their respective compute instance, or
2 dockerized (non-swarm-mode) on their respective compute instance, or
3 dockerized in a swarm with at least 3 manager nodes, or
4 yet another option

Are there reasons not to run the load balancers dockerized (e.g. in a dedicated swarm for the loadbalancing layer) as well?
If docker, would you run those loadbalancer services rather in the same swarm with the load balanced services or in a separate swarm which is 1 tier "higher up" in the service design?

I hope you can make sense of my question, thanks in advance.
Mark

[BretFisher]

Hey Mark,

Let me make sure I understand your requirements:

1. You can't use a cloud-provided LB service like AWS ELB
2. You want to run multiple published Swarm services on the same external ports (80/443)
3. You want everything highly available (multiple swarm nodes for incoming traffic, multiple replicas of each service across those nodes)

Assuming that's true, you'll have two layers that need load balancing/proxying:

1. External traffic to multiple nodes
   This would be where a AWS ELB or load balancer service by the cloud provider would be great. Those services are highly available and solve the problem of "one IP for many servers". The only other option here that is highly available, is that if you are only serving HTTP to browsers, you can use multiple DNS A records (one for each Swarm node you want to receive incoming traffic) for each DNS name hosted by the Swarm, and all browsers are designed to fail over to a new IP if one stops responding (it's surprising how well it works for for simple websites). YMMV.
2. Requests for various websites on the same swarm on the same published ports
   This requires a HTTP Layer 7 reverse proxy and it's normal for a container solution to have them. Traefik and Docker Flow Proxy are two good ones that listen to the Swarm API and auto-update for new services and DNS names you want to route around inside the Swarm