In Managed IT
Load balancing refers to the efficient distribution of incoming network traffic among a group of backend servers
Management
What is load balancing?
The task of a load balancer, on the other hand, is to distribute the load among several installed backend servers.
There are several types of load balancers:
- An application load balancer.
- A network load balancer.
- A gateway load balancer.
- A classic load balancer.
There are also many load balancers themselves and they all have different uses:
- Haproxy
- Nginx
- mod_athena
- Varnish
- Balance
- Linux Virtual Server (LVS)
In this article we will take a look at Nginx. Read also about advantages and disadvantages of the most popular hosting types.
What is Nginx load balancing?
Nginx is a high-performance web server that can also be used as a load balancer - it is the process of distributing web traffic across multiple servers using Nginx.
It ensures that no server is overloaded and that all requests are handled in a timely manner. Nginx uses various algorithms to determine optimal traffic distribution and can also be configured to provide fault tolerance in case one of the servers fails.
You can use either Nginx open source or Nginx Plus to load balance HTTP traffic across a group of servers.
Personally, I use Nginx open source to set up my load balancers and that is what I am going to show you in this article.
Benefits of load balancing
Load balancing helps to scale the application by dealing with traffic spikes without increasing cloud costs. It also helps eliminate the single point of failure problem. Since the load is distributed, if one of the servers fails - the service will still continue to operate.
Configuring Nginx
Installing Nginx
The first step is to install Nginx. It can be installed on Debian, Ubuntu or CentOS. I am going to use Ubuntu, which I have already configured on my virtual server.
sudo apt-get update
sudo apt-get install nginx
Configure Nginx as a load balancer
Create a new configuration file for the load balancer:
cd /etc/nginx/sites-available/
sudo nano default
http {
upstream app{
server 10.2.0.100;
server 10.2.0.101;
server 10.2.0.102;
}
server {
listen 80;
server_name mydomain.com;
location / {
include proxy_params;
proxy_pass http://app;
proxy_redirect off;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}
}
In the file, you need to define upstream and server directive. Upstream defines where Nginx will forward requests once they are received. It contains IP addresses of a group of servers (backend) to which requests may be sent depending on the load balancing method chosen. By default, Nginx uses the round-robin balancing method to distribute power among servers.
The server segment defines port 80, through which Nginx will receive requests. It also contains the proxy_pass variable.
The proxy_pass variable is used to tell NGINX where to send the received traffic. In this case, the proxy_pass variable points to 3 servers. This allows NGINX to direct received traffic to any of the upstream servers' IP addresses. Nginx acts as both a reverse proxy and a load balancer.
A reverse proxy is a server that resides between backend servers and intercepts requests from clients.
Selecting a load balancing method
The next step is to determine the load balancing method. There are several methods that we can use. These include:
Round Robin
Round Robin is a load balancing method where each server in the cluster is given an equal opportunity to process requests. This method is often used in web servers where each server request is evenly distributed across servers.
The power is allocated in turn, which means that each server will have a different amount of time to complete the request. For example, if you have three upstream servers, A, B and C, the load balancer will first distribute the load to A, then B and finally C before distributing the load to A. This method is fairly straightforward but has some limitations.
One limitation is that some servers will be idle simply because they will be waiting their turn. In our example, if A receives a job and completes it in a second, this will mean that it will be idle until the next job. By default, it is the round robin method that Nginx uses to distribute the load between servers.
Round Robin with added weight
To solve the problem of server downtime, we can use server weights to tell Nginx which servers should have the highest priority. Weighted Round Robin is one of the most popular load balancing methods used today.
This method involves assigning a weight to each server and then distributing traffic between servers based on these weights. This ensures that the servers with more bandwidth receive more traffic and helps prevent any of the servers from becoming overloaded.
This method is often used in conjunction with other methods, such as Session Persistence, to ensure equal capacity allocation to all servers. The application server with the highest weighting parameter will have priority (more traffic) over the server with the lowest number (weighting).
In order to enable server weighting parameters, it is necessary to update the Nginx configuration:
http {
upstream app{
server 10.2.0.100 weight=5;
server 10.2.0.101 weight=3;
server 10.2.0.102 weight=1;
}
server {
listen 80;
location / {
proxy_pass http://app;
}
}
}
Least Connection
The Least Connection method is a popular technique used to distribute the workload evenly among multiple servers. It works by routing each new connection request to the server with the fewest active connections. This ensures that all servers are used equally and none are overloaded.
http {
upstream app{
least_conn;
server 10.2.0.100;
server 10.2.0.101;
server 10.2.0.102;
}
server {
listen 80;
location / {
proxy_pass http://app;
}
}
}
Least Connection with Weighting
This method is used to distribute the workload among multiple compute resources (such as servers) to optimize performance and minimize response times. This method takes into account the number of active connections on each server and assigns the appropriate weights. The goal is to distribute the workload in such a way as to balance the load and minimise the response time.
http {
upstream app{
least_conn;
server 10.2.0.100 weight=5;
server 10.2.0.101 weight=4;
server 10.2.0.102 weight=1;
}
server {
listen 80;
location / {
proxy_pass http://app;
}
}
}
IP Hash
The IP Hash balancing method uses a hashing algorithm to determine which server should receive each of the incoming packets. This is useful when there are multiple servers behind the same IP address and you want to make sure that every packet from the client IP address is routed to the same server. It takes the source IP address and destination IP address and creates a unique hash key. This is then used to distribute the client to certain servers.
This is a very important point in the case of a canary deployment. This allows us developers to release changes to a select portion of the users so that they can test and provide feedback before they are sent to the release.
The advantage of this approach is that it can provide better performance than other methods such as round-robin.
http {
upstream app{
ip_hash;
server 10.2.0.100;
server 10.2.0.101;
server 10.2.0.102;
}
server {
listen 80;
location / {
proxy_pass http://app;
}
}
}
URL Hash
URL Hash load balancing also uses a hashing algorithm to determine which server will receive each of the requests based on the URL.
It's also similar to the IP Hash balancing method, but the difference is that we're hashing specific URLs rather than IPs. This ensures that all requests are evenly distributed across servers, providing better performance and reliability.
Restart Nginx
Once you have configured the load balancer with the load balancing method you want, you can restart Nginx for the changes to take effect.
sudo systemctl restart nginx
Proxying HTTP traffic to a server group
A proxy server is a server that mediates between a client and another server. A proxy server can be used to provide clients with access to a server group, e.g. a group of web servers, by forwarding requests from the client to the appropriate server.
A proxy server can also provide caching to improve performance by reducing the need to send requests to the server for each page view. Nginx can act as either a forward or reverse proxy server. The difference between a forward proxy and a reverse proxy is that the forward proxy is in front of multiple clients and the reverse proxy is in front of multiple backends.
The forward proxy protects client identity data, while the reverse proxy protects backend server data.
Load balancing with HTTPS enabled
Load balancing with HTTPS enabled is a great way to improve the performance of your site. Using a load balancer, you can distribute the workload of your site across multiple servers, which helps reduce the resource consumption of each individual server.
Also, by enabling HTTPS you can ensure that all communications between your site's servers and visitors are encrypted, which helps provide additional security for your site.
However, it also has a number of definite problems. One of them is that you have to do a TLS handshake and DNS lookup for each request, which can lead to delayed responses.
I prefer to have my backend application servers communicate with the load balancer over HTTP (SSL termination) and the load balancer communicate with clients over HTTPS. This way, I don't have to consider TLS handshakes and DNS lookups when routing requests to application servers.
Performance test
Load balancers are often used to distribute incoming traffic among a group of servers to avoid overloading any single server. Health checks are an important part of ensuring that traffic is properly routed by the load balancer as it helps to identify when a server is down or not working properly.
Nginx can monitor the health of our HTTP servers in the upstream group. It can also perform passive health checks and active health checks.
Passive health checks
Nginx analyses operations as they occur and attempts to recover failed connections for passive health checks.
It identifies the server as unreachable and temporarily stops sending requests to it until it is deemed active again if the operation still cannot be resumed.
To mark a upstream server as unreachable we need to define two parameters in the upstream directive: failed_timeout and max_fails.
failed_timeout specifies the time during which a certain number of failed attempts must occur for a server to be flagged as unreachable.
max_fails specifies the number of failed attempts.
upstream app{
server 10.2.0.100 max_fails=3 fail_timeout=60s;
server 10.2.0.101;
server 10.2.0.102;
}
Active health checks
Nginx can periodically check the health of upstream servers by sending special health check requests to each server and checking that the response is correct.
server {
location / {
proxy_pass http://app;
health_check;
}
}
Unfortunately, we haven't used active health checks because they seem to only apply to Nginx Plus. You can read more about active health checks here.
Self-Managed Load Balancer vs. Managed Service
There are two main types of load balancers: self-managed and managed. Self-managed load balancers are usually cheaper but require more technical knowledge to set up and maintain. Managed load balancers are more expensive but often have more features and support.
If you want to sleep peacefully at night, managed load balancers are much better, as they take the headache of constant debugging away.
Limiting the number of connections
One way to improve the performance of a load balancer is to limit the number of connections it tries to make to each server. This will help prevent the load balancer from overloading one of the servers, and ensure that the traffic is evenly distributed across all of them.
Are reverse proxy and load balancer similar?
Yes, reverse proxy and load balancer are similar in many ways. They can both be used to improve the performance and availability of a website or application and also to distribute traffic between multiple servers.
Conclusion
Load balancing is a good way of distributing requests across application instances. It ensures high availability and ensures that the server is always up and running. Using Nginx load balancer for load balancing is also beneficial because it serves as both a reverse proxy and a load balancer. It is also open source, so you can always get exactly what you need from Nginx. I hope this article has helped you set up the Nginx load balancer.