Run Apache, Nginx & HAProxy on Same Server (Debian, Ubuntu, CentOS)

January 22, 2020

If you’re a server admin, you probably have a web server of your choice like Apache, or Nginx. Apache is a well-known web server since the 1990s. Nginx was first developed in 2004 and quickly gained traction due to its lightweight memory footprint and fast processing speed for static HTML files.

Both Apache and Nginx support virtual hosting, which means you can host multiple websites or web applications on the same server. However, you will encounter situations in which you have an existing web server running, but a particular web application requires using a different web server. Port 80 or 443 on the public IP address can be used by only one process. If Apache is using the port, then Nginx can’t use (or bind to) it. So what can you do?

You can configure Nginx as a reverse proxy to Apache, so Nginx can redirect HTTP requests to Apache. In my experience, I found that this isn’t always the best way because it has once caused weird problems that I cannot troubleshoot. Instead, I prefer to use HAProxy as a reverse proxy for both Nginx and Apache. HAProxy is a free, open-source high availability load balancer and proxy server for TCP and HTTP-based applications.

Run Apache, Nginx and HAProxy on the Same Server

Here’s how it works.

  • Nginx listens on 127.0.0.1:80 and 127.0.0.1:443
  • Apache listens on 127.0.0.2:80 and 127.0.0.2:443
  • HAProxy listens on port 80 and 443 of the public IP address. It redirects HTTP request on port 80 to port 443. When a request arrives on port 443, it will choose between Nginx and Apache back end by analyzing the SNI (server name indication) header in the HTTPS request.

Run Apache, Nginx & HAProxy on Same Server (Debian, Ubuntu, CentOS)

Actually, Cloudflare (a CDN provider) is also using the SNI header to determine how to route HTTPS requests to origin servers.

Step 1: Stop Nginx and Apache

To stop Nginx on Debian, Ubuntu and CentOS, run

sudo systemctl stop nginx

To stop Apache on Debian/Ubuntu, run

sudo systemctl stop apache2

To stop Apache on CentOS, run

sudo systemctl stop httpd

Step 2: Change the Listen Port in Nginx

We need to make Nginx listen on 127.0.0.1:80. Open your Nginx configuration files in /etc/nginx/conf.d/ or /etc/nginx/sites-enabled/ and find the following line.

listen 80;

Change it to

listen 127.0.0.1:80;

If https is enabled on the Nginx server block, then also find

listen 443 ssl;

And change it to

listen 127.0.0.1:443 ssl;

The Nginx main configuration file /etc/nginx/nginx.conf might include a default virtual host listening on port 80 or 443, so you might need to edit this file too.

Restart Nginx for the changes to take effect.

sudo systemctl restart nginx

Step 3: Change the Listen Port in Apache

We need to make Apache listen on 127.0.0.2:80.

Debian/Ubuntu

On Debian and Ubuntu, edit the /etc/apache2/ports.conf file.

sudo nano /etc/apache2/ports.conf

Change

Listen 80
Listen 443

To

Listen 127.0.0.2:80
Listen 127.0.0.2:443

Save and close the file. Also go to /etc/apache2/sites-enabled/ directory, edit the virtual host files. Change

<VirtualHost *:80>

To

<VirtualHost 127.0.0.2:80>

If there are SSL virtual host, then also change

<VirtualHost *:443>

To

<VirtualHost 127.0.0.2:443>

Restart Apache.

sudo systemctl restart apache2

CentOS

On CentOS, edit the /etc/httpd/conf/httpd.conf file.

sudo nano /etc/httpd/conf/httpd.conf

Find

Listen 80

Change it to

Listen 127.0.0.2:80

Save and close the file. Then go to /etc/httpd/conf.d/ directory, edit the virtual host files. Change

<VirtualHost *:80>

To

<VirtualHost 127.0.0.2:80>

If there are SSL virtual host, then also change

<VirtualHost *:443>

To

<VirtualHost 127.0.0.2:443>

In the /etc/httpd/conf.d/ssl.conf file, there’s

Listen 443 https

Change it to:

Listen 127.0.0.2:443 https

Save and close the file. Restart Apache for the changes to take effect.

sudo systemctl restart httpd

Step 4: Configure HAProxy

Install HAProxy on your distro.

Debian/Ubuntu

sudo apt install haproxy

CentOS

sudo dnf install haproxy

Edit HAProxy configuration file.

sudo nano /etc/haproxy/haproxy.cfg

Add the following code snippet at the end of the file, which will make HAPorxy listen on port 80 of the public IP address and redirect HTTP requests on port 80 to port 443. Replace 12.34.56.78 with your server’s public IP address.

frontend http
    bind 12.34.56.78:80
    mode http
    redirect scheme https code 301

Now we also need to add an HTTPS front end.

frontend https
    bind 12.34.56.78:443
    mode tcp
    tcp-request inspect-delay 5s
    tcp-request content accept if { req_ssl_hello_type 1 }

Then define the Nginx and Apache back ends.

backend nginx
    mode tcp
    option ssl-hello-chk
    server nginx 127.0.0.1:443 check
backend apache
    mode tcp
    option ssl-hello-chk
    server apache 127.0.0.2:443 check

You can define a default back end with:

default_backend nginx

We will use SNI header in the HTTPS request to redirect to the correct back end. For example, if Nginx is serving domain1.com and Apache is serving domain2.com, then you add the following two lines.

use_backend nginx if { req_ssl_sni -i domain1.com }
use_backend apache if { req_ssl_sni -i domain2.com }

Note that the default_backend and use_backend directives should be placed above the backend definitions.

In the configuration above, we utilized the SNI (Server Name Indication) feature in TLS to differentiate HTTPS traffic.

  • When domain1.com is in the TLS Client Hello, HAProxy redirect traffic to the nginx backend.
  • When domain2.com is in the TLS Client Hello, HAProxy redirect traffic to the apache backend.

If the client doesn’t specify the server name in TLS Client Hello, then HAproxy will use the default backend (nginx).

Save and close the file. Then restart HAproxy.

sudo systemctl restart haproxy

Now Apache, Nginx and HAProxy are able to run on the same server.

How to Forward Client’s IP address to Backend

By default, Apache and Nginx can only see HAProxy’s IP address. To get client’s real IP address, make sure you added the send-proxy-v2 option in the HAProxy’s back end definition like below.

server nginx 127.0.0.1:443 send-proxy-v2 check
server apache 127.0.0.2:443 send-proxy-v2 check

We also need to add some configuration in Nginx and Apache.

Nginx

Add proxy_protocol in the Nginx listen directive like below.

listen 127.0.0.2:443 ssl http2 proxy_protocol;

Then add the following two directives in the Nginx http { } block.

set_real_ip_from 127.0.0.1;
real_ip_header proxy_protocol;

Save and close the file. Then reload Nginx.

sudo systemctl reload nginx

Apache

If you use Apache on Debian/Ubuntu, you need to enable the remoteip module. (This module is enabled on CentOS by default.)

sudo a2enmod remoteip

Then add the following 3 lines in your Apache virtual host configuration file.

RemoteIPProxyProtocol On
RemoteIPHeader X-Forwarded-For
RemoteIPTrustedProxy 127.0.0.1

Like this.

<VirtualHost 127.0.0.2:443>
    ServerName www.example.com
    RemoteIPProxyProtocol On
    RemoteIPHeader X-Forwarded-For
    RemoteIPTrustedProxy 127.0.0.1

Save and close the file. Then we also need to change the combined log format. Edit the Apache main configuration file.

sudo nano /etc/apache2/apache2.conf

or

sudo nano /etc/httpd/conf/httpd.conf

Find the following line.

LogFormat "%h %l %u %t "%r"" %>s %O ""%{Referer}i"" ""%{User-Agent}i"""" combined