I Can HA(z)Proxy Pump?

on

pump.io is selfish. It's intended to run on a standard web port, but it doesn't want to share that port with other services. This poses a problem to people who, like me, want to host their own pump.io server, but don't want to dedicate an entire host to it. So let's proxy it.

Disclaimer

The pump.io documentation advises against running pump.io behind a proxy, as doing so may cause the service to perform at a degraded level. The method presented seems to work okay for me, but your mileage may vary.

Also, keep in mind that if you already run a pump.io server, changing your server port may (will) break your existing OAuth relationships.

Why proxy?

If you're not already running services on standard web ports on your host, and you don't plan to do so in the future, then you don't need to worry about setting up a proxy in front of pump.io. Just follow the documentation to run pump.io directly on port 80 or 443. I strongly recommend running on port 443 with SSL enabled. I like Namecheap for cheap SSL certificates and StartSSL for free SSL certificates. Neither of these would be a good choice for an e-commerce site, but they're not bad for basic personal sites.

If you are already running services on standard web ports, then you won't be able to run pump.io on those ports. This is a common issue for web-facing software, and there are generally a couple of common solutions to this issue. One common solution is to configure a webserver to route certain incoming requests on a standard web port to an application handler via FastCGI or similar protocol. This cannot (currently?) be done with pump.io

Another common method of running web-facing services on a machine that already runs webservers is to proxy certain incoming requests on standard web ports to a service running on a separate port. This can currently be done with pump.io ... sort of. The pump.io documentation suggests that proxying pump.io behind a web server is going to make WebSockets "work less well." My webserver of choice, lighttpd, doesn't even have support for WebSockets. Other webservers such as nginx may support proxying WebSockets; in fact, there are articles describing how to configure pump.io behing nginx, such as this one. But this article describes a different solution: running pump.io and a separate webserver behind HAProxy.

The Case for HAProxy

HAProxy is a high-availability load balancer and proxy for TCP and HTTP connections. In a common scenario, a high-traffic site would run HAProxy on a web-facing machine and distribute the traffic to a number of separate hosts. That's all overkill for our application. But HAProxy also provides TCP and HTTP proxying which makes it an ideal WebSockets proxy.

Architecture

The proxy configuration makes use of HAProxy, lighttpd, and pump.io. A separate, dedicated subdomain (pump2.thisshitistemp.com) is created to access the pump.io service.

HAProxy sits in front of everything. Incoming requests on port 80 and 443 are received by HAProxy and routed to the appropriate destination based on simple configuration directives.

Within HAProxy, HTTP connections on port 80 are forwarded directly to lighttpd. However, since port 80 is not in use by HAProxy, lighttpd is modified to listen for HTTP requests on port 8080. Similarly, lighttpd is modified to listen for HTTPS requests on port 8443. But connections received by HAProxy on port 443 aren't immediately forwarded to lighttpd. First, HAProxy reads the destination hostname from the Server Name Indication (SNI). If the destination matches the pump.io subdomain, then HAProxy forwards that request directly to the pump.io service running on port 31337. Otherwise, the request is forwarded to lighttpd on port 8443.


           +-----------+        all HTTP            +------------+
---->  :80 | --------- | ------------------>  :8080 |            |
           |           |                            |            |
           |  HAProxy  |                            |  lighttpd  |
           |           |     other HTTPS            |            |
----> :443 | ---SNI--- | ------------------>  :8443 |            |
           +-----------+                            +------------+
                 |           
                 |                                  +------------+
                 |                                  |            |
                 |                                  |            |
                 |                                  |  pump.io   |
                 | pump2.thisshitistemp.com         |            |
                 +-------------------------> :31337 |            |
                                                    +------------+

Host Configuration

pump.io

Here are the relevant bits of the pump.io config file.

pump.io.json:

{
  "hostname": "pump2.thisshitistemp.com",
  "address": "127.0.0.1",
  "port": 31337,
  "urlPort": 443,
  "noweb": false,
  "site": "pump2.thisshitistemp.com",
}

HAProxy

And here is the HAProxy config file.

global
    log         127.0.0.1 local2

    chroot      /usr/share/haproxy
    pidfile     /run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon

defaults
    log                     global
    option                  httplog
    option                  dontlognull
    option http-server-close
    option forwardfor       except 127.0.0.0/8
    option                  redispatch
    retries                 3
    timeout http-request    10s
    timeout queue           1m
    timeout connect         10s
    timeout client          1m
    timeout server          1m
    timeout http-keep-alive 10s
    timeout check           10s
    maxconn                 3000

frontend http-in *:80
    mode http
    default_backend servers

frontend https-in *:443
    mode tcp
    option tcplog
    option socket-stats
    option ssl-hello-chk

    tcp-request inspect-delay 5s
    tcp-request content accept if { req_ssl_hello_type 1 }

    use_backend pumpio-ssl if { req_ssl_sni -i pump2.thisshitistemp.com }
    default_backend servers-ssl

backend servers
    mode http
    option httpclose
    option forwardfor
    cookie JSESSIONID prefix

    server server1 127.0.0.1:8000 cookie A check

backend servers-ssl
    server server-ssl1 127.0.0.1:8001 check

backend pumpio-ssl
    use-server pump if { req_ssl_sni -i pump2.thisshitistemp.com }
    server pump 127.0.0.1:31337 weight 0