How to Setup VPN & Reverse Proxy for Containers in Kubernetes?

  docker, docker-compose, kubernetes, nginx, proxy

So, I have a very specific application I’m trying to migrate from docker-compose to kubernetes. Everything I’m about to describe works perfectly in docker-compose, it’s just how to replicate it in kubernetes that’s throwing me off.

The Goal

I have an application, let’s call it "my-app". my-app needs to route all of its requests over a VPN for security reasons. However, I want my-app to be able to receive instructions over the "normal" network, ie sent to its regular public IP Address/Hostname.

So I setup docker-compose. I created the VPN container, and routed all of my-app’s traffic through it. I then created an Nginx reverse-proxy and linked it directly to my-app.

The result is that I can use my-app’s API to send commands to it, and then my-app performs the actions it needs to over the VPN.

I have proper security on my-app, so I’m not worried about malicious attacks. The reason for this setup is I want to have my-app continue to receive commands at its current host address for ease-of-use, the added VPN security is only needed for the requests my-app does behind the scenes.

Docker Compose Setup

Here is a stripped-down example of the docker-compose configuration:

version: "3"
services:
    vpn:
        image: vpn-image
        container_name: vpn
        # More VPN configuration

    my-app:
        image: my-app-image
        container_name: my-app
        network_mode: service:vpn
        # More my-app configuration

    my-proxy:
        image: my-proxy-image
        container_name: my-proxy
        links:
            - vpn:my-app
        ports:
            - 8080:80
        # More my-proxy configuration

Where I’m At

So I’ve got my-app and my-proxy running in two separate Kubernetes pods, and working perfectly. I also have the vpn container setup as a "sidecar" in the same pod as my-app.

As long as I only deploy my-app and my-proxy, everything works fine. The moment I include the vpn, the proxy is no longer able to route to my-app.

I know this is because the VPN is changing how my-app handles all requests. I can shell into the pod and validate that the VPN is working, that network connectivity exists, etc.

Here is a stripped-down version of my kubernetes config for the deployment containing both the VPN and my-app:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app-vpn
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-app-vpn
  template:
    metadata:
      labels:
        app: my-app-vpn
    spec:
      containers:
        - name: vpn
          image: vpn-image
          securityContext:
            capabilities:
              add:
                - NET_ADMIN
                - SYS_MODULE
          # More vpn configuration
        - name: my-app
          image: my-app-image
          ports:
            - containerPort: 30007
          # More my-app configuration
---
apiVersion: v1
kind: Service
metadata:
  name: my-app-vpn-service
spec:
  type: NodePort
  selector:
    app: my-app-vpn
  ports:
    - port: 30007
      targetPort: 30007
      nodePort: 30007
      protocol: TCP

my-proxy is just a very simple nginx reverse-proxy, one that I’ve already been using for a lot of apps. Here is the proxy directive for my-app:

location /my-app/ {
    proxy_set_header Host $http_host;
    rewrite ^/my-app/(.*)$ /$1 break;
    proxy_pass http://my-app-vpn-service:30007;
}

As I said, my-proxy perfectly redirects to my-app using the above config so long as the vpn container is commented out and not running.

What I Need

I don’t have enough understanding of Kubernetes networking to know what I’m doing wrong. I know that there has to be a way to setup a service of some kind that can bridge the gap here, do the equivalent of the docker "link", but I just don’t know how to do it.

Guidance is appreciated. Thank you.

Source: Docker Questions

LEAVE A COMMENT