At Altenar, we often use RabbitMQ as an entry point for our products. In this specific case, we will discuss a system that calculates sports data and provides it to other systems within the organization as well as outside the perimeter. When designing the system, the RabbitMQ component had the following requirements:

The endpoint is secured with an SSL certificate
RabbitMQ is hosted in Kubernetes. We use the RabbitMQ cluster operator and RabbitMQ messaging topology operator

For this project, we are using Google Cloud. Like Let’s Encrypt, you can use a Google Managed SSL certificate for any public HTTP load balancer in Google Cloud. However, it has several limitations, with the key ones being:

It only works for HTTP Load balancers.
It only works for public Load balancers.

According to RabbitMQ documentation, there are two common approaches for client connections:

Configure RabbitMQ to handle TLS connections
Use a proxy or load balancer (such as HAproxy) to perform TLS termination of client connections and use plain TCP connections to RabbitMQ nodes.

Our idea was to use a Google Managed certificate for TCP/SSL proxy load balancer in Google Cloud. However, due to the aforementioned limitations, a managed certificate is not supported for TCP load balancers. On the other hand, Google allows you to use an SSL certificate for several load balancers, which we decided to explore. Overall the blueprint would look like this:

We wanted to have a default dummy HTTP service that we would expose over port 443 and use for the sake of managing the SSL certificate and automatic renewals.
We would have a separate endpoint reusing the same IP address and SSL certificate.

Let’s examine each part of the solution in detail.

GKE Cluster

As the main hosting platform, we use GKE. We use the Google Terraform module for the private GKE cluster, configure Workload Identity and install an AutoNEG controller to the cluster.

In this article, we won’t delve into the specifics of how Workload Identity works with AutoNEG, as they are used as described in the official documentation.

With all preparations in place, we deploy a dummy “Hello World” service to the cluster and attach it to the Google Load balancer via AutoNEG. Below is the YAML configuration for that:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: gcp-tls-certificate-issuer
  labels:
    app: gcp-tls-certificate-issuer
  annotations:
    deployment.kubernetes.io/revision: '1'
spec:
  replicas: 2
  selector:
    matchLabels:
      app: gcp-tls-certificate-issuer
  template:
    metadata:
      labels:
        app: gcp-tls-certificate-issuer
    spec:
      containers:
        - name: ok
          image: assemblyline/ok:latest
          ports:
            - containerPort: 8888
              protocol: TCP
          imagePullPolicy: Always
          securityContext:
            capabilities:
              drop:
                - ALL
            runAsUser: 1000
            runAsGroup: 3000
            runAsNonRoot: true
            readOnlyRootFilesystem: true
      restartPolicy: Always
      terminationGracePeriodSeconds: 30
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 25%
      maxSurge: 25%
  revisionHistoryLimit: 10
  progressDeadlineSeconds: 600
---
apiVersion: v1
kind: Service
metadata:
  name: gcp-tls-certificate-issuer
  labels:
    app: gcp-tls-certificate-issuer
  annotations:
    cloud.google.com/neg: '{"exposed_ports": {"8888":{"name": "gcp-tls-certificate-issuer"}}}'
    controller.autoneg.dev/neg: '{"backend_services":{"8888":[{"name":"envcode-rabbit-https-backend-service","max_connections_per_endpoint":10000}]}}'
spec:
  ports:
    - name: http
      protocol: TCP
      port: 8888
      targetPort: 8888
  selector:
    app: gcp-tls-certificate-issuer
  clusterIP: 10.10.12.130
  clusterIPs:
    - 10.10.12.130
  type: ClusterIP

Notice the service's annotation. This is how AutoNEG adds it to the Load balancer as a backend.

Google load balancer

The next part is managed outside GKE and created separately. Google Load balancer is not a single object but rather a set of different objects combined. Below is the Terraform code with comments:

resource "google_compute_managed_ssl_certificate" "rabbitmq" {
  project = var.project
  name    = "${var.environment_name}-google-managed-certificate-rabbitmq"

  managed {
    domains = ["rabitmq.example.com."]  # Replace with your domain
  }
}

# reserved IP address
resource "google_compute_global_address" "default" {
  project = var.project
  name    = "tcp-proxy-xlb-ip"
}

output "rabbitmq-ip" {
  value = google_compute_global_address.default.address
}

# forwarding rule for TCP Loadbalanser
resource "google_compute_global_forwarding_rule" "default" {
  project               = var.project
  name                  = "${var.environment_name}-tcp-global-loadbalancer"
  provider              = google
  ip_protocol           = "TCP"
  load_balancing_scheme = "EXTERNAL"
  port_range            = "5671"
  target                = google_compute_target_ssl_proxy.default.id
  ip_address            = google_compute_global_address.default.id
}
# https://cloud.google.com/load-balancing/docs/ssl
# When you use Google-managed SSL certificates with SSL Proxy Load Balancing, the frontend port for traffic must be 443 to enable the Google-managed SSL certificates to be provisioned and renewed.

# forwarding rule for HTTPS Loadbalanser
resource "google_compute_global_forwarding_rule" "https" {
  project               = var.project
  name                  = "${var.environment_name}-https-global-loadbalancer"
  provider              = google
  ip_protocol           = "TCP"
  load_balancing_scheme = "EXTERNAL"
  port_range            = "443"
  target                = google_compute_target_ssl_proxy.https.id
  ip_address            = google_compute_global_address.default.id
}

resource "google_compute_target_ssl_proxy" "default" {
  project          = var.project
  name             = "${var.environment_name}-global-loadbalancer-tcp-proxy"
  backend_service  = google_compute_backend_service.default.id
  ssl_certificates = [google_compute_managed_ssl_certificate.rabbitmq.id]
}

resource "google_compute_target_ssl_proxy" "https" {
  project          = var.project
  name             = "${var.environment_name}-global-loadbalancer-https-proxy"
  backend_service  = google_compute_backend_service.https.id
  ssl_certificates = [google_compute_managed_ssl_certificate.rabbitmq.id]
}

# backend service For RabbitMQ Autoneg
resource "google_compute_backend_service" "default" {
  project = var.project
  name    = "${var.environment_name}-tcp-backend-service"
  protocol              = "TCP"
  port_name             = "tcp"
  load_balancing_scheme = "EXTERNAL"
  timeout_sec           = 10
  health_checks         = [google_compute_health_check.default.id]
  session_affinity      = "CLIENT_IP"

  # We don't want TF to remove whatever was configured by AutoNEG
  lifecycle {
    ignore_changes = [backend]
  }
}

# backend service For HTTPS Autoneg
resource "google_compute_backend_service" "https" {
  project               = var.project

                          #that's what you use in the service annotations
  name                  = "${var.environment_name}-https-backend-service"  
  protocol              = "TCP"
  port_name             = "tcp"
  load_balancing_scheme = "EXTERNAL"
  timeout_sec           = 10
  health_checks         = [google_compute_health_check.https.id]

  # We don't want TF to remove whatever was configured by AutoNEG
  lifecycle {
    ignore_changes = [backend]
  }
}

resource "google_compute_health_check" "default" {
  project            = var.project
  name               = "tcp-proxy-health-check"
  description        = "Backend service for AutoNEG"
  timeout_sec        = 1
  check_interval_sec = 1

  tcp_health_check {
    port = "5672" #use container port
  }
}

resource "google_compute_health_check" "https" {
  project            = var.project
  name               = "https-proxy-health-check"
  description        = "Backend service for AutoNEG"
  timeout_sec        = 1
  check_interval_sec = 1

  tcp_health_check {
    port = "8888" #use container port
  }
}

]

As you can see, we created one IP address and one SSL certificate and then used them in two forwarding rules. This allowed us to have a managed SSL certificate used for the TCP Load Balancer.

Don’t forget to configure DNS and point the IP to the right hostname for the whole thing to work.

Tip: GKE has thel7-default-backend deployment. Perhaps it would’ve been enough just creating a service with AutoNEG annotations and pointing it to that deployment’s pods. Try that and let me know in the comments if it works.

Managed SSL for TCP Loadbalancer in GKE