A WebSocket server is easy to demonstrate and surprisingly difficult to operate. The handshake is rarely the problem. The real engineering begins when thousands of clients remain connected, one reader stops consuming, Redis disappears, credentials expire, or a deployment asks every node to shut down without leaking goroutines.

This tutorial builds a production-oriented Go service with a hard per-node connection limit, bounded outbound queues, slow-client eviction, authenticated handshakes, ping/pong heartbeats, graceful shutdown, health checks, metrics, and Redis-backed fan-out across replicas.

Architecture and delivery contract

Each process owns its local WebSocket connections. A client message is validated, stamped with the authenticated identity and publishing node, then published to Redis. Every process subscribes to the same channel and forwards each event to its local clients.

The design deliberately uses Redis Pub/Sub. It is fast and appropriate for ephemeral presence, dashboards, notifications, and live collaboration updates, but it is not durable. A disconnected subscriber can miss events. Applications requiring replay should use a durable log such as Redis Streams or another broker and introduce message IDs, acknowledgements, and retention.

Memory remains bounded in two places:

• MAX_CLIENTS limits connections accepted by one process.
• Every client receives a fixed-capacity outbound channel. If it fills, that client is disconnected instead of slowing the entire hub or consuming unbounded memory.

All nodes receive messages through Redis, including the originating node. That single path avoids duplicate local delivery and keeps ordering semantics consistent within the limits of Pub/Sub.

Prerequisites and project layout

You need Go 1.22 or newer, Docker with Compose for the deployment example, and OpenSSL for generating a secret. The application uses github.com/gorilla/websocket 1.5.3 and github.com/redis/go-redis/v9 9.7.0.

realtime/
├── cmd/
│   ├── check/main.go
│   └── token/main.go
├── Dockerfile
├── compose.yaml
├── go.mod
├── haproxy.cfg
└── main.go

Create go.mod:

module example.com/realtime

go 1.22.0

require (
	github.com/gorilla/websocket v1.5.3
	github.com/redis/go-redis/v9 v9.7.0
)

Implement the WebSocket service

The server maintains exactly one reader and one writer per connection, matching Gorilla WebSocket’s concurrency contract. Only the read pump parses client frames. Only the write pump sends ordinary frames and heartbeats.

package main

import (
	"context"
	"crypto/hmac"
	"crypto/sha256"
	"crypto/subtle"
	"encoding/base64"
	"encoding/json"
	"fmt"
	"log/slog"
	"net/http"
	"os"
	"os/signal"
	"strconv"
	"strings"
	"sync"
	"sync/atomic"
	"syscall"
	"time"

	"github.com/gorilla/websocket"
	"github.com/redis/go-redis/v9"
)

const (
	writeWait = 10 * time.Second
	pongWait  = 60 * time.Second
	pingEvery = 25 * time.Second
	maxFrame  = 4096
	queueSize = 64
)

type counters struct {
	connected, accepted, published, publishErrors, slowDisconnects atomic.Int64
}

type config struct {
	addr, redisURL, channel, node string
	secret                       []byte
	maxClients                   int
	origins                      map[string]struct{}
}

type app struct {
	ctx     context.Context
	cfg     config
	redis   *redis.Client
	hub     *hub
	slots   chan struct{}
	metrics *counters
	pumps   sync.WaitGroup
}

type hub struct {
	register, unregister chan *client
	inbound             chan []byte
	metrics             *counters
}

type client struct {
	app  *app
	conn *websocket.Conn
	send chan []byte
	user string
}

func env(name, fallback string) string {
	if value := os.Getenv(name); value != "" {
		return value
	}
	return fallback
}

func loadConfig() config {
	maxClients, err := strconv.Atoi(env("MAX_CLIENTS", "5000"))
	if err != nil || maxClients < 1 {
		panic("MAX_CLIENTS must be a positive integer")
	}
	secret := []byte(os.Getenv("AUTH_SECRET"))
	if len(secret) < 32 {
		panic("AUTH_SECRET must contain at least 32 characters")
	}
	origins := make(map[string]struct{})
	for _, origin := range strings.Split(env("ALLOWED_ORIGINS", ""), ",") {
		if origin = strings.TrimSpace(origin); origin != "" {
			origins[origin] = struct{}{}
		}
	}
	host, _ := os.Hostname()
	return config{
		addr:       env("LISTEN_ADDR", ":8080"),
		redisURL:   env("REDIS_URL", "redis://127.0.0.1:6379/0"),
		channel:    env("REDIS_CHANNEL", "ws:broadcast"),
		node:       env("NODE_ID", host),
		secret:     secret,
		maxClients: maxClients,
		origins:    origins,
	}
}

func (a *app) authenticate(r *http.Request) (string, error) {
	scheme, token, ok := strings.Cut(r.Header.Get("Authorization"), " ")
	if !ok || !strings.EqualFold(scheme, "Bearer") {
		return "", fmt.Errorf("missing bearer token")
	}
	parts := strings.Split(token, ".")
	if len(parts) != 3 {
		return "", fmt.Errorf("malformed token")
	}
	signingInput := parts[0] + "." + parts[1]
	supplied, err := base64.RawURLEncoding.DecodeString(parts[2])
	if err != nil {
		return "", fmt.Errorf("malformed signature")
	}
	mac := hmac.New(sha256.New, a.cfg.secret)
	mac.Write([]byte(signingInput))
	if subtle.ConstantTimeCompare(supplied, mac.Sum(nil)) != 1 {
		return "", fmt.Errorf("invalid signature")
	}
	subjectBytes, err := base64.RawURLEncoding.DecodeString(parts[0])
	if err != nil {
		return "", fmt.Errorf("malformed subject")
	}
	subject := string(subjectBytes)
	if subject == "" || len(subject) > 128 {
		return "", fmt.Errorf("invalid subject")
	}
	expiry, err := strconv.ParseInt(parts[1], 10, 64)
	if err != nil || time.Now().Unix() >= expiry {
		return "", fmt.Errorf("expired token")
	}
	return subject, nil
}

func (h *hub) run(ctx context.Context) {
	clients := make(map[*client]struct{})
	drop := func(c *client, slow bool) {
		if _, exists := clients[c]; !exists {
			return
		}
		delete(clients, c)
		close(c.send)
		c.conn.Close()
		h.metrics.connected.Add(-1)
		if slow {
			h.metrics.slowDisconnects.Add(1)
		}
	}

	for {
		select {
		case c := <-h.register:
			clients[c] = struct{}{}
			h.metrics.connected.Add(1)
		case c := <-h.unregister:
			drop(c, false)
		case message := <-h.inbound:
			for c := range clients {
				select {
				case c.send <- message:
				default:
					drop(c, true)
				}
			}
		case <-ctx.Done():
			for c := range clients {
				drop(c, false)
			}
			return
		}
	}
}

func (c *client) readPump() {
	defer c.conn.Close()
	c.conn.SetReadLimit(maxFrame)
	_ = c.conn.SetReadDeadline(time.Now().Add(pongWait))
	c.conn.SetPongHandler(func(string) error {
		return c.conn.SetReadDeadline(time.Now().Add(pongWait))
	})

	for {
		messageType, data, err := c.conn.ReadMessage()
		if err != nil {
			return
		}
		if messageType != websocket.TextMessage {
			continue
		}
		var input struct {
			Body string `json:"body"`
		}
		if json.Unmarshal(data, &input) != nil || input.Body == "" {
			continue
		}
		event := struct {
			Type, User, Body, Node, At string
		}{
			Type: "message", User: c.user, Body: input.Body,
			Node: c.app.cfg.node, At: time.Now().UTC().Format(time.RFC3339Nano),
		}
		payload, err := json.Marshal(event)
		if err != nil {
			return
		}

		ctx, cancel := context.WithTimeout(c.app.ctx, 1500*time.Millisecond)
		err = c.app.redis.Publish(ctx, c.app.cfg.channel, payload).Err()
		cancel()
		if err != nil {
			c.app.metrics.publishErrors.Add(1)
			slog.Error("publish failed", "user", c.user, "error", err)
			_ = c.conn.WriteControl(
				websocket.CloseMessage,
				websocket.FormatCloseMessage(1011, "broker unavailable"),
				time.Now().Add(writeWait),
			)
			return
		}
		c.app.metrics.published.Add(1)
	}
}

func (c *client) writePump() {
	ticker := time.NewTicker(pingEvery)
	defer ticker.Stop()

	for {
		select {
		case message, ok := <-c.send:
			_ = c.conn.SetWriteDeadline(time.Now().Add(writeWait))
			if !ok {
				_ = c.conn.WriteControl(
					websocket.CloseMessage,
					websocket.FormatCloseMessage(1001, "server closing"),
					time.Now().Add(writeWait),
				)
				return
			}
			if c.conn.WriteMessage(websocket.TextMessage, message) != nil {
				return
			}
		case <-ticker.C:
			_ = c.conn.SetWriteDeadline(time.Now().Add(writeWait))
			if c.conn.WriteMessage(websocket.PingMessage, nil) != nil {
				return
			}
		case <-c.app.ctx.Done():
			return
		}
	}
}

func (a *app) serveWS(w http.ResponseWriter, r *http.Request) {
	user, err := a.authenticate(r)
	if err != nil {
		http.Error(w, "unauthorized", http.StatusUnauthorized)
		return
	}
	select {
	case a.slots <- struct{}{}:
	default:
		w.Header().Set("Retry-After", "5")
		http.Error(w, "connection capacity reached", http.StatusServiceUnavailable)
		return
	}

	upgrader := websocket.Upgrader{
		ReadBufferSize:  4096,
		WriteBufferSize: 4096,
		CheckOrigin: func(r *http.Request) bool {
			origin := r.Header.Get("Origin")
			if origin == "" {
				return true
			}
			_, allowed := a.cfg.origins[origin]
			return allowed
		},
	}
	conn, err := upgrader.Upgrade(w, r, nil)
	if err != nil {
		<-a.slots
		return
	}
	c := &client{app: a, conn: conn, send: make(chan []byte, queueSize), user: user}
	select {
	case a.hub.register <- c:
	case <-a.ctx.Done():
		conn.Close()
		<-a.slots
		return
	}

	a.metrics.accepted.Add(1)
	var once sync.Once
	cleanup := func() {
		once.Do(func() {
			conn.Close()
			select {
			case a.hub.unregister <- c:
			case <-a.ctx.Done():
			}
			<-a.slots
		})
	}

	a.pumps.Add(2)
	go func() {
		defer a.pumps.Done()
		c.readPump()
		cleanup()
	}()
	go func() {
		defer a.pumps.Done()
		c.writePump()
		cleanup()
	}()
}

func subscribe(ctx context.Context, rdb *redis.Client, channel string, inbound chan<- []byte) {
	backoff := time.Second
	for ctx.Err() == nil {
		pubsub := rdb.Subscribe(ctx, channel)
		_, err := pubsub.Receive(ctx)
		if err == nil {
			backoff = time.Second
			for {
				message, receiveErr := pubsub.ReceiveMessage(ctx)
				if receiveErr != nil {
					err = receiveErr
					break
				}
				select {
				case inbound <- []byte(message.Payload):
				case <-ctx.Done():
					pubsub.Close()
					return
				}
			}
		}
		pubsub.Close()
		if ctx.Err() != nil {
			return
		}
		slog.Error("subscription interrupted", "error", err, "retry_in", backoff)
		timer := time.NewTimer(backoff)
		select {
		case <-timer.C:
		case <-ctx.Done():
			timer.Stop()
			return
		}
		if backoff < 15*time.Second {
			backoff *= 2
		}
	}
}

func main() {
	cfg := loadConfig()
	ctx, cancel := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
	defer cancel()

	options, err := redis.ParseURL(cfg.redisURL)
	if err != nil {
		panic(err)
	}
	options.DialTimeout = 2 * time.Second
	options.ReadTimeout = 2 * time.Second
	options.WriteTimeout = 2 * time.Second
	options.PoolTimeout = 2 * time.Second
	options.PoolSize = 20
	rdb := redis.NewClient(options)

	startupCtx, startupCancel := context.WithTimeout(ctx, 2*time.Second)
	err = rdb.Ping(startupCtx).Err()
	startupCancel()
	if err != nil {
		panic(fmt.Sprintf("Redis unavailable: %v", err))
	}

	metrics := &counters{}
	h := &hub{
		register: make(chan *client), unregister: make(chan *client),
		inbound: make(chan []byte, 1024), metrics: metrics,
	}
	a := &app{
		ctx: ctx, cfg: cfg, redis: rdb, hub: h,
		slots: make(chan struct{}, cfg.maxClients), metrics: metrics,
	}
	go h.run(ctx)
	go subscribe(ctx, rdb, cfg.channel, h.inbound)

	mux := http.NewServeMux()
	mux.HandleFunc("/ws", a.serveWS)
	mux.HandleFunc("/healthz", func(w http.ResponseWriter, _ *http.Request) {
		w.WriteHeader(http.StatusOK)
	})
	mux.HandleFunc("/readyz", func(w http.ResponseWriter, r *http.Request) {
		checkCtx, checkCancel := context.WithTimeout(r.Context(), 500*time.Millisecond)
		defer checkCancel()
		if rdb.Ping(checkCtx).Err() != nil {
			http.Error(w, "not ready", http.StatusServiceUnavailable)
			return
		}
		w.WriteHeader(http.StatusOK)
	})
	mux.HandleFunc("/metrics", func(w http.ResponseWriter, _ *http.Request) {
		w.Header().Set("Content-Type", "text/plain; version=0.0.4")
		fmt.Fprintf(w,
			"ws_connected %d\nws_accepted_total %d\nws_published_total %d\n"+
				"ws_publish_errors_total %d\nws_slow_disconnects_total %d\n",
			metrics.connected.Load(), metrics.accepted.Load(),
			metrics.published.Load(), metrics.publishErrors.Load(),
			metrics.slowDisconnects.Load(),
		)
	})

	server := &http.Server{
		Addr: cfg.addr, Handler: mux,
		ReadHeaderTimeout: 5 * time.Second,
		IdleTimeout:       60 * time.Second,
		MaxHeaderBytes:    16 << 10,
	}
	serverErrors := make(chan error, 1)
	go func() {
		slog.Info("server listening", "address", cfg.addr, "node", cfg.node)
		serverErrors <- server.ListenAndServe()
	}()

	select {
	case <-ctx.Done():
	case err := <-serverErrors:
		if err != nil && err != http.ErrServerClosed {
			slog.Error("HTTP server stopped", "error", err)
		}
		cancel()
	}

	shutdownCtx, shutdownCancel := context.WithTimeout(context.Background(), 10*time.Second)
	_ = server.Shutdown(shutdownCtx)
	shutdownCancel()
	cancel()

	done := make(chan struct{})
	go func() {
		a.pumps.Wait()
		close(done)
	}()
	select {
	case <-done:
	case <-time.After(5 * time.Second):
		slog.Warn("timed out waiting for connection pumps")
	}
	_ = rdb.Close()
}

The server intentionally omits http.Server.WriteTimeout. A conventional short response timeout can terminate long-lived upgraded connections. WebSocket writes are bounded individually with deadlines instead.

Mint short-lived authentication tokens

This compact HMAC token format is suitable when one trusted authentication service issues credentials to this WebSocket tier. It contains a base64url subject, Unix expiry, and SHA-256 HMAC. It is not presented as JWT and should not be mixed with JWT tooling.

package main

import (
	"crypto/hmac"
	"crypto/sha256"
	"encoding/base64"
	"fmt"
	"os"
	"time"
)

func main() {
	if len(os.Args) != 3 {
		panic("usage: token SUBJECT DURATION")
	}
	secret := []byte(os.Getenv("AUTH_SECRET"))
	if len(secret) < 32 {
		panic("AUTH_SECRET must contain at least 32 characters")
	}
	duration, err := time.ParseDuration(os.Args[2])
	if err != nil || duration <= 0 {
		panic("duration must be positive, for example 15m")
	}
	subject := base64.RawURLEncoding.EncodeToString([]byte(os.Args[1]))
	unsigned := fmt.Sprintf("%s.%d", subject, time.Now().Add(duration).Unix())
	mac := hmac.New(sha256.New, secret)
	mac.Write([]byte(unsigned))
	fmt.Printf("%s.%s\n", unsigned,
		base64.RawURLEncoding.EncodeToString(mac.Sum(nil)))
}

Browser JavaScript cannot set an arbitrary Authorization header during the native WebSocket handshake. For browser deployments, exchange the normal authenticated session for a short-lived, single-use ticket and pass that ticket in the URL. Ensure the edge proxy redacts query strings from logs. Do not place reusable API keys in URLs.

Containerize and scale horizontally

The container runs without root privileges. HAProxy terminates the public HTTP connection and distributes upgraded connections across application replicas; WebSockets require no sticky sessions because every node uses the shared broker.

FROM golang:1.22-alpine AS build
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" -o /out/realtime .

FROM alpine:3.20
RUN addgroup -S app && adduser -S -G app app
COPY --from=build /out/realtime /usr/local/bin/realtime
USER app
EXPOSE 8080
ENTRYPOINT ["/usr/local/bin/realtime"]

Create haproxy.cfg:

global
    log stdout format raw local0

defaults
    mode http
    log global
    option httplog
    timeout connect 3s
    timeout client 1h
    timeout server 1h
    timeout http-request 10s

resolvers docker
    nameserver dns 127.0.0.11:53
    resolve_retries 3
    timeout resolve 1s
    timeout retry 1s
    hold valid 10s

frontend public
    bind *:8080
    default_backend websocket_nodes

backend websocket_nodes
    balance leastconn
    option httpchk GET /readyz
    http-check expect status 200
    server-template app 1-10 app:8080 check resolvers docker init-addr libc,none

Create compose.yaml:

services:
  app:
    build: .
    environment:
      AUTH_SECRET: "${AUTH_SECRET:?set AUTH_SECRET}"
      REDIS_URL: "redis://redis:6379/0"
      REDIS_CHANNEL: "ws:broadcast"
      ALLOWED_ORIGINS: "https://app.example.com"
      MAX_CLIENTS: "5000"
    expose:
      - "8080"
    depends_on:
      redis:
        condition: service_healthy
    ulimits:
      nofile:
        soft: 65536
        hard: 65536
    restart: unless-stopped

  redis:
    image: redis:7.4-alpine
    command: ["redis-server", "--save", "", "--appendonly", "no"]
    expose:
      - "6379"
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 2s
      timeout: 1s
      retries: 20
    restart: unless-stopped

  proxy:
    image: haproxy:3.0-alpine
    volumes:
      - "./haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro"
    ports:
      - "127.0.0.1:8080:8080"
    depends_on:
      - app
    restart: unless-stopped

Generate dependencies and start three application replicas:

go mod tidy
export AUTH_SECRET="$(openssl rand -base64 32)"
docker compose up --build --scale app=3

The example binds HAProxy only to loopback. In production, terminate TLS at a hardened edge proxy, expose only the intended HTTPS port through the host firewall, and keep Redis on a private network. A remote Redis deployment should use an authenticated rediss:// URL with certificate verification. Do not expose /metrics publicly; restrict it by network policy or a separate internal listener.

Verify delivery across connections

Add cmd/check/main.go. It opens two authenticated connections, publishes through one, and confirms receipt through the other.

package main

import (
	"fmt"
	"net/http"
	"os"
	"time"

	"github.com/gorilla/websocket"
)

func connect(token string) *websocket.Conn {
	headers := http.Header{"Authorization": {"Bearer " + token}}
	conn, _, err := websocket.DefaultDialer.Dial("ws://127.0.0.1:8080/ws", headers)
	if err != nil {
		panic(err)
	}
	return conn
}

func main() {
	if len(os.Args) != 2 {
		panic("usage: check TOKEN")
	}
	first := connect(os.Args[1])
	second := connect(os.Args[1])
	defer first.Close()
	defer second.Close()

	if err := first.WriteJSON(map[string]string{"body": "cross-node hello"}); err != nil {
		panic(err)
	}
	_ = second.SetReadDeadline(time.Now().Add(5 * time.Second))
	_, message, err := second.ReadMessage()
	if err != nil {
		panic(err)
	}
	fmt.Println(string(message))
}

TOKEN="$(go run ./cmd/token alice 15m)"
go run ./cmd/check "$TOKEN"
curl -fsS http://127.0.0.1:8080/readyz
curl -fsS http://127.0.0.1:8080/metrics

The event’s node field identifies the publishing replica. With several long-lived clients, HAProxy’s least-connections policy distributes connections among healthy nodes, while Redis makes delivery independent of which replica owns either connection.

Failure drills and operational tuning

Pause Redis with docker compose pause redis. Readiness should return 503, subscription loops should retry with bounded exponential backoff, and a client attempting to publish should receive close code 1011 rather than a false success. Resume it with docker compose unpause redis; subscriptions should recover automatically.

Test backpressure with a client that completes the handshake but never reads. Once its 64-message queue fills, ws_slow_disconnects_total should increase and healthy clients should continue receiving messages. This is the essential isolation property: one slow consumer costs one connection, not the hub.

Capacity planning must account for file descriptors, goroutine stacks, TLS buffers at the edge, the 64-slot client queues, and the maximum 4 KiB frame. The Compose file raises the container limit, but the host’s file-descriptor ceiling must also be inspected and configured by an administrator. Increase limits only after measuring memory and reconnect behavior.

Add edge-level handshake rate limiting, connection limits per identity or source, and token-issuance throttling. Rotate authentication secrets with an overlap strategy rather than replacing the only key instantaneously. For stronger key management, use versioned signing keys and include a key identifier in the token format.

Common production failures

• Every client shares one writer: a blocked socket stalls unrelated users. Keep one bounded writer per connection.
• Outbound queues grow dynamically: memory exhaustion merely arrives later. Fix the capacity and define the eviction policy.
• Only TCP keepalive is enabled: dead peers can survive too long through proxies. Use application-level ping/pong deadlines.
• The originating node broadcasts locally and through Redis: local clients receive duplicates. Use one delivery path.
• Redis Pub/Sub is described as reliable storage: it provides live fan-out, not replay or durable acknowledgement.
• Origin checks are disabled: authenticated browser sessions become vulnerable to cross-site WebSocket hijacking.
• Readiness and liveness are identical: an unhealthy dependency can leave a replica receiving new connections. Keep process health separate from traffic readiness.

Final verification checklist

• Invalid, altered, and expired tokens receive HTTP 401 before upgrade.
• Disallowed browser origins fail the handshake.
• The process refuses connections beyond MAX_CLIENTS with HTTP 503.
• Ping/pong deadlines remove dead peers.
• Slow consumers are disconnected without blocking healthy clients.
• Redis operations have explicit dial, pool, read, write, and per-publish timeouts.
• Multiple replicas exchange messages through the shared channel.
• Redis failure makes readiness fail and publishing closes honestly.
• SIGTERM stops admission, closes clients, and waits briefly for pumps.
• TLS, firewall rules, metrics access, and Redis exposure are controlled at deployment boundaries.

A scalable WebSocket service is not defined by how quickly it accepts an upgrade. It is defined by the limits it enforces when the network becomes unfair. Bound every resource, make failure visible, and disconnect clients the system cannot safely serve. Those choices turn a persistent socket from an operational liability into a dependable production primitive.