The HTTP 502 Bad Gateway error is one of the most frequent and frustrating issues encountered when deploying microservices with Docker and Nginx. To the outside browser, Nginx returns a blank error page or generic 502 status, while your application container appears to be running normally according to docker ps.
A 502 Bad Gateway means exactly one thing: Nginx acted as a reverse proxy, contacted an upstream address, but received an invalid response or could not establish a TCP handshake.
In Dockerized environments, this failure almost always boils down to one of four specific network boundary traps. This runbook walks through the exact diagnostic commands to isolate the failure in 60 seconds and provides permanent production fixes.
1. The 60-Second Diagnostic Triage
Before changing Nginx configuration files, check Nginx's error logs to identify the exact operating system socket failure:
# Tail the Nginx error log:
docker logs --tail 50 -f nginx-proxy
| Log Error Signature | Root Cause | Target Fix |
|---|---|---|
| connect() failed (111: Connection refused) | App listening on 127.0.0.1 or crashed on boot | Bind backend to 0.0.0.0; inspect app logs |
| [emerg] host not found in upstream "api" | Nginx and container are on different Docker networks | Attach both containers to a shared user-defined network |
| upstream timed out (110: Connection timed out) | Heavy backend computation or unbuffered token streaming | Increase proxy_read_timeout; disable buffering |
| connect() to 172.20.0.4:8000 failed (113: No route to host) | Stale Docker IP cached by Nginx after container restart | Use Nginx dynamic variable resolution with 127.0.0.11 |
2. Root Cause 1: The '127.0.0.1' Localhost Binding Trap
This is the single most common cause of 111: Connection refused. In Python (Uvicorn, Flask), Node.js (Express, Next.js), or Go, developers frequently configure their servers to listen on localhost or 127.0.0.1:
# ❌ WRONG: Listens only inside the container's isolated loopback namespace
uvicorn.run("app:app", host="127.0.0.1", port=8000)
Inside a container, 127.0.0.1 refers exclusively to that specific container's loopback interface. Incoming packets from Nginx arrive over the Docker bridge network interface (e.g. eth0 at 172.20.0.3). Because the application is listening only on loopback, the Linux kernel immediately rejects the connection with ECONNREFUSED.
The Solution
Always bind your web application to all network interfaces (0.0.0.0):
# ✅ CORRECT: Listens on all container interfaces
uvicorn.run("app:app", host="0.0.0.0", port=8000)
3. Root Cause 2: Container Network Isolation
If Nginx logs output [emerg] host not found in upstream "api", Nginx's embedded DNS cannot resolve the container name. By default, containers defined in separate docker-compose.yml files or launched with raw docker run commands are placed on different bridge networks and cannot communicate by service name.
The Production Solution: Shared External Network
Create an explicit Docker bridge network:
docker network create proxy_network
Attach both your Nginx proxy and your upstream application service to this shared network in their respective docker-compose.yml files:
# In your application docker-compose.yml:
services:
api:
image: my-app:latest
restart: unless-stopped
networks:
- proxy_network
networks:
proxy_network:
external: true
4. Root Cause 3: Docker DNS Caching (Stale Upstream IP)
When Nginx starts, it resolves upstream domain names (e.g. proxy_pass http://api:8000;) once at startup and caches the resolved IP address in memory indefinitely.
When you rebuild or restart the api container, Docker assigns it a new IP address on the internal bridge (e.g. moving from 172.20.0.4 to 172.20.0.5). Nginx continues sending packets to the old, nonexistent IP address, resulting in an intermittent 502 Bad Gateway until Nginx itself is restarted.
The Permanent Solution: Dynamic DNS Resolution via 127.0.0.11
Docker runs an internal DNS server at 127.0.0.11 inside every user-defined network. By assigning the upstream to an Nginx variable, Nginx is forced to re-query the Docker DNS server respecting the DNS TTL:
server {
listen 80;
server_name api.example.com;
# Point to Docker internal DNS resolver with 10-second cache TTL
resolver 127.0.0.11 valid=10s ipv6=off;
# Setting upstream via a variable forces dynamic runtime resolution
set $upstream_endpoint http://api:8000;
location / {
proxy_pass $upstream_endpoint;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
Summary Checklist
- Ensure your application server binds to
0.0.0.0, never127.0.0.1. - Verify both Nginx and your application are attached to the same user-defined Docker network.
- Ensure the internal container port in
proxy_passmatches what the application is listening on (not the external host port mapping). - Use
resolver 127.0.0.11 valid=10s;to prevent stale IP caching across container deployments.
Related Architecture & Developer Guides
- Caddy vs. Nginx: The Reverse Proxy Decision Tree for Self-Hosters in 2026 (Automated reverse proxies without manual DNS traps).
- How to Fix 'Docker: permission denied while trying to connect to the Docker daemon socket' (Fixing Linux container permissions).
- Self-Hosting Uptime Kuma: The Complete Production Guide (Monitoring container upstreams).
This technical guide and reference runbook was verified and tested on production Linux systems by Nabil Massaoudi (Founder & Technical Publisher) and the RepoBenchmarks Editorial Team. For configuration files, compose templates, or troubleshooting inquiries, contact editorial@repobenchmarks.com or inspect our public repository at github.com/repobenchmarks/devops-runbooks.
Discussion & Issues
Comments
Post a Comment