The diagram everyone draws is the happy-path one: user hits app, app hits database, done. It is fine for a whiteboard and useless for running a system. Production has a different shape, because production has to survive traffic spikes, node failures, deploys and the occasional bad afternoon. Visualizing that reality, not the tidy sketch, is what turns a diagram into an operational tool.
The trick is to stop drawing "what talks to what" and start drawing three overlapping views: the traffic path, the failure domains, and the scaling boundaries. Most production diagrams are muddled because they try to show all three at once with no intent. Draw them deliberately and the picture becomes something you can actually reason with at 3 a.m.
View one: the traffic path
Trace a single request from the outside world to the data and back, and draw every hop it really takes. In production that is longer than the sketch admits:
- A CDN or edge in front, serving cache hits before your origin ever sees them.
- A load balancer spreading traffic across instances.
- A tier of stateless app servers, not one box.
- A cache that most reads hit before the database.
- A primary database for writes and read replicas for the rest.
- An async path: a queue and workers for the work that should not happen inline.
View two: the failure domains
Now draw the lines that group things by "what fails together." Availability zones are the obvious one: two or three columns, with your app tier and data tier spread across them, so a zone outage does not take you down. But failure domains are broader than zones. A shared cache is a domain. A single primary database is a domain. The queue is a domain. Drawing these boundaries makes your single points of failure impossible to miss, because they show up as the box with everything depending on it and no partner beside it.
A production diagram earns its keep the moment it shows you the box you cannot afford to lose.
View three: the scaling boundaries
Mark what scales and how. The app tier scales horizontally, so draw it as a group that grows, not a fixed pair. The database scales differently, up for writes, out with replicas for reads, and that asymmetry is worth showing because it is where production systems hit a wall. Autoscaling groups, worker pools that expand under queue depth, and any fixed-capacity component all deserve a visible note. The scaling story is half of capacity planning, and it lives in this view.
Do not forget the pieces that only exist in prod
The design doc leaves these out; production cannot. Observability (metrics, logs, traces) taps nearly every component. Secrets and config come from somewhere. There is a bastion or a private access path for humans. Backups run against the data tier. You do not need to draw all of it in one diagram, and you should not, but a production picture that omits every one of them is a fiction. Give the important ones a box.
Keep it current, or it lies
A production diagram is only useful if it matches production, and production changes constantly. This is the case for deriving it from your Terraform or manifests and regenerating on a cadence, rather than lovingly hand-maintaining a drawing that drifts. Pair the generated skeleton with your annotations, embed it in the runbook, and refresh it when the infrastructure moves. A current production diagram is one of the highest-leverage documents a team can own.
Draw the traffic path, mark the failure domains, show the scaling boundaries, and generate the base from real infrastructure so it stays honest. That is the diagram you actually want open during an incident.