Why LLM-drawn architecture diagrams are wrong

Paste a repository into a language model and ask it to draw the architecture. You will get a diagram. It will have boxes with sensible names, arrows pointing in plausible directions, and a layered layout that looks like every architecture diagram you have seen.

Some of it will be wrong. The problem is not that some of it is wrong — every model output has that property, and we have learned to live with it. The problem is that you cannot tell which parts, and a diagram gives you no way to find out.

Why a wrong diagram is worse than no diagram

A wrong sentence and a wrong picture fail differently.

When a model writes "the checkout service calls the auth service", you read it as a claim. It sits in your head as something asserted, and some part of you keeps a receipt.

When a model draws an arrow from Checkout to Auth, you do not read it as a claim. You read it as a map. Diagrams carry an implicit authority that prose does not — they look like the output of a process rather than the output of an opinion, and you start navigating by them immediately.

So a diagram that is 85% right is not 85% useful. It is a map with unmarked errors, and you find them by walking into them.

What the model is actually doing

A model asked to draw your architecture is doing pattern completion over the identifiers it saw. It read files named auth-service.ts, checkout.ts, session-manager.ts, and it knows — from an enormous amount of other people's code — roughly how components with those names relate in a typical system.

That is genuinely useful information. It is also not a fact about your system.

The failure mode is specific and predictable: the model draws the architecture your code looks like it should have. Which is exactly the architecture you no longer have, because the whole reason you wanted the diagram is that the codebase drifted from the design.

The diagram is most confident precisely where it is least useful.

The distinction that fixes this

The fix is not a better prompt or a bigger context window. It is deciding, per element of the diagram, whether a computer or a model produced it.

Some things about a codebase are facts, and a parser can extract them exactly:

• which modules import which
• which functions call which
• which HTTP routes exist and which handlers serve them
• which code touches which database tables
• what changed in the schema
• what changed in the dependency manifest

Other things are judgements, and a parser cannot produce them at all:

• what to call a cluster of twelve files
• which of forty relationships matter enough to draw
• how to phrase what a change does
• what is worth a reader's attention first

The mistake is asking one system for both. Ask a parser for the judgements and you get "37 imports found", which is true and useless. Ask a model for the facts and you get a beautiful diagram of a system that does not exist.

The structure should come from analysis. The naming and the prose should come from the model. Nothing should come from both.

What this looks like in practice

Under that split, every arrow in the diagram exists because a parser found an import or a call, and can name the file and line where it found it. The label on the box may be the model's — "Authentication domain" is a judgement, and a good one — but the box exists because eleven files were found to belong together by a rule you can inspect.

Drawn per change rather than for the whole system, that is an architecture diff: the same two pictures before and after, where every difference between them was computed rather than recalled.

The practical test is whether you can click an arrow and land on the line of code that produced it. If you can, the diagram is checkable, and its errors are findable in seconds rather than in production. If you cannot, you are holding an illustration.

This also changes what the tool does when it is unsure. A model asked to draw something it does not understand will draw something anyway — that is what completion means. A system grounded in analysis can decline: show fewer nodes, say how many it left out, and name what it chose not to claim. When it is uncertain, it should show less, not something plausible.

That is an unglamorous property and it is the only one that makes the output worth trusting.

The honest limits

Static analysis is not omniscient either, and a tool built on it should say so loudly:

• Dynamic dispatch is invisible. getattr, dependency-injection containers, Django's app registry, duck typing — no static tool sees these, and none should claim to.
• Line numbers are only valid at one commit, so a citation has to carry the sha it was taken at or it decays into a wrong link.
• Grouping files into components involves thresholds, and thresholds are tuned rather than proven. They should be inspectable.

A tool that names these is more trustworthy than one that does not, for the same reason the whole argument above holds: the value is not in being right about everything. It is in the reader being able to tell where the ground is.

---

This is the principle Cutplane is built on — the structure comes from real static analysis and the model only names and explains what the analysis found. The example report is the visible version of that argument: every claim on it links to the line of code it was computed from, on a public repository you can go and check.