What leaves your machine when a code tool calls an LLM

Every developer tool that calls a language model faces the same question, and most of them answer it badly: what, exactly, do you upload?

The bad answers are recognisable. "Your code is never used for training" answers a different question. "We take security seriously" answers no question. "Data is encrypted in transit" describes the pipe, not the payload. A privacy page that says only these things is telling you that nobody wrote down the answer.

The question worth asking is narrower and much harder to dodge.

The question

Name the bytes. How many files, which ones, how large, and what happens if I say no?

A tool that has thought about this can answer in four sentences. A tool that has not will change the subject to compliance.

There is a good reason the answer matters beyond the obvious one. It is not only that source code is sensitive — it is that metadata is sensitive in ways people underestimate. A file path can name an unreleased product, an internal service, or a customer. src/features/acme-migration/ leaks a fact about your business before a single line of code is read. Any tool that says "we only send metadata, not code" has not finished thinking about it.

The answer for this one

Cutplane's CLI runs locally, reads git objects directly, and makes exactly two network calls to a model. Here is what each one carries.

By default, source is sent. Up to twelve changed files, whole, at most 156 KB in total — the highest-priority changed files from the components the change touched. This exists so the report can say what a change does and not only what it touched, which is the difference between a useful report and a file census.

Structural metadata goes with them: paths, symbol names, component names, counts, relationships. As above, this is not the harmless half.

Two flags change the answer:

default        structure, plus up to 12 changed files (≤156 KB)
--no-source    structure only — what every version before 0.2 sent
--offline      nothing at all

--offline means no network call is made. Not "no source is sent" — no call. You get the structural report, and it says at the top that it is structure only.

Don't take that on trust

The paragraph above is a claim by an interested party, which is worth roughly what such claims are usually worth. So there is a flag that makes it checkable:

$ npx cutplane analyze --pr 482 --print-llm-payload

It prints the exact bytes of both calls. Not a summary of them, not a description — the payload. If the paragraph above is wrong, this is where it becomes obvious.

That is the property that matters, and it generalises past this tool: a privacy claim you can verify locally is worth more than a policy you have to believe. When you are evaluating anything in this category, look for the verification path. Its absence is the finding.

Why two calls and not one

The two calls carry different payloads on purpose, and collapsing them would make the privacy story worse.

One call phrases the report's sentences and names its components. It sends structure only — no source at all. It is the call that turns "these eleven files cluster together" into "Authentication domain".

The other reads the key files and writes what the change actually does, at the top of the report. It is the only call that carries source.

Splitting them means --no-source can disable exactly one of them and still produce a full structural report, rather than being all-or-nothing.

The part that is not negotiable

An API key is required. Without one, analyze stops and names the fix rather than silently degrading — because a tool that quietly produced a worse report when a key was missing would be lying by omission about which sentences a model wrote.

--offline is how you ask for the structural report deliberately. The distinction between failed to call a model and chose not to call a model belongs to you, not to a fallback path.

What to ask your other tools

Take the four questions to anything else in your stack that calls a model on your code:

1. How many files, and which ones — by what rule?
2. Is there a byte ceiling?
3. Can I turn source off and still get output?
4. Can I see the payload before it leaves?

A tool that answers all four has thought about it. A tool that answers none has not, whatever its security page says.

The full flag list and the rest of the CLI's behaviour is on the CLI page, and it is published on npm as cutplane under MIT if you would rather read it than read about it.