Skip to report

Durable text batches get their own AI SDK layer

Follow the AI SDK’s experimental batch API from a new batch component in the ai package to the provider batch interface that language models declare support for.

vercel/ai #18346 · PR by n1ckoates · Merged

vercel/ai 2b608269..c90d57a8

base and head as given

  • Experimental text batch operations are now exposed by the AI SDK.
  • Language models can now declare durable batch-processing capability.
  • Core AI now depends on its batch-handling layer.
files changed
14
in a component
11
lines in components
780
components
5
public routes changed
0
schema statements
0
new dependencies
0
excluded by role
3
Based on 11 of 11 eligible changed files

Durable text batches

Added
Before
Text batch operations did not exist
After
Text batches can be started and queried

Applications can submit multiple text-generation requests as one durable batch, check its lifecycle status, and stream completed item results. Only models declaring batch support are accepted.

In favour

  • In favour: Rejects empty or duplicate request IDs before provider calls
  • In favour: Cancelling result streams aborts the provider operation

Against

  • Against: Core and batch code now import each other, three edges total
Files changed in 5 components. Dependencies shown: 4 new
new · 42 components added

4 new dependencies between components — AI Batch → AI Package Source · AI Package Source → AI Batch · Provider Package Source → Provider Batch · V4 Language Models → Provider Batch

What else could behave differently

7 components · 11 files

These components import changed code, directly or through other files. A traced dependency is a place to review, not a confirmed runtime failure.

Reached 3

ComponentSymbolCited atFilesReached by
AI Package Sourcepackages/ai/src/Added 1 of 15 changed symbols reaching itpackages/ai/src/batch/batch-types.ts:231none direct · 1 at two steps
Trace path

Shortest component path from the changed symbol:

  1. AI Batch
  2. AI Package Source

Later steps follow file imports; they do not prove this symbol is called.

Provider Batchpackages/provider/src/batch/Added 1 of 7 changed symbols reaching itpackages/provider/src/batch/v4/batch-v4.ts:10-1521 direct · 1 at two steps
Trace path

Shortest component path from the changed symbol:

  1. Provider Batch

Later steps follow file imports; they do not prove this symbol is called.

AI Batchpackages/ai/src/batch/Added 1 of 15 changed symbols reaching itpackages/ai/src/batch/batch-types.ts:2332 direct · 1 at two steps
Trace path

Shortest component path from the changed symbol:

  1. AI Batch

Later steps follow file imports; they do not prove this symbol is called.

Held back 2

ComponentFilesReached byResolution
packages/ai/1none direct · 1 at three stepsprecise
packages/provider/src/1none direct · 1 at three stepsprecise

Ranked lower 2

ComponentFilesReached by
packages/provider/src/language-model/1none direct · 1 at two steps
packages/provider/src/language-model/v4/22 direct

Statements 3

The added BatchLanguageModel, alongside all 13 names in its file, reached 1 file in AI Package Source. 1
packages/ai/src/batch/batch-types.ts:23
The added BatchV4Error, alongside all 7 names in its file, reached 2 files in Provider Batch. 11
packages/provider/src/batch/v4/batch-v4.ts:10-15
The added BatchLanguageModel, alongside all 13 names in its file, reached 3 files in AI Batch. 21
packages/ai/src/batch/batch-types.ts:23

2 reached files belong to no component — tests, docs and config.

Routes, schema, dependencies

No public URL, no migration and no new package.

Not stated

  • 389 routes were found on files whose role excludes them, so no route is claimed for them.

Inspect components, files and symbols

Component

AI Package SourceModified
packages/ai/src/

1 file · 1 line

13 dependents

Provider Package SourceModified
packages/provider/src/

1 file · 1 line

Provider BatchAdded
packages/provider/src/batch/

3 files · 114 lines

2 dependents

Provider Batch added 14 exported batch-related names.
AI BatchAdded
packages/ai/src/batch/

4 files · 621 lines

1 dependent

AI Batch added 30 exported batch-related names.
V4 Language ModelsModified
packages/provider/src/language-model/v4/

2 files · 43 lines

3 dependents

V4 Language Models added 4 exported batch-language-model names.

File

AI Package Sourcepackages/ai/src/
packages/ai/src/index.ts

+1 −0

Provider Package Sourcepackages/provider/src/
packages/provider/src/index.ts

+1 −0

Provider Batchpackages/provider/src/batch/
packages/provider/src/batch/v4/batch-v4.ts

+104 −0

Provider Batchpackages/provider/src/batch/
packages/provider/src/batch/v4/index.ts

+9 −0

Provider Batchpackages/provider/src/batch/
packages/provider/src/batch/index.ts

+1 −0

AI Batchpackages/ai/src/batch/
packages/ai/src/batch/batch.ts

+336 −0

AI Batchpackages/ai/src/batch/
packages/ai/src/batch/batch-types.ts

+134 −0

AI Batchpackages/ai/src/batch/
packages/ai/src/batch/batch-types.test-d.ts

+132 −0

AI Batchpackages/ai/src/batch/
packages/ai/src/batch/index.ts

+19 −0

V4 Language Modelspackages/provider/src/language-model/v4/
packages/provider/src/language-model/v4/language-model-v4-batch.ts

+39 −0

V4 Language Modelspackages/provider/src/language-model/v4/
packages/provider/src/language-model/v4/index.ts

+4 −0

Function

imports what changeddownstream of it, one mark per step

Diff

AI Package Sourcepackages/ai/src/
packages/ai/src/index.ts

35–35

Provider Package Sourcepackages/provider/src/
packages/provider/src/index.ts

1–1

Provider Batchpackages/provider/src/batch/
packages/provider/src/batch/v4/batch-v4.ts

1–104

Provider Batchpackages/provider/src/batch/
packages/provider/src/batch/v4/index.ts

1–9

Provider Batchpackages/provider/src/batch/
packages/provider/src/batch/index.ts

1–1

AI Batchpackages/ai/src/batch/
packages/ai/src/batch/batch.ts

1–336

AI Batchpackages/ai/src/batch/
packages/ai/src/batch/batch-types.ts

1–134

AI Batchpackages/ai/src/batch/
packages/ai/src/batch/batch-types.test-d.ts

1–132

AI Batchpackages/ai/src/batch/
packages/ai/src/batch/index.ts

1–19

V4 Language Modelspackages/provider/src/language-model/v4/
packages/provider/src/language-model/v4/language-model-v4-batch.ts

1–39

V4 Language Modelspackages/provider/src/language-model/v4/
packages/provider/src/language-model/v4/index.ts

23–26

Analysis scope

7438 → 5
  1. 7,438
    files at head
  2. 241
    components
  3. 5
    changed by this change

Where the new files landed

8 new files
New fileBelongs toUsed byCallsServes
packages/provider/src/batch/3 new files
Provider Batchnew territory
Provider Package SourceV4 Language Models
packages/provider/src/shared/
packages/ai/src/batch/4 new files
AI Batchnew territory
AI Package Source
AI Package Sourcepackages/ai/src/error/packages/ai/src/logger/packages/ai/src/model/packages/ai/src/prompt/packages/ai/src/types/…and 1 more component
packages/provider/src/language-model/v4/language-model-v4-batch.ts
V4 Language Modelsbase frame
its own component only
Provider Batch

3 added files carry no component — 1 docs, 1 example, 1 test.

5components changed
2reached, not changed
2components added/removed
11dependencies added between components
0dependencies removed between components
0files moved to another path

14 files changed, 15 claims stated. 7 components reached in total.

naming: “model

narrative: “model

analyzer 0.2.0, config v1.9.0-phase8

Every line above is derived from 2b608269..c90d57a8 and nothing else.