@use-crux/ai

import {
  generate,
  stream,
  createUIMessageStreamResponse,
  pipeUIMessageStreamToResponse,
  embedding,
  retrievalModel,
  reranker,
  aiSdkProviderRuntime,
} from "@use-crux/ai";
import {
  generateObjectFn,
  generateTextFn,
  createCruxAi,
  CruxAIError,
} from "@use-crux/ai";
import { resolve } from "@use-crux/ai/agent";

Architecture

@use-crux/ai exports aiSdkProviderRuntime, a normal Crux provider runtime with ownership: 'loop-owned' for the Vercel AI SDK. Core owns all policy (prompt resolution, fallback()/router()/cascade() routing, validation retry, the Safety session for constraints/guardrails, the tool-approval protocol, instrumentation, timeouts); the adapter delegates mechanics to AI SDK natives (stopWhen, prepareStep, experimental_repairText, SDK-owned tool approval hooks, abortSignal, experimental_transform for streaming guardrails). The bound runtime also exposes AI SDK embedding through the same SdkGateway seam.

The only module that calls AI SDK functions is the SdkGateway — inject a scripted one via createCruxAi to test without module mocks.

Internally, @use-crux/ai keeps AI SDK request planning and result projection in a private call-plan codec. The loop runtime asks the codec for a generateText, generateObject, streamText, streamObject, or replay plan, invokes the selected SdkGateway method, then decodes or attaches the raw SDK result. This is not a public extension API; it exists to keep the gateway seam stable and the adapter behavior testable.

For Anthropic AI SDK models, Crux converts resolved systemBlocks into AI SDK system messages and places providerOptions.anthropic.cacheControl on the single cacheBoundary block at the end of the stable cached prefix.

Multimodal content

@use-crux/ai accepts canonical ContentPart[] message content and encodes it to AI SDK text, image, and file parts. AI SDK-shaped message arrays from convertToModelMessages() normalize back to canonical parts when no fields would be dropped; legacy v6 media parts decode as file-data. SDK control parts such as tool calls and approval responses move into Message.metadata, while genuinely unknown part shapes pass through with a diagnostics warning.

Cross-adapter parity: switching a prompt between @use-crux/ai and a native adapter (@use-crux/openai, @use-crux/anthropic, @use-crux/google) changes nothing observable except model behavior itself — same default tool-loop budget (maxSteps: 10), same corrective-retry messages, same approval suspend/resume protocol, same canonical tool.call / tool.args / raw and model-facing tool.result telemetry, same routing and audit metadata. This is enforced by a cross-dialect parity suite in @use-crux/core. Note this differs from the raw AI SDK's own generateText default (stopWhen: stepCountIs(1)): Crux adapters loop tools by default; pass maxSteps: 1 for single-step behavior.

Hallucinated tool calls are part of that parity: where the raw AI SDK throws NoSuchToolError/InvalidToolInputError and aborts the loop, @use-crux/ai uses the SDK's native experimental_repairToolCall to feed the model the same error tool result core-driven adapters produce ({"error":"Tool \"x\" not found"}), so the model self-corrects and the loop continues. The SDK's tool-error taxonomy never reaches your code.

@use-crux/ai/agent

Use @use-crux/ai/agent for AI SDK-compatible agent frameworks that own their own model loop, such as Convex Agent or Mastra. It composes instructions through the normal core prompt pipeline, then wraps the model with AI SDK middleware when Crux execution hooks are installed.

import { resolve } from "@use-crux/ai/agent";

const { instructions, model } = await resolve(chatPrompt, {
  model: languageModel,
  input: { mode: "support" },
  tools: Object.keys(tools),
});

The returned model reports generate/stream traces, stream progress, tool timing estimates, provider metadata cost, and parent-child links back to the prompt-resolution trace.

generate(prompt, options)

Execute a prompt using the Vercel AI SDK.

Returns: GenerateResult<TRaw, TOutput> — accumulated .text, optional complete .usage, optional .cost, .steps, .finalStep, provider-neutral .messages, retained ._meta, and typed .raw for the AI SDK result. With an output schema, .object is typed from the prompt schema.

stream(prompt, options)

Stream a prompt execution. Same options as generate(), except cascade() is not supported (cascade needs full results for tier evaluation). Streaming guardrails execute automatically on text streams through the Safety stream transform, so sentence-gated holds, rewrites, and blocks reach textStream consumers; constraints run report-only at end-of-stream and audits land on the completion meta.

Returns: StreamResult<TRawStream, TOutput> — canonical .textStream, typed .raw for the AI SDK stream result, and .completion, which resolves to the canonical completion envelope.

const result = await stream(editDraft, {
  model: openai("gpt-4o"),
  input: { instruction: "Fix the intro" },
});
for await (const partial of result.raw.partialObjectStream) {
  console.log(partial);
}

Stream results additionally carry a typed completion promise:

const result = await stream(chatPrompt, { model, input: { message } });
for await (const delta of result.textStream) process.stdout.write(delta);
const completion = await result.completion;
// { text, usage?, cost?, steps, finalStep, messages }

For AI SDK UI routes, pass the canonical result to the helper:

const result = await stream(chatPrompt, { model, input: { message } });
return createUIMessageStreamResponse(result);

Use pipeUIMessageStreamToResponse(result, options) for Node ServerResponse targets.

toParams(resolved, options) / fromResponse(response)

toParams() converts a ResolvedPrompt plus an AI SDK { model, settings?, extra? } object into generateText() args. fromResponse() normalizes an AI SDK generate result into Crux AdapterResponse facts.

These codecs are translation-only. Use managed generate()/stream() when Crux should run tools, approvals, validation retry, memory capture, safety, and observability.

prepare(prompt, options)

Prepare a headless AI SDK call. Crux resolves the prompt and returns the planned SdkGateway params without invoking the AI SDK. Feed the raw SDK result back with finish(response).

generate(prompt, { ...options, transport }) is the BYO-wire mode where Crux keeps owning the loop and invokes your callback for each SdkGateway request. AI SDK structured validation retry still happens when the transport throws the same validation/parse errors the SDK would throw. stream() with transport is intentionally unsupported and rejects with CruxTransportStreamUnsupportedError.

createCruxAi(options?)

Build a @use-crux/ai instance bound to a specific SdkGateway. The package-level exports are an instance created with the live gateway; reach for this when you need the test seam.

Returns: CruxAi — { generate, stream, prepare, generateTextFn, generateObjectFn, embedding, retrievalModel, reranker }

import { createCruxAi } from "@use-crux/ai";

const ai = createCruxAi({ gateway: myScriptedGateway });
const result = await ai.generate(myPrompt, { model, input });

createAiSdkLoopRuntime(gateway)

The low-level adapter from a SdkGateway to core's LoopRuntimePort — the gateway-closed runtime that aiSdkProviderRuntime binds and createCruxAi drives. Reach for it only when you build a bespoke runtime compiler or run the loopRuntimePortConformance() suite directly; most callers use createCruxAi or aiSdkProviderRuntime.

Returns: AiSdkLoopRuntime — a LoopRuntimePort<LanguageModel> with runTextLoop/runStructuredAttempt/runStream/replayStream.

CruxAIError

Coded error wrapper. generate()/stream() propagate underlying errors unchanged (existing ValidationExhaustedError/AggregateError handling keeps working); use CruxAIError.classify(error) at your boundary for a stable, machine-readable code.

generateObjectFn

Pre-bound GenerateObjectFn for @use-crux/core APIs (judges, extraction). Pass model per-call.

Signature: GenerateObjectFn — (opts: { model, system?, prompt, schema }) => Promise<{ object: T }>

generateObjectFn shares the same AI SDK structured-attempt mechanics used by generate() for prompts with an output schema: provider-specific schema sanitation, core-backed experimental_repairText, and router/cascade model resolution all happen before the helper returns { object }.

import { judge } from "@use-crux/core/scoring";

const evaluator = judge({
  id: "helpfulness",
  criteria: "Does the answer resolve the request?",
  scale: { min: 1, max: 5 },
  generate: generateObjectFn,
  model,
})

For a GenerateObjectFn that also runs through full adapter prompt execution semantics, use createGenerateObjectFnFromGenerate(generate) from @use-crux/core/compaction.

generateTextFn

Pre-bound GenerateTextFn for @use-crux/core APIs (summarization, compaction).

Signature: GenerateTextFn — (opts: { model, system?, prompt }) => Promise<{ text: string }>

embedding(config)

Create a dense Crux embedding backed by AI SDK embedMany().

const docsEmbedding = embedding({
  name: "docs-embedding",
  model: openai.textEmbeddingModel("text-embedding-3-small"),
  dimensions: 1536,
  maxInputTokens: 8192,
});

Use it with memory, indexers, or retrievers when you want the AI SDK model registry and provider abstraction, but still want a first-class Crux embedding object with batching and telemetry.

retrievalModel(config)

Create a RetrievalModel for model-backed recipe steps such as rerank() and compressToBudget().

import { retrievalModel } from "@use-crux/ai";

const model = retrievalModel({ model: openai("gpt-4o-mini") });

reranker(config)

Create a core Reranker backed by the AI SDK native rerank() endpoint.

import { cohere } from "@ai-sdk/cohere";
import { reranker } from "@use-crux/ai";

const engine = reranker({
  model: cohere.reranking("rerank-v3.5"),
  topN: 12,
});

Use this with rerank({ engine }) in retrieval recipes. Native reranking models are dedicated ranking calls; they do not spend generation tokens.

Tool approvals

When toolApproval requires approval for a tool call, generation suspends: the result has _meta.finishReason === 'tool_approval_required', pendingApprovals carries the minted requests (with anti-forgery tokens), and messages ends with the approval-request message. Persist the messages, collect a decision, append a tool-approval-response, and call generate() again.

import { appendToolApprovalResponse } from "@use-crux/core/tool-middleware";

const first = await generate(assistant, {
  model,
  input,
  tools,
  toolApproval: { deletePost: "always" },
});

if (first.pendingApprovals?.length) {
  const approval = first.pendingApprovals[0];
  const resumed = await generate(assistant, {
    model,
    input,
    tools,
    toolApproval: { deletePost: "always" },
    messages: appendToolApprovalResponse(first.messages, {
      approvalId: approval.approvalId,
      approved: true,
      approvalToken: approval.approvalToken,
    }),
  });
}

The approved tool executes during resume, its result is replayed into the loop as a normal tool round, and generation continues. Denied tools produce an execution-denied output the model can reason about.

What is not exported

AI SDK helpers and AI SDK types are no longer re-exported — import provider-native helpers from 'ai' directly when you need them under extra. Portable loop helpers (maxSteps, hasToolCall) come from @use-crux/core. Agent compositions come from @use-crux/core/agent or from loopRuntimeAdapter(). The legacy toMessages/fromMessages/createAIExecutor exports were removed (RFC use-crux/crux#28).

Types

import type {
  AIGenerateOptions,
  AIEmbeddingConfig,
  AIRerankerConfig,
  AiSdkLoopRuntime,
  CruxAi,
  CruxAiOptions,
  CruxAIErrorCode,
  SdkGateway,
  TextStreamResult,
  ObjectStreamResult,
  GenerateReturn,
  StreamReturn,
} from "@use-crux/ai";

Related

• Guide: Execution
• Reference: Retrieval
• Reference: Prompts
• Cookbook: Streaming with tools