Token Management

Assembled prompts can outgrow the model's context window or your cost budget. Set tokenBudget and Crux drops the least important contexts first: it preserves the prompt's own system text and any provider-cache prefix, then drops the lowest-priority uncached contexts until the total fits. Provider-specific adaptations let you tweak prompts per model without breaking portability.

import { prompt, context } from '@use-crux/core'
import { generate } from '@use-crux/ai'

const critical = context({ priority: 100, system: '## Critical Rules' })  // dropped last in the uncached tail
const guidelines = context({ priority: 50, system: '## Guidelines\n...' })
const examples = context({ priority: 20, system: '## Examples\n...' })    // dropped first

const myPrompt = prompt({
  use: [critical, examples, guidelines],
  system: 'You are an assistant.',
})

const result = await generate(myPrompt, {
  model,
  input: { ... },
  tokenBudget: 2000,
})

How it works

1. Conditional contexts are evaluated first: when predicates, when() wrappers, and match() specs are resolved. Excluded contexts are never even resolved.
2. The prompt's own system text is always included.
3. Cached contexts (cache: true) form a stable prefix after the prompt's own system text and are never budget-dropped.
4. Uncached context contributions stay in use order for rendering, then the lowest-priority uncached contexts are dropped until the total fits within budget.
5. Use .inspect() to see exactly what was dropped, excluded, and why

Conditional exclusion vs token-budget dropping

These are two different mechanisms with different semantics:

Use when/match when a context is irrelevant to the current request (wrong mode, missing data, feature disabled). Use priority for graceful degradation under token pressure.

// Conditional: don't include research context if there's no research data
const research = context({
  input: z.object({ synthesis: z.string().optional() }),
  when: ({ input }) => !!input.synthesis, // excluded when no data
  priority: 40, // AND droppable under pressure unless it uses cache: true
  system: ({ input }) => `## Research\n${input.synthesis}`,
});

Custom tokenizer

The default tokenizer estimates tokens as chars / 4. For accurate counts, provide a real tokenizer via config():

config({
  generation: {
    tokenizer: (text) => encode(text).length,
  },
});

Or standalone: setTokenizer((text) => encode(text).length)

Context Caching

Contexts with expensive async resolvers can use memo to skip redundant resolver calls. Stable blocks can also set cache: true to request provider-level token caching and join the stable prefix.

const brand = context({
  id: "brand-voice",
  system: async ({ input }) => fetchBrandProfile(input.orgId),
  memo: { ttl: 300_000 }, // 5min resolver memoization
  cache: true, // provider cache hint
});

See the Context Caching guide for full details on resolver memoization, provider behavior, and observability.

Provider-specific adaptations

Different models sometimes need different prompting strategies. The adapt field lets you apply provider-specific tweaks:

const myPrompt = prompt({
  system: "You are a helpful assistant.",
  adapt: {
    anthropic: {
      appendSystem: "\nReturn raw JSON, no markdown fences.",
    },
    openai: {
      prependPrompt: "Think step by step.\n\n",
      settings: { temperature: 0.1 },
    },
    "*": { appendSystem: "\nRespond with valid JSON only." },
  },
});

Resolution priority: exact provider match > modelId prefix (for OpenRouter-style routing) > '*' wildcard.

System adaptations participate in the same block model as prompt and context text. Prompt-owned system text remains first; prependSystem is inserted after that own-system block when it exists, and appendSystem is appended at the end. In messages mode, the final adapted system text is folded into the messages array instead of returning a parallel system field.

Settings merge with priority: config.settings < adapt.settings < call-site overrides.

Next steps