Context Caching

Context resolvers that call databases, RAG systems, or APIs run on every prompt.resolve() call. Crux names the two caching layers separately:

1. memo: { ttl }: skip the resolver function when the same declared inputs produce the same output.
2. cache: true: mark the resolved block for provider-level token caching.

Quick Start

import { context } from "@use-crux/core";
import { z } from "zod";

const brand = context({
  id: "brand-voice",
  input: z.object({ orgId: z.string() }),
  system: async ({ input }) => {
    const data = await fetchBrandProfile(input.orgId); // expensive DB call
    return `## Brand Voice\n${data.guidelines}`;
  },
  memo: { ttl: 300_000 }, // app-side resolver memoization
  cache: true, // provider cache hint
});

The first call runs the resolver and memoizes the result. Subsequent calls with the same orgId within 5 minutes skip the resolver entirely. Adapters mark the block for provider caching; see the Anthropic adapter reference for exact marker placement.

Option Forms

Fine-Grained Control

Use the two fields independently when the layers need different settings:

// Cache the resolver but DON'T mark as a provider cache breakpoint
// (content varies too much for prefix caching)
const searchResults = context({
  id: "search",
  input: z.object({ query: z.string() }),
  system: async ({ input }) => ragSearch(input.query),
  memo: { ttl: 30_000 },
});

// Provider cache only, cheap to compute but stable across calls
const rules = context({
  id: "rules",
  system: "## Rules\nAlways respond in JSON.",
  cache: true,
});

How It Works

Application-Level (Resolver Caching)

When memo.ttl > 0:

1. A cache key is computed from contextId + a stable hash of the input fields declared in the context's inputSchema
2. On cache hit: the cached text is returned, systemFn() is never called, and inspect/artifact metadata reports servedFrom: "memo", the original resolvedAt, and age
3. On cache miss: systemFn() runs, the result is stored with memo.ttl, and metadata reports servedFrom: "live" plus resolvedAt
4. Unrelated prompt-level input fields don't affect the cache key

Provider-Level (Token Caching)

When cache: true:

1. The resolution pipeline includes systemBlocks on ResolvedPrompt: one block per system contribution, each with a providerCache flag and a single cacheBoundary marker on the final cached block
2. Cached contexts form a stable prefix after the prompt's own system text; uncached contexts form the droppable tail. Token budgets can drop only the uncached tail.
3. @use-crux/anthropic converts blocks to Anthropic TextBlockParam[] with cache_control: { type: 'ephemeral' } on the cacheBoundary block
4. @use-crux/google creates server-side CachedContent objects for the cacheable system prefix via Google's caching API, then references them in generateContent() calls while sending any uncached remainder as systemInstruction. A single CachedContent lifecycle handles creation, reuse, per-call TTL, concurrency dedup, and graceful error fallback automatically; configure it (TTL, max entries, onError: 'throw', a custom cache port, or a fully custom lifecycle) via createGoogle(client, { cachedContent }). See the @use-crux/google reference.
5. @use-crux/ai (Vercel AI SDK) converts blocks to SystemModelMessage[] with providerOptions: { anthropic: { cacheControl: { type: 'ephemeral' } } } on the cacheBoundary block when the model is Anthropic
6. OpenAI prefix caching works automatically: Crux's deterministic context ordering already provides stable prefixes

Requirements

• id is required when memo is set: needed for cache key derivation. An error is thrown at definition time if missing.
• Static contexts cannot use memo because there is no resolver call to memoize. Use cache: true on stable static content for provider prefix caching.
• Short memo with provider cache warns when cache: true and memo.ttl < 300000, because the provider may reuse the block longer than your declared freshness tolerance.
• Dynamic prompt system with cached contexts warns because provider cache prefixes must be byte-stable before the native breakpoint.

Observability

Resolved contributions carry freshness facts through .inspect() and context.contribution artifacts:

• Live resolutions report servedFrom: "live" and resolvedAt
• Memo hits report servedFrom: "memo", the original resolvedAt, and age
• Structured segments can also carry observedAt and sourceVersion when a primitive already knows source freshness

compilePrompt().inspect() uses the same memo-cache path as .resolve(), but runs with quiet observability and instrumentation ports so inspection does not emit spans, artifacts, or cache events.

Next Steps