Semantic Cache

Import from the cache subpath:

import {
  createSemanticCache,
  semanticCachePolicies,
} from "@use-crux/core/cache";

`createSemanticCache(config)`

Creates a Crux plugin that wraps generate() and stream() calls for prompts that opt into cache.semantic.

createSemanticCache({
  store,
  embedding,
  ttl: 60_000,
  threshold: 0.95,
  scope: ({ input }) => `tenant:${input.tenantId}`,
});

Required fields:

Field	Description
`store`	`RecordStore` with TTL, dense vector search, and isolated semantic-cache namespace capability.
`embedding`	`DenseEmbedding`. Sparse and hybrid lookup are intentionally not accepted.
`ttl`	Maximum cache lifetime in milliseconds. Must be greater than zero.
`scope`	`'global'` or a function returning a safety boundary string.

Optional fields:

Field	Description
`threshold`	Minimum similarity score. Defaults to `0.95`.
`namespace`	Logical cache namespace inside the store. Defaults to `'default'`.
`shouldLookup`	Callback that can skip lookup.
`shouldCache`	Callback that can skip writes after generation.

Prompt-level cache.semantic can set mode, version, ttl, threshold, and query. Prompt TTLs only shorten the plugin TTL. Prompt thresholds only make matching stricter.

`semanticCachePolicies`

Built-in policy helpers:

Helper	Use
`finishReason(...allowed)`	Cache only selected finish reasons.
`noErrors()`	Cache only when no error is present.
`promptIds(ids)`	Apply lookup/write only to specific prompt ids.
`skipWhenToolsPresent()`	Disable lookup when tools are available.
`skipWhenToolCallsPresent()`	Disable writes when the model used tools.
`all([...])`	Compose policies with AND.
`any([...])`	Compose policies with OR.
`not(policy)`	Negate a policy.

Custom shouldLookup and shouldCache callbacks receive the prompt id, operation, input, prepared provider arguments, tools presence, version, and threshold. shouldCache additionally receives the generated result on write.

Dense-Only Embeddings

createSemanticCache() accepts a DenseEmbedding only; sparse and hybrid embeddings are rejected.

Sparse and hybrid retrieval are supported elsewhere in Crux through retriever() and VectorStore.search(), and they are excellent for document search. Response caching has a different job: decide whether a completed prompt result can be reused. That decision needs a single threshold with predictable meaning. Hybrid retrieval introduces fusion choices and adapter-specific ranking behavior, which makes a default response-cache threshold misleading.

If you need sparse or hybrid cache lookup, implement it as a custom cache or retrieval policy. Crux does not block that pattern; it only avoids presenting it as the default semantic response cache.

Events

Instrumentation hooks include:

onSemanticCacheLookupStart;
onSemanticCacheLookupEnd;
onSemanticCacheHit;
onSemanticCacheMiss;
onSemanticCacheWrite;
onSemanticCacheSkip;
onSemanticCacheReplayStart;
onSemanticCacheReplayEnd;

The devtools wire protocol uses matching semantic-cache:* event names.

Store Capability Rule

createSemanticCache() refuses stores that cannot prove both requirements:

records.capabilities().ttl !== false; // record store must support TTL
vectors.capabilities().filter === "pre"; // vector store must pre-filter

This is deliberate. Cache entries expire through record-store TTL, and lookups scope by namespace and version before ranking, so a vector store that only post-filters can let unrelated vectors crowd out cache entries. Keep semantic-cache entries in a dedicated vector namespace/index rather than sharing one with memory or RAG vectors.

inMemoryStorage() satisfies both by default. Durable adapters qualify when the record store supports TTL and the vector backend applies filters before ranking. Plain Redis stores do not expose vector search unless product-specific vector hooks are configured.

Convex bundled storage provides records but not vector search. Pair convexRecordStore() with an explicit pre-filter-capable VectorStore, such as upstashVectorStore() in a dedicated semantic-cache namespace.