@use-crux/indexer

@use-crux/indexer owns the TypeScript source analysis used by @use-crux/local during crux dev, crux index reindex, and related index/lint flows.

It is an internal package, not a user-facing SDK surface. The public contract for index data remains @use-crux/core/project-index, and users interact with the read model through crux dev, crux lint, GET /api/project/index, or GET /api/index.

TypeScript compiler runtime

@use-crux/indexer is the Crux package that executes TypeScript-family compiler runtimes for Project Index source intelligence. The default semantic backend is verified with the stable JavaScript typescript package across the Crux support range (>=5.5 <7). Semantic cache identity records the selected compiler runtime identity, so native TypeScript-Go and JavaScript TypeScript runs do not share stale semantic fact caches.

TypeScript 7 native-preview support is intentionally separate from the public package type-surface preview lane. Crux exposes native-preview semantic indexing only as an explicit experimental backend until the TypeScript 7 compiler API is stable enough for default source-intelligence use.

Experimental native backend

Crux can run semantic Project Index enrichment through an experimental native backend. The default remains the JavaScript typescript compiler API backend. Enable the native path only through the top-level experimental config:

import { config } from "@use-crux/core";

export default config({
  experimental: {
    indexer: {
      native: true,
    },
  },
});

The current native engine is TypeScript-Go. When you need to point Crux at a specific native preview server binary, pass the path explicitly:

import { config } from "@use-crux/core";

export default config({
  experimental: {
    indexer: {
      native: {
        engine: "tsgo",
        tsserverPath: "/path/to/tsgo",
      },
    },
  },
});

Omit experimental.indexer.native, or set it to false, to keep the default TypeScript backend. Crux does not expose indexer.semantic, experimental_backend, or public unstableApi switches.

Both semantic backends produce compiler-free semantic evidence through the same backend interface, worker, cache, compiler, and read-model path. The TypeScript backend is still the default correctness baseline; the native backend is marked experimental because its current TypeScript-Go engine uses the upstream @typescript/native-preview API, which is unstable. Crux does not intentionally support a smaller semantic fact set for the native backend. Semantic backend changes must keep the parity matrix green for schema facts, source refs, relations, imported and conditional injection surfaces, tool maps, data-access intelligence, and semantic lint output.

When native semantic indexing is selected, TypeScript-Go owns semantic project setup, checker calls, declaration lookup, and AST traversal. Unsupported direct-projector shapes continue through the tsgo-backed shared analyzer path; they do not fall back to the JavaScript TypeScript semantic backend.

Internally, the native backend may use TypeScript-Go fast paths for source shapes it can prove exactly. Those fast paths are driven by internal primitive projection manifests where the shape is expressible as data, such as call names, definition identity fields, schema properties, local dependency relations, and source-ref roles. When a shape is unsupported by the native projector, Crux routes it through the native backend's complete shared semantic analyzer rather than emitting partial native-only facts. This applies to first-party and extension-authored primitives: native acceleration is optional, but backend parity is required.

Use crux config inspect to verify the selected config. The Project Model includes experimental.indexer.nativeAst, experimental.indexer.native, experimental.indexer.nativeEngine, and experimental.indexer.tsserverPath provenance when those flags are present.

Experimental native static AST compiler

Static Project Index extraction has a separate native beta:

import { config } from "@use-crux/core";

export default config({
  experimental: {
    indexer: {
      nativeAst: true,
    },
  },
});

experimental.indexer.nativeAst: true or { frontend: 'oxc' } lets @use-crux/local run the Go-orchestrated Rust/Oxc native static compiler for the AST/source pass. When the beta gates pass, the native compiler emits Project Index patch facts directly instead of sending Rust/Oxc syntax records through TypeScript projection. Node can still start for config/static-plan inspection and for TypeScript-authored extension or rule compatibility work. This flag does not select the native semantic backend; use experimental.indexer.native separately for TypeScript-Go semantic enrichment.

Native AST beta readiness is gated by exact normalized parity with the TypeScript Static Index baseline. The release gate builds the Rust/Oxc worker, runs Rust tests, runs the full @use-crux/indexer suite with CRUX_STATIC_INDEX_WORKER, runs repository static parity, runs Go production parity with CRUX_INDEXER_PARITY_REQUIRED=1, and verifies make local. TypeScript extension fallback is part of that gate: a configured TypeScript extension extractor or rule may run through the trusted Node host, but its facts still finalize through the same Project Index patch contract as native first-party output.

If a native AST run is degraded, inspect the indexing diagnostics before treating missing source facts as absent code. Common causes are a missing Rust worker binary, failed worker startup, configured TypeScript extension work that requires Node, or config import failures. Remove the flag or set experimental.indexer.nativeAst: false to return to the TypeScript Static Index baseline.

Responsibilities

discover authored Crux definitions in TypeScript and JavaScript source
resolve import-safe definitions in CRUX_INDEX=1 mode
statically recover prompts, contexts, tools, agents, flows, compositions, model routing parents (routing.router, routing.split, routing.retry, routing.cascade, routing.fallback) and their child definitions (routing.router.route, routing.split.route, routing.retry.target, routing.cascade.tier, routing.fallback.option), RAG, memory, workspaces, safety primitives, scorers, evals, and suites
statically recover prompt/context injection possibilities, including contributor-family definitions serialized with the legacy injectable taxonomy value, nested context use, when(...), match(...), guarded use entries, context tool contributors, and simple injectable return-object contributions
emit source locations, sourceRefs, authored paths, relations, runtime join hints, foldable-child index presentation hints, and intelligence metadata
connect routing children to their parents with router.includes_route, split.includes_route, retry.uses_target, cascade.includes_tier, and fallback.includes_option, and enrich them with compiler-resolved target relations such as router.route.uses_cascade, split.route.uses_prompt, retry.target.uses_prompt, cascade.tier.uses_fallback, and fallback.option.uses_agent; when proven, child definitions also expose metadata.targetKind and metadata.targetDefinitionId
project RouteArgs context evidence from router classify and split seed callbacks as routingContextType plus routingContextRequired, and preserve JSON-safe literal call-profile settings other than model as route-child profile facts; context-free callbacks and raw-model routes omit fields the compiler cannot prove
emit authored-graph lint findings, suppression metadata, and available rule descriptor metadata through the index snapshot
classify candidate files before AST parsing so generated bundles, base64 artifacts, and oversized non-authored files do not consume the local devtools heap

Boundary

@use-crux/core/project-index owns the JSON-safe Project Index contract. @use-crux/indexer emits facts and semantic presentation hints, but does not own persistence, HTTP routes, subscriptions, or concrete UI layout. The @use-crux/local Go service preserves and serves the merged read model; the @use-crux/devtools web UI and local TUI render it. definition.metadata.indexPresentation lets those clients fold first-class child records (flow.step, routing routes/tiers/options/targets, composition branches/stages, RAG stages, memory blocks/stores) under their authored parents without removing the child definitions from search, relations, lints, or direct inspection.

The serialized Project Index taxonomy still uses the injectable kind, family, relation, and lint vocabulary for this contributor-family surface. That legacy name is retained for cross-runtime persisted contracts; the stable authoring API for custom contributions is contributor().

The indexer is intentionally layered. The fast AST/source pass should return a useful Project Index quickly, while deeper TypeScript semantic analysis is background enrichment. Semantic analysis may use a long-lived ts.Program / TypeChecker to resolve aliases, barrels, imported symbols, nested schemas, callbacks, and primitive graph edges that are not safe to infer from syntax alone. For injection intelligence, that includes imported contributor-family definitions serialized as injectable, imported injectable input schemas and inject callback source refs, import-safe prompt/context/injectable use arrays with spreads, resolved useEntries for imported/spread arrays and helper-shaped conditional entries, condition-specific source refs for when(...), match(...), and guarded && use entries, imported/spread tool maps, simple injectable inject functions that return tool maps, and returned constraints/guardrails/metadata keys. Computed semantic use/tool shapes are kept as dynamic or partial facts where possible, so clients can show that the indexer saw the injection surface even when it could not safely resolve every target.

Bundled source indexing is owned by Crux Local: Go drives the Rust/Oxc Static Index compiler, Rust emits first-party facts and bundled lint findings, and Node is used for config/runtime/semantic work plus third-party Static Index extension evidence. The @use-crux/indexer package is the extension-authoring SDK and JSON-safe contract package, not a public in-process bundled indexer.

resolveProjectModel(...) returns the ResolvedProjectModel read model from @use-crux/core/project-index. It is the config-inspection surface used by crux config inspect: selected root, package name, config status, source roots, ignored conventions, definitions, Quality discovery defaults, and diagnostics all carry provenance so config remains policy/override state rather than a duplicate primitive registry. Missing config is reported as source-only discovery, while selected lint findings such as missing stable ids and runtime-dependent tool maps become Project Model diagnostics with source provenance.

@use-crux/indexer/extensions is an experimental extension boundary. It models the Project Index compiler with role-based slots such as extractors, resolvers, rules, and emitters. Extractors return immutable intermediate facts and unresolved references; Crux resolves, validates, merges, projects, caches, and emits index snapshots or patches. Explicit allowlisted package loading is supported for extractor/relation-spec manifests. Compiler profiles, compiler-owned projections, runtime construction details, custom third-party rules, resolvers, emitters, sources, parsers, and sandboxing remain internal/reserved.

Stability

Stable beta surfaces are granular. The package root, @use-crux/indexer/testing, @use-crux/indexer/source-resolver, and JSON-safe contracts/* subpaths are intended to avoid breaking shape changes after the pre-launch rename/cache-bump phase. @use-crux/indexer/extensions remains experimental: its ExtractContext reader/builder object is frozen, but third-party loading, trust UX, custom rule execution, and fixture package contracts are not stable plugin ecosystem promises yet. @use-crux/indexer/host/* subpaths are Crux-owned worker bridges and should not be treated as application SDK APIs.

The loading model is config-first and trust-aware. @use-crux/core accepts inert indexer configuration for extension references, trust policy, and rule options. The indexer exposes loadIndexerExtensionReferences(...) for package-resolution/import and resolveIndexerExtensionReferences(...) for pure manifest-only validation. The loader preflights package-name trust before import, reads installed package metadata, checks requested package versions, and reports trust, version, manifest, or compatibility diagnostics before a compiler profile receives an extension.

The package export map is intentionally small: @use-crux/indexer, @use-crux/indexer/extensions, @use-crux/indexer/testing, and @use-crux/indexer/source-resolver. Internal indexer/* modules are not public package entry points.

The host package exposes Crux-owned worker bridges for static planning/config, semantic enrichment, runtime patching, and Static Index extension-host calls. It also exports the relation model facade used internally: resolveRelationModel, relationIdentity, mergeRelationsByIdentity, createRelationPolicyTable, and relationDiagnosticsFromReport, plus withResolvedRelationReadModel for callers that already have resolved relations and only need the compiler's definition projection. These helpers are the supported way to keep relation identity, fidelity-aware replacement, policy-table validation, and unresolved-reference accounting aligned with compiler behavior. Semantic analyzer output is merged with the same relation identity contract, so resolved semantic facts replace provisional static facts instead of producing duplicate graph edges. Relation types are expected to be declared in the active policy table before they reach compiler output. Undeclared relation refs or pre-resolved relations are preserved as evidence but reported through RelationResolutionReport.policyGaps and relation.policy_gap diagnostics. Runtime use entries that name ambient resources are resolved from RuntimeUseTargetRules; app or host-specific aliases belong in that data rather than in SDK resolver code. Use @use-crux/indexer/testing for extension tests; it wraps that same engine with an in-memory source reader, disabled cache, the TypeScript syntax-record producer as the fixture default, and trace.syntaxFrontend reporting for the producer that ran.

The experimental extractor context exposes compiler-owned readers and builders for the current migration path: argument reads, static object reads, schema projection, definition/reference builders, and source-ref builders for properties, callbacks, schemas, template interpolations, and helper functions. Its public shape is frozen as extension, extractor, match, source, args, config, define, ref, and sourceRef. Raw TypeScript nodes remain an unstable first-party escape hatch on internal module paths only.

See Writing an Indexer Extension for the author walkthrough and @use-crux/indexer/extensions for the API reference.

Crux Local's Go-owned incremental indexer consumes the graph-backed planner and applies planner-approved source closures to the live Project Index snapshot. In AST-only runs it produces exact-invalidation static patches; semantic runs follow with TypeScript semantic enrichment for known index-owning source files and semantic source-ref support files. Unknown files, config changes, old snapshots, unresolved imports, and first-run support files without durable source graph evidence fall back to the full Go/Rust static reindex path.

The local Go service can now call the incremental worker with a previous index plus changed/deleted file sets and apply the returned ordered patches to the live index state. The Go patch applier honors exact file/definition invalidation and unions source-row evidence across AST and semantic phases. POST /api/project/index/reindex and POST /api/index/reindex accept optional files and deletedFiles arrays for delta-triggered incremental refreshes. During crux dev, a Go fsnotify watcher debounces source/config changes and feeds the same incremental bridge.

Semantic enrichment is organized as focused analyzers with a shared semantic evidence contract. semanticIndexFacts(root, files) remains the top-level behavior, while analyzers handle narrower responsibilities such as schema enrichment, source references, relations, and definition enrichment. Shared semantic infrastructure lives under indexer/semantic/: program setup, candidate discovery, analyzer input selection, registry wiring, result merging, and evidence projection are separate from the analyzer implementations. Analyzer-level tests should use real temporary TypeScript source when compiler resolution is part of the behavior.

Indexer Extensions remain backend-neutral. Extension authors use Crux manifests, extractors, rules, facts, and read models; semantic backends do not expose raw TypeScript or TypeScript-Go AST/checker objects as a public extension API. Extension primitives that do not have native projector coverage still work through the native shared analyzer path when native indexing is enabled.

The Go service remains the owner of the final read model. Today it serves and broadcasts the index snapshot with explicit indexing status, including failed indexing states when the worker cannot produce a snapshot. AST, semantic, and incremental workers emit index patches against that same Go-owned read model. Source-indexer workers emit index facts; they do not make UI-specific presentation decisions.

Index Facts Contract

The indexer emits facts into the public @use-crux/core/project-index contract. Clients should render these fields directly and should not parse source snippets or rebuild architecture client-side.

Every ProjectDefinition can expose the following fact buckets:

definition.metadata.inputSchema, outputSchema, argsSchema, configSchema, and schema for direct contract access.
definition.metadata.intelligence.contract for normalized args/input/output/config/schema summaries, nested schema refs, and field summaries.
definition.metadata.intelligence.control for how the primitive runs: mode, ordering, child ids, retry/fallback policy, suspension points, and budgets.
definition.metadata.intelligence.data for visible memory, blackboard, workspace, store, block, artifact, and retrieval reads/writes.
definition.metadata.intelligence.dependencies for local detail-panel summaries of prompt/context/tool/agent/flow/memory/store/router dependencies. The canonical graph remains index.relations.
definition.metadata.facts.useEntries and definition.metadata.facts.tools for attributed prompt/context/injectable injection possibilities when they are visible statically. These facts preserve conditionality hints such as when, match-case, match-default, and binary-guard; runtime traces remain the source of truth for which contributions activated for a concrete input.
definition.metadata.intelligence.runtime and definition.metadata.runtimeJoin for authored-to-runtime span/resource join hints.
definition.metadata.facts for typed primitive-specific summaries such as prompt shape, agent prompt/tool/handoff lists, flow step names, composition participants, routing route/tier/option counts, memory block/store summaries, workspace mounts, and eval coverage hints.
definition.sourceRefs for source-backed schema, callback, handler, constant, template, prompt fragment, tool-map, store, block, evaluator, classifier, injection condition, and route-target declarations.
index.relations for architecture edges such as agent.uses_prompt, flow.includes_step, flow.step.reads_memory, memory.includes_block, router.includes_route, and eval.covers_definition.
index.lintFindings for backend-owned authored-graph issues produced from those facts.
index.ruleDescriptors for backend-owned rule metadata, including built-in rules and extension-provided rules that did not fire a concrete finding.

Known fact fields are typed in @use-crux/core/project-index. Future integrations may use explicit extensions bags, but core Crux primitives should prefer typed fields whenever the indexer can prove the fact.

Cache Versioning

Project Index caches are local implementation details, but their identities are part of Crux's migration discipline. Static cache keys include source hashes, direct import dependency hashes, config boundary hashes, extension/extractor/rule identity, compiler profile/intrinsic identity, and the TypeScript syntax frontend version. Semantic cache keys include the analyzed source closure, config boundary hashes, selected compiler runtime identity, and the semantic compiler-options identity. If indexer code would produce different facts for unchanged user source and those structured inputs do not already change, the matching cache epoch must change in the same patch.

Bump packages/indexer/src/indexer/cache-identity.ts STATIC_PARSE_CACHE_EPOCH when static AST extraction changes definition, relation, metadata, schema, source-ref, diagnostic, path/source, file-classification, or presentation-hint output.
Bump packages/indexer/src/indexer/cache-identity.ts SEMANTIC_FACTS_CACHE_EPOCH when semantic enrichment changes schema/callback/source-ref resolution, runtime joins, intelligence metadata, relations, lint facts, compiler runtime identity, or compiler-option meaning.
Bump packages/local/internal/projectindex/cache/identity.go ProjectIndexSnapshotCacheEpoch when an existing .crux/cache/index-v2/epoch-* snapshot could hide a new IndexData / ProjectDefinition read-model field or changed cache semantics after restart.

Moving semantic logic between analyzers without changing emitted facts does not require a semantic cache version bump.

Changes that span AST facts, semantic facts, and the Go snapshot should bump all three. After rebuilding, restarting, and running crux index reindex, the new snapshot should appear without asking users to delete .crux/cache.

Reference: Project Index
Reference: Devtools Integration
Reference: @use-crux/local

@use-crux/indexer

On this page