Safety

import {
  boundary,
  constraint,
  createSafety,
  createSafetyPlugin,
  defaultConstraintFeedbackFormatter,
  evaluateConstraint,
  evaluateGuardrail,
  guardrail,
  toolPolicy,
} from "@use-crux/core/safety";

@use-crux/core/safety is the runtime policy boundary engine. Policies are authored as frozen objects and executed through one per-call Safety session. Adapters construct that session internally; application code usually authors policies and attaches them globally, to prompts/contexts, or per call.

Boundaries

Policies bind to typed boundary descriptors:

Boundary ids are stable decision/config ids. boundary.stream.* is not a public boundary family; streaming is configured on output guardrails.

Multimodal content

Both media helpers infer { part, origin }. Narrow part.type for canonical variant fields and origin.kind for original message, step, or completed- operation coordinates. Indexes refer to the pre-strip canonical arrays and do not shift when an earlier policy removes a sibling.

Media policies support allow, warn, block, and strip; the latter three require a reason. Enforced strips remove optional canonical parts, while report mode records intent without mutation. Removing required media—such as the last generated image, generated speech audio, transcription audio, or an edit mask's final reference—escalates to a block.

See Safety for content primitives for the full primitive/boundary matrix, completed-operation write-back rules, step coverage, and stream temporal caveat.

boundary.input.text() remains the projected-text boundary. Text parts remain verbatim while media becomes a bounded descriptor containing its kind and MIME type plus the full URL or byte size and a 12-character SHA-256 prefix. Payloads over 256KB use sha256:omitted, Blob hashes use sha256:unavailable, and file descriptors include the filename.

Guardrails

guardrail(config) creates a policy that can allow, block, warn, rewrite, or hold stream segments. Media guardrails use strip instead of rewrite or hold.

const pii = guardrail({
  id: "pii",
  category: "pii",
  on: [boundary.input.text(), boundary.output.text()],
  run: guardrail.pii({ strategy: "mask" }),
});

Built-in strategies:

• guardrail.pii({ strategy })
• guardrail.secrets()
• guardrail.injection({ action })
• guardrail.classifier({ classifier, blockWhen, findings })
• guardrail.media({ mediaTypes, size, sources, action })

Text strategies default to sentence-level streaming checks. Classifier and object/path guardrails run at final output unless tuned.

guardrail.media(options)

Creates a callback restricted to media-only boundaries, including an ordered input/output media tuple. At least one rule group is required. Configured rules run in stable mediaTypes → size → sources order and aggregate all failures into one privacy-safe decision.

Top-level options:

MIME and size rules:

Source rules:

sources: {} enables all source defaults in the table; omitting sources disables source evaluation entirely. Invalid configuration throws SafetyConfigError when the strategy is constructed.

MIME inspection prefers the declared part.mediaType, then source-owned metadata, then a data-URL type. A bounded image signature sniff is used only for undeclared local bytes; it does not verify a declared MIME type against payload contents. Size inspection uses local payload bytes or declared asset/provider metadata. It never fetches remote URLs or calls providers.

The exported public types are MediaGuardrailOptions, MediaTypeGuardrailRule, MediaSizeGuardrailRule, MediaSourceGuardrailRule, MediaGuardrailAction, and MediaTypePattern. The returned callback carries normalized, frozen runtime metadata as { kind: 'guardrail.media', config }. Project Index records authored helper calls with static strategy kind media and includes config only when the whole argument is statically resolvable.

Reasons and observability can contain MIME types and byte counts, but never raw media, URLs, observed hostnames, paths, queries, userinfo, filenames, or provider/file identifiers.

Constraints

constraint(config) creates a retryable semantic assertion. Constraints run on guarded output candidates.

const citeSources = constraint({
  id: "cite-sources",
  on: boundary.output.text(),
  severity: "assert",
  maxRetries: 2,
  run: async (text) =>
    text.includes("[1]")
      ? { pass: true }
      : { pass: false, feedback: "Include at least one citation." },
});

Built-in strategies include constraint.judge({ judge, minScore }) and constraint.citations(...). Use judge() from @use-crux/core/scoring to create the judge instance, then bridge it through constraint.judge(...) for runtime enforcement.

Tool Policies

toolPolicy() is the Safety-owned surface for tool decisions. It returns ordinary tool middleware so it fits the adapter protocol, but blocked and approval-requested tools carry Safety decision evidence, Project Index rows, and Devtools Explain entries.

const blockDeletes = toolPolicy({
  id: "block-delete",
  match: { tool: "deleteUser" },
  action: "block",
  reason: "Delete user requires an operator override.",
});

const screenResults = toolPolicy.result({
  id: "screen-customer-result",
  match: { tool: "lookupCustomer" },
  run: async ({ output }) =>
    typeof output === "string" && output.includes("ssn")
      ? { action: "block", reason: "Tool result contained sensitive data." }
      : { action: "allow" },
});

Use toolMiddleware() instead for execution plumbing such as logging, timing, caching, retry wrapping, or argument normalization.

Registration

Global policies are installed with createSafetyPlugin():

import { config } from "@use-crux/core";
import { createSafetyPlugin } from "@use-crux/core/safety";

config({
  plugins: [
    createSafetyPlugin({ guardrails: [pii], constraints: [citeSources] }),
  ],
});

Prompt, context, and call-site policies compose into the same registry. Duplicate policy ids are invalid by default. Use one guardrail with an on array when the same policy should apply to several boundaries.

Consumption

Adapters consume safety through createSafety(). Custom adapter dialects call:

1. guardInput() before the provider call.
2. finalizeOutput() after schema validation.
3. stamp() before returning metadata.
4. openStream() for streamed text.

const safety = createSafety({
  call: opts,
  resolved,
  promptId,
  model,
  systemPrompt,
});

({ messages } = await safety.guardInput({ messages }));

const final = await safety.finalizeOutput(output, regenerate, {
  suspended: finishReason === "tool_approval_required",
  messages,
});

const meta = safety.stamp(traceMeta);

finalizeOutput() applies output guardrails before constraints. Regenerated candidates are guarded before constraints re-check them. Structured output rewrites keep returned text and object synchronized or fail closed.

openStream() exposes the streaming sub-protocol:

• feed(chunk) returns { kind: 'emit', content } or { kind: 'hold' }.
• finish() runs final-only guards, report-mode constraints, and returns the final seal with any unreleased pending text.
• transform() returns a TransformStream<string, string>.

Tuning And Capture

Per-call safety.tune can change policy posture without replacing policy logic:

await adapter.generate(prompt, {
  input,
  safety: {
    tune: {
      pii: { mode: "report" },
      "cite-sources": { enabled: true },
    },
  },
});

Allowed tune fields are mode, stream, and enabled. Unknown ids throw.

Safety artifacts participate in observability.capture. Use capture presets and per-artifact overrides when traces leave a trusted boundary. Public audit, errors, and observability records are safe-by-default. Media audit entries add the canonical part type and original pre-strip indexes, never the media source.

Choosing The Primitive

Testing

Use evaluateGuardrail() and evaluateConstraint() for focused policy tests. Use Eval decision-report matchers for runtime behavior:

ctx.expect.decisionReport.safety.toHaveOutcome("pii", "rewrite");

Security Utilities

Input sanitization helpers ship from the core root, not the safety subpath:

import {
  safe,
  raw,
  limit,
  wrap,
  escapeXml,
  truncate,
  userContent,
  detectSuspiciousPatterns,
} from "@use-crux/core";

escapeXml(text)

Replaces the five characters that can break out of XML tag attributes or content:

truncate(text, maxLength?)

Truncates a string to a maximum length. Default maxLength is 10,000. When the text exceeds the limit, the result is sliced and a … [truncated] suffix is appended:

import { truncate } from "@use-crux/core";

truncate(longArticle); // → first 10,000 chars + "… [truncated]"
truncate(userQuery, 500); // → first 500 chars + "… [truncated]"

raw(text)

Returns a SafeWrapper that safe will not escape. Use only inside safe templates, for content you have already sanitized or that comes from a trusted source:

safe`
  System-generated HTML: ${raw(trustedHtml)}
  User query: ${userQuery}
`;
// trustedHtml is interpolated as-is; userQuery is escaped

raw() bypasses all escaping. Only use it when you are certain the content is safe.

limit(text, maxLength?)

Combines truncate and escapeXml in a single call. Returns a SafeWrapper: use only inside safe templates:

safe`Query: ${limit(userQuery, 500)}`;
// userQuery is truncated to 500 chars, then XML-escaped

wrap(text, tag?)

Escapes the text and wraps it in <user-input> delimiters. Returns a SafeWrapper: use only inside safe templates. Delimiters make it clear to the model where user content starts and ends, reducing the chance of instruction following from injected content:

safe`Instruction: ${wrap(instruction)}`;
// → Instruction: <user-input>escaped content</user-input>

safe`${wrap(feedback, "customer-feedback")}`;
// → <customer-feedback>escaped content</customer-feedback>

Never use wrap() in regular template literals: it returns an object, not a string, so you get [object Object] in your prompt. Use userContent() instead for regular templates.

userContent(text, tag?)

Standalone version of wrap() that returns a plain string. Escapes the text and wraps it in <user-input> delimiters. Use this in regular template literals or anywhere outside safe templates:

const prompt = `Instruction: ${userContent(instruction)}`;
// → Instruction: <user-input>escaped content</user-input>

const prompt = `Feedback: ${userContent(feedback, "customer-feedback")}`;
// → Feedback: <customer-feedback>escaped content</customer-feedback>

detectSuspiciousPatterns(text)

A dev-time heuristic that returns an array of warning strings. It checks for three categories of suspicious input:

import { detectSuspiciousPatterns } from "@use-crux/core";

const warnings = detectSuspiciousPatterns(userInput);
if (warnings.length > 0) {
  console.warn("Suspicious input:", warnings);
}

detectSuspiciousPatterns is a heuristic, not a guarantee: treat it as an extra layer of defense, not the only one.

Security warning records

When generation.securityWarnings is enabled (the default outside production), suspicious input patterns are emitted as canonical security.warning graph records (and related security.report artifacts) with full trace correlation: each warning links to the generate() call that triggered it. emitSecurityWarningSpan opens a real span, not a free-floating span:event. Subscribe to the graph stream for custom integrations:

import { subscribeObservability } from "@use-crux/core/observability";

const unsubscribe = subscribeObservability((record) => {
  // emitSecurityWarningSpan opens a real span (not a free-floating span:event).
  if (record.type === "span:start" && record.primitive === "security.warning") {
    // attributes: promptId, field, pattern, inputPreview
    myLogger.send(record);
  }
  if (record.type === "artifact" && record.kind === "security.report") {
    // JSON preview carries severity, message, field, pattern
    myLogger.send(record);
  }
});

// later: unsubscribe()

Related

• Guide: Safety
• Guide: Guardrails
• Guide: Constraints
• Guide: Security