Patterns

Compose router, split, retry, fallback, and cascade into production model policies.

The routing primitives are intentionally small. Product behavior comes from composing named policies.

Retry Then Fallback

Retry transient failures on the preferred model before moving sideways.

const resilient = fallback([
  retry(gpt5, { id: "gpt5-retry", attempts: 2, backoff: "exponential" }),
  sonnet,
  { model: gemini, maxTokens: 2000 },
]);

Router To Cascade

Use a router for up-front task classification, then a cascade for result-aware quality escalation.

const productionModel = router({
  id: "support-routing-policy",
  classify: ({ input, context }: RouteArgs<AuthContext, SupportInput>) =>
    context.tier === "enterprise" || input.question.length > 2000
      ? "quality"
      : "fast",
  routes: {
    fast: fallback([gpt5nano, haiku]),
    quality: answerCascade,
    default: answerCascade,
  },
});

Canary Inside A Route

Use split() when one branch of a router needs a stable rollout.

const fastBranch = split({
  id: "fast-model-canary",
  seed: ({ context }: RouteArgs<{ sessionId: string }>) => context.sessionId,
  routes: {
    stable: { model: gpt4oMini, weight: 95 },
    canary: { model: gpt5mini, weight: 5 },
  },
});

Cascade With Fallback Tiers

Wrap tier models in fallback when provider resilience and quality escalation are both needed.

const productionCascade = cascade({
  id: "production-answer-cascade",
  tiers: [
    { model: fallback([gpt4oMini, haiku]), evaluate: fastQualityCheck },
    { model: fallback([sonnet, gpt5]), evaluate: strongQualityCheck },
    { model: opus },
  ],
});

Manual Overrides

Keep authored policy stable and pass request-specific controls at the call site.

await generate(prompt, {
  model: productionModel,
  input,
  routing,
  route: adminMode === "deep" ? "quality" : undefined,
});

Checklist

Give durable product policies a stable id.
Include default on routers.
Use routing: for app context, not prompt input.
Use route: for deterministic overrides.
Inspect result.routing instead of legacy _meta.router, _meta.cascade, or _meta.fallback.