Types

The types you would touch when embedding Meted.

SufficiencyDecision

What an engine returns.

type SufficiencyDecision = {
  taskType: string
  complexity: 'trivial' | 'simple' | 'moderate' | 'complex' | 'open_ended'
  targetOutputTokens: number
  maxOutputTokens: number
  confidence: number
  rationaleCodes: string[]
}

EngineDecision is the same, plus an optional optimizerOverheadTokens for engines that spend tokens of their own.

SufficiencyRequest

Everything an engine is allowed to look at.

interface SufficiencyRequest {
  provider: string
  model: string
  messages: EngineMessage[]
  requestedMaxOutputTokens?: number | null
  stream?: boolean
  temperature?: number | null
  toolNames?: string[]
  responseFormat?: 'text' | 'json_object' | 'json_schema'
  capabilities?: ModelCapabilities
  metadata?: Record<string, unknown>
}

interface EngineMessage {
  role: 'system' | 'developer' | 'user' | 'assistant' | 'tool'
  content: string
  name?: string
  hasNonTextParts?: boolean
}

Tool names only. The current engine uses the names to detect that tools are available and does not read parameter schemas or descriptions, so a request whose expected answer length depends on a tool definition is sized from the messages alone.

SufficiencyEngine

interface SufficiencyEngine {
  readonly name: string
  readonly version: string
  readonly proprietary?: boolean
  evaluate(
    request: SufficiencyRequest,
    options?: { signal?: AbortSignal },
  ): Promise<EngineDecision>
  close?(): void | Promise<void>
}

EngineOutcome

What runEngine returns. It never throws.

type EngineOutcome =
  | {
      ok: true
      decision: EngineDecision
      durationMs: number
      engine: { name: string; version: string }
    }
  | {
      ok: false
      failure: EngineFailure
      durationMs: number
      engine: { name: string; version: string }
    }

interface EngineFailure {
  kind: 'timeout' | 'invalid_decision' | 'threw' | 'load_failed' | 'unavailable'
  message: string
}

Allocation

What a policy decides.

interface Allocation {
  mode: Mode
  policy: ModePolicy
  applied: boolean
  appliedMaxOutputTokens: number | null
  suggestedMaxOutputTokens: number
  skipReason?:
    'observe_mode' | 'low_confidence' | 'caller_max_lower' | 'engine_failed'
  lengthHint: string | null
}

suggestedMaxOutputTokens is populated in every mode, including observe, so the CLI can show what you are not yet doing.

ModelCapabilities

interface ModelCapabilities {
  id: string
  provider: string
  maxOutputTokens: number
  contextWindow: number
  reasoning: boolean
  reasoningReserveTokens?: number
  known?: boolean
}

known: false means the model was not in the registry and a conservative ceiling was used. It surfaces as the model_unknown rationale code.

RequestSummary

What the gateway knows about one finished request, passed to onRecord once the response has completed. It carries counts, durations and labels, and no message content.

interface RequestSummary {
  id: string
  ts: number
  provider: string
  model: string
  mode: Mode
  deployment: 'local' | 'cloudflare'
  stream: boolean

  taskType: string | null
  complexity: Complexity | null
  confidence: number | null
  rationaleCodes: string[]

  targetOutputTokens: number | null
  maxOutputTokens: number | null
  suggestedMaxOutputTokens: number | null
  appliedMaxOutputTokens: number | null
  allocationApplied: boolean
  allocationSkipReason: string | null

  inputTokens: number | null
  outputTokens: number | null
  reasoningTokens: number | null
  usageMeasured: boolean

  estimatedNaturalOutputTokens: number | null
  naturalEstimateSource: 'measured' | 'prior' | null
  optimizerOverheadTokens: number

  engineMs: number
  gatewayMs: number
  ttftMs: number | null
  providerMs: number
  totalMs: number

  engineName: string
  engineVersion: string
  engineFailed: boolean
  engineFailureKind: string | null

  finishReason: string | null
  status: number
  errorKind: string | null
}

ProviderAdapter

See Provider adapters.

Embedding the gateway

import { createGateway, type RequestSummary } from '@meted/gateway'
import { loadEngine } from '@meted/engine-interface'
import { loadConfig, resolveApiKey } from '@meted/config/node'

const { config } = loadConfig()
const { engine, source } = await loadEngine()

const gateway = createGateway({
  config,
  engine,
  engineSource: source,
  apiKey: resolveApiKey(config).key,
  // Send request summaries to your metrics system.
  onRecord: (summary: RequestSummary) => metrics.record(summary),
})

export default gateway.app