Provider adapters

The provider adapter interface.

The gateway never talks to a provider directly. It talks to a ProviderAdapter. Adding a provider means writing one file.

OpenAI ships in v0.1.

import { getProvider, registerProvider } from '@meted/providers'

registerProvider(anthropicAdapter)

The interface

interface ProviderAdapter {
  readonly id: string
  readonly name: string

  isCompletionPath(path: string): boolean
  normalize(body: unknown): NormalizedRequest | null
  applyAllocation(input: ApplyAllocationInput): ApplyAllocationResult
  requestStreamUsage(body: Record<string, unknown>): {
    body: Record<string, unknown>
    clientRequestedUsage: boolean
  }
  buildUpstream(options: BuildUpstreamOptions): UpstreamTarget
  parseUsage(json: unknown): ProviderUsage
  parseFinishReason(json: unknown): string | null
  parseContentChars(json: unknown): number
  createStreamParser(options: CreateStreamParserOptions): StreamParser
  modelMaxOutputTokens(model: string): number | null
  describeError(status: number, body: string): ProviderError
}

What each method is for

normalize reduces a request body to the fields that decide answer size: model, messages flattened to text, the caller's own ceiling, whether it streams, tool names, response format. Return null for a body you cannot size, and the gateway proxies it verbatim.

applyAllocation writes the ceiling into a copy of the body. Mutating the caller's object is a bug. It also returns overheadTokens, the input tokens the adapter added, typically a length hint.

requestStreamUsage asks the provider for usage on streamed responses, and reports whether the caller had already asked.

createStreamParser measures a stream without buffering it. push returns the text to forward, so an adapter can drop an event. This is how the OpenAI adapter hides the usage event it requested on the caller's behalf.

describeError classifies a failure as auth, rate_limit, invalid_request, provider_error or http_error, which is what the meted dev log shows for a failed request.

The OpenAI adapter as a worked example

About 350 lines, and worth reading before writing another. Things it handles that are easy to miss:

• max_tokens vs max_completion_tokens. Reasoning models reject the former. The adapter follows whichever field the caller used, and otherwise picks by model family.
• Reasoning reserve. Hidden reasoning tokens bill against the completion ceiling, so the ceiling has to cover them.
• Multimodal content. content may be a string or an array of parts. Text parts are flattened; other parts are counted and flagged.
• Header hygiene. Hop-by-hop headers, Authorization and any inbound x-meted-* are stripped before forwarding.

Testing an adapter

packages/providers/src/openai.test.ts is the specification. An adapter should be able to demonstrate:

• A normalized request that captures the caller's ceiling from either field
• An allocation that does not mutate the input body
• A stream forwarded byte-identically
• A usage event stripped when the caller did not ask for it, and kept when they did
• Errors classified correctly