OpenAI compatibility

What changes when you route through Meted, and what does not.

Meted speaks the OpenAI API. Point a client's baseURL at it and everything else stays the same.

const client = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
  baseURL: 'http://localhost:8787/v1',
})

What Meted handles

POST /v1/chat/completions is sized, allocated and forwarded.

Everything else under /v1 is proxied: /v1/models, /v1/embeddings, /v1/files, anything your provider offers. Method, query string and body are forwarded unchanged; headers are rewritten as described under What changes. Proxied responses carry no x-meted-* headers.

A POST to the completions path whose body is valid JSON but has no model field cannot be sized, and takes the same proxy path. A body that is not valid JSON is rejected with a 400 before any provider call. See Errors.

What is preserved

• Streaming. Every event is forwarded except a usage-only chunk Meted requested on your behalf. See Streaming.
• Tool and function calling, including tool_choice and parallel calls.
• response_format, JSON mode, and structured outputs.
• Multimodal content. Image and audio parts pass through untouched; the engine reads the text parts.
• Error bodies and status codes, as the provider returned them.
• Your max_tokens or max_completion_tokens, which Meted will never raise.

What changes

The credential. The inbound Authorization header is removed and replaced with the credential from the gateway's environment, so a client cannot use Meted to relay an arbitrary key.

Other headers. host, connection, content-length and accept-encoding are dropped and re-derived, inbound x-meted-* headers are dropped, and content-type: application/json is set. Any provider.headers from the config are added.

The ceiling, in an active mode. max_tokens, or max_completion_tokens for reasoning models, is set to the applied value. In observe mode no ceiling is set.

stream_options, on a streaming request where the caller did not set include_usage. This happens in every mode. See Streaming.

The system message, in aggressive mode only, which appends one sentence of length guidance. See Policies.

Response headers

A response from the completions path carries the decision:

x-meted-request-id          a19f0c4e91b3d27a
x-meted-mode                observe
x-meted-task-type           definition
x-meted-complexity          simple
x-meted-target-tokens       38
x-meted-max-tokens          64
x-meted-confidence          0.96
x-meted-engine-ms           0.41
x-meted-overhead-tokens     0
x-meted-applied             false
x-meted-skip-reason         observe_mode

When the engine failed, x-meted-engine-failure names the kind and the decision headers are absent. When a ceiling was applied, x-meted-applied-max-tokens carries the limit that was sent upstream. That header reports the limit, not whether the response reached it: check finish_reason for that.

These headers are written before the response body, so they carry nothing that is known only after the answer completes. Output tokens, finish_reason and latency reach the embedding process through onRecord.

import { metedRequestId } from '@meted/gateway/sdk'

const { data, response } = await client.chat.completions
  .create({ model, messages })
  .withResponse()

const id = metedRequestId(response)
const target = response.headers.get('x-meted-target-tokens')

Turn the decision headers off with gateway.responseHeaders: false. The request id header remains. The decision is still reported through the meted dev log and the onRecord summary.

Meted's own routes

/_meted/health is not authenticated and reports the configured mode, provider and engine. Block it at the edge on a public deployment.