Policies

The four modes, and the rules that hold in all of them.

A policy turns a Sufficiency decision into a concrete instruction for the provider. The engine estimates the output tokens the request needs; the policy decides which ceiling, if any, to send upstream.

The four modes

observe computes and reports the allocation without applying a ceiling.

conservative applies the engine's ceiling multiplied by 1.5, with a floor of 256 tokens. This is the highest of the three active ceilings, so it is the least likely to truncate and it saves the least.

balanced applies the engine's ceiling unchanged, with a floor of 96 tokens.

aggressive applies the ceiling multiplied by 0.8, with a floor of 48 tokens, and adds a length hint to the prompt that costs a few input tokens.

Each mode changes how often a response is truncated and how many output tokens are saved. Measure both on your own requests with meted eval before choosing one.

The three rules

These hold in every mode, including aggressive, and are not configurable.

Meted never raises a ceiling you set yourself. If your request carries max_tokens: 40, Meted may lower it or leave it alone. When your ceiling is already below what Meted would apply, Meted leaves the request alone and reports caller_max_lower.

Meted never allocates below the engine's target. The applied ceiling is always at least targetOutputTokens. Modes tighten the headroom above the target rather than the target itself.

Meted never acts on a decision the engine was not confident about. Below the mode's minConfidence, the request is forwarded unchanged and reported as low_confidence.

The length hint

aggressive appends one sentence to the system message:

Aim for roughly 28 words unless the task genuinely needs more.

The phrasing lets the model exceed it. A hint that reads as a hard cap produces truncated answers.

It costs input tokens, which are counted as optimizer overhead and subtracted from every reduction Meted reports.

Tuning a policy

Any part of a mode can be overridden:

// meted.config.json
{
  "mode": "balanced",
  "policy": {
    "ceilingMultiplier": 1.2,
    "floorTokens": 128,
    "minConfidence": 0.6,
    "lengthHint": false,
  },
}

A reasonable middle ground is balanced with minConfidence raised, so only clear-cut requests are touched.

Why not per-route policies?

A route-level ceiling is either too low for the hard requests on that route or too high for the easy ones, and usually both.