Optimizer overhead
How optimizer tokens affect reported reductions.
Optimizer overhead is the input tokens Meted adds to a request. It is
measured per request and reported as optimizerOverheadTokens on the
onRecord summary and as
x-meted-overhead-tokens on the response.
What counts
Length-hint tokens. In aggressive mode Meted appends one sentence to the
system or developer message:
Aim for roughly 28 words unless the task genuinely needs more.
That is about 14 input tokens on the requests that receive it. The other three modes add no hint, so their overhead is 0.
Engine tokens. An engine reports what it spent through
optimizerOverheadTokens on its decision. The baseline engine is lexical and
makes no model calls, so it reports 0. An engine that called a model would
report a real number.
Where it appears
meted eval prints overhead as its own row and subtracts it from the reduction:
NATURAL METED
Output tokens 91,482 60,817
Median output 287 183
Median latency 1.4s 1.1s
Optimizer overhead - 412
Estimated net reduction 33.0%
Sufficiency 97.6%
Under-allocation 2.4%
Estimated net reduction is:
1 − (metered output + overhead) ÷ natural output
1 − (60,817 + 412) ÷ 91,482 = 33.0%
Both output totals come from the provider's usage figures for two real runs, so
this comparison is measured on both sides. The metric name retains Estimated
from an earlier release.
Two different comparisons
The DIFF column in the meted dev log and the reduction in a
meted eval report answer different questions, and only one of them involves
overhead.
meted dev DIFF | meted eval net reduction | |
|---|---|---|
| Compares | Actual output against the engine's target | Two runs of the same prompt |
| Both sides measured | No. The target is an estimate | Yes |
| Overhead | Not subtracted | Subtracted |
| Answers | How far the output was from the estimate | What a ceiling changed |
In observe mode nothing is added to the request, so overhead is 0 and the two figures differ only by what the ceiling did.
Input tokens and output tokens
Overhead is input tokens and the reduction is output tokens. Meted reports both as token counts and does not convert to currency, because per-token prices differ by provider and model.
For most providers output tokens cost several times more than input tokens, so subtracting them one for one reports a smaller net reduction than a price-weighted calculation would.