Iris 0.14.0The dashboard is a product.

Added

• Labels exist, and a rule's local precision changes its published number on this deployment at twenty labels. On any trace, every fired rule asks whether it was right to fire; POST /api/v1/labels writes your answer on that fire (a rule that did not fire is refused — labels are written on fires, so they measure precision only, and no surface ever says "local accuracy"). Right ÷ (right + wrong) over your labels is the rule's local precision with a 95% Wilson interval, and at twenty labels it is in force: a fire of that detection or inference carries uncertainty: { basis: "local_labels" } instead of the published positive predictive value, the risk estimate reads it in place of the published number (its assumptions say so, and a stored row re-composes on read from its own stamp), and the prior is estimated from your labels — *f·p̂ / sensitivity* from the rule that fires on a fraction *f* of your traffic — unless you set eval.prior, which always wins; every verdict's provenance.composer now records the prior and whose it was. GET /api/v1/labels/stats is the table (/rules shows it: labels, local precision, in force, published, fire rate) with the estimated prior and which rule to label next (the traffic-weighted narrowing of the interval); GET /api/v1/issues groups fires by rule and by what the rule found, so ten fires of one pattern read as one issue with a count (the Failures page lists them); POST /api/v1/evaluations/:id/reevaluate re-scores a trace under the rules and labels as they stand now, keeps the earlier row and names it in provenance.supersedes (the trace page has the button). Migration 011 adds verdict_labels; the engine reads the labels at startup and after every write.
• A `regression-alarm` moment kind exists and the moments filter accepts it. Drift compares two windows; a CUSUM watches the stream. Per agent and rule — and per run and rule where the traffic carries runs — each evaluation is a Bernoulli draw, the statistic accumulates the exact log-likelihood ratio for a ten-point rise in the fail rate over the agent's own baseline (Jeffreys-smoothed, set once ten fails are expected rather than after a count), and it crosses a line derived by a seeded simulation for an in-control run length of about 500, never typed. The trace whose evaluation crossed the line is a regression-alarm moment that names the rule, both rates, the run when the stream is run-scoped, and the reset; it reports and never gates, resets on alarm and re-baselines, and any threshold move stays yours. Every kind list derives from the one list, so GET /api/v1/moments?significance_kind=regression-alarm, the preferences store, the dashboard's filters, legends and command palette accept it; the Failures page ranks it above a first failure and below a cost spike.
• `cost_anomaly`, a rule that reads the agent's own history. The cost bundle gains a measurement: the trace cost against this agent's last 200 costed traces as the Iglewicz–Hoaglin modified z (0.6745 · (cost − median) / MAD), a spike at z > 3.5; when every recent trace cost the same, a spike is more than 10% over every prior value. It reports and never decides the verdict (cost_under_threshold stays the explicit dollar policy), skips as insufficient_history below twenty prior costed traces, and names the dearest tool call — as recorded when the trajectory prices its calls, or the largest estimated share by output size when it does not. The engine feeds it the same per-agent history the moment classifier reads, on log_trace with evaluate, on iris-eval ingest, on evaluate_runs, and on evaluate_output when a trace is linked. Its proof family measures conformance to the formula (14 cases, arithmetic in every note).
• `compare_runs` tests each rule and corrects twenty comparisons at once, and can state equivalence within a margin. The per-rule table was a number with no test behind it — failed_before, failed_after, a delta — so a reader could not tell a regression in one rule from noise, and twenty rules each read at 5% manufacture a "worse" somewhere in most comparisons of runs that did not change. Every rule now carries its own one-sided test in the regression direction (McNemar exact on that rule's discordant pairs when the runs pair, else the z read off its Newcombe difference), the p-values are corrected together with Benjamini–Hochberg, and a row is marked worse only at q ≤ 0.05; rules_tested names the family. A seeded guard over a thousand comparisons of twenty null rules holds the any-rule false-"worse" rate: about 48% uncorrected, 2.4% corrected. The comparison can also say a third thing, distinct from "worse" and from "not distinguishable": equivalent_within — two one-sided tests at α = 0.05, the 90% interval on the difference inside ±δ, with δ from equivalence_margin or, absent, the smallest difference these sizes could detect (and margin_source says which). The dashboard's compare view renders p, q, the worse mark and the equivalence chip; POST /api/v1/compare carries the same fields.
• A judge sharing a model family with the agent it judges says so. evaluate_with_llm_judge reads the agent's model from the linked trace (metadata.model, or a span's gen_ai.request.model) or from the new agent_model argument, and when it shares a family with the judge model the response carries warnings: [{ code: "IRIS_JUDGE_SAME_FAMILY", … }]. A judge from the agent's own lineage tends to forgive the errors it would make itself; the score stands and is stored, and nothing is refused.
• The header reads the server, not a constant. The status pill used to be the word "live" typed into the layout; it could not go red. It now comes from the client's own record of its last answer (any request that fails turns it red at once) and one health poll every 30 s — live, paused (the tab is in the background, or the server asked the page to slow down), degraded (the server's own word when its storage is down), unreachable, signed out — each with a sentence behind it. A judge chip says whether the LLM judge is on and, when it is off, lists the server's own steps to enable it; a DEMO chip appears on a server started with --demo; the account menu shows the retention window once, and the running server's version beside the UI's build when the two differ.
• One rule-result row, every stamped field. The trace page's evaluation card and the moment page each drew a rule result their own way — a mark, a name, a message — and dropped everything the composer stamps. Both now render one RuleResultRow: the kind and the role as chips with a sentence each, criticality with its source, the evidence (spans quoted from the text when the page has it, patterns with their count, a link to the tool call the evidence names, citations with their status, counts with their threshold and where it came from), the error bar with its basis (a fired detection's PPV interval at the prior in force, a quiet one's miss rate, conformance for a rule right by definition, local precision from your labels, or the published table's interval when the row carries none), the measured value with its unit, the skip reason and its class, and the rule's definition behind one disclosure. A row stamped before 0.9.0 says so instead of showing blanks. Tool-call rows on the moment page show the call's error; the trace page's tool calls carry anchors the evidence links to; the trace page renders its query error typed.
• The verdict panel. Above every evaluation's rows, on the trace page and the moment page: the verdict word with its basis (one sentence per basis: gate, veto, critical unknown, required evidence missing, risk over loss, clean, no rules), the rules it was decided by, the risk estimate with its interval and its confidence when the risk layer decided, coverage by question with how many of that question's rules ran and why the others did not, and the composer's own interpretations as sentences — a fired rule that did not decide names the one setting that would make it. The eight-step ladder is progressive disclosure: the panel shows the result and the reasons, a failed row shows its evidence, and one control — *How was this computed?* — opens method, computation and uncertainty on every row plus the composer's facts (eval.falsePassCost and the τ it yields, eval.defaultsGate, eval.onCriticalSkipped) and the provenance. An evaluation from before the composer says it has no verdict rather than showing one. The moment-level verdict tooltips no longer describe the score era.
• Runs and cases have pages, and two runs can be compared from the dashboard. /runs lists every run with its counts and pass rate and carries the compare action; /runs/:id shows one run's provenance and its evaluations, one per trace as a comparison counts them; /cases/:key shows every attempt at a case across runs and says whether it is flaky. The compare action calls a new route, POST /api/v1/compare, which is the compare_runs tool's own handler over HTTP — the same implementation, the same answer (method, the two summaries with their Wilson intervals, the difference with its interval, McNemar's exact test when the runs share case keys, the smallest change the runs could have seen, and the per-rule movement). Runs sits in the sidebar and the command palette (g u).
• Four navigation entries for three concepts, named in one place. The sidebar's six entries (Dashboard, Decision Moments, Custom Rules, Audit Log, Traces, Evaluations) become four — what failed, the data, and authoring twice — with traces and evaluations reachable from Runs as raw views of the same data and the moments timeline reachable from the landing page and the palette. Every surface that names an entry (sidebar, palette, page titles, the e2e specs) reads NAV_LABELS; the proposed names (Failures · Runs · Rules · Audit) are written beside the current ones and one edit flips them. /rules now shows the built-in roster with its proof — kind, mechanism, the question each rule answers, criticality with its source, and the published precision on the labelled corpus or "no family" — above the rules you deployed. The welcome banner and the auto-opening tour are gone: the Failures empty state says what they said, and the tour stays one palette command away.
• Drift by run. The Drift view gains a cohort selector: split by run — the grouping compare_runs tests, so the picture and the test agree on what "before" means — and each run gets a panel with n and the 95% Wilson interval on its pass rate in this window and the prior one, the tested difference with its interval or "not compared" with the smallest change the windows could have seen, and its own trend line. GET /api/v1/eval-stats/drift now carries interval on each window, from the same statistics module the proof harness uses; the dashboard computes nothing.

Fixed

• The `--api-key` dashboard's sign-in form works from a browser. Under the security headers' default Referrer-Policy: no-referrer a browser sends Origin: null on a form POST, so POST /session reached the DNS-rebinding guard as an unknown origin and was refused — the page showed {"error":"Forbidden: invalid Origin header"} and only the ?key= link ever signed a browser in. The policy is now same-origin (the referrer, and so the Origin, goes only to this server's own origin; nothing leaves for a third party). Found by the api-key state spec, confirmed by a request probe; held by a guard test and the end-to-end sign-in.
• A failure can be opened at phone width. At 390 px the sidebar kept its full 256 px — its width is an inline style, so the stylesheet rule meant to compress it below 768 px had been inert since the v2 chrome — and the failure row's four-column layout left its link with no width in the 134 px that remained: the list rendered and nothing in it could be opened. Below 768 px the sidebar is now the icon rail whatever the preference says (the preference is kept and applies again on a wider viewport), and the row's meta column drops under the body. The landing page's first-run hint names the palette command by its own title ("Onboarding tour").
• The Stream view survives a trace that recorded no cost. The live trace tail formatted cost_usd whenever it was not undefined, and the API serializes a missing cost as null — so one trace logged without a cost (the shape every REST or CLI ingest without a price produces) threw null.toFixed and the whole Stream view showed "This page hit an error it could not recover from". Found by the labels state spec, whose own stored trace carries no cost; the row now shows "—" for a null cost, and a test holds it.
• The cost-spike moment's label carries its dollar sign again. Since the cost-anomaly change earlier in this release the label read "Cost: 0.2000 (5.2× MAD over baseline)"; it now reads "Cost: $0.2000 (modified z 5.2)", or "(12% over the prior maximum)" when the flat-history fallback decided.
• The moment detail carries every rule result whole, and the evaluation's verdict, coverage, interpretations and provenance with it. The server's moments route remapped each rule result to six fields — name, passed, score, message, skipped, skipReason — so the stamp the engine has put on every rule since 0.9.0 (kind, role, evidence, uncertainty, criticality with its source, the question, the classes) never reached the dashboard, and the dashboard's own evaluation type had no field for the verdict, the coverage or the reasons. It now receives the same object the tool returns, and a test holds the dashboard's types to every key the server sends (tests/dashboard-types-contract.test.ts).
• A trace nobody judged is no longer labelled a pass. A trace with no evaluation, or one whose rules all skipped or whose verdict was unknown, was classified normal-pass ("no rules fired") on every list. It is its own moment kind, unevaluated ("No verdict"), excluded from the failure counts and the significance streams, and the moments filter and the saved preferences accept it. The three hand-typed copies of the kind list (the moments route, the preferences route, the preference store) now derive from one exported list.
• A failed query says what happened, in its own widget. Every API failure surfaced as one string ("API error: 503 Service Unavailable") that a page rendered if it thought to, and a page that did not went blank. The client now throws a typed ApiError — unreachable, unauthorized, not-found, rate-limited, server-error, bad-request — each with its own sentence and the server's own reason when it gave one; the data hook exposes it; one QueryError component renders it beside the widget that asked, with a retry where a retry can help and the sign-in path where a key is needed. A boundary per route keeps the shell when a page throws, and an unknown address gets a page that says so and links the sections instead of an empty shell.
• The `iris-eval-capture` plugin records turns again. Its Stop hook spawned the ingest itself, detached, with the trace on a stdin pipe and stderr on another, and exited a millisecond later; on the published package the ingest died with those pipes and no turn was ever stored — found by the stranger harness at the 0.13.0 release (row V1), which passed only when the hook was made to wait. The hook now writes the turn to a file under its data directory and detaches a pipe-free runner (hooks/ingest-runner.mjs) that hands the file to iris-eval ingest, treats exit 0 alone as success, logs the outcome, and keeps a payload it could not ingest under pending/. The plugin is served from this repository, so the fix reaches every new install without a package release.

Changed

• A flat cost history now uses the approved fallback. When every one of an agent's recent traces cost exactly the same (a zero MAD), the cost-spike moment used to read the trace against a floor of a tenth of the median; it now reads it as the approved algorithm states — a spike is more than 10% over every prior value — and the moment's reason says the fallback decided. The z arithmetic is unchanged and is now written as the reference writes it (0.6745 · (x − median) / MAD, which equals the 1.4826 scaling it replaced).
• The judge's price table is read from both providers' pricing pages (2026-09-20), and it is the one table. claude-opus-4-7 was listed at $15/$75 per million tokens and is $5/$25 — a threefold overstatement that made the cost cap refuse calls that fit (found and fixed by Roy Tong, #478; carried here with his credit). Added: claude-fable-5-1, claude-opus-5, claude-sonnet-5, claude-opus-4-8, claude-opus-4-6, claude-opus-4-5, claude-sonnet-4-5, gpt-5, gpt-5-mini, gpt-4.1-mini, o4-mini, o3-mini. o1-mini is absent from OpenAI's page and is kept as retired at its last known price, so an existing configuration keeps working. The cost estimator's separate table (older ids, older prices, since 0.3.1) now derives from the judge's; the guide's table is held to the code by a test; both tools' model sentences derive from the table and an unknown id is refused with the full priced list.
• Cost spikes are judged against the agent's own history, not a fixed dollar figure. The cost-spike moment fired on any trace over $0.10 — a literal in the classifier that flagged a haiku-class summariser and a research agent against the same line. It now asks whether the trace is expensive *for this agent*: a robust z-score over the agent's most recent costs (up to 200; the median and the median absolute deviation, src/eval/cost-anomaly.ts), a spike at z > 3.5, and the moment's reason names the baseline it was read against. An agent with fewer than twenty prior costed traces gets no cost verdict at all — nothing said, as the novelty classes say nothing below their floor — and the Failures page now builds the same per-agent history the moments page does, so the two rank the same traces. cost_under_threshold, the explicit dollar policy a deployment sets, is unchanged.
• The verdict's risk layer and the statistics module share one Beta sampler. src/eval/risk.ts carried its own Beta draw — the same Marsaglia–Tsang gamma as stats.ts, with the shape < 1 boost drawing its uniform in a different order — so the verdict's credible interval and the published PPV interval were computed by two implementations of one idea (arc 6 measured the difference and deferred the change to ship under its own proof regeneration). The risk layer now draws from stats.beta, seeded-random.ts is gone, and proof/COMPOSITE.md is regenerated: across the composite corpus's 141 cases no point estimate, verdict, confidence or accuracy rate changed; 489 interval bounds moved by 0.23 points on average and 1.65 points at most (silent-037, risk hi 0.948 → 0.931).
• Invariant 13 has its lock: tests/unit/eval/nothing-dropped.test.ts holds that every field the engine sets to a value appears in the serialized evaluation unless it is in the documented private set exported beside the serializer (PRIVATE_RESULT_KEYS: the caller's own text, the storage timestamp, the judge's spend). interpretations[] vanished for three releases because no such lock existed.
• The stranger harness grades on the 0.13.0 product: an evaluation counts on either door (log_trace with evaluate: true or evaluate_output); an evaluation returned inline satisfies the read-the-verdict row; the untold task is graded in a fresh session; the answer is read in the composer's vocabulary (must not ship, veto, basis) rather than the score era's; the twelve-call ceiling excludes the host's own mechanics (deferred-tool search, auto-memory, spilled results) and fetches of the prompt's own "Start here" pointer; --regrade grades a recorded run under the current rules without running it. Each re-derivation is proven against the 0.13.0 transcript that exposed it.

Earlier

One line per release, the bold lead of its changelog section; each links to the release notes built from that section.