Iris 0.20.0Iris says what it checked. A verdict tells a pass from a check that never ran, names who recorded the evidence it judged, and cannot be replaced by the agent it judges.

Security

• A long run of full stops, question marks or exclamation marks no longer holds the event loop. The sentence splitter read a run of terminators inside a word again from each terminator in it, and read the word before every full stop back to the last space, so the time to evaluate such an output grew with the square of its length: 20,000 full stops between two letters took 1.6 seconds, and a.B repeated over 20,000 characters 1.3, each doubling taking four times as long. A run is now read once, and only as much of the word before a full stop as the abbreviation lists need: the same outputs take 4 ms and 18 ms, and 80,000 characters 8 ms and 69 ms. Sentences are split exactly as before; the old and new splitters agree on 226,190 texts, 26,190 of them from the repository's proof corpora, fixtures and documents. The playground's copy has the same change.
• Two ways past the critical detectors are closed. (1) The fold that rejoins a word an evader split with a tab or a line break (previ|ous) read every lone break between two letters that way, so a phrase whose words were separated by anything but a plain space became one unbroken string: ignore all previous instructions written one word per line, with tabs, or with no-break spaces passed no_injection_patterns; a blocked phrase passed no_blocklist_words; a recovery phrase listed down the page passed no_pii; and a directive wrapped across lines in a tool result was missed by no_injection_compliance. These rules now read the text both ways, as joined tokens and as separate words, and report whichever reading holds the match, with the span still indexing the raw text. The structural injection detectors, which are line-shaped, read only the first. (2) Invisible characters were dropped from a list of eight. A right-to-left override, an invisible separator, an Arabic letter mark, a directional isolate, a variation selector or a tag character inside a Social Security number defeated no_pii. Every Unicode format character and every variation selector is dropped now. The published transforms table at https://iris-eval.com/proof shows the difference on the no-break-space and line-break rows, and the public playground reads the same way as the server.
• The agent being judged can no longer replace its own verdict. Every reader of a trace's verdict takes the newest evaluation carrying the trace's id, and every evaluation a caller linked to a trace carried it, whatever was judged. So after a trace failed on a leaked Social Security number, one evaluate_output call with that trace_id and clean text made the trace read pass in its run, in compare_runs and in the export, with no flag; the leaking text with eval_type: "cost", or with an empty tool_calls, did the same, and so did a lenient evaluate_with_llm_judge template. A trace's verdict is now only the server's own scoring of the trace as stored. Anything else is stored with the trace named in a separate column (reference_trace_id, migration 020) and no trace id of its own, so no reader of a trace's verdict can pick it up: the readers that list a trace's evaluations ask for both columns, and a reader written later that forgets the new column shows less rather than a replaced verdict. Deleting the trace erases the text of both kinds.
• A swapped rule and a changed gating setting change the fingerprint, and the audit log says what a rule was. provenance.rulesetHash named a deployed rule by its name alone: a rule that blocked a competitor's name, replaced under the same name by one matching nothing, turned the verdict from fail to pass under the same hash. provenance.configHash left out every composer setting, so the same output passed and failed under one fingerprint when eval.defaultsGate, the loss ratio, the handling of a critical check that could not answer, the required evidence or an estimated prior changed, and compare_runs called the two runs comparable. Both are covered now, appended only where set, so a deployment at the shipped settings with built-in rules keeps its hashes. An audit entry for a rule now carries details.contentSha256, the same hash the ruleset fingerprint carries for it, on deploy, toggle and delete. The entry is written before the change, and a failed write refuses the change: the append used to sit inside an empty catch, so a full or read-only disk let a rule be deployed, disabled or deleted, or a trace removed, with no record.
• A verdict that was `unknown` for missing required evidence read back as a pass. A stored evaluation's verdict is composed again on read from the settings stored with it, and eval.requiredEvidence was not among them, so the row read back under "nothing required". It is stored with the row now.
• A critical detection is no longer hidden behind a policy failure. The verdict's layers are asked in order (a policy you configured, a critical detector, a critical check that could not answer, required evidence, the risk estimate), verdict.basis names one of them, and everything that acted on one basis read basis alone. So for an output that broke a configured policy and leaked a Social Security number: iris-eval ingest --fail-on detector_veto exited 0, the detector_veto webhook did not fire, and the agent was told only that its cost ceiling had failed. verdict.also now lists every later layer that would have decided the verdict on its own, each as { basis, state, by }, in the order the layers are asked; it is absent when one layer decided alone and on every pass. --fail-on, the detector_veto and verdict_fail webhooks, the gate action's receipt and the dashboard's verdict panel read it, the evaluation CSV carries it as verdict_also (latest_verdict_also on the trace CSV), and interpretations gains one sentence to the agent naming the later layers and saying that clearing the first does not clear the verdict. basis, by and verdictPath() are unchanged, so nothing that read them reads differently. The risk estimate is now computed on a verdict an earlier layer decided, which is how a risk_over_loss entry reaches also; verdict.risk is still carried only when the risk layer decided or the verdict is clean. A stored evaluation reads back with also, since the verdict is composed on read. Tests: each pair of layers through the composer, the command-line gate end to end on the output above, the webhook, the receipt and the panel; the gate and webhook cases failed before the change.
• Dependency advisories closed in every workspace: fast-uri 3.1.7 → 3.1.8 (GHSA-hrr3-gc8f-f4qj, inconsistent host case normalization through percent-encoded octets; it ships, through ajv), and brace-expansion 5.0.9 → 5.0.12 and 1.1.18 → 1.1.21 (GHSA-qhr7-859c-m2p7, GHSA-6j4f-fj2g-mc7p and GHSA-q2hr-2g5m-vwhr, CPU and stack exhaustion on crafted brace patterns; development tooling only, through ESLint's minimatch, in the root, dashboard and website). Each override's floor is now the fixed version. The lockfiles were regenerated on Linux, and only these packages moved in them.
• Citation fetching refuses NAT64 addresses that lead to a private network. The SSRF guard already refused private IPv4 re-encoded as 6to4, Teredo, IPv4-mapped and IPv4-compatible IPv6. It now also checks NAT64. For the well-known prefix 64:ff9b::/96, the IPv4 address inside it goes through the same blocklist: on a network with a NAT64 gateway, 64:ff9b::7f00:1 reaches 127.0.0.1. The local-use prefix 64:ff9b:1::/48 is refused whole. The gap was found while triaging GHSA-2vr4-cq9g-pvrc, which reported the same range missing from the ip-address package. ip-address moves from 10.4.0 to 10.7.2 (GHSA-rpw4-54j3-4h4q, GHSA-2vr4-cq9g-pvrc) and the dashboard's test-only undici from 7.29.0 to 7.30.0 (GHSA-3wwx-pv8p-q78v). Neither was reachable in Iris; SECURITY-EXPOSURE.md records why.
• One search can no longer stall the server (#703). Full-text search is new in this release, and before this fix one query could cost as much as reading most of the index, once per term, on the event loop that every MCP and HTTP request waits on. On the benchmark machine, w* w* w* w* took 2.5 s at 10,000 traces and 23.9 s at 100,000, and 32 repeats of w* took 26.4 s at 10,000. Anyone who could call get_traces with q, GET /api/v1/traces?q= or the dashboard's search box could send such a query. Search is now bounded in three ways. - Normalising: a repeated term, and a one-word prefix that another one-word term implies (ref* beside refund), is searched once. The traces matched do not change; a property test checks 500 generated queries against the terms as typed. The response lists each term left out under search.ignored. - Limits: a prefix needs at least 3 letters or digits before the *, or 1 character in Chinese, Japanese or Korean, where one character is a word. A query carries at most 16 terms and 4 prefix terms. A query over a limit is refused before anything is read, with a message that names the term and the limit (an error from get_traces, a 400 from the HTTP API), and the dashboard shows the prefix rule as a hint. Three letters, because a prefix costs more the more words it starts: on text built from English-like syllables at 10,000 traces, one broad prefix took 62 ms at one letter, 45 at two, 26 at three and 3 at four, and the two-letter one matched 99.97% of traces, narrowing nothing. Every query from the review now answers in under 1 ms, refused. - A time budget: a search reads matches newest first for at most 1,000 ms (storage.searchBudgetMs in config.json, or IRIS_SEARCH_BUDGET_MS, 50 to 60,000). If it reaches the budget first, it answers with the best matches among the traces it read, sets search.complete to false and search.budget_ms to the budget, and total counts only the matches found. This covers the search without FTS5 too, which is also what runs while the index is built after an upgrade, and which read every trace however long that took. - What comes before the first match cannot be stopped: expanding a prefix, and counting the traces each term is in for the ranking. The limits bound it. At 100,000 traces the costliest query they allow took 0.6 s (four of the broadest three-letter prefixes on English-like text), and 0.4 s with Chinese terms. - A search on a store in a file now runs on a worker thread with its own read-only connection, so none of that work runs on the event loop, whatever the store's size. The thread starts with the first search and builds the page's snippets as well, and the server's thread only reads the page's rows. At 100,000 traces, measured by a 1 ms timer on the server's thread with the median of 7 runs, the longest wait for four of the broadest three-letter prefixes fell from 428 ms to 17 ms with better-sqlite3, and from 455 ms to 16 ms with node:sqlite. With CJK text, four one-character prefixes and twelve characters fell from 202 ms to 16 ms. 16 ms is the smallest wait the timer can show on Windows, and a search for a word in 0.1% of traces showed 11 ms on either thread. The thread adds 0.3 to 1.0 ms to a search: a word in 0.1% of traces took 4.8 ms against 4.0 on the server's thread, and 8.6 against 8.2 with CJK text. Its answers are the same as a search on the server's own connection; tests compare them across sort orders, pages, filters, CJK, and without FTS5. If the thread fails, the search it was running fails with the reason, and the next search starts a new thread. If the thread cannot start, searches run on the server's thread instead: the server logs the reason once, /health reports it under search_worker (unavailable, with the reason), and --self-test prints it. If the thread does not answer within the budget plus 10 s, the search answers as stopped with nothing read and the thread is replaced. Closing the store closes the thread's connection before its own, and waits at most 5 s for the thread: one inside a long statement cannot be stopped, so it is left to end when the statement does. A store in memory searches on its own connection. - Searches that were already allowed cost about the same. Measured on 100,000 traces with main and this change in one process, alternating, median of 7: a word in every trace 208 ms against 246, sorted by timestamp 347 against 398, a phrase in half the traces 84 against 123, one agent's traces 167 against 154, and sixteen words that match no trace together 14 against 9. The search without FTS5 now reads the traces by rowid rather than sorting the tenant's rows again for every batch it read: 2.1 s against 4.1 for a word in every trace at 100,000. - An FTS5 prefix index (prefix='3') was measured and not added. It made four broad three-letter prefixes 3 times faster (186 ms against 559 at 100,000 traces), but made the file 20% larger and each insert 31% slower, for every trace, while the limits already hold those queries under the budget. - Tests: the review's queries and 60 generated ones on a store shaped like the review's (every word starts with w), each on the index and on the search without FTS5, each refused or answered within a bound on CPU time; before this change, 8 repeats of w* on that store used 23.5 s of CPU. The budget is tested for which traces it keeps and how it ranks them, on both paths, and for the CPU time the search without FTS5 uses against reading everything.
• `delete_trace` no longer leaves the deleted text readable in `iris.db` (#703). The database zeroes the pages a delete frees, but in write-ahead-log mode the zeroed pages go to iris.db-wal, and iris.db kept the old ones, with the trace's text, until the next checkpoint: for as long as the process ran. The retention sweep and --purge already checkpointed after deleting; delete_trace now does too, before it returns. Tests read the bytes of iris.db and iris.db-wal straight after the delete, for Latin text and for Chinese text and the two-character pieces the index keeps of it, and for a trace written since the last checkpoint. Each failed before the change. A delete takes longer: 24 ms against 15 at 100,000 traces, median of 20. A trace carrying an agent loop's spans takes 85 to 100 ms: taking a trace's words out of the search index grows with how many different words it holds. The API reference's delete_trace section states the cost. The checkpoint does not wait for a search that is reading the file, because waiting held the event loop for the rest of that search (470 to 620 ms at 100,000 traces). It is tried again every 20 ms until the reader finishes, so the text leaves the file then; a test deletes a trace while a search reads and checks the file's bytes once the search has finished. - When a test finds a swept trace's word still in iris.db, its message now names the page of the file that holds it, the page's b-tree type, whether the file's free list names it, the connection's secure_delete and free-page count, and the rows of merge the sweep still owes. The test failed once on a hosted Windows runner with the word whole in iris.db and the log empty, and did not repeat in 50 local runs (#758).
• Revoking an API key takes effect on the next request, with no restart. Remove a key from security.apiKeys in config.json, or delete its key file, and the next request with it is refused on the MCP transport and the dashboard alike. Every browser session it opened is signed out at the same moment, and other keys and their sessions carry on. Before, a leaked key kept working until the server restarted. On each request the server checks config.json and every key file it names with a few stat calls, and re-reads the keys only when one has changed: in a unit test, 50 requests with the files unchanged read config.json 0 times, and 50 after one edit read it once. Deleting one key's file revokes that key alone. A config.json that cannot be read fails closed: until it is fixed, only a key from IRIS_API_KEY or --api-key is accepted, and the log says why. A new expiresAt also applies without a restart. Proven through the real CLI: with a server serving both ports, rewriting config.json without a key refuses that key's next request with 403 on both ports, and its session's next request gets 401; the same test fails, with the revoked key still answered 200, when the server uses the old key ring. The key in IRIS_API_KEY or --api-key, and whether authentication is on at all, still change only on a restart.
• A browser session ends when the API key it signed in with expires. Each dashboard session now records the key it was opened with and checks that key on every request, so a key past its expiresAt, or one no longer configured, signs its browsers out at the same moment the Bearer check starts refusing it. The session and its cookie's Max-Age are also capped at the key's expiry. Before, a browser signed in with an expiring key kept full access for up to 30 days. Proven through the real dashboard: a session opened with a key expiring in one hour answers 401 one hour and one second later, while a session from a key with no expiry is unaffected.
• The Python client is published only from a tag on `main`, built with hash-pinned tools. publish-python.yml now refuses a py-v* tag whose commit is not on main, as release.yml does for v*. It installs build and the hatchling backend, with their 6 dependencies, from .github/requirements/build.txt, where every file is pinned by SHA-256 (pip install --require-hashes --only-binary :all:). It then builds with --no-isolation, so the build no longer downloads an unpinned backend. build.txt is generated from build.in with pip-compile --generate-hashes in python:3.12-slim, and Dependabot watches it.
• The SDK, the LangChain.js handler and the `iris-eval` launcher can be published by the release workflow, with npm provenance, once each is set up on npm. A package joins a release when the workflow can publish it: its trusted publisher (this repository, release.yml) set on npmjs.com, or for its first version a one-time token stored, as RELEASED_WITH_SERVER in scripts/claims/packages.mjs records. None is yet, so this release publishes the server alone, and a package the workflow cannot publish no longer fails the run after the server is public. Two new release jobs handle them. build-packages runs with no publishing identity: it checks the packages (scripts/check-npm-packages.mjs), builds and typechecks them, and packs them. publish-packages runs only npm: it publishes each tarball whose version is new, in dependency order, after the server, with --provenance. verify-release then checks from outside that each package's version is on npm with an SLSA provenance statement. Each package's publishConfig requires provenance, so a publish from a machine that cannot attest its build is refused. @iris-eval/sdk and @iris-eval/langchain are no longer private. The launcher is at 1.0.1 in this repository, the same launcher with provenance once the workflow publishes it; 1.0.0, on npm, was published by hand without provenance and works the same. The release page carries these notes whole when they fit and, when they do not, everything before the first ### heading whole with each later entry by its lead sentence and a link to the rest; validate refuses a tag whose notes do not fit even so, and verify-release checks that the page ends with the supply-chain block and the list of merged pull requests.
• CI installs the MCP clients it tests against from a lockfile. Claude Code 2.1.282 and Gemini CLI 0.61.0 come from .github/real-clients/package-lock.json with npm ci, every package pinned by integrity hash, on Linux, macOS and Windows. Before, they came from npm install -g. Dependabot proposes client upgrades.
• `SECURITY.md` covers everything the repository publishes: the container image, the MCPB bundle, the launcher, the JavaScript SDK, the LangChain.js handler, the Python client and the CI gate action, as well as the server, the dashboard and the website. Its self-hosting advice, and the README's data-on-disk section, now recommend encrypting the disk or volume that holds the Iris home. Iris does not encrypt at rest: its database files are owner-only, and it stores no LLM provider keys.
• The storage threads start from fixed code. Run from its TypeScript sources (its tests, npx tsx), Iris used to start the search and checkpoint threads from code with the entry file's URL written into it. They now start from one constant and receive the URL as an argument (src/storage/source-thread.ts), which closes two CodeQL js/bad-code-sanitization alerts. The published package never took this path: it starts the compiled worker files directly.

Added

• `state` on every rule result, bundle row, run result and case attempt, and a count of what was not checked beside every pass count. A rule result carries state (pass, fail, not_checked): a rule that skipped still carries passed: false and score: 0 as placeholders, and state is what to read. A rule that skipped for missing evidence carries lacked (the inputs it reads that the call did not carry) and, when somebody had asked for it, asked (config or call). GET /api/v1/runs/:id and GET /api/v1/cases/:key carry state on each row; GET /api/v1/runs, the case route, GET /api/v1/eval-stats, each drift window, the summary (eval_not_checked), compare_runs and evaluate_runs (not_checked) say how many verdicts were not checked. Pass rates keep those in the denominator, so leaving out what a check reads never raises a rate. The dashboard draws such a verdict as NOT CHECKED in grey where it drew a red FAIL, says "2 not checked" beside the counts, and its result filter reads "Did not pass". The recent-failures list leaves such verdicts out and names the rule that decided, and the count of leaks no longer counts a no_pii that skipped. The gate action's receipt has a Verdict column (failed, or not checked). Migration 021-eval-verdict-state adds the column the counts read; rows from earlier releases are filled in the background after the first start and are counted as before until then, and a stored row that cannot be read no longer stops that background step at itself.
• `verdict_not_checked` webhook event, for a verdict that was not checked and that no layer failed, with the missing input or the check that could not answer as its subject. An alert on verdict_fail alone stayed silent when an agent stopped sending the evidence a check reads.
• `proof/INVARIANTS.md`: what a verdict does when evidence is left out, and when a failure is added. npm run proof -- --invariants takes each of the 145 labelled cases, sends the call without one field at a time (deleted, or a blank in its place) and counts the verdicts that got better, then adds one more failure to each and counts the ones rescued by it. With nothing said about what a call must carry the counts are not zero and are published as they are: of 72 failing cases that carry tool calls, 39 pass when the calls are left out. Where a contract is in force (the deployment requires the input, the deployment set the threshold of a rule that reads it, or the call supplied what it is compared with) no case passes with the field left out, and none of five fixed additions rescues a case; CI holds both at zero. Two things are measured and published without being held at zero, each with its reason: an explicit empty list of tool calls under a ceiling on the calls, and a sentence of refusal appended to the answer, which does rescue cases. The page also rewrites each output seven ways: doubled spaces, wrapped lines and the output sent as a JSON string field change no verdict and no deciding rule's answer, held at zero in CI; curly quotes, a Markdown quote prefix and upper or lower case are published with the count of answers each changes and why it is not held.
• `storage.searchIndex: "off"` (`IRIS_SEARCH_INDEX=off`) keeps no full-text index, for a feed that needs the ingest rate more than fast search (#729). A write then stores the trace and nothing more: sustained OTLP ingest of traces with two model spans stored 4,237 traces/s at 100,000 traces and 4,393 at 10,000, against 1,735 and 2,000 with the index (0.19.0: 2,330 and 2,620). A search reads the traces themselves within storage.searchBudgetMs, newest first, and says so (search.index: "scan"): a word in one trace in a hundred took about 180 ms at 10,000 traces, against 4 ms, and at 100,000 took about 1.7 s and answered with 527 to 584 of its 1,000 matches (search.complete: false), against 7 ms and all of them. Turning it off erases the index the file kept, in steps after the start; turning it on again builds a new one in the background. Health reports the index as off, and so does --self-test when IRIS_SEARCH_INDEX sets it (the self-test does not read config.json). The index stays on by default, and span text stays fully indexed.
• A trace that reports no cost gets one estimated from its tokens, marked as estimated (#702). Before, a gpt-4o-mini trace with 150,000 input and 10,000 output tokens and no cost attribute was stored with cost_usd null, and every cost rule and the cost alert skipped it. It is now stored at $0.0285, its list price, with cost_source: "estimated". A reported cost always wins and is never replaced, and is marked cost_source: "reported". - Every door prices the trace before it stores or scores it: log_trace, POST /api/v1/traces, POST /v1/traces (which the SDK and Python wrappers and the LangChain handlers send to) and iris-eval ingest, and the storage adapter prices any trace a door did not. When spans record model calls, each call's model is the one its span names (gen_ai.response.model, then gen_ai.request.model); calls to more than one model are priced each at its own. Of the 8 framework fixtures the OTLP door is tested against, 7 are now priced and the Google ADK one (gemini-2.5-flash) is not. - Model ids are matched exactly, ignoring case, with one provider prefix (openai/, openai:, anthropic/, anthropic:) when it names the right provider, and 11 dated snapshots each checked against the provider's own model page. No date is stripped by rule: gpt-4o-2024-05-13, which OpenAI prices at twice gpt-4o, is not matched. An unknown model, no model or no token counts leaves the cost null, and cost_estimate says why (unknown_model, no_model, no_tokens); a trace with one unpriced call is not priced at all. - Prices come from the judge's table, read from the providers' pricing pages on 2026-09-28. pricing.models in config.json prices models the table does not know, or at another price, and pricing.asOf dates them; pricing.estimate: false turns estimates off. The server and iris-eval ingest read the same file, and --self-test prints the setting. - cost_estimate stores the calls, tokens and prices an estimate used. The rules' messages end with what was estimated and their cost evidence carries costSource: "estimated"; the cost_anomaly webhook's detail carries cost_source. The dashboard shows est. after an estimated cost, with a tooltip naming the tokens, the model and the date of the prices, says why a trace has no cost, and names the estimated part of the total cost. GET /api/v1/summary adds estimated_cost_usd, GET /api/v1/eval-stats adds estimatedCost, and the cost_by_agent view adds estimatedTraces and estimatedCostUsd. evaluate_output with a trace_id and no cost_usd now uses the trace's stored cost. - Cached input is priced as the provider bills it. Cache reads (gen_ai.usage.cache_read.input_tokens, which the SDK and Python wrappers and the LangChain handlers send, OpenInference's llm.token_count.prompt_details.cache_read, or token_usage.cache_read_tokens, or OpenAI's prompt_tokens_details.cached_tokens as sent) are priced at the model's cache-read price, and cache writes at its cache-write price. The judge's table gains both, read from Anthropic's and OpenAI's pricing pages; the OpenAI models in it charge no write premium. A gpt-4o-mini call with 150,000 input tokens, 100,000 of them cached, is estimated at $0.021, not $0.0285. A claude-sonnet-5 call with 100,000 cache reads and 20,000 cache writes is estimated at $0.084, not $0.254. A model priced in config.json without cache prices has its cached tokens priced at its input price, and cost_estimate.notes says so. Anthropic's cache counts reported beside the input count, rather than inside it, are added to it, and the notes say that too. A 1-hour cache write is priced at Anthropic's 1-hour price (2 times the input, where a 5-minute write is 1.25 times) when the trace carries Anthropic's split of its writes. The SDK and Python wrappers now send that split as iris.usage.cache_creation.ephemeral_1h_input_tokens, read from the response's usage.cache_creation. Both LangChain handlers send it too: from input_token_details in Python (langchain-anthropic) and from the raw usage in response_metadata in JavaScript (@langchain/anthropic). A shared fixture holds the two handlers to the same attributes. The Python handler also counts the cache writes when langchain-anthropic moves them into the lifetimes and sets cache_creation to 0, where before it sent 0. token_usage.cache_creation_1h_tokens carries the split on the other doors. A claude-sonnet-5 call whose 20,000 writes are 15,000 1-hour ones is estimated at $0.1065, not $0.084. Without the split every write is priced as a 5-minute write, and the notes say so. An estimate does not know batch discounts. docs/cost.md has the rules and the limits. - Migration 016 adds two nullable columns and rewrites nothing: 6 ms on a file of 100,000 traces, where rewriting the costed half took 2.3 s. Costs stored before it read as reported. Tested by upgrading a database written by the 0.19.0 release itself, on both SQLite drivers.
• `install --upgrade` moves every MCP client on the machine to this release, and `install --list` shows which Iris each one runs (#704). It reads the config of each of the ten clients install knows and moves every Iris entry pinned to an older release, or unpinned (which npx resolves to whatever version it has cached), to this one, keeping the rest of the entry (trailing arguments such as --dashboard, an env block) and moving an entry under the older iris key to iris-eval. It never moves a client back: a pin to a newer release is left as it is, as is an entry that runs something other than the npm package, and each is listed with the reason. A config it cannot parse is reported and left untouched, and the command exits 1. Proven for each of the ten clients' config formats, and end to end in a new CI job on Linux, macOS and Windows that installs the released 0.19.0 from npm, has it install Claude Code, Cursor and Codex and store traces, upgrades that database with this release, and moves the three pins.
• A database is copied before it is migrated (#704). When a start finds migrations to apply to a file that already holds data, it first writes a consistent copy next to it with VACUUM INTO, named for the release that last changed the file's schema and the one about to (iris.db.0.19.0-to-0.20.0.20260928T101500Z.bak), created owner-only like the database, and keeps the newest three. The start prints one line on stderr with the copy's path, which older releases can no longer open the file, and how to go back; when a client on the machine is pinned to such a release, a second line names each one and the install --upgrade command. When the disk has less free space than the copy needs plus 64 MB, or the copy cannot be written, the start migrates without it and the line says why. Because a copy holds the traces as they were, --purge deletes every copy, the retention sweep deletes one older than retention.days, and --demo-clear deletes the demo database's. The README has a new "Downgrading" section: restore the copy and pin the older release. The end-to-end job restores a copy and 0.19.0 opens it again.
• A migration names the oldest release that can still use the database, so a later additive migration no longer locks out the release before it (#704). Each migration declares a compatibility floor, and the migration ledger stores it beside the migration's id (rows written before 0.20.0 are filled in with the release that introduced each migration). A release that finds a migration it does not know now opens the file when that migration's floor is at or below its own version, and refuses, as before, when the floor is above it or not recorded; the refusal names the version needed, install --upgrade, and the copy to restore. 015-trace-search has floor 0.20.0: run against a file with the index, the released 0.19.0's deletes, retention sweep, metadata patch and span inserts all kept the index correct through its triggers, but 0.19.0 inserts traces without indexing them, so a 0.20.0 server beside it would search an index that lacks them. 0.19.0 refuses the file regardless, since its own check cannot change. 016-trace-cost-source has floor 0.19.0: the released 0.19.0, writing into a file with its two columns, stored rows this release reads with the right cost basis (a reported cost as reported, none as none), and its metadata patch, delete and sweep left an estimated trace's cost, source and estimate as they were. 015 still holds such a file at 0.20.0. 017-relevance-judge-spend has floor 0.19.0 too: 0.19.0 has no relevance judge and never touches the table, and the same run left a day's recorded spend exactly as it was. 018-eval-risk-estimate and 019-read-paths take 0.20.0, the release that adds them; 019's reads name its indexes, so a later migration that drops one takes its own release as floor.
• Export traces and evaluations as CSV or JSON Lines, with the list's filters and search applied (#4). The Traces and Evaluations pages have an Export button that downloads every matching row across all pages, not the page on screen; GET /api/v1/traces/export and GET /api/v1/evaluations/export take format=csv|jsonl and the list routes' own parameters, validated by the same schemas; and iris-eval export traces|evaluations writes the same bytes without the dashboard running. A JSON Lines trace is exactly what GET /api/v1/traces/:id answers, with its spans and evaluations inside it. The CSV has a fixed, documented column set, including cost_source beside cost_usd so an estimated cost is never read as a reported one, is UTF-8 with a byte-order mark so Excel reads non-ASCII text correctly, and prefixes ' to any text cell starting with =, +, -, @, a tab or a carriage return, so a stored formula shows as text instead of running. The export streams: the server reads 250 rows at a time, waits for a slow reader, and breaks the connection on a failure part-way instead of ending a file that looks complete. It is behind the same key, rebinding guard, tenant scoping and rate limit as every read, and carries the stored text as the dashboard shows it (docs/api-reference.md says what that includes). Measured with node --expose-gc --import tsx scripts/bench-export.ts on an Intel Core i9-12900HK laptop (20 threads, 32 GB, Windows 11), Node 24.11.0, better-sqlite3, 100,000 traces of about 140 words, two tool calls and one evaluation each (a 330 MB file), through the real server over a socket: the CSV export (100 MB) took 3.3 s and the JSON Lines export (173 MB) 3.5 s, the server's live heap grew by at most 2 MB, the event loop was held for at most 25 ms, and a list request sent every 50 ms during the export was answered in at most 60 ms (74 ms with no export running). A search is ranked on the search thread with no time budget, since an export must hold every match, and then read a batch at a time: one matching 50,000 traces exported in 2.0 s with the event loop held for at most 27 ms (202 ms when it is the process's first search, which starts the search thread, as the list's first search does); 100,000 evaluations as CSV (35 MB) took 1.3 s. The audit log's CSV download now applies the same formula guard, byte-order mark and CRLF line endings.
• Recipes for the OpenAI Agents SDK (Python and JavaScript) and LlamaIndex, each run in CI against a real Iris server. docs/otel-recipes.md gives each one's OpenInference setup lines with Iris as the OTLP endpoint, what Iris reads from the spans (the input, the output, each tool call with its arguments, the token usage, the model, the session, the tools offered to the model) and what they do not carry. Each fixture is the framework's real export of a tool-calling run against the scripted provider, captured with pinned versions (openai-agents 0.22.3 with openinference-instrumentation-openai-agents 2.5.0; @openai/agents 0.18.0 with @arizeai/openinference-instrumentation-openai-agents 0.2.15; llama-index-core 0.14.25 with openinference-instrumentation-llama-index 4.5.2). A new CI job runs each recipe's script against a server built from the checkout and requires the trace back with its words, tool call, usage (50 in, 20 out over two model calls), its cost estimated from them ($0.0000195 at gpt-4o-mini's list price, cost_source: "estimated"), session and a passing verdict, and a leaked SSN failed by no_pii: 6 end-to-end cases. The captures showed what the OTLP door read wrong, now fixed and held by tests on the three captures: the Python Agents SDK puts no words on its root span, so the output was the first model call's tool request; LlamaIndex's workflow steps record Python reprs, the root's output among them; OpenInference's llm.input_messages.* / llm.output_messages.*, agent.name and llm.tools.N.tool.json_schema were not read; a tool's input.value JSON (and LlamaIndex's {"kwargs": {...}}) was checked by valid_tool_arguments as a string and failed. The scripted provider now echoes the offered tools in a Responses answer, as OpenAI's API does.
• Every OpenAI and Anthropic call recorded and scored, without the model choosing to (#648). Wrap the provider client once: wrap_openai / wrap_anthropic in the Python client, and wrapOpenAI / wrapAnthropic plus irisMiddleware for the Vercel AI SDK in a new JavaScript package, @iris-eval/sdk (in this repository, not yet published to npm). Each call becomes one standard OpenTelemetry GenAI span (gen_ai.*) sent to POST /v1/traces, so there is no new trace model: Iris stores it with the input, the output, the token usage, the tool calls and the model, and returns a verdict. Covered: Chat Completions (plain, streamed, parse, the stream helper), Responses (plain, streamed, the stream helper), Anthropic Messages (plain, streamed, the stream helper), sync and async in Python. The original client is not changed, a refused call raises the provider's own error, and an unreachable Iris never breaks a call or makes it wait. Proven in CI against the official SDKs and a scripted provider, with each call read back from a real Iris server: 19 end-to-end cases in JavaScript (Node 22 and 24) and 18 in Python (3.10 to 3.13, and once on openai 1.66.0 and anthropic 0.43.0); one fixture holds the two languages to the same span for 8 calls.
• LangChain and LangGraph runs scored, in Python and JavaScript (#642). IrisCallbackHandler in iris_eval.langchain and in @iris-eval/langchain (rebuilt; in this repository, not yet published to npm): each top-level run (a graph, a chain, an agent or a model call) becomes one trace of GenAI spans, the run with its model calls, tool calls, graph nodes and retrievers, sent through the same recorder as the provider wrappers with a verdict asked for. Iris stores the run's input and output, each tool call with its arguments and result, the token usage of every model call, the model, the tool catalogue, the latency, and a LangGraph thread_id as the session. A run that raises arrives with its error and is not scored. Proven in CI against a real LangGraph app (the tool loop with LangGraph's ToolNode) and a scripted chat model: 8 end-to-end cases in Python, 7 in JavaScript (Node 22 and 24).
• An OTLP sender can ask for a verdict on its own traces. iris.evaluate: true on the resource or the root span scores that trace with otel.evaluateOnIngest off, and iris.eval_type names the bundle; an unknown bundle stores the trace unscored and says so in evaluation_error.
• The LLM judge prices Claude Opus 5.5 (claude-opus-5-5, $4 in and $20 out per million tokens, read from claude.com/pricing on 2026-09-25), so it can be named as a judge and costed as an agent's model. An unpriced model is refused, because the cost cap cannot hold without a price. claude-opus-5 stays priced and is marked legacy, as the provider's page lists it.
• Claude Desktop installs Iris in one click (#650). Each release attaches iris-eval.mcpb, an MCP Bundle: open it and Claude Desktop shows an install dialog, with an optional Anthropic key, an optional OpenAI key and a dashboard switch that starts off. The bundle is built from the npm tarball the release publishes plus the production dependencies in package-lock.json (scripts/mcpb/pack.mjs); built from the 0.19.0 package it is 6.3 MB, 109 dependency packages. It leaves out better-sqlite3, whose binary fits one platform and one Node ABI, and starts Iris on Node's built-in SQLite (IRIS_SQLITE_DRIVER=node) under the Node range the package declares. The build is deterministic, and the release builds it twice and requires the same bytes; it is cosign-signed (iris-eval.mcpb.sigstore.json) and carries a build-provenance attestation, and verify-release checks both and that the bundle holds the files of the package npm serves, byte for byte. server.json names the bundle as a second package, with the SHA-256 of the attached file written in at release time, as the MCP Registry requires. CI unpacks and validates each bundle with the reference @anthropic-ai/mcpb 2.1.2 CLI, rebuilds it on Linux, macOS and Windows and compares the bytes, and starts it from the launch config that package builds on Node 22.13.0 and 24, requiring all 12 tools in tools/list and a stored trace; a second job starts it in an Electron 40.4.1 utility process (Node 24.13.0, what Claude Desktop 1.1.6679 ships) on macOS and Windows. The Smithery listing copy now carries the steps to publish that bundle.
• `npx iris-eval` starts Iris. The unscoped npm package iris-eval is a launcher: it depends on @iris-eval/mcp-server alone, at an open-ended range, and starts it, so npx iris-eval --self-test, npx iris-eval install <client> and npx iris-eval --dashboard are the server's own commands at its latest release. An MCP client config should still name the server package and a version, which is what install writes.
• Decision Moments can be ranked by significance, and the Moments page ranks by default (#409). Newest first put a safety violation behind every trace that came after it. GET /api/v1/moments?sort_by=significance reads the newest window traces that match the agent and time filters (default and maximum 500), classifies each, applies the verdict and significance filters, and ranks them: significance.score descending, then newest first, then trace id. Pages are cut from that one ranking, so total is exact within the window and page two begins where page one ended. The response states the window it read (window.size, scanned, tracesInRange, newest, oldest), and passing until = window.newest keeps later pages on the same ranking while traces arrive. The significance tiers, most serious first, are severity (a safety or veto rule failed, 1.0), change against the agent's own baseline (cost spike 0.9, regression alarm 0.85), and rarity (first failure 0.8, new combination 0.75), then the ordinary outcomes; docs/api-reference.md has the table and the history each class needs. On a 240-trace fixture with a safety violation, a cost spike and a first failure, newest first shows them at positions 210, 150 and 90, and the ranking shows them at 1, 2 and 3. A 500-trace window over a 2,000-trace store answers in a median 334 ms locally, against 66 ms for a newest-first page of 50. The Moments page asks for the ranking by default, polls it every 10 seconds instead of 3, says how far back it reached ("Ranked by significance within the last 500 traces of 1,200 …"), and offers Newest first as an order (?sort=newest). sort_by defaults to timestamp, so existing callers are unchanged. A window without sort_by=significance, or sort_order=asc with it, is refused with a 400 rather than ignored.
• An off-topic answer fails when you turn on the relevance judge (#649). Set IRIS_RELEVANCE_JUDGE_MODEL to a priced model id, with that provider's key, and answers_the_ask asks a new relevance judge template whether the output addresses its ask. It then gates on the judge's verdict: an off-topic answer fails with verdict.basis: "policy_gate" and verdict.by: ["answers_the_ask"]. A correct paraphrase that shares none of the ask's words passes, and a one-word answer or a one-term ask is judged rather than skipped. That is one judge call per evaluation that carries an input, on evaluate_output, log_trace with evaluate: true, HTTP and OTLP ingest, iris-eval ingest and evaluate_runs alike, capped by IRIS_LLM_JUDGE_MAX_COST_USD_PER_EVAL. A provider key alone never turns it on. The rule's result carries kind: "judgment", role: "gate", the judge's score against its 0.60 pass line, its rationale and what the call cost. When the judge cannot answer (an unpriced model, no key, the cost cap, a provider error), the result records why, the rule falls back to its lexical reading, which advises, and interpretations says so. A judge from the agent's own model family still decides, with the same-family warning in interpretations. Installing a judge changes the ruleset hash, so evaluate_runs re-scores traces judged without it. iris://capabilities → judge.relevance and --self-test show whether it is on and whether it can be called. Embedders install it with engine.setRelevanceJudge(createRelevanceJudge({ model, apiKey })).
• The `relevance` judge template, for `evaluate_with_llm_judge` too. It asks whether the output addresses this request rather than another subject or question, and it does not judge correctness or helpfulness: a wrong answer to the question asked is relevant. It scores against five anchored points with a 0.60 pass line, requires input (refused with IRIS_INVALID_ARGUMENT before any spend), and returns addresses_request, on_subject and specific_to_request. It wraps the request and the output in the per-call nonce tags, and it tells the judge to score an instruction aimed at itself as absent. A new template, so no existing score shifts. Its proof set is 36 labelled cases in proof/judge/cases/relevance.json: 18 that should pass, 9 of them shaped to fool word matching; 12 violations; and 6 injection twins. npm run proof:judge also measures answers_the_ask with the judge installed on the rule's own 45-case corpus family, beside the lexical rule. It then measures the whole composite verdict with the judge installed, on the test, dev and real-transcript splits, beside the same corpus without it. All three need a key and are pending until a keyed run is published. The composite harness now gates on a judgment as the product's composer does, so that measurement is of the composer the product runs. It is proved without a key on a stand-in judge that fails everything: the four off-task composite cases the shipped config lets through then fail. What CI measures without a key: on those 36 cases the lexical answers_the_ask fails 10 of the 18 it should fail and wrongly fails 2 of the 18 it should pass. One relevance judgment's worst case is $0.0060 on claude-haiku-4-5-20251001 and $0.0008 on gpt-4o-mini.
• The relevance judge spends within a daily budget, keeps personal data and credentials from the provider, and says when it reaches the network (#649). With IRIS_RELEVANCE_JUDGE_MODEL set, every evaluation that carries an input makes a judge call on your key, and the provider wrappers ask for evaluation on every call they record. Until this change only each call was capped; nothing capped the total, and the ask and the answer went to the provider exactly as they were. - IRIS_RELEVANCE_JUDGE_DAILY_BUDGET_USD, default 1, is what the judge may spend per UTC day, per tenant. It is kept in the database (migration 017, table relevance_judge_spend), so a restart, a second server on the same file and iris-eval ingest draw on one balance. A call is made only if its worst case, priced as the per-call cap prices it, fits in what is left. The worst case is replaced by the actual cost when the call returns, so the day's total never passes the budget. At the $0.0060 worst case above, the default admits at least 165 judgments a day. Past it, answers_the_ask reads the ask lexically, judge.withheld is daily_budget with the reason in judge.error, and the server logs one warning line per tenant per day (once per process). 0 stops every call. - IRIS_RELEVANCE_JUDGE_MAX_CALLS_PER_REQUEST, default 20, caps judge calls in one request. An OTLP request of up to 2,000 traces, or an evaluate_runs re-score of a whole run, judges its first 20 and reads the rest lexically with judge.withheld: "request_cap". The OTLP answer carries iris-eval.relevance_judge: { calls, withheld, max_calls_per_request }, and the evaluate_runs summary says how many were withheld. - Before a call, every span Iris's own no_pii rule flags in the input and the output is replaced by a numbered marker such as [REDACTED:Email#1]. That covers the personal-data patterns, the API keys and tokens, private key blocks, seed phrases, and any of those inside a base64 run. It uses the rule's detector, not a second list, and every match is replaced, not only the 25 the rule keeps as evidence. A value that appears in both texts gets one marker, and the prompt tells the judge to read a marker as the value it replaced. judge.redacted counts the replacements. IRIS_RELEVANCE_JUDGE_REDACT=off sends the text as it is and marks each record sentUnredacted: true. Redaction is on by default because relevance is about the subject, not the digits of a card number. - The README, docs/llm-as-judge.md and the enable steps say what leaves the machine: the evaluation's input and output, to Anthropic or OpenAI, on your key, and nothing else from the trace. - With a relevance judge installed, evaluate_output, log_trace and evaluate_runs advertise openWorldHint: true, and evaluate_output advertises idempotentHint: false. Without one they are unchanged. - iris://capabilities → judge.relevance carries egress, redact, maxCallsPerRequest and today's budget (spent, remaining, calls, refused, when it resets). --self-test prints the same line, with today's spend read from the install's database. GET /api/v1/health carries judge.relevance.budget_exhausted. It does not report the amount, since the endpoint answers without a key. - A relevance judge that is configured and cannot run (no key for its provider, or an unpriced model) no longer fails silently. The server and iris-eval ingest print one warning line at startup naming what is missing, and --self-test fails its judge step and exits 1 instead of printing PASS. Each result still carries judge.error and the lexical reading. - The Claude Desktop bundle (iris-eval.mcpb) asks for a relevance judge model (optional, empty by default) and the daily budget (default 1), beside the two provider keys, which stay masked. Its description says what naming a model sends. - Proven end to end with the provider replaced at fetch, so the tests read the request body that would go out: tests/integration/relevance-judge-limits.test.ts, tests/unit/eval/llm-judge/judge-budget.test.ts and tests/unit/eval/llm-judge/redact.test.ts.
• Full-text search over traces: find the run where the agent said it (#7). get_traces, GET /api/v1/traces and the Python client's get_traces() take q, and the dashboard's trace list has a search box. It searches each trace's input, output, and the values in its tool calls and metadata (never their keys). Every word must appear; "a phrase" matches in order; word* is a prefix; case and accents are ignored. Results are ranked by relevance (BM25) unless sort_by says otherwise, the other filters still apply, and each trace carries match: the field it matched in, a snippet, and the snippet as fragments with the matched words marked. Anything typed is read as words, never as SQLite query syntax: NEAR(, OR, input:, stray quotes and parentheses cannot fail or widen a search (a property test runs 400 generated hostile queries per driver and requires the index and a plain read of the traces to return the same traces). A q with no letter or digit is refused with a message rather than answered with an empty page. - The index is an SQLite FTS5 table in the same file, holding no second copy of the text (except, for a trace with Chinese, Japanese or Korean in it, the pieces its CJK text is indexed as: see the entry for words inside CJK text). Migration 015 creates it empty and the server indexes the traces already stored in the background after it starts, and builds the covering index a search's filters read after the start as well; a batch of 100 traces or more is indexed in the transaction that writes it, and a smaller one is queued there and indexed in batches after it (a search indexes the queue first, so it misses nothing just stored), and triggers take a trace out on every delete and update, including the retention sweep, --purge and a DELETE run by hand. A deleted trace's words are removed from the index as well as its row: tests delete a trace, run a sweep and a purge, checkpoint, and find none of the removed words in iris.db or iris.db-wal. - better-sqlite3's SQLite has FTS5; Node's built-in node:sqlite has it from Node 22.16.0, not on 22.13.0 to 22.15.0. On a SQLite without FTS5, search still works by reading the traces with the same matching (slower; search.index says scan), and the index is built on the first start that has FTS5. A new CI job checks both drivers on Linux, macOS and Windows, the built-in at Node 22.13.0 (the scan, on a real SQLite without FTS5) and 24. - Measured with npx tsx scripts/bench-trace-search.ts on an Intel Core i9-12900HK laptop (20 threads, 32 GB, Windows 11), Node 24.11.0, better-sqlite3 with SQLite 3.53.2, synthetic traces of about 140 words each, a page of 50 with the total counted, median of 15 runs. At 10,000 traces: 1.3 ms for a word in 0.1% of traces, 4.7 ms for one in 10%, 33.7 ms for one in every trace. At 100,000 traces: 3.1 ms, 35.6 ms and 258.5 ms; 9.0 ms for a word in 1% of traces and 168.8 ms for a phrase in half of them. The same queries without the index took 8 to 14 s at 100,000 traces. - What it costs, from the same run: at 100,000 traces the file grows from 125 MB to 211 MB, an insert takes 364 µs per trace instead of 149 µs, deleting one trace takes 14.5 ms, and a retention sweep of 3,007 traces 4.3 s. Building the index for 100,000 existing traces took 8.5 s in the background, measured separately with the server started in 228 ms (an integration test starts the real server over stdio on 40,000 unindexed traces and gets an answer, and a correct search, before the index is ready). - The dashboard's clickable table rows are no longer themselves buttons: a row opens on a click anywhere and from a named button in its first cell, so the copy-id button inside a trace row is reachable by keyboard and screen reader (axe reported it as nested-interactive).
• Search reads what a trace's spans say: the later model calls, tool results and exceptions of an agent loop sent over OTLP (#683). A trace from POST /v1/traces, which is how the provider wrappers and LangChain send, keeps only its first model call's input and output on the trace row; the rest was only in its spans, so a word from the third model call, a tool result or an exception message found nothing. q now also searches spans: the string values of each span's attributes and of its events' attributes, in span start order. - Never keys, numbers or booleans, and not the otel.span_id and otel.parent_span_id attributes Iris adds. A string that holds JSON, such as gen_ai.input.messages, is read by the strings inside it, so role and content are not words of every trace. Each distinct string is indexed once per trace, so the conversation history every model call re-sends adds nothing. At most 4,096 characters of one value and 32,768 of one trace's span text are indexed: words past either cap are not searchable. - match.field is spans for a match there, and match.span names the span (span_id, name); the dashboard shows the span's name beside the excerpt. Span text is weighted like tool-call and metadata values (half of input and output). - Deletes stay exact. The span text is computed in SQL from the spans table by every write and every trigger, and no copy of it is stored; a trace's words leave the index before its spans are deleted with it, and a span inserted, changed or deleted after its trace was indexed (by insertSpan, or by hand) re-indexes the trace. Tests check FTS5's own integrity check after each of those, and that a deleted trace's span words are not left in iris.db or its WAL. - Found on both SQLite drivers, and without FTS5, where the search reads the traces and reads a trace's span text only when its own fields lack a word and its stored span JSON could hold it. - What it costs, measured on the benchmark machine with the code before this change and after it run side by side in one process, each step alternated, on traces that are each an agent loop of three model calls and two tool calls: | | 10,000 traces, before | 10,000, after | 100,000, before | 100,000, after | |---|---|---|---|---| | File | 84 MB | 102 MB | 841 MB | 993 MB | | The index in it | 7.7 MB | 23.0 MB | 73.4 MB | 225.1 MB | | Insert, per trace | 493 µs | 1,320 µs | 564 µs | 1,675 µs | | delete_trace, median of 20 | 18.3 ms | 73.7 ms | 27.3 ms | 118.3 ms | | Retention sweep of 3% | 0.46 s | 1.30 s | 6.7 s | 22.9 s | | Search, a word in 1% of outputs | 4.1 ms | 7.7 ms | 8.7 ms | 12.0 ms | | Search, a word only in spans (1% of traces) | not found | 21.4 ms | not found | 28.3 ms | | Without FTS5, a word in 1% of outputs | 625 ms | 2,271 ms | 10.7 s | 23.5 s | | Without FTS5, a word only in spans | not found | 1,406 ms | not found | 36.0 s | The retention sweep row is how long the sweep takes; it now runs in the background, in steps, after the server answers (see Check before upgrading). Without FTS5 (and while the index is being built) a search reads the traces, and now reads span text too, for the traces whose own fields lack a word and whose stored spans could hold it: two to four times as long. Traces without spans cost the same as before: at 100,000, 211 MB either way, 317 µs per insert against 308, and delete_trace 14.7 ms against 15.2.
• A word inside Chinese, Japanese or Korean text is found (#682). Those languages put no spaces between words, and the index kept a run of CJK characters as one word, so 批准 ("approved") did not find 退款已经批准了; only the whole run did. A field that holds CJK is now also indexed as the overlapping two-character pieces of each run, and a CJK search word is cut the same way and searched as those pieces in order. A word of one, two or more characters is found anywhere inside a run, in order and within one run (批准了请 does not match across the comma in 批准了，请), with the index and without FTS5, and the snippet marks the matched characters inside the run. Latin written against CJK is a word of its own there (iphone finds iPhone充电器). - Search now splits text exactly as the index does. It used to approximate SQLite's tokenizer, which split Korean syllables into letters: a Korean word searched with the index found nothing, while the same search without FTS5 found it. The split is now read from a table of what SQLite's unicode61 tokenizer does with each of the 1,112,063 code points, generated from SQLite by npm run unicode61:render; a CI job regenerates it and fails when it differs, and a test checks 2,000 random strings drawn from Latin, Greek, Cyrillic, Hangul, kana, Han, emoji, private-use characters and combining marks against FTS5 itself. - Traces without CJK cost the same as before, measured with the code before this change and after it run side by side in one process, each step alternated, on traces that are each an agent loop of three model calls and two tool calls. At 10,000 traces: 1,410 µs per insert against 1,388, 102 MB either way, and a search for a word in 1% of traces 3.3 ms against 6.6 (the snippet's word split is faster now). At 100,000: 1,871 µs per insert against 1,793, 993 MB either way, delete_trace 276 ms against 268, a 3% retention sweep 24.8 s against 22.9 (an earlier run of this change came out the other way on each, by as much: the machine's own variation). With accented Latin text: 1,968 µs per insert against 1,877, and 130 MB either way. Their words never enter the CJK tables. - What CJK text costs, on the same agent loops written in Chinese, at 10,000 traces: | | Before | After | |---|---|---| | File | 128 MB | 251 MB | | Insert, per trace | 1,936 µs | 6,622 µs | | delete_trace, median of 20 | 24.5 ms | 244.9 ms | | Retention sweep of 3% | 1.96 s | 4.89 s | | Search, a CJK word inside a run (1% of traces) | not found | 28.5 ms | | Search, a Latin word in every trace | 54.6 ms | 86.6 ms | | Without FTS5, a Latin word in 1% of traces | 4.2 s | 6.9 s | | Index built after upgrading from 0.19.0 | 7.4 s | 50.8 s | Every CJK character is a word of the index twice over (in two pieces), and each is erased from the index when its trace is deleted, which is where the file size and the delete time go. - A trace with CJK text keeps a second, derived copy of that text in the file: the pieces its CJK is indexed as. The rest of the index holds no copy of any trace's text; SQLite recomputes what a delete must hand the index from the trace row itself. The pieces are cut by JavaScript, which a trigger cannot run, so the text each trace's pieces were given is stored beside them, and every delete hands it back. That copy leaves the file with the trace on every route: delete_trace, the retention sweep, --purge, a DELETE run by hand, and the metadata patch when it replaces the text. Its pages are zeroed as the trace's row is. Tests check the file's bytes after each route: none of the deleted trace's CJK pieces is left in iris.db or its WAL. A trace without CJK has no such copy.
• Every verdict says who recorded what it judged, and a capture harness can certify that no tool was called. An empty list of tool calls says two different things: from software that watched the agent and records every call, that none were made; from the agent's own report, only that it reported none, which an agent that hid a failed call can say as easily as one that made no call. A capture source now declares itself on the trace: capture: { name, version, complete } on POST /api/v1/traces and iris-eval ingest, or iris.capture.name, iris.capture.version and iris.capture.complete on an OTLP resource, where complete lists what it records in full (input, tool_calls, tool_outputs; over OTLP input and tool_outputs only, since one trace can arrive in several requests and a request without a TOOL span cannot show that none was made). Its empty list of tool calls is then evidence that no tool was called: it meets eval.requiredEvidence: ["tool_calls"], and the verdict's coverage says "no tool was called". A field it declares and the trace leaves out is a hole in the record, and the verdict is unknown (required_evidence_missing), with a sentence to the operator naming the capture source and the field; on the labelled corpus, leaving a declared field out never yields a pass (proof/INVARIANTS.md, four held rows). log_trace takes no capture, since the agent calls it and a record cannot vouch for itself. Each evaluation carries provenance.evidence: recordedBy (harness, agent, not_declared), the capture source, the inputs the call carried, and how many tool calls it carried; a stored evaluation reads back with the same, a trace scored again by its id is judged on the declaration it was stored with, and the dashboard's verdict panel says who recorded the evidence. The Claude Code capture plugin declares itself on every turn, and declares complete only what that part of the turn holds. Iris cannot check a declaration, as it cannot check the calls themselves: the contract says who made each claim, so a deployment can decide which sources it trusts. The evidence contract.

Changed

• `provenance.composer.calibration` names the calibration table, not the corpus it was measured on. The table behind the confidence label (src/eval/published-calibration.ts) is regenerated whenever a rule moves a verdict, while the composite corpus stays the same, so a stamp of the corpus's version would let a later table label a stored verdict again. The table now carries its own version, a hash over its content, and verdicts are stamped with it; a stored verdict is labelled again only under the table it was given with. Rows written by 0.19.0 already read back without a label, and still do. proof/COMPOSITE.md adds the AUC beside Brier and ECE: the chance a bad output gets a higher estimate than a good one, which does not move when every estimate moves.
• The published verdict accuracy is measured through the composer the server runs. The composite table was scored by a second composer kept in the harness, which agreed with the server's on every case and would have kept publishing its own answer if the two had drifted. The harness now calls compose(), a test holds that the verdict scored on each case is the one the engine gave, and the second composer is removed. proof/COMPOSITE.md also counts the three states apart, so a verdict that was not checked is never folded into the blocks or the passes. No published number changed.
• A Decision Moment whose trace has one passing evaluation and one that was not checked reads "No verdict". It read pass: the evaluation that could not be reached was dropped before the count. Its reason now names all three ways a verdict is not reached, where it named two.
• The 24 real transcripts: bundle verdicts agreeing with the answer key went from 78 to 82 of 96, and transcripts whose four bundles all agree from 11 to 13, because a bundle row is now the verdict read for its rules. Five recorded gaps closed and one opened (the placeholder detector lives in the safety bundle, and the answer key files a deferral under completeness); proof/TRANSCRIPTS.md lists each.
• iris-eval.com shows what exists today: the open-source server. The home page and the pricing page show it alone; hosted and team features are under consideration, not under construction. The planned hosted plans shown beside it, the waitlist form and the "Join Cloud Waitlist" button in the hero are removed; several features those plans listed were already in the open-source server. The signup route (POST /api/waitlist on the site) is removed and /waitlist redirects to /pricing. The terms' section for a future paid service is removed. The security page says Iris is self-hosted today, that none of your traces reach us, that Iris holds no certification and that tenant isolation is already enforced in the storage code, with what exists today and what does not, in place of "pre-SOC-2" and "enterprise customers". The privacy policy says the form is removed, that an address given while it was open is still held and is deleted on request, and that text pasted into the playground is scored and returned without being stored. The repository's sponsor link, which pointed at the waitlist, is removed.
• Community links point to GitHub, npm and X. The footer and the home page's community section list the places Iris is active; the "Discord (coming soon)" entry is removed.
• Iris's own test runs stay out of its public download count. The upgrade test takes the previous release's tarball from a cache and fetches it from the npm registry only on a cache miss. It used to install from the registry on every run, on three operating systems, and those installs were most of what the package's download count measured. The test installs the tarball it is given, refuses a missing one rather than falling back to the registry, and checks the version it installed. The total-downloads badge, whose all-time number includes those runs, is removed from the README and the site.
• Background work's steps are sized to stay under 50 ms, and the index build no longer runs FTS5's merges inside them. Each step of the index build, the retention sweep, the merge a sweep owes, the erasure of a retired index and the fill of stored risk estimates after an upgrade is now sized to 35 ms, from the work the last step did and how long it took, its commit included. A step grows by at most half the last one, so the next step has room before the 50 ms bound. The index build also turns FTS5's automerge off inside its own steps. Automerge had merged older segments inside whichever step's commit flushed, and those were the build's slowest steps. The build now merges the index's levels in steps of their own. Those merge steps are sized by the rows each one wrote, never by a budget it left unused, and capped at 512 pages, past which one merge's work comes in lumps. Measured on the benchmark machine against the code before this change, alternated, at 100,000 agent-loop traces, while other work shared the machine. A hold is read as this thread's CPU time, so time the process spent waiting to be scheduled is not counted. - Over stdio, the longest hold after the first answer was 78 and 62 ms instead of 125 and 126 ms while the index was built after an upgrade from 0.19.0. It was 78 and 64 ms instead of 125 and 126 ms while a retired index was erased and the index rebuilt, and 78 and 78 ms instead of 94 and 93 ms during a retention sweep of 3,000 traces. At 10,000 traces the build's longest hold was 48 to 63 ms instead of 94 to 109 ms; the sweep's did not improve (62 to 79 ms against 63 ms). - In one process, the build after an upgrade had 2 and 3 steps over 50 ms instead of 536 and 584, and took 70.4 and 70.7 s against 71.2 and 75.7 s. The sweep had 6 and 7 instead of 33 and 35. The rebuild after a retired index had 32 and 39 instead of 655 and 752, measured while the machine was busier. - The fill of stored risk estimates after an upgrade from 0.19.0 with 20,000 evaluations stopped each step at 50 ms of estimates and then committed, and the commit rewrote every row the step touched: 349 of 351 steps held the event loop over 50 ms, with 62 to 78 ms of this thread's CPU at the median. Sized with its commit, 45 of 644 steps did, and a step took 31 ms of CPU at the median (both drivers, two runs each).
• After upgrading, the server answers its client before it copies and migrates the database. The copy taken before a migration and the migrations themselves ran before the MCP connection opened, on the event loop: from 0.19.0 at 100,000 agent-loop traces, the first answer (initialize) came 5.0 to 7.3 s after the server started (six runs over stdio, median 6.7 s). The server now answers first, in 0.68 to 1.02 s (median 0.82 s), and copies and migrates after, on the checkpoint worker's connection, so neither holds the event loop: the copy is finished before the first migration writes, and nothing else writes until both are done. A tool call, a resource read or an HTTP request that arrives meanwhile waits for them, at most 30 s, and is then refused with a sentence saying what the server is doing (IRIS_STORAGE_ERROR, retryable; HTTP 503 with Retry-After); it is never answered from the older schema. The bound is for large files: the 790 MB file of 100,000 agent-loop traces copied in 3.5 s, so at that rate a million traces would take about 35 s to copy alone, and a request arriving then is refused with the reason rather than left waiting. The first get_traces call, sent right after initialize, still waits for the copy and the migrations: it was answered 6.2 to 7.8 s after the start (median 7.1 s), against 5.0 to 7.3 s (median 6.7 s) when both ran before the connection opened, so running them on the worker's connection costs about 0.4 s at the median on this first call after an upgrade. GET /health answers throughout, and its new upgrade field says whether the store is copying or migrating, or why the upgrade failed. The deployment's verdict labels are read before any request is let through, as they were before the connection opened. The stall guard now runs an upgrade from 0.19.0 at 10,000 traces, the start included: its longest hold was 48 ms, and 421 ms with the copy and the migrations back on the event loop. iris-eval ingest, --purge, --self-test and an embedder's initialize() still return with the file migrated; upgradeAfterStart on the adapter's options is what the server sets.
• After upgrading, the server answers before the covering index for search filters is built. Migration 015 built idx_traces_search_filter before the MCP connection opened, in one statement that reads every trace row. The server now builds it after the start, on the checkpoint worker's own connection, so the event loop stays free while it runs. It logs Search index: the covering index for search filters was built after the start, in N s when it is done. Until then a search is correct, and slower, because it joins through the traces' own key. The index holds the write lock while it builds: the store's own background work waits for it, and a trace written in that moment waits for it too, as it would behind any other writer. Measured in one process at 100,000 agent-loop traces from 0.19.0, the start, migrations included, took 1.87 and 2.23 s instead of 2.41 and 2.53 s; the index is 0.62 to 0.68 s of that. The rest of the start is migration 019's indexes: 1.69 to 1.87 s.
• Each connection compiles a statement once (#711). Both drivers keep what they prepare, up to 256 statements, and drop the least recently used first. A kept statement stays correct when the schema changes under it, including when another process changes it. Before this, every call compiled its SQL again, and with the search index that was most of a write: compiling the insert into spans compiles six triggers. log_trace over stdio takes 1.29 ms at 10,000 traces and 1.34 ms at 100,000 (2.25 and 2.34 ms without the cache, and 0.99 and 0.96 ms in 0.19.0). What still separates it from 0.19.0 is the search index's own write: a stored trace writes 38 pages to the log where 0.19.0 wrote 13.5.
• Storing one trace no longer writes the search index in the same commit (#729). log_trace and POST /api/v1/traces store the trace and queue its id. Indexing a trace on its own took 558 µs; in a batch of 1,000 it takes 95 µs. The queue is indexed when writes pause for 25 ms, in steps of about 35 ms under their own write lock, with FTS5's merges in steps of their own. During a stream of writes it is indexed only once this process has queued 1,000 traces, and a write that finds more queued indexes some of it first, so the queue never holds more. A batch of 100 traces or more (an OTLP request, iris-eval ingest) is indexed in the transaction that stores it, as before, because it is already a batch. A search indexes the queue first, in the same steps with other requests answered between them, so it finds every trace stored before it and ranks exactly as before. At 100,000 traces, a search that first indexed 990 queued agent-loop traces answered in 594 to 623 ms, complete, and held the event loop for 65 ms at the longest, with a checkpoint running. No step starts while the checkpoint worker truncates the log, which holds the write lock. Every delete route takes a queued trace off the queue, and its text leaves the file as an indexed trace's does. Measured over stdio on the machine in #711, five alternated rounds of 200 log_trace calls sent back to back: at 10,000 traces the median is 1.22 ms and the 95th percentile 1.65 ms (without this change 1.72 and 4.72 ms; 0.19.0 1.02 and 2.46 ms), and at 100,000 traces 1.32 and 2.64 ms (without it 1.92 and 3.50; 0.19.0 1.25 and 2.13). In a stream of 2,000 back to back, where the queue passes 1,000 and is indexed between writes, the 95th percentile at 100,000 traces is 3.9 ms against 3.0 without this change, and the 99th 36 ms against 27. Sustained OTLP ingest is unchanged by it: 1,999 traces/s at 100,000 traces against 1,935 without it, and 2,054 at 10,000 against 2,057 (0.19.0, which had no index: 2,396 and 2,680).
• `GET /api/v1/health` and `GET /health` carry `search`: the index's `state` (`ready`, `building` or `unavailable`), `index` (what a search reads now, `fts5` or `scan`) and `progress`, the share of stored traces indexed, from 0 to 0.99 while it is built. A share and not a count, because the endpoint answers without a key. It is not a check: a search answers in every state, so it never makes the status degraded. --self-test gains a line for the configured database's index (ready: 100,000 trace(s) indexed, or how many of how many, or that the SQLite has no FTS5), read without changing the file.
• GenAI message arrays are read down to their words, for every framework. The words reader added for LangSmith's LangChain envelopes now reads gen_ai.input.messages / gen_ai.output.messages (the conventions' parts schema) and OpenAI's [{ role, content }] the same way. The rules judge the answer's words and search finds them: before, ?q=role matched every GenAI trace and the rules read a JSON document. Proved per framework by the conventions fixtures (Pydantic AI, Semantic Kernel and LangSmith's documented shape change; the others already carried words), and through the OTLP door by one test that stores, searches and scores a message-array trace.
• The agent's own span is filed as the agent, not as a model call. invoke_agent and create_agent spans map to INTERNAL whatever else they carry; before, any non-empty gen_ai.operation.name made a span LLM, so the Agent Framework's invoke_agent (which carries the model's name and the run's usage) showed as a third model call beside two real ones. Usage is summed over the leaf carriers whatever their kind, so the totals are unchanged: the Agent Framework, Google ADK and Pydantic AI fixtures keep 1,742/136, 318/44 and 812/133 tokens, and the door test reads the same totals from GET /api/v1/traces/:id.
• Installing Iris succeeds where `better-sqlite3` cannot be built (#661). better-sqlite3 moves from dependencies to optionalDependencies. On a platform with no prebuilt binary and no C++ toolchain, npm now prints the build error, skips the module and finishes the install, and Iris runs on Node's built-in SQLite (node:sqlite), which it already fell back to when the module could not load. The warning, the startup log and --self-test now say which driver holds the file and why: better-sqlite3 loaded (the default), IRIS_SQLITE_DRIVER=node chose Node's built-in SQLite, better-sqlite3 is not installed …, or better-sqlite3 could not load (<reason>) …, and the advice differs (npm install better-sqlite3 for an absent module, npm rebuild better-sqlite3 for a broken one). A CI job installs the packed server into an empty project with the native build forced to fail (no prebuilt download, a compiler that always fails), requires the module to be absent and the install to finish, then requires --self-test to start the server, store a trace and read it back on node:sqlite, and iris-eval ingest --evaluate to store and score one more. The lockfile, regenerated on Linux, marks better-sqlite3 and the 32 packages only it pulls in as optional, and the two it shares with development tooling (detect-libc, semver) as dev-optional; no version moved. The MCP bundle's packer (scripts/mcpb/pack.mjs) now walks the root's optional dependencies as well as its dependencies, so it still finds better-sqlite3 and leaves it out by name.
• A filtered Decision Moments list returns full pages (#657). GET /api/v1/moments in its default time order applied the verdict, min_significance and significance_kind filters after reading a page of traces: on one agent's 240 traces with a failure every twelfth, ?verdict=fail&limit=10 returned 1 moment, the next page returned 0, and total said 240, with 20 failures in the store. With a filter set, the route now reads a window, the first window traces (default and maximum 500) matching the agent and time filters in the order asked for, the way sort_by=significance does. It filters every moment in it and cuts pages from that list. The same query returns 10, then the other 10, with total 20, and the response states the window. The dashboard's Moments page says how far back a filtered list reached ("Filtered within the last 500 traces of 1,200 …"), the Stream view's list of significant moments asks for a 200-trace window, the number of traces it read on each poll before, and the recent-moments row beside it now reads 200 traces instead of 48, so it finds moments further back. Unfiltered reads are unchanged.
• The dashboard loads each page's code when the page is shown (#662). Every page's code used to arrive in one bundle before the first page rendered: 576,945 bytes, 166,296 gzipped. Each page, and each of the four views of / (Failures, Health, Drift, Stream), is now its own chunk. The first load (the shell, the router, the design system and the data hooks the shell shares with the pages) is 308,822 bytes, 97,068 gzipped, and the default landing view needs 335,744 bytes, 109,015 gzipped, 34% less than before. The charting code Health and Drift share arrives only with them. Once the first page has rendered, the other pages' code is fetched while the browser is idle, so moving between pages does not wait on the network. A page whose code is still arriving shows "Loading page…" in a polite status region, visible only after 250 ms so a fast load does not flash, with the sidebar and header usable. A page whose code cannot be fetched, usually because Iris was upgraded while the tab stayed open, says so and offers a reload instead of the generic error. Hashed files under /assets/ are now served as immutable for a year, so a reload does not re-request each chunk against the rate limit the dashboard's polling shares, and a missing asset is a 404 instead of the app's HTML. The bundle budget gate now reads Vite's build manifest and holds the first load to 350 KB raw and 110 KB gzipped, and every other chunk to a budget of its own (the shared d3 chunk 52 / 19 KB, any other chunk 28 / 10 KB); a named budget that matches no chunk fails the gate. Upgrading React to 19.3 measures at 605,618 bytes, 175,048 gzipped, as a single bundle, over the old 168 KB gzip budget; on top of this change its first load is 337,430 bytes, 105,353 gzipped, inside the new one.
• Behaviour change: `verify_citations` reports a judge failure under `judge_error`, not `resolve_error` (#407). When a source resolved and the judge then gave no verdict (the cost cap stopped the call, the provider failed, or the reply could not be read), the citation carried resolve_status: "ok" and the judge's error under resolve_error, a field that says the source failed when it had not. A resolve_error.kind of timeout could have been either the fetch or the judge. Each citation now carries at most one of the two: resolve_error when the source was not resolved (resolve_status skipped or error), and judge_error: { kind, message } when it was and judge is absent. The kinds are unchanged: cost_cap_reached, malformed_judge_response and the provider kinds (auth, rate_limit, bad_request, server_error, timeout, malformed_response, unknown). Neither is scored as unsupported, and IRIS_JUDGE_FAILED still fires when every resolved citation fails at the judge. The stored evaluation's rule message now names what was not judged, by stage, for example 1/1 judged sources supported the output. Not judged: the judge failed on 1 resolved source (timeout); 1 source was not resolved (bad_status)., which the dashboard shows on the evaluation. A caller that read judge failures from `resolve_error` on a citation with `resolve_status: "ok"` reads them from `judge_error` instead.
• `evaluate_output` is back to 0.19.0's speed, reading evaluations is faster than 0.19.0, and every number is unchanged (#717). The risk estimate's 95% interval comes from 2,000 seeded draws. Since answers_the_ask advises instead of gating at the defaults, most evaluations reach the estimate, and on main it took about 8 ms of every call. It also ran on every read, because a stored evaluation re-composes its verdict when it is read: at 10,000 traces with 200 evaluated, GET /api/v1/failures took 2,946 ms on main, against 327 ms on 0.19.0. Three changes: - Each draw rebuilt the failure classes with Maps and filters. Now the class structure is worked out once per estimate, and each draw is flat loops over the same values in the same order. The draws cannot be shared between detectors, because all of them come from one seeded stream. - A finished estimate is cached (1,024 entries, least recently used evicted first), because it depends only on its detectors, prior and mode. - The estimate is stored with each evaluation (migration 018, two columns and an index), under the key of the inputs it came from, and a read uses it only when the row's own inputs give the same key. Evaluations stored before the upgrade get theirs in the background after the start: on 20,000 evaluations written by 0.19.0, a tenth of them with a labelled fire of their own, the start took 138–183 ms, the fill took 14.8 s in steps of median 59 ms (the longest 140 ms), and a page of 50 read during it took 58–62 ms. Once every row is filled, a start reads nothing. Creating the index at the upgrade took 608 ms at 100,000 evaluations. Measured with the same inputs, v0.19.0, main and this release alternated: | | 0.19.0 | main | now | |---|---|---|---| | evaluate_output over stdio, median (300 calls × 3 rounds) | 3.1–3.7 ms | 11.3–13.9 ms | 3.2–3.9 ms | | evaluate_output, p95 | 11.5–12.4 ms | 15.2–19.0 ms | 4.9–8.4 ms | | GET /api/v1/failures, a fresh process, 10,000 traces with 200 evaluated | 319 ms | 2,971–3,006 ms | 131–134 ms | | GET /api/v1/moments, the same | 106–109 ms | 790–795 ms | 80–83 ms | | 500 evaluations read at once, each fire carrying a different label count | 4,452–4,484 ms | 4,462–4,498 ms | 107–113 ms | HTTP ingest with evaluate: true goes from 25.7–26.8 ms on main to 14.6–15.4 ms, against 7.1–7.2 ms on 0.19.0. What remains is the agent failure-log query (#711), not the estimate. tests/unit/eval/risk-draws.test.ts keeps the 0.19.0 loop and sampler as a reference. It requires every one of the 2,000 draws to match bit for bit over 800 generated inputs (fired and silent classes, local labels, duplicate rule names, both prior modes, and priors equal to three decimals), and the finished estimate to match whether it was cached or not. tests/unit/storage/migration-016.test.ts requires the verdict read back to equal composing the row again. It plants numbers under a row's own key to show a read uses the stored estimate, and shows that a stored estimate under another key, or one that is not an estimate, is ignored. It also checks the background fill and the fill's query plans. Against main's build, 10,000 generated inputs gave no difference, and the verdicts and interpretations of the 500 labelled evaluations hash the same. npm run proof -- --check --composite regenerates the composite corpus's results unchanged.
• The dashboard sends compressed files and API responses (#718). The dashboard build writes a brotli (.br) and a gzip (.gz) copy of each file it emits, at the strongest settings. The server sends the copy the browser accepts, with Vary: Accept-Encoding, and never serves a copy older than its file. JSON answers of 1 KB or more to a read (GET or HEAD) from another machine are compressed per request (brotli quality 4 or gzip level 6, off the event loop). The answer to a write is sent as it is: it is an acknowledgement, and compressing it only kept the writer waiting. On the same 10,000-trace store, HTTP ingest with evaluate: true took a median 9.0–13.9 ms with this rule and 10.7–15.6 ms when its 201 was compressed, 1.7 ms less in each of three paired rounds, while every read kept the sizes below. Measured on a 10,000-trace store: | response | before | brotli | gzip only | |---|---|---|---| | first load of the dashboard | 363,007 bytes | 95,652 bytes | 110,169 bytes | | a page of 50 traces | 59,523 bytes | 18,791 bytes | | | a search page of 50 traces | 79,239 bytes | 20,338 bytes | | | /api/v1/capabilities | 62,640 bytes | 20,124 bytes | | A browser on plain http:// to a LAN address offers gzip only. A client on the same machine (a loopback address) gets API answers uncompressed: there is no network to save, and compressing and decoding a page of 50 traces took GET /api/v1/traces?limit=50 from 6.3 to 8.5 ms at 100,000 traces (#761). A reverse proxy on the same host connects over loopback too, and compresses for its own clients. The dashboard's files stay compressed on loopback, since they cost no CPU to send. Streamed responses (res.write, server-sent events) and non-JSON bodies pass through unchanged, as do responses that are marked Cache-Control: no-transform or already encoded. A browser request from another site (Sec-Fetch-Site: cross-site or same-site) is never compressed. This guards against BREACH: with compression, an attacker who can make a signed-in browser send requests carrying text of their choosing can read a secret in the same response from the response's length. The session cookie is SameSite=Lax, so the only cross-site requests that carry it are top-level navigations, and browsers mark those as cross-site. The compressed copies add 389 KB to the npm package. Proved by tests/unit/dashboard/compression.test.ts, which reads the bytes on the wire and decodes them, and by tests/e2e/compression.spec.ts, where Chromium and Firefox load the built dashboard compressed.
• The public claims schema (iris-eval.com/claims-schema-v1.json) requires sdkPackage in version and in version.published (the version of @iris-eval/sdk, and false until it is on npm), and now describes the top-level clients block .claims.json already carried, so the published file validates against its own schema again; a test holds the two together.

Fixed

• `evaluate_output` with an `expected_trajectory` no longer fails with `IRIS_INTERNAL_ERROR`. On the default bundle, and on completeness and cost, the response failed the tool's own output schema, which did not list expected_trajectory among the inputs a rule reads (rule_results[].saw). The schema now names every input, and a test holds it to the engine's list.
• A comparison no longer counts a rule that did not run as a pass. A run row held only the rules that fired, so a rule that skipped on a case read the same as one that passed it. A second run that stopped sending its tool calls had every trajectory rule skip, passed every case, and compare_runs answered better: true with "This is an improvement" (reproduced end to end: 5 of 12 against 12 of 12, one-sided p 0.0078). Each rule is now compared only over the cases it ran on, and over the shared cases it ran on in both runs when they pair; a rule that ran in one run and on no case in the other has no difference and no test. coverage.lost names the rules that ran on fewer cases in the second run, the summary says so before it gives the rates, each case that "recovered" names the rules that did not run on it, and better is not declared while any coverage was lost. A regression is declared either way.
• A comparison's words match its numbers. "This is a regression" could sit beside "95% interval [-28.6, +0.9]" and significant: false: the test was one-sided at 5% and the interval printed was the two-sided 95% one (45 of 50 against 38 of 50). The interval is now the 90% one that test reads, and the one-sided p is that interval inverted, so the word, the field and the interval agree on every input; a test checks every pair of rates at four run sizes, paired and unpaired. When the runs pair, the summary gives the share of changed cases that fell, with its exact interval, which is what McNemar's test reads.
• A comparison no longer calls runs "equivalent" within a margin nobody chose. With no equivalence_margin, the margin defaulted to the smallest difference the run sizes could detect, so two runs of eight cases, both 8 of 8, read "Equivalent within 40.3 points", and a rise from 1 to 5 leaked-data failures in 12 cases read "Equivalent within 26.3 points". Equivalence is now tested only against a margin the caller supplies, on an interval over every case rather than only the changed ones, and never while a critical rule fires on new cases or coverage was lost.
• A critical rule firing on new cases is named in the comparison. critical_rises lists each critical rule that fires on more cases in the second run, with the shared cases where it is new, and the summary states the count. The per-rule significance test could read "none reads worse" on five new leaks in twelve cases; the count is now given whatever the test says.
• `compare_traces` no longer prints a 95% interval of [100%, 100%]. The run-level interval was a percentile bootstrap over cases. Resampling cases that all passed returns the same answer every time, so the interval had no width whenever every case passed, and at 10 cases with a true pass rate of 95% it held the true rate two times in five (a seeded simulation of 2,000 trials). It is now the exact interval for that many cases, each case counted once at its own rate, which holds the true rate at least 93 times in 100 at 10, 20 and 50 cases for rates from 50% to 99%.
• "Could not have detected a change smaller than X" now means what it says. X was the half-width of the 95% interval at a pass rate of one half, which is the difference a test detects about half the time. It is now the difference detected four times in five (80% power), for the one-sided test a run comparison makes and the two-sided one the dashboard's drift view makes, and the sentence says "would have missed, more often than one time in five".
• The dashboard no longer draws a rule that never ran as "100% pass". The Drift page's per-rule meters counted a rule as passing on every trace where it was not in the failed list, so a rule that skipped on all 84 evaluations of a run showed a full bar titled "84 firings, 100% pass", and the header counted 84 × 25 "rule firings". A meter's rate is now over the traces the rule ran on; a rule that ran on none reads "not run (84)" with an empty bar; the header counts the checks that ran and says how many did not. The moments API carries ruleSnapshot.passed, the rules that ran and passed, beside failed and skipped.
• The run comparison page shows what each run was judged on. It lists the rules that ran on fewer cases in the second run and the critical rules firing on new cases, shows each rule's failures as "3 of 20" (the cases it ran on) or "not run", and marks a comparison where more cases passed under fewer checks as "JUDGED ON LESS" instead of "BETTER". The summary rendered literal ** markers; it is plain text now.
• Several server processes on one Iris home run the same rules and lose no rule. install gives each MCP client its own server process, and they share custom-rules.json. Each process read the file once at start and wrote its own copy back on every change. A rule deployed through one client was not applied by the others, two processes stamped different ruleset_hash values on the same output at the same moment, and the next deploy from a process that started earlier deleted every rule deployed since. A process now checks the file before an evaluation (one stat, no read, at most once every 20 ms) and re-reads it when it changed, and a change is made under a lock file on what the file holds at that moment. rules_changed counts a change another process made. A rules file that could not be parsed is read again once it is fixed, without a restart. A test starts two servers on one home and holds each of these; another has three processes deploy at once and counts the rules afterwards.
• A label written through one server process is read by the others. Labels are in the database, which the processes share, but each read them once at start, so a label moved the local precision and the prior only in the process it was written through. Each process now reads the database's change counter before an evaluation (at most once every 20 ms) and re-reads the labels when another process wrote one.
• The retention sweep no longer deletes the pinned baseline run. A baseline is the run every later run is compared against, and the sweep deleted it at retention.days like any other trace: a run pinned 31 days earlier compared against nothing. Its traces and evaluations are now kept until the run is unpinned, and each sweep that keeps some of it says so in the log. delete_trace and --purge remove them as before. Dataset cases and labels were never swept; a test now holds that.
• The HTTP endpoint serves more than one MCP client. --transport http created one transport for the life of the process. The first client to initialize owned it, a second got 400 Server already initialized, and after the first ended its session every later client got 404 Session not found until the server was restarted. Each initialize now opens a session of its own over the one engine and store, up to 256 at a time. Clients often leave without DELETE /mcp, so at the limit a new client takes the place of the session used least recently when that session has no event stream open and has been quiet for a minute; when every session is in use the new client gets 503 with Retry-After. Shutdown ends the open sessions first, so an open event stream no longer holds it for ten seconds.
• The site and the docs say what the code does, and a test holds each of these statements to the thing it describes. - What leaves the machine. The sentence the README, the site, llms.txt and the MCP manifest print (brand.dataResidency) names all four ways out, each off until you configure it: the OpenTelemetry exporter, the LLM judge, the citation check that fetches the pages an output cites (verify_citations), and webhooks. It named the first two, and the security page, which now prints the sentence, said "No data ever leaves your environment". tests/data-residency-names-every-exit.test.ts lists every source file that opens an outbound connection with the words the sentence must carry for it, so a fifth way out fails until it is named. - Which checks block a merge. The list is .github/required-checks.json: a test holds CONTRIBUTING.md and docs/proof.md to it, and a new job in the claims workflow compares it with the live branch protection on every pull request and push. The proof job is a required check, so a rule change cannot merge without its numbers, as docs/proof.md says. CONTRIBUTING.md had typed a shorter list and a Prettier check CI does not run. - The proof corpus. docs/proof.md and proof/README.md describe the blind sample as twenty cases from each of seven families (they called it "40-case"); say the real transcripts are measured on their own, every transcript-derived case having left the families on 2026-09-23; name the judgement families no human has sampled, where they said every rule outside the sample is arithmetic; and describe the risk composer as what the server composes every verdict with, where they called it "run in the harness only" and something "a future release may adopt". tests/proof/proof-docs-match-corpus.test.ts checks each against the corpus headers, the sample manifest and the tree. - The capability map. compare_traces has taken question since 0.15.0, so four multi-run cells are answered (has) on the same evidence the completion row already cited, and two keep a limit that still holds; all six had said "a per-question filter is not shipped". No cell says GET /api/v1/cases/:key takes the filter, which it does not. Three more cells agree with the rows beside them: recall by entity and recall under evasion transforms, which those rows publish, and which rule reads the structure of the ask (ask_coverage). The contract test reads the tool's schema and the route's. - The site. The live playground says beside the box that the text is posted to the site's server, and what is kept: a salted hash of the address for the rate limit, the category, a length bucket, the result and the duration, never the text. Its description had said the rules run "in your browser". The Cursor button installs Iris under the key iris-eval, as every other surface does, in place of server. Every homepage card metric reads from the truthbase, with its unit; "60s to first trace" and "$0.07 avg cost visibility per trace", which nothing measured, are replaced. The Output Quality Score page says the composer never consults the score for passed and that /proof publishes the verdict's numbers, where it said passed is "decided by the threshold" and that how the composed verdict performs "is not yet measured".
• The compare pages hold against the vendors' pages as of 2026-10-01. Every statement about the fourteen vendors was read again against their own docs, pricing and changelogs, not only the quoted sentence, and the cells that had stopped being true were corrected: Phoenix runs LLM-as-a-judge and code evaluators server-side, and its MCP server is the remote one built into Phoenix 19 and later; Langfuse and DeepEval score with Jev as a judge, and Confident AI offers it; Langfuse's MCP server has read and write tools on Cloud and self-hosted; Promptfoo's trajectory assertions read OpenTelemetry spans the app sends; LangSmith's standalone MCP server is deprecated for its hosted Remote MCP, and LangSmith and Weave accept OpenTelemetry spans; Weights & Biases is part of CoreWeave Forge; Patronus AI documents SDK prompt management, and the MCP server repository its blog links returned 404; Opik lists 30+ pre-built metrics, LLM-as-a-judge and heuristic; Latitude has no prompt manager in V2 and lists its ISO 27001 audit and GDPR review as in progress; Judgment's $32M covers its Seed and Series A; Helicone's AI Gateway repository is GPL-3.0. npm run compare:verify finds all 179 quoted sentences on their pages, and the dates on the pages say 2026-10-01.
• Closing the store releases the file before it returns (#750). The checkpoint worker's connection was asked to close and given 5 s; past that the thread was terminated and close() returned at once, while the thread was still in its statement and still held iris.db open. On Windows the file could not then be moved or removed until that statement ended, and two storage suites failed on windows-latest removing their directories (EPERM) after every assertion had passed. close() now resolves only once the thread has ended, and answers any request the thread was in with an error rather than leaving it waiting. A test holds the worker in a four-second statement with a 100 ms timeout: before the change close() returned with the thread still running (4 runs in 4, both drivers); now it waits for it. - The checkpoint thread stays held from the start of close() until it has ended. The answer to a checkpoint in flight used to release it again. If the thread then outlasted close()'s timeout, a short-lived process (iris-eval ingest, say) could exit with code 13 while close() still waited. The worker-exit CI job showed that release when ending the thread directly, in 1 process in 10.
• A search's page of trace ids, and the start's check for evaluations with no stored risk estimate, keep their plans when the file has statistics (#742). Iris never writes statistics, but ANALYZE run by hand does, and under them SQLite read the search index's whole trace_id index to find a page's ids, and every evaluation to find the ones without a risk estimate, at every start. The page's ids are now looked up one doc id at a time in a fixed join, in one statement for any number of matches (100,000 in 80 ms at 100,000 traces, against 92 ms in chunks of 1,000), and the fill names its index. The query-plan test covers every export read, and getEvalsByTraceIds names the index it already used.
• The website builds only when something its build reads has changed. Every pull request used to create a preview build of iris-eval.com, and one day of pull requests used up the host's daily build allowance, which the live site's production deploys share. A preview now builds only for a change to website/ or docs/blog/, and a production deploy also for a .claims.json change other than its timestamp and commit stamp. A branch's first deployment compares with main, and a base the host's shallow clone lacks is fetched; anything that fails builds. tests/website-build-scope.test.ts runs the real skip script in a shallow clone, as the host does, for six kinds of change, a branch's first deployment and an unreachable base, and fails if a website source reads any other path in the repository. CI's website job now runs the production build (next build) on every pull request, so a .claims.json change that would break the site is still caught without a preview.
• A database file is never opened by both copies of SQLite in one process (#739). better-sqlite3 carries its own SQLite and node:sqlite is Node's. Their locks cannot see each other within one process, and one can reset the WAL's shared-memory file while the other has it mapped, which kills the process with SIGBUS or corrupts the file. Measured on Linux with Node 24.21.0: SIGBUS in 3 runs of 5, against 0 of 10 with one copy. Iris never did this: the store and its two worker threads use the copy the store's connection got. The one place it happened was a test's stand-in search thread, which died of SIGBUS in 2 of 30 runs on macOS. Now the driver seam enforces it: a file already open in a thread is opened again only with the copy that holds it, and the other copy is refused before it opens anything. A new CI job ends worker threads that hold a connection every way Iris ends them, on Linux, macOS and Windows, Node 22 and 24, both drivers: 9,000 per cell. One process in one cell has since exited with an access violation, on Windows with Node 24. That is a Node.js bug in a thread's teardown (nodejs/node#65778), which the job reached by starting its threads with tsx's loader; an installed server's threads have none (#804). A cell takes 20 to 60 minutes, so the job runs on pull requests that touch storage, its fixtures or the dependencies, on every push to main, and nightly.
• Scoring a trace no longer reads every evaluation in the store (#711). Each evaluation with a cost reads its agent's recent history (the failure log), and so do /api/v1/moments, /api/v1/failures, the views and webhook moment events. There are two ways to read that history, and which one is fast depends on the data. 0.19.0 always read every evaluation of the store; the covering index added for search moved SQLite to walking every trace instead; with statistics it picked the slow way for a store where every trace is evaluated. The read now chooses between two queries that name their index and fix their join order, using counts of index entries. At 100,000 traces with every trace evaluated, log_trace with evaluate: true took 291 ms in 0.19.0 and takes 25.7 ms, POST /api/v1/traces with evaluate 291 ms and 24.9 ms, and GET /api/v1/moments 2.1 s and 0.8 s (medians of three rounds). The failure-log read itself went from 160 to 176 ms to 3 ms. On a store with few evaluations, where 0.19.0 was already fast, it stays between 0.1 and 3.5 ms. A test checks every result against the plain query on 12 seeded random stores per driver, through all three of the read's paths.
• Every hot read keeps its query plan, whatever SQLite's statistics say (#711). Iris never runs ANALYZE. With statistics from a store of 100,000 evaluated traces, SQLite made the failure log 35 times slower and the 30-day summary 2.4 times slower. The reads that run on every evaluation or dashboard render (the failure log, case results, get_traces with and without q, the trace search join, the filter lists, the summary and the eval stats) now name their index and fix their join order. tests/unit/storage/query-plans.test.ts checks each statement's plan on both drivers, with and without FTS5, with no statistics and under three sets of them. The test fails if a statement reads a whole table, or if a join starts from the wrong side or reaches the other side through the wrong index.
• The dashboard's first render reads index entries, not trace rows (#711). At 100,000 traces: GET /api/v1/filters 198 ms in 0.19.0, 1.4 ms now (one index seek per distinct agent and framework); GET /api/v1/summary?hours=720 284 ms and 62 ms; GET /api/v1/eval-stats?period=30d 172 ms and 33 ms. get_traces sorted by cost_usd or latency_ms sorts index entries and reads the 50 rows of the page: 187 ms and 27 ms.
• `get_traces` with `min_score` or `max_score` no longer walks every evaluation once per trace (#711). Finding each trace's latest evaluation took the tenant's evaluations in time order instead of the trace's own one or two. At 100,000 evaluated traces the count for one page ran for more than two minutes; the page and its count take 0.2 s now. 0.19.0 had the same plan.
• Case results for one case key or one session start from that case's traces (#711). A webhook reads them for each evaluation. At 100,000 evaluated traces the read took 290 ms and takes under 0.1 ms. 0.19.0 had the same plan.
• On Node's built-in `node:sqlite`, the first read after another process changed the schema returns every column (#711). node:sqlite counts a statement's columns before SQLite recompiles it for the new schema. That first read dropped a column that had just been added, and failed on a table rebuilt with fewer columns. This happened even to a statement prepared fresh. A read now checks the schema version first and has the connection load the new schema when it moved.
• Rebuilding the search index no longer holds the server while the old index is erased (#695). When the index has to be rebuilt from scratch (a file opened by a SQLite without FTS5, then by one with it), the old index used to be dropped in one statement, which rewrites and zeroes every page of it: at 100,000 agent-loop traces the event loop was held for 3.1 s in both of two runs over stdio (5.2 s in the issue, and up to 19.7 s in one run of the adapter alone). The old index is now erased a few hundred rows at a time before the rebuild indexes anything, each freed page zeroed as before, and the emptied tables are then dropped. Over stdio at 100,000 the longest hold during the erasure and the rebuild was 247 ms and 211 ms; at 10,000, 90 to 236 ms against 336 ms to 3.1 s. FTS5's own tables are read-only to SQL on a connection in SQLite's defensive mode, which better-sqlite3 turns on, so each step turns it off for its own statements and back on; a connection that refuses the write anyway drops the old index in one statement, as before, and logs a warning that says so.
• A write no longer waits for SQLite to copy the whole write-ahead log into the database file, and the log stays short. SQLite does that copy (a checkpoint) inside whichever commit takes the log past 1,000 pages, and on a large file that commit was the slow one: at 100,000 agent-loop traces, sweep steps that committed in 3 ms at the median took up to 567 ms when a checkpoint landed in them, and with the automatic checkpoint switched off the slowest of them took 20 ms (60 steps, two runs each). A worker thread with its own connection to the file now copies the log, 250 ms after each copy ends. A copy on another connection cannot start the log over while writes keep coming, so the server's own connection still checkpoints, at 16,000 pages (64 MB) instead of 1,000, and copies only what the worker has not. Before each step of background work, once the log holds 8,000 pages, the worker empties it and the step waits for that without holding the event loop. The worker also empties the log when a delete, a sweep or --purge asks. When the worker's own work holds that off (its connection still opening after the store's first write, or a copy in progress), the delete waits for the worker and has it empty the log, trying again for up to 250 ms with the event loop free, so the text is gone when the call returns. Every write of the server's own also waits for a TRUNCATE in flight, off the event loop, as the background steps already did: the TRUNCATE takes the write lock without waiting, and a write that landed in that instant turned it away, so under steady writes a delete could return with its text still in the file (with Node's built-in SQLite, in 2 runs of 3 of 20 deletes while another client wrote). A test now deletes 20 traces while another client writes on every turn of the event loop, and checks the file's bytes after each one. Only a search, or another process using the file, still leaves the erasure to the retry every 20 ms. Before, the worker opening was treated like a reader, and on a Windows runner with Node's built-in SQLite a deleted trace's words were still in iris.db when delete_trace returned. A stress test deletes 96 traces across 12 stores, each while the index is built, the worker copies and, for the first delete of each store, the worker is still starting, and checks the file's bytes after every call. It also found delete_trace failing with "database is locked" when a worker TRUNCATE landed between its read and its write (3 times in 26 runs): the delete now takes the write lock before it reads, after any TRUNCATE in flight. Until the worker is running, and if it ever stops, the server's own connection checkpoints as before, and a warning says so. A database in memory has no worker. Measured at 100,000 agent-loop traces against the worker alone, which let the log grow with every write: - 1,000 log_trace calls over stdio, three runs: 1.05 to 1.26 ms at the median instead of 1.28 to 1.46 ms, the slowest 1% 22 to 25 ms instead of 25 to 27 ms, and a 63 to 65 MB log instead of 153 MB. With the worker alone, every read and write on the server's connection looked through the whole log. 0.19.0 takes 0.67 to 0.77 ms; what is left of the difference is the search index's own write (the entry on the statement cache has it). - The search index built after an upgrade from 0.19.0, in one process, two runs: a 10 to 22 MB log at the end instead of 2.6 GB, the longest hold of the event loop 62 to 72 ms instead of 193 to 195 ms, and 66 to 69 s instead of 64 s. - How far the log runs past 64 MB before it starts over depends on the disk. The tail checkpoint waits while the worker's own copy holds the checkpoint lock, and SQLite reuses the log file without shrinking it, so the file keeps its largest cycle. Under 6,000 back-to-back writes at 10,000 traces the log started over about every 500 to 600 ms both on a desktop, where the file reached 63 to 67 MB, and on a hosted Linux runner, where cycles reached 105 to 138 MB. With the worker alone it started over twice and reached 1.1 GB. The stall guard asserts the restarts (the longest stretch over 64 MB without one stays under 2 s) and a 256 MB ceiling.
• A failing test in the truthbase capture is reported as a failing test, by name, not as drift. On main at 969719e the CI truthbase job said only "claims.json drifted from generator output". The tree was byte-identical to a pull-request head whose same job had passed; one test had failed during the capture (3,729 of 3,730), was recorded as failed: 1, and no line of the log named it. The capture now refuses a run with a failing test and lists each one with its file and the first line of its failure, and claims:check names every field that differs from the generator's output (for that run it would have printed tests.vitestRoot.passed: committed 3730, generated 3729).
• The truthbase capture refuses a run that exited non-zero with every test passing. An unhandled error leaves vitest's JSON report at success: true with every test passed; only the exit code says the run failed, and the report and its output do not carry the error. So the capture's rule for a run that failed outside any test (it read success: false) never fired, and a branch recorded 4,016 of 4,016 while CI failed its test jobs on an unhandled rejection. The capture now reads the exit code and refuses such a run, saying to run npx vitest run in the scope to see the error. A scope that wrote no report is unchanged: it keeps the committed counts, which the CI job that cannot install the dashboard relies on.
• The Python client's publish workflow parses again, and every workflow is linted on every pull request. Since the hash-pinned build landed, publish-python.yml held a plain YAML value with :all: -r in it, which YAML reads as a mapping, so GitHub refused the file on every push. Nothing noticed, because the workflow ran only on a py-v* tag. The step is now a block scalar. A new CI job runs actionlint 1.7.12, from an image pinned by digest, over every workflow in .github/workflows, including each run: script through shellcheck. It reported the parse error on the old file, and 6 shellcheck findings elsewhere, now fixed (four unguarded ls *.tgz globs in ci.yml, an unused loop variable and an unquoted kill $(cat …) in lighthouse.yml). publish-python.yml now also runs its build job on every pull request (the hashed install, the build and the fresh-environment import), and publishes only for a tag.
• `o1-mini` is priced at $1.10 in and $4.40 out per million tokens, not $3 and $12. The judge's table carried the model's launch price; OpenAI's model page (developers.openai.com/api/docs/models/o1-mini) lists $1.10 / $4.40, and its pricing page no longer lists the model. The judge's cost cap and cost report for o1-mini were 2.7 times too high, so the cap refused calls that fit. Every other row was checked against claude.com/pricing and developers.openai.com/api/docs/pricing on 2026-09-28 and matches, and the table's read date is now 2026-09-28.
• On Node 24, Iris no longer loads a better-sqlite3 that would abort the server when it frees a statement (#719). Node 24.19.0 changed the node::ObjectWrap header without the runtime change it needs (nodejs/node#65446). A better-sqlite3 compiled against that header, as npm does when the prebuilt download fails, aborts the process with Assertion failed: (env) != nullptr the first time V8 collects one of its statements, whether the database is open or closed. On Node 24.21.0 with 12.11.1, measured: a binary built from source aborted in 10 runs of 10, the prebuilt binary in 0 of 10. It aborted CI once, in 1 of the 62 failed jobs of the last 400 runs, the one whose install had compiled the addon. Iris now reads the binary before loading it. When the binary carries the new call and the runtime lacks the fix (every 24.x release so far, and 26.x before 26.4.0), Iris holds the store with Node's built-in SQLite and says why on stderr and in --self-test; GET /health reports driver: "node". IRIS_SQLITE_DRIVER=native refuses instead. npm rebuild better-sqlite3 restores the prebuilt binary, which is safe. A new CI job compiles the addon from source on Node 24 on Linux, macOS and Windows, and on Node 22 on Linux and macOS. It requires the binary to abort under collection exactly when Iris predicts it, and a real server started and stopped over stdio six times to end every session cleanly. With the check turned off, that stdio test failed 5 runs of 5 on the compiled binary.
• A stdio server shuts down in order when the client closes its stdin. MCP clients end a stdio session that way. The server used to drain and exit without closing the store, and a search-index build kept running for the client that had gone: 1.5 s on 80,000 traces, against 32 ms now. With the dashboard running, the end of stdin still does not stop the process.
• A published precision's interval is the one for the prevalence asked (#717). The intervals behind a rule's positive predictive value and miss rate were remembered per rule and prevalence to three decimals, but computed at the exact prevalence. So the first prevalence asked answered for every other that rounded the same: after an interval at 0.0101, one at 0.0104 came back as 0.0101's. An estimated prior is a continuous number, so a deployment with labels could see an interval for another prior. The intervals are now remembered per exact prevalence (4,096 entries, least recently used evicted first). The posterior draws, which depend on the rule alone, are drawn once per rule. tests/unit/eval/accuracy.test.ts fails on the old key, and it checks every rule at eight prevalences against intervals drawn afresh as before.
• `--self-test` no longer says PASS on a database the server then refuses (#704). It checked that the configured home was writable and ran everything else in a temporary home, so a file migrated by a newer release, or MCP clients pinned to a release that could no longer open it, passed. Two new steps read the configured database on a read-only connection and each MCP client's config: the first fails when this version cannot open the file, naming the version it needs, install --upgrade and the newest copy to restore, and otherwise says what the next start will apply; the second fails when a client is pinned to a release that cannot open the file, and warns when the next start will strand one. Neither changes the file: the tests compare its SHA-256 before and after.
• A trace stored by a process that does not index it is found by the next search. A trace inserted by hand, or by a release from before the search index, had no entry in the index, and a server already running answered searches from the index without it until its next start. Before each search the server now looks for traces added since the last one that the index lacks (a lookup of the newest row id, then of each new row in the index's own id table); if it finds any, that search reads the traces and the index takes them in behind it.
• A start refused over the database prints its sentence on its own line. The refusal reached the terminal only inside a structured log record with a stack trace; the sentence is now written plain to stderr first, as iris-eval: This database was migrated by a newer Iris ….
• Every test file is type-checked in CI (#721). tsconfig.json covers src/ only, so no file under tests/ was ever type-checked. On main, a config that includes tests/ and resolves modules the way Vitest does found 95 errors across the test files and 1 in the Playwright setup. - 53 errors cast client.callTool's result union to { content?: unknown }, and 4 read a resource's text without checking it was not a blob. Shared helpers in tests/helpers/ now narrow both and throw naming what arrived. - Three fixtures had drifted from the types: a count evidence item used name where the type has stat and unit, an e2e rule result said it saw cost_usd (a rule needs cost), and failure-log entries lacked costUsd. - Test files added since found 12 more: an export test's untyped id list and query objects, two more resource reads not checked for a blob (now resourceTextOf), an upgrade test reading cost_source off resolveTraceCost's inferred literal type, and checkReport's exitCode option, which the script's own types did not name (it now has a JSDoc signature). - npm run typecheck:tests checks tests/ under tsconfig.tests.json, and the Playwright specs under tsconfig.e2e.json with the DOM library, since their page callbacks run in the browser. The new CI job typecheck-tests runs it. The dashboard's tests were already checked by its own typecheck.
• The published test counts are no longer recorded from a run in which a test file failed to load. npm run claims:capture-tests read only vitest's totals, and a file that does not parse or throws at import counts none of its tests and no failure, so a run eleven tests short once recorded every test passing. The capture now refuses such a run, naming each file and the first line of its error, and writes nothing; it also refuses a run vitest marks failed with no failing test, and a run with a skipped test (which it used to fall back from to the committed counts). --report root=<file> and --report dashboard=<file> read an existing vitest JSON report instead of running the suite, which is how tests/unit/scripts/capture-report.test.ts proves the refusal on a real report of a file that fails to parse and one that throws at import.
• The Python recorder batches as it says it does. IrisRecorder waits flush_interval (0.25 s by default) to gather traces into one request, but every new trace woke that wait early, so an application recording steadily sent one request per trace and a full queue was never reached. The interval is now a deadline that only flush() or close() ends early. test_the_queue_is_bounded_and_drops_the_oldest yields between records, and fails on the old recorder every run.
• The Failures and Moments pages no longer re-run the regression-alarm watcher once per trace (#680). To find the alarms each trace in its window carries, the route re-ran every rule's CUSUM watcher over the agent's log up to that trace, so the cost grew with the square of the window and with the number of rules. The watchers now run once per request, and once more for each size of the rule family the log grows through; a log whose rules and runs are there from the start takes one pass. Every trace gets exactly the alarms it got before: a test checks the one pass against the per-trace answer for every trace of seeded logs where rules and runs start part-way through, where traces share a timestamp, and where alarms fire. On one agent's 500 traces the Failures and ranked Moments responses are byte-identical before and after. Measured on that window with no other load: with 25 rules, a warm request takes 286–344 ms, down from 2.5–2.6 s, and the first request after a start 4.5 s, down from 6.8 s; with 3 rules, 33–45 ms, down from 94–101 ms. The first-request cost that remains is the simulation that draws each stream's alarm line, which is memoised once drawn.
• The first Failures or Moments request after a start no longer spends seconds drawing alarm lines. Each stream's regression-alarm line came from a seeded simulation of about 150 ms, run the first time the stream's baseline settled, so a server's first request with 25 rules spent most of its time simulating. The lines are now generated ahead of time into src/eval/cusum-thresholds.generated.ts: every baseline a stream can first settle at, 2,267 of them up to 2,000 evaluations, which covers a rule whose tenth fail comes at a fail rate down to about 0.5%. A stream that settles later than that is simulated on first use, as before. Each row is exactly what the simulation returns, and a CI job, npm run cusum:check, regenerates every row from its seed and fails when the committed table differs. On one agent's 500 traces the Failures, ranked Moments and newest-first Moments responses are identical before and after, apart from the rank scores, which decay with the clock. Measured on that window with no other load, three runs each: with 25 rules the first request takes 188–216 ms, down from 4.5–4.9 s; with 3 rules, 98–103 ms, down from 497–541 ms.
• A busy host no longer gets ordinary custom regex rules killed as backtracking (#677). The regex sandbox's 100 ms budget ran from the moment a match was posted, so the time the worker thread waited to be scheduled was charged to the pattern. With every core busy, a pattern as plain as forbidden on a 40-character output was killed and reported as a budget skip, and a critical rule came back in critical_skipped with the verdict unknown, exactly when the server was busiest. Two changes fix it. The budget now starts when the worker starts the match. A fresh worker also warms the match path before it reports ready, so its first match does not pay inside the budget for code the thread has not run yet. A worker that has not started a match within 5 s is reported as a sandbox error, never as backtracking. A catastrophic pattern is still killed one budget after it starts. Under 20 busy-loop processes on a 20-core machine, the root test suite failed critical-rule-budget-killed in 2 of 5 runs and rules-preview's sample check in 1 of 5 before, and neither in 10 runs after. A harness with 60 busy processes (scripts/sandbox-load-harness.ts) ran 7,600 matches of forbidden with no false timeout, against 2 in 1,000 before. ^(a|a)*$ on 41 characters was still killed every time.
• LangGraph traces from LangSmith's OpenTelemetry export arrive readable. LangSmith sends gen_ai.prompt / gen_ai.completion as bytes, which OTLP/JSON writes as base64, so the trace's input and output were stored as base64. Bytes that hold UTF-8 text are now read as that text, and a LangChain envelope (a run's whole state as {"messages": [...]}, or a model's {"generations": ...}) is read down to the last question asked and the last answer given. Proven by a captured export of a real LangGraph run (tests/fixtures/otlp/langsmith-langgraph.pb) and by CI running the recipe against a real server.
• A tool call whose arguments arrive as a JSON string is checked as the object it encodes. gen_ai.tool.call.arguments (and tool_call.function.arguments, ai.toolCall.args) are JSON strings by convention, and the OpenTelemetry Python SDK can send no other form; valid_tool_arguments read the string and failed every such call against its tool's schema, which turned a correct tool loop into a fail verdict.
• The LangGraph recipe works as written. LangSmith posts to OTEL_EXPORTER_OTLP_ENDPOINT without appending /v1/traces and does not percent-decode OTEL_EXPORTER_OTLP_HEADERS; the recipe's endpoint and header lines now say so, and OTEL_SERVICE_NAME names the agent.
• The LangChain and CrewAI examples are complete programs, not scaffolds. examples/langchain/observe-agent.py is a LangGraph agent with IrisCallbackHandler (CI runs the same graph with a scripted model); examples/crewai/observe-crew.py is the CrewAI OpenInference recipe as a script. Each needs a model key to run.
• The published list of judge models is the list the judge accepts. llmJudgeTemplates.supportedModels in .claims.json was typed by hand. It named o1-mini, which the judge marks retired, and no Claude 5 model. It is now read from the pricing table the judge enforces, and a test holds the two equal.
• `answers_the_ask` has blocked at the shipped defaults since 0.18.0, contrary to its release notes; it now advises there, as documented. The 0.18.0 notes said it advises unless you set a relevance threshold. On every server built from its config it blocked. Its two measurements decided whether a threshold was yours by comparing the value with the number in the rule's own file, and the shipped config carries topic_consistency: 0.33 where the rule says 1/3, so the shipped default read as your setting. The unit tests built engines without the shipped config, so none of them saw it. keyword_overlap, topic_consistency, min_output_length, sentence_count and tool_choice now read where a threshold came from — your config file, or the context a caller passes — never its value, like every other rule. A deployment that sets the shipped number has still set it. tests/unit/eval/shipped-thresholds-read-as-shipped.test.ts builds the engine from loadConfig() and defaultConfig and requires every threshold stamp to read default at the defaults; it fails on the old code.
• The published composite numbers move, because they were measured with the rule blocking. The composite corpus runs the shipped config, so answers_the_ask was blocking in the measurement too, and failing stub and off-task cases that no other layer catches. At the shipped settings the risk composer is now right about shipping on 64.7% of the test split (95% interval 47.9–78.5%; was 76.5%) and 75.7% of dev (66.9–82.7%; was 81.1%). It now misses 45.8% of the outputs that should be blocked on test (was 29.2%) and 26.3% on dev (was 18.4%). False blocks on clean outputs do not change (10.0% on test, 20.0% on dev), nor do the real-transcript and per-class rows. Twelve composite verdicts change: five stub cases, four off-task cases and one stub-and-over-budget case now pass; two cases that also carry injection still fail, one on the risk estimate and one on a detector's veto. The per-output advantage over the legacy composer on test falls from 41.2 to 29.4 points (5.7–48.9). In the calibration table behind verdict.confidence, the 0.1–0.2 band holds 16 bad verdicts of 43 (was 12 of 39), and 69 of 112 labelled verdicts move from decisive to marginal (was 63 of 101). No rule's precision, recall or F1 moved.
• The response schema accepts a judge's `sample` evidence and the rule result's judge object. Before, a response carrying either failed the output-schema check.
• The evaluator matrix reads a measured `proof/judge-results.json`. It looked templates up by name in an array, so a keyed run would still have read as pending. It also derives the judge case count from the case files instead of the stale "165 cases across the five templates".
• A fresh clone builds with `pnpm install && pnpm run build` again. Since the dashboard joined build in 0.13.0, a clone that installed only the root package had no dashboard dependencies, so the build stopped at Cannot find package 'vite'. Directories that build Iris from its repository, Glama among them, have failed to build it since, and kept showing 0.10.0 and nine tools. build:dashboard now installs the dashboard's dependencies from its lockfile when they are missing, and a CI job builds from a fresh checkout with pnpm and requires every tool in tools/list.
• `npm run demo` and the release and tool-list checks no longer start Iris on your own data. npm run demo started the server on ~/.iris, so its traces were stored in the database of the Iris you use and the checkout's migrations were applied to it; a database migrated by a newer checkout is then refused by the installed release. The demo, scripts/check-tools-listed.mjs and npm run verify:release now start it with IRIS_HOME and IRIS_DB_PATH both set to a temporary directory they remove, over any the caller has set. tests/unit/scripts/check-tools-listed.test.ts runs the check against a stand-in server that answers only when both are set, and requires the caller's own home to be left untouched. The test setup also clears an IRIS_DB_PATH exported in the caller's shell, so npm test keeps every database in its scratch home.
• The dashboard works from the keyboard and reads correctly to a screen reader, and CI now checks it on every page in both themes (#724). - The view tabs carry the ids their panels name (each panel pointed at a tab id that was not on the page), are one Tab stop, and move with Left, Right, Home and End. The period selector sits beside the tablist instead of inside it, which axe rated critical (aria-required-children). The Time period and Cohort radio groups are one Tab stop each and move with the arrow keys. - Each sparkline on the Drift and Health views is named for its agent or rule; axe found 25 unnamed on the Drift view (svg-img-alt). - The account menu and the notifications popover move focus into themselves when opened, give it back to their button on Escape, and close when focus leaves them. The menu's items move with Up and Down, and Down on its button opens it. The command palette and the keyboard-shortcuts overlay keep focus inside while open and give it back on close; opening the tour from the palette used to leave focus on the page body. The palette's input is a combobox that controls a named listbox, and the selected option scrolls into view as the arrows move it. - The links on the not-found page are underlined, so they differ from the text around them by more than colour. - The Export button on the Traces and Evaluations lists names its panel in aria-controls only while the panel is open; closed, it named an id that was not on the page. - tests/e2e/a11y.spec.ts runs axe-core 4.12.1 (WCAG 2.0, 2.1 and 2.2 at A and AA, and best practice) on every route: 17 addresses, counting each dashboard view and the not-found page, in the dark and light themes, and on the five overlays open. It checks that every id an ARIA attribute or a <label> names is on the page, walks each page with Tab to its end (no trap, a visible focus indicator on every stop, every stop on screen, every control reached or in a group whose Tab stop is), and fails if a route in dashboard/src/routes.tsx has no entry. It runs in Chromium and Firefox; against the dashboard before this change, 19 of its Chromium cases fail. Colour contrast is enforced too (next entry).
• Dashboard text reaches WCAG AA contrast (4.5:1) in both themes (#725). axe counted 934 text elements below it on the 17 addresses above, counting each page separately (432 in the dark theme, 502 in the light): the muted text at 2.86 to 3.38:1, and in the light theme the pass, warn and fail colours at 1.72 to 3.76:1. Each colour keeps its hue and moves only in lightness (in OKLCH), to the nearest value that reaches 4.5:1 on every surface and tint it is drawn on: - --text-muted: dark #5e5e72 to #7d7e93, light #7a908e to #5a6f6e. - Light theme only: --eval-pass #22c55e to #007532, --eval-warn #eab308 to #886600, --eval-fail #ef4444 to #c50b20. In the dark theme --eval-fail goes from #ef4444 to #fc514f, the same hue lifted in lightness, because as text on its own chip tint it measured 4.1 to 4.4:1 (the verdict panel's basis chip on a moment, and a delete in the audit log, in Firefox); it is now 4.7:1 or more on every dark surface and tint. The solid destructive button keeps #ef4444 (--danger-fill). The dark theme's pass and warn colours already pass. - --eval-skipped #71717a: dark #8b8b94, light #6a6a73. - The red of the fail badges on their #450a0a fill: #ef4444 to #f44948. - Text on a teal fill (primary buttons, the notification count) is the near-black --text-on-accent in both themes. The light theme drew it in #fafcfc on #14b8a6 (2.4:1), and .iris-btn--primary drew white on #0d9488 (3.7:1), and on #14b8a6 when hovered (2.5:1). - tests/e2e/a11y.spec.ts now fails on any color-contrast violation. After the change: 0 on the same 17 addresses in both themes, and on the five overlays.

Earlier

One line per release, the bold lead of its changelog section; each links to the release notes built from that section.