v0.20.0Iris says what it checked. A verdict tells a pass from a check that never ran, names who recorded the evidence it judged, and cannot be replaced by the agent it judges→

Comparison · Observability

Iris vs Weave

MCP-Native Agent Eval vs Decorator-Instrumented Agent Observability Platform.

TL;DR

Iris is an MCP server your agent discovers and uses on connect — one config block, no SDK, one SQLite file, every rule's precision and recall published at iris-eval.com/proof. Weave — from its own pages, read 2026-10-01: Weave is Weights & Biases' observability and evaluation platform for agents and LLM applications, integrated through a Python or TypeScript SDK with @weave.op decorators or existing OpenTelemetry spans. It traces calls with token usage and cost, runs evaluations with custom or predefined scorers, versions prompts, and can trace MCP servers and clients. CoreWeave acquired W&B on 2025-05-05; since 2026-09-30 W&B is a product within CoreWeave Forge, where Agent Lens (public preview) is the recommended experience for tracing agents. (source)

Iris grades what an agent did with its tools — the trace, the answer, the cost — not whether an MCP server honours its own contract; a server test harness answers that question, and Iris runs beside it. For the method, see the agent eval guide.

Feature comparison

Side by side.

13 features, the same 13 on every comparison. Every Weave cell links the page it was read from and the date; where the sentence it was read from was found verbatim on a plain download of that page (checked 2026-09-22), the link carries it as its title, and where it was not, the cell says so. The highlighted cells are Iris's own call on which side is stronger for a team running MCP agents — 3 to Iris, 3 to Weave — not a measurement.

FeatureIrisWeave
Integration methodOne block in the MCP config, no code — the agent discovers Iris and its tools on connectPython/TypeScript SDK: install weave, call weave.init, add @weave.op to functions; supported LLM clients auto-tracked; or send existing OpenTelemetry spansWeave's page · read 2026-10-01
Self-hostingOne process, one SQLite file; Docker image with a health checkSelf-managed: Kubernetes 1.29+, Helm, ClickHouse, S3-compatible storage, existing W&B Platform, Weave-enabled licenseWeave's page · read 2026-10-01
Where it runsNothing in the agent's process — Iris is a separate server the agent callsIn-process SDK: after importing and initializing weave inside your code, it records each request as a CallWeave's page · read 2026-10-01
Evaluation26 built-in deterministic rules and 9 custom-rule types, in-process; 7 judge templates on a key you supply; every rule's precision and recall publishedEvaluation over a Dataset with scorers: function-based, Scorer class, LLM-as-a-judge, predefined and local SLM scorersWeave's page · read 2026-10-01
Cost trackingPer-trace USD cost and tokens; a cost spike judged against the agent's own historyAutomatic token usage and cost per call for supported providers; custom per-token costs; not in the TypeScript SDKWeave's page · read 2026-10-01
MCP supportProtocol-native — Iris is an MCP server with 12 tools; OTLP traces inTraces MCP servers and clients (patches FastMCP and ClientSession); W&B also ships an MCP server to query Weave tracesWeave's page · read 2026-10-01
LicenseMIT, the whole packageWeave SDK is Apache-2.0; the hosted platform and the self-managed server require a license from W&BWeave's page · read 2026-10-01
OwnershipIndependent and founder-ledBuilt by Weights & Biases; CoreWeave completed its acquisition of Weights & Biases on 2025-05-05Weave's page · read 2026-10-01
DashboardA local dashboard on its own port: traces, moments, regressions, five viewsWeb UI with trace tree and calls table; Signals score every production trace automatically with 13 preset signals (monitors are the previous approach)Weave's page · read 2026-10-01
Framework supportAny MCP client (2 verified, 8 claimed — see /clients); OTLP/HTTP from anything elseOpenAI, Anthropic, Bedrock, Google, Mistral, LiteLLM; LangChain, LlamaIndex, CrewAI, DSPy, OpenAI and Claude Agent SDKsWeave's page · read 2026-10-01
Prompt managementNot includedPrompt objects (StringPrompt, MessagesPrompt) published with weave.publish and versioned on the project's Prompts pageWeave's page · read 2026-10-01
Enterprise and complianceSelf-hosted. Nothing leaves your machine unless you turn one of these on: an OpenTelemetry endpoint (IRIS_OTEL_ENDPOINT), which exports traces to the collector you name; the LLM judge with your own key, which sends the text it judges to that provider, and whose citation check fetches the pages an output cites; or a webhook, which posts ids, the verdict and rule names, never the text, to the address you set. No compliance certification is claimed before it is heldISO 27001/27017/27018, SOC 2 Type 2, HIPAA; SSO via OIDC, LDAP or SAML; SaaS, dedicated cloud, customer-managedWeave's page · read 2026-10-01
Cost to runFree — MIT, one process on your machine; the only spend is a judge call on a key you supply, when you opt inFree $0/mo for personal use; Pro starts at $60/month billed monthly, Weave data ingestion billed in arrears; Enterprise custom; self-hosting needs a licenseWeave's page · read 2026-10-01

Decision guide

Which one fits your stack?

When to choose Iris

  • You are building with MCP-compatible agents and want the integration to be one config block
  • You want the evaluation to be deterministic and local — no model call, nothing leaving the machine
  • You want to read what each rule is worth before you trust it: every rule's precision and recall is published
  • You want self-hosting to be one process and one file
  • You want a fully permissive MIT license on the whole package

When to choose Weave

  • You need LLM tracing and evaluation in the same platform as model experiment tracking (W&B Models). source
  • You need a prompt playground and leaderboards to compare evaluation runs side by side. source
  • You need auto-instrumented tracing across many providers and frameworks such as LangChain, LlamaIndex, CrewAI and the OpenAI Agents SDK. source
  • You need pre-built scorers for toxicity, bias, PII detection and hallucinations out of the box. source
  • You need an enterprise deployment with ISO 27001, SOC 2 Type 2 and HIPAA compliance, including dedicated cloud or customer-managed options. source

FAQ

The questions buyers ask.

What is the difference between Iris and Weave?
Iris is an MCP-native agent eval server that needs no SDK. Add the config block, restart your client, and every session lists Iris's tools on connect. Iris never intercepts: it runs when your agent calls one of its tools, when a host hook or `iris-eval ingest` hands it a trace, or when you POST one to its HTTP API. Weave is Weights & Biases' observability and evaluation platform for agents and LLM applications. You instrument Python or TypeScript code with the weave SDK and @weave.op decorators, or send existing OpenTelemetry spans, and it records traces, runs evaluations with scorers, and tracks token usage and cost. source
Is Iris better than Weave for MCP agent evaluation?
Weave offers things Iris does not: a prompt playground, leaderboards, pre-built scorers for toxicity, bias, PII and hallucinations, and a self-managed enterprise deployment on Kubernetes under a W&B license. source
How much does Weave cost to run?
Iris: Free — MIT, one process on your machine; the only spend is a judge call on a key you supply, when you opt in. Weave, from its own page read 2026-10-01: Free $0/mo for personal use; Pro starts at $60/month billed monthly, Weave data ingestion billed in arrears; Enterprise custom; self-hosting needs a license. source
Can I self-host Weave?
Iris: One process, one SQLite file; Docker image with a health check. Weave, from its own page read 2026-10-01: Self-managed: Kubernetes 1.29+, Helm, ClickHouse, S3-compatible storage, existing W&B Platform, Weave-enabled license. source
Does Weave work with MCP agents?
Iris: Protocol-native — Iris is an MCP server with 12 tools; OTLP traces in. Weave, from its own page read 2026-10-01: Traces MCP servers and clients (patches FastMCP and ClientSession); W&B also ships an MCP server to query Weave traces. source

Sources

Every Weave statement on this page was read from one of these pages on the date shown. The file behind this page is website/src/lib/compare/weave.json.

Last verified: 2026-10-01. This comparison is based on publicly available documentation and may not reflect recent changes to Weave. We aim to keep this page accurate and fair.

See something outdated or incorrect? Report an inaccuracy — we review and update within 48 hours.

Ready to see what your agents are doing?

Add Iris to your MCP config. First trace in 60 seconds. No SDK, no signup, no infrastructure.

Install IrisTry the playgroundEvery comparison