v0.20.0Iris says what it checked. A verdict tells a pass from a check that never ran, names who recorded the evidence it judged, and cannot be replaced by the agent it judges→

Comparison · Observability

Iris vs Opik

MCP-Native Agent Eval vs Open-Source SDK-Traced LLM Observability.

TL;DR

Iris is an MCP server your agent discovers and uses on connect — one config block, no SDK, one SQLite file, every rule's precision and recall published at iris-eval.com/proof. Opik — from its own pages, read 2026-10-01: Opik is the open-source LLM observability and evaluation platform for AI agent tracing, LLM evaluation, prompt management, and production monitoring, built by Comet. It is Apache-2.0 licensed and free to self-host as the full platform. (source)

Iris grades what an agent did with its tools — the trace, the answer, the cost — not whether an MCP server honours its own contract; a server test harness answers that question, and Iris runs beside it. For the method, see the agent eval guide.

Feature comparison

Side by side.

13 features, the same 13 on every comparison. Every Opik cell links the page it was read from and the date; where the sentence it was read from was found verbatim on a plain download of that page (checked 2026-09-22), the link carries it as its title, and where it was not, the cell says so. The highlighted cells are Iris's own call on which side is stronger for a team running MCP agents — 3 to Iris, 3 to Opik — not a measurement.

FeatureIrisOpik
Integration methodOne block in the MCP config, no code — the agent discovers Iris and its tools on connectPython and TypeScript SDKs with a @track decorator, framework integrations, a REST API, and OpenTelemetry.Opik's page · read 2026-10-01
Self-hostingOne process, one SQLite file; Docker image with a health checkSelf-host via Docker Compose (opik.sh; MySQL, Redis, ClickHouse, MinIO) or a Kubernetes Helm chart for production.Opik's page · read 2026-10-01
Where it runsNothing in the agent's process — Iris is a separate server the agent callsIn-process SDK instrumentation (decorator or integration) sends traces to the Opik server; OpenTelemetry supported.Opik's page · read 2026-10-01
Evaluation26 built-in deterministic rules and 9 custom-rule types, in-process; 7 judge templates on a key you supply; every rule's precision and recall publishedLLM-as-a-judge natural-language assertions, 30+ pre-built metrics (LLM-as-a-judge and heuristic), custom metrics, annotation queues for human review.Opik's page · read 2026-10-01
Cost trackingPer-trace USD cost and tokens; a cost spike judged against the agent's own historyToken usage measured per trace; USD cost estimates for OpenAI, Anthropic, Google, Bedrock, Groq; custom cost settable.Opik's page · read 2026-10-01
MCP supportProtocol-native — Iris is an MCP server with 12 tools; OTLP traces inOfficial MCP server (opik-mcp) lets Claude Code, Cursor, VS Code and others read traces, log scores, save prompts.Opik's page · read 2026-10-01
LicenseMIT, the whole packageApache 2.0 for the full platform (server, web app, evaluation); self-hosted deployments lack user-management features.Opik's page · read 2026-10-01
OwnershipIndependent and founder-ledBuilt by Comet (Comet ML, Inc.), headquartered in New York City, launched 2017, with $70M in funding per its about page.Opik's page · read 2026-10-01
DashboardA local dashboard on its own port: traces, moments, regressions, five viewsWeb dashboard for traces, experiments, feedback scores, trace counts and token usage over time; online evaluation rules.Opik's page · read 2026-10-01
Framework supportAny MCP client (2 verified, 8 claimed — see /clients); OTLP/HTTP from anything elseIntegrations: LangChain, LangGraph, LlamaIndex, CrewAI, OpenAI Agents, Pydantic AI, Vercel AI SDK, Google ADK, more.Opik's page · read 2026-10-01
Prompt managementNot includedPrompt Library versions each change automatically and links the version that ran to its trace; retrieved via get_prompt.Opik's page · read 2026-10-01
Enterprise and complianceSelf-hosted. Nothing leaves your machine unless you turn one of these on: an OpenTelemetry endpoint (IRIS_OTEL_ENDPOINT), which exports traces to the collector you name; the LLM judge with your own key, which sends the text it judges to that provider, and whose citation check fetches the pages an output cites; or a webhook, which posts ids, the verdict and rule names, never the text, to the address you set. No compliance certification is claimed before it is heldEnterprise plan lists SOC 2, ISO 27001, ISO 9001, HIPAA and GDPR compliance, single sign-on, and flexible deployments.Opik's page · read 2026-10-01
Cost to runFree — MIT, one process on your machine; the only spend is a judge call on a key you supply, when you opt inOpen source free to self-host (same codebase as hosted); Free Cloud $0 (10 team members, 25k spans a month); Pro Cloud $19/month (100k spans, 60-day retention)Opik's page · read 2026-10-01

Decision guide

Which one fits your stack?

When to choose Iris

  • You are building with MCP-compatible agents and want the integration to be one config block
  • You want the evaluation to be deterministic and local — no model call, nothing leaving the machine
  • You want to read what each rule is worth before you trust it: every rule's precision and recall is published
  • You want self-hosting to be one process and one file
  • You want a fully permissive MIT license on the whole package

When to choose Opik

  • You need a fully Apache-2.0 platform you can self-host, backend included, without a commercial license. source
  • You need native integrations for LangChain, LlamaIndex, CrewAI, OpenAI Agents and dozens of other frameworks. source
  • You need a Prompt Library that versions every change and links the exact version to each trace. source
  • You need estimated USD cost tracking per trace across OpenAI, Anthropic, Google, Bedrock and Groq. source
  • You need annotation queues for human-in-the-loop review alongside 30+ pre-built metrics, LLM-as-a-judge and heuristic. source

FAQ

The questions buyers ask.

What is the difference between Iris and Opik?
Iris is an MCP-native agent eval server that needs no SDK. Add the config block, restart your client, and every session lists Iris's tools on connect. Iris never intercepts: it runs when your agent calls one of its tools, when a host hook or `iris-eval ingest` hands it a trace, or when you POST one to its HTTP API. Opik is Comet's open-source LLM observability and evaluation platform: you instrument your app with its SDK or OpenTelemetry, and traces, experiments and prompts are managed in its dashboard. source
Is Iris better than Opik for MCP agent evaluation?
Opik offers production monitoring dashboards with online evaluation rules, a Prompt Library with automatic versioning, estimated cost tracking, and a large set of native framework integrations. source
How much does Opik cost to run?
Iris: Free — MIT, one process on your machine; the only spend is a judge call on a key you supply, when you opt in. Opik, from its own page read 2026-10-01: Open source free to self-host (same codebase as hosted); Free Cloud $0 (10 team members, 25k spans a month); Pro Cloud $19/month (100k spans, 60-day retention). source
Can I self-host Opik?
Iris: One process, one SQLite file; Docker image with a health check. Opik, from its own page read 2026-10-01: Self-host via Docker Compose (opik.sh; MySQL, Redis, ClickHouse, MinIO) or a Kubernetes Helm chart for production. source
Does Opik work with MCP agents?
Iris: Protocol-native — Iris is an MCP server with 12 tools; OTLP traces in. Opik, from its own page read 2026-10-01: Official MCP server (opik-mcp) lets Claude Code, Cursor, VS Code and others read traces, log scores, save prompts. source

Sources

Every Opik statement on this page was read from one of these pages on the date shown. The file behind this page is website/src/lib/compare/opik.json.

Last verified: 2026-10-01. This comparison is based on publicly available documentation and may not reflect recent changes to Opik. We aim to keep this page accurate and fair.

See something outdated or incorrect? Report an inaccuracy — we review and update within 48 hours.

Ready to see what your agents are doing?

Add Iris to your MCP config. First trace in 60 seconds. No SDK, no signup, no infrastructure.

Install IrisTry the playgroundEvery comparison