v0.14.0The dashboard is a product→

Comparison · Observability

Iris vs Opik

MCP-Native Agent Eval vs Open-Source SDK-Traced LLM Observability.

TL;DR

Iris is an MCP server your agent discovers and uses on connect — one config block, no SDK, one SQLite file, every rule's precision and recall published at iris-eval.com/proof. Opik — from its own pages, read 2026-09-21: Opik is the open-source LLM observability and evaluation platform for AI agent tracing, LLM evaluation, prompt management, and production monitoring, built by Comet. It is Apache-2.0 licensed and free to self-host as the full platform. (source)

Iris grades what an agent did with its tools — the trace, the answer, the cost — not whether an MCP server honours its own contract; a server test harness answers that question, and Iris runs beside it. For the method, see the agent eval guide.

Feature comparison

Side by side.

Twelve features, the same twelve on every comparison. Every Opik cell links the page it was read from and the date. The highlighted cells are Iris's own call on which side is stronger for a team running MCP agents — 3 to Iris, 3 to Opik — not a measurement.

FeatureIrisOpik
Integration methodOne block in the MCP config, no code — the agent discovers Iris and its tools on connectPython and TypeScript SDKs with a @track decorator, framework integrations, a REST API, and OpenTelemetry.Opik's page · read 2026-09-21
Self-hostingOne process, one SQLite file; Docker image with a health checkSelf-host via Docker Compose (opik.sh; MySQL, Redis, ClickHouse, MinIO) or a Kubernetes Helm chart for production.Opik's page · read 2026-09-21
Where it runsNothing in the agent's process — Iris is a separate server the agent callsIn-process SDK instrumentation (decorator or integration) sends traces to the Opik server; OpenTelemetry supported.Opik's page · read 2026-09-21
Evaluation21 built-in deterministic rules and 9 custom-rule types, in-process; 6 judge templates on a key you supply; every rule's precision and recall publishedLLM-as-a-judge natural-language assertions, 30+ pre-built metrics, custom metrics, annotation queues for human review.Opik's page · read 2026-09-21
Cost trackingPer-trace USD cost and tokens; a cost spike judged against the agent's own historyToken usage measured per trace; USD cost estimates for OpenAI, Anthropic, Google, Bedrock, Groq; custom cost settable.Opik's page · read 2026-09-21
MCP supportProtocol-native — Iris is an MCP server with 12 tools; OTLP traces inOfficial MCP server (opik-mcp) lets Claude Code, Cursor, VS Code and others read traces, log scores, save prompts.Opik's page · read 2026-09-21
LicenseMIT, the whole packageApache 2.0 for the full platform (server, web app, evaluation); self-hosted deployments lack user-management features.Opik's page · read 2026-09-21
OwnershipIndependent and founder-ledBuilt by Comet (Comet ML, Inc.), headquartered in New York City, launched 2017, with $70M in funding per its about page.Opik's page · read 2026-09-21
DashboardA local dashboard on its own port: traces, moments, regressions, five viewsWeb dashboard for traces, experiments, feedback scores, trace counts and token usage over time; online evaluation rules.Opik's page · read 2026-09-21
Framework supportAny MCP client (1 verified, 9 claimed — see /clients); OTLP/HTTP from anything elseIntegrations: LangChain, LangGraph, LlamaIndex, CrewAI, OpenAI Agents, Pydantic AI, Vercel AI SDK, Google ADK, more.Opik's page · read 2026-09-21
Prompt managementNot includedPrompt Library versions each change automatically and links the version that ran to its trace; retrieved via get_prompt.Opik's page · read 2026-09-21
Enterprise and complianceSelf-hosted. Nothing leaves your machine unless you set IRIS_OTEL_ENDPOINT, which exports traces to the collector you name, or enable the LLM judge with your own key. No compliance certification is claimed before it is heldEnterprise plan lists SOC 2, ISO 27001, ISO 9001, HIPAA and GDPR compliance, single sign-on, and flexible deployments.Opik's page · read 2026-09-21

Decision guide

Which one fits your stack?

When to choose Iris

  • You are building with MCP-compatible agents and want the integration to be one config block
  • You want the evaluation to be deterministic and local — no model call, nothing leaving the machine
  • You want to read what each rule is worth before you trust it: every rule's precision and recall is published
  • You want self-hosting to be one process and one file
  • You want a fully permissive MIT license on the whole package

When to choose Opik

  • You need a fully Apache-2.0 platform you can self-host, backend included, without a commercial license. source
  • You need native integrations for LangChain, LlamaIndex, CrewAI, OpenAI Agents and dozens of other frameworks. source
  • You need a Prompt Library that versions every change and links the exact version to each trace. source
  • You need estimated USD cost tracking per trace across OpenAI, Anthropic, Google, Bedrock and Groq. source
  • You need annotation queues for human-in-the-loop review alongside 30+ pre-built LLM-as-a-judge metrics. source

Sources

Every Opik statement on this page was read from one of these pages on the date shown. The file behind this page is website/src/lib/compare/opik.json.

Last verified: 2026-09-21. This comparison is based on publicly available documentation and may not reflect recent changes to Opik. We aim to keep this page accurate and fair.

See something outdated or incorrect? Report an inaccuracy — we review and update within 48 hours.

Ready to see what your agents are doing?

Add Iris to your MCP config. First trace in 60 seconds. No SDK, no signup, no infrastructure.

Install IrisEvery comparison