v0.21.0A security release: what an agent or a client sends Iris is bounded before Iris reads it, and a tool argument can narrow what the operator allows but never widen it→

Comparison · Observability

Iris vs Arize

MCP-Native Eval vs Enterprise ML Observability.

TL;DR

Iris is an MCP server your agent discovers and uses on connect — one config block, no SDK, one SQLite file, every rule's precision and recall published at iris-eval.com/proof. Arize — from its own pages, read 2026-10-01: Arize describes itself as 'the agent observability and evaluation platform': Phoenix (open-source observability and evals, Elastic License 2.0) for self-hosted tracing, evaluation, experimentation and prompt iteration, plus Arize AX (managed AI engineering platform) with online evals, dashboards, the Alyx agent, SOC 2 Type II, SSO and RBAC. Embedding drift monitoring lives in its ML observability product. In August 2026 Arize announced a definitive agreement to be acquired by Dynatrace. (source)

Iris grades what an agent did with its tools — the trace, the answer, the cost — not whether an MCP server honours its own contract; a server test harness answers that question, and Iris runs beside it. For the method, see the agent eval guide.

Feature comparison

Side by side.

13 features, the same 13 on every comparison. Every Arize cell links the page it was read from and the date; where the sentence it was read from was found verbatim on a plain download of that page (checked 2026-09-22), the link carries it as its title, and where it was not, the cell says so. The highlighted cells are Iris's own call on which side is stronger for a team running MCP agents — 4 to Iris, 3 to Arize — not a measurement.

FeatureIrisArize
Integration methodOne block in the MCP config, no code — the agent discovers Iris and its tools on connectOpenTelemetry SDK + auto-instrumentationArize's page · read 2026-10-01
Self-hostingOne process, one SQLite file; Docker image with a health checkPhoenix: pip install arize-phoenix; SQLite by default, PostgreSQL supported; Docker, Kubernetes/Helm optionsArize's page · read 2026-10-01
Where it runsNothing in the agent's process — Iris is a separate server the agent callsOpenTelemetry SDK + OpenInference auto-instrumentation in the app, exporting to the Phoenix collector endpointArize's page · read 2026-10-01
Evaluation26 built-in deterministic rules and 9 custom-rule types, in-process; 7 judge templates on a key you supply; every rule's precision and recall publishedPhoenix: server-side LLM-as-a-judge and code evaluators, response and retrieval evals; AX: LLM-as-a-Judge, agent-as-a-judge or code evaluatorsArize's page · read 2026-10-01
Cost trackingPer-trace USD cost and tokens; a cost spike judged against the agent's own historyToken and cost tracking across modelsArize's page · read 2026-10-01
MCP supportProtocol-native — Iris is an MCP server with 12 tools; OTLP traces inRemote MCP built into Phoenix 19+, generated from its REST API (traces, datasets, experiments, prompts), with SQL over traces; the npm server is in maintenanceArize's page · read 2026-10-01
LicenseMIT, the whole packagePhoenix: Elastic License 2.0 (ELv2)Arize's page · read 2026-10-01
OwnershipIndependent and founder-ledDefinitive agreement signed for acquisition by Dynatrace (announced Aug 2026); close pendingArize's page · read 2026-10-01
DashboardA local dashboard on its own port: traces, moments, regressions, five viewsAX dashboards from templates or blank, widget-based; Alyx AI engineering agent; Prompt PlaygroundArize's page · read 2026-10-01
Framework supportAny MCP client (2 verified, 8 claimed — see /clients); OTLP/HTTP from anything else40+ models, frameworks and AI tools; 21 Python tracing integrations in Phoenix (OpenAI Agents, LangGraph, CrewAI, DSPy…)Arize's page · read 2026-10-01
Prompt managementNot includedPrompt Playground + Prompt Hub versioning; optimization manually, via Alyx, or automated Prompt LearningArize's page · read 2026-10-01
Enterprise and complianceSelf-hosted. Nothing leaves your machine unless you turn one of these on: an OpenTelemetry endpoint (IRIS_OTEL_ENDPOINT), which exports traces to the collector you name; the LLM judge with your own key, which sends the text it judges to that provider, and whose citation check fetches the pages an output cites; or a webhook, which posts ids, the verdict and rule names, never the text, to the address you set. No compliance certification is claimed before it is heldSOC 2 Type II (Pro/Enterprise); Enterprise SSO, org and space RBAC, self-hosted deployment; online evals on all tiersArize's page · read 2026-10-01
Cost to runFree — MIT, one process on your machine; the only spend is a judge call on a key you supply, when you opt inAX Free $0 (25k spans a month, 1 GB, 15-day retention); AX Pro $50/month (50k spans, 10 GB, 30 days); Enterprise custom; Phoenix is free to self-hostArize's page · read 2026-10-01

Decision guide

Which one fits your stack?

When to choose Iris

  • You are building with MCP-compatible agents and want the integration to be one config block
  • You want the evaluation to be deterministic and local — no model call, nothing leaving the machine
  • You want to read what each rule is worth before you trust it: every rule's precision and recall is published
  • You want self-hosting to be one process and one file
  • You want a fully permissive MIT license on the whole package

When to choose Arize

  • You need embedding drift monitoring for NLP, computer vision, or multi-variate tabular model data source
  • You need LLM-as-a-Judge, agent-as-a-judge, or code evaluators source
  • You need enterprise compliance and RBAC today source
  • You're using non-MCP frameworks and need broad OpenTelemetry-based instrumentation source
  • You want a Prompt Playground and Prompt Hub with versioning and automated Prompt Learning source
  • You're monitoring traditional ML models alongside LLM applications source

FAQ

The questions buyers ask.

What is the difference between Iris and Arize?
Iris is an MCP-native agent eval server that needs no SDK. Add the config block, restart your client, and every session lists Iris's tools on connect. Iris never intercepts: it runs when your agent calls one of its tools, when a host hook or `iris-eval ingest` hands it a trace, or when you POST one to its HTTP API. Arize calls itself 'the agent observability and evaluation platform': Phoenix (open-source observability and evals, Elastic License 2.0) for self-hosting, and Arize AX (managed AI engineering platform) in the cloud. In August 2026 Arize announced a definitive agreement to be acquired by Dynatrace. source
Is Iris better than Arize for MCP agent evaluation?
Arize requires OpenTelemetry instrumentation (Phoenix is built on top of OpenTelemetry with OpenInference auto-instrumentation) but offers embedding drift monitoring for ML models, LLM-as-a-Judge, agent-as-a-judge and code evaluators, and enterprise features such as SSO and RBAC. source
How much does Arize cost to run?
Iris: Free — MIT, one process on your machine; the only spend is a judge call on a key you supply, when you opt in. Arize, from its own page read 2026-10-01: AX Free $0 (25k spans a month, 1 GB, 15-day retention); AX Pro $50/month (50k spans, 10 GB, 30 days); Enterprise custom; Phoenix is free to self-host. source
Can I self-host Arize?
Iris: One process, one SQLite file; Docker image with a health check. Arize, from its own page read 2026-10-01: Phoenix: pip install arize-phoenix; SQLite by default, PostgreSQL supported; Docker, Kubernetes/Helm options. source
Does Arize work with MCP agents?
Iris: Protocol-native — Iris is an MCP server with 12 tools; OTLP traces in. Arize, from its own page read 2026-10-01: Remote MCP built into Phoenix 19+, generated from its REST API (traces, datasets, experiments, prompts), with SQL over traces; the npm server is in maintenance. source

Sources

Every Arize statement on this page was read from one of these pages on the date shown. The file behind this page is website/src/lib/compare/arize.json.

Last verified: 2026-10-01. This comparison is based on publicly available documentation and may not reflect recent changes to Arize. We aim to keep this page accurate and fair.

See something outdated or incorrect? Report an inaccuracy — we review and update within 48 hours.

Ready to see what your agents are doing?

Add Iris to your MCP config. First trace in 60 seconds. No SDK, no signup, no infrastructure.

Install IrisTry the playgroundEvery comparison