v0.20.0Iris says what it checked. A verdict tells a pass from a check that never ran, names who recorded the evidence it judged, and cannot be replaced by the agent it judges→

Comparison · Observability

Iris vs Latitude

MCP-Native Agent Eval vs Open-Source Agent Observability Platform.

TL;DR

Iris is an MCP server your agent discovers and uses on connect — one config block, no SDK, one SQLite file, every rule's precision and recall published at iris-eval.com/proof. Latitude — from its own pages, read 2026-10-01: Latitude is an open-source, MIT-licensed observability platform for production AI agents, operated by Latitude Data, S.L. in Barcelona. It ingests traces through a TypeScript or Python SDK or OpenTelemetry, groups failures into signals, scores sessions with LLM-as-judge, deterministic or JavaScript evaluations, tracks token cost, exposes an MCP server for coding agents, and can be self-hosted for free or run as Latitude Cloud. (source)

Iris grades what an agent did with its tools — the trace, the answer, the cost — not whether an MCP server honours its own contract; a server test harness answers that question, and Iris runs beside it. For the method, see the agent eval guide.

Feature comparison

Side by side.

13 features, the same 13 on every comparison. Every Latitude cell links the page it was read from and the date; where the sentence it was read from was found verbatim on a plain download of that page (checked 2026-09-22), the link carries it as its title, and where it was not, the cell says so. The highlighted cells are Iris's own call on which side is stronger for a team running MCP agents — 3 to Iris, 2 to Latitude — not a measurement.

FeatureIrisLatitude
Integration methodOne block in the MCP config, no code — the agent discovers Iris and its tools on connectTypeScript (@latitude-data/telemetry) or Python (latitude-telemetry) SDK; OpenTelemetry exporter for other languagesLatitude's page · read 2026-10-01
Self-hostingOne process, one SQLite file; Docker image with a health checkFree self-hosting (Docker Compose or Helm): five services + Postgres/pgvector, ClickHouse, Redis, Temporal, object storeLatitude's page · read 2026-10-01
Where it runsNothing in the agent's process — Iris is a separate server the agent callsIn-process telemetry SDK in your app, or your existing OpenTelemetry pipeline exporting to Latitude; no proxy describedLatitude's page · read 2026-10-01
Evaluation26 built-in deterministic rules and 9 custom-rule types, in-process; 7 judge templates on a key you supply; every rule's precision and recall publishedLLM-as-judge, deterministic conditions or custom JavaScript score completed sessions; human annotations check alignmentLatitude's page · read 2026-10-01
Cost trackingPer-trace USD cost and tokens; a cost spike judged against the agent's own historyInput/output/total tokens and estimated cost per trace and session; Cost dashboard with per-model spend, cache adviceLatitude's page · read 2026-10-01
MCP supportProtocol-native — Iris is an MCP server with 12 tools; OTLP traces inRemote OAuth MCP server (api.latitude.so/v1/mcp) with tools generated from the Latitude API, for coding agentsLatitude's page · read 2026-10-01
LicenseMIT, the whole packageMIT on the default branch (development); the legacy V1 branches (main-old, latitude-v1) still carry an LGPL-3.0 LICENSELatitude's page · read 2026-10-01
OwnershipIndependent and founder-ledOperated by Latitude Data, S.L., Barcelona, Spain; funding and investors are not stated on its pagesLatitude's page · read 2026-10-01
DashboardA local dashboard on its own port: traces, moments, regressions, five viewsWeb UI with Traces, Sessions, Signals, Evaluations, Cost, Users, Tools, Memory, Search, Behaviors and Experiments viewsLatitude's page · read 2026-10-01
Framework supportAny MCP client (2 verified, 8 claimed — see /clients); OTLP/HTTP from anything elseOpenAI, Anthropic, Bedrock, Vercel AI SDK, LangChain and more, plus any OpenTelemetry-compatible applicationLatitude's page · read 2026-10-01
Prompt managementNot includedNo prompt manager in V2: prompts live in your codebase; the V1 prompt manager and gateway stop serving traffic on 30 October 2026Latitude's page · read 2026-10-01
Enterprise and complianceSelf-hosted. Nothing leaves your machine unless you turn one of these on: an OpenTelemetry endpoint (IRIS_OTEL_ENDPOINT), which exports traces to the collector you name; the LLM judge with your own key, which sends the text it judges to that provider, and whose citation check fetches the pages an output cites; or a webhook, which posts ids, the verdict and rule names, never the text, to the address you set. No compliance certification is claimed before it is heldSOC 2 Type II; ISO 27001 audit and GDPR readiness review in progress per its docs; SAML SSO, RBAC, custom on-prem or cloud deployment on the Enterprise planLatitude's page · read 2026-10-01
Cost to runFree — MIT, one process on your machine; the only spend is a judge call on a key you supply, when you opt inStarter free (20K credits a month, 30-day retention, unlimited seats); Pro $99/month (100K credits, 90-day retention); Enterprise custom; MIT to self-hostLatitude's page · read 2026-10-01

Decision guide

Which one fits your stack?

When to choose Iris

  • You are building with MCP-compatible agents and want the integration to be one config block
  • You want the evaluation to be deterministic and local — no model call, nothing leaving the machine
  • You want to read what each rule is worth before you trust it: every rule's precision and recall is published
  • You want self-hosting to be one process and one file
  • You want a fully permissive MIT license on the whole package

When to choose Latitude

  • You need to self-host the entire platform for free on Docker Compose or Kubernetes, with your data staying in your own infrastructure. source
  • You need recurring production failures auto-grouped into tracked signals with status, size and trend, Sentry-style. source
  • You need a fix loop that dispatches Claude Code or Cursor with the failing traces and replays the fix against those traces. source
  • You need a Cost dashboard with per-model economics and cache recommendations computed from your own traffic. source
  • You need OpenTelemetry-native ingestion from any language, not only Python or TypeScript. source

FAQ

The questions buyers ask.

What is the difference between Iris and Latitude?
Iris is an MCP-native agent eval server that needs no SDK. Add the config block, restart your client, and every session lists Iris's tools on connect. Iris never intercepts: it runs when your agent calls one of its tools, when a host hook or `iris-eval ingest` hands it a trace, or when you POST one to its HTTP API. Latitude is an open-source, MIT-licensed platform for improving production AI agents: it captures agent traffic through a telemetry SDK or OpenTelemetry, groups repeated failures into signals, scores sessions with evaluations, and can be self-hosted or used as Latitude Cloud. source
Is Iris better than Latitude for MCP agent evaluation?
Latitude offers things Iris does not: a free self-hostable platform with Sessions, Signals, Cost and Experiments views, LLM-as-judge and JavaScript evaluations that keep scoring live traffic, and a dispatch loop that hands failing traces to Claude Code or Cursor. source
How much does Latitude cost to run?
Iris: Free — MIT, one process on your machine; the only spend is a judge call on a key you supply, when you opt in. Latitude, from its own page read 2026-10-01: Starter free (20K credits a month, 30-day retention, unlimited seats); Pro $99/month (100K credits, 90-day retention); Enterprise custom; MIT to self-host. source
Can I self-host Latitude?
Iris: One process, one SQLite file; Docker image with a health check. Latitude, from its own page read 2026-10-01: Free self-hosting (Docker Compose or Helm): five services + Postgres/pgvector, ClickHouse, Redis, Temporal, object store. source
Does Latitude work with MCP agents?
Iris: Protocol-native — Iris is an MCP server with 12 tools; OTLP traces in. Latitude, from its own page read 2026-10-01: Remote OAuth MCP server (api.latitude.so/v1/mcp) with tools generated from the Latitude API, for coding agents. source

Sources

Every Latitude statement on this page was read from one of these pages on the date shown. The file behind this page is website/src/lib/compare/latitude.json.

Last verified: 2026-10-01. This comparison is based on publicly available documentation and may not reflect recent changes to Latitude. We aim to keep this page accurate and fair.

See something outdated or incorrect? Report an inaccuracy — we review and update within 48 hours.

Ready to see what your agents are doing?

Add Iris to your MCP config. First trace in 60 seconds. No SDK, no signup, no infrastructure.

Install IrisTry the playgroundEvery comparison