v0.14.0The dashboard is a product→

Comparison · Observability

Iris vs Galileo

MCP-Native Agent Eval vs SDK-Instrumented Enterprise AI Observability.

TL;DR

Iris is an MCP server your agent discovers and uses on connect — one config block, no SDK, one SQLite file, every rule's precision and recall published at iris-eval.com/proof. Galileo — from its own pages, read 2026-09-21: Galileo is an observability, evaluation, and guardrail platform for GenAI and agentic applications, logged through its SDK and scored with out-of-the-box or custom metrics. As of August 7, 2026 it is Splunk Agent Observability, following Cisco's acquisition. (source)

Iris grades what an agent did with its tools — the trace, the answer, the cost — not whether an MCP server honours its own contract; a server test harness answers that question, and Iris runs beside it. For the method, see the agent eval guide.

Feature comparison

Side by side.

Twelve features, the same twelve on every comparison. Every Galileo cell links the page it was read from and the date. The highlighted cells are Iris's own call on which side is stronger for a team running MCP agents — 4 to Iris, 3 to Galileo — not a measurement.

FeatureIrisGalileo
Integration methodOne block in the MCP config, no code — the agent discovers Iris and its tools on connectPython/TypeScript SDK: GalileoLogger, @log decorator, wrapped OpenAI client, LangChain callback; plus OpenTelemetry.Galileo's page · read 2026-09-21
Self-hostingOne process, one SQLite file; Docker image with a health checkEnterprise plan lists hosted, VPC, or on-prem deployment; Free and Pro tiers are hosted.Galileo's page · read 2026-09-21
Where it runsNothing in the agent's process — Iris is a separate server the agent callsIn-process SDK instrumentation; SDK integrations capture prompts and responses without explicit logging code.Galileo's page · read 2026-09-21
Evaluation21 built-in deterministic rules and 9 custom-rule types, in-process; 6 judge templates on a key you supply; every rule's precision and recall publishedOut-of-the-box metrics plus custom LLM-as-a-judge or code-based metrics; Luna-2 small language models score at scale.Galileo's page · read 2026-09-21
Cost trackingPer-trace USD cost and tokens; a cost spike judged against the agent's own historyExperiment comparison shows cost per response and metric cost, plus input/output token counts and latency.Galileo's page · read 2026-09-21
MCP supportProtocol-native — Iris is an MCP server with 12 tools; OTLP traces inShips an MCP server (api.galileo.ai/mcp/http/mcp) for IDEs: datasets, prompt templates, experiments, log-stream signals.Galileo's page · read 2026-09-21
LicenseMIT, the whole packagePlatform is proprietary (paid tiers); the galileo-python SDK on GitHub is under the Apache License 2.0.Galileo's page · read 2026-09-21
OwnershipIndependent and founder-ledCisco acquisition announced 2026-04-09, completed per Splunk; as of 2026-08-07 Galileo is Splunk Agent Observability.Galileo's page · read 2026-09-21
DashboardA local dashboard on its own port: traces, moments, regressions, five viewsWeb console at app.galileo.ai with projects and Log streams; a Playground runs experiments in the console.Galileo's page · read 2026-09-21
Framework supportAny MCP client (1 verified, 9 claimed — see /clients); OTLP/HTTP from anything elseIntegrations: OpenAI, OpenAI Agents SDK, LangChain, LangGraph, CrewAI, Google ADK, Strands, Vercel AI, Bedrock, OTel.Galileo's page · read 2026-09-21
Prompt managementNot includedPrompt templates created, stored and retrieved by name via the SDK for experiments; docs advise versioning them.Galileo's page · read 2026-09-21
Enterprise and complianceSelf-hosted. Nothing leaves your machine unless you set IRIS_OTEL_ENDPOINT, which exports traces to the collector you name, or enable the LLM judge with your own key. No compliance certification is claimed before it is heldSOC 2 Type II and HIPAA (BAAs) stated on its trust page; Enterprise plan lists RBAC, SSO, VPC or on-prem deployment.Galileo's page · read 2026-09-21

Decision guide

Which one fits your stack?

When to choose Iris

  • You are building with MCP-compatible agents and want the integration to be one config block
  • You want the evaluation to be deterministic and local — no model call, nothing leaving the machine
  • You want to read what each rule is worth before you trust it: every rule's precision and recall is published
  • You want self-hosting to be one process and one file
  • You want a fully permissive MIT license on the whole package

When to choose Galileo

  • You need Luna-2 small language models tuned for low-latency, reduced-cost metric evaluation at production scale. source
  • You need an enterprise VPC or on-prem deployment with RBAC, SSO and real-time guardrails. source
  • You need out-of-the-box metrics you can extend with custom LLM-as-a-judge or code-based metrics. source
  • You need SDK integrations for OpenAI, LangChain/LangGraph, CrewAI and OpenTelemetry that capture traces without explicit logging code. source
  • You need a platform with SOC 2 Type II certification and HIPAA Business Associate Agreements. source

Sources

Every Galileo statement on this page was read from one of these pages on the date shown. The file behind this page is website/src/lib/compare/galileo.json.

Last verified: 2026-09-21. This comparison is based on publicly available documentation and may not reflect recent changes to Galileo. We aim to keep this page accurate and fair.

See something outdated or incorrect? Report an inaccuracy — we review and update within 48 hours.

Ready to see what your agents are doing?

Add Iris to your MCP config. First trace in 60 seconds. No SDK, no signup, no infrastructure.

Install IrisEvery comparison