See what your AI agents are
Stop shipping agents on vibes. Install once. Every MCP client lists its tools on connect. Zero SDK. Decision Moments classify what matters, so safety violations and cost spikes surface before happy-path passes.
Works with any MCP-compatible agent
The Problem
Your agents pass every health check.
Infrastructure monitoring tells you the request succeeded. It cannot tell you the answer was wrong. Your agents need a quality gate — something that scores every output for safety, accuracy, and cost before it reaches a user.
Product
12 tools. One quality standard.
Iris registers as an MCP server. Your client lists its tools on connect, and your agent calls them. No SDK.
Every run you send it. Every tool call. Every token.
log_trace captures full agent runs with hierarchical spans, per-tool-call latency, token usage, and cost in USD.
- Hierarchical span tree with OpenTelemetry-compatible span kinds
- Per-tool-call latency tracking
- Token usage breakdown (prompt, completion, total)
- Arbitrary metadata for custom attribution
Built for
Three problems. One MCP server.
Every team building AI agents hits the same walls. Iris was built to tear them down — without touching your code.
“You deployed an agent and you have no idea what it's doing.”
Add one config block and every session lists Iris's tools on connect. No SDK. Your agent calls log_trace, a host hook forwards each turn, or iris-eval ingest loads a trace file, and every run Iris receives is scored.
“Your agent burned $0.47 on a single query and your APM showed 200 OK.”
Iris tracks cost per trace, per agent, per time window. Set budget thresholds and get flagged when agents overspend — before finance finds out.
“Your agent leaked a Social Security number in its output and nobody noticed for 3 months.”
Iris evaluates every output against 26 built-in rules including PII detection across 21 patterns (SSN, credit card, phone, email, IBAN, DOB, medical record number, IP address, API keys and vendor tokens, private key blocks, seed phrases), prompt injection (38 patterns), stub-output detection, and hallucination markers. Real-time, every trace.
Join the community
Open Source — Free to Self-Host
60 seconds to first trace.
Install Iris locally and start seeing what your agents are doing. Verified on every CI run in Claude Code, Gemini CLI; claimed for 8 more MCP clients from their own documentation — every row, with its source. Free, MIT-licensed. Nothing leaves your machine unless you turn one of these on: an OpenTelemetry endpoint (IRIS_OTEL_ENDPOINT), which exports traces to the collector you name; the LLM judge with your own key, which sends the text it judges to that provider, and whose citation check fetches the pages an output cites; or a webhook, which posts ids, the verdict and rule names, never the text, to the address you set.
{
"mcpServers": {
"iris-eval": {
"command": "npx",
"args": ["-y", "@iris-eval/mcp-server"]
}
}
}$ npx @iris-eval/mcp-server --self-test
✓ PASS — this install works
$ npx -y @iris-eval/mcp-server install claude-desktop
Added iris-eval in the Claude Desktop config: npx -y @iris-eval/mcp-server@0.21.0
$ npm install -g @iris-eval/mcp-server
$ iris-eval --dashboard
Dashboard available at http://localhost:6920
One-click install for Cursor IDE.
No config file needed.
Pricing
Free to self-host.
The open-source core is MIT licensed, with no usage limits and no account.
Self-Hosted
Everything you need to evaluate your MCP agents in production. Your machine, your data, your eval rules.
- 12 MCP tools — full lifecycle + LLM judge + semantic citation verify (SSRF-guarded)
- LLM-as-judge + citation verify use your own Anthropic/OpenAI API key (BYOK, no proxy)
- 26 built-in eval rules + custom rules
- Web dashboard with trace visualization
- SQLite storage — zero infrastructure
- Production security (auth, rate limiting)
- Cost tracking per trace
- Docker + npm + npx install
- Community support (GitHub Issues + Discussions)
The self-hosted server includes unlimited eval rules, both transports (stdio + HTTP), and the full API.
If something you need is missing, tell us: hello@iris-eval.com.
“I kept running into the same problem building AI agents: once they're running, you have no visibility into what they're actually doing. Traditional monitoring tells you the request succeeded. It can't tell you the agent leaked PII, hallucinated an answer, or burned through your budget on a single query.
So I built Iris — an MCP server every agent finds on connect. No SDK. Add it to your config, and every run your agent, a hook or the CLI hands it gets scored.
Research
Publications and insights.
Original research on MCP agent observability, evaluation methodology, and the evolving landscape of AI agent infrastructure.
Iris vs Langfuse vs Phoenix vs Promptfoo: where each wins, where each loses
Four ways to evaluate an AI agent, read from each vendor's own pages on 2026-10-01: how each gets into your stack, where the evaluation runs, what it costs to run, how it self-hosts, and what it does with MCP.
Read postThe State of MCP Agent Observability
The gap between deploying AI agents and understanding what they're doing. Covers protocol-native observability, heuristic vs. semantic eval, cost visibility, and EU AI Act implications.
Read reportEvery rule's precision and recall, published
Each built-in rule is measured on a public corpus, and the proof check is required on every pull request: a rule change cannot merge without its numbers.
Roadmap
Built in public. Shipping fast.
The dashboard is a product
Released 2026-09-20.
Reach, honestly
Released 2026-09-21.
Earn the comparison
Released 2026-09-22.
Hostile input, honest verdicts, a harder release path
Released 2026-09-23.
Honest numbers, a lighter footprint, nothing done silently
Released 2026-09-24.
Verdicts that say how sure they are, detectors that see through disguises, and one command to set up any client
Released 2026-09-25.
Iris says what it checked. A verdict tells a pass from a check that never ran, names who recorded the evidence it judged, and cannot be replaced by the agent it judges
Released 2026-10-03.
A security release: what an agent or a client sends Iris is bounded before Iris reads it, and a tool argument can narrow what the operator allows but never widen it
Released 2026-10-25.
What Iris cannot judge yet
Every question Iris answers, answers in part, or cannot answer yet, generated from the code.