AI summaries, semantic search filters, and the remote MCP connector
Sessions, traces, and test results now open with an AI summary — how the session went, what a trace did, and the dominant failure modes behind a test result, with drill-down from each failure mode to the exact rows behind it.
Semantic search filters
Build test subpopulations with embedding-based semantic matching instead of exact or keyword matching — a filter for "billing problems" also catches "I was charged twice", with no shared keywords.
Remote MCP connector
The Openlayer MCP server now runs as a hosted, OAuth-protected connector — add https://mcp.openlayer.com to Claude Code, claude.ai, Cursor, VS Code, and other MCP clients, and work with your workspace in natural language. Your coding agent can pull test results and LLM-judge explanations to auto-improve your codebase, and validate changes against the rules and policies of your governance frameworks.
Also new this quarter
- Openlayer Agent Skills. Packaged skills that teach Claude Code, Cursor, and other AI coding assistants to wire a codebase into Openlayer — tracing, offline evals, tests, guardrails, and CI gating. Learn more →
- LLM cost explorer. A free public tool for browsing and comparing LLM pricing across providers, at llm-costs.openlayer.com — the same feed that powers Openlayer's cost tracking. Try it →
- Workspace dashboards. An at-a-glance view of cost, usage, and project health across the whole workspace. Open your dashboard →
- Webhooks for test results. Subscribe a webhook to test-suite outcomes and get notified when tests pass or fail. Learn more →
- openlayer init and headless CLI. One guided command takes a fresh directory to a linked, monitored, instrumented project;
link,login, and resource creation now run non-interactively for agents and CI. Learn more → - IBM watsonx Orchestrate integration. Joins Agentforce and Copilot Studio as a first-party agent integration. Learn more →
- Gateway upgrades. Per-user usage limits and guardrails, multimodal
/responsesand/messagesendpoints, semantic model-based routing, and a Helm chart for Kubernetes deployments. Learn more →
Full feature, improvement, and fix lists are below.
Features
- Integrations. Openlayer Agent Skills — packaged skills that teach AI coding assistants to integrate a codebase with Openlayer (tracing, offline evals, tests, guardrails, CI gating); ships as a Claude Code plugin, Cursor plugin, or via the skills CLI, and covers traditional/tabular ML as well as LLM workflows
- Integrations. Remote, OAuth-protected MCP connector — the MCP server runs hosted instead of local-only, usable in Claude Code and claude.ai, with in-chat workspace switching
- Integrations. Governance and lifecycle tools via MCP — frameworks, rules, and rule results are reachable through MCP, plus 12 new session, trace, and test-lifecycle tools (44 → 53 tools)
- UI/UX. Workspace-level dashboards — at-a-glance cost, usage, and project health across the workspace, with a table of contents and drill-in widgets
- UI/UX. AI-generated session summaries — session detail views open with an AI summary of the session arc across its traces
- UI/UX. AI-generated trace summaries — trace detail views show an AI summary instead of requiring line-by-line span reading
- UI/UX. AI-generated test-result summaries — test results include an AI summary of dominant failure modes
- UI/UX. Semantic search filters — embedding-based subpopulation filtering for tests, beyond exact and keyword matching
- API. Webhooks for test results — subscribe a webhook endpoint to test-suite outcomes and get notified when tests pass or fail (enabled in production — confirmed by Rishab 2026-08-08)
- Platform. Semantic (model-based) routing — routing rules can score each prompt in-process and send it to a strong or weak target by threshold, alongside static first-match rules
- CLI. `openlayer init` — one guided command takes a fresh directory to a pushable, linked, monitored project with tracing instrumentation
- CLI. Non-interactive CLI mode — `link`, `login`, and resource creation run headless, making the CLI usable from AI coding agents and CI
- Integrations. Native IBM watsonx Orchestrate integration — connect a watsonx Orchestrate instance, modeled on the existing Agentforce and Copilot Studio integrations
- SDKs. Gemini and Vertex tracing in Python and TypeScript — dedicated tracer for the unified Gemini SDK including Vertex, where previously only the legacy SDK was auto-instrumented
- SDKs. Claude Agent SDK tracing for TypeScript — drop-in import auto-traces agent runs (the Python equivalent shipped last window)
- Platform. Per-project data retention — set a rolling retention window (7/30/90/180/365 days or indefinite); monitoring data older than the window is deleted
- Platform. Pause and resume individual tests — pause one monitoring test without disabling the whole project
- Platform. Bulk session deletion — multi-select sessions and delete with a full cascade to their traces, scores, and embeddings
- Platform. Missing-guardrails governance rule — a new integrity test that flags traces missing an expected guardrail
- Platform. US National Provider Identifier as a native PII check — detectable in Contains-PII tests and redactable in traces
- Platform. Per-user usage limits and guardrails in the Gateway — limits and guardrails can target an individual user, including per-user caps within a team
- Platform. Multimodal Gateway support — the /responses and /messages endpoints support media sent and received, not just text
- Platform. Content safety guardrail — a new guardrail type for Gateway requests and responses
- Platform. LLM cost explorer (NEW: llm-cost-service) — a free public tool at llm-costs.openlayer.com for browsing and comparing LLM pricing across providers, including per-token-category pricing (cached, reasoning, audio); the same feed that powers Openlayer's cost tracking
- On-Prem. Helm chart for the Gateway — the Gateway ships an official Helm chart for Kubernetes deployments
- On-Prem. Deployment admin area — operators get background-job queue health, metrics, history, and license/feature status, gated by a dedicated deployment-admin role
Improvements
- Platform. Granular token usage and cost — cached, reasoning, and audio token categories are priced and stored, with per-API-key cost visible in the UI; LLM-as-judge evaluator runs are now cost-tracked per test result
- API. Expanded API-key surface — workspace API keys now work for updating projects, cost-override endpoints, test retrieval/update/deletion, and seven previously session-only read routes
- API. Update a project's name and description via API — new PATCH endpoint, picked up by the Python, Go, Java, and Ruby SDKs and the MCP server
- API. Sorting and filtering on more list endpoints — workspace rules, project members, and user sessions; rules tables also gained server-side group filtering and a more concise redesign
- CLI. Browser-based CLI login — sign in with a device-code browser flow instead of pasting an API key
- CLI. CLI export gains formats and scores — `openlayer export` supports CSV and XLSX (previously JSON only) and can include scores
- UI/UX. XLSX export in development and monitoring modes, all metrics included in downloads, clearer download date formats, and a batch of rules/frameworks/audit-trail quick wins
- UI/UX. Suggested filter values — filter fields suggest likely values as you type, served from the backend and respecting other filters already applied
- Platform. Faster row search and listing — row list and full-text search now return in sub-second time
- UI/UX. Drill down from an AI summary's failure modes to the exact rows behind each pattern
- UI/UX. Redesigned test creation page, revamped in-app settings, improved date filtering, and trace navigation search with jump-to-top/bottom controls
- UI/UX. Framework management — sorting, search, and filtering; confirmation before discarding changes; clearer framework documentation and a revised custom-framework creation flow
- UI/UX. Bulk test actions and metric-file management — duplicate several tests at once, schedule runs for several tests at once, and view/download/delete uploaded custom metric files
- SDKs. LangChain and LangGraph tracing modernized for v1 in Python and TypeScript, with LangGraph metadata mapping and large-trace handling; TypeScript also gained offline trace buffering and first-class session/user columns
- Platform. Gateway spend visibility — per-model breakdown on the usage endpoint, a per-user spend aggregation endpoint, projected savings shown when creating a limit, per-key caps within team limits, key rotation, and provider secrets settable from the admin UI
- On-Prem. On-prem deployment options — CSI Secrets Store support, external/managed ClickHouse, a domain and DNS assignment guide, native Oracle Object Storage as a storage backend, the deployed version shown in the UI, and a multi-arch Gateway image for ARM hosts
Fixes
- Platform. Batch uploads reported success while writing zero rows— fixed
- SDKs. Batch inference uploads from the Python SDK failed asynchronously while the client reported success — fixed
- Platform. Test evaluations could be silently skipped or dropped, and stalled monitoring evaluations never recovered — both fixed, with projects now auto-pausing after repeated data source connection failures
- Platform. High-volume session evaluations ran the worker out of memory, leaving tests stuck running or queued for hours — fixed
- Platform. Large-volume exports timed out or ran out of memory; exports now stream in bounded-memory chunks, preserve custom trace fields, and upload via multipart
- Platform. Session-level test scores read as null, went missing, or bled between two tests of the same subtype — fixed
- Platform. Session-based LLM-judge rubrics returned false positives — judges whose reasoning concluded a session passed were recorded as failing
- Integrations. Dialogflow and Agentforce sync reliability — memory-safe batch handling, missing custom parameters, stalled syncs, and cross-agent session scoping all fixed; Agentforce agents now auto-register
- UI/UX. AI summaries crashed or hung on content from certain providers, single-span traces, and rich-text test descriptions — fixed, with loading states corrected
- SDKs. LangChain traces using Gemini reported zero cost with empty usage details in both the Python and TypeScript SDKs — fixed
- SDKs. Background trace publishing grew memory without bound under sustained load, and the offline buffer could lose data under concurrency — both fixed in the Python SDK
- CLI. CLI fixes — `metrics pull` failed to extract some bundles, `metrics push` was rejected at registration and dropped enum parameter values, `push` gave a generic error when config was incomplete, and dataset-only commits failed server-side
- Platform. Gateway errors are returned in the client's native wire format, so tools no longer retry silently against an unreadable error; cost estimation now counts cached tokens and works for Azure AI Foundry; routing savings no longer display as negative
- Security. Activity-feed notifications surfaced projects the viewer had no access to — now scoped to visible projects
- On-Prem. On-prem fixes — deployments on Redis 8 no longer crash on index-not-found, signup is no longer blocked when SMTP is unconfigured, and the Helm chart wires the license key and correct app/worker replica counts
