# AI summaries, semantic search filters, and the remote MCP connector

Published: 2026-08-10

[AI-generated session summary](https://cdn.sanity.io/images/m9qfap23/production/b2f1e3179a80a78920388f12b616cf3dff9383a0-2400x1534.jpg?w=1400&fit=max&auto=format)

Sessions, traces, and test results now open with an AI summary — how the session went, what a trace did, and the dominant failure modes behind a test result, with drill-down from each failure mode to the exact rows behind it.

[Read the docs →](https://www.openlayer.com/docs/monitoring/ai-summaries)

## Semantic search filters {#semantic-search-filters}

Build test subpopulations with embedding-based semantic matching instead of exact or keyword matching — a filter for "billing problems" also catches "I was charged twice", with no shared keywords.

[Semantic search filters](https://cdn.sanity.io/images/m9qfap23/production/cc331d612170b5793cdd396b76e95e898cf2b055-2400x1534.jpg?w=1400&fit=max&auto=format)

[Read the docs →](https://www.openlayer.com/docs/tests/semantic-search-filters)

## Remote MCP connector {#remote-mcp-connector}

The Openlayer MCP server now runs as a hosted, OAuth-protected connector — add https://mcp.openlayer.com to Claude Code, claude.ai, Cursor, VS Code, and other MCP clients, and work with your workspace in natural language. Your coding agent can pull test results and LLM-judge explanations to auto-improve your codebase, and validate changes against the rules and policies of your governance frameworks.

[Openlayer remote MCP connector](https://cdn.sanity.io/images/m9qfap23/production/51306e6d49f712a3e5234a63e337f33a5c6c7f86-2400x1534.jpg?w=1400&fit=max&auto=format)

[Read the docs →](https://www.openlayer.com/docs/openlayer-mcp)

## Also new this quarter {#also-new-this-quarter}

- **Openlayer Agent Skills. **Packaged skills that teach Claude Code, Cursor, and other AI coding assistants to wire a codebase into Openlayer — tracing, offline evals, tests, guardrails, and CI gating. [Learn more →](https://www.openlayer.com/docs/openlayer-skills)
- **LLM cost explorer. **A free public tool for browsing and comparing LLM pricing across providers, at llm-costs.openlayer.com — the same feed that powers Openlayer's cost tracking. [Try it →](https://llm-costs.openlayer.com/)
- **Workspace dashboards. **An at-a-glance view of cost, usage, and project health across the whole workspace. [Open your dashboard →](https://app.openlayer.com)
- **Webhooks for test results. **Subscribe a webhook to test-suite outcomes and get notified when tests pass or fail. [Learn more →](https://www.openlayer.com/docs/security/webhooks/overview)
- **openlayer init and headless CLI. **One guided command takes a fresh directory to a linked, monitored, instrumented project; `link`, `login`, and resource creation now run non-interactively for agents and CI. [Learn more →](https://www.openlayer.com/docs/api-reference/cli/commands/init)
- **IBM watsonx Orchestrate integration. **Joins Agentforce and Copilot Studio as a first-party agent integration. [Learn more →](https://www.openlayer.com/docs/integrations/ibm-watson-orchestrate)
- **Gateway upgrades. **Per-user usage limits and guardrails, multimodal `/responses` and `/messages` endpoints, semantic model-based routing, and a Helm chart for Kubernetes deployments. [Learn more →](https://www.openlayer.com/docs/gateway/overview)

Full feature, improvement, and fix lists are below.

## Features

- Openlayer Agent Skills — packaged skills that teach AI coding assistants to integrate a codebase with Openlayer (tracing, offline evals, tests, guardrails, CI gating); ships as a Claude Code plugin, Cursor plugin, or via the skills CLI, and covers traditional/tabular ML as well as LLM workflows
- Remote, OAuth-protected MCP connector — the MCP server runs hosted instead of local-only, usable in Claude Code and claude.ai, with in-chat workspace switching
- Governance and lifecycle tools via MCP — frameworks, rules, and rule results are reachable through MCP, plus 12 new session, trace, and test-lifecycle tools (44 → 53 tools)
- Workspace-level dashboards — at-a-glance cost, usage, and project health across the workspace, with a table of contents and drill-in widgets
- AI-generated session summaries — session detail views open with an AI summary of the session arc across its traces
- AI-generated trace summaries — trace detail views show an AI summary instead of requiring line-by-line span reading
- AI-generated test-result summaries — test results include an AI summary of dominant failure modes
- Semantic search filters — embedding-based subpopulation filtering for tests, beyond exact and keyword matching
- Webhooks for test results — subscribe a webhook endpoint to test-suite outcomes and get notified when tests pass or fail (enabled in production — confirmed by Rishab 2026-08-08)
- Semantic (model-based) routing — routing rules can score each prompt in-process and send it to a strong or weak target by threshold, alongside static first-match rules
- `openlayer init` — one guided command takes a fresh directory to a pushable, linked, monitored project with tracing instrumentation
- Non-interactive CLI mode — `link`, `login`, and resource creation run headless, making the CLI usable from AI coding agents and CI
- Native IBM watsonx Orchestrate integration — connect a watsonx Orchestrate instance, modeled on the existing Agentforce and Copilot Studio integrations
- Gemini and Vertex tracing in Python and TypeScript — dedicated tracer for the unified Gemini SDK including Vertex, where previously only the legacy SDK was auto-instrumented
- Claude Agent SDK tracing for TypeScript — drop-in import auto-traces agent runs (the Python equivalent shipped last window)
- Per-project data retention — set a rolling retention window (7/30/90/180/365 days or indefinite); monitoring data older than the window is deleted
- Pause and resume individual tests — pause one monitoring test without disabling the whole project
- Bulk session deletion — multi-select sessions and delete with a full cascade to their traces, scores, and embeddings
- Missing-guardrails governance rule — a new integrity test that flags traces missing an expected guardrail
- US National Provider Identifier as a native PII check — detectable in Contains-PII tests and redactable in traces
- Per-user usage limits and guardrails in the Gateway — limits and guardrails can target an individual user, including per-user caps within a team
- Multimodal Gateway support — the /responses and /messages endpoints support media sent and received, not just text
- Content safety guardrail — a new guardrail type for Gateway requests and responses
- LLM cost explorer (NEW: llm-cost-service) — a free public tool at llm-costs.openlayer.com for browsing and comparing LLM pricing across providers, including per-token-category pricing (cached, reasoning, audio); the same feed that powers Openlayer's cost tracking
- Helm chart for the Gateway — the Gateway ships an official Helm chart for Kubernetes deployments
- Deployment admin area — operators get background-job queue health, metrics, history, and license/feature status, gated by a dedicated deployment-admin role

## Improvements

- Granular token usage and cost — cached, reasoning, and audio token categories are priced and stored, with per-API-key cost visible in the UI; LLM-as-judge evaluator runs are now cost-tracked per test result
- Expanded API-key surface — workspace API keys now work for updating projects, cost-override endpoints, test retrieval/update/deletion, and seven previously session-only read routes
- Update a project's name and description via API — new PATCH endpoint, picked up by the Python, Go, Java, and Ruby SDKs and the MCP server
- Sorting and filtering on more list endpoints — workspace rules, project members, and user sessions; rules tables also gained server-side group filtering and a more concise redesign
- Browser-based CLI login — sign in with a device-code browser flow instead of pasting an API key
- CLI export gains formats and scores — `openlayer export` supports CSV and XLSX (previously JSON only) and can include scores
- XLSX export in development and monitoring modes, all metrics included in downloads, clearer download date formats, and a batch of rules/frameworks/audit-trail quick wins
- Suggested filter values — filter fields suggest likely values as you type, served from the backend and respecting other filters already applied
- Faster row search and listing — row list and full-text search now return in sub-second time
- Drill down from an AI summary's failure modes to the exact rows behind each pattern
- Redesigned test creation page, revamped in-app settings, improved date filtering, and trace navigation search with jump-to-top/bottom controls
- Framework management — sorting, search, and filtering; confirmation before discarding changes; clearer framework documentation and a revised custom-framework creation flow
- Bulk test actions and metric-file management — duplicate several tests at once, schedule runs for several tests at once, and view/download/delete uploaded custom metric files
- LangChain and LangGraph tracing modernized for v1 in Python and TypeScript, with LangGraph metadata mapping and large-trace handling; TypeScript also gained offline trace buffering and first-class session/user columns
- Gateway spend visibility — per-model breakdown on the usage endpoint, a per-user spend aggregation endpoint, projected savings shown when creating a limit, per-key caps within team limits, key rotation, and provider secrets settable from the admin UI
- On-prem deployment options — CSI Secrets Store support, external/managed ClickHouse, a domain and DNS assignment guide, native Oracle Object Storage as a storage backend, the deployed version shown in the UI, and a multi-arch Gateway image for ARM hosts

## Fixes

- Batch uploads reported success while writing zero rows— fixed
- Batch inference uploads from the Python SDK failed asynchronously while the client reported success — fixed
- Test evaluations could be silently skipped or dropped, and stalled monitoring evaluations never recovered — both fixed, with projects now auto-pausing after repeated data source connection failures
- High-volume session evaluations ran the worker out of memory, leaving tests stuck running or queued for hours — fixed
- Large-volume exports timed out or ran out of memory; exports now stream in bounded-memory chunks, preserve custom trace fields, and upload via multipart
- Session-level test scores read as null, went missing, or bled between two tests of the same subtype — fixed
- Session-based LLM-judge rubrics returned false positives — judges whose reasoning concluded a session passed were recorded as failing
- Dialogflow and Agentforce sync reliability — memory-safe batch handling, missing custom parameters, stalled syncs, and cross-agent session scoping all fixed; Agentforce agents now auto-register
- AI summaries crashed or hung on content from certain providers, single-span traces, and rich-text test descriptions — fixed, with loading states corrected
- LangChain traces using Gemini reported zero cost with empty usage details in both the Python and TypeScript SDKs — fixed
- Background trace publishing grew memory without bound under sustained load, and the offline buffer could lose data under concurrency — both fixed in the Python SDK
- CLI fixes — `metrics pull` failed to extract some bundles, `metrics push` was rejected at registration and dropped enum parameter values, `push` gave a generic error when config was incomplete, and dataset-only commits failed server-side
- Gateway errors are returned in the client's native wire format, so tools no longer retry silently against an unreadable error; cost estimation now counts cached tokens and works for Azure AI Foundry; routing savings no longer display as negative
- Activity-feed notifications surfaced projects the viewer had no access to — now scoped to visible projects
- On-prem fixes — deployments on Redis 8 no longer crash on index-not-found, signup is no longer blocked when SMTP is unconfigured, and the Helm chart wires the license key and correct app/worker replica counts