November 17, 2023

Enhanced onboarding, redesigned navigation, and new goals

We’re thrilled to announce a new and improved onboarding flow, designed to make your start with us even smoother. We’ve also completely redesigned the app navigation, making it more intuitive than ever.

You can now use several new consistency and integrity goals — fine-grained feature & label drift, dataset size-ratios, new category checks and more. These are described in more detail below.

You’ll also notice a range of improvements — new Slack and email notifications for monitoring projects, enhanced dark mode colors and improved transactional email deliverability. We’ve reorganized several features for ease of use, including the subpopulation filter flow and the performance goal page layout.

If you’re working in dev mode, check out the dedicated commit page where you can view all the commit’s metadata and download your models and data to use locally.

Stay tuned for more updates and join our Discord community to be a part of our ongoing development journey. 🚀👥

Features

New and improved onboarding for monitoring mode
Redesigned app navigation
New goals
- Column drift (consistency) — choose specific columns and specific test types to measure drift in production
- Column values match (consistency) — specify a cohort that must have matching values for a set of features in both production and reference data
- New categories (consistency) — check for new categories present for features in your production data
- Size-ratio (consistency) — specify a required size ratio between your datasets
- Character length (integrity) — enforce character limits on your text-based columns
- Ill-formed rows for LLMs (integrity) — check that your input and output columns don’t contain ill-formed text
Dedicated commit page to view all commit metadata and download artifacts

Improvements

Updated Slack, email notifications in monitoring mode
Color improvements for dark mode
Text no longer resets when toggling between block types in prompt playground
Text highlight color is now standard blue for browsers
Better transactional email deliverability
Navigate to notification settings directly from the notifications modal
Improved readability of prompt block content
Volume graphs in monitoring mode are more real-time
You may now invite team members in the workspace dropdown
Reorganized subpopulation filter flow
Reorganized create performance goal page layout
Improved multi-select for subpopulation filters
Requesting an upgrade in-app now opens a new tab
You can now specify arbitrary column names in goal thresholds and subpopulation filters

Fixes

Back navigation didn’t maintain project mode
Residual plots were missing cohorts in performance diagnosis page
Null metric values would cause all metrics to appear empty
Sample projects missing “sample” tag in projects page
Icon for comment in the activity log was incorrect
Metrics table was busted when missing subpopulation information
Performance and diagnostics page would freeze when using 1000s of classes
Aggregate metrics would sometimes get cut off
Filtering project page by LLMs or tabular-regression would not work
App links returned by client API now navigate to the correct project mode
Auto-conversion of input variables with spaces to underscores for inference

Enhanced onboarding, redesigned navigation, and new goals

Stop guessing. Ship with confidence.

The automated AI evaluation and monitoring platform.