Row-by-row observability over your production requests 🔍

Introducing support for multi-turn interactions. You can now log and refer back to the full chat history of each of your production requests in Openlayer. Sort by timestamp, token usage, or latency to dig deeper into your AI’s usage. And view graphs of these metrics over time.

There’s more: we now support Google’s new Gemini model. Try out the new model and compare its performance against others.

⬇️ Read the full changelog below for all the tweaks and improvements we’ve shipped over the last few weeks and, as always, stay closer to our development journey by joining our Discord!

New features

• Log multi-turn interactions in monitoring mode, and inspect individual production requests to view the full chat history alongside other meta like token usage and latency
• Sort and filter through your production requests
• View a graph of the token usage and latency across all your requests over time
• Support for Gemini is now available in-platform: experiment with Google’s new model and see how it performs on your tests
• View row-by-row explanations for tests using GPT evaluation

Improvements

• Expanded the Openlayer TypeScript/JavaScript library to support all methods of logging requests, including those using other providers or workflows than OpenAI
• Improved commit selector shows the message and date published for each commit
• New notifications for uploading reference datasets and data limits exceeded in monitoring mode
• Only send email notifications when test statuses have changed from the previous evaluation in monitoring
• Added sample projects for monitoring
• Enhancements to the onboarding, including a way to quickstart a monitoring project by sending a sample request through the UI
• No longer navigate away from the current page when toggling between development and monitoring, unless the mode does not apply to the page
• Allow reading and setting project descriptions from the UI
• Update style of selected state for project mode toggles in the navigation pane for clarity
• Clarify that thresholds involving percentages currently require inputting floats
• Allow computing PPS tests for columns other than the features
• Test results automatically update without having to refresh the page in monitoring mode
• Add dates of last/next evaluation to monitoring projects and a loading indication when they recompute
• Surface error messages when tests fail to compute
• Add callouts for setting up notifications and viewing current usage against plan limits in the navigation
• Graphs with only a single data point have a clearer representation now
• Improvements to the experience of creating tests with lots of parameters/configuration
• Improvements to the experience of creating Great Expectations tests
• Add alert when using Openlayer on mobile
• Default request volume, token usage, and latency graphs to monthly view

Bug fixes

• Title suggestions for certain tests during creation were unavailable or inaccurate
• Fixes to test parameters, including incorrectly labeled and invalid options
• Certain LLM tests would not allow selecting target columns that are not input variables
• Code in development onboarding modals was not syntax highlighted
• Create test card content would overflow improperly
• Sample projects would not show button for creating suggested tests after some were created
• Graphs in monitoring test cards were cut off
• Requests table would break when rows were missing columns
• Full-screen onboarding pages would not allow scrolling when overflowed
• Options were sometimes duplicated in heatmap dropdowns
• Thresholds would not faithfully appear in test result graphs
• Skipped evaluations would not appear in test result graphs