⏳ More ways to test latency

We’ve added more ways to test latency. Beyond just mean, max, and total, you can now make test latency with minimum, median, 90th percentile, and 99th percentile metrics. Just head over to the Performance page and the new test types are there.

You can also create more granular data tests by applying subpopulation filters to run the tests on specific clusters of your data. Just add filters in the Data Integrity or Data Consistency pages, and the subpopulation will be applied.

New features

• Ability to apply subpopulation filters to data tests
• More latency tests
  • Min Latency
  • Median Latency
  • 90th Percentile Latency
  • 95th Percentile Latency
  • 99th Percentile Latency
• Support for logging and testing runs of the OpenAI Assistants API with our Python and TypeScript clients: See how it works

Improvements

• Updated OpenAI model pricing
• Support for OpenAI assistants with example notebook
• Improved performance for monitoring projects
• Requests are updated every 5 seconds live on the page
• Ability to search projects by name in the project overview
• You can now view rows per evaluation window in test modals
• Date picker for selecting data range in test modal
• Show only the failing rows for tests
• Allow opening rows to the side in test modal tables
• Enable collapsing the metadata pane in test modals
• Skipped test results now render the value from the last successful evaluation in monitoring

Bug fixes

• Langchain version bug is fixed
• Metric score and explanations did not appear in data tables in development mode
• Request table layout was broken
• Now able to navigate to subsequent pages in requests page
• Fixed bug with opening request metadata
• Requests and inference pipeline occasionally did not load
• Some LLM metrics had null scores in development mode
• There was a redundant navigation tab bar in monitoring test modals
• Monitoring tests with no results loaded infinitely