Setting up alerts is an essential first step to monitoring your LLMs, but in order to understand why issues arise in production, it’s helpful to have human eyes to review requests.

This process is now easier than ever in Openlayer - you can add annotations to any requests, with custom values. If you’ve set up tracing, you can annotate each individual step of the trace for more granularity.

Every request can also be rated as a thumbs up or thumbs down, making it easy to scan through good and bad responses and figure out where your model is going wrong.

We’ve released some other huge features and improvements this month, so make sure to read the full changelog below!

New features

• Ability to export data from the UI
  • Now you can download requests data right from the workspace. This is especially helpful if you’ve applied filters and want to download the filtered cohort of data
• Updated navigation
  • Our navigation has a new layout featuring breadcrumbs at the top, making it much easier to navigate between projects and understand the hierarchy
• Annotation and human feedback
  • You can now annotate any request with custom values. You can also give every request a thumbs up or thumbs down to make identifying error patterns even easier.

Improvements

• More project templates
• Improved OpenAI SDK
• Improvements to billing page in settings
• Project-level Git repository settings now available
• Ability to now edit branch and root directory in project-level git settings
• With new navigation, ability to copy project name and inference pipeline ID
• Ability to add ground truths to requests and edit existing ground truths
• Data in individual test modals is now filtered by selected evaluation window

Bug fixes

• Some tests were improperly skipped
• Openlayer Assistant was broken
• Spaces in No PII test caused errors
• Issue with metric tests when input variable names were null
• Hovering over graph with no results shows broken tooltip