Trace every step of your requests 🕵️‍♂️📡

We’re thrilled to share with you the latest update to Openlayer: comprehensive tracing capabilities and enhanced request streaming with function calling support.

Now, you can trace every step of a request to gain detailed insights in Openlayer. This granular view helps you to debug and optimize performance.

Additionally, we’ve expanded our request streaming capabilities to include support for function calling. This means that requests you stream to Openlayer are no longer a black box, giving you improved control and flexibility.

New features

• Tracing
  • Trace every step of a request and view details, including latency and function inputs & outputs, in the UI
  • Support for function calling in request streaming
• Added support for using Azure OpenAI models

Improvements

• Improved performance of the UI, including several networking optimizations
• Toggle button color improvements to make it easier to understand which is selected
• Improvement to color of background behind modals
• Column A mean / sum etc. grouped by column B values
• Surface generated question for answer relevancy metric
• Easily duplicate/fork test configurations
• Enable creating more tests without dismissing modal
• Improved design of request pane
• Warning displays in request pane when no prompt has been added
• Request pane can be closed with the Esc key
• Navigate through requests in the pane by using the arrow keys
• Improved design of prompt roles in prompt blocks
• Ability to copy values of blocks and columns in request page
• RAG tracing example added to Openlayer examples gallery
• Azure GPT example added to Openlayer examples gallery
• Performance improvement: only automatically load inference pipelines and project versions if the user is in the relevant mode
• Remove Intercom app which was not utilized and was blocking core UI components
• Navigation callout components now have dark-mode purple styling
• Update notification page titles in settings
• Improvements and bug fixes for rendering content and metadata in selected row pane
• Updated copy icon
• Updated inconsistent delete icons throughout the app
• Render inputs in row pane even when no prompt is available
• Render metric scores and explanation columns further left in tables so they are in view without scrolling
• Updated format of date strings
• Enabled ability to collapse sections in row panels
• Enabled ability to collapse chat history blocks

Bug fixes

• In-app Google Colab links were incorrect
• Checkboxes for suggested tests were not default selected
• Graph in test modal rendered too short sometimes
• Prompt roles did not render correctly when set to an unknown value
• Handle cases where data contains non-utf8 codes
• Create test pages overflow before enabling scroll
• Test modal overflows page
• Boolean values would not render in request pane metadata
• Labels in request pane overflowed improperly with long content
• Tests rendered broken graphs when all results were skipped
• Inference pipelines did not automatically load
• Inference pipelines did not automatically update tests or requests
• Commits did not automatically load nor update tests once processed
• Projects did not automatically appear when added from API
• API key and project name were not auto-filling in TypeScript code snippet for starting monitoring
• Clicking to browse a commit always went to monitoring mode
• Monitoring test graphs did not show hovered results on initial load until refreshing
• Opening requests page showed no data until refreshed
• Column drift test wouldn’t run on non-feature columns
• Timeline page showed monitoring tests
• Checkboxes for suggested tests did not check properly on click
• Multiple copies of tests got created on successive clicks
• Unselected tests got created, and not all selected tests got created
• Tests loaded for too long when skipped or unavailable
• Copy button rendered twice in code labels
• Chat history input in row panels sometimes showed text editor