We’ve added lots of features and enhancements across our platform, focused on improving performance, expanding functionality, and streamlining workflows. To highlight a few:
🔐 Increased security with SAML SSO directory sync. You can now sync SAML SSO on Openlayer with existing security groups. Openlayer can now more seamlessly fit into your organization’s security policies.
🧑⚖️ New LLM-as-a-judge models. We’ve expanded the models available to act as judges for LLM-as-a-judge tests. You can now use models from Cohere and Vertex AI when running these tests.
🎨 Website refresh. We’ve given our website a brand refresh, including lots of fun animations showcasing the Openlayer product in action and case studies from some of our customers.
Features
•SDKsFaster batch uploads with pyarrow support
•SDKsPush commits to the platform via the Python SDK
•UI/UXTabular view of test results in test modals
•UI/UXAdd pie graph for test results in project home
•EvalsAdd Faithfulness and Answer Correctness metrics for RAG systems
•PlatformUse Cohere, Vertex AI models as options for LLM-as-a-judge metrics
•APIAdd `expand` to inference pipeline GETs so projects and workspaces are included in the response body
•PlatformNew "Viewer" role in workspaces that doesn’t have write, update or delete permissions on resources
•SDKsSupport for async data uploads, and faster upload speeds
•PlatformDirectory sync with SAML
Improvements
•APILower latency for data stream endpoint
•UI/UXUpdate tooltips and rendering of statuses in test cards
•UI/UXMake sections in test modals collapsible
•APIAdd skipped and failing test counts in project version and inference pipeline objects
•APIBetter error messages for invalid data configs when streaming data
•PlatformMore intuitive status messages for skipped tests
•DocumentationAdd code samples in Java
Fixes
•PlatformGenerate outputs step was not failing gracefully
•UI/UXSurface user-facing error messages upon SSO login failures
•UI/UXBetter failure message when password reset link has expired
•PlatformImproved rate limiting
•IntegrationsSlack notifications for create pipeline now includes name
•PlatformAnswer Correctness metric was breaking when output was not a string
$ openlayer push
Stop guessing.
Ship with confidence.
The automated AI evaluation and monitoring platform.
