Configurations
The configurations available are:Data
Data
The “Data” section allows you to select the data on top of which this test will run.In monitoring mode, it means choosing among your data
sources (e.g., production data, staging data, etc.) In
development mode, it means selecting among your datasets
(e.g., training, validation, etc.)You can also apply filters to narrow the test to specific slices of the data.
Parameters
Parameters
Some tests include configurable parameters that control their behavior.
For example, a test such as the PII detection test
lets you specify which type of PII to detect (e.g., credit card numbers, social security numbers, etc.)Refer to the specific test page in the Catalog for details
on its parameters.
Threshold
Threshold
The “Threshold” section allows you to define the condition for the test to pass.You can set the threshold manually (e.g., saying that the test passes if the
metric is greater than 0.5). However, in monitoring mode,
you can also use automatic thresholds, which is available for most tests.When using automatic thresholds, Openlayer analyzes historical data and
learns a time-series model. Then, for each new observation, Openlayer
predicts an expected range. The test fails if the observed metric falls outside this range.
Advanced settings
Advanced settings
The “Advanced settings” section allows you to set additional configurations and
metadata for the test.Here’s where you can define the criticality of the test, and for monitoring,
the evaluation and delay windows.