# AI Glossary

Definitions of AI evaluation, observability, and governance terms from the Openlayer team.

- [AI compliance certification](/glossary/ai-compliance-certification.md): Explore what AI compliance certification means, the leading frameworks available, and how organizations can build AI systems that meet regulatory and ethical standards.

AI compliance certification is the process of validating that an AI system meets specific legal, regulatory, or ethical standards. As global attention on responsible AI grows, certifications provide proof that systems are safe, fair, and accountable.

- [AI governance](/glossary/ai-governance.md): AI governance defines how organizations build and manage AI responsibly, aligning systems with ethical principles, security guardrails, and global frameworks like the EU AI Act, NIST RMF, and ISO 42001.

- [AI model testing](/glossary/ai-model-testing.md): Learn how to test AI models for accuracy, bias, drift, and reliability. Explore best practices for both ML and generative AI systems.

AI model testing is the process of evaluating how well an AI system performs across tasks, scenarios, and data conditions. It includes structured test cases, real-world simulations, and ongoing validation throughout the development and deployment lifecycle.

- [AI quality assurance](/glossary/ai-quality-assurance.md): Learn what AI quality assurance is and how it helps test, monitor, and validate AI systems. Explore key practices across ML and GenAI.

AI quality assurance (AI QA) is the discipline of validating that AI systems perform reliably, ethically, and safely across development and production environments. It involves testing models, monitoring outputs, and identifying failures before they impact users.

- [Data quality monitoring dashboard](/glossary/data-quality-monitoring-dashboard.md): Learn what a data quality monitoring dashboard is, why it matters in machine learning, and what features it should include to catch data drift, nulls, and anomalies.

A data quality monitoring dashboard provides a visual interface for tracking the health of your datasets over time. It helps data teams identify data drift, schema changes, missing values, anomalies, and other quality issues that can degrade AI and ML models.

- [Data quality monitoring framework](/glossary/data-quality-monitoring-framework.md): Learn what a data quality monitoring framework is and how to implement one to track schema drift, anomalies, and data validation in machine learning systems.

A data quality monitoring framework is a structured system that continuously checks the integrity, consistency, and fitness of data used in machine learning pipelines. It helps detect issues like schema changes, missing values, drift, and outliers before they affect model performance.

- [EU AI Act compliance](/glossary/eu-ai-act-compliance.md): Understand the core requirements of the EU AI Act and how to build AI systems that comply with its transparency, safety, and risk management standards.

EU AI Act compliance refers to the set of practices and safeguards AI developers and organizations must implement to align with the European Union’s Artificial Intelligence Act. This legislation aims to ensure that AI systems used within the EU are safe, transparent, and uphold fundamental rights.

- [Generative AI testing tools](/glossary/generative-ai-testing-tools.md): Explore top testing tools for generative AI, including methods to catch hallucinations, assess prompt quality, and monitor LLM reliability.

Generative AI testing tools help teams evaluate the behavior, performance, and safety of large language models (LLMs) and other generative systems. These tools are essential for identifying edge cases, hallucinations, and prompt failures in applications like chatbots, content generation, and copilots.

- [How to evaluate LLMs](/glossary/how-to-evaluate-llms.md): Discover the most effective methods for evaluating LLMs, including LLM-as-a-judge, human review, and prompt-based testing. Learn how to track quality, reliability, and safety.

- [LLM benchmarks](/glossary/llm-benchmarks.md): Discover standard benchmarks used to evaluate large language models (LLMs). Includes HELM, MMLU, TruthfulQA, and more.

LLM benchmarks are standardized tests and datasets used to evaluate the performance of large language models. These benchmarks provide a way to compare models across tasks like reasoning, question answering, coding, and ethics.

- [LLM evaluation metrics](/glossary/llm-evaluation-metrics.md): Discover common metrics used to evaluate LLMs, including rubric-based scoring, LLM-as-a-judge, and quality benchmarks for text generation.

LLM evaluation metrics are methods for quantifying the quality, safety, and utility of outputs generated by large language models. These metrics help teams evaluate models across tasks like summarization, Q&A, reasoning, and multi-turn interaction.

- [LLM guardrails](/glossary/llm-guardrails.md): Discover what LLM guardrails are and how they help control behavior, enforce structure, and prevent unsafe outputs in generative AI systems.

LLM guardrails are mechanisms used to constrain, validate, or intervene in the outputs of large language models (LLMs). They help ensure that LLMs behave safely, stay on topic, respect user boundaries, and comply with ethical or regulatory standards.

- [LLM test](/glossary/llm-test.md): Learn how to create and run LLM tests to evaluate safety, accuracy, and consistency. Includes prompt testing, rubric scoring, and output analysis.

An LLM test is a structured evaluation designed to measure the behavior, accuracy, or robustness of a large language model (LLM). These tests help ensure that the model performs reliably across tasks, use cases, and prompt structures.

- [LLM visualization](/glossary/llm-visualization.md): Discover how LLM visualization tools help explain prompt flows, output reasoning, and multi-step interactions. Ideal for debugging and analysis.

LLM visualization refers to tools and techniques used to interpret, trace, or debug the behavior of large language models (LLMs). As LLM applications grow more complex—especially with agents, tool use, and chaining—visualization helps teams understand how prompts are processed and outputs are generated.

- [ML evaluation metrics](/glossary/ml-evaluation-metrics.md): Learn the most common machine learning evaluation metrics including accuracy, precision, recall, F1, MSE, and AUC. Choose the right metric for your model type and task.

ML evaluation metrics help determine how well a machine learning model performs on a given task. Choosing the right metric is critical for building reliable, fair, and effective models.

- [Model drift vs. data drift](/glossary/model-drift-vs-data-drift.md): Learn the difference between model drift and data drift in machine learning. Understand how each affects model performance and how to detect and address them.

Understanding the difference between model drift and data drift is essential for maintaining machine learning model performance over time.

- [Prompt evaluation](/glossary/prompt-evaluation.md): Learn how to evaluate LLM prompts using scoring frameworks, LLM-as-a-judge methods, hallucination detection, and automated testing to improve reliability before production deployment.

Prompt evaluation is the process of assessing the effectiveness of prompts used to query large language models (LLMs). As prompt engineering becomes a critical component of GenAI development, understanding how to evaluate prompt quality is essential for improving LLM outputs.