# LLM test

Learn how to create and run LLM tests to evaluate safety, accuracy, and consistency. Includes prompt testing, rubric scoring, and output analysis.

An LLM test is a structured evaluation designed to measure the behavior, accuracy, or robustness of a large language model (LLM). These tests help ensure that the model performs reliably across tasks, use cases, and prompt structures.

## What is an LLM test? {#what-is-an-llm-test}

An LLM test is typically a repeatable prompt (or prompt set) used to:

- Measure output quality
- Catch regressions
- Test for specific risks (e.g., hallucinations, formatting, refusals)
- Validate model alignment with task instructions or user expectations

LLM tests can be standalone or part of a broader GenAI testing framework.

## Why LLM testing matters {#why-llm-testing-matters}

Without structured testing, LLMs may:

- Produce unpredictable outputs across prompts or contexts
- Fail to meet quality or tone requirements
- Regress in performance during updates

Running systematic tests helps ensure LLMs are production-ready.

## Types of LLM tests {#types-of-llm-tests}

### 1. **Prompt-based tests**

- Fixed input prompts used to evaluate consistency, accuracy, and safety
- Track how output changes over time or across versions

### 2. **Scenario-based tests**

- Multi-turn or contextual tests simulating user interactions
- Helps evaluate agent behavior, memory, or chain-of-thought reasoning

### 3. **Rubric-driven evaluations**

- Tests scored using defined criteria (e.g., clarity, helpfulness, tone)
- Can be scored manually or with LLM-as-a-Judge

### 4. **Behavioral and stress tests**

- Evaluate how the model handles edge cases, adversarial inputs, or conflicting instructions

### 5. **Regression tests**

- Compare new outputs against baselines from previous versions

## Related {#related}

- [GenAI testing](https://www.openlayer.com/products/genai-testing)
- [LLM evaluation metrics](https://openlayer.com/glossary/llm-evaluation-metrics)
- [Prompt evaluation](https://openlayer.com/glossary/prompt-evaluation)
- [LLM monitoring](https://www.openlayer.com/products/llm-monitoring)

*Designing a robust LLM test suite is key to continuous evaluation and quality assurance in generative AI applications.*