Log question for RAG systems

Retrieval-Augmented Generation (RAG) systems typically receive a user query that drives both retrieval and generation. Openlayer uses the question to compute metrics such as context relevancy (how well the retrieved context matches the query), answer relevancy (how well the answer addresses the query), and more. In most cases, you don’t need to do anything — Openlayer automatically infers the question from the first argument of the outermost traced function. Explicit logging is only necessary when:

the first argument is not the question (e.g. it’s a config object or a dict), or
the question is constructed or transformed inside the pipeline before being used.

How to log the question

There are two main ways to provide the question to Openlayer:

The question must be a string (str) containing the user query.

1. Use `@trace` with `question_kwarg`

If your function receives the user query as a keyword argument, you can tell Openlayer which argument contains it.

from openlayer.lib import trace

@trace(question_kwarg="query")
def generate_answer(query: str, context: list[str]) -> str:
    return llm_call(query, context)

2. Call `log_question` directly

If you prefer not to rely on a kwarg, or if the question is derived at a different point in the pipeline, you can log it explicitly:

from openlayer.lib import trace
from openlayer.lib.tracing import log_question

@trace()
def generate_answer(raw_input: dict) -> str:
    query = raw_input["text"]
    # Log the user query manually
    log_question(query)

    context = retrieve_context(query)
    return llm_call(query, context)

Get started

Workspace setup

Governance

Observability

Offline testing

Tests

Gateway

Data quality monitoring

Administration

Notifications

Other resources

Log question for RAG systems

How to log the question

1. Use `@trace` with `question_kwarg`

2. Call `log_question` directly

​How to log the question

​1. Use @trace with question_kwarg

​2. Call log_question directly

How to log the question

1. Use `@trace` with `question_kwarg`

2. Call `log_question` directly