Cookbook: RAG Evaluation

criteria.rag plus Knowledge Bases and sources toggles.

Cookbook 2 — RAG Evaluation

Upload policy docs on Studio /knowledge (PII redacted before persist).
Attach the KB to a rubric (knowledge_base_id) or pass kb_id / sources.documents=true.
Score with criteria.rag faithfulness / citation criteria.

result = client.evaluate(
    messages=[...],
    criteria=["Penalize responses that invent policy details not present in retrieved context."],
    kb_id="kb_...",
    sources={"documents": True, "annotations": True, "memory": False, "calibration_tau": True},
)

Cookbook: Response Quality & CI Gate

Assert assurance passes in CI with Pytest or the TypeScript SDK.

Cookbook: Agent Trajectory Evaluation

AgentTracer ContextVar spans + post-hoc criteria.agent (one evaluate, zero extra LLM localization).

On this page

Cookbook 2 — RAG Evaluation