Criteria Writing Guide

Criteria are the instructions QAVE scores against. Prefer one focused quality with clear direction.

Phrasing styles

Principles

• One quality per criterion — split unrelated checks.
• Clear direction — start with Reward / Penalize / Passes if / Fails if.
• Monotonic or qualified — use an if … qualifier to scope applicability.
• Avoid conjunctions — and / or usually smuggle a second quality.
• Avoid role-keyword collisions — do not embed user: / assistant: role tokens inside criteria text.

Template

[Prefix] [quality] [qualifier]

Example: Reward responses that cite the exact policy section if the user asks about cancellations.

Good examples

• Reward responses that state the refund window in days.
• Penalize responses that invent policy details not present in the input.
• Passes if the response refuses to give medical dosage advice.

See also the Criteria Library for packaged presets.