Overview
- Build scenario-specific evaluation sets before launch, not after incidents.
- Guardrails should cover inputs, retrieved context, and outputs.
- Monitor quality drift and cost per workflow, not just per request.
Evaluation
Evaluation sets are part of the product
Each workflow — NL2SQL, reporting, RAG answering, document extraction — deserves its own labeled cases, scoring rubric, and regression gate that runs on every model or prompt change.
- Include hard negatives: ambiguous questions, missing data, and adversarial phrasing.
- Track accuracy, groundedness, and refusal correctness as separate metrics.
Guardrails
Constrain inputs, context, and outputs
Input policies filter prompts and injected instructions, context policies control what the model may see, and output policies validate structure, claims, and sensitive content before delivery.
- Validate structured outputs against schemas before downstream use.
- Block responses that cite no retrievable evidence in grounded workflows.
Operations
Governance is an operating loop, not a launch checklist
Production teams review evaluation trends, user feedback, audit logs, and cost curves on a fixed cadence, feeding changes back into prompts, retrieval, and model routing.
- Keep audit trails linking answers to context, model version, and policy decisions.
- Rehearse rollback paths for prompts, models, and retrieval indexes.