On this page
Learning objectives
- Define stable input and output contracts around model calls
- Validate outputs before downstream use
- Handle timeouts, refusals, truncation, and schema failures
Prerequisites
Basic HTTP and JSON · Ability to call an API from one programming language
Answer first
Canonical definition
A production LLM call is an API boundary with versioned instructions, typed inputs, a constrained output schema, validation, telemetry, and a fallback path.
Operating context
Why it matters
Free-form text is useful for people but brittle for software. Structured output makes downstream behavior testable.
The surrounding contract—not the prompt alone—determines how safely a model can participate in a workflow.
Method
Step by step
- 01
Define the schema
Describe the minimum fields, types, enums, null behavior, and evidence needed by the next step.
- 02
Constrain the request
Separate system rules, user data, retrieved context, and tool results rather than concatenating an opaque prompt.
- 03
Validate twice
Use provider constraints when available, then validate again in application code before acting.
- 04
Handle non-success
Make refusals, missing fields, timeouts, rate limits, and low-confidence cases explicit states.
Hands-on lab
Create a typed classifier
Classify support requests into a fixed schema with category, urgency, evidence, and escalation reason.
Deliverable
A schema, ten test cases, validation code, retry rule, and fallback queue.
What breaks
Common failure modes
- F1Parsing prose with brittle regular expressions
- F2Retrying invalid outputs indefinitely
- F3Changing model or prompt versions without re-running evals
Beyond the demo
Production notes
- Log schema version, model, latency, token use, validation result, and request ID without storing unnecessary sensitive content.
- Use bounded enums and explicit unknown states instead of forcing a confident classification.
Further reading
Sources
Related Tenten resources