Structured Output: Get Reliable JSON From an LLM

Structured output is an answer in a shape you decided in advance: a JSON object with named keys, a table with fixed columns, or a template with the same headings every time. You need it whenever a program reads the answer, and it helps when a person does too, because every answer then looks the same and is easy to scan or compare.

The trick is to describe the shape so precisely that the model has nothing left to choose. The word "JSON" on its own is not a description. The block below extracts details from a bug report. Compare a request for "the key details" with a request that spells out the schema.

Free textJSON with a schema

EditCopy

Pull out the key details from this bug report: "Since yesterday's update the app crashes when I tap Export on a project with more than 50 photos. Smaller projects export fine. I'm on Android 14, app version 3.2.0. This is blocking my client delivery."

Try it

Example replyReplies vary between models and runs.

Here are the key details from the bug report:

• Issue: The app crashes when exporting projects
• Trigger: Tapping Export on projects with more than 50 photos
• Started: After the most recent update
• Platform: Android 14
• App version: 3.2.0
• Impact: High, since it is blocking a client delivery

Smaller projects appear to export without problems.

The free text answer is accurate and readable, but no two runs will use the same labels, the severity is a sentence rather than a value, and a program would have to guess where each field starts. The JSON answer can go straight into a bug tracker. Notice that the reply still arrived inside a code fence: chat apps usually wrap JSON that way, which matters when you parse it in code.

How to ask for JSON

A good JSON request answers every question the model would otherwise answer for you:

• Every key, spelled exactly. Write the key names in quotes as they must appear. Say "exactly these keys" so the model does not add extra ones.
• The type of every value. String, number, boolean, array of strings, nested object.
• The allowed values for any field with a fixed set, such as severity or category. Without the list, you get "High", "high", "severe" and "P1" across four runs.
• What to do when the input lacks a value. If you do not say "null if not stated", the model tends to fill the gap with a plausible guess, and a guessed app version looks exactly like a real one.
• Nothing around it. "Return only the JSON, with no text before or after it" removes the friendly opening sentence.

When the shape is nested or unusual, showing one complete example object works better than describing it. That is few-shot prompting applied to format. Keep the example's values clearly different from the real input so the model does not copy them.

A reusable extraction prompt

This block has the same request split into parts. Switch off the format part and the model still returns JSON, because the constraints ask for it, but it picks its own key names, such as title instead of job_title, and code that expects your keys breaks. The constraints part is what stops it from completing a half-written phone number or inferring a company from an email domain. Paste a real signature into the input field to test it.

Extract contact details as JSON

EditCopy

Fill insignature

PartsTaskFormatConstraintsInput

Task

Extract the contact details from the email signature below.

Format

Return a JSON object with exactly these keys: { "name": "string", "job_title": "string or null", "company": "string or null", "email": "string or null", "phone": "string or null" }

Constraints

Use null for any value that is not in the text. Do not guess or complete partial values. Return only the JSON.

Input

Priya Nair | Head of Data, Northwind Labs | priya.nair@northwind.example

Try it

Example replyReplies vary between models and runs.

{
  "name": "Priya Nair",
  "job_title": "Head of Data",
  "company": "Northwind Labs",
  "email": "priya.nair@northwind.example",
  "phone": null
}

Tables and fixed templates

Structured output is not only for programs. When you read the answer yourself, a markdown table or a fixed template gives you the same benefit: you know where each piece of information will be before you look.

Table

EditCopy

Compare Python lists, tuples and sets in a markdown table with these columns: Type, Ordered, Mutable, Allows duplicates, Typical use. One row per type. No text outside the table.

Try it

Example replyReplies vary between models and runs.

Type	Ordered	Mutable	Allows duplicates	Typical use
list	Yes	Yes	Yes	A sequence you add to, remove from or sort
tuple	Yes	No	Yes	A fixed group of values, such as coordinates
set	No	Yes	No	Removing duplicates and fast membership checks

A template works the same way for longer text: give the headings in order and say what goes under each. "Answer with three bold labels: Cause, Fix, How to check" produces the same three labels every time, which makes a batch of answers easy to compare.

JSON mode in the API

Several model APIs have a setting that forces syntactically valid JSON. In the OpenAI Python SDK it is response_format. JSON mode requires the word "JSON" to appear somewhere in your messages, so the system prompt below names it and lists the keys.

import json
from openai import OpenAI

client = OpenAI()
MODEL = "your-model-id"  # e.g. from your provider's model list

report_text = "Since yesterday's update the app crashes when I tap Export..."

response = client.chat.completions.create(
    model=MODEL,
    response_format={"type": "json_object"},
    messages=[
        {
            "role": "system",
            "content": (
                "Extract the bug report into JSON with the keys "
                "summary (string), severity (one of low, medium, high, critical) "
                "and steps_to_reproduce (array of strings)."
            ),
        },
        {"role": "user", "content": report_text},
    ],
)

data = json.loads(response.choices[0].message.content)

JSON mode makes sure the text parses (unless the reply is cut off at the token limit), not that it matches your schema. A key can still be missing or a severity can still be "urgent". Several providers also accept a full JSON Schema, as an output format or as the input schema of a tool (function) definition, and some of these modes constrain the answer to the schema. Check your provider's documentation for the exact parameter, since these features differ between APIs.

Validate the result in code

Treat the model's JSON the way you treat any input from outside your program: parse it, then check it. Parsing catches broken syntax. A check catches a valid object with the wrong content.

ALLOWED_SEVERITIES = {"low", "medium", "high", "critical"}

def problems(data):
    if not isinstance(data, dict):
        return ["the answer must be a JSON object"]
    found = []
    if not isinstance(data.get("summary"), str):
        found.append("summary must be a string")
    if data.get("severity") not in ALLOWED_SEVERITIES:
        found.append("severity must be low, medium, high or critical")
    steps = data.get("steps_to_reproduce")
    if not isinstance(steps, list) or not all(isinstance(s, str) for s in steps):
        found.append("steps_to_reproduce must be an array of strings")
    return found

When the check fails, a single retry often fixes it: send the model its own output together with the list of problems and ask for corrected JSON. Cap the number of retries, and log the failures, because a field that fails often is a sign the prompt is unclear. In larger projects a validation library such as Pydantic or a JSON Schema validator replaces the hand-written function.

Two more habits prevent quiet errors. Set the output token limit high enough for the largest answer you expect, because a truncated object never parses. And when the input is long or comes from users, mark it off from your instructions with delimiters or XML tags so text inside it is less likely to be read as an instruction. If one prompt has to both reason and produce JSON, consider prompt chaining: let one step think in free text and a second step turn the result into the structure.