OpenAI REST API: Structured Output

The original method was to simply ask the model in your prompt to return JSON.

Extract the movie title, year, and director from the following text. Return the output as a JSON object with keys "title", "year", and "director".

Text: "My favorite movie is The Matrix, which came out in 1999 and was directed by the Wachowskis."
            

Why it's brittle? The model might not perfectly follow instructions, leading to common parsing failures. For example, it could add extraneous text, use different key names than requested, or generate malformed JSON.

A more recent and reliable feature is JSON Mode. By setting the response_format parameter to { "type": "json_object" } in your API call, you instruct the model to guarantee that its output is a syntactically valid JSON object.

Requirements:

• You must use a newer model like gpt-4-turbo-preview or gpt-3.5-turbo-1106.
• Your prompt must contain the word "JSON" to remind the model of the constraint.

{
  "model": "gpt-4-turbo-preview",
  "response_format": { "type": "json_object" },
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful assistant designed to output JSON."
    },
    {
      "role": "user",
      "content": "Who won the world series in 2020?"
    }
  ]
}
            

This is a huge improvement, as it eliminates parsing errors. However, it doesn't guarantee the schema of the JSON (e.g., that a "year" field will be present).

Function Calling is the most powerful and recommended method for getting structured output. It was designed to let the model call external tools, but its real superpower is forcing the model's output into a specific, predefined JSON schema.

Here's how it works:

1. Define a Schema: In your API request, you provide a JSON schema describing the exact structure you want. You define the function's name, its purpose, and the parameters it accepts (your desired fields), including their types (string, integer, array) and descriptions.
2. Instruct the Model: You tell the model to call this function with data extracted from the user's prompt.
3. Get Perfect JSON: The API returns a response containing a JSON object that perfectly matches the schema you defined.

-- POST https://api.openai.com/v1/chat/completions
-- Content-Type: application/json
-- Authorization: Bearer $OPENAI_API_KEY

{
  "model": "gpt-4o-mini",
  "messages": [
    {
      "role": "system",
      "content": "You are a JSON generator. Given a movie name, produce a single JSON object with a specific schema."
    },
    {
      "role": "user",
      "content": "Get movie details for: The Matrix\nRespond by calling the function 'create_movie_details'."
    }
  ],
  "functions": [
    {
      "name": "create_movie_details",
      "description": "Create a movie details object.",
      "parameters": {
        "type": "object",
        "properties": {
          "title": { "type": "string" },
          "year": { "type": "integer" },
          "director": { "type": "string" },
          "description": { "type": "string" },
          "genres": { "type": "array", "items": { "type": "string" } },
          "actors": { "type": "array", "items": { "type": "string" } }
        },
        "required": ["title", "year", "director", "description", "genres", "actors"]
      }
    }
  ],
  "function_call": "auto",
  "temperature": 0.0
}
            

The API's response won't contain a typical text message. Instead, it will contain a tool_calls object. Inside, you'll find the name of the function the model chose to call (create_movie_details) and, most importantly, an arguments field. This field contains a JSON string with the extracted data, perfectly matching your schema.

{
  "id": "chatcmpl-CD3GM3Gc1Qfr40FKxOTf8DZG7afGn",
  "object": "chat.completion",
  "created": 1757226842,
  "model": "gpt-4o-mini-2024-07-18",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "function_call": {
          "name": "create_movie_details",
          "arguments": "{\"title\":\"The Matrix\",\"year\":1999,\"director\":\"Lana Wachowski, Lilly Wachowski\",\"description\":\"A computer hacker learns from mysterious rebels about the true nature of his reality and his role in the war against its controllers.\",\"genres\":[\"Action\",\"Sci-Fi\",\"Adventure\"],\"actors\":[\"Keanu Reeves\",\"Laurence Fishburne\",\"Carrie-Anne Moss\",\"Hugo Weaving\",\"Gloria Foster\"]}"
        },
        "refusal": null,
        "annotations": []
      },
      "logprobs": null,
      "finish_reason": "function_call"
    }
  ],
  "usage": {
    "prompt_tokens": 104,
    "completion_tokens": 96,
    "total_tokens": 200,
    "prompt_tokens_details": {
      "cached_tokens": 0,
      "audio_tokens": 0
    },
    "completion_tokens_details": {
      "reasoning_tokens": 0,
      "audio_tokens": 0,
      "accepted_prediction_tokens": 0,
      "rejected_prediction_tokens": 0
    }
  },
  "service_tier": "default",
  "system_fingerprint": "fp_8bda4d3a2c"
}}
            

A Deeper Look at Constrained Decoding

The magic behind the reliability of Function Calling and JSON Mode is a process called constrained decoding. Unlike traditional text generation where the model predicts the next most likely word from its entire vocabulary, constrained decoding restricts the model's choices at each step.

When you provide a JSON schema, the API uses it as a blueprint. As it generates the output, it only allows the model to select tokens that are valid for the current position in the JSON structure. For example, if it's generating a key, it will only allow string tokens. If it expects a number, it will restrict the output to digits. This programmatic enforcement is what guarantees syntactically correct JSON that conforms to your schema, moving from "please give me JSON" to "you must give me JSON."

Limitations and Known Issues

While powerful, the feature isn't foolproof. Here are a few things to keep in mind:

• Performance: The first request with a new or complex schema can take a few seconds to process due to caching. Subsequent calls with the same schema are much faster.
• Interruption: The model may still violate the schema if the generation is interrupted by a stop condition (like reaching max_tokens) before the JSON is complete.
• Complexity: Overly complex schemas can sometimes confuse the model, leading to incorrect or incomplete output.
• Glitches: Users have reported occasional glitches, like the model returning a JSON string with excess newline characters (e.g., \n\n\n).

Beyond OpenAI: A Broader Perspective

The concept of enforcing a schema on AI output is becoming an industry standard. Other platforms like Azure OpenAI offer similar structured output capabilities, supporting a JSON Schema subset and enforcing strict rules like requiring defined fields (`required`) and prohibiting extra ones (`additionalProperties: false`). This provides a comparable level of reliability for developers on different platforms.

This is the technique used in the demo below. We define a MovieDetails schema and ask the model to populate it. The result is a clean, predictable object that we can directly bind to our UI without any guesswork. It's the most robust way to build AI-powered features into your applications.