A 200 Response Is Not a Complete Response: Test Truncated Structured Outputs

There Are Three Different Successes

A structured-output call crosses three independent boundaries. Collapsing them into one boolean is the bug.

1. Transport success: the server accepted the HTTP request and returned a response object.
2. Generation success: the response reached the terminal state your application accepts, normally completed.
3. Application success: the completed content has the expected type, passes domain validation, and is safe to use in the current state.

The OpenAI Responses API reference lists completed, failed, in_progress, cancelled, queued, and incomplete as response statuses. It also defines max_output_tokens as an upper bound that includes visible output and reasoning tokens. That makes an HTTP-successful response object an envelope, not a promise that generation reached your requested end state. See the current Responses create reference.

Structured Outputs solves a different problem. With a JSON Schema format, the model is constrained to the supplied shape. OpenAI's Structured Outputs guide still shows an explicit check for response.status === "incomplete" and response.incomplete_details.reason === "max_output_tokens" before content is consumed. Schema adherence does not erase the response lifecycle.

Put One Adapter Between the SDK and Your Domain

Do not spread status checks across every controller. Give the provider response one narrow exit into trusted application code. The adapter below accepts a provider-shaped object, refuses every non-completed state, distinguishes a refusal from output text, and calls the domain parser after those checks pass.

class IncompleteModelOutput extends Error {
  constructor(readonly reason: string) {
    super(`Model output incomplete: ${reason}`);
  }
}

type ResponseLike = {
  status: string;
  incomplete_details?: { reason?: string } | null;
  output: Array<{
    type: string;
    content?: Array<{ type: string; text?: string; refusal?: string }>;
  }>;
};

export function requireCompletedJson<T>(
  response: ResponseLike,
  parseDomainValue: (value: unknown) => T
): T {
  if (response.status === "incomplete") {
    throw new IncompleteModelOutput(
      response.incomplete_details?.reason ?? "unknown"
    );
  }
  if (response.status !== "completed") {
    throw new Error(`Model response not completed: ${response.status}`);
  }

  const message = response.output.find((item) => item.type === "message");
  const content = message?.content?.[0];

  if (content?.type === "refusal") {
    throw new Error(`Model refused: ${content.refusal ?? "no reason"}`);
  }
  if (content?.type !== "output_text" || typeof content.text !== "string") {
    throw new Error("Completed response had no output text");
  }

  return parseDomainValue(JSON.parse(content.text));
}

The order is intentional. Status is an envelope property, so it is checked first. Content type comes second because a refusal is not your requested JSON payload. Syntax comes third. Domain validation comes last and should reject a well-formed but impossible value: an unknown account, an unsupported state transition, a negative quantity, or a resource outside the authenticated tenant.

This adapter also prevents a common convenience bug: reading output_text and trusting it because a helper returned a string. Convenience accessors are useful after the state gate, not instead of it. If you later change SDK versions, add a fixture for every response state your adapter handles and compare it with the current retrieve-response schema.

Force the Failure in a Contract Test

A mocked object proves your branch logic. It does not prove that your request can actually produce the provider state you expect. Keep both levels: a fast unit test with a hand-built response, plus a quarantined contract test that calls the API with a deliberately restrictive output budget.

The contract test should request something much larger than the budget. For example, require an array of 25 objects with several string fields, then set max_output_tokens to a small value. The exact token count at which a model stops is not the contract, so do not assert a count or a text prefix. Assert the documented state and reason.

import OpenAI from "openai";
import { describe, expect, it } from "vitest";

const openai = new OpenAI();

describe.runIf(process.env.OPENAI_API_KEY)("Responses completion contract", () => {
  it("reports output-budget exhaustion before the result is consumed", async () => {
    const response = await openai.responses.create({
      model: process.env.OPENAI_TEST_MODEL ?? "gpt-5.6",
      input: "Return exactly 25 detailed migration steps.",
      max_output_tokens: 32,
      text: {
        format: {
          type: "json_schema",
          name: "migration_plan",
          strict: true,
          schema: {
            type: "object",
            properties: {
              steps: {
                type: "array",
                minItems: 25,
                maxItems: 25,
                items: {
                  type: "object",
                  properties: {
                    name: { type: "string" },
                    verification: { type: "string" }
                  },
                  required: ["name", "verification"],
                  additionalProperties: false
                }
              }
            },
            required: ["steps"],
            additionalProperties: false
          }
        }
      }
    });

    expect(response.status).toBe("incomplete");
    expect(response.incomplete_details?.reason).toBe("max_output_tokens");
  });
});

Run that test on demand or on a scheduled provider-contract job, not in every unit-test loop. It consumes API capacity, depends on a live service, and may need its model identifier updated. The OpenAI API changelog is the primary place to review relevant platform changes before adjusting the fixture. Keep the unit test deterministic so every pull request still protects the boundary.

The Test Must Also Prove Zero Side Effects

The response-state assertion is necessary but incomplete. The production failure is not “we logged the wrong status.” It is “partial model output reached an effect.” Put parsing and persistence behind one orchestration function, inject the effect, and assert that the effect was called zero times when the envelope is incomplete.

async function buildAndSavePlan(
  response: ResponseLike,
  savePlan: (plan: MigrationPlan) => Promise<void>
) {
  const plan = requireCompletedJson(response, MigrationPlan.parse);
  await savePlan(plan);
}

it("does not save partial structured output", async () => {
  const savePlan = vi.fn();
  const truncated: ResponseLike = {
    status: "incomplete",
    incomplete_details: { reason: "max_output_tokens" },
    output: [{
      type: "message",
      content: [{ type: "output_text", text: '{"steps":[]}' }]
    }]
  };

  await expect(buildAndSavePlan(truncated, savePlan))
    .rejects.toThrow("Model output incomplete: max_output_tokens");
  expect(savePlan).not.toHaveBeenCalled();
});

The fixture deliberately contains parseable JSON. That removes syntax as an accidental guard and proves the status check is doing the work. Add sibling fixtures for failed, cancelled, a refusal content item, missing output text, invalid JSON, and a completed value that fails the domain schema. This is a compact application of the broader testing workflow in Testing with AI: name the external contract, isolate it behind a seam, and make the failure observable.

If saving the plan triggers more consequential work, preserve the same boundary all the way down. Queue publication after successful validation. Generate an idempotency key after completion. If a retry can repeat an external mutation, use the separate patterns in Idempotent Tool Calls for AI Agents. Rejecting incomplete output prevents a bad first attempt; idempotency protects a valid attempt delivered twice.

Output Exhaustion Is Not Input Truncation

Two similarly named controls operate on opposite ends of the request. Mixing them up produces a misleading test and the wrong remediation.

• max_output_tokens limits generation. The reference defines it as an upper bound on generated tokens, including visible and reasoning tokens. When that budget is exhausted, inspect response status and incomplete details.
• truncation handles oversized input context. The create reference says auto can drop items from the beginning of the conversation to fit the context window, while disabled makes an oversized input fail instead. This option does not certify output completion.

Test them separately. The output-exhaustion test uses a small output budget and asserts an incomplete response. The input-overflow test supplies a context that exceeds the chosen model's limit and asserts the configured input policy: rejection when truncation is disabled, or explicitly accepted loss of old conversation items when automatic truncation is enabled. Do not make one giant live test cover both; when it fails, you will not know which contract moved.

Decide the Recovery Policy Before Production

Once you reject incomplete output, you need a bounded next step. Choose one per use case and make it visible in metrics.

• Retry with a larger budget when the requested object is legitimately larger than your first limit. Cap attempts, record the reason, and keep the same side-effect gate.
• Ask for a smaller object when unbounded arrays or prose fields make the contract unpredictable. Pagination or one section per request is often easier to test than one enormous schema.
• Return a typed application error when latency or cost makes retry unacceptable. The caller can show “generation did not complete” without exposing provider-shaped partial data.
• Escalate unknown reasons instead of treating every incomplete response as a token problem. Log the response id, status, reason, model, configured budget, and token usage, while keeping sensitive input and output out of ordinary logs.

Do not silently persist the partial value with an is_partial flag unless your entire downstream domain is deliberately designed for partial records. That flag tends to be lost at the next queue or query boundary. The safer default is a separate failure record containing operational metadata but no actionable domain object.

Related Guides