Streaming AI Responses: SSE, Real-Time UI, and Production Patterns

Why Streaming Matters for AI UX

AI models emit output incrementally. Without streaming, your server waits for the complete response before forwarding it. With streaming, the user can see useful output before generation finishes. Measure time to first event on your own model, region and workload; it is not guaranteed to be under a second.

The underlying mechanism is Server-Sent Events (SSE) — the server holds the HTTP connection open and pushes chunks as they arrive. The Anthropic API, OpenAI API, and most other AI providers use SSE for their streaming endpoints.

Streaming with the Anthropic SDK

The Anthropic SDK provides a stream() method that returns an async iterator of events. Each event contains a text delta you append to the output.

import Anthropic from '@anthropic-ai/sdk';

const anthropic = new Anthropic();

async function streamResponse(userMessage: string): Promise<void> {
  const stream = anthropic.messages.stream({
    model: 'claude-sonnet-5-5',
    max_tokens: 1024,
    messages: [{ role: 'user', content: userMessage }],
  });

  for await (const event of stream) {
    if (
      event.type === 'content_block_delta' &&
      event.delta.type === 'text_delta'
    ) {
      process.stdout.write(event.delta.text);
    }
  }

  const finalMessage = await stream.finalMessage();
  console.log('\nUsage:', finalMessage.usage);
}

The finalMessage() call at the end returns the complete assembled message with usage statistics — input tokens, output tokens, and cache metrics if you're using prompt caching.

Using the Text Stream Helper

For simple text-only use cases, the SDK also exposes a stream.textStream async iterable that yields only the text deltas:

const stream = anthropic.messages.stream({ ... });

for await (const text of stream.textStream) {
  process.stdout.write(text);   // just the text, no event unwrapping
}

Passing Streaming Through a Node.js Backend

In production you typically don't expose your Anthropic API key to the browser. Instead, your frontend calls your own backend, which calls Anthropic and proxies the stream. The browser receives SSE from your server.

Express SSE Endpoint

import express from 'express';
import Anthropic from '@anthropic-ai/sdk';

const app = express();
app.use(express.json());

const anthropic = new Anthropic();

app.post('/api/chat/stream', async (req, res) => {
  const { message } = req.body;

  // Set SSE headers
  res.setHeader('Content-Type', 'text/event-stream');
  res.setHeader('Cache-Control', 'no-cache');
  res.setHeader('X-Accel-Buffering', 'no');
  res.flushHeaders();

  const stream = anthropic.messages.stream({
    model: 'claude-sonnet-5-5',
    max_tokens: 1024,
    messages: [{ role: 'user', content: message }],
  });

  for await (const event of stream) {
    if (
      event.type === 'content_block_delta' &&
      event.delta.type === 'text_delta'
    ) {
      // SSE format: "data: ...\n\n"
      res.write(`data: ${JSON.stringify({ text: event.delta.text })}\n\n`);
    }
  }

  res.write('data: [DONE]\n\n');
  res.end();
});

Handling Client Disconnects

If the user navigates away or closes the browser, the HTTP connection closes. Detect this and abort the upstream stream to avoid continuing to pay for tokens the user will never see:

app.post('/api/chat/stream', async (req, res) => {
  const stream = anthropic.messages.stream({
    model: 'claude-sonnet-5-5',
    max_tokens: 1024,
    messages: [...],
  });

  res.on('close', () => {
    if (!res.writableEnded) stream.abort();
  });

  try {
    for await (const event of stream) { ... }
  } catch (err) {
    if (err.name !== 'APIUserAbortError') throw err;
    // Client disconnected — silently exit
  }

  res.end();
});

Consuming Streams in React

On the frontend, use the browser's fetch API with a ReadableStream reader. The EventSource API only supports GET requests; for POST requests (with a body), read the response stream directly.

async function streamChat(
  message: string,
  onChunk: (text: string) => void,
  signal: AbortSignal
) {
  const response = await fetch('/api/chat/stream', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ message }),
    signal,
  });

  if (!response.ok) throw new Error(`HTTP ${response.status}`);
  if (!response.body) throw new Error('Streaming body unavailable');

  const reader = response.body.getReader();
  const decoder = new TextDecoder();
  let buffer = '';

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;

    buffer += decoder.decode(value, { stream: true });
    const events = buffer.split(/\r?\n\r?\n/);
    buffer = events.pop() ?? '';

    for (const rawEvent of events) {
      const payload = rawEvent
        .split(/\r?\n/)
        .filter((line) => line.startsWith('data:'))
        .map((line) => line.slice(5).trimStart())
        .join('\n');
      if (!payload) continue;
      if (payload === '[DONE]') return;
      const { text, error } = JSON.parse(payload);
      if (error) throw new Error(error);
      if (typeof text === 'string') onChunk(text);
    }
  }
}

Network chunks do not align with SSE event boundaries. Keep an incomplete event in a buffer as above; splitting and parsing each chunk independently will eventually truncate JSON. In production, prefer a maintained SSE parser and cap buffered event size.

React Hook

function useStreamingChat() {
  const [output, setOutput] = useState('');
  const [isStreaming, setIsStreaming] = useState(false);
  const abortRef = useRef<AbortController | null>(null);

  const send = useCallback(async (message: string) => {
    abortRef.current?.abort();
    abortRef.current = new AbortController();
    setOutput('');
    setIsStreaming(true);

    try {
      await streamChat(message, (text) => {
        setOutput((prev) => prev + text);
      }, abortRef.current.signal);
    } finally {
      setIsStreaming(false);
    }
  }, []);

  const stop = useCallback(() => {
    abortRef.current?.abort();
    setIsStreaming(false);
  }, []);

  return { output, isStreaming, send, stop };
}

SSE vs WebSockets

Both protocols keep a connection open for real-time updates, but they serve different patterns:

For a request followed by one streamed model response, SSE framing over an HTTP response is often the simpler choice. Test buffering, timeouts, disconnects and concurrent-connection limits on every proxy and CDN in the deployed path.

Use WebSockets when you need true bidirectionality: the server needs to push unsolicited updates (e.g., a background agent completing a task), or multiple clients need to share a live collaborative state.

Error Handling in Streams

Errors in a streaming context behave differently from request/response errors. The HTTP response has already started (200 OK headers sent) when the error occurs, so you can't change the status code. Instead, send an error event in the stream:

import type { Response } from 'express';

// Server: send an error event in the stream
function sendError(res: Response, message: string) {
  res.write(`data: ${JSON.stringify({ error: message })}\n\n`);
  res.write('data: [DONE]\n\n');
  res.end();
}

// In your handler:
try {
  for await (const event of stream) { ... }
} catch (err) {
  if (err instanceof Anthropic.APIError) {
    sendError(res, `API error: ${err.status}`);
  } else {
    sendError(res, 'An error occurred. Please try again.');
  }
}

// Client: check for error events
const { text, error } = JSON.parse(payload);
if (error) {
  showErrorToUser(error);
  return;
}
onChunk(text);

Retry on Network Interruption

Network interruptions mid-stream will cause the fetch to throw. Decide at the product level whether to automatically retry (acceptable for read-only queries) or surface the error to the user and let them resend. Automatic retry is risky for write operations or when the model has already partially responded.

Streaming with Tool Use

When your AI calls tools (function calling), the stream emits tool_use blocks in addition to text. You need to handle both:

const pendingTools = new Map<number, {
  id: string;
  name: string;
  inputJson: string;
}>();

for await (const event of stream) {
  if (
    event.type === 'content_block_delta' &&
    event.delta.type === 'text_delta'
  ) {
    process.stdout.write(event.delta.text);
  }

  if (
    event.type === 'content_block_start' &&
    event.content_block.type === 'tool_use'
  ) {
    pendingTools.set(event.index, {
      id: event.content_block.id,
      name: event.content_block.name,
      inputJson: '',
    });
  }

  if (
    event.type === 'content_block_delta' &&
    event.delta.type === 'input_json_delta'
  ) {
    const tool = pendingTools.get(event.index);
    if (tool) tool.inputJson += event.delta.partial_json;
  }

  if (event.type === 'content_block_stop') {
    const tool = pendingTools.get(event.index);
    if (!tool) continue;

    const input = JSON.parse(tool.inputJson);
    validateAgainstToolSchema(tool.name, input);
    const result = await executeToolByName(tool.name, input);
    // Send a tool_result block using tool.id in the next Messages request.
    pendingTools.delete(event.index);
  }
}

Track tool blocks by content-block index: more than one block can occur in a message. Execute by the declared tool name, validate the complete JSON against that tool's input schema, and return a tool_result with the matching tool-use ID. Do not execute partial JSON, and do not treat the model-generated arguments as trusted input.

Production Considerations

Timeouts

Configure and test every timeout in the path: application server, load balancer, reverse proxy, CDN and client. Do not copy a single total-duration estimate. Nginx's proxy_read_timeout is an inactivity timeout between upstream reads, not a cap on the whole response. Choose a value from measured gaps and your failure budget; comments can serve as heartbeats if every hop forwards them.

# In your Nginx location block for the streaming endpoint
proxy_read_timeout 120s;  # example only; set from measured idle gaps
proxy_buffering off;

Disabling Buffering

HTTP proxies and load balancers may buffer responses. Buffering defeats incremental delivery — the user gets one batch instead of a stream. Nginx buffering is enabled by default and can be disabled in configuration or with X-Accel-Buffering: no, unless the proxy is configured to ignore that header. Confirm behaviour in the deployed path:

res.setHeader('X-Accel-Buffering', 'no');   // tells Nginx not to buffer
res.setHeader('Cache-Control', 'no-cache');
res.flushHeaders();

Backpressure

If your Node.js server receives tokens faster than the network can flush them to the client, you'll accumulate data in memory. Check res.write()'s return value — it returns false when the write buffer is full, signaling you should pause until the drain event fires:

for await (const event of stream) {
  if (event.type === 'content_block_delta' && event.delta.type === 'text_delta') {
    const canWrite = res.write(`data: ${JSON.stringify({ text: event.delta.text })}\n\n`);
    if (!canWrite) {
      await new Promise((resolve) => res.once('drain', resolve));
    }
  }
}

For most AI streaming use cases this is a non-issue — tokens arrive slower than they can be sent. But it matters if you're building a high-throughput proxy or aggregating multiple model outputs.

Official Sources and Review Trigger

• Anthropic TypeScript SDK: message streaming helpers — textStream, finalMessage(), abort behaviour and accumulated events, checked Sep 22 2026.
• Claude Platform: streaming Messages — SSE event types, deltas, errors and tool-input streaming, checked Sep 22 2026.
• Anthropic: model status — active model ID used in the examples, checked Sep 22 2026.
• MDN: using server-sent events — UTF-8 event framing, multi-line data and EventSource reconnection, checked Sep 22 2026.
• Node.js HTTP documentation — response lifecycle and write()/drain backpressure, checked Sep 22 2026.
• Nginx proxy module — buffering, X-Accel-Buffering and inactivity timeouts, checked Sep 22 2026.

Review this guide again by Oct 22 2026, or earlier if the Anthropic SDK changes its streaming helpers, model ID or event types, or if the deployed Node.js, proxy or CDN stack changes.

Related Guides