Streaming

Set stream: true to receive typed events as they arrive. The SDK handles SSE framing, partial chunks, and the final usage event for you.

Request

const events = await client.responses.create({
  model: "ninja/auto",
  input: "Write a haiku about coding",
  stream: true,
});

Code examples

const events = await client.responses.create({
  model: "ninja/auto",
  input: "Write a haiku about coding",
  stream: true,
});

for await (const event of events) {
  if (event.type === "response.output_text.delta") {
    process.stdout.write(event.delta ?? "");
  }
}

events = client.responses.create(
    model="ninja/auto",
    input="Write a haiku about coding",
    stream=True,
)

for event in events:
    if event.get("type") == "response.output_text.delta":
        print(event.get("delta", ""), end="", flush=True)

const chunks = await client.chat.completions.create({
  model: "gpt-5.6-luna",
  messages: [{ role: "user", content: "Write a haiku about coding" }],
  stream: true,
  stream_options: { include_usage: true },
});

for await (const chunk of chunks) {
  process.stdout.write(chunk.choices[0]?.delta.content ?? "");
  if (chunk.usage) {
    console.log("\nTokens:", chunk.usage.total_tokens);
  }
}

SSE format

Both endpoints stream text/event-stream. While the gateway waits for the first token from a provider it may send comment frames — lines that begin with : — as keepalives. They carry no data: a spec-compliant SSE parser ignores them, and a hand-rolled parser must skip any line starting with : rather than trying to parse it as JSON.

Responses

Each frame carries an event: name and a data: JSON object whose type repeats the name and whose sequence_number increases monotonically:

event: response.output_text.delta
data: {"type":"response.output_text.delta","item_id":"msg_req_...","output_index":0,"content_index":0,"delta":"The","sequence_number":3}

Event	When
`response.created`	The stream opened; `response.status` is `in_progress`
`response.output_item.added`	A message item (`output_index` 0) or a function-call item begins
`response.content_part.added`	The text part of the message item begins
`response.output_text.delta`	A text delta in `delta`
`response.function_call_arguments.delta`	An arguments delta for a function-call item
`response.output_text.done`	The full `text` of the message
`response.content_part.done`	The completed text part
`response.output_item.done`	The completed message or function-call item
`response.function_call_arguments.done`	The full `arguments` for a function-call item
`response.completed`	The final `response` with `model`, `output_text`, `usage`, `cost_usd`, `provider`, `request_id`, and `routing`
`error`	An in-band failure: `{ "type": "error", "error": { "type", "code", "message" } }`. No `response.completed` follows

The stream closes after response.completed; there is no [DONE] sentinel on this endpoint.

Chat Completions

OpenAI-compatible chat.completion.chunk objects on data: lines, ending with data: [DONE]:

data: {"id":"chatcmpl_req_...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"The"},"finish_reason":null}]}
data: {"id":"chatcmpl_req_...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":" capital"},"finish_reason":null}]}
data: [DONE]

The final chunk — empty delta, finish_reason set — always carries the request receipt, whether or not you set stream_options.include_usage:

{
  "id": "chatcmpl_req_...",
  "object": "chat.completion.chunk",
  "created": 1787529600,
  "model": "gemini-3-flash",
  "resolved_model": "gemini-3-flash",
  "provider": "google",
  "choices": [{ "index": 0, "delta": {}, "finish_reason": "stop" }],
  "usage": {
    "prompt_tokens": 14,
    "completion_tokens": 8,
    "total_tokens": 22,
    "prompt_tokens_details": { "cached_tokens": 0, "cache_creation_tokens": 0 },
    "completion_tokens_details": { "reasoning_tokens": 0 }
  },
  "cost_usd": 0.00003,
  "request_id": "req_...",
  "routing": {
    "strategy": "balanced",
    "requested_models": ["gemini-3-flash"],
    "resolved_model": "gemini-3-flash",
    "provider": "google",
    "fallbacks_allowed": true,
    "data_policy": "default"
  }
}

If a provider fails after the HTTP status has been sent, the last data frame before [DONE] is an error object instead: partial_stream when some output was already delivered (only the delivered tokens are billed) or stream_error when nothing was (you are not charged).

Billing

Streaming uses the same metered rates as a non-streaming request and settles on the actual usage reported in the final frame.
If you abort the connection mid-stream, only the tokens delivered before the abort are billed.

Notes

Use the SDK unless you specifically need to own the SSE parser.
Abort TypeScript streams with AbortSignal in the request options.

Get Started

Chat

Orchestration

Production

Media & search

Reference

Request

Code examples

SSE format

Responses

Chat Completions

Billing

Notes

​Request

​Code examples

​SSE format

​Responses

​Chat Completions

​Billing

​Notes

Request

Code examples

SSE format

Responses

Chat Completions

Billing

Notes