Streaming and interrupted responses

Set stream: true on a Chat request to receive output incrementally through SSE. Streaming changes how output arrives; it does not guarantee fixed latency or completion time.

Complete Python example

Install dependencies using the SDK guide, then set APIONE_API_KEY and APIONE_MODEL. Save as stream.py and run python stream.py.

import os
from openai import OpenAI

with OpenAI(
    base_url="https://api.apione.ai/v1",
    api_key=os.environ["APIONE_API_KEY"],
    timeout=60.0,
    max_retries=0,
) as client:
    stream = client.chat.completions.create(
        model=os.environ["APIONE_MODEL"],
        messages=[{"role": "user", "content": "Write a short product description."}],
        stream=True,
    )
    try:
        for chunk in stream:
            if not chunk.choices:
                continue
            text = chunk.choices[0].delta.content
            if text:
                print(text, end="", flush=True)
        print()
    finally:
        stream.close()

The example uses a 60-second SDK timeout and disables SDK retries to make a single request easier to inspect. Adjust these settings for your application. Exceptions end execution and close the stream; already printed text remains available.

Inspect raw events with cURL

-N disables cURL output buffering so you can inspect incoming SSE events.

: "${APIONE_API_KEY:?Set APIONE_API_KEY first}"
: "${APIONE_MODEL:?Set APIONE_MODEL first}"

curl -N -sS https://api.apione.ai/v1/chat/completions \
  -H "Authorization: Bearer $APIONE_API_KEY" \
  -H "Content-Type: application/json" \
  --data "{\"model\":\"$APIONE_MODEL\",\"messages\":[{\"role\":\"user\",\"content\":\"Write a one-sentence product description.\"}],\"stream\":true}"

The content type is text/event-stream. Text updates generally appear in choices[0].delta.content, and data: [DONE] marks completion. A network byte chunk may not contain a complete SSE event. Custom parsers must buffer complete events and handle UTF-8 boundaries.

Empty chunks, errors, and interruptions

Situation	Handling
Role-only chunk, empty `choices`, or no text	Skip text rendering and check fields before indexing
Usage arrives near completion	Continue parsing to the end
Error or disconnect after HTTP 200	Mark the response incomplete, keep displayed content, and notify the user
Cancellation or application shutdown	Close the stream and release the connection
A retry is needed	Let the application decide explicitly; silently repeating a whole request can duplicate generation and charges

Browser applications should call your server, with the API Key kept on the server. See APIONE streaming documentation and troubleshooting.

Cancellation and billing

Model calls finally classified as failed by the platform are not charged. A client timeout, cancellation, disconnected stream, or failed download after successful generation does not by itself mean the platform call failed. If generation completed or billable usage was confirmed, the applicable model and group rules determine the charge. For streaming, check the final status and actual usage rather than HTTP 200 alone, the client’s error message, or whether the client received every output chunk. Incomplete delivery does not prove that no generation occurred. For a disputed status or charge, retain the request ID, timestamp and timezone, model, final status, and error details, then contact support. Confirmed erroneous or duplicate charges are corrected after investigation. Avoid repeatedly creating tasks while the original outcome is unknown. See failed and interrupted call billing.

Addresses and authentication

Model APIs

Output and advanced usage

Image APIs

Video APIs

Streaming and interrupted responses

Complete Python example

Inspect raw events with cURL

Empty chunks, errors, and interruptions

Cancellation and billing