spicyapiDocs
Main content

Text and streaming

Call text models in the official OpenAI, Anthropic or Google Gemini format, and handle conversations, tools, streams and costs correctly.

Text models answer in the same HTTP request: there is no task to poll. You can wait for the whole answer, or stream it โ€” receive the text in small pieces while the model is still writing, the way a chat app types out a reply. Streamed pieces arrive as Server-Sent Events (SSE): plain text lines that start with data:, sent one after another over a connection that stays open until the answer is complete.

Select an available text model

The shell examples on this page are written for a macOS or Linux terminal and use jq, a small command-line JSON tool (brew install jq on macOS, sudo apt install jq on Debian or Ubuntu). On Windows, run them in WSL, or use the Python SDK examples instead.

First put your API key in an environment variable โ€” a named value your terminal hands to the commands it runs, so the key never has to be pasted into a command:

export SPICY_API_KEY="sk-spicy-paste-your-key-here"

List the text models this key can call right now:

curl -s "https://api.spicyapi.ai/api/v1/models?modality=text" \
  -H "Authorization: Bearer $SPICY_API_KEY" \
  | jq -r '.data.items[] | select(.enabled and .available) | .model'

You should see one model ID per line. Tools that read the OpenAI-format list can ask for the same set with GET /v1/models?modality=text; there each item also carries task, context_length, max_output_tokens and supported_parameters. Copy one ID and keep it in SPICY_MODEL; every example below reads it:

export SPICY_MODEL="MODEL_ID_FROM_CATALOG"   # replace with one line from the list above

MODEL_ID_FROM_CATALOG is only a placeholder โ€” never send it literally. Before you rely on tools, images, reasoning settings or structured output, read that model's inputSchema from GET /api/v1/models/{model}: those features work only if the schema lists them. Protocol compatibility does not give every model every capability.

The same model ID works with any of the protocols below, regardless of who built the model: you can call another vendor's model in Anthropic format, or a non-Google model in Gemini format.

Looking for complete programs?

Chat recipes has runnable Python, Node.js and curl programs for multi-turn conversations, character role-play, streaming into your own interface and tool calling. Moving existing OpenAI or OpenRouter code? Start with Migrate from OpenAI or OpenRouter.

Supported protocols

All of these endpoints share the same API key and model catalog, along with the same rate limits, parameter validation and billing; only the request and response formats differ. Use whichever one your existing code or SDK already speaks.

ProtocolEndpointAPI key header
OpenAI Chat CompletionsPOST /v1/chat/completionsAuthorization: Bearer
OpenAI ResponsesPOST /v1/responsesAuthorization: Bearer
Anthropic MessagesPOST /v1/messagesx-api-key or Authorization: Bearer
Google GeminiPOST /v1beta/models/{model}:generateContentx-goog-api-key or Authorization: Bearer
Google Gemini (streaming)POST /v1beta/models/{model}:streamGenerateContentx-goog-api-key or Authorization: Bearer
Native stream with a confirmed pricePOST /api/v1/jobs/streamAuthorization: Bearer

On /v1 and /v1beta, responses and errors follow their own protocols and carry no native {code,msg,data,request_id} envelope, so do not parse them with a client that only reads body.data. The Gemini format puts the model ID in the URL path; slashes in the ID stay as they are and need no encoding.

Send the key in a request header only. x-api-key and x-goog-api-key work only on /v1 and /v1beta; the native /api/v1 accepts only Authorization: Bearer. A ?key= query parameter is never accepted: once a key lands in a URL, it lingers in browser history, proxy servers and access logs. The official SDKs all send the key in a header, so they are unaffected.

Use an official SDK

Change only the base URL and the key; everything else works the way each SDK normally does.

SDKBase URL
OpenAI (openai)https://api.spicyapi.ai/v1
Anthropic (anthropic)https://api.spicyapi.ai
Google GenAI (google-genai)https://api.spicyapi.ai

The Anthropic and Google SDKs append /v1 or /v1beta themselves, so their base URL has no version path; the OpenAI SDK's base URL must include /v1. All three Python examples below read SPICY_API_KEY from the environment, along with a SPICY_MODEL chosen from the live catalog.

OpenAI
import os
from openai import OpenAI

client = OpenAI(api_key=os.environ["SPICY_API_KEY"], base_url="https://api.spicyapi.ai/v1")
reply = client.chat.completions.create(
    model=os.environ["SPICY_MODEL"],
    messages=[{"role": "user", "content": "Explain a rainbow in one sentence."}],
)
print(reply.choices[0].message.content)
Anthropic
import os
import anthropic

client = anthropic.Anthropic(api_key=os.environ["SPICY_API_KEY"], base_url="https://api.spicyapi.ai")
message = client.messages.create(
    model=os.environ["SPICY_MODEL"],
    max_tokens=256,
    messages=[{"role": "user", "content": "Explain a rainbow in one sentence."}],
)
print(message.content[0].text)
Google GenAI
import os
from google import genai
from google.genai import types

client = genai.Client(
    api_key=os.environ["SPICY_API_KEY"],
    http_options=types.HttpOptions(base_url="https://api.spicyapi.ai"),
)
response = client.models.generate_content(
    model=os.environ["SPICY_MODEL"],
    contents="Explain a rainbow in one sentence.",
)
print(response.text)

The compatibility layer prices at acceptance and takes no quote

/v1 and /v1beta charge the price in force when the request is accepted. They do not accept quoteId or expectedCost, so this path has no ceiling on a single call โ€” an agent loop that runs away keeps spending.

Two ways to bound it, and most integrations want both:

  • One call at a time โ€” quote with the native POST /api/v1/jobs/quote, then submit POST /api/v1/jobs/stream carrying the returned quoteId and expectedCost. If the price moved in between, acceptance fails with 40901 instead of charging more than you agreed to. See Native stream with a confirmed price.
  • Everything this key spends โ€” set the key's daily, monthly and lifetime spend caps. They apply to every surface, this one included, and a breach returns 40202 instead of a larger bill. See Authentication.

Idempotency keys on text endpoints

An Idempotency-Key header stops a network retry from charging twice. On text endpoints it does not let you get a lost answer back, because the answer is streamed to you and is not stored:

  • Resending a key whose request completed, with the same body, returns 400 (This Idempotency-Key was already used for a completed request, and its answer cannot be replayed. Use a new Idempotency-Key to send a new request) and charges nothing more. The task record will not contain the text either.
  • If that earlier request failed, resending the same key and body returns the same error it did, and charges nothing.
  • Resending it with a different body returns 409.

So: send a new key with every new message (for example $(uuidgen) in a shell, uuid.uuid4() in Python), and reuse a key only when you resend the exact same request after a network error โ€” accepting that the reply to that request cannot be recovered. If you want an answer again, send it with a new key and pay for it.

In the SDKs, pass the key per call, never on the client object โ€” with one key for the whole client, every message after the first would be rejected with 409 for 24 hours:

Per-call Idempotency-Key
import os, uuid

# OpenAI SDK (client from the OpenAI example above). The Anthropic SDK takes
# the same extra_headers argument on client.messages.create().
reply = client.chat.completions.create(
    model=os.environ["SPICY_MODEL"],
    messages=[{"role": "user", "content": "Explain a rainbow in one sentence."}],
    extra_headers={"Idempotency-Key": str(uuid.uuid4())},
)

# Google GenAI SDK (client and types from the Google GenAI example above):
# http_options inside the per-call config
response = client.models.generate_content(
    model=os.environ["SPICY_MODEL"],
    contents="Explain a rainbow in one sentence.",
    config=types.GenerateContentConfig(
        http_options=types.HttpOptions(headers={"Idempotency-Key": str(uuid.uuid4())}),
    ),
)

The OpenAI and Anthropic SDKs retry timeouts on their own with the same headers. If such a retry reaches a request we had already accepted, the SDK raises a 400 bad-request error with the message above: the first attempt was processed and charged, and its reply is lost.

Use SpicyAPI models from the Grok command-line client

This configures xAI's command-line client to send its requests to SpicyAPI, so it talks to the models in the SpicyAPI catalog rather than to xAI's own endpoint. That includes the Grok family published in the catalog: call it by its SpicyAPI model ID, like any other model. The client can point a custom model at any OpenAI-compatible endpoint. Add the entry below to ~/.grok/config.toml; env_key names the environment variable the client reads, so the key itself never sits in the file.

~/.grok/config.toml
[model.spicyapi]
model = "MODEL_ID_FROM_CATALOG"
base_url = "https://api.spicyapi.ai/v1"
name = "SpicyAPI"
env_key = "SPICY_API_KEY"

[models]
default = "spicyapi"

Replace MODEL_ID_FROM_CATALOG with an exact text model ID from the live catalog, in the form <publisher>/<model>/<task>; never leave the placeholder in place. Then start with grok -m spicyapi, or switch inside a session with /model spicyapi. The client speaks Chat Completions, so the parameter mapping, stream endings and error shapes described below apply unchanged.

Parameter mapping across protocols

Protocol-level fields are first converted to the platform's canonical parameter names, then validated against the selected model's inputSchema. A parameter the model does not support returns 400; it is never silently dropped while the request is billed as usual. The table below shows the mapping only, and "โ€”" means the protocol has no equivalent field. Whether a given model accepts a parameter, and which values it allows, is defined by that model's inputSchema.

Canonical parameterChat CompletionsResponsesMessagesGemini
messagesmessagesinstructions, inputsystem, messagessystemInstruction, contents
max_tokensmax_tokens, max_completion_tokensmax_output_tokensmax_tokensgenerationConfig.maxOutputTokens
temperaturetemperaturetemperaturetemperaturegenerationConfig.temperature
top_ptop_ptop_ptop_pgenerationConfig.topP
toolstoolstools (function type)tools (custom tools)tools[].functionDeclarations
tool_choicetool_choicetool_choicetool_choicetoolConfig.functionCallingConfig
response_formatresponse_formattext.formatโ€”generationConfig.responseMimeType plus responseSchema or responseJsonSchema
reasoning_effortreasoning_effortreasoning.effortโ€”generationConfig.thinkingConfig

Fields not listed in the table are likewise converted to canonical parameters and validated by the model; anything missing from its inputSchema returns 400. For example, only some models list seed in their inputSchema; on the others, Chat's seed and Gemini's generationConfig.seed return 400. Sampling settings and stop sequences are the exception: on a model that does not take them they are ignored (named in the X-Spicy-Ignored-Params response header), and a value outside what the model takes is changed to the nearest accepted one (named in X-Spicy-Adjusted-Params). That covers temperature, top_p and the penalties, Chat's stop, Messages' stop_sequences, Gemini's generationConfig.stopSequences, and reasoning_effort levels. Read the schema of the model you chose rather than assuming a field from another model.

A few exceptions:

  • Fields that only mean something to the protocol's own platform are ignored and do not affect generation: user, metadata and store in Chat; user, metadata, store and include in Responses; metadata in Messages.
  • Responses does not support previous_response_id; send the full history in input.
  • Gemini's safetySettings is accepted but has no effect. cachedContent and server-side tools such as googleSearch and codeExecution are not supported and return 400. Pass images as inlineData (base64) or fileData.fileUri (an https URL); whether a model accepts images is defined by its inputSchema.

Send a Chat stream

With SPICY_API_KEY and SPICY_MODEL set as above, this writes the request to a file and streams the answer. Replace the sample prompt as needed, and validate model parameters against the current schema before sending. Each run sends a fresh Idempotency-Key, because each run is a new message.

jq -n --arg model "$SPICY_MODEL" '{
  model: $model,
  messages: [{role: "user", content: "Explain a rainbow in one sentence."}],
  stream: true
}' > chat-request.json

curl --fail-with-body --no-buffer --max-time 120 \
  https://api.spicyapi.ai/v1/chat/completions \
  -H "Authorization: Bearer $SPICY_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  --data-binary @chat-request.json

--no-buffer prints each piece as soon as it arrives, and --max-time 120 gives up after two minutes. You should see lines like the ones below arriving one by one, and finally data: [DONE]. The answer is spread across the delta.content pieces; join them in order to get the full text. Streams follow OpenAI's usage rules: without stream_options.include_usage, no chunk carries usage and every chunk has a non-empty choices list. Send "stream_options": {"include_usage": true} to receive the token counts: one extra chunk with an empty choices list and the final usage then arrives just before [DONE], and every other chunk has usage: null. Skip choices on that last chunk (the Python and Node.js examples under Timeouts and streaming check choices first, so they work either way). Each choice ends with exactly one chunk that carries its finish_reason.

data: {"id":"chatcmpl-req_01k...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"A rainbow"}}]}

data: {"id":"chatcmpl-req_01k...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":" forms when"}}]}

data: [DONE]

uuidgen ships with macOS; on Linux it comes with the uuid-runtime package.

Send a Gemini request

This uses the same SPICY_API_KEY and SPICY_MODEL, and again a new Idempotency-Key: it is a different request from the Chat example, so reusing that key would return 409. The model ID goes in the URL, so the request body carries no model field. Wrap the URL in double quotes and write the variable as ${SPICY_MODEL}, so the shell does not read the colon that follows it as a variable modifier.

jq -n '{
  systemInstruction: {parts: [{text: "Answer in one sentence."}]},
  contents: [{role: "user", parts: [{text: "Explain a rainbow."}]}],
  generationConfig: {temperature: 0.7, maxOutputTokens: 256}
}' > gemini-request.json

curl --fail-with-body --no-buffer --max-time 120 \
  "https://api.spicyapi.ai/v1beta/models/${SPICY_MODEL}:streamGenerateContent?alt=sse" \
  -H "x-goog-api-key: $SPICY_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  --data-binary @gemini-request.json

If you do not need streaming, replace :streamGenerateContent?alt=sse with :generateContent and read the answer from candidates[0].content.parts[].text; usage is in usageMetadata. Without alt=sse, a streaming request returns a single JSON array written out incrementally; the official SDKs add alt=sse for you.

Native stream with a confirmed price

POST /api/v1/jobs/stream is the one text endpoint that accepts a quote, so it can refuse to run if the price changed. The request body has the same shape as createTask: model plus an input whose fields come from the model's inputSchema (for text models, messages and the other canonical parameters in the table above).

# 1. Write the request once.
jq -n --arg model "$SPICY_MODEL" '{
  model: $model,
  input: {messages: [{role: "user", content: "Explain a rainbow in one sentence."}]}
}' > native-request.json

# 2. Quote it. This creates nothing and charges nothing.
QUOTE=$(curl -s https://api.spicyapi.ai/api/v1/jobs/quote \
  -H "Authorization: Bearer $SPICY_API_KEY" \
  -H "Content-Type: application/json" \
  --data-binary @native-request.json)
echo "$QUOTE" | jq '{code, msg, estimatedCost: .data.estimatedCost, maxCharge: .data.maxCharge, expiresAt: .data.expiresAt}'

# 3. Within five minutes, add quoteId and expectedCost to the unchanged request and stream it.
jq --argjson quote "$QUOTE" \
  '. + {quoteId: $quote.data.quoteId, expectedCost: $quote.data.estimatedCost}' \
  native-request.json > native-confirmed.json

curl --fail-with-body --no-buffer --max-time 120 \
  https://api.spicyapi.ai/api/v1/jobs/stream \
  -H "Authorization: Bearer $SPICY_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  --data-binary @native-confirmed.json

Step 2 prints code: 200 and the price in US dollars; any other code means the quote failed, and msg says why. If the stream call fails before any text is sent โ€” for example with 40901 because the price or the request changed, or 40201 for too little balance โ€” you get the normal {code,msg,data,request_id} JSON and nothing is held. Once streaming has started, a failure ends the stream with event: error and data: {"code":"stream_interrupted"}, and nothing is charged for it. Image, video and audio models return 503 here; use createTask for them.

The answer arrives as SSE data: lines. Unlike /v1/chat/completions, this endpoint does not promise one event format for every model, so test your parser with the model you chose; if you need a single stable format, use a /v1 endpoint and bound spending with the key's caps.

Send images, video and audio to a text model

Some text models read images as well as text, and a few also read video and audio. You send the media with a question, and the answer is still text. What a model accepts is written in its schema, not implied by the protocol. Check the model you chose:

curl -s "https://api.spicyapi.ai/api/v1/models/$SPICY_MODEL" \
  -H "Authorization: Bearer $SPICY_API_KEY" \
  | jq -c '[.data.inputSchema.properties.messages.items.properties.content.items.oneOf[]?.properties.type.const]'

The list names the content parts the model takes. ["text","image_url"] means text and images; a model that also reads video and audio lists video_url, audio_url and input_audio too. An empty list means text only. Sending a part the model does not list returns 400 and charges nothing.

To send media, make the message content a list of parts instead of a string โ€” a text part for the question and one part per file:

jq -n --arg model "$SPICY_MODEL" '{
  model: $model,
  stream: true,
  messages: [{
    role: "user",
    content: [
      {type: "text", text: "What is in this picture?"},
      {type: "image_url", image_url: {url: "https://example.com/photo.jpg"}}
    ]
  }]
}' > image-request.json

curl --fail-with-body --no-buffer --max-time 120 \
  https://api.spicyapi.ai/v1/chat/completions \
  -H "Authorization: Bearer $SPICY_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  --data-binary @image-request.json

Each kind of media has its own part:

PartShapeWhere the file comes from
Image{"type": "image_url", "image_url": {"url": "โ€ฆ"}}A public HTTPS URL, an uploaded file (spicy://f/โ€ฆ), or a small data: URI
Video{"type": "video_url", "video_url": {"url": "โ€ฆ"}}An uploaded file (spicy://f/โ€ฆ), or a public HTTPS URL
Audio file{"type": "audio_url", "audio_url": {"url": "spicy://f/โ€ฆ"}}An uploaded file only
Inline audio{"type": "input_audio", "input_audio": {"data": "<base64>", "format": "wav"}}Base64 in the request, wav or mp3

How to upload a file and get its spicy://f/โ€ฆ URI is described in Media; uploads last one day. A few rules follow from how each kind reaches the model:

  • Images can be a public HTTPS URL that returns the image file itself (not a page that shows it), an uploaded file, or a data: URI within the inline limits. JPEG and PNG work on every model that accepts images.
  • Video and audio work best as uploads. We read the uploaded file and hand it to the model inside the request, so the uploaded video and audio in one request can total at most 14 MB โ€” MP4 or WebM video, MP3 or WAV audio. A larger total returns 400 and charges nothing. A public video URL is passed on as a link instead, and works only if the model's provider can download it.
  • Inline audio (input_audio) follows the OpenAI format and has to fit in the 2 MiB request body, so it suits short clips; upload anything longer and send it as audio_url.

Several files can go in one message, each as its own part, and earlier messages in the history may carry media too.

The other protocols carry images in their own shape, and all of them reach the model identically. Video and audio parts are sent through Chat Completions and the native stream.

ProtocolImage part
Chat Completions{"type": "image_url", "image_url": {"url": "โ€ฆ"}}
Responses{"type": "input_image", "image_url": "โ€ฆ"}
Anthropic Messages{"type": "image", "source": {"type": "url", "url": "โ€ฆ"}}, or a base64 source
GeminiinlineData (base64) or fileData.fileUri (an HTTPS URL)

What it costs. Media is counted as input tokens at the model's normal input rate โ€” there is no separate per-file price. An image typically adds a few hundred to a few thousand tokens depending on the model and the image size; video and audio grow with their length. The usage block of the response reports the exact number.

What is held up front. Before a request runs we hold an upper bound and release the difference when it settles. For a request whose only media are images you uploaded or sent inline, the bound adds a fixed allowance per image. For a request that carries video or audio, or an image given as a public URL, the bound is sized to the model's whole input limit, because we cannot know its size in advance โ€” so the held amount can be far above the final charge. Upload images to keep the hold small, keep enough balance for it, or quote the request first with POST /api/v1/jobs/quote to see it.

In the Playground. On a model that accepts media, the message box shows an attach button. You can also paste a screenshot or drop files onto the box; each message takes up to four attachments, and the video and audio among them can total 14 MB. Models that read text only show no button.

Conversations and tool calls

Complete tool-calling programs in Python, Node.js and curl are in Chat recipes.

Chat Completions

  • Non-streaming: read choices[0].message.
  • Streaming: process choices[].delta one SSE event at a time, not per network packet. Assemble tool_calls by index, and validate and execute a call only once its arguments are complete.
  • Keep the assistant message in the history, and return each result in a tool message with the matching tool_call_id.

Responses

  • Send the full history in input; previous_response_id is rejected.
  • Return function results as function_call_output items with the matching call_id.

Messages

  • Use text, tool_use and tool_result content blocks rather than the Chat message shape.

Gemini

  • contents alternate between two roles, user and model.
  • When the model wants to call a tool, it returns a functionCall part. After running the tool, wrap the result in a functionResponse part and send it back in the next user content, using the same name as the call.

Completion, errors and disconnects

ProtocolNormal completionUsage
Chat Completionsdata: [DONE]Only with stream_options.include_usage: a chunk with empty choices and usage, just before [DONE]
Responsesresponse.completed eventresponse.usage in that event
Messagesmessage_stop after message_deltausage in message_delta
GeminiLast event with finishReason; there is no [DONE]usageMetadata

Check the HTTP status and Content-Type before reading a stream; errors come back as each protocol's own JSON:

ProtocolError body
Chat Completions, Responses{"error": {"message", "type", "param", "code"}}
Messages{"type": "error", "error": {"type", "message", "code"}}
Gemini{"error": {"code", "message", "status", "details"}}

HTTP status codes are the same across all four protocols. Gemini's status uses Google's status names: 400 is INVALID_ARGUMENT, 401 is UNAUTHENTICATED, 402 is FAILED_PRECONDITION, 403 is PERMISSION_DENIED, 404 is NOT_FOUND, 409 is ABORTED, 429 is RESOURCE_EXHAUSTED, 500 is INTERNAL, 503 is UNAVAILABLE and 504 is DEADLINE_EXCEEDED. Branch your code on the status code and the stable reason code, never on the message text.

The reason code tells apart errors that share a status. It is error.code in the OpenAI shape, the extra error.code field in the Anthropic shape (the official Anthropic SDKs ignore it, so read it from the raw body), and the reason of the google.rpc.ErrorInfo entry in Gemini's details, upper-cased (for example CONTENT_REJECTED). The ones to know:

HTTPCodeMeaning
400unsupported_parameterThe model does not accept this field, or this kind of message part (for example an image_url part on a text-only model). Remove it
400invalid_requestA field is missing or has a value the model does not accept, or the model is not a chat model. The message says what to change
400content_rejectedThe model's content policy refused the request. Sending the same input again is refused again
404model_not_foundNo model has this ID. The message shows the ID shape and, when there is one, the closest callable chat model

When the error is about one field, the OpenAI shape names it in param as a dot path, for example temperature, messages.1.content.0 or model.

If something fails after streaming begins, Chat and Gemini send an event carrying an error object, Responses sends an error event and Messages sends event: error. Your client must handle these error events, as well as timeouts and a connection that closes without the normal completion marker. If you run the stream through your own proxy, turn off response buffering there, or the pieces arrive in bursts.

What you pay when a stream stops early

  • You disconnect (close the tab, abort the request, hit a client timeout): disconnecting does not stop the bill. You are charged for the usage the model reports, which for many models covers the whole answer โ€” including text you never received โ€” and never more than the amount held when the request was accepted. Text you did not receive cannot be fetched later. To spend less, lower max_tokens rather than disconnecting.
  • The model or our service fails mid-stream (you receive an error event, or the stream ends without its completion marker): the request is marked failed and nothing is charged.
  • The connection to the model breaks only after the answer is complete (every choice already has its finish_reason and the final usage has arrived): nothing is missing from your answer, so the stream ends normally with its completion marker and the request is charged for the usage delivered, like any completed request.
  • The model sent you text or reasoning, then its answer was judged empty or refused: what you already received is billed by the usage the model reports, never more than the amount held, and the stream ends normally. Only when nothing at all reached you is it refunded in full.
  • The request runs past its server deadline: it ends as a failure and nothing is charged.

Timeouts and streaming

Without streaming, the answer arrives in one piece once it is ready, and long answers and long reasoning take time โ€” models such as Grok, GLM, DeepSeek Pro and Claude Opus can think for minutes before they write anything, especially at a high reasoning_effort. The network edge in front of the API closes a connection on which nothing has arrived for about 100 seconds, so requests of both kinds are kept open while the model works.

  • A non-streamed request still running after about 30 seconds starts its response early. You receive status 200, Content-Type: application/json and the header X-Spicy-Keepalive: 1, then a newline every 15 seconds, then the JSON body. JSON allows whitespace before a value, so standard parsers (json.loads, JSON.parse, response.json()) and the official SDKs read the body unchanged. A request that finishes sooner gets a normal response with its real status code and without the header.
  • After that point, a failure is in the body, not in the status. The 200 has already been sent, so the body holds the protocol's usual error object from the table above, with the same code and message; in the Gemini format, error.code still carries the HTTP status the error would have had. When X-Spicy-Keepalive is present, check the body for an error object before reading the answer โ€” the official SDKs do not treat a 200 as an error. A failed request is not charged.
  • Streaming keeps the connection open too. While the model is still working, a streamed response carries keep-alive lines โ€” : keep-alive comments over SSE, blank lines in the Gemini format โ€” so the connection is never idle, however long the model thinks. Your parser must skip them, as the SSE format requires; the official SDKs already do.
  • Prefer stream: true for anything that may run long. You see the answer as it is written, and a failure arrives as an error event that the SDKs understand. Chat Completions, Responses and Messages take stream: true; in the Gemini format, call :streamGenerateContent?alt=sse instead of :generateContent. A changed body is a new request, so send it with a new Idempotency-Key: reusing the old one returns 409.
  • A request cut off by the network edge is not charged. If nothing at all reaches you for about 100 seconds and the connection closes, the request appears under Console โ†’ Logs and in GET /api/v1/jobs as failed, with errorCode timeout and a message that says what happened. Resending it with the same Idempotency-Key returns that message as the error.
  • Your own time limits still apply. A total timeout in your HTTP client, or in a proxy you run, ends a request too, and once anything has reached you โ€” a keep-alive line included โ€” that counts as you disconnecting (see What you pay when a stream stops early). The official OpenAI SDKs wait up to 10 minutes by default; the curl examples on this page give up after 2 minutes, so raise --max-time for long reasoning.

Turning streaming on in the OpenAI SDKs and in curl:

OpenAI Python
import os
from openai import OpenAI

client = OpenAI(api_key=os.environ["SPICY_API_KEY"], base_url="https://api.spicyapi.ai/v1")
stream = client.chat.completions.create(
    model=os.environ["SPICY_MODEL"],
    messages=[{"role": "user", "content": "Explain a rainbow in one sentence."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
print()
OpenAI Node.js
import OpenAI from "openai";

const client = new OpenAI({ apiKey: process.env.SPICY_API_KEY, baseURL: "https://api.spicyapi.ai/v1" });
const stream = await client.chat.completions.create({
  model: process.env.SPICY_MODEL,
  messages: [{ role: "user", content: "Explain a rainbow in one sentence." }],
  stream: true,
});
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}
process.stdout.write("\n");
curl
curl --fail-with-body --no-buffer --max-time 900 \
  https://api.spicyapi.ai/v1/chat/completions \
  -H "Authorization: Bearer $SPICY_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  --data-binary @chat-request.json

The Node.js example uses top-level await, so run it as an ES module (for example, save it as stream.mjs). The curl command sends chat-request.json from Send a Chat stream, which already sets stream: true; --max-time 900 lets the answer take up to 15 minutes.

Quotes and billing

For explicit cost confirmation, use the native stream with a quote. /v1 and /v1beta do not accept quote credentials; they apply the price at acceptance.

Every text request that is accepted runs as a task, and the response names it in the X-Spicy-Task-Id header (job_โ€ฆ), streamed or not, on all four protocols and on /api/v1/jobs/stream. The header is also present on an error returned after acceptance, whose task shows the refund; a request rejected before acceptance, such as one that fails parameter validation, creates no task and has no header. An end-of-text marker is not proof of the final charge: read the task with GET /api/v1/jobs/recordInfo?taskId=<job_โ€ฆ> and treat cost as final only when settled is true. Items in GET /api/v1/jobs also carry the requestId of the call that created them (the X-Request-Id response header, which is also the suffix of the chatcmpl-req_โ€ฆ ID), so you can match either way; Console โ†’ Logs shows the same tasks. See Task history and recovery and Billing.

reasoning_tokens is already included in completion_tokens: do not add it twice.

Other request shapes

These examples describe protocol envelopes only. Replace MODEL_ID_FROM_CATALOG with a live model that supports the fields shown; never send the placeholder itself.

POST /v1/responses
{
  "model": "MODEL_ID_FROM_CATALOG",
  "input": "Explain a rainbow in one sentence.",
  "stream": false
}
POST /v1/messages
{
  "model": "MODEL_ID_FROM_CATALOG",
  "max_tokens": 256,
  "messages": [{"role": "user", "content": "Explain a rainbow in one sentence."}],
  "stream": false
}

Continue reading

Chat recipes ยท Migrate from OpenAI or OpenRouter ยท Quotes and protocol compatibility ยท Idempotency ยท Billing ยท Production integration ยท Troubleshooting and recovery

On this page