Text and streaming
Call text models in the official OpenAI, Anthropic or Google Gemini format, and handle conversations, tools, streams and costs correctly.
Text models answer in the same HTTP request: there is no task to poll. You can wait for the whole answer, or stream it โ receive the text in small pieces while the model is still writing, the way a chat app types out a reply. Streamed pieces arrive as Server-Sent Events (SSE): plain text lines that start with data:, sent one after another over a connection that stays open until the answer is complete.
Select an available text model
The shell examples on this page are written for a macOS or Linux terminal and use jq, a small command-line JSON tool (brew install jq on macOS, sudo apt install jq on Debian or Ubuntu). On Windows, run them in WSL, or use the Python SDK examples instead.
First put your API key in an environment variable โ a named value your terminal hands to the commands it runs, so the key never has to be pasted into a command:
export SPICY_API_KEY="sk-spicy-paste-your-key-here"List the text models this key can call right now:
curl -s "https://api.spicyapi.ai/api/v1/models?modality=text" \
-H "Authorization: Bearer $SPICY_API_KEY" \
| jq -r '.data.items[] | select(.enabled and .available) | .model'You should see one model ID per line. Tools that read the OpenAI-format list can ask for the same set with GET /v1/models?modality=text; there each item also carries task, context_length, max_output_tokens and supported_parameters. Copy one ID and keep it in SPICY_MODEL; every example below reads it:
export SPICY_MODEL="MODEL_ID_FROM_CATALOG" # replace with one line from the list aboveMODEL_ID_FROM_CATALOG is only a placeholder โ never send it literally. Before you rely on tools, images, reasoning settings or structured output, read that model's inputSchema from GET /api/v1/models/{model}: those features work only if the schema lists them. Protocol compatibility does not give every model every capability.
The same model ID works with any of the protocols below, regardless of who built the model: you can call another vendor's model in Anthropic format, or a non-Google model in Gemini format.
Looking for complete programs?
Chat recipes has runnable Python, Node.js and curl programs for multi-turn conversations, character role-play, streaming into your own interface and tool calling. Moving existing OpenAI or OpenRouter code? Start with Migrate from OpenAI or OpenRouter.
Supported protocols
All of these endpoints share the same API key and model catalog, along with the same rate limits, parameter validation and billing; only the request and response formats differ. Use whichever one your existing code or SDK already speaks.
| Protocol | Endpoint | API key header |
|---|---|---|
| OpenAI Chat Completions | POST /v1/chat/completions | Authorization: Bearer |
| OpenAI Responses | POST /v1/responses | Authorization: Bearer |
| Anthropic Messages | POST /v1/messages | x-api-key or Authorization: Bearer |
| Google Gemini | POST /v1beta/models/{model}:generateContent | x-goog-api-key or Authorization: Bearer |
| Google Gemini (streaming) | POST /v1beta/models/{model}:streamGenerateContent | x-goog-api-key or Authorization: Bearer |
| Native stream with a confirmed price | POST /api/v1/jobs/stream | Authorization: Bearer |
On /v1 and /v1beta, responses and errors follow their own protocols and carry no native {code,msg,data,request_id} envelope, so do not parse them with a client that only reads body.data. The Gemini format puts the model ID in the URL path; slashes in the ID stay as they are and need no encoding.
Send the key in a request header only. x-api-key and x-goog-api-key work only on /v1 and /v1beta; the native /api/v1 accepts only Authorization: Bearer. A ?key= query parameter is never accepted: once a key lands in a URL, it lingers in browser history, proxy servers and access logs. The official SDKs all send the key in a header, so they are unaffected.
Use an official SDK
Change only the base URL and the key; everything else works the way each SDK normally does.
| SDK | Base URL |
|---|---|
OpenAI (openai) | https://api.spicyapi.ai/v1 |
Anthropic (anthropic) | https://api.spicyapi.ai |
Google GenAI (google-genai) | https://api.spicyapi.ai |
The Anthropic and Google SDKs append /v1 or /v1beta themselves, so their base URL has no version path; the OpenAI SDK's base URL must include /v1. All three Python examples below read SPICY_API_KEY from the environment, along with a SPICY_MODEL chosen from the live catalog.
import os
from openai import OpenAI
client = OpenAI(api_key=os.environ["SPICY_API_KEY"], base_url="https://api.spicyapi.ai/v1")
reply = client.chat.completions.create(
model=os.environ["SPICY_MODEL"],
messages=[{"role": "user", "content": "Explain a rainbow in one sentence."}],
)
print(reply.choices[0].message.content)import os
import anthropic
client = anthropic.Anthropic(api_key=os.environ["SPICY_API_KEY"], base_url="https://api.spicyapi.ai")
message = client.messages.create(
model=os.environ["SPICY_MODEL"],
max_tokens=256,
messages=[{"role": "user", "content": "Explain a rainbow in one sentence."}],
)
print(message.content[0].text)import os
from google import genai
from google.genai import types
client = genai.Client(
api_key=os.environ["SPICY_API_KEY"],
http_options=types.HttpOptions(base_url="https://api.spicyapi.ai"),
)
response = client.models.generate_content(
model=os.environ["SPICY_MODEL"],
contents="Explain a rainbow in one sentence.",
)
print(response.text)The compatibility layer prices at acceptance and takes no quote
/v1 and /v1beta charge the price in force when the request is accepted. They do not accept quoteId or expectedCost, so this path has no ceiling on a single call โ an agent loop that runs away keeps spending.
Two ways to bound it, and most integrations want both:
- One call at a time โ quote with the native
POST /api/v1/jobs/quote, then submitPOST /api/v1/jobs/streamcarrying the returnedquoteIdandexpectedCost. If the price moved in between, acceptance fails with40901instead of charging more than you agreed to. See Native stream with a confirmed price. - Everything this key spends โ set the key's daily, monthly and lifetime spend caps. They apply to every surface, this one included, and a breach returns
40202instead of a larger bill. See Authentication.
Idempotency keys on text endpoints
An Idempotency-Key header stops a network retry from charging twice. On text endpoints it does not let you get a lost answer back, because the answer is streamed to you and is not stored:
- Resending a key whose request completed, with the same body, returns
400(This Idempotency-Key was already used for a completed request, and its answer cannot be replayed. Use a new Idempotency-Key to send a new request) and charges nothing more. The task record will not contain the text either. - If that earlier request failed, resending the same key and body returns the same error it did, and charges nothing.
- Resending it with a different body returns
409.
So: send a new key with every new message (for example $(uuidgen) in a shell, uuid.uuid4() in Python), and reuse a key only when you resend the exact same request after a network error โ accepting that the reply to that request cannot be recovered. If you want an answer again, send it with a new key and pay for it.
In the SDKs, pass the key per call, never on the client object โ with one key for the whole client, every message after the first would be rejected with 409 for 24 hours:
import os, uuid
# OpenAI SDK (client from the OpenAI example above). The Anthropic SDK takes
# the same extra_headers argument on client.messages.create().
reply = client.chat.completions.create(
model=os.environ["SPICY_MODEL"],
messages=[{"role": "user", "content": "Explain a rainbow in one sentence."}],
extra_headers={"Idempotency-Key": str(uuid.uuid4())},
)
# Google GenAI SDK (client and types from the Google GenAI example above):
# http_options inside the per-call config
response = client.models.generate_content(
model=os.environ["SPICY_MODEL"],
contents="Explain a rainbow in one sentence.",
config=types.GenerateContentConfig(
http_options=types.HttpOptions(headers={"Idempotency-Key": str(uuid.uuid4())}),
),
)The OpenAI and Anthropic SDKs retry timeouts on their own with the same headers. If such a retry reaches a request we had already accepted, the SDK raises a 400 bad-request error with the message above: the first attempt was processed and charged, and its reply is lost.
Use SpicyAPI models from the Grok command-line client
This configures xAI's command-line client to send its requests to SpicyAPI, so it talks to the models in the SpicyAPI catalog rather than to xAI's own endpoint. That includes the Grok family published in the catalog: call it by its SpicyAPI model ID, like any other model. The client can point a custom model at any OpenAI-compatible endpoint. Add the entry below to ~/.grok/config.toml; env_key names the environment variable the client reads, so the key itself never sits in the file.
[model.spicyapi]
model = "MODEL_ID_FROM_CATALOG"
base_url = "https://api.spicyapi.ai/v1"
name = "SpicyAPI"
env_key = "SPICY_API_KEY"
[models]
default = "spicyapi"Replace MODEL_ID_FROM_CATALOG with an exact text model ID from the live catalog, in the form <publisher>/<model>/<task>; never leave the placeholder in place. Then start with grok -m spicyapi, or switch inside a session with /model spicyapi. The client speaks Chat Completions, so the parameter mapping, stream endings and error shapes described below apply unchanged.
Parameter mapping across protocols
Protocol-level fields are first converted to the platform's canonical parameter names, then validated against the selected model's inputSchema. A parameter the model does not support returns 400; it is never silently dropped while the request is billed as usual. The table below shows the mapping only, and "โ" means the protocol has no equivalent field. Whether a given model accepts a parameter, and which values it allows, is defined by that model's inputSchema.
| Canonical parameter | Chat Completions | Responses | Messages | Gemini |
|---|---|---|---|---|
messages | messages | instructions, input | system, messages | systemInstruction, contents |
max_tokens | max_tokens, max_completion_tokens | max_output_tokens | max_tokens | generationConfig.maxOutputTokens |
temperature | temperature | temperature | temperature | generationConfig.temperature |
top_p | top_p | top_p | top_p | generationConfig.topP |
tools | tools | tools (function type) | tools (custom tools) | tools[].functionDeclarations |
tool_choice | tool_choice | tool_choice | tool_choice | toolConfig.functionCallingConfig |
response_format | response_format | text.format | โ | generationConfig.responseMimeType plus responseSchema or responseJsonSchema |
reasoning_effort | reasoning_effort | reasoning.effort | โ | generationConfig.thinkingConfig |
Fields not listed in the table are likewise converted to canonical parameters and validated by the model; anything missing from its inputSchema returns 400. For example, only some models list seed in their inputSchema; on the others, Chat's seed and Gemini's generationConfig.seed return 400. Sampling settings and stop sequences are the exception: on a model that does not take them they are ignored (named in the X-Spicy-Ignored-Params response header), and a value outside what the model takes is changed to the nearest accepted one (named in X-Spicy-Adjusted-Params). That covers temperature, top_p and the penalties, Chat's stop, Messages' stop_sequences, Gemini's generationConfig.stopSequences, and reasoning_effort levels. Read the schema of the model you chose rather than assuming a field from another model.
A few exceptions:
- Fields that only mean something to the protocol's own platform are ignored and do not affect generation:
user,metadataandstorein Chat;user,metadata,storeandincludein Responses;metadatain Messages. - Responses does not support
previous_response_id; send the full history ininput. - Gemini's
safetySettingsis accepted but has no effect.cachedContentand server-side tools such asgoogleSearchandcodeExecutionare not supported and return400. Pass images asinlineData(base64) orfileData.fileUri(an https URL); whether a model accepts images is defined by itsinputSchema.
Send a Chat stream
With SPICY_API_KEY and SPICY_MODEL set as above, this writes the request to a file and streams the answer. Replace the sample prompt as needed, and validate model parameters against the current schema before sending. Each run sends a fresh Idempotency-Key, because each run is a new message.
jq -n --arg model "$SPICY_MODEL" '{
model: $model,
messages: [{role: "user", content: "Explain a rainbow in one sentence."}],
stream: true
}' > chat-request.json
curl --fail-with-body --no-buffer --max-time 120 \
https://api.spicyapi.ai/v1/chat/completions \
-H "Authorization: Bearer $SPICY_API_KEY" \
-H "Content-Type: application/json" \
-H "Idempotency-Key: $(uuidgen)" \
--data-binary @chat-request.json--no-buffer prints each piece as soon as it arrives, and --max-time 120 gives up after two minutes. You should see lines like the ones below arriving one by one, and finally data: [DONE]. The answer is spread across the delta.content pieces; join them in order to get the full text. Streams follow OpenAI's usage rules: without stream_options.include_usage, no chunk carries usage and every chunk has a non-empty choices list. Send "stream_options": {"include_usage": true} to receive the token counts: one extra chunk with an empty choices list and the final usage then arrives just before [DONE], and every other chunk has usage: null. Skip choices on that last chunk (the Python and Node.js examples under Timeouts and streaming check choices first, so they work either way). Each choice ends with exactly one chunk that carries its finish_reason.
data: {"id":"chatcmpl-req_01k...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"A rainbow"}}]}
data: {"id":"chatcmpl-req_01k...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":" forms when"}}]}
data: [DONE]uuidgen ships with macOS; on Linux it comes with the uuid-runtime package.
Send a Gemini request
This uses the same SPICY_API_KEY and SPICY_MODEL, and again a new Idempotency-Key: it is a different request from the Chat example, so reusing that key would return 409. The model ID goes in the URL, so the request body carries no model field. Wrap the URL in double quotes and write the variable as ${SPICY_MODEL}, so the shell does not read the colon that follows it as a variable modifier.
jq -n '{
systemInstruction: {parts: [{text: "Answer in one sentence."}]},
contents: [{role: "user", parts: [{text: "Explain a rainbow."}]}],
generationConfig: {temperature: 0.7, maxOutputTokens: 256}
}' > gemini-request.json
curl --fail-with-body --no-buffer --max-time 120 \
"https://api.spicyapi.ai/v1beta/models/${SPICY_MODEL}:streamGenerateContent?alt=sse" \
-H "x-goog-api-key: $SPICY_API_KEY" \
-H "Content-Type: application/json" \
-H "Idempotency-Key: $(uuidgen)" \
--data-binary @gemini-request.jsonIf you do not need streaming, replace :streamGenerateContent?alt=sse with :generateContent and read the answer from candidates[0].content.parts[].text; usage is in usageMetadata. Without alt=sse, a streaming request returns a single JSON array written out incrementally; the official SDKs add alt=sse for you.
Native stream with a confirmed price
POST /api/v1/jobs/stream is the one text endpoint that accepts a quote, so it can refuse to run if the price changed. The request body has the same shape as createTask: model plus an input whose fields come from the model's inputSchema (for text models, messages and the other canonical parameters in the table above).
# 1. Write the request once.
jq -n --arg model "$SPICY_MODEL" '{
model: $model,
input: {messages: [{role: "user", content: "Explain a rainbow in one sentence."}]}
}' > native-request.json
# 2. Quote it. This creates nothing and charges nothing.
QUOTE=$(curl -s https://api.spicyapi.ai/api/v1/jobs/quote \
-H "Authorization: Bearer $SPICY_API_KEY" \
-H "Content-Type: application/json" \
--data-binary @native-request.json)
echo "$QUOTE" | jq '{code, msg, estimatedCost: .data.estimatedCost, maxCharge: .data.maxCharge, expiresAt: .data.expiresAt}'
# 3. Within five minutes, add quoteId and expectedCost to the unchanged request and stream it.
jq --argjson quote "$QUOTE" \
'. + {quoteId: $quote.data.quoteId, expectedCost: $quote.data.estimatedCost}' \
native-request.json > native-confirmed.json
curl --fail-with-body --no-buffer --max-time 120 \
https://api.spicyapi.ai/api/v1/jobs/stream \
-H "Authorization: Bearer $SPICY_API_KEY" \
-H "Content-Type: application/json" \
-H "Idempotency-Key: $(uuidgen)" \
--data-binary @native-confirmed.jsonStep 2 prints code: 200 and the price in US dollars; any other code means the quote failed, and msg says why. If the stream call fails before any text is sent โ for example with 40901 because the price or the request changed, or 40201 for too little balance โ you get the normal {code,msg,data,request_id} JSON and nothing is held. Once streaming has started, a failure ends the stream with event: error and data: {"code":"stream_interrupted"}, and nothing is charged for it. Image, video and audio models return 503 here; use createTask for them.
The answer arrives as SSE data: lines. Unlike /v1/chat/completions, this endpoint does not promise one event format for every model, so test your parser with the model you chose; if you need a single stable format, use a /v1 endpoint and bound spending with the key's caps.
Send images, video and audio to a text model
Some text models read images as well as text, and a few also read video and audio. You send the media with a question, and the answer is still text. What a model accepts is written in its schema, not implied by the protocol. Check the model you chose:
curl -s "https://api.spicyapi.ai/api/v1/models/$SPICY_MODEL" \
-H "Authorization: Bearer $SPICY_API_KEY" \
| jq -c '[.data.inputSchema.properties.messages.items.properties.content.items.oneOf[]?.properties.type.const]'The list names the content parts the model takes. ["text","image_url"] means text and images; a model that also reads video and audio lists video_url, audio_url and input_audio too. An empty list means text only. Sending a part the model does not list returns 400 and charges nothing.
To send media, make the message content a list of parts instead of a string โ a text part for the question and one part per file:
jq -n --arg model "$SPICY_MODEL" '{
model: $model,
stream: true,
messages: [{
role: "user",
content: [
{type: "text", text: "What is in this picture?"},
{type: "image_url", image_url: {url: "https://example.com/photo.jpg"}}
]
}]
}' > image-request.json
curl --fail-with-body --no-buffer --max-time 120 \
https://api.spicyapi.ai/v1/chat/completions \
-H "Authorization: Bearer $SPICY_API_KEY" \
-H "Content-Type: application/json" \
-H "Idempotency-Key: $(uuidgen)" \
--data-binary @image-request.jsonEach kind of media has its own part:
| Part | Shape | Where the file comes from |
|---|---|---|
| Image | {"type": "image_url", "image_url": {"url": "โฆ"}} | A public HTTPS URL, an uploaded file (spicy://f/โฆ), or a small data: URI |
| Video | {"type": "video_url", "video_url": {"url": "โฆ"}} | An uploaded file (spicy://f/โฆ), or a public HTTPS URL |
| Audio file | {"type": "audio_url", "audio_url": {"url": "spicy://f/โฆ"}} | An uploaded file only |
| Inline audio | {"type": "input_audio", "input_audio": {"data": "<base64>", "format": "wav"}} | Base64 in the request, wav or mp3 |
How to upload a file and get its spicy://f/โฆ URI is described in Media; uploads last one day. A few rules follow from how each kind reaches the model:
- Images can be a public HTTPS URL that returns the image file itself (not a page that shows it), an uploaded file, or a
data:URI within the inline limits. JPEG and PNG work on every model that accepts images. - Video and audio work best as uploads. We read the uploaded file and hand it to the model inside the request, so the uploaded video and audio in one request can total at most 14 MB โ MP4 or WebM video, MP3 or WAV audio. A larger total returns
400and charges nothing. A public video URL is passed on as a link instead, and works only if the model's provider can download it. - Inline audio (
input_audio) follows the OpenAI format and has to fit in the 2 MiB request body, so it suits short clips; upload anything longer and send it asaudio_url.
Several files can go in one message, each as its own part, and earlier messages in the history may carry media too.
The other protocols carry images in their own shape, and all of them reach the model identically. Video and audio parts are sent through Chat Completions and the native stream.
| Protocol | Image part |
|---|---|
| Chat Completions | {"type": "image_url", "image_url": {"url": "โฆ"}} |
| Responses | {"type": "input_image", "image_url": "โฆ"} |
| Anthropic Messages | {"type": "image", "source": {"type": "url", "url": "โฆ"}}, or a base64 source |
| Gemini | inlineData (base64) or fileData.fileUri (an HTTPS URL) |
What it costs. Media is counted as input tokens at the model's normal input rate โ there is no separate per-file price. An image typically adds a few hundred to a few thousand tokens depending on the model and the image size; video and audio grow with their length. The usage block of the response reports the exact number.
What is held up front. Before a request runs we hold an upper bound and release the difference when it settles. For a request whose only media are images you uploaded or sent inline, the bound adds a fixed allowance per image. For a request that carries video or audio, or an image given as a public URL, the bound is sized to the model's whole input limit, because we cannot know its size in advance โ so the held amount can be far above the final charge. Upload images to keep the hold small, keep enough balance for it, or quote the request first with POST /api/v1/jobs/quote to see it.
In the Playground. On a model that accepts media, the message box shows an attach button. You can also paste a screenshot or drop files onto the box; each message takes up to four attachments, and the video and audio among them can total 14 MB. Models that read text only show no button.
Conversations and tool calls
Complete tool-calling programs in Python, Node.js and curl are in Chat recipes.
Chat Completions
- Non-streaming: read
choices[0].message. - Streaming: process
choices[].deltaone SSE event at a time, not per network packet. Assembletool_callsbyindex, and validate and execute a call only once itsargumentsare complete. - Keep the assistant message in the history, and return each result in a
toolmessage with the matchingtool_call_id.
Responses
- Send the full history in
input;previous_response_idis rejected. - Return function results as
function_call_outputitems with the matchingcall_id.
Messages
- Use
text,tool_useandtool_resultcontent blocks rather than the Chat message shape.
Gemini
contentsalternate between two roles,userandmodel.- When the model wants to call a tool, it returns a
functionCallpart. After running the tool, wrap the result in afunctionResponsepart and send it back in the nextusercontent, using the samenameas the call.
Completion, errors and disconnects
| Protocol | Normal completion | Usage |
|---|---|---|
| Chat Completions | data: [DONE] | Only with stream_options.include_usage: a chunk with empty choices and usage, just before [DONE] |
| Responses | response.completed event | response.usage in that event |
| Messages | message_stop after message_delta | usage in message_delta |
| Gemini | Last event with finishReason; there is no [DONE] | usageMetadata |
Check the HTTP status and Content-Type before reading a stream; errors come back as each protocol's own JSON:
| Protocol | Error body |
|---|---|
| Chat Completions, Responses | {"error": {"message", "type", "param", "code"}} |
| Messages | {"type": "error", "error": {"type", "message", "code"}} |
| Gemini | {"error": {"code", "message", "status", "details"}} |
HTTP status codes are the same across all four protocols. Gemini's status uses Google's status names: 400 is INVALID_ARGUMENT, 401 is UNAUTHENTICATED, 402 is FAILED_PRECONDITION, 403 is PERMISSION_DENIED, 404 is NOT_FOUND, 409 is ABORTED, 429 is RESOURCE_EXHAUSTED, 500 is INTERNAL, 503 is UNAVAILABLE and 504 is DEADLINE_EXCEEDED. Branch your code on the status code and the stable reason code, never on the message text.
The reason code tells apart errors that share a status. It is error.code in the OpenAI shape, the extra error.code field in the Anthropic shape (the official Anthropic SDKs ignore it, so read it from the raw body), and the reason of the google.rpc.ErrorInfo entry in Gemini's details, upper-cased (for example CONTENT_REJECTED). The ones to know:
| HTTP | Code | Meaning |
|---|---|---|
| 400 | unsupported_parameter | The model does not accept this field, or this kind of message part (for example an image_url part on a text-only model). Remove it |
| 400 | invalid_request | A field is missing or has a value the model does not accept, or the model is not a chat model. The message says what to change |
| 400 | content_rejected | The model's content policy refused the request. Sending the same input again is refused again |
| 404 | model_not_found | No model has this ID. The message shows the ID shape and, when there is one, the closest callable chat model |
When the error is about one field, the OpenAI shape names it in param as a dot path, for example temperature, messages.1.content.0 or model.
If something fails after streaming begins, Chat and Gemini send an event carrying an error object, Responses sends an error event and Messages sends event: error. Your client must handle these error events, as well as timeouts and a connection that closes without the normal completion marker. If you run the stream through your own proxy, turn off response buffering there, or the pieces arrive in bursts.
What you pay when a stream stops early
- You disconnect (close the tab, abort the request, hit a client timeout): disconnecting does not stop the bill. You are charged for the usage the model reports, which for many models covers the whole answer โ including text you never received โ and never more than the amount held when the request was accepted. Text you did not receive cannot be fetched later. To spend less, lower
max_tokensrather than disconnecting. - The model or our service fails mid-stream (you receive an error event, or the stream ends without its completion marker): the request is marked failed and nothing is charged.
- The connection to the model breaks only after the answer is complete (every choice already has its
finish_reasonand the final usage has arrived): nothing is missing from your answer, so the stream ends normally with its completion marker and the request is charged for the usage delivered, like any completed request. - The model sent you text or reasoning, then its answer was judged empty or refused: what you already received is billed by the usage the model reports, never more than the amount held, and the stream ends normally. Only when nothing at all reached you is it refunded in full.
- The request runs past its server deadline: it ends as a failure and nothing is charged.
Timeouts and streaming
Without streaming, the answer arrives in one piece once it is ready, and long answers and long reasoning take time โ models such as Grok, GLM, DeepSeek Pro and Claude Opus can think for minutes before they write anything, especially at a high reasoning_effort. The network edge in front of the API closes a connection on which nothing has arrived for about 100 seconds, so requests of both kinds are kept open while the model works.
- A non-streamed request still running after about 30 seconds starts its response early. You receive status
200,Content-Type: application/jsonand the headerX-Spicy-Keepalive: 1, then a newline every 15 seconds, then the JSON body. JSON allows whitespace before a value, so standard parsers (json.loads,JSON.parse,response.json()) and the official SDKs read the body unchanged. A request that finishes sooner gets a normal response with its real status code and without the header. - After that point, a failure is in the body, not in the status. The
200has already been sent, so the body holds the protocol's usual error object from the table above, with the same code and message; in the Gemini format,error.codestill carries the HTTP status the error would have had. WhenX-Spicy-Keepaliveis present, check the body for anerrorobject before reading the answer โ the official SDKs do not treat a200as an error. A failed request is not charged. - Streaming keeps the connection open too. While the model is still working, a streamed response carries keep-alive lines โ
: keep-alivecomments over SSE, blank lines in the Gemini format โ so the connection is never idle, however long the model thinks. Your parser must skip them, as the SSE format requires; the official SDKs already do. - Prefer
stream: truefor anything that may run long. You see the answer as it is written, and a failure arrives as an error event that the SDKs understand. Chat Completions, Responses and Messages takestream: true; in the Gemini format, call:streamGenerateContent?alt=sseinstead of:generateContent. A changed body is a new request, so send it with a newIdempotency-Key: reusing the old one returns409. - A request cut off by the network edge is not charged. If nothing at all reaches you for about 100 seconds and the connection closes, the request appears under Console โ Logs and in
GET /api/v1/jobsas failed, witherrorCodetimeoutand a message that says what happened. Resending it with the sameIdempotency-Keyreturns that message as the error. - Your own time limits still apply. A total timeout in your HTTP client, or in a proxy you run, ends a request too, and once anything has reached you โ a keep-alive line included โ that counts as you disconnecting (see What you pay when a stream stops early). The official OpenAI SDKs wait up to 10 minutes by default; the curl examples on this page give up after 2 minutes, so raise
--max-timefor long reasoning.
Turning streaming on in the OpenAI SDKs and in curl:
import os
from openai import OpenAI
client = OpenAI(api_key=os.environ["SPICY_API_KEY"], base_url="https://api.spicyapi.ai/v1")
stream = client.chat.completions.create(
model=os.environ["SPICY_MODEL"],
messages=[{"role": "user", "content": "Explain a rainbow in one sentence."}],
stream=True,
)
for chunk in stream:
if chunk.choices and chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="", flush=True)
print()import OpenAI from "openai";
const client = new OpenAI({ apiKey: process.env.SPICY_API_KEY, baseURL: "https://api.spicyapi.ai/v1" });
const stream = await client.chat.completions.create({
model: process.env.SPICY_MODEL,
messages: [{ role: "user", content: "Explain a rainbow in one sentence." }],
stream: true,
});
for await (const chunk of stream) {
process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}
process.stdout.write("\n");curl --fail-with-body --no-buffer --max-time 900 \
https://api.spicyapi.ai/v1/chat/completions \
-H "Authorization: Bearer $SPICY_API_KEY" \
-H "Content-Type: application/json" \
-H "Idempotency-Key: $(uuidgen)" \
--data-binary @chat-request.jsonThe Node.js example uses top-level await, so run it as an ES module (for example, save it as stream.mjs). The curl command sends chat-request.json from Send a Chat stream, which already sets stream: true; --max-time 900 lets the answer take up to 15 minutes.
Quotes and billing
For explicit cost confirmation, use the native stream with a quote. /v1 and /v1beta do not accept quote credentials; they apply the price at acceptance.
Every text request that is accepted runs as a task, and the response names it in the X-Spicy-Task-Id header (job_โฆ), streamed or not, on all four protocols and on /api/v1/jobs/stream. The header is also present on an error returned after acceptance, whose task shows the refund; a request rejected before acceptance, such as one that fails parameter validation, creates no task and has no header. An end-of-text marker is not proof of the final charge: read the task with GET /api/v1/jobs/recordInfo?taskId=<job_โฆ> and treat cost as final only when settled is true. Items in GET /api/v1/jobs also carry the requestId of the call that created them (the X-Request-Id response header, which is also the suffix of the chatcmpl-req_โฆ ID), so you can match either way; Console โ Logs shows the same tasks. See Task history and recovery and Billing.
reasoning_tokens is already included in completion_tokens: do not add it twice.
Other request shapes
These examples describe protocol envelopes only. Replace MODEL_ID_FROM_CATALOG with a live model that supports the fields shown; never send the placeholder itself.
{
"model": "MODEL_ID_FROM_CATALOG",
"input": "Explain a rainbow in one sentence.",
"stream": false
}{
"model": "MODEL_ID_FROM_CATALOG",
"max_tokens": 256,
"messages": [{"role": "user", "content": "Explain a rainbow in one sentence."}],
"stream": false
}Continue reading
Chat recipes ยท Migrate from OpenAI or OpenRouter ยท Quotes and protocol compatibility ยท Idempotency ยท Billing ยท Production integration ยท Troubleshooting and recovery

