Chat recipes

Each recipe on this page is a whole program: set two environment variables, copy it, run it. They all call POST https://api.spicyapi.ai/v1/chat/completions with the official OpenAI libraries, so the same code works with every text model in the catalog. Text and streaming explains the protocol itself; this page shows how to build the usual things on top of it.

Before you start

1. Put your API key in an environment variable, so it never has to be pasted into code:

export SPICY_API_KEY="sk-spicy-paste-your-key-here"

2. Choose a text model from the live catalog and keep its ID in SPICY_MODEL. Every recipe reads it. This lists the text models your key can call right now (it needs jq):

curl -s "https://api.spicyapi.ai/api/v1/models?modality=text" \
  -H "Authorization: Bearer $SPICY_API_KEY" \
  | jq -r '.data.items[] | select(.enabled and .available) | .model'

export SPICY_MODEL="MODEL_ID_FROM_CATALOG"   # replace with one line from the list above

Text model IDs have the form publisher/model/chat. MODEL_ID_FROM_CATALOG is only a placeholder: never send it literally.

3. Check what that model accepts. Models differ: some take temperature, some do not; reasoning_effort has different values on different models; only some accept images. Unsupported sampling fields such as temperature or frequency_penalty are ignored and named in the X-Spicy-Ignored-Params response header. Common client defaults are handled too: unsupported stop is ignored, an unsupported n: 1 means one reply, and a recognized reasoning level is mapped to the nearest supported level when necessary. The X-Spicy-Adjusted-Params header shows adjusted settings. Unsupported functional fields such as tools or response_format still produce 400. This prints the fields, the reasoning_effort values and the kinds of message content the model takes:

curl -s "https://api.spicyapi.ai/api/v1/models/$SPICY_MODEL" \
  -H "Authorization: Bearer $SPICY_API_KEY" \
  | jq '.data.inputSchema.properties | {
      fields: keys,
      reasoning_effort: .reasoning_effort.enum,
      content_parts: [.messages.items.properties.content.items.oneOf[]?.properties.type.const]
    }'

For a quick overview of every text model at once, GET /v1/models?modality=text returns each model's supported_parameters, context_length and max_output_tokens; it lists field names only, so use the command above when you need the allowed values.

4. Install the OpenAI library for your language: pip install openai for Python, or npm install openai for Node.js 18 or newer. Save the Node.js programs with an .mjs extension (they use top-level await) and run them with node file.mjs.

Multi-turn conversation

The API keeps no memory between calls. A conversation is a list of messages that you keep and resend in full on every turn: the system message first, then alternating user and assistant messages. After each reply, append it to the list as an assistant message.

Every message you resend is billed again as input, so trim the oldest turns once the conversation gets long. Always keep the system message, and remove turns in user/assistant pairs so the history still starts with a user message.

import os
from openai import OpenAI

client = OpenAI(api_key=os.environ["SPICY_API_KEY"], base_url="https://api.spicyapi.ai/v1")
MODEL = os.environ["SPICY_MODEL"]
KEEP = 20  # messages kept after the system prompt; an even number keeps user/assistant pairs together

history = [{"role": "system", "content": "You are a patient cooking coach. Keep every answer under 80 words."}]

while True:
    try:
        user_text = input("you> ").strip()
    except EOFError:  # Ctrl-D
        break
    if user_text in ("", "quit", "exit"):
        break
    history.append({"role": "user", "content": user_text})
    reply = client.chat.completions.create(model=MODEL, messages=history)
    answer = reply.choices[0].message.content or ""
    print(f"coach> {answer}\n")
    history.append({"role": "assistant", "content": answer})
    # The API remembers nothing between calls: resend the conversation every turn,
    # and trim it so the prompt (and what you pay for it) stops growing.
    history = history[:1] + history[1:][-KEEP:]

The model answers the last message using everything before it: ask it afterwards what ingredients you mentioned, and it knows. Reasoning models also return reasoning_content next to content; for a plain conversation you do not need to send it back.

Role-play with a character card

A character is a system message written with care. Because the API keeps no memory, the character card travels with every request, which is also what keeps the character consistent on turn fifty. A card that works well answers six questions:

Two habits make role-play feel right. Put the character's opening line into the history as an assistant message, so the first thing the user sees is already in character. And end every reply with a question or a choice, so the user always knows how to continue; for interactive fiction, numbered choices let the user answer with a single digit.

import os
from openai import OpenAI

client = OpenAI(api_key=os.environ["SPICY_API_KEY"], base_url="https://api.spicyapi.ai/v1")
MODEL = os.environ["SPICY_MODEL"]

CARD = {
    "name": "Captain Ilsa Varn",
    "identity": "Navigator of the airship Kestrel. Forty years old, calm under pressure, dry sense of humour, fiercely loyal to her crew.",
    "voice": "Short, vivid sentences full of wind and weather. Calls the user 'deckhand'. No modern slang.",
    "world": "Islands drift on warm updrafts above an endless sea of cloud. Storms here are alive and hungry, and maps go out of date within a week.",
    "boundaries": (
        "Stay in character and inside this world. If the user asks about the real world, react as Ilsa would: puzzled but polite. "
        "If the user writes (OOC: ...), step out of the story for one short, plain answer, then return. "
        "Keep the story suitable for a general audience."
    ),
    "narration": "Describe the scene in the second person ('You feel the deck tilt...'). Put Ilsa's spoken lines in quotation marks.",
    "format": "At most 120 words. End every reply with a question or a choice, so the user always knows how to continue.",
}

system_prompt = "\n".join(
    [f"You are {CARD['name']}, a character in an interactive story."]
    + [f"{key.capitalize()}: {value}" for key, value in CARD.items() if key != "name"]
)
history = [
    {"role": "system", "content": system_prompt},
    {"role": "assistant", "content": "\"Welcome aboard, deckhand. Storm wall to the north, a lost island to the east. Which way do we steer?\""},
]
print(history[-1]["content"])

while True:
    try:
        user_text = input("\nyou> ").strip()
    except EOFError:  # Ctrl-D
        break
    if user_text in ("", "quit", "exit"):
        break
    history.append({"role": "user", "content": user_text})
    answer = ""
    stream = client.chat.completions.create(model=MODEL, messages=history, stream=True)
    for chunk in stream:
        if chunk.choices and chunk.choices[0].delta.content:
            piece = chunk.choices[0].delta.content
            answer += piece
            print(piece, end="", flush=True)
    print()
    history.append({"role": "assistant", "content": answer})

The same six parts work for any genre. A science-fiction ship's computer, a companion that helps with a daily routine, or a narrator for a text adventure only need a different card:

You are Juniper, a friendly running coach inside a fitness app.
Identity: a former trail runner, upbeat and practical, never judgemental.
Voice: warm and brief; uses the user's name when they give it; celebrates small wins.
World: the user's own week. Remember goals and aches they mention earlier in this conversation.
Boundaries: give general fitness encouragement only; for pain, injury or medical questions, suggest seeing a professional. Stay in character; answer (OOC: ...) plainly.
Narration: first person, present tense, talking directly to the user.
Format: at most 80 words; end with one concrete suggestion for today.

For long stories, keep the card short and stable, and when the history grows, replace the oldest turns with a brief "Story so far" note in a second system message rather than letting the prompt grow without limit.

Stream into your own interface

Streaming shows the reply as it is written, which is what makes a chat interface feel responsive. Six rules cover every model in the catalog:

1. Ask for usage if you want token counts. Streams follow OpenAI's rules: without stream_options.include_usage, no chunk carries usage. With "include_usage": true, one extra chunk arrives just before [DONE] with the final usage and an empty choices list, and every other chunk has usage: null.
2. Check choices before reading choices[0]. Every chunk except that final usage chunk has at least one choice, so this matters only when you ask for usage, but the check costs nothing and keeps the loop safe either way.
3. Thinking is separate. Reasoning models stream their thinking in delta.reasoning_content and the answer in delta.content. Show the first dimmed, collapsed or not at all; only the second is the reply.
4. finish_reason: "length" means cut off. The model reached max_tokens. Raise it, or lower reasoning_effort.
5. Errors can arrive mid-stream. A failure after streaming has started arrives as an event with an error object, and the OpenAI libraries raise it as an exception. Such a failed request is not charged. If the model had already finished the answer when its connection broke, you get no error at all: the stream ends normally and the answer is charged like any other. Details: Completion, errors and disconnects.
6. Stopping early does not stop the bill. Closing the connection does not stop the model; you are charged for what it reports. Lower max_tokens instead (What you pay when a stream stops early).

import os
from openai import OpenAI

client = OpenAI(api_key=os.environ["SPICY_API_KEY"], base_url="https://api.spicyapi.ai/v1")

stream = client.chat.completions.create(
    model=os.environ["SPICY_MODEL"],
    messages=[{"role": "user", "content": "Write a four-line poem about a cat who lives in a lighthouse."}],
    stream=True,
    stream_options={"include_usage": True},
)

finish_reason = None
usage = None
thinking_shown = False
for chunk in stream:
    # With include_usage, only the last chunk before [DONE] carries usage; it has no choices.
    if chunk.usage:
        usage = chunk.usage
    if not chunk.choices:
        continue
    choice = chunk.choices[0]
    thinking = getattr(choice.delta, "reasoning_content", None)  # reasoning models only
    if thinking:
        thinking_shown = True
        print(f"\033[2m{thinking}\033[0m", end="", flush=True)  # dimmed
    if choice.delta.content:
        if thinking_shown:
            print("\n")
            thinking_shown = False
        print(choice.delta.content, end="", flush=True)
    if choice.finish_reason:
        finish_reason = choice.finish_reason
print()

if finish_reason == "length":
    print("[stopped at max_tokens: raise it, or lower reasoning_effort]")
if usage:
    print(f"[{usage.prompt_tokens} prompt tokens, {usage.completion_tokens} completion tokens]")

Tool calling

Tools let the model ask your program to do something (roll dice, look up an order, read a calendar) and then answer with the result. The loop is always the same:

1. Send the messages together with tools, a list of functions described with JSON Schema.
2. If the reply has tool_calls, append that assistant message exactly as it came back, run each call yourself, and append one tool message per call with the matching tool_call_id.
3. Send everything again. Repeat until a reply has no tool_calls: that reply is the answer.

Treat the arguments as untrusted input: parse them, check them, and send an error back as the tool result rather than crashing. Cap the number of rounds so a confused model cannot loop forever. Every text model in the catalog currently accepts tools; check the model's tool_choice values before forcing a particular tool, because some models accept only auto and none.

import json
import os
import random
from openai import OpenAI

client = OpenAI(api_key=os.environ["SPICY_API_KEY"], base_url="https://api.spicyapi.ai/v1")
MODEL = os.environ["SPICY_MODEL"]


def roll_dice(count, sides):
    rolls = [random.randint(1, sides) for _ in range(count)]
    return {"rolls": rolls, "total": sum(rolls)}


LOCAL_TOOLS = {"roll_dice": roll_dice}
TOOLS = [{
    "type": "function",
    "function": {
        "name": "roll_dice",
        "description": "Roll dice for a tabletop game and return every roll and the total.",
        "parameters": {
            "type": "object",
            "properties": {
                "count": {"type": "integer", "minimum": 1, "maximum": 10, "description": "How many dice to roll"},
                "sides": {"type": "integer", "enum": [4, 6, 8, 10, 12, 20], "description": "Sides on each die"},
            },
            "required": ["count", "sides"],
        },
    },
}]

messages = [
    {"role": "system", "content": "You are the game master of a fantasy adventure. Whenever the rules call for a roll, use roll_dice; never invent a result. A roll of 12 or more on a d20 succeeds."},
    {"role": "user", "content": "I try to pick the lock on the old tower door. Roll a d20 for me."},
]

for _ in range(5):  # a hard stop, so a confused model cannot loop forever
    reply = client.chat.completions.create(model=MODEL, messages=messages, tools=TOOLS)
    message = reply.choices[0].message
    # Send the assistant turn back exactly as it came, tool calls included.
    messages.append(message.model_dump(exclude_none=True))
    if not message.tool_calls:
        print(message.content)
        break
    for call in message.tool_calls:
        tool = LOCAL_TOOLS.get(call.function.name)
        try:
            result = tool(**json.loads(call.function.arguments or "{}")) if tool else {"error": "unknown tool"}
        except (TypeError, ValueError) as error:  # the model sent arguments the tool cannot use
            result = {"error": str(error)}
        print(f"[tool] {call.function.name}({call.function.arguments}) -> {result}")
        messages.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(result)})

The first reply ends with finish_reason: "tool_calls" and content set to null; the final one ends with "stop". Reasoning models may also include reasoning_content in the assistant turn: keep it when you send the turn back, as the examples do by appending the message unchanged. When you stream a tool-calling reply, the call arrives in pieces; join them by index as described in Conversations and tool calls.

JSON output

When your program needs data rather than prose, ask for JSON with response_format, on models whose inputSchema lists it, and describe the fields in the prompt. Always parse the reply defensively: a reply cut off by max_tokens is not valid JSON.

import json
import os
from openai import OpenAI

client = OpenAI(api_key=os.environ["SPICY_API_KEY"], base_url="https://api.spicyapi.ai/v1")
reply = client.chat.completions.create(
    model=os.environ["SPICY_MODEL"],
    messages=[
        {"role": "system", "content": "Reply with a JSON object only, with the keys title (string), genre (string) and hook (one sentence)."},
        {"role": "user", "content": "Invent a cosy mystery novel set in a mountain bakery."},
    ],
    response_format={"type": "json_object"},
)
try:
    book = json.loads(reply.choices[0].message.content)
    print(book["title"], "-", book["hook"])
except (json.JSONDecodeError, KeyError, TypeError):
    print("Not the JSON we asked for:", reply.choices[0].message.content)

On a model that does not list response_format, leave the field out and rely on the system prompt alone; the try block above matters even more there.

When something goes wrong

Errors come back in the OpenAI shape, {"error": {"message", "type", "param", "code"}}, so the OpenAI libraries raise their usual exceptions. Branch on the HTTP status and error.code; show error.message to people, because it says what to change. When the error is about one field, param names it as a dot path, for example temperature or messages.1.content.0.

Why a request can fail with 402 while your balance looks fine: before a text request runs, an amount is held that covers your messages plus the full max_tokens of output at the model's prices (the model's default max_tokens when you leave it out). You pay only for the tokens actually used, and the rest of the hold is released when the request finishes, but the whole hold must fit into your available balance before the request starts. Setting a realistic max_tokens lowers the hold. See Billing.

Find what a call cost

Every request that is accepted runs as a task, and the response names it in the X-Spicy-Task-Id header, streamed or not. Read the header, then look the task up: cost is final once settled is true. A request rejected before acceptance (a validation error, for example) creates no task and has no header.

import os
from openai import OpenAI

client = OpenAI(api_key=os.environ["SPICY_API_KEY"], base_url="https://api.spicyapi.ai/v1")
raw = client.chat.completions.with_raw_response.create(
    model=os.environ["SPICY_MODEL"],
    messages=[{"role": "user", "content": "Say hello in one sentence."}],
)
reply = raw.parse()  # the usual ChatCompletion object
print(reply.choices[0].message.content)
print("task:", raw.headers.get("x-spicy-task-id"))

With a stream, the header is on the response before the first chunk; with_raw_response in Python and .withResponse() in Node.js work the same way with stream: true. Items in GET /api/v1/jobs also carry the requestId of the call that created them, which is the X-Request-Id response header.

Continue reading

Text and streaming · Migrate from OpenAI or OpenRouter · Model catalog and schemas · Billing · Errors