AI inference

OpenAI-compatible inference

Use one OpenAI-compatible base URL for your application code and your coding tools. Chat completions, embeddings, images and video all bill from the organization credit you already use in Maxlayer.

Get started #

Three values configure every client on this page, and the rest of the guide is detail about them. If you take nothing else:

Base URL: https://inference.maxlayer.cloud/v1 — the same for every client and every route. Keep the /v1; the library appends what it needs beneath it. If a tool asks for a “host” rather than a base URL, it usually wants this without the /v1.
Organization API key: Created under Settings → API keys, shown once, and stored by you as MAXLAYER_API_KEY. It identifies which organization is calling and which balance pays. The browser session that signs you into the dashboard is not one of these and cannot spend credit — that is deliberate, so a stolen session cannot run up a bill.
Model id: Copied from the catalogue exactly as written, e.g. anthropic/claude-sonnet-5. Ids contain a slash, and some carry a :variant suffix that is part of the id rather than decoration. Switching model means changing this string and nothing else.

Four routes are served beneath that base URL, and each takes a model listed for it: /chat/completions for conversation, /embeddings for vectors, /images for pictures, and /videos for clips. Calling a route with a model that belongs to another one returns a clear error rather than a strange answer.

OpenAI SDK quickstart #

Install the openai package, create its client with your organization key and Maxlayer base URL, then use the same chat.completions.create call you would use against OpenAI.

openai.ts

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.MAXLAYER_API_KEY,
  baseURL: "https://inference.maxlayer.cloud/v1",
});

const completion = await client.chat.completions.create({
  model: process.env.MAXLAYER_MODEL_ID,
  messages: [{ role: "user", content: "Explain this pull request." }],
});

console.log(completion.choices[0]?.message.content);

The full SDK guide covers timeouts and retries, aborting a stream and what it costs, tool calling, the typed exception classes, and the short list of things that differ from calling OpenAI directly.

HTTP and Python #

The endpoint accepts the usual bearer token and JSON request body. The Python SDK uses base_url in the same way the TypeScript SDK uses baseURL.

curl #

bash

curl https://inference.maxlayer.cloud/v1/chat/completions \
  -H "Authorization: Bearer $MAXLAYER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "$MAXLAYER_MODEL_ID",
    "messages": [{"role": "user", "content": "Explain this pull request."}]
  }'

Python #

python

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["MAXLAYER_API_KEY"],
    base_url="https://inference.maxlayer.cloud/v1",
)

completion = client.chat.completions.create(
    model=os.environ["MAXLAYER_MODEL_ID"],
    messages=[{"role": "user", "content": "Explain this pull request."}],
)

print(completion.choices[0].message.content)

Models and prices #

GET /v1/models is public, so an OpenAI-compatible tool can fill its model picker before a key is configured. It returns the models listed by Maxlayer along with their capabilities and what each one charges.

public model catalogue

curl https://inference.maxlayer.cloud/v1/models

Two bases appear there, and billing_mode says which a model uses. token is a rate card — the pricing object carries an input and an output price per million tokens, and a request costs what its tokens come to. upstream_cost means there is no single rate card: image and video models are priced per image, per second or per megapixel, and a multimodal embedding model charges a different per-token rate for text, image and audio input. Either way a request is charged what it cost to serve plus our markup. Those models publish their provider’s own prices in price_lines and carry zeros throughout pricing, which is the one field a client must not read as free.

The catalogue is the source of truth for availability and price. A model can be listed, repriced, or removed without changing this documentation, so check it before pinning a model id in an integration.

Browse the model marketplace to compare the current catalogue by task, provider, capability, and price. Zero data retention is stated on each model that offers it rather than something to filter on.

Chat completions #

Send POST /v1/chat/completions with a listed chat model and at least one message. Maxlayer accepts OpenAI request fields it does not need to interpret and passes them to the selected model, so settings such as temperature and tools are not narrowed by this gateway.

A normal response has OpenAI's completion shape. Requests against an embedding model are rejected instead of being silently routed to a chat model.

What passes through #

Anything this gateway does not need to interpret reaches the model unchanged, which means the OpenAI fields you already use keep working without being listed here one by one: temperature, top_p, max_tokens, stop, seed, tools and tool_choice, response_format for JSON and structured output, and multimodal content arrays for models that accept images.

tools is the one that is more than a field. A model that wants a tool run replies asking for it, and your code has to execute it and come back — a loop rather than a request. Tool calling has the whole cycle, parallel calls, streaming, and what a loop costs.

Sending an image #

Models with the vision capability take the same multimodal content array OpenAI uses — a text part and an image_url part, where the URL may be a link or a data: URI. Images are charged as input tokens on a rate-carded model; how many depends on the image, which is why a vision-heavy workload is worth measuring on Activity rather than estimating.

vision.ts

const completion = await client.chat.completions.create({
  model: process.env.MAXLAYER_VISION_MODEL_ID,
  messages: [
    {
      role: "user",
      content: [
        { type: "text", text: "What has changed in this screenshot?" },
        { type: "image_url", image_url: { url: "https://example.com/before.png" } },
      ],
    },
  ],
});

Context and length #

Each model publishes a context_window covering everything in the request plus what it generates, and a max_output_tokens where the provider states one. Exceeding the window is an error rather than a silent truncation, which is the behaviour you want — a quietly truncated prompt produces a confident answer to a question you did not ask.

Set max_tokens deliberately. It bounds a runaway answer, and since output usually costs several times input, it is a cost control as much as a correctness one.

Streaming chat completions #

Set stream: true to receive server-sent OpenAI chunks. The first delta carries the assistant role. After the final content delta, a separate terminal chunk carries finish_reason, then the stream closes with [DONE]. This is the sequence SDKs and editor integrations expect when they wait for a completion to finish.

streaming.ts

const stream = await client.chat.completions.create({
  model: process.env.MAXLAYER_MODEL_ID,
  messages: [{ role: "user", content: "Summarize this deployment." }],
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta.content ?? "");
}

Embeddings #

Send POST /v1/embeddings with a listed embedding model and either one string or a batch. Repository-aware coding tools use this route to index source before they can answer questions about it; batching files avoids one request per file, which is what makes indexing a repository practical rather than merely possible.

embeddings.ts

const embedding = await client.embeddings.create({
  model: process.env.MAXLAYER_EMBEDDING_MODEL_ID,
  input: ["README.md", "src/index.ts"],
});

const vectors = embedding.data.map((item) => item.embedding);

Embedding more than text #

Some models embed images and audio into the same vector space as text, which is what lets you search a picture library with a sentence. For those, input takes OpenRouter’s structured content form instead of a string — an object whose content array carries text and image_url entries. Plain strings and arrays of strings keep working and are still the right shape for text.

multimodal.sh

curl https://inference.maxlayer.cloud/v1/embeddings \
  -H "Authorization: Bearer $MAXLAYER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<a multimodal embedding model from /v1/models>",
    "input": [
      { "content": [
        { "type": "text", "text": "a red bicycle leaning on a wall" },
        { "type": "image_url", "image_url": { "url": "https://example.com/bike.jpg" } }
      ] }
    ]
  }'

Check the catalogue before assuming a model accepts it: most embedding models are text-only and will reject an image. The marketplace lists what each one takes.

Note

A multimodal embedding model charges a different rate per kind of input.

google/gemini-embedding-2 prices text, image and audio input separately — at the time of writing $0.20, $0.45 and $6.50 per million tokens — and a response reports one prompt_tokens total covering all three. No single rate describes that, so these models carry billing_mode: "upstream_cost" like the generation models: charged what the request actually cost plus our markup, with the provider’s own rates listed in price_lines. Budget from the kind of input you are sending rather than from the headline figure, because embedding an hour of audio and embedding an hour of text are not the same bill. Text-only embedding models — most of the catalogue — keep an ordinary rate card.

Images #

Send POST /v1/images with a listed image model and a prompt. The response is OpenAI’s image shape — a data array whose entries carry b64_json — so a client already pointing at OpenAI’s images API changes only its base URL. Parameters beyond model and prompt are passed through to the provider, so whatever a given model supports for resolution, aspect ratio or a seed works without us listing them here.

images.sh

curl https://inference.maxlayer.cloud/v1/images   -H "Authorization: Bearer $MAXLAYER_API_KEY"   -H "Content-Type: application/json"   -d '{
    "model": "<an image model from /v1/models>",
    "prompt": "A serene mountain landscape at sunset"
  }'

Video #

Video generation takes minutes rather than seconds, so it is the one route here that does not answer in the request that starts it. POST /v1/videos accepts the job and returns an id and a polling URL; poll that URL until the status is completed, then fetch the result.

video.sh

# 1. submit — returns an id and a polling url
curl https://inference.maxlayer.cloud/v1/videos   -H "Authorization: Bearer $MAXLAYER_API_KEY"   -H "Content-Type: application/json"   -d '{
    "model": "<a video model from /v1/models>",
    "prompt": "A serene mountain landscape at sunset"
  }'

# 2. poll until status is "completed"
curl "$POLLING_URL" -H "Authorization: Bearer $MAXLAYER_API_KEY"

# 3. fetch the video from the url the finished job carries
curl "$VIDEO_URL" -H "Authorization: Bearer $MAXLAYER_API_KEY" -o video.mp4

Coding tools #

Cursor, Continue, Cline, Aider, Zed and opencode all work through their OpenAI-compatible provider settings. Each wants the same three values — the base URL https://inference.maxlayer.cloud/v1, your organization API key, and a model id copied from the catalogue — in a different place under a different name.

Cursor: Settings → Models → Override OpenAI Base URL.
Continue: an openai provider with apiBase, plus a separate embeddingsProvider for @codebase.
Cline and Roo: choose OpenAI Compatible as the API provider.
Aider: OPENAI_API_BASE, OPENAI_API_KEY, and an openai/ prefix on the model.
Zed: language_models.openai.api_url, with max_tokens per model.
opencode: an OpenAI-compatible provider with baseURL.

Configuration for each of them, including the two that have a real gotcha, and a table mapping every error this endpoint returns to what actually caused it.

Frameworks and SDKs #

The same three values work in every framework that can be given an OpenAI base URL, which is most of them. There is no Maxlayer package to install: the endpoint speaks OpenAI’s wire format, so the OpenAI integration a framework already ships is the integration.

LangChain: ChatOpenAI with base_url, in Python and JavaScript.
LlamaIndex: OpenAILike with api_base, plus OpenAIEmbedding for the index.
Vercel AI SDK: createOpenAICompatible from @ai-sdk/openai-compatible.
PydanticAI: an OpenAIProvider inside an OpenAIChatModel.
Mastra, TanStack AI, Effect: all take an AI SDK provider, so the one above serves them.

Working configuration for each, what to set so a library that reads OpenAI’s own environment variables finds Maxlayer, and how tracing tools behave against this endpoint.

Choosing a model #

There are several hundred, which is a worse problem than having too few. Nothing here recommends one — the catalogue changes without this page changing, and a recommendation written today ages badly. What follows is how to narrow it yourself.

Start from the shelf, not the list. The marketplace divides the catalogue by what a model emits — text, images, video, embeddings — because that is the first thing that has to match. An embedding model cannot answer a question and a chat model cannot fill a vector index, and calling the wrong route returns a clear error rather than a bad answer.

Then narrow on whichever of these you actually care about:

Price, sorted cheapest or dearest. Rates span four orders of magnitude — the gap between a small open model and a frontier one is far larger than any difference in how you call them.
Pricing basis. Fixed rate is a published rate card, so you can work out a bill before you send anything. Per request is charged from what the call cost and is knowable only afterwards. If you are costing a workload in advance, that filter is the one that matters.
Context window, if you are sending long documents. Exceeding it is an error rather than a silent truncation.
Capabilities — streaming, tool calling, vision. A model that cannot call tools will ignore the field rather than fail, which is the kind of thing worth checking before you build on it.
Zero data retention, where a data policy requires it. Labelled per model, and enforced per request on deployments configured for it.

What a call costs #

Every call draws on the same prepaid organization balance your applications and databases use. There is no separate AI subscription, no invoice and no card on file for it.

How the charge is worked out depends on the model, and billing_mode in the catalogue says which:

A rate card (token) — input tokens times the input rate, output tokens times the output rate. Both are published per million tokens, so the bill is predictable before the call.
Billed per request (upstream_cost) — image and video models, and the few embedding models that charge a different rate per kind of input. These have no single rate card to multiply, so the charge is what the request cost to serve, plus our markup. The amount lands on your usage page like any other call.

Four things move a bill, in roughly the order they matter:

Which model. Larger than everything else combined. Trying a cheaper model is one string.
Output length. Output usually costs several times input, so max_tokens is a cost control as much as a correctness one.
How much you send. Whole files in a prompt are input tokens on every request. For repeated questions over the same corpus, an index built with embeddings is usually cheaper than re-sending the corpus.
Request count. Embeddings accept a batch, so indexing a repository in one request per file is a round trip you are paying for by the file.

Dashboard → AI → Activity reports spend by model and by API key, with a request log you can filter to either and export. Giving each service its own key is what makes that breakdown useful — otherwise every request is attributed to one key and the question "which of my services spent this" has no answer.

Errors #

Every failure comes back in one envelope: Maxlayer’s code, message and requestId, plus OpenAI’s type and param so an OpenAI SDK raises the exception class you expect. Quote the requestId if you contact support — it is how a request is found in the logs.

insufficient credit

{
  "error": {
    "code": "insufficient_credit",
    "message": "This organization has no credit remaining. Top up to resume inference.",
    "type": "insufficient_quota",
    "param": null,
    "requestId": "b3f8c2a1"
  }
}

The ones worth recognising:

401 — the key is missing, revoked, or is a browser session token rather than an organization API key. Only the latter can spend credit.
402 insufficient_credit — your organization is out of credit. Top up; nothing else is wrong.
404 — the model id is not listed. Ids are exact, including the slash and any :variant suffix, and a model can be withdrawn upstream. Re-read /v1/models before assuming a typo.
400 — the request shape is wrong for the route, most often a chat model sent to the embeddings route or the reverse. The message says which.
502 — something failed upstream of us. This is ours to fix rather than yours; the requestId is what we need.