OpenAI SDK integration

Install and configure #

npm install openai      # TypeScript / JavaScript
pip install openai      # Python

Two settings and nothing else. Keep the /v1 on the base URL — the SDK appends the route beneath it, and dropping it produces a 404 that looks like a missing endpoint.

import OpenAI from "openai";

/*
 * Two settings and nothing else. baseURL keeps its /v1 — the SDK appends
 * /chat/completions beneath it — and the key is an organization API key from
 * Settings → API keys, never the token your browser session uses.
 */
export const client = new OpenAI({
  apiKey: process.env.MAXLAYER_API_KEY,
  baseURL: "https://inference.maxlayer.cloud/v1",

  // Worth setting explicitly. The SDK default is 2 retries and 10 minutes, and a
  // coding tool that hangs for ten minutes reads as a broken endpoint.
  maxRetries: 2,
  timeout: 60_000,
});

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["MAXLAYER_API_KEY"],
    base_url="https://inference.maxlayer.cloud/v1",
    max_retries=2,
    timeout=60.0,
)

The key must be an organization API key from Settings → API keys. A browser session token cannot call inference: it identifies a person rather than an organization, so there would be no balance to charge.

Chat completions #

The same call you would make against OpenAI. Request fields Maxlayer does not need to interpret are passed through to the model, so temperature, top_p, seed, stop and anything a provider adds next month work without waiting on this endpoint.

const completion = await client.chat.completions.create({
  model: process.env.MAXLAYER_MODEL_ID,
  messages: [
    { role: "system", content: "You review pull requests." },
    { role: "user", content: diff },
  ],
  temperature: 0.2,
  max_tokens: 1024,
});

console.log(completion.choices[0]?.message.content);
console.log(completion.usage);   // prompt_tokens, completion_tokens, total_tokens

usage comes back on every non-streamed response, and it is the same count the request was billed on. The dashboard's AI → Activity shows the resulting spend per model and per API key.

Streaming #

Set stream: true for server-sent OpenAI chunks. The first delta carries the assistant role, a terminal chunk carries finish_reason, and the stream closes with [DONE] — the sequence clients that wait on finish_reason rather than [DONE] depend on.

const stream = await client.chat.completions.create({
  model: process.env.MAXLAYER_MODEL_ID,
  messages: [{ role: "user", content: "Summarise this deployment." }],
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta.content ?? "");
}

Aborting a stream #

Pass an AbortSignal to stop reading early. Worth knowing what that does to the bill: tokens already generated are still charged. The provider produced them and the platform was invoiced for them, so hanging up does not undo the cost — it only stops you receiving the rest. Abandoned requests are recorded and shown as client disconnected in the activity log.

const controller = new AbortController();
setTimeout(() => controller.abort(), 5_000);

const stream = await client.chat.completions.create(
  { model: process.env.MAXLAYER_MODEL_ID, messages, stream: true },
  { signal: controller.signal },
);

Tool calling #

Tool definitions pass straight through to the model. Whether a given model honours them depends on the model, not on this endpoint — the catalogue records a function-calling capability per model, and the marketplace lets you filter by it.

const completion = await client.chat.completions.create({
  model: process.env.MAXLAYER_MODEL_ID,
  messages: [{ role: "user", content: "What is failing in build 4182?" }],
  tools: [
    {
      type: "function",
      function: {
        name: "get_build",
        description: "Fetch a build by id",
        parameters: {
          type: "object",
          properties: { id: { type: "string" } },
          required: ["id"],
        },
      },
    },
  ],
});

const call = completion.choices[0]?.message.tool_calls?.[0];

That is the first request, which is the half nobody gets wrong. The model has not run anything — it has asked you to — so what follows is a loop: execute, hand the result back with the tool_call_id it answers, and ask again. Tool calling covers the whole cycle, parallel calls, streaming with tools, what a loop costs, and the failure modes worth handling.

Embeddings #

Send one string or a batch. Batching matters for a repository index: one request per file is one authorization and one billing record per file, and the round trips dominate.

const embedding = await client.embeddings.create({
  model: process.env.MAXLAYER_EMBEDDING_MODEL_ID,
  input: ["README.md contents…", "src/index.ts contents…"],
});

// One vector per input, in order, and each carries its own index.
const vectors = embedding.data.map((item) => item.embedding);

Chat and embedding models are separate catalogue entries. Calling /v1/embeddings with a chat model returns a request error naming the mismatch rather than an unusable vector, and the reverse is true too.

Handling errors #

Every error carries OpenAI's type alongside Maxlayer's own code, so the SDKs raise the exception class you would expect. The code is the one to branch on — it distinguishes cases OpenAI's type collapses together.

import OpenAI from "openai";

try {
  await client.chat.completions.create({ model, messages });
} catch (error) {
  if (error instanceof OpenAI.AuthenticationError) {
    // 401 — the key is wrong, revoked, or a session token rather than an mxl_ key.
  } else if (error instanceof OpenAI.NotFoundError) {
    // 404 — the model id is unknown *or* not listed. Deliberately the same answer.
  } else if (error instanceof OpenAI.RateLimitError) {
    // 429 from the upstream provider, or 402 when the balance is spent. Read
    // error.code to tell them apart: insufficient_credit means top up, not retry.
    if ((error.error as { code?: string })?.code === "insufficient_credit") {
      // Stop. Retrying will not help and the balance has to be topped up.
    }
  } else if (error instanceof OpenAI.APIError) {
    // 5xx. error.requestId is the thread to quote to support.
  }
  throw error;
}

Every error also carries a requestId. It is the thread from a failure to the log line for the same call, and the one thing worth quoting in a support conversation.

Every error this endpoint returns, with its cause and fix — including the ones that look like something else.

What differs from OpenAI #

Short list, and none of it needs code changes:

• Model ids name a publisher, not a provider — anthropic/claude-sonnet-5. Copy one from the catalogue verbatim; they contain a slash and are not interchangeable with an OpenAI model name. Some carry a variant suffix after a colon — anthropic/claude-opus-5:batch — which is a different price for the same model, not decoration. The suffix is part of the id.
• GET /v1/models needs no key, so a model picker can populate before anything is configured.

  curl https://inference.maxlayer.cloud/v1/models

• Billing is prepaid credit, not a monthly invoice. Input and output tokens draw on the same organization balance your apps and databases use, metered per request and deducted hourly.
• No organization or project headers. The API key identifies the organization. OpenAI-Organization and OpenAI-Project are accepted and ignored, so a client that sends them unprompted still works.
• The catalogue is the contract. A model can be listed, repriced or withdrawn without this page changing, so read /v1/models rather than pinning what you read here.