Framework integrations

How this works #

There is no Maxlayer plugin to install, and there does not need to be one. The endpoint speaks OpenAI’s wire format, so the OpenAI integration a framework already ships is the Maxlayer integration. Three values are all any of them need:

• Base URL — https://inference.maxlayer.cloud/v1, with the /v1 kept. The library appends the route beneath it.
• API key — an organization key from Settings → API keys. A browser session token cannot call inference.
• Model id — copied from the catalogue exactly as written, slash and any :variant suffix included.

Environment #

The snippets below read these. Setting OPENAI_BASE_URL and OPENAI_API_KEY as well is worth doing: several libraries and most command-line tools read OpenAI’s own variables and accept no arguments at all, and with those two set they work against Maxlayer without further configuration.

MAXLAYER_API_KEY=mxl_...
MAXLAYER_MODEL_ID=<a chat model from /v1/models>
MAXLAYER_EMBEDDING_MODEL_ID=<an embedding model from /v1/models>

# Some libraries read OpenAI's own variables and take no arguments.
# Pointing those two at Maxlayer works everywhere the SDK is used.
OPENAI_BASE_URL=https://inference.maxlayer.cloud/v1
OPENAI_API_KEY=$MAXLAYER_API_KEY

LangChain #

Use ChatOpenAI and give it a base URL. Everything built on top of it — chains, agents, LangGraph — takes the model object and needs no further change.

# pip install -U langchain-openai
import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model=os.environ["MAXLAYER_MODEL_ID"],
    base_url="https://inference.maxlayer.cloud/v1",
    api_key=os.environ["MAXLAYER_API_KEY"],
)

print(llm.invoke("Explain this pull request.").content)

The JavaScript package puts the URL one level down, under configuration, which is passed through to the OpenAI client underneath.

// npm i @langchain/openai
import { ChatOpenAI } from "@langchain/openai";

const llm = new ChatOpenAI({
  model: process.env.MAXLAYER_MODEL_ID,
  apiKey: process.env.MAXLAYER_API_KEY,
  configuration: { baseURL: "https://inference.maxlayer.cloud/v1" },
});

console.log((await llm.invoke("Explain this pull request.")).content);

LlamaIndex #

OpenAILike is the wrapper for an OpenAI-compatible endpoint, and it takes api_base rather than base_url. Set is_chat_model=True: without it the wrapper calls the legacy completions route, which this endpoint does not serve.

# pip install llama-index-llms-openai-like llama-index-embeddings-openai
import os
from llama_index.llms.openai_like import OpenAILike
from llama_index.embeddings.openai import OpenAIEmbedding

llm = OpenAILike(
    model=os.environ["MAXLAYER_MODEL_ID"],
    api_base="https://inference.maxlayer.cloud/v1",
    api_key=os.environ["MAXLAYER_API_KEY"],
    # Without this the wrapper calls the legacy completions route, which
    # this endpoint does not serve.
    is_chat_model=True,
)

embed_model = OpenAIEmbedding(
    model_name=os.environ["MAXLAYER_EMBEDDING_MODEL_ID"],
    api_base="https://inference.maxlayer.cloud/v1",
    api_key=os.environ["MAXLAYER_API_KEY"],
)

Vercel AI SDK #

Install @ai-sdk/openai-compatible and build a provider with createOpenAICompatible. The name is a label for traces and errors; it can be anything.

// npm i @ai-sdk/openai-compatible ai
import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { generateText } from "ai";

const maxlayer = createOpenAICompatible({
  name: "maxlayer",
  baseURL: "https://inference.maxlayer.cloud/v1",
  apiKey: process.env.MAXLAYER_API_KEY,
});

const { text } = await generateText({
  model: maxlayer(process.env.MAXLAYER_MODEL_ID),
  prompt: "Explain this pull request.",
});

The provider slots into generateText, streamText and the React hooks unchanged. Streaming works because the endpoint emits the same SSE chunk sequence OpenAI does — see streaming chat completions.

PydanticAI #

Wrap an OpenAIProvider carrying the base URL and key, hand it to OpenAIChatModel, and give that to your agent.

# pip install pydantic-ai
import os
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider

model = OpenAIChatModel(
    os.environ["MAXLAYER_MODEL_ID"],
    provider=OpenAIProvider(
        base_url="https://inference.maxlayer.cloud/v1",
        api_key=os.environ["MAXLAYER_API_KEY"],
    ),
)

agent = Agent(model)
print(agent.run_sync("Explain this pull request.").output)

Structured output and tool calling work through the same request fields OpenAI uses. Whether a given model honours them is a property of the model rather than of this endpoint — the catalogue lists what each one supports.

Other frameworks #

The same three values work anywhere an OpenAI-compatible base URL can be set. These are in common use and are configured through their OpenAI provider rather than a Maxlayer-specific one:

• Mastra — takes an AI SDK provider, so the createOpenAICompatible instance above is what you pass it.
• TanStack AI — React, Solid and Preact, on an AI SDK provider in the same way.
• Effect AI SDK — TypeScript, through its OpenAI-compatible client layer.
• Semantic Kernel, Spring AI and other server-side frameworks — each has an OpenAI connector that accepts a custom endpoint.

For editors and terminal agents rather than libraries, see coding tool setup, which carries the exact configuration for each.

Observability #

Tracing tools that wrap the OpenAI client — Langfuse, OpenLLMetry, LangSmith and similar — work unchanged, because they instrument the client rather than the endpoint. They record the base URL and model id you configured, so traces name the Maxlayer model id rather than a provider’s.