Baseten

Baseten provides scalable infrastructure for deploying and serving machine learning models, including language models and custom AI applications. Braintrust integrates seamlessly with Baseten through direct API access, wrapper functions for automatic tracing, and proxy support.

Setup

To use Baseten models, configure your Baseten API key in Braintrust.

Get a Baseten API key from the Baseten console.
Add the Baseten API key as an organization or project AI provider.
Optional: Set API base URL if your models are served from a custom Baseten deployment endpoint (for example, a dedicated deployment). If you leave this blank, Braintrust uses the default Baseten endpoint.
Set the Baseten API key and your Braintrust API key as environment variables.

.env

BASETEN_API_KEY=<your-baseten-api-key>
BRAINTRUST_API_KEY=<your-braintrust-api-key>

# For organizations on the EU data plane, use https://api-eu.braintrust.dev
# For self-hosted deployments, use your data plane URL
# BRAINTRUST_API_URL=<your-braintrust-api-url>

API keys are stored as one-way cryptographic hashes, never in plaintext.

Use Baseten with Braintrust gateway

The Braintrust gateway allows you to access Baseten models through a unified interface. Use any supported provider’s SDK to call Baseten models. Install the braintrust and openai packages.

# pnpm
pnpm add braintrust openai
# npm
npm install braintrust openai

pip install braintrust openai

Then, initialize the client and make a request to a Baseten model via the Braintrust gateway.

import { OpenAI } from "openai";

const client = new OpenAI({
  baseURL: "https://gateway.braintrust.dev/v1",
  apiKey: process.env.BRAINTRUST_API_KEY,
});

const response = await client.chat.completions.create({
  model: "gpt-oss-120b",
  messages: [{ role: "user", content: "Hello, world!" }],
});

import os

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.braintrust.dev/v1",
    api_key=os.environ["BRAINTRUST_API_KEY"],
)

response = client.chat.completions.create(
    model="gpt-oss-120b",
    messages=[{"role": "user", "content": "Hello, world!"}],
)

Trace logs with Baseten

Trace your Baseten LLM calls for observability and monitoring. When using the Braintrust gateway, API calls are automatically logged to the specified project.

import { OpenAI } from "openai";
import { initLogger } from "braintrust";

initLogger({
  projectName: "My Project",
  apiKey: process.env.BRAINTRUST_API_KEY,
});

const client = new OpenAI({
  baseURL: "https://gateway.braintrust.dev/v1",
  apiKey: process.env.BRAINTRUST_API_KEY,
});

// All API calls are automatically logged
const result = await client.chat.completions.create({
  model: "gpt-oss-120b",
  messages: [{ role: "user", content: "What is machine learning?" }],
});

import os

from braintrust import init_logger
from openai import OpenAI

init_logger(project="My Project")

client = OpenAI(
    base_url="https://gateway.braintrust.dev/v1",
    api_key=os.environ["BRAINTRUST_API_KEY"],
)

# All API calls are automatically logged
result = client.chat.completions.create(
    model="gpt-oss-120b",
    messages=[{"role": "user", "content": "What is machine learning?"}],
)

The Braintrust gateway is not required. For more control, learn how to customize traces.

Evaluate with Baseten

Evaluations distill the non-deterministic outputs of Baseten models into an effective feedback loop that enables you to ship more reliable, higher quality products. Braintrust Eval is a simple function composed of a dataset of user inputs, a task, and a set of scorers. To learn more about evaluations, check out the Experiments guide.

import { Eval } from "braintrust";
import { OpenAI } from "openai";

const client = new OpenAI({
  baseURL: "https://gateway.braintrust.dev/v1",
  apiKey: process.env.BRAINTRUST_API_KEY,
});

Eval("Baseten Evaluation", {
  data: () => [
    { input: "What is 2+2?", expected: "4" },
    { input: "What is the capital of France?", expected: "Paris" },
  ],
  task: async (input) => {
    const response = await client.chat.completions.create({
      model: "openai/gpt-oss-120b",
      messages: [{ role: "user", content: input }],
    });
    return response.choices[0].message.content;
  },
  scores: [
    {
      name: "accuracy",
      scorer: (args) => (args.output === args.expected ? 1 : 0),
    },
  ],
});

import os

from braintrust import Eval
from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.braintrust.dev/v1",
    api_key=os.environ["BRAINTRUST_API_KEY"],
)


def task(input):
    response = client.chat.completions.create(
        model="openai/gpt-oss-120b",
        messages=[{"role": "user", "content": input}],
    )
    return response.choices[0].message.content


def accuracy_scorer(output, expected, **kwargs):
    return 1 if output == expected else 0


Eval(
    "Baseten Evaluation",
    data=[
        {"input": "What is 2+2?", "expected": "4"},
        {"input": "What is the capital of France?", "expected": "Paris"},
    ],
    task=task,
    scores=[accuracy_scorer],
)

To learn more about tool use, multimodal support, attachments, and masking sensitive data with Baseten, visit the customize traces guide.

AI providers

SDKs

Developer tools

Setup

Use Baseten with Braintrust gateway

Trace logs with Baseten

Evaluate with Baseten

​Setup

​Use Baseten with Braintrust gateway

​Trace logs with Baseten

​Evaluate with Baseten

Setup

Use Baseten with Braintrust gateway

Trace logs with Baseten

Evaluate with Baseten