Act on findings from Topics

​

Build datasets from topics

1. Go to Logs and click Filter.
2. Select Topics and choose the topic you want to filter by. Alternately, click SQL and enter a filter clause. See the SQL reference for more query patterns.
   Filter by specific topic

   Filter by multiple topics

   Combine topics across facets

   classifications.Task.label = "Dataset creation"
   

   classifications.Task.label IN ("Dataset creation", "API errors")
   

   classifications.Task.label = "Checkout flow"
     AND classifications.Sentiment.label = "NEGATIVE"
   
3. Select the logs you want to include.
4. Click + Dataset and choose an existing dataset or create a new one.

• “Error investigation” tasks → test your error handling.
• Negative sentiment interactions → improve responses.
• “Pricing questions” → evaluate your pricing explanations.

​

Score logs based on topics

import braintrust from "braintrust";
import { z } from "zod";

const project = braintrust.projects.create({ name: "my-project" });

project.scorers.create({
  name: "Checkout experience",
  slug: "checkout-experience",
  description: "Flag traces with negative checkout experiences",
  parameters: z.object({
    trace: z.any(),
  }),
  handler: async ({ trace }) => {
    if (!trace) return { score: null };

    const spans = await trace.getSpans();
    const rootSpan = spans.find((s) => s.span_id === s.root_span_id);
    if (!rootSpan) return { score: null };

    const classifications = rootSpan.classifications || {};
    const taskClassification = (classifications.Task || [{}])[0];
    const sentimentClassification = (classifications.Sentiment || [{}])[0];

    if (
      taskClassification.label === "Checkout Flow" &&
      sentimentClassification.label === "NEGATIVE"
    ) {
      return {
        score: 0,
        metadata: { reason: "Negative sentiment during checkout" },
      };
    }

    return { score: 1 };
  },
});

bt functions push topic_scorer.ts

import braintrust
from pydantic import BaseModel

project = braintrust.projects.create(name="my-project")

class TraceParams(BaseModel):
    trace: dict

async def checkout_experience_scorer(trace):
    if not trace:
        return {"score": None}

    spans = await trace.get_spans()
    root_span = next(
        (s for s in spans if s.get("span_id") == s.get("root_span_id")),
        None
    )
    if not root_span:
        return {"score": None}

    classifications = root_span.get("classifications", {})
    task_classification = classifications.get("Task", [{}])[0]
    sentiment_classification = classifications.get("Sentiment", [{}])[0]

    if (
        task_classification.get("label") == "Checkout Flow"
        and sentiment_classification.get("label") == "NEGATIVE"
    ):
        return {
            "score": 0,
            "metadata": {"reason": "Negative sentiment during checkout"},
        }

    return {"score": 1}

project.scorers.create(
    name="Checkout experience",
    slug="checkout-experience",
    description="Flag traces with negative checkout experiences",
    parameters=TraceParams,
    handler=checkout_experience_scorer,
)

bt functions push topic_scorer.py

1. Go to Settings > Automations and click + Rule.
2. Select your scorer, set Scope to Trace, configure the sampling rate, and click Create rule.

​

Assign topics for review

1. Go to Logs and click Filter.
2. Select Topics and choose the topic you want to filter by. Alternately, click SQL and enter a filter clause. See the SQL reference for more query patterns.
   Filter by specific topic

   Filter by multiple topics

   Combine topics across facets

   classifications.Task.label = "Dataset creation"
   

   classifications.Task.label IN ("Dataset creation", "API errors")
   

   classifications.Task.label = "Checkout flow"
     AND classifications.Sentiment.label = "NEGATIVE"
   
3. Select the logs you want to assign.
4. Select Assign and choose a team member.

​

Send a daily Slack digest

• Summary: How activity shifted, with a table of topic movements between the latest topic generations.
• Biggest mover: The topic that changed the most since the last generation, with context on what’s driving it.
• Highlights: Specific traces worth a closer look, tied to the biggest movements.
• Takeaways: A clear, actionable recommendation when one is evident.

1. Make sure the Slack integration is enabled for your organization.
2. Go to Settings > Feature flags and turn on Topics digest.
3. Go to Topics and expand Automation settings in the status panel.
4. Turn on Topics digest.
5. Select the Slack workspace and Slack channel to post to.
6. Set the Send time. The digest is sent once a day at this local time.
7. Click Save.

​

Next steps

• Manage Topics to tune the pipeline that produces these classifications.
• Custom facets to define new dimensions to act on.