Tracing

The @mutagent/sdk/tracing module provides custom instrumentation primitives for tracing LLM calls, agent workflows, tool usage, and more. Use it when the Integration packages do not cover your framework, or when you need fine-grained control over span creation.

For concepts, span kinds, and the overall tracing architecture, see the Tracing Overview. For the full server-side API reference, see the Tracing API.

Installation

The tracing module is included in @mutagent/sdk. No additional packages are required.

npm install @mutagent/sdk

Quick Start

import { initTracing, shutdownTracing } from '@mutagent/sdk/tracing';

// Initialize once at application startup
initTracing({
  apiKey: process.env.MUTAGENT_API_KEY!,
  environment: 'production',
  debug: false,
});

// ... your application code using @trace, withTrace, or startSpan ...

// Shutdown gracefully (flushes remaining spans)
await shutdownTracing();

Lazy Initialization

If you do not call initTracing() explicitly, the module automatically initializes from the MUTAGENT_API_KEY environment variable on the first startSpan() call. This is a one-time attempt — if the variable is not set, tracing remains disabled for the rest of the process lifecycle.

// No initTracing() call needed if MUTAGENT_API_KEY is set
import { startSpan, endSpan } from '@mutagent/sdk/tracing';

// First call triggers lazy init from env var
const span = startSpan({ kind: 'llm.chat', name: 'openai-chat' });

Architecture

The tracing module uses a singleton architecture:

SpanManager — Creates and manages spans with AsyncLocalStorage for parent-child propagation
BatchCollector — Buffers finished spans in memory and flushes when either the batch size (default 10) or the flush interval (default 5s) is reached. Retries with exponential backoff on failure (up to 3 attempts).
TraceHTTPClient — Sends grouped span payloads to POST /api/traces

Instrumentation Approaches

The SDK provides three levels of instrumentation, from highest to lowest abstraction.

1. `withTrace()` Wrapper

The primary instrumentation API. Wraps any function with a tracing span and provides a SpanHandle for manual enrichment during execution.

import { withTrace } from '@mutagent/sdk/tracing';

const result = await withTrace(
  { kind: 'chain', name: 'rag-pipeline' },
  async (span) => {
    span.setAttributes({ 'rag.corpus': 'knowledge-base-v2' });

    const docs = await retrieve(query);
    span.addEvent('retrieval-complete', { documentCount: docs.length });

    const answer = await generate(query, docs);
    span.setOutput({ text: answer });
    span.setMetrics({
      model: 'gpt-4o',
      provider: 'openai',
      inputTokens: 150,
      outputTokens: 300,
      totalTokens: 450,
    });

    return answer;
  }
);

When tracing is not initialized, withTrace runs the function directly with a no-op SpanHandle — your code executes normally without any overhead.

Wrapper Options

interface WithTraceOptions {
  kind: SpanKind;                        // Required: span kind
  name?: string;                         // Span name
  input?: SpanIO;                        // Initial structured input
  attributes?: Record<string, unknown>;  // Custom attributes
}

SpanHandle Methods

Method	Description
`setAttributes(attrs)`	Add custom key-value attributes to the span
`addEvent(name, attrs?)`	Record a lifecycle event with timestamp and optional attributes
`setOutput(output)`	Set structured output (`SpanIO`)
`setInput(input)`	Set or override structured input (`SpanIO`)
`setMetrics(metrics)`	Set token counts, cost, latency, and model info
`spanId`	Read-only span ID
`traceId`	Read-only trace ID

2. `@trace()` Decorator

For class-based code. Automatically captures method arguments as input and the return value as output. Requires experimentalDecorators: true in tsconfig.json.

import { trace } from '@mutagent/sdk/tracing';

class MyAgent {
  @trace({ kind: 'agent', name: 'research-agent' })
  async run(query: string): Promise<string> {
    // Your agent logic here
    return result;
  }

  @trace({ kind: 'tool', name: 'web-search' })
  async search(query: string): Promise<string[]> {
    // Tool implementation
    return results;
  }
}

The decorator:

Creates a span with the specified kind and name
If name is omitted, falls back to the method name, then the kind
Captures arguments as input.raw and the return value as output.raw
Sets status: 'ok' on success, status: 'error' on exceptions (with message)
Propagates parent-child span relationships via AsyncLocalStorage context
Works with both sync and async methods, preserving this context
No-ops gracefully when tracing is not initialized

Decorator Options

interface TraceDecoratorOptions {
  kind: SpanKind;                        // Required: span kind
  name?: string;                         // Span name (defaults to method name)
  attributes?: Record<string, unknown>;  // Custom attributes
}

3. `startSpan()` / `endSpan()` Manual API

The lowest-level API for integration adapters or scenarios where automatic span lifecycle does not fit.

import {
  startSpan,
  endSpan,
  runInSpanContext,
  getCurrentSpan,
  getCurrentTraceId,
} from '@mutagent/sdk/tracing';

const span = startSpan({
  kind: 'llm.chat',
  name: 'openai-chat',
  input: {
    messages: [
      { role: 'user', content: 'Hello!' },
    ],
  },
});

if (span) {
  // Run nested code within span context (enables parent-child propagation)
  const result = await runInSpanContext(span, async () => {
    // getCurrentSpan() returns this span inside the callback
    // getCurrentTraceId() returns the active trace ID
    const response = await callLLM();

    endSpan(span, {
      status: 'ok',
      output: {
        messages: [
          { role: 'assistant', content: response.text },
        ],
      },
      metrics: {
        model: 'gpt-4o',
        provider: 'openai',
        inputTokens: response.usage.prompt_tokens,
        outputTokens: response.usage.completion_tokens,
        totalTokens: response.usage.total_tokens,
      },
    });

    return response;
  });
}

Low-Level API Functions

Function	Description
`startSpan(options)`	Create a new span. Returns `MutagentSpan \| undefined`
`endSpan(span, endOptions?)`	End a span, compute duration, deliver to batch collector
`runInSpanContext(span, fn)`	Run `fn` within the span’s async context for parent-child propagation
`getCurrentSpan()`	Get the active span from async context
`getCurrentTraceId()`	Get the active trace ID from async context

SpanOptions

interface SpanOptions {
  kind: SpanKind;                        // Required: span kind
  name?: string;                         // Span name
  input?: SpanIO;                        // Structured input
  attributes?: Record<string, unknown>;  // Custom attributes
  parentSpanId?: string;                 // Explicit parent span (auto-inferred from context if omitted)
}

SpanEndOptions

interface SpanEndOptions {
  status?: SpanStatus;                   // 'ok' | 'error' | 'unset'
  statusMessage?: string;                // Error message or status detail
  output?: SpanIO;                       // Structured output
  metrics?: SpanMetrics;                 // Token counts, cost, latency
  attributes?: Record<string, unknown>;  // Additional attributes
  events?: SpanEvent[];                  // Lifecycle events
}

Configuration Reference

The initTracing() function accepts a TracingConfig object:

Property	Type	Default	Description
`apiKey`	`string`	— (required)	MutagenT API key for authentication
`endpoint`	`string`	`https://api.mutagent.io`	API endpoint URL
`environment`	`string`	—	Environment name (e.g., `production`, `staging`)
`batchSize`	`number`	`10`	Number of spans to buffer before flushing
`flushIntervalMs`	`number`	`5000`	Flush interval in milliseconds
`debug`	`boolean`	`false`	Log span start/end events to console
`source`	`string`	`sdk`	Source identifier for traces

Lifecycle Functions

Function	Description
`initTracing(config)`	Initialize tracing. Idempotent: calling again replaces the previous configuration (shuts down the old collector first).
`shutdownTracing()`	Flush remaining spans, clear timers, and reset singleton state. Returns a `Promise<void>`.
`isTracingInitialized()`	Check whether tracing has been initialized. Returns `boolean`.

Type Reference

SpanKind

Taxonomy aligned with OTel Gen AI semantic conventions (v1.37+):

Category	Values
Generation	`llm.chat`, `llm.completion`, `llm.embedding`
Orchestration	`chain`, `agent`, `graph`, `node`, `edge`, `workflow`, `middleware`
Tools	`tool`
Retrieval	`retrieval`, `rerank`
Safety	`guardrail`
Other	`custom`

SpanIO

Structured input/output for spans:

Field	Type	Description
`messages`	`Array<{ role, content, name?, toolCallId? }>`	Chat messages (most common for LLM spans). Roles: `system`, `user`, `assistant`, `tool`
`toolCalls`	`Array<{ id, name, arguments, result? }>`	Tool calls for LLM spans requesting tool use
`documents`	`Array<{ content, metadata?, score? }>`	Retrieved documents (for retrieval spans)
`text`	`string`	Raw text (for completions, generic spans)
`raw`	`unknown`	Framework-specific data (escape hatch)

SpanStatus

'ok' | 'error' | 'unset'

SpanMetrics

Field	Type	Description
`model`	`string`	Model identifier (e.g., `gpt-4o`, `claude-sonnet-4`)
`provider`	`string`	Provider name (e.g., `openai`, `anthropic`)
`inputTokens`	`number`	Input token count
`outputTokens`	`number`	Output token count
`totalTokens`	`number`	Total token count
`costUsd`	`number`	Estimated cost in USD
`latencyMs`	`number`	Total latency in milliseconds
`ttftMs`	`number`	Time to first token in milliseconds

SpanEvent

Field	Type	Description
`name`	`string`	Event name
`timestamp`	`Date`	When the event occurred (auto-set by `addEvent`)
`attributes`	`Record<string, unknown>`	Optional event attributes

MutagentSpan

Internal mutable span representation (returned by startSpan):

Field	Type	Description
`id`	`string`	Unique span ID
`traceId`	`string`	Trace ID this span belongs to
`parentSpanId`	`string \| undefined`	Parent span ID (if nested)
`kind`	`SpanKind`	Span kind
`name`	`string`	Span name
`startedAt`	`Date`	When the span started
`input`	`SpanIO \| undefined`	Structured input
`output`	`SpanIO \| undefined`	Structured output
`status`	`SpanStatus`	Current status
`statusMessage`	`string \| undefined`	Error/status message
`metrics`	`SpanMetrics`	Token counts, cost, etc.
`attributes`	`Record<string, unknown>`	Custom attributes
`events`	`SpanEvent[]`	Lifecycle events

FinishedSpan

Immutable span ready for serialization and transport:

Field	Type	Description
`spanId`	`string`	Unique span ID
`traceId`	`string`	Trace ID
`parentSpanId`	`string \| undefined`	Parent span ID
`name`	`string`	Span name
`kind`	`SpanKind`	Span kind
`startTime`	`string`	ISO 8601 start time
`endTime`	`string`	ISO 8601 end time
`durationMs`	`number`	Duration in milliseconds
`status`	`SpanStatus`	Final status
`statusMessage`	`string \| undefined`	Error/status message
`model`	`string \| undefined`	Model used
`provider`	`string \| undefined`	Provider name
`inputTokens`	`number \| undefined`	Input token count
`outputTokens`	`number \| undefined`	Output token count
`totalTokens`	`number \| undefined`	Total token count
`costUsd`	`number \| undefined`	Estimated cost

Exports Summary

All exports from @mutagent/sdk/tracing:

// Lifecycle
import {
  initTracing,
  shutdownTracing,
  isTracingInitialized,
} from '@mutagent/sdk/tracing';

// Decorator
import { trace } from '@mutagent/sdk/tracing';
import type { TraceDecoratorOptions } from '@mutagent/sdk/tracing';

// Wrapper
import { withTrace } from '@mutagent/sdk/tracing';
import type { WithTraceOptions } from '@mutagent/sdk/tracing';

// Low-level API
import {
  startSpan,
  endSpan,
  getCurrentSpan,
  getCurrentTraceId,
  runInSpanContext,
} from '@mutagent/sdk/tracing';

// Types
import type {
  SpanKind,
  SpanIO,
  SpanStatus,
  SpanEvent,
  SpanMetrics,
  SpanOptions,
  SpanEndOptions,
  SpanHandle,
  MutagentSpan,
  FinishedSpan,
  TracingConfig,
  TracePayload,
  TraceHTTPResponse,
} from '@mutagent/sdk/tracing';

Next Steps

Tracing Overview

Concepts, span hierarchy, and tracing architecture

Tracing API Reference

Full server-side API reference for ingesting and querying traces

Integrations

Framework adapters with automatic tracing (recommended for most users)

TypeScript SDK Setup

Install and configure the TypeScript SDK

Getting Started

CLI

Integrations

Tracing

SDK

Tracing

Tracing

Installation

Quick Start

Lazy Initialization

Architecture

Instrumentation Approaches

1. `withTrace()` Wrapper

Wrapper Options

SpanHandle Methods

2. `@trace()` Decorator

Decorator Options

3. `startSpan()` / `endSpan()` Manual API

Low-Level API Functions

SpanOptions

SpanEndOptions

Configuration Reference

Lifecycle Functions

Type Reference

SpanKind

SpanIO

SpanStatus

SpanMetrics

SpanEvent

MutagentSpan

FinishedSpan

Exports Summary

Next Steps

Tracing Overview

Tracing API Reference

Integrations

TypeScript SDK Setup

Getting Started

CLI

Integrations

Tracing

SDK

​Tracing

​Installation

​Quick Start

​Lazy Initialization

​Architecture

​Instrumentation Approaches

​1. withTrace() Wrapper

​Wrapper Options

​SpanHandle Methods

​2. @trace() Decorator

​Decorator Options

​3. startSpan() / endSpan() Manual API

​Low-Level API Functions

​SpanOptions

​SpanEndOptions

​Configuration Reference

​Lifecycle Functions

​Type Reference

​SpanKind

​SpanIO

​SpanStatus

​SpanMetrics

​SpanEvent

​MutagentSpan

​FinishedSpan

​Exports Summary

​Next Steps

Tracing Overview

Tracing API Reference

Integrations

TypeScript SDK Setup

Tracing

Installation

Quick Start

Lazy Initialization

Architecture

Instrumentation Approaches

1. `withTrace()` Wrapper

Wrapper Options

SpanHandle Methods

2. `@trace()` Decorator

Decorator Options

3. `startSpan()` / `endSpan()` Manual API

Low-Level API Functions

SpanOptions

SpanEndOptions

Configuration Reference

Lifecycle Functions

Type Reference

SpanKind

SpanIO

SpanStatus

SpanMetrics

SpanEvent

MutagentSpan

FinishedSpan

Exports Summary

Next Steps