Optimization Streaming

Get real-time updates on optimization progress using WebSocket streaming. Build responsive UIs and monitor long-running optimization jobs as they execute. Connect to a job’s event stream to receive updates as they happen:

import { Mutagent } from '@mutagent/sdk';

const client = new Mutagent({ bearerAuth: 'sk_live_...' });

// Subscribe to job events
const unsubscribe = client.optimization.subscribe('job_xxxx', {
  onEvent: (event) => {
    switch (event.type) {
      case 'iteration_start':
        console.log(`Starting iteration ${event.iteration}`);
        break;

      case 'mutation_generated':
        console.log('New prompt variant generated');
        console.log('Preview:', event.preview.substring(0, 100) + '...');
        break;

      case 'evaluation_start':
        console.log('Evaluating variant...');
        break;

      case 'evaluation_complete':
        console.log(`Evaluation score: ${event.score.toFixed(2)}`);
        break;

      case 'new_best':
        console.log('=== NEW BEST PROMPT FOUND ===');
        console.log(`Score: ${event.score.toFixed(2)} (+${event.improvement.toFixed(2)})`);
        break;

      case 'iteration_complete':
        console.log(`Iteration ${event.iteration} complete\n`);
        break;
    }
  },

  onComplete: (results) => {
    console.log('=== OPTIMIZATION COMPLETE ===');
    console.log(`Final score: ${results.finalScore.toFixed(2)}`);
    console.log(`Total improvement: +${(results.improvement * 100).toFixed(1)}%`);
  },

  onError: (error) => {
    console.error('Optimization error:', error.message);
  },
});

// Later: cleanup subscription
// unsubscribe();

Event Types

Event	Description	Key Fields
`job_started`	Optimization job has begun	`jobId`, `timestamp`
`iteration_start`	New iteration beginning	`iteration`, `timestamp`
`mutation_generated`	Prompt variant created	`preview`, `mutationType`
`evaluation_start`	Variant evaluation beginning	`iteration`
`evaluation_complete`	Variant evaluation finished	`score`, `metrics`
`new_best`	New best-performing prompt found	`score`, `improvement`, `prompt`
`iteration_complete`	Iteration finished	`iteration`, `bestScore`
`job_paused`	Job was paused	`iteration`, `reason`
`job_resumed`	Job was resumed	`iteration`
`job_complete`	Optimization finished successfully	`results`
`job_failed`	Optimization failed	`error`
`job_cancelled`	Optimization was cancelled	`reason`

Event Payloads

job_started

interface JobStartedEvent {
  type: 'job_started';
  jobId: string;
  promptId: string;
  datasetId: string;
  config: OptimizationConfig;
  timestamp: Date;
}

iteration_start

interface IterationStartEvent {
  type: 'iteration_start';
  iteration: number;
  currentBestScore: number;
  timestamp: Date;
}

mutation_generated

interface MutationGeneratedEvent {
  type: 'mutation_generated';
  iteration: number;
  preview: string;           // First 200 chars of mutated prompt
  mutationType: 'rephrase' | 'restructure' | 'expand' | 'simplify';
  timestamp: Date;
}

evaluation_complete

interface EvaluationCompleteEvent {
  type: 'evaluation_complete';
  iteration: number;
  score: number;
  metrics: Record<string, number>;  // Per-metric scores
  isImprovement: boolean;
  timestamp: Date;
}

new_best

interface NewBestEvent {
  type: 'new_best';
  iteration: number;
  score: number;
  improvement: number;              // Score increase from previous best
  totalImprovement: number;         // Score increase from original
  prompt: {
    content: string;
    variables: Record<string, string>;
  };
  timestamp: Date;
}

job_complete

interface JobCompleteEvent {
  type: 'job_complete';
  results: OptimizationResults;
  timestamp: Date;
}

UI Integration

Build a real-time progress UI using streaming events:

React Example

import { useState, useEffect } from 'react';
import { Mutagent } from '@mutagent/sdk';

const client = new Mutagent({ bearerAuth: 'sk_live_...' });

interface IterationData {
  iteration: number;
  score: number;
  isNewBest: boolean;
}

function OptimizationProgress({ jobId }: { jobId: string }) {
  const [status, setStatus] = useState<string>('connecting');
  const [iterations, setIterations] = useState<IterationData[]>([]);
  const [bestScore, setBestScore] = useState<number>(0);
  const [currentIteration, setCurrentIteration] = useState<number>(0);
  const [bestPrompt, setBestPrompt] = useState<string>('');

  useEffect(() => {
    const unsub = client.optimization.subscribe(jobId, {
      onEvent: (event) => {
        switch (event.type) {
          case 'job_started':
            setStatus('running');
            break;

          case 'iteration_start':
            setCurrentIteration(event.iteration);
            break;

          case 'evaluation_complete':
            setIterations(prev => [...prev, {
              iteration: event.iteration,
              score: event.score,
              isNewBest: event.isImprovement,
            }]);
            break;

          case 'new_best':
            setBestScore(event.score);
            setBestPrompt(event.prompt.content);
            break;
        }
      },

      onComplete: (results) => {
        setStatus('completed');
        setBestScore(results.finalScore);
      },

      onError: (error) => {
        setStatus('error');
        console.error(error);
      },
    });

    return () => unsub();
  }, [jobId]);

  return (
    <div className="optimization-progress">
      <div className="header">
        <h2>Optimization Progress</h2>
        <span className={`status ${status}`}>{status}</span>
      </div>

      <div className="stats">
        <div className="stat">
          <label>Current Iteration</label>
          <value>{currentIteration}</value>
        </div>
        <div className="stat">
          <label>Best Score</label>
          <value>{bestScore.toFixed(2)}</value>
        </div>
      </div>

      <div className="chart">
        {/* Score progression chart */}
        {iterations.map((it, i) => (
          <div
            key={i}
            className={`bar ${it.isNewBest ? 'best' : ''}`}
            style={{ height: `${it.score * 100}%` }}
            title={`Iteration ${it.iteration}: ${it.score.toFixed(2)}`}
          />
        ))}
      </div>

      {bestPrompt && (
        <div className="best-prompt">
          <h3>Current Best Prompt</h3>
          <pre>{bestPrompt}</pre>
        </div>
      )}
    </div>
  );
}

export default OptimizationProgress;

Vue Example

<template>
  <div class="optimization-progress">
    <h2>Optimization: {{ status }}</h2>

    <div class="progress-bar">
      <div
        class="fill"
        :style="{ width: `${(currentIteration / maxIterations) * 100}%` }"
      ></div>
    </div>

    <div class="scores">
      <p>Iteration: {{ currentIteration }} / {{ maxIterations }}</p>
      <p>Best Score: {{ bestScore.toFixed(2) }}</p>
    </div>

    <ul class="iterations">
      <li
        v-for="it in iterations"
        :key="it.iteration"
        :class="{ best: it.isNewBest }"
      >
        Iteration {{ it.iteration }}: {{ it.score.toFixed(2) }}
        <span v-if="it.isNewBest">*</span>
      </li>
    </ul>
  </div>
</template>

<script setup>
import { ref, onMounted, onUnmounted } from 'vue';
import { Mutagent } from '@mutagent/sdk';

const props = defineProps(['jobId']);

const client = new Mutagent({ bearerAuth: 'sk_live_...' });

const status = ref('connecting');
const currentIteration = ref(0);
const maxIterations = ref(10);
const bestScore = ref(0);
const iterations = ref([]);

let unsubscribe;

onMounted(() => {
  unsubscribe = client.optimization.subscribe(props.jobId, {
    onEvent: (event) => {
      if (event.type === 'iteration_complete') {
        currentIteration.value = event.iteration;
        iterations.value.push({
          iteration: event.iteration,
          score: event.score,
          isNewBest: event.isNewBest,
        });
      }
      if (event.type === 'new_best') {
        bestScore.value = event.score;
      }
    },
    onComplete: () => {
      status.value = 'completed';
    },
  });
});

onUnmounted(() => {
  if (unsubscribe) unsubscribe();
});
</script>

Connection Management

Automatic Reconnection

The SDK handles reconnection automatically:

const unsubscribe = client.optimization.subscribe('job_xxxx', {
  // Event handlers...

  // Optional: customize reconnection
  reconnect: {
    enabled: true,           // default: true
    maxAttempts: 5,          // default: 5
    delay: 1000,             // initial delay in ms
    maxDelay: 30000,         // max delay with exponential backoff
  },

  onReconnect: (attempt) => {
    console.log(`Reconnecting (attempt ${attempt})...`);
  },

  onReconnectFailed: () => {
    console.error('Failed to reconnect after max attempts');
  },
});

Manual Reconnection

// Disconnect
unsubscribe();

// Reconnect later
const newUnsubscribe = client.optimization.subscribe('job_xxxx', handlers);

Error Handling

Handle various error scenarios:

client.optimization.subscribe('job_xxxx', {
  onEvent: (event) => { /* ... */ },

  onError: (error) => {
    switch (error.code) {
      case 'JOB_NOT_FOUND':
        console.error('Job does not exist');
        break;
      case 'UNAUTHORIZED':
        console.error('Invalid API key');
        break;
      case 'CONNECTION_LOST':
        console.error('WebSocket connection lost');
        break;
      default:
        console.error('Unknown error:', error.message);
    }
  },

  onComplete: (results) => { /* ... */ },
});

Best Practices

Clean up subscriptions

Always call the unsubscribe function when the component unmounts or the job completes to prevent memory leaks.

Handle reconnection gracefully

Show appropriate UI feedback during reconnection attempts so users know the connection is being restored.

Batch UI updates

If receiving many events quickly, consider batching state updates to avoid excessive re-renders.

Persist critical state

For long-running optimizations, consider persisting the best result locally in case of browser refresh.

Prompts

Datasets

Evaluations

Optimization

Providers

Optimization Streaming

Optimization Streaming

Event Types

Event Payloads

job_started

iteration_start

mutation_generated

evaluation_complete

new_best

job_complete

UI Integration

React Example

Vue Example

Connection Management

Automatic Reconnection

Manual Reconnection

Error Handling

Best Practices

Prompts

Datasets

Evaluations

Optimization

Providers

​Optimization Streaming

​Subscribe to Updates

​Event Types

​Event Payloads

​job_started

​iteration_start

​mutation_generated

​evaluation_complete

​new_best

​job_complete

​UI Integration

​React Example

​Vue Example

​Connection Management

​Automatic Reconnection

​Manual Reconnection

​Error Handling

​Best Practices

Optimization Streaming

Subscribe to Updates

Event Types

Event Payloads

job_started

iteration_start

mutation_generated

evaluation_complete

new_best

job_complete

UI Integration

React Example

Vue Example

Connection Management

Automatic Reconnection

Manual Reconnection

Error Handling

Best Practices