Skip to main content

​
Optimization Jobs

An optimization job runs multiple mutation-evaluation cycles to improve a prompt. This guide covers job configuration, lifecycle management, and result handling.

​
Creating a Job

# Start an optimization job
mutagent prompts optimize start 123 \
  --dataset 456 \
  --max-iterations 10 \
  --model claude-sonnet-4-6

# With target score and patience
mutagent prompts optimize start 123 \
  --dataset 456 \
  --max-iterations 15 \
  --target-score 0.9 \
  --model claude-sonnet-4-6

​
Configuration Options

OptionTypeDefaultDescription
maxIterationsnumber10Maximum optimization cycles (1-100)
targetScorenumber—Stop early when this score is reached (0.0-1.0)
patiencenumber—Stop after N iterations without improvement (1-50)
modelstring—LLM model to use for mutation and evaluation
dryRunbooleanfalseTest mode with mock LLM calls
tuningParamsobject—Additional tuning parameters

​
Configuration Examples

Conservative optimization: 
{
  "maxIterations": 20,
  "patience": 5,
  "model": "claude-sonnet-4-6"
}
Aggressive optimization: 
{
  "maxIterations": 10,
  "targetScore": 0.95,
  "model": "claude-sonnet-4-6"
}
Dry run (testing): 
{
  "maxIterations": 3,
  "dryRun": true
}

​
Job States

Jobs progress through these states:
StateDescriptionTransitions
queuedWaiting to start-> running, cancelled
runningActively optimizing-> completed, paused, failed, cancelled
pausedTemporarily stopped-> running, cancelled
completedSuccessfully finishedTerminal
failedError occurredTerminal
cancelledManually stoppedTerminal

​
Managing Jobs

​
Check Status

# Get job status
mutagent prompts optimize status <job-id>

​
List Jobs

const jobs = await client.optimization.listOptimizations({
  status: 'running',
  limit: 20,
});

jobs.data.forEach(job => {
  console.log(`${job.id}: ${job.status} - Score: ${job.currentScore}`);
});

​
Pause a Job

Temporarily stop a running job (can be resumed later): 
await client.optimization.pauseOptimization({ id: jobId });
Pausing preserves the current best prompt and all progress. The job can be resumed from where it left off.

​
Resume a Job

Continue a paused job: 
await client.optimization.resumeOptimization({ id: jobId });

​
Cancel a Job

Permanently stop a job (cannot be resumed): 
await client.optimization.cancelOptimization({ id: jobId });
Cancellation is permanent. If you might want to continue later, use pause instead.

​
Getting Results

Retrieve results when a job completes: 
# Get optimization results
mutagent prompts optimize results <job-id>

​
Job Response Structure

interface OptimizationJobResponse {
  id: string;                        // UUID
  promptId: number;                  // Source prompt ID
  promptGroupId: string;             // Prompt group UUID
  datasetId: number;                 // Dataset used for evaluation
  status: string;                    // Job state
  executionMode: 'worker_loop' | 'bun_worker';
  config: Record<string, unknown>;   // Job configuration
  progress: number;                  // Completion percentage
  currentIteration: number;          // Current iteration
  maxIterations: number;             // Maximum iterations
  currentStage: string | null;       // Current optimization stage
  currentScore: number | null;       // Latest score
  bestScore: number | null;          // Best score achieved
  bestIteration: number | null;      // Iteration of best score
  resultPromptId: number | null;     // ID of optimized prompt (on completion)
  error: string | null;              // Error message (on failure)
  createdAt: string;                 // Job creation timestamp
  startedAt: string | null;          // Execution start time
  completedAt: string | null;        // Completion time
}

​
Applying Results

When optimization completes, it automatically creates a new prompt version with the optimized content. The resultPromptId field points to this new version: 
const status = await client.optimization.getOptimization({ id: jobId });

if (status.status === 'completed' && status.resultPromptId) {
  console.log('Optimized prompt created:', status.resultPromptId);
  console.log('Best score:', status.bestScore);

  // The new prompt version is already linked to the same prompt group
  // and marked as the latest version
}

​
Monitoring Progress

​
Polling

Check status periodically via CLI or SDK: 
# CLI polling
watch -n 5 mutagent prompts optimize status <job-id>

async function waitForJob(jobId: string) {
  while (true) {
    const job = await client.optimization.getOptimization({ id: jobId });

    console.log(`Iteration ${job.currentIteration}/${job.maxIterations}`);
    console.log(`Current: ${job.currentScore} | Best: ${job.bestScore}`);

    if (['completed', 'failed', 'cancelled'].includes(job.status)) {
      return job;
    }

    await new Promise(r => setTimeout(r, 5000));
  }
}
Use WebSocket streaming for real-time updates. See Streaming for full details.

​
Best Practices

Optimization is only as good as your test cases. Ensure your dataset is representative and well-designed before optimizing.
A target score of 1.0 is rarely achievable. Set targets based on your baseline and acceptable quality levels.
Set patience (e.g., 3-5) to avoid wasting iterations when the optimizer has converged.
After optimization completes, review the optimized prompt to ensure it maintains the intended behavior and variable structure.
Due to the stochastic nature of optimization, running multiple jobs and comparing results can yield better outcomes.

​
Troubleshooting

Check provider configuration and rate limits. Jobs queue when resources are constrained. Verify you have a configured provider in Settings > Providers.
The prompt may be near optimal for the given dataset. Try a different model, adjust the dataset, or review the evaluation criteria.
Free-tier workspaces have a limited number of optimization iteration-runs. The error message shows your usage and limit. Upgrade to increase your limit.
Check the error field in the job status. Common causes: provider API errors, invalid prompt variables, or dataset format issues.