Skip to main content
Stage ③ of the loop · owned by the Evaluator skill · also on npm as @mutagent/evaluator.
Evaluate turns your traces into a trustworthy verdict. It doesn’t ask you to write the eval — it derives the criteria from your own runs, judges each one, and checks the judge against your labels before it trusts it.

​
Run it

Ask for a verdict against your real runs:
Other ways people ask:
  • Score the Deep Research agent’s answers for citation accuracy and flag which criteria fail.
  • Check how often the Refund Processing agent approves disputes that should have been escalated.

​
What it does

1

Derive criteria

Reads your traces and derives binary, actionable success criteria from the runs that worked and the runs that didn’t — no criteria list to hand-write.
2

Judge

Fans out one critique-before-verdict judge per criterion, running in parallel, and scores each run: pass or fail, with a confidence. Each judge reasons before it rules, so every verdict arrives with its rationale.
3

Validate the judge

Measures each judge against your own labels. Below a threshold of labels, a judge is marked unvalidated rather than trusted — you always know which verdicts to believe.
4

Gate

Rolls the per-criterion results into one verdict on the whole run — a critical failure sinks it.

​
Judge only — it never fixes

The evaluator decides; it does not change your agent. Failures route to Diagnose, which proposes the fixes. Separating the grader from the fixer is what keeps the verdict independent of the change it triggers.

​
Use it on its own

The evaluator publishes independently as @mutagent/evaluator, so you can build a trustworthy eval suite without adopting the rest of Helix.

Next: Diagnose

Root-cause what evaluation flagged.