Runbook: Eval

Auto-generated from ./src/eval/index.ts. Do not edit by hand — run node scripts/gen-runbooks.mjs.

Import path: personaforge/eval · Public symbols: 0 · Guide: /guide/eval

What it is

personaforge/eval is a public entry point of personaforge. Import it directly; you only pull in this feature's code (subpath exports are tree-shakeable and optional native deps load lazily).

Install

npm i personaforge
# or: bun add personaforge · pnpm add personaforge · yarn add personaforge

Import

import 'personaforge/eval';

Public API surface

• No named runtime exports; import for side effects or types.

Minimal use

Real example from the eval guide:

import {
  runBenchmark,
  exactMatchScorer,
  containsScorer,
  wordOverlapScorer,
  rougeLScorer,
  llmJudgeScorer,
  formatBenchmarkReport,
} from 'personaforge/eval';

const report = await runBenchmark({
  name: 'qa-benchmark-v3',
  dataset: [
    { input: 'What is the boiling point of water?', expected: '100°C' },
    { input: 'Who invented the telephone?',          expected: 'Alexander Graham Bell' },
  ],
  // `run` receives the input string as the first argument
  run: async (input) => {
    const result = await agent.run(input);
    return result.text;
  },
  // Built-in scorers take no arguments
  scorers: [
    exactMatchScorer(),
    containsScorer(),
    wordOverlapScorer(),
    rougeLScorer(),
    llmJudgeScorer({ llm, rubric: 'Award full marks for a correct, complete answer.' }),
  ],
  concurrency: 5,
});

console.log(formatBenchmarkReport(report));

Verify it works

• Type-check: npx tsc --noEmit resolves personaforge/eval with no missing-module error.
• Runtime: node -e "import('personaforge/eval').then(m => console.log(Object.keys(m)))" lists the exports above.
• Behavior: follow the runnable example in /guide/eval.

Common failures

• Cannot find module 'personaforge/eval' — package not installed or stale build; run npm i personaforge and rebuild.
• Cannot find module '<peer>' at call time — this feature lazy-loads an optional native/SDK dep; install the one named in the error.
• Type errors after upgrade — check CHANGELOG.md for the symbol you import; names above are the current contract.

Rollback

• Remove the import and the feature is gone from your bundle (subpaths are isolated; nothing else depends on importing it).
• Pin a known-good version: npm i personaforge@<version>.

Related

• Full index: /runbooks/ · llms.txt
• Concept guide: /guide/eval