Skip to content

Repository files navigation

personai

Persona generator for LLM response divergence. Creates randomized agent personas by combining curated sentence-form descriptions across multiple dimensions, then injects them into system prompts to produce measurably different LLM responses.

Install

pnpm install

Usage

Generate personas

# Single persona (printed to stdout)
pnpm personai generate

# Batch of 10
pnpm personai generate -n 10

# Copy to clipboard
pnpm personai generate -c

# JSON output
pnpm personai generate -n 5 --format json

# Reproducible with seed
pnpm personai generate --seed 42

# Write to file
pnpm personai generate -n 10 -o personas.json --format json

Evaluate divergence

Runs the same prompt through Claude with different personas and measures whether personas produce statistically significant response divergence.

# Basic eval (5 personas, 3 trials each, LLM-judge metric)
pnpm personai eval -p "What causes inflation?"

# More personas, more trials
pnpm personai eval -p "What causes inflation?" -n 10 --trials 5

# Fast local metric (no LLM judge calls)
pnpm personai eval -p "What causes inflation?" --metric tfidf

# Verbose (show individual responses)
pnpm personai eval -p "What causes inflation?" --verbose

Exit code 0 = statistically significant divergence (p < 0.05). Exit code 1 = not significant.

How it works

  1. Dimensions — YAML files in data/ define pools of curated sentence-form options (epistemic stances, values, contrarian modes, personalities)
  2. Generation — randomly selects one option per dimension and fills a template to produce a flowing paragraph
  3. Evaluation — generates baseline responses (no persona) and persona responses via claude CLI, then measures semantic divergence using TF-IDF or LLM-judge scoring with permutation testing for statistical significance

Extending

Add new options to any dimension by editing the YAML files in data/. No code changes needed.

Web UI

A Next.js web app under src/app/ exposes the generator in the browser: pick dimensions, generate single or batch personas, copy to clipboard. Run it with:

pnpm dev

Development

pnpm test            # Run tests (Vitest)
pnpm lint            # Biome lint + format check
pnpm dev             # Next.js dev server (web UI)
pnpm personai:build  # Build CLI binary

About

PersonAI - A generator used to create random personas to inject into system prompts in order to introduce divergence in LLM output.

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages