Evaluate Codex skills and plugins from chat with a beginner-friendly start command, local-first reports, token budget explanations, and guided benchmarking. Skills for evaluating plugins and skills, improving skills and designing metric packs, with local-first reports and token budget explanations. Listed in OpenAI's curated Codex marketplace (openai-curated); the manifest declares skills.
Also available as
Install
- 1Add it in ChatGPT
Open Plugins in the ChatGPT sidebar (web or desktop), search for “Plugin Eval” and select +. You'll be asked to sign in to any connected service on first use. Then type @ in a chat to call it.
- 2Or install it in Codex
codex plugin add plugin-eval@openai-curatedStart a new Codex session afterwards. You can also run /plugins inside the CLI to browse and install interactively.
Skills 5
- Evaluate Plugin
Evaluate a local Codex plugin
- Evaluate Skill
Evaluate a local Codex skill
- Improve Skill
Turn plugin-eval findings into a rewrite brief
- Metric Pack Designer
Design custom metric packs for plugin-eval
- Plugin Eval
Choose the right local evaluation workflow
Things to ask
- “Give me an analysis of the game studio plugin.”
- “Evaluate this plugin.”
- “Why did this score that way?”
Disconnect or uninstall
- 1In ChatGPT
Disconnect an account in Settings → Apps or Plugins. To remove the plugin itself, open Plugin Eval in the Plugin Directory and select Uninstall plugin. Uninstalling can disconnect every account connected through it, and plugins your workspace installs may not offer the option.
- 2In Codex
codex plugin remove plugin-eval@openai-curated
Learn more
Plugin Eval alternatives
SuperpowersPlan, develop, and debug codeplugin · by Jesse Vincent · 294k stars
ShopifyBuild and manage your storeplugin · by Shopify · 7.2k stars