Agents Index

Nonobench

Online

An open-source benchmark of how well LLMs solve nonogram puzzles, from 5x5 to 20x20.

Visit publisherConnectJSON

What it can do: 11 tools

  • get_leaderboard read-onlyModels ranked by accuracy; defaults to all effort levels for compatibility.
  • list_providers read-onlyProvider ids, names, families and variant counts.
  • list_families read-onlyModel families, available efforts and best variants.
  • compare_models read-onlySide-by-side core overall and per-size accuracy, cost, latency and token results for model or family names.
  • get_model_results read-onlyAccuracy, cost, latency and token use for one model, broken down by grid size.
  • list_puzzles read-onlyThe benchmark puzzles with their ids and row/column clues.
  • get_puzzle read-onlyOne puzzle, including the clue text models were prompted with. The reference solution is only included on request; some puzzles have several valid solutions.
  • check_solution read-onlyCheck a grid against a puzzle's clues, using the same rule as the benchmark grader. Reports which rows and columns do not match.
  • get_puzzle_results read-onlyPer-model outcomes for one puzzle. Answers are omitted unless requested.
  • get_model_puzzles read-onlyWhich puzzles one model solved, missed, timed out on, or has not run.
  • list_runs read-onlyIndividual benchmark runs (one model on one puzzle), optionally with the raw prompt and model output.

Connect it

Claude Code
claude mcp add --transport http nonobench https://www.nonobench.com/mcp
Claude.ai / ChatGPT
Settings → Connectors → Add custom connector
URL: https://www.nonobench.com/mcp
Cursor (~/.cursor/mcp.json)
{
  "mcpServers": {
    "nonobench": {
      "url": "https://www.nonobench.com/mcp"
    }
  }
}
VS Code (.vscode/mcp.json)
{
  "servers": {
    "nonobench": {
      "type": "http",
      "url": "https://www.nonobench.com/mcp"
    }
  }
}

Questions

What does Nonobench do?

An open-source benchmark of how well LLMs solve nonogram puzzles, from 5x5 to 20x20. It exposes 11 tools to AI assistants.

What tools does Nonobench provide?

11 tools, including get_leaderboard, list_providers, list_families, compare_models, get_model_results, list_puzzles, get_puzzle, check_solution, get_puzzle_results, get_model_puzzles, list_runs.

How do I connect Nonobench to Claude, ChatGPT or Cursor?

Add the remote MCP endpoint https://www.nonobench.com/mcp as a custom connector or MCP server. Copy-paste setup is on this page.

Does Nonobench require an account or API key?

No sign-in was needed to list its tools. Individual tools may still need an account or API key.

Is Nonobench online?

As of 2026-11-19 our probe found it online. It answered 100% of our last 3 checks.

Who publishes Nonobench?

@mauricekleine on GitHub. The official MCP Registry verified that the publisher controls this GitHub account before accepting the listing.

Similar in Agents & Orchestration