OpenMark
OnlineBenchmark your own prompt on 100+ LLMs with real API calls; pick the cheapest model that passes.
What it can do: 10 tools
get_task_formatHow to write an OpenMark task: the YAML structure, all 18 scoring modes, and tips. Call this before writing a task. Pass a topic for the full detail on one subject (the app's Task Guide): fields, scoring, composite, chaivalidate_taskCheck a task YAML the way OpenMark will before running it: structure, scoring settings, and the test limit of the user's plan. Free, saves nothing. Returns the problems to fix and warnings worth a look.list_modelsCurrent OpenMark model ids with list prices ($ per 1M tokens), as text and as a structured list. Use these ids in run_benchmark. Append @low, @medium or @high to a reasoning model id to set its reasoning effort.prepare_attachmentsGet an upload_uid and a one-hour upload link (no API key needed) for images, PDFs or text documents, with the exact curl command. Upload the files, reference them by file name in each test's attachments list, then pass testimate_costCredits a run_benchmark call would use, before spending anything: the expected total, the split per model, the balance needed to start, the worst case, and the user's balance. Same inputs and run settings as run_benchmarrun_benchmarkSave a task (YAML, see get_task_format) to the user's OpenMark account and run it on the given models with real API calls. Returns a job_id and the expected cost; poll get_results. Spends the user's credits; pass max_creget_resultsStatus of a benchmark job, and once completed the ranked results (score, stability, cost per run, latency), the tests each model missed with its answer, a recommendation for the chosen objective and quick picks for the oexport_openclaw_csvA completed job's results as the CSV the OpenMark Router plugin for OpenClaw reads (the same file as the app's Export > OpenClaw). Save it in the plugin's benchmarks folder, by default ~/.openclaw/workspace/plugins/openmget_accountThe user's plan, credit balance and limits, and what a credit pack, Pro or Expert would unlock. Use it before large runs, or when a run is refused for credits or limits.watch_taskWatch the task of a completed job: when new or cheaper models ship, OpenMark re-tests up to 4 of them next to the user's pick and emails the user only when one beats it. Each check uses the user's credits at the normal r
Connect it
claude mcp add --transport http openmark https://openmark.ai/mcpSettings → Connectors → Add custom connector
URL: https://openmark.ai/mcp{
"mcpServers": {
"openmark": {
"url": "https://openmark.ai/mcp"
}
}
}{
"servers": {
"openmark": {
"type": "http",
"url": "https://openmark.ai/mcp"
}
}
}Questions
What does OpenMark do?
Benchmark your own prompt on 100+ LLMs with real API calls; pick the cheapest model that passes. It exposes 10 tools to AI assistants.
What tools does OpenMark provide?
10 tools, including get_task_format, validate_task, list_models, prepare_attachments, estimate_cost, run_benchmark, get_results, export_openclaw_csv, get_account, watch_task.
How do I connect OpenMark to Claude, ChatGPT or Cursor?
Add the remote MCP endpoint https://openmark.ai/mcp as a custom connector or MCP server. Copy-paste setup is on this page.
Does OpenMark require an account or API key?
No sign-in was needed to list its tools. Individual tools may still need an account or API key.
Is OpenMark online?
As of 2026-11-19 our probe found it online. It answered 100% of our last 3 checks.
Who publishes OpenMark?
openmark.ai. The official MCP Registry verified that the publisher controls this domain before accepting the listing.
Similar in Agents & Orchestration
-
Upwork MCPOnline · sign-in requiredOfficialPopular
The official Upwork MCP server, letting AI agents connect to Upwork and act on your behalf.
-
Postman MCP ServerOnline · sign-in requiredOfficialPopular
Give your AI agents trusted access to the full Postman platform.
-
OpenWeather for agentsOnlineOfficialPopular
OpenWeather data for AI agents: list feeds, sign up, check balance, get a top-up link, fetch data.
-
Creator MCPOnline · sign-in requiredOfficialNew
Hosted motion engine for agents. Builds layers and keyframes, exports MP4, WebM or Lottie.
-
Agent economy data (counts from the Official MCP Registry)OnlineNew
MCP server counts from the official registry, new servers per day, and MCP/A2A spec changes. As is.
-
Rota NacionalOnlineOfficialNew
AI API compatible with OpenAI and Anthropic, with personal data handled before the model.