Pack local or remote codebases into AI-friendly files that LLMs and coding agents can read or search
Nothing to run from here
Eval Runner is installed from its publisher's own source and answers where it runs, so this marketplace is not in the path of a single call. There is no address here to send one to, and a panel that pretended otherwise would be showing you an answer we made up. Install it and call it from your own client — the Installation tab has the entry for each one.
93/100
trust score
3
downloads a day
local
runs on your machine
140
downloads in 30d
What it does
A standardized testing harness for MCP servers and agent workflows
Quickstart
# 1 — add it to Claude Code
claude mcp add --scope user --env 'YOUR_API_KEY=${YOUR_API_KEY}' --transport stdio eval-runner-mcp '--' npx -y mcp-eval-runner
# 2 — ask your agent for something these tools do
# run_suite, run_case, list_cases and 7 more
It needs YOUR_API_KEY: this reads it from your environment, so export it first.
Eval Runner costs nothing on mcprush: there is no plan to choose here, no cap to set and nothing mcprush can bill you for.
Publishernot claimed
DBAll 6 servers
dbsectrainer6 servers
5 categories
Collected from a public index. Nobody has claimed this account, so nothing here was written by its author — claim it if it is yours.
Installation
Where are you running it?
Every route below installs the same thing and ends at the same approval screen. This one runs on your machine: your client starts Eval Runner as a process under your own user, with your files and your network, so the tool surface below is what it can do to your computer rather than to a server somewhere else. Its scan result is on the Security tab — read the surface before you approve it.
Claude Codedeclared by this listing
This is a public server: you run it yourself and this marketplace is not in the path. You need Node.js — the current LTS (24) is the safe choice; npx comes with it. Claude Code registers it in one command; --scope user makes it available in every project. Claude Code reads YOUR_API_KEY (Your API key for the service) from your environment each time it starts the server, so the value stays out of its config: export it in your shell profile (macOS, Linux), or run setx YOUR_API_KEY … and open a new terminal (Windows). The '--' is in quotes so that PowerShell passes it on.
claude mcp add --scope user --env 'YOUR_API_KEY=${YOUR_API_KEY}' --transport stdio eval-runner-mcp '--' npx -y mcp-eval-runnerReconnect, or start a new session, and the tools appear in the model’s tool list.
Free on mcprush, and it runs from its publisher’s own package: this marketplace is not in the path, so there is no key of ours to hold and no card to add.
What it asks of you
It reaches a system of yours, so it needs your own credential rather than the publisher’s: YOUR_API_KEY — Your API key for the service. You set it in your own client’s config, on your machine — this marketplace never holds it.
What it writes
One config entry your client uses to start the process locally. A local server runs with your file system and your network, which is why it is priced without metering.
What you approve
10 tools, with what each one reads, writes and reaches shown before you agree — the same list on every route above. Read the tool surface.
Tool surface
What the model actually sees. Descriptions are diffed on every release — see version history.
No parameter schema on file.
No parameter schema on file.
No parameter schema on file.
No parameter schema on file.
No parameter schema on file.
No parameter schema on file.
No parameter schema on file.
No parameter schema on file.
No parameter schema on file.
No parameter schema on file.
Free
$0 forever
- Every tool, no call limit
- No card, and mcprush charges nothing for it
- Source published under MIT
- Runs on your machine — nothing of it reaches our gateway
- Nothing to cap, because mcprush bills nothing
What counts against your monthly calls
| Tool | Unit | Calls used | Out of the allowance |
|---|
Nothing here is billable: mcprush charges nothing to install Eval Runner or to call it, at any volume.
Call allowance · this installenforced at the gateway
Monthly allowance, in calls
—
—
Alert at 80% · then the tool returns a normal MCP error
Used this month: —
ATrust score 93 / 100
Release 1.0.0 · scanned 14 Oct 2026
Two independent axes, because powerful and malicious are different questions. The grade is threat only. The capability level is blast radius, and it is never a penalty on the grade — it is priced as one subtract-only term in the score, where you can see it.
Threat score100
Client score93
Findings1 threat · 0 capability
Capabilityhigh
Coveragesource
Verificationrepo
Surface digest4afdeef94935a653cdd5bc172b8d1c76ffac80a26938034cf77dadc920e168fb
Score vectorthreat 100 → client 93
| Term | Level | What it prices | Points |
|---|---|---|---|
| capability-exposure | high | capability blast radius (high) — client exposure if the model is manipulated | −6 |
| verification-discount | repo | publisher verification (public source) — no provenance, but the source is public and inspectable | −1 |
| coverage-honesty | source | inspection depth (source) — how much of the target the scan could see | −0 |
score = clamp(round(threatScore − E_cap − E_ver − E_cov), 0, 100). No model, no opinion, no network call at scoring time: the same methodology version against the same bytes always produces this number, and every line above is a field of the report rather than a summary of it.
What the scan could actually read
A grade is only as meaningful as its coverage, so the scanner publishes its own depth before it publishes its result.
Tool surfaceread
Implementation sourceread
Package metadataread
Live transportnot available
Tools were statically extracted from the published source (10 recovered), not enumerated from a running server. Tool-poisoning, Unicode-smuggling, capability and toxic-flow analysis ran on this inferred surface, but a mis-parsed registration could be missed or mis-attributed, so tool-derived findings are capped below “confirmed”. To grade the real runtime surface, scan the running server: --command "npx -y <package>".
Capability — what it could do if the model were manipulated
Tags derived from each tool’s schema and the implementation, not from what the tool calls itself. high is the level these add up to.
| Tool | Capability tags | Why the tag was assigned |
|---|---|---|
| run_suite | state-write | state-write: declares readOnlyHint: false |
| run_case | state-write | state-write: declares readOnlyHint: false |
| list_cases | no tags | |
| create_test_case | state-write | state-write: declares readOnlyHint: false |
| regression_report | no tags | |
| compare_results | no tags | |
| generate_html_report | no tags | |
| scaffold_fixture | state-write | state-write: declares readOnlyHint: false |
| evaluate_deployment_gate | no tags | |
| discover_fixtures | no tags |
Toxic-flow graph
The lethal trifecta, checked as a graph rather than as a checklist: untrusted input, a sensitive source and an external sink have to meet before there is a path worth worrying about.
The public result for this release does not print the flow graph, so there is nothing to show here. That is not the same as "no paths were found": what the scan did read is above, under coverage.
Supply chain and provenance
Integrityfirst-seen
Surface digest4afdeef94935a653
Scanned2026-10-14
This is the first scan of this surface here, so there is nothing yet to compare it against.
Every result on this tab comes from one deterministic pass over the published package — offline, rule by rule, and auditable line by line above. Same methodology version, same bytes, same score.
Release history
Always latest — the install command names no version, so it installs the newest release when it runs; the current release on file is 1.0.0. Pin a release below and the command asks for exactly that one.
1.0.0
undated
latestNo release note was published with this version.
—
no reviews yet
Who can review
One review per account, from a signed-in account that does not publish this listing. Nothing else is asked — you can say what you think before you install it. Publishers can reply once per review, and a review can be edited or deleted by whoever wrote it.
Sign in to write one
Any signed-in account that does not publish Eval Runner can review it, once — before installing it or after. A review from an account that has installed it is marked as one.
No reviews yet
Nobody has reviewed this listing. The rating on the card is the mean of the reviews written here and nothing else, so there is no rating until somebody writes the first — which takes a signed-in account that does not publish it, and nothing else.