RAG Eval Agent Skill
Filesystem RAG benchmarks: corpus/, train.json, evaluate_rag.py (RAGAS quality) for a deployed NVIDIA RAG Blueprint.
- Context cost
- 10.9k tokensestimated from the bundle, loaded when it triggers
- Bundle
- 9 files · 43.8 kBtext throughout, nothing executable
- Licence
- Apache-2.0the author’s own terms
- Last change
- —v2.6.0
- Servers it uses
- Noneruns standalone
What it does
Filesystem RAG benchmarks: corpus/, train.json, evaluate_rag.py (RAGAS quality). Not for prod monitoring, latency/throughput benchmarking (use rag-perf), or evals outside this repo layout.
Installed, it changes the agent in these ways.
What this skill changes about the agent has not been set out here as a list. Its SKILL.md, shown below, says it in the author’s own words.
And these are the jobs people reach for it for.
The skill itself
A skill is instructions the model reads, so you can see what you are getting: below is the start of its SKILL.md. The front matter loads with every session, the body loads when the skill triggers, and the whole file comes with the install.
Nothing in a skill executes. The client loads the text and the model follows it, so a skill can be audited the way a runbook is — by reading it.
What it costs in context
Skills are not billed by the call. They are paid for in context: every token the instructions occupy is a token your code, your diff and your conversation cannot use. Here is what this one takes and when it takes it.
10.9k tokens, estimated from the bundle at four bytes to the token, held for the rest of the session once it triggers. Heavy. Teams tend to install this one per project rather than globally, and load it only when the job comes up.
A server charges by the month. A skill charges once per session, in context, and then keeps charging it for as long as the session lives.
Before and after
The same question, put to the same model twice: once as it comes, and once with these instructions loaded.
No worked example has been published for this skill yet.
The procedure it runs
The procedure has not been set out here step by step. It is in the body of its SKILL.md: the start of it is shown above, and the whole file comes with the install.
These steps are written for a model to follow, not executed by a runtime. It can still be told to skip one, and it will say so when it does.
Servers it uses
None. This skill calls no MCP servers at all.
Everything it needs is in the instructions, so it works in a project with nothing connected — the model reads the file and changes how it works with what it can already reach.
It asks the agent to write files, using whatever file access your client already has. Whether it reaches the network is not declared.
Read from the allowed-tools line of this skill’s own SKILL.md. It also asks for Bash, which could do either, so what is not named there is not ruled out. A skill grants no permissions of its own — it can only ask for tools your client already has.
What it will not do
Every skill is narrow, and the useful ones say where they stop. These are the jobs this one is the wrong tool for.
What this skill is not for has not been published here. Nothing is implied by that: it is a section the author has not filled in.
What is in the bundle
9 files, 43.8 kB on disk. A bundle is text throughout: the instructions the model reads, plus the templates it fills in. 1 file of its source is not in the download — listed after these, with why.
- SKILL.md9.4 kB
- BENCHMARK.md2.9 kB
- skill-card.md2.7 kB
- eval/h100.json2.9 kB
- eval/nvidia_hosted.json2.0 kB
- references/benchmark-execution.md8.3 kB
- references/dataset-and-conversion.md8.4 kB
- references/evaluate-rag-cli.md4.3 kB
- references/result-analysis.md2.9 kB
Not in the download
Not carried is a type a skill download does not hold. Take it from its source.
- skill.oms.signot carried
No dependencies and nothing executable: a skill is text the agent reads, so the download is 9 files — not the whole of its source: 1 file is left out. It is listed under the files, with why. The Apache-2.0 licence covers the templates and examples as well as the instructions.
Install
Installing copies the bundle into your project. Nothing runs at install time — the files sit on disk until the model reads them.
# RAG Eval · 10.9k tokens when loaded · macOS, Linux
mkdir -p .claude/skills && t=$(mktemp -d) &&
curl -fsSL -o "$t/bundle.tar.gz" https://mcprush.com/api/skills/rag-eval/bundle.tar.gz &&
tar -xzf "$t/bundle.tar.gz" -C "$t" && [ -d "$t/rag-eval" ] &&
{ [ ! -e .claude/skills/rag-eval ] || { mv .claude/skills/rag-eval "$t/.previous" && echo "Previous copy moved to $t/.previous"; }; } &&
mv "$t/rag-eval" .claude/skills/ && echo "Installed .claude/skills/rag-eval"npx mcprush@latest skill add nvidia/rag-evalWindows PowerShell says 'running scripts is disabled on this system'? Run Set-ExecutionPolicy -Scope CurrentUser RemoteSigned once, or type npx.cmd instead of npx (codex.cmd, gemini.cmd, grok.cmd likewise). PowerShell 7 needs neither. Writes to .claude/skills/rag-eval/ in the current project. For every project instead, run mkdir -p ~/.claude/skills && t=$(mktemp -d) && curl -fsSL -o "$t/bundle.tar.gz" https://mcprush.com/api/skills/rag-eval/bundle.tar.gz && tar -xzf "$t/bundle.tar.gz" -C "$t" && [ -d "$t/rag-eval" ] && { [ ! -e ~/.claude/skills/rag-eval ] || { mv ~/.claude/skills/rag-eval "$t/.previous" && echo "Previous copy moved to $t/.previous"; }; } && mv "$t/rag-eval" ~/.claude/skills/ && echo "Installed ~/.claude/skills/rag-eval" (macOS, Linux) — or add --global to the npx line. The curl line replaces .claude/skills/rag-eval/ only after the download succeeds, so files left there by another skill of the same name, or by an older release, do not mix in. A copy already there is moved aside, not deleted, and the line prints where. Not in this download: 1 file of its source (which one, and why). Take it from its source.
Which clients pick it up on their own
A skill is a folder of text. Most of these read a skills folder and pick it up without being told; the rest take the same folder another way — an upload in the app, a parameter on an API call — and then choose it by its description just the same.
Versions
v2.6.0 is what is on the shelf; no release here carries a date. Instructions change more often than APIs do — a skill can be rewritten entirely without anything it depends on moving.
- No earlier releases have been published to the marketplace.
The install command always fetches the release that is current the day you run it — v2.6.0 today — and cannot ask for an earlier one. The copy on your disk does not change until you run it again.
Reviews
Nobody has reviewed this skill yet. The rating is the mean of the reviews written here, so there is none until somebody writes the first.
A review is tied to an account so it can be answered, edited and taken down by the person who wrote it.
Sign inAnybody with an account, one review each. It carries the version that was current when you wrote it, and the publisher may reply once.