AI Socratic Dialogue Designer Agent Skill

Design a multi-round questioning sequence for interrogating AI chatbot answers, tracking how responses shift and distinguishing genuine…

Context cost
6.5k tokensestimated from the bundle, loaded when it triggers
Bundle
1 file · 26.2 kBtext throughout, nothing executable
Licence
Source-availablethe author’s own terms
Last change
—v1.0
Servers it uses
Noneruns standalone

What it does

Design a multi-round questioning sequence for interrogating AI chatbot answers, tracking how responses shift and distinguishing genuine updates from sycophantic capitulation. Use when teaching students to probe AI critically.

What this skill changes about the agent has not been set out here as a list. Its SKILL.md, shown below, says it in the author’s own words.

ai-behaviourai-literacycapitulationcritical-thinkingdocumentationmulti-roundsocratic-questioningsycophancy

The skill itself

A skill is instructions the model reads, so you can see what you are getting: below is the start of its SKILL.md. The front matter loads with every session, the body loads when the skill triggers, and the whole file comes with the install.

SKILL.md26.2 kB · preview
--- # AGENT SKILLS STANDARD FIELDS (v2) name: ai-socratic-dialogue-designer disable-model-invocation: false user-invocable: true effort: medium # EXISTING FIELDS skill_id: "ai-literacy/ai-socratic-dialogue-designer" skill_name: "AI Socratic Dialogue Designer" domain: "ai-literacy" version: "1.0" contributor: "Gareth Manning" evidence_strength: "moderate" evidence_sources: - "Paul & Elder (2008) — The Miniature Guide to Critical Thinking Concepts and Tools" - "Walsh & Sattes (2005) — Quality Questioning: research-based practice to engage every learner" - "Nystrand et al. (1997) — Opening Dialogue: understanding the dynamics of language and learning in English classrooms" - "Perez et al. (2022) — Sycophancy to Subterfuge: investigating reward tampering in language models" - "Wei et al. (2022) — Chain-of-Thought Prompting Elicits Reasoning in Large Language Models" input_schema: required: - field: "interrogation_topic" type: "string" - field: "student_level" type: "string" optional: - field: "subject_area" type: "string" - field: "rounds" type: "integer" - field: "capitulation_focus" type: "string" - field: "discussion_format" type: "string" output_schema: type: "object" fields: - field: "question_sequence" type: "array" - field: "answer_drift_tracker" type: "object" - field: "capitulation_taxonomy" type: "object" - field: "facilitation_notes" type: "string" - field: "debrief_guide" type: "object" chains_well_with: - "socratic-questioning-sequence-generator" - "ai-output-critical-audit-designer" - "critical-thinking-task-designer" teacher_time: "4 minutes" tags: ["AI-literacy", "Socratic-questioning", "sycophancy", "AI-behaviour", "critical-thinking", "multi-round", "capitulation"] ---
70# AI Socratic Dialogue Designer
71
72## What This Skill Does
73
74Generates a multi-round questioning sequence specifically designed for interrogating AI chatbots — probing their answers through iterative questioning, tracking how their responses shift across rounds, and teaching students to distinguish genuine logical concession (the AI updates because a new argument is logically compelling) from sycophantic capitulation (the AI agrees because it is trained to defer to user pushback). This addresses a fundamental asymmetry between AI Socratic dialogue and human Socratic dialogue: AI systems are trained to be helpful and agreeable, which means they will often revise their answers in response to user pushback regardless of whether the pushback is logically valid. A student who pushes back on an AI answer and receives an updated, more agreeable response may conclude that persistence equals correctness — a false inference with significant implications for how they evaluate evidence. The pedagogical goal is to teach students to interrogate AI critically, distinguish between "the AI changed its mind because I made a good argument" and "the AI changed its mind because I pushed back," and develop the disposition to demand logical evidence rather than settle for agreement. The output includes a multi-round questioning sequence using Paul & Elder's question types adapted for AI, an answer drift tracker protocol, a capitulation taxonomy, facilitation notes, and a debrief guide.
75
76## Evidence Foundation
77
78Paul & Elder (2008) classified Socratic questions into six types: clarification, probing assumptions, probing reasons and evidence, viewpoints and perspectives, implications and consequences, and questions about the question. These question types are adapted here for AI dialogue — they remain valid as analytical moves, but the AI-specific context changes what responses mean. Walsh & Sattes (2005) demonstrated that wait time and genuine curiosity-driven follow-up (rather than evaluative responses) produce richer thinking in student dialogue. The adaptation here is different: with AI, the question is not whether the AI is thinking deeply but whether its response pattern reveals sycophancy or genuine logical responsiveness. Nystrand et al. (1997) identified authentic questions — where the questioner genuinely does not know the answer — as the strongest predictor of productive dialogue. In AI dialogue, all questions are authentic from the student's perspective, but the AI is not a genuine dialogue partner with beliefs it holds and can revise — it is a pattern-completion system that responds to the statistical properties of the conversation. Perez et al. (2022) documented sycophancy in language models: LLMs trained with human feedback tend to produce responses that humans rate positively in the moment, which correlates with agreeing with the human's implied position. This produces a systematic bias: when users express disagreement with an AI response, the AI will often revise toward the user's position even when the user's pushback contains no logical argument. Wei et al. (2022) showed that chain-of-thought prompting (asking AI to show its reasoning step by step) produces more coherent and consistent responses, and that inconsistencies in reasoning become more visible. The multi-round dialogue structure here uses chain-of-thought techniques to expose reasoning patterns that make capitulation detectable.
79
80## Input Schema
81
82The teacher must provide:
83- **Interrogation topic:** The AI claim to probe. *e.g. "AI's claim that nuclear energy is safer than renewable energy" / "An AI explanation of why homework improves learning" / "AI's assertion that social media is primarily harmful to teenagers" / "AI's summary of the causes of WWI, which oversimplifies the role of German aggression"*
84- **Student level:** Year group and questioning experience. *e.g. "Year 12, familiar with Socratic method from Philosophy class" / "Year 10, basic questioning skills"*
85
86Optional (injected by context engine if available):
87- **Subject area:** The discipline
88- **Rounds:** Number of questioning rounds
This is a preview of SKILL.md. Download or install the skill to get the whole thing.
In the file
SKILL.md3,874 words
Files1
LicenceSource-available
Why you can read it

Nothing in a skill executes. The client loads the text and the model follows it, so a skill can be audited the way a runbook is — by reading it.

What it costs in context

Skills are not billed by the call. They are paid for in context: every token the instructions occupy is a token your code, your diff and your conversation cannot use. Here is what this one takes and when it takes it.

≈830
always loaded
The name and description, so the model knows the skill exists and when to reach for it.
5,720
on trigger
The instruction body, read only when the skill fires.
3.3%
of a 200k window
Ten skills this size would take about 33% of the window before you open a file.
050k100k150k200k context window

6.5k tokens, estimated from the bundle at four bytes to the token, held for the rest of the session once it triggers. Heavy. Teams tend to install this one per project rather than globally, and load it only when the job comes up.

Servers bill, skills cost

A server charges by the month. A skill charges once per session, in context, and then keeps charging it for as long as the session lives.

Before and after

The same question, put to the same model twice: once as it comes, and once with these instructions loaded.

No worked example has been published for this skill yet.

The procedure it runs

The procedure has not been set out here step by step. It is in the body of its SKILL.md: the start of it is shown above, and the whole file comes with the install.

Prose, not code

These steps are written for a model to follow, not executed by a runtime. It can still be told to skip one, and it will say so when it does.

Servers it uses

None. This skill calls no MCP servers at all.

Everything it needs is in the instructions, so it works in a project with nothing connected — the model reads the file and changes how it works with what it can already reach.

Its SKILL.md does not declare which tools it asks the agent for, so what it touches is whatever the instructions themselves ask for — read them before you install it.

What it asks for
Writes filesnot declared
Network accessnot declared

This SKILL.md carries no allowed-tools line, so neither is declared. A skill grants no permissions of its own — it can only ask for tools your client already has.

What it will not do

Every skill is narrow, and the useful ones say where they stop. These are the jobs this one is the wrong tool for.

What this skill is not for has not been published here. Nothing is implied by that: it is a section the author has not filled in.

What is in the bundle

1 file, 26.2 kB on disk. A bundle is text throughout: the instructions the model reads, plus the templates it fills in.

  • SKILL.md26.2 kB
What is not in it

No dependencies and nothing executable: a skill is text the agent reads, so the bundle is 1 file you can review in full before installing. The Source-available licence covers the templates and examples as well as the instructions.

Install

Installing copies the bundle into your project. Nothing runs at install time — the files sit on disk until the model reads them.

# AI Socratic Dialogue Designer · 6.5k tokens when loaded · macOS, Linux mkdir -p .claude/skills && t=$(mktemp -d) && curl -fsSL -o "$t/bundle.tar.gz" https://mcprush.com/api/skills/ai-socratic-dialogue-designer/bundle.tar.gz && tar -xzf "$t/bundle.tar.gz" -C "$t" && [ -d "$t/ai-socratic-dialogue-designer" ] && { [ ! -e .claude/skills/ai-socratic-dialogue-designer ] || { mv .claude/skills/ai-socratic-dialogue-designer "$t/.previous" && echo "Previous copy moved to $t/.previous"; }; } && mv "$t/ai-socratic-dialogue-designer" .claude/skills/ && echo "Installed .claude/skills/ai-socratic-dialogue-designer"
On Windows, or any OS with Node 20+
npx mcprush@latest skill add garethmanning/ai-socratic-dialogue-designer

Windows PowerShell says 'running scripts is disabled on this system'? Run Set-ExecutionPolicy -Scope CurrentUser RemoteSigned once, or type npx.cmd instead of npx (codex.cmd, gemini.cmd, grok.cmd likewise). PowerShell 7 needs neither. Writes to .claude/skills/ai-socratic-dialogue-designer/ in the current project. For every project instead, run mkdir -p ~/.claude/skills && t=$(mktemp -d) && curl -fsSL -o "$t/bundle.tar.gz" https://mcprush.com/api/skills/ai-socratic-dialogue-designer/bundle.tar.gz && tar -xzf "$t/bundle.tar.gz" -C "$t" && [ -d "$t/ai-socratic-dialogue-designer" ] && { [ ! -e ~/.claude/skills/ai-socratic-dialogue-designer ] || { mv ~/.claude/skills/ai-socratic-dialogue-designer "$t/.previous" && echo "Previous copy moved to $t/.previous"; }; } && mv "$t/ai-socratic-dialogue-designer" ~/.claude/skills/ && echo "Installed ~/.claude/skills/ai-socratic-dialogue-designer" (macOS, Linux) — or add --global to the npx line. The curl line replaces .claude/skills/ai-socratic-dialogue-designer/ only after the download succeeds, so files left there by another skill of the same name, or by an older release, do not mix in. A copy already there is moved aside, not deleted, and the line prints where.

Which clients pick it up on their own

A skill is a folder of text. Most of these read a skills folder and pick it up without being told; the rest take the same folder another way — an upload in the app, a parameter on an API call — and then choose it by its description just the same.

Claude Code.claude/skills/
Claude DesktopCustomize › Skills
ChatGPTSkills › Create › Upload
Cursor.cursor/skills/
VS Code.github/skills/
Codex CLI.agents/skills/
Gemini CLI.gemini/skills/
Grok Build.grok/skills/
Microsoft CopilotSkills › Upload a skill
PerplexityComputer › Skills › Upload a skill
DeepSeek Harness.dsh/skills/
Zed.agents/skills/
Devin Desktop.devin/skills/
Windsurf (legacy).windsurf/skills/
OpenAI Agents SDKInstructions or sandbox Skills
HTTP / APIa request parameter
This release
Version1.0
Publishedno release date on file
PriceFree
Referencegarethmanning/ai-socratic-dialogue-designer

Versions

v1.0 is what is on the shelf; no release here carries a date. Instructions change more often than APIs do — a skill can be rewritten entirely without anything it depends on moving.

v1.0
  • No earlier releases have been published to the marketplace.
Pinning

The install command always fetches the release that is current the day you run it — v1.0 today — and cannot ask for an earlier one. The copy on your disk does not change until you run it again.

Reviews

—no reviews yet

Nobody has reviewed this skill yet. The rating is the mean of the reviews written here, so there is none until somebody writes the first.

Sign in to review it

A review is tied to an account so it can be answered, edited and taken down by the person who wrote it.

Sign in
Who can post

Anybody with an account, one review each. It carries the version that was current when you wrote it, and the publisher may reply once.

Publisher
Servers0
Claim this skill