10# PaperJury (CS-conference paper review and editing)
11
12PaperJury edits and hardens any CS-conference paper. It runs in
13three modes. In **direct-edit mode** (the common case) the user describes a change
14in Chinese or English and the LaTeX is edited directly through a CS-venue writing
15toolkit, with author sign-off. In **review mode** (occasional, pre-submission) it
16exposes the manuscript to a harsh, multi-perspective courtroom review engine that
17adjudicates each issue (N holistic domain reviewers -> contestability routing ->
18two-sided trial -> three-way verdict, with a polish track and a clerk-converged
19multi-round loop), gates every change behind consensus, and tracks issues in a durable
20ledger. In **auto mode** (unattended, opt-in via /goal) it runs that same engine
21toward a verifiable goal, applying safe fixes under a drift-bounded policy and
22queueing the risky ones for one human pass on return. All modes share the same
23writing toolkit, hard rules, ledger, and author sign-off (auto via up-front policy
24sign-off plus the queue, see hard rule 1).
25
26This skill is **fully generic**. It ships no hardcoded paths, no project files,
27and no embedded paper. Everything specific to a given paper (where the
28manuscript is, the venue, who signs off, the house style) is resolved at runtime
29or supplied by a config the *project* owns. The skill itself is the backbone;
30any concrete paper is just an instantiation of it.
31
32Scope: CS conferences only. Three venue families, each with its own style profile:
33- **Vision**: CVPR, ICCV, ECCV, WACV
34- **NLP**: ACL, EMNLP, NAACL, COLING
35- **ML**: ICLR, NeurIPS, ICML, AAAI, COLM
36
37## When to use / when not
38
39Three modes, one skill. Pick by what the user is asking for:
40- **Direct-edit mode (the common case).** The user describes a change in Chinese
41 (or English) and wants the LaTeX edited directly: "把这段改成...", "polish this
42 paragraph", "把我对 intro 的想法写成 LaTeX", "tighten this". No review panel; go
43 straight to drafting the patch through the writing toolkit, with author sign-off.
44- **Review mode (occasional, pre-submission).** The user wants the paper critiqued
45 or hardened: review / critique / 审稿 / 评审 / mock-review, or iterating a draft
46 to clear reviewer-raised issues. This runs the courtroom review engine
47 (references/review-engine-v3.md).
48- **Auto mode (unattended).** The user opts in via /goal (or config mode: auto)
49 to run the review-revise loop AFK toward a verifiable goal. Establish the spine
50 up front (the one human step), then the engine applies safe fixes under the
51 bounded-aggressive policy and queues the rest. The drafter input passes the
52 significance floor (node scripts/ledger.js floor: valid-fixable majors only) and
53 the ledger view is initialized collapsed (--display collapse: minors fold into a
54 Minor digest, majors stay itemized). See references/auto-mode.md.
55 Never self-detect auto; it is explicit only.
56
57Do NOT use for: writing a paper from scratch (use ml-paper-writing), figure or
58diagram generation (use academic-plotting), or an official-venue rebuttal (this
59is a pre-submission self-hardening loop, no score gate).
60
61Soft update reminder: at the start of each PaperJury invocation, before choosing
62the mode or editing a manuscript, run node scripts/check-update.js from the
63skill root unless PAPERJURY_DISABLE_UPDATE_CHECK=1 is set. If it reports an
64available update, show the notice once and continue. If the check is skipped,
65silent, or cannot reach GitHub, continue without mentioning it; update checks are
66never allowed to block review or editing.
67
68## The three primitives
69
70This paradigm is expressed as **Skill + Workflow + Memory**. Each carries one
71concern; together they replace the heavy per-round file-and-flag machinery a
72hand-rolled version accumulates.
73
741. **Skill (this folder) = entry point + methodology.** The protocol, the
75 reviewer panel, the contestability routing, the writing toolkit, the human gates.
76 Detail in references/review-engine-v3.md, references/reviewer-personas.md,
77 references/writing-toolkit.md.
782. **Workflow = fan-out engine.** The semantic, no-human-in-the-middle steps run as
79 Workflows (parallelism + schema-validated output by construction). The simple
80 panel is workflows/review-panel.workflow.js; the v3 courtroom engine is
81 assign-reviewers -> reading-check -> coverage-auditor -> merge ->
82 {trial (+ escalate) || polish} -> recall-audit -> drafter ->
83 {edit-audit | meaning-audit} -> clerk. The DETERMINISTIC guards run
84 orchestrator-side via Bash between workflow calls (the Workflow sandbox has no fs):
85 scripts/ holds decompose, extract-docx, ledger, journal, apply-patch,
86 anchor-diff, cross-ref, spine, rekey, compile-guard, compliance-check
87 (plus doctor, the install/repo health check: npm run doctor). Build note: this harness
88 delivers a workflow's args as a JSON STRING, so every workflow parses it
89 defensively. Protocol + every orchestrator seam: references/review-engine-v3.md.
903. **Memory = durable state + learned conventions.** Two layers:
91 - **Ledger** (LEDGER.json resolved at runtime = the machine source of truth,
92 plus a rendered LEDGER.md view; managed by scripts/ledger.js): the live,
93 mutable issue state across rounds and sessions. Schema + status state machine:
94 references/ledger-schema.md.
95 - **Claude memory** (the active project's memory): stable conventions worth
96 recalling next session, e.g. this paper's house style, venue, persona tuning.
97
98## Resolving inputs at runtime (no hardcoded paths)
99
100The skill ships ZERO hardcoded paths or project files. On trigger it resolves
101each input by **discovery first, then asking**:
102
103- **manuscript**: detect the main source, then route it through the INTAKE FORMAT
104 GATE by extension. Four routes, none silent:
105 - .tex: the native LaTeX path. Detect the main source (the .tex with
106 \documentclass / \begin{document}, or the file the user names). If
107 several candidates, ask.
108 - .md / .markdown / .txt: the native text path. The full multi-round
109 engine runs; compile checks are not applicable (compile-guard returns
110 compiled:null plus a markdown sanity lint, an honest UNKNOWN, never a
111 fake pass); LaTeX-only compliance checks are skipped and reported as
112 skipped_checks.
113 - .docx: if a .paper-review/ working copy AND a ledger already exist,
114 REUSE them, never re-extract. If the sha256 of the docx no longer matches
115 the ledger's meta.original_sha256, STOP and ask: continue on the working
116 copy, or extract --force knowingly discarding the applied edits (an
117 explicit new-intake event). Otherwise run
118 node scripts/extract-docx.js extract <file.docx> (one time) and tell the
119 user explicitly: the original Word file is never modified; all rounds run
120 on .paper-review/<basename>.md (print the full working-copy path); they
121 get back the edited Markdown plus a per-edit change list; the extraction
122 report lists everything dropped or degraded. Write ledger meta
123 `{manuscript: <working copy>, working_format: 'markdown', source_format:
124 'docx', original, original_sha256, extracted_at, extraction_report}`. If the
125 report shows nonzero tracked-change counts, seed a round-1 author-required
126 ledger row ("manuscript contains unresolved tracked changes; accepted-all
127 for review").
128 - any other extension (.doc, .pdf, .rtf, .odt, ...): explicitly
129 unsupported. Say so and suggest exporting .docx / .md / .tex; never
130 silently degrade.
131
132 After intake, the working copy IS the manuscript for every rule and gate in
133 this file (sign-off, spine freeze, round-0 baseline, edit safety, journal);
134 the original uploaded file is permanently read-only.
135- **venue_family**: the user can name it, or an agent reads the class file to
136 GUESS the family (e.g. a cvpr/iccv style, an acl style, a neurips/iclr style).
137 There is no hardcoded venue list and no deterministic detector; if unclear, ask.
138- **ledger**: default to <manuscript-dir>/.paper-review/LEDGER.json (the machine
139 source of truth; scripts/ledger.js also renders a LEDGER.md view). Create if
140 absent, reuse if present. The user may point elsewhere.
141- **author**: ask who signs off on edits (default: the current user). Every edit
142 needs explicit authorization.
143- **personas**: default to N domain-expert holistic reviewers assigned at runtime
144 (assign-reviewers, from the project gatekeeper core + a generated domain overlay);
145 the three generic lenses in references/reviewer-personas.md are the degrade
146 fallback. If the project defines its own named reviewer subagents, use them as
147 agentType; otherwise inline the persona prompts.
148- **style_profile**: start from the venue-family default; refine from any
149 conventions recalled from memory or pinned in a project config.
150
151A project MAY pin these by dropping a config in ITS OWN repo (see
152configs/config-template.md for the shape). That file is owned by the project,
153never by this skill. At round start, recall any pinned conventions from memory.
154
155## Direct-edit mode (the common case)
156
157The user states a change in Chinese or English; you draft and apply the LaTeX edit.
158No panel, no ledger, no discussion. Minimal flow:
159
1601. **Locate.** Resolve the manuscript and find the target passage the instruction
161 refers to (a paragraph, sentence, caption, table cell). If it is ambiguous on a
162 large file, ask which passage; do not guess. On a .docx: if a working copy
163 already exists, it IS the manuscript, edit it; if none exists, offer an
164 explicit choice between (a) paste-back, returning the rewritten passage as
165 text for the user to apply in Word (no working copy), and (b) running the
166 one-time intake extraction and editing the working copy. Never edit the
167 .docx file itself.
1682. **Draft.** Pick the writing-toolkit prompt matching the instruction
169 (translate-to-english for a Chinese idea, polish-english / de-ai for a
170 rewrite, compress / expand for length, caption / experiment-analysis
171 for those units) and draft the patch to do exactly what was asked. The Common
172 guards apply (markup-safe for the working format, plain CS prose, no log
173 leakage into the manuscript).
1743. **Self-gate.** Run logic-check on the drafted passage.
1754. **Sign-off.** Show the patch and get explicit author approval (hard rule 1).
1765. **Apply.** Write only the patch into the manuscript; keep any back-translation
177 or note author-side.
178
179This is the writing toolkit used on its own. Escalate to review mode only when the
180user wants the paper critiqued or hardened, not for a single asked-for edit.
181
182## Why fan-out is a Workflow and the rest is conversation
183
184The reviewer panel and the trial jury are pure fan-out: spawn, collect,
185merge. A Workflow does this deterministically (parallelism enforced by
186construction, structured outputs via schema, isolation by default since each
187agent sees only the prompt you give it). That isolation is what replaces the
188snapshot-and-whitelist defense: a reviewer cannot see peers, the ledger, or prior
189rounds because you simply do not put them in its prompt.
190
191But the loop has genuine human gates (the author reviews the issue list, gives
192per-issue direction, authorizes edits, breaks ties). Workflows run to completion
193and return a result; they do not pause mid-run for hours of human input. So:
194
195- fan-out steps (reviewers, trial, polish, recall, merge) -> **Workflow**
196- human gates (per-issue direction, authorization, override) -> **main conversation turns**
197- cross-round truth (the ledger) + stable conventions -> **Memory**
198
199## Review mode: one round, end to end
200
201The full adversarial loop (the v3 courtroom engine). Use it to harden the paper, not
202for a single asked-for edit (that is direct-edit mode). Full protocol + the 14
203orchestrator seams: references/review-engine-v3.md. [WF] = Workflow step,
204[det] = deterministic Node guard run orchestrator-side between workflow calls,
205[HUMAN] = author gate, [LEDGER] = state write.
206
2071. **Resolve + recall.** Resolve the inputs above; recall this paper's conventions
208 from memory. Pick scope: full (whole paper) or passage (one section / para / claim).
2092. **[det] decompose.** Split the manuscript into reading units + stable
210 passage_ids + the canonical section list.
2113. **[WF] assign-reviewers** + **[HUMAN] confirm.** Name N subfields (2-4,
212 default 3); instantiate N holistic domain reviewers from the gatekeeper core + a
213 generated overlay. An unconfirmable slot degrades per slot to a generic gatekeeper
214 (the three generic lenses in reviewer-personas.md are the fallback). The author
215 confirms the assignment (or pins it via config).
2164. **[WF] reading-check.** Each reviewer reads the WHOLE paper → weaknesses
217 {significance(major|minor), kind(mechanical|substantive), verbatim quote —
218 cannot quote = did not read} + one overall_confidence + a per-section coverage
219 report. Anti-skim is three layers: [det] per-section quote-verify, [WF]
220 coverage-auditor, [WF] targeted re-invoke.
2215. **[WF] merge.** Semantic dedup across reviewers; derive significance (MAX) /
222 kind (substantive-dominates) / corroboration. [LEDGER] intake as raised.
2236. **[det] route.** mechanical → polish; substantive&minor → polish;
224 substantive&major → trial (two parallel tracks).
2257. **[WF] trial.** Per substantive-major charge: a whole-paper DEFENSE → 5
226 decorrelated local-context jurors (+ on-demand expansion) → a deterministic verdict
227 (decide iff quorum surviving >= ceil(0.8*jurySize) AND one side > 60% of
228 surviving votes; else escalate to 12). Verdict ∈ {invalid-drop, valid-fixable,
229 author-required, escalate}; the judge sets a close_criterion ONLY for a
230 valid-fixable charge, satisfiable by editing existing text (no new data). [WF]
231 polish runs the off-gate mechanical/minor track in parallel (never silently dropped).
2328. **[WF] recall-audit.** Mode A revives wrongly-dropped charges; Mode B spot-checks
233 strong-consensus majors BEFORE the edit. Runs before the drafter.
2349. **[HUMAN] Authorize + [WF] drafter + edit-safety.** On authorization, the
235 drafter writes the minimal patch per surviving valid-fixable. The edit-safety chain
236 gates it: [det] anchor-diff + cross-ref → [WF] meaning-audit (frozen anchor,
237 four-state) / edit-audit (risky non-anchor); [det] apply-patch + compile-guard land
238 a passing patch and [LEDGER] mark closed; a drift / anchor / failed edit is
239 reverted and queued. Revision logs / back-translations stay author-side.
24010. **[WF] clerk + report.** The clerk reconciles the round boundary (carried
241 open-questions vs this round's edits, via a passage_id + similarity merge key) and
242 emits convergence counts. Summarize new/closed counts with the minor/polish part
243 as a one-line digest (counts), never per-item paragraphs; in review mode do not
244 auto-start the next round (auto mode drives the outer loop via /goal). The
245 rendered LEDGER.md obeys meta.display_mode (flip anytime:
246 node scripts/ledger.js mode <ledger.json> <show|collapse>; review defaults to
247 the flat table, auto initializes collapsed). At round end run
248 node scripts/rekey.js <working file> <ledger> <journal> to re-link open rows
249 whose passage_id no longer resolves after this round's edits (both formats).
250
251GATE: node scripts/ledger.js gate = 0 gate-blocking active major (gate-blocking =
252{raised, in-trial, re-trial, valid-fixable}; author-required / queued / dropped /
253closed are gate-OK and author-required accumulates to the queue). Full protocol +
254ledger schema + status machine: references/review-engine-v3.md,
255references/ledger-schema.md. The legacy single-pass 3-reviewer panel
256(workflows/review-panel.workflow.js, the discussion-mode flow in
257references/methodology.md) is kept only as a quick check.
258
259## Hard rules (load-bearing, venue-agnostic)
260
2611. **Never edit the manuscript without explicit author sign-off.** Auto-mode
262 carve-out: the rule HOLDS; auto satisfies it via UP-FRONT sign-off (the spine
263 confirmation + the pre-authorized bounded-aggressive policy) plus the return
264 queue, not per-edit sign-off. Nothing outside the authorized envelope is applied.
2652. **Reviewers / jurors are isolated.** Fresh eyes per round: no cross-talk, no
266 prior-round leakage, no sight of the ledger. Enforced by (a) what goes into each
267 agent's prompt AND (b) an explicit ISOLATION instruction in every reviewer-type
268 prompt telling the agent to judge only the quoted text and not read files
269 (workflow agents have read tools and will otherwise sometimes roam).
2703. **A valid-fixable issue carries a close_criterion** (one concrete sentence an
271 edit must satisfy), set by the judge at trial; it is null at intake.
2724. **No leakage into the reviewed text.** Revision logs, back-translations, and
273 self-check verdicts are author-side aids; they never enter the manuscript or
274 any frozen snapshot.
2755. **Disagreement resolves through discussion, then override** (logged), never a
276 silent dismissal.
2776. **No hardcoded paths or project files in the skill.** Resolve at runtime.
278
279## Memory convention
280
281- At round start: recall the paper's conventions (house style, venue, persona
282 tuning) from memory; read the resolved LEDGER.json for open issues.
283- During the round: the ledger is the only mutable truth; update it at merge,
284 trial verdicts, recall, and close.
285- After the round: persist any newly learned stable convention to memory (e.g. a
286 house-style rule a reviewer surfaced), not the transient issue state.
287
288## Maximizing it under ultracode
289
290The fan-out engine implements the strong form directly
291(workflows/review-panel.workflow.js):
292
293- **loop-until-dry**: re-runs independent fresh panels and accumulates only issues
294 not seen before, stopping after dryStop consecutive passes that add no
295 surviving issue (hard cap maxRounds). Raises recall past a single pass.
296- **adversarial verify**: each new issue faces perspective-diverse skeptics
297 (misreading / already-addressed / scope-or-severity) and is kept unless a
298 majority refute it, filtering plausible-but-wrong issues before they reach the
299 ledger. Bias is to keep, so real flaws are not lost.
300
301Toggle via args: ultracode on -> defaults (maxRounds 4, dryStop 2,
302verify true); ultracode off -> pass {maxRounds:1, verify:false} for the basic
303single-panel form. The loop is budget-aware and stops early if the token budget
304runs low.
305
306
307
308## Capabilities and status
309
310Built: the review engine; the submission-readiness checker (deterministic desk-reject screening plus a real LaTeX compile, degrading to a structural lint when no toolchain is present); auto mode (the review-revise loop toward a goal under a drift-bounded policy, applying safe fixes and queueing risky ones for author review); and the significance floor (ledger.js floor gates the drafter to valid-fixable majors; the collapsed ledger view folds minors into a digest so trivia never floods the author's attention -- render-only, full detail kept in LEDGER.json). Roadmap: vision-based layout verification, automatic venue detection from the class file, and reviewer personas tuned to each venue community.
311
312## Related skills
313
314- ml-paper-writing: from-scratch drafting, citation verification (never
315 hallucinate citations), conference checklists. This loop borrows its
316 sentence-level guidance for the edit-drafting step rather than duplicating it.
317- academic-plotting: figure and architecture-diagram generation (out of scope
318 here; this loop edits text and captions, not figure images).
319