Research Publishability Critic

Give a researcher an honest, empirical, fast read on whether an experimental strategy paper is worth pursuing — and exactly what to fix first. The skill audits the draft against the elements a strategy experiment must have, scans the live literature for the closest published work and the real gap, scores the work on a multi-item rubric from three deliberately different reviewer voices, and writes a single digestible report the researcher can act on.

The output is a decision aid, not a compliment. A weak idea should get a clear "pivot" or "perish" with reasons, because the most valuable thing this skill can do is save a researcher months spent on a paper that a target-journal reviewer will reject on contribution.

Design intent: fast and cheap by construction

Publishability review is easy to make slow and expensive — full PDFs, long literature dumps, one giant context. This skill is deliberately built the other way, and the instructions below exist to keep it that way. Two ideas do most of the work:

Fan out, summarize, discard. The heavy, parallelizable work — the literature scan and the three persona reviews — runs in sub-agents. Each sub-agent does its own reading in its own context and returns a compact, structured result (scores, one-line rationales, a short list of closest papers). The orchestrator never ingests raw search results or full PDFs; it holds only the structured draft plus these small returns. This is what keeps the token cost roughly flat regardless of how big the paper or the literature is, and it is why the three persona reviews cost about the same as one.

Read abstracts, not papers. The literature step ranks and clusters on abstracts only, via a bundled deterministic search script — never by browsing or downloading PDFs. Deterministic search is faster, cheaper, and reproducible compared with letting the model surf.

If sub-agents are not available (e.g., a single-context runtime), the skill still works: run the same steps sequentially in the main context, but cap the literature candidates hard (see the inline-fallback note in Step 3) so the context stays small. Prefer sub-agents whenever they exist.

Inputs and file structure

The skill expects this layout (create output/ if missing):

input/personal/      the draft: working paper, abstract, slides, writing
input/inspiration/   journal + working articles that reflect the researcher's taste
output/              where the report and score file are written

input/inspiration/ is not decoration — it calibrates the review. Those articles encode what this researcher considers a good paper and which venues they read, so use them to infer likely target journals, house framing, and the "taste" against which relevance is judged. Read their titles/abstracts, not every full text.

If input/personal/ holds several files, ask which is the draft under review (or treat the longest prose document as the draft and the rest as supporting material) before proceeding.

Workflow at a glance

Step 0  Clarify goals & context        (interactive; sensible defaults if unattended)
Step 1  Structure the draft            (map to the required elements; recommend for blanks)
Step 2  Extract keywords & venues      (from draft + inspiration taste)
Step 3  Literature scan  ── sub-agent  (litscan.py → rate abstracts 1–10 → cluster → gap)
Step 4  Three persona reviews ── 3 sub-agents in parallel (rubric-scored, with why)
Step 5  Synthesize & write report      (orchestrator merges → output/*.md + scores.json)

Steps 3 and 4 are independent — launch them together. Step 5 waits on both.

Step 0 — Clarify goals and context

Publishability is relative to a target, so establish the target before judging. Ask the researcher (one short round, not an interrogation):

• Career stage and how much time they can commit to this paper.
• Career goals — what this paper needs to do for them.
• Target venue(s). Default to A* if unspecified.
• Timeline — when a submission needs to happen.
• What "success" means here — a specific journal? any peer-reviewed outlet? a strong conference? citations? societal reach?

Record the answers at the top of the report as the frame every downstream judgment is made against. If the session is unattended or the researcher wants to just run it, do not block — assume an A* target, a standard 6–12 month timeline, and success = accepted at a target venue, and state these assumptions explicitly at the top of the report so a returning reader can correct them.

Step 1 — Structure the draft against the required elements

A strategy experiment is only reviewable if it names its moving parts. Map the draft onto these elements. For any element that is missing or thin, do not just flag it — recommend what it should say, drawing on the draft's own material and the literature. A recommendation the researcher can react to is worth far more than a blank checkbox.

1. Setup / motivation for a strategy audience — why a strategy scholar should care, in strategy's own terms.
2. Setting & subjects — where, who, how many, how selected.
3. Business challenge / friction — the real-world problem with teeth.
4. The insight — the non-obvious idea that motivates the intervention.
5. Solution (the intervention) — what is actually manipulated or done.
6. Why & when will it work — the mechanism and its boundary conditions.
7. Null / the right comparison — what is the correct counterfactual, and what would it look like if the effect were not there. A muddled comparison is the single most common fatal flaw; treat this element with extra care.
8. Impact on the business, and how it will be measured — the outcome that matters and the concrete measure(s) for it. Prefer behavioral/objective measures over self-report where possible, and say how each is operationalized.

Produce a compact element table: element · present? · one-line summary or recommendation. This table becomes a section of the report and feeds the rubric.

Step 2 — Extract keywords and candidate venues

From the structured draft plus the inspiration folder's taste, produce:

• 6–12 keywords / search phrases that actually describe the contribution (constructs, methods, mechanisms, population, context) — not generic terms.
• Candidate target journals (honor Step 0; infer additions from the inspiration set). These scope "closest published work in target venues" and the call-for-papers check.

Keep this as a short list; it is the input to the literature sub-agent.

Step 3 — Literature scan (sub-agent)

Dispatch a lit-scout sub-agent with the keywords, candidate venues, and the structured draft summary. Its job is to answer three questions cheaply and return a compact result — not to write prose.

It should use the bundled script for search (deterministic, no API key, abstracts only):

scripts/litscan.py --keywords "<...>" --venues "<...>" --max 40

litscan.py queries OpenAlex first (free, no key, structured abstracts), with Crossref, arXiv, and Semantic Scholar as fallbacks, and returns candidate records (title, venue, year, abstract, DOI) as JSON. If the script or network is unavailable, fall back to WebSearch; if that too is unavailable, scan input/inspiration/ only and say so in the report. Never download or read full PDFs for this step.

The sub-agent then:

• Rates each candidate 1.0–10.0 on relevance to this draft, judged from the abstract, with a one-line reason per paper.
• Clusters the papers into topic subgroups. The clusters double as study modules / a reading map; within a cluster the relevance score is the reading priority.
• Assesses the gap: is something genuinely unaddressed — a missing test, method, mediator, or moderator? Is the gap important (who cares, why now, what societal impact)? Is there an active call for papers at a target venue this work could answer?

It returns (compact): the top ~10 closest published articles in target venues (title, venue, year, DOI, relevance, one-line why), the clustered reading list, and a short gap verdict (real gap / crowded / incremental, with reasons). Not the raw 40-record dump.

Step 4 — Three persona reviews (parallel sub-agents)

Launch three reviewer sub-agents in parallel, each scoring the same multi-item rubric in references/rubric.md but through a different lens. Diversity of lens is the point: each catches failure modes the others miss, and running them in parallel costs about the same as one.

• The supportive advisor — a PhD supervisor championing the researcher's curiosity. Finds the strongest version of the idea and the shortest path to it. Generous on potential, honest on effort.
• The hostile editor — the exact reviewer at the target journal who would trash this paper to the co-editors over a beer when the author isn't there. Ruthless on contribution, identification, and the comparison/null. This voice exists to surface the desk-reject reasons before a real editor does.
• The curious peer — a smart colleague from outside the subfield who has no idea about the specifics but asks the great, naive questions that expose unstated assumptions and weak motivation.

Each sub-agent receives the structured draft, the element table, and the lit-scout's gap verdict + closest-papers list, and returns: a rubric score per dimension (with a one-to-two sentence "why"), its top 3 concerns, and its single highest-leverage suggestion. Full rubric dimensions, anchors, and the "why it matters / what this reviewer looks for" notes live in references/rubric.md; read that file before scoring. Persona detail lives in references/personas.md.

Every score must carry a reason. A number with no "why" is not usable feedback and defeats the point of the rubric.

Step 5 — Synthesize and write the report

The orchestrator merges the element table, the lit-scout return, and the three persona scorecards into one digestible markdown report written to output/ (use the template in assets/report_template.md). Also write a machine-readable output/scores.json with the per-dimension, per-persona scores so results are trackable across revisions.

Reconcile the three personas rather than averaging them blindly: where they disagree, say so and explain the disagreement — a split between the advisor and the hostile editor is itself signal about risk. Translate scores into real magnitudes and plain reasons, not just numbers.

The report must contain, in this order:

1. Goals & frame — Step 0 answers (or the stated default assumptions).
2. Element completeness table — present/missing, with recommendations for gaps.
3. Rubric scorecard — each dimension: score, the why, and which persona(s) flagged it. Show disagreement explicitly.
4. Improvement checklist, ordered by severity — the core deliverable. Each item: what to fix, why it matters (which rubric dimension / which reviewer), and a concrete next action. Most severe first.
5. Closest published work — the top ~10 in target venues, with one-line positioning of the draft against each.
6. Assessment of necessary resources — data, sample, time, collaborators, and skills the plan implies, versus what the researcher said they have.
7. Three-reviewer read — a short, distinct paragraph in each persona's voice, so the researcher feels the range of reactions.
8. Final recommendation — pursue / pivot / perish — one clear verdict with its reasoning. "Pivot" must name the specific pivot; "perish" must say why the effort is better spent elsewhere. This is the sentence the researcher came for; make it honest and unambiguous.

Deliver the report file to the researcher and point them to the severity checklist and the verdict first.

Keeping the review honest (applies throughout)

These are general standards for good experimental-strategy work; hold the draft to them regardless of subfield.

• Nulls are findings, not failures. Never coach the researcher to bury a null or dress it up. A clean, well-powered null against the right comparison can be more publishable than a noisy positive; judge it on design, not direction.
• The comparison is everything. Most fatal flaws hide in the counterfactual. Spend disproportionate scrutiny on whether the null/comparison actually isolates the claimed mechanism.
• Behavioral over self-report. Prefer outcomes that capture what people did over what they said they'd do; flag when a headline claim rests only on self-report.
• Talk in magnitudes. Push the draft toward real effect sizes and practical significance rather than p-value/asterisk framing.
• Don't confuse polish with contribution. A beautifully written paper with no delta still perishes; a rough draft with a real gap can pursue. Score contribution and design first, prose last.

Reference and bundled files

• references/rubric.md — the multi-item scoring rubric: every dimension, its anchored scale, why it matters, and what the hostile editor specifically looks for. Read before Step 4.
• references/personas.md — full behavioral briefs for the three reviewer voices, so their feedback stays distinct and in-character.
• scripts/litscan.py — deterministic literature search (OpenAlex primary; Crossref/arXiv/Semantic Scholar fallback) returning ranked-ready JSON. Used in Step 3.
• assets/report_template.md — the exact structure for the output report, so every run produces the same digestible shape.

Edge cases

• Just an abstract, not a full draft. Fine — run the same flow; expect more "missing element → recommendation" entries and a gap verdict weighted toward potential. Say the review is idea-stage.
• Idea outside strategy. The rubric is tuned for experimental strategy research; if the draft is clearly another field, say so and offer a best-effort review rather than forcing a bad venue fit.
• Empty or unreadable inspiration folder. Proceed, but note that relevance and venue inference are weaker without taste calibration, and lean more on the Step 0 target venues.
• No network / no search. Degrade to inspiration-folder-only relevance and state the limitation prominently — do not present a thin scan as comprehensive.
• Sub-agents unavailable. Run Steps 3 and 4 sequentially in the main context; cap literature candidates at ~15 and keep persona reviews terse to protect the context budget.