Real-World Evidence Suitability Checklist Agent Skill

--- id: rwe-suitability-checklist name: "Real-World Evidence Suitability Checklist" version: "0.1.0" category: observational_research subcategory: real_world_evidence summary: "Check whether a data source and design can support a real-world evidence question, and what to disclose." intended_users: - pharmacoepidemiologists - health data scientists - regulatory affairs learners use_cases: - RWE study planning - data source assessment - methods education domain: - epidemiology - clinical_research task_type: - checklist - evaluate modality: - text - ehr lifecycle_stage: - protocol - analysis tags: - real-world evidence - pharmacoepidemiology - target trial emulation - claims data risk_class: R1 maturity: draft runtime: - prompt - checklist permissions: phi_allowed: false real_patient_data_allowed: false clinical_decision_making: false medication_dosing: false diagnosis: false internet_required: false external_api_required: false inputs: - name: study_plan type: string outputs: - name: checklist_json type: object parameters: [] prompt_system: > You are a real-world evidence methods reviewer. Draft a suitability checklist for a proposed RWE plan. Name threats and disclosure items; leave design decisions to the team. Do not produce evidence and do not give clinical advice. prompt_user: | Review the following real-world evidence plan and draft a suitability checklist. Plan: {study_plan} Return JSON with: { "question_readiness": [], "data_source_fitness": [], "design_threats": [], "transparency_items": [] } examples: - input: | plan: Does new users of a fictional oral anticoagulant have lower gastrointestinal bleeding risk than new users of an older comparator, using 5 years of a national claims database, new-user active-comparator design with propensity weighting? output: | { "question_readiness": [ "State the target trial: who is eligible, what is assigned, when follow-up starts, and what the contrast means; the observational plan should emulate it", "Define the outcome with the code set and a validation strategy (validated algorithms or chart review on a sample)" ], "data_source_fitness": [ "Claims capture dispensing well; verify bleeding outcomes are coded reliably in this database (validation studies exist for several code sets)", "Check linkage to any laboratory or hospital source the outcome needs", "Confirm follow-up length supports the event window" ], "design_threats": [ "Confounding by indication: record the indication and severity covariates the claims capture, and state what remains unmeasured", "Immortal time: cohort entry at dispensing with immediate follow-up; no person-time before treatment counted as treated", "Channeling: prior treatment history drives allocation; require new-user status for both groups" ], "transparency_items": [ "Pre-register the protocol and analysis plan", "Report the code sets, the propensity model specification, and diagnostics", "Report negative and positive control outcomes where available" ] } safety_notes: - "Planning and education tool only." - "Regulatory use of RWE requires formal frameworks and qualified review." limitations: - "Cannot assess data quality without the data; the checklist names what to check." - "Does not replace target-trial specification work by the study team." human_review_required: true references: - name: "Hernán MA, Robins JM" note: "Using big data to emulate a target trial when a randomized trial is not available. Am J Epidemiol 2016;183(8):758-64." - name: "Ray WA" note: "Evaluating medication effects outside of clinical trials: new-user designs. Am J Epidemiol 2003;158(9):915-20." - name: "GRACE principles" note: "Good ReseArch for Comparative Effectiveness (GRACE) principles for database studies." - name: "ICH M14" note: "ICH guideline work on real-world data suitability for regulatory decision-making (2024-2026 drafts)." status: active license: CC-BY-4.0 maintainers: - name: OpenMedSkills maintainers last_updated: "2026-08-29" content_reviewed: "2026-08" ---