Penn Policy Summit on AI for Health Care

Penn Policy Summit on AI for Health Care

The goal of these exercises is to get a sense of where there might be broad agreement or different points of view, and to identify issues that might benefit the most from discussion by the group. Answers are confidential.
Name (optional; only to be used to help facilitate discussion)

First NameLast Name
Instructions

For each activity below assign 1 "Primary" actor and no more than 3 "Supporting" entitites. For more detail on what each activity could entail see https://pmc.ncbi.nlm.nih.gov/articles/PMC11645404/ and https://jamanetwork.com/journals/jamainternalmedicine/fullarticle/2840933.

Stage: Pre-Deployment Development and Testing

Rows	Primary	Supporting	Supporting	Supporting	Notes & Assumptions	Questions
Method Validation (Analytical)
Clinical Knowledge & Skills Benchmarking (In Silico/Simulation Tests)
Supervised Practice (Real-World, with Oversight)
Scope of Practice Determination/Certification
Local Verification & Credentialing
Documentation & Transparency

Stage: Real-World Monitoring

Rows	Primary	Supporting	Supporting	Supporting	Notes/Assumptions	Questions
Regular Performance Checks
Input Checks
Documentation & Transparency
Inspection

Stage: Discipline & Compliance Actions

Rows	Primary	Supporting	Supporting	Supporting	Notes/Assumptions	Questions
Discipline & Compliance Actions
Liability Determination

Instructions

For each idea below assign a value: Good Idea, Bad Idea, I Don't Know, or No Strong Feeling.

Risk Assessment

Rows	Based on the Type of Clinical Action (Triage vs. Diagnosis vs. Treatment Initiation)	Narrowness of Task	Reversibility	Exceptions/Special Paths Where Benefit-Risk Ratio May Warrant Reduced Pre-Market Evidence	Notes/Assumptions	Questions
How should risk be tiered?

Evidence Method

Rows	RCT (device model)	Simulated exams and supervised practice (practioner model)	Cost effectiveness (service model)	What would occur in absence of the AI (i.e., taking current access limitation into account?)	Notes and Assumptions	Questions
What type of evaluation should be used?

Evidence Level

Rows	Human Clinician as the Benchmark	Non- Inferior to Human Clinician Benchmark	Superior to Human Clinician Benchmark	Absolute Safety Threshold	Notes and Assumptions	Questions
What type of evaluation should be used?

Evidence Type

Rows	Clinical Knowledge	Analytic Capability	Safety Behavior	Communication	Generalizability	Questions	Notes and Assumptions
What type of evaluation should be used?

Reporting Method & Cadence

Rows	Defined Intervals (e.g., Yearly)	Random Simulation Testing (Modeled on CLIA)	Adverse Event Reporting System	Machine-Based Supervisory Agents	Notes and Assumptions	Questions
What type of evaluation should be used?

Payment

Rows	Fee-for- Service	Value-Based Episodic Payments	Per Patient	Value-Based (Per Result)	Transitional Financing & Adoption Support	Not Reimbursing AI Separately	Notes and Assumptions	Questions
How should clinical AI be reimbursed?

What are the potentially effective ways to pay for autonomous clinical AI?
What are the potentially effective compliance actions?
What existing policy frameworks or proposals for autonomous AI are the most promising? Why?
What action at the local, state, or Federal level would help your organization advance autonomous AI safely and equitably?
Should be Empty: