Publications

Deployment-Centered Evaluation: Predicting Query-Level Rejection Risk in a Clinical LLM System.
Alyssa Unell, Miguel Fuentes, Brenna Li, Bridget Lin, Meena Jagadeesan, Sanmi Koyejo, Nigam Shah.
NeurIPS 2026 Workshop on Responsible Communication of Machine Learning Research in Biomedicine.
Oral Presentation.

Metric Match: A Subset Selection Approach to Evaluating LLM Judge Reliability.
Alyssa Unell*, Natalie Dullerud*, Naomi Boneh, Meena Jagadeesan, Tatsunori Hashimoto, Nigam Shah, Sanmi Koyejo.
NeurIPS 2026 Workshop on Trustworthy AI Evaluation.

Holistic Evaluation of Large Language Models for Medical Tasks with MedHELM.
Suhana Bedi*, Hejie Cui*, Miguel Fuentes*, Alyssa Unell*, … Percy Liang, Mike Pfeffer, Nigam Shah.
Nature Medicine 2025.

CancerGUIDE: Cancer Guideline Understanding via Internal Disagreement Estimation.
Alyssa Unell, Noel C.F. Codella, Sam Preston, Peniel Argaw, … Matthew Lungren, Hoifung Poon.
ML4H 2025.
Presented at NeurIPS 2025 Workshop on GenAI for Health.

TIMER: Temporal Instruction Modeling and Evaluation for Longitudinal Clinical Records.
Hejie Cui*, Alyssa Unell*, Bowen Chen, Jason Alan Fries, Emily Alsentzer, Sanmi Koyejo, Nigam Shah.
npj Digital Medicine 2025.
Presented at ICLR 2025 Workshop on Synthetic Data.

Real-World Usage Patterns of Large Language Models in Healthcare.
Alyssa Unell*, Mehr Kashyap*, Michael Pfeffer, Nigam Shah.
medRxiv 2025.

Why are Visually-Grounded Language Models Bad at Image Classification?
Yuhui Zhang, Alyssa Unell, Xiaohan Wang, Dhruba Ghosh, Yuchang Su, Ludwig Schmidt, Serena Yeung-Levy.
NeurIPS 2024.

µ-Bench: A Vision-Language Benchmark for Microscopy Understanding.
Alejandro Lozano, Jeffrey Nirschl, James Burgess, Sanket Rajan Gupte, Yuhui Zhang, Alyssa Unell, Serena Yeung-Levy.
NeurIPS 2024 Datasets and Benchmarks Track.

Feasibility of Automatically Detecting Practice of Race-Based Medicine by Large Language Models.
Akshay Swaminathan, Sid Salvi, Philip Chung, Alison Callahan, Suhana Bedi, Alyssa Unell, Mehr Kashyap, Roxana Daneshjou, Nigam Shah, Dev Dash.
AAAI 2024 Spring Symposium on Clinical Foundation Models.

From Clear to Noise: Investigating Neural Noise Progression in Visual System Robustness.
Hojin Jang, Alyssa Unell, Suayb Arslan, Walt Dixon, Michael Fux, Matt Groth, Joydeep Munshi, Pawan Sinha.
Vision Sciences Society 2024.

Transformation Tolerance of Machine-based Face Recognition Systems.
Ashika Verma, Kyle Keane, Alyssa Unell, Anna Musser, Pawan Sinha.
ICLR 2021 Workshop on Generalization Beyond the Training Distribution in Brains and Machines.

Influence of Visual Feedback Persistence on Visuo-Motor Skill Improvement.
Alyssa Unell, Zachary M. Eisenstat, Ainsley Braun, Abhinav Gandhi, Sharon Gilad-Gutnick, Shlomit Ben-Ami, Pawan Sinha.
Nature Scientific Reports 2021.

Open-Source Contributions

Distributed Collaborative Learning (DisCo).
Added security guarantees to the DisCo platform, allowing clients to securely train models in a decentralized fashion.