Katie Collins

Hi, I’m Katie! I’m a Machine Learning PhD candidate in the Computational and Biological Learning (CBL) Lab at the University of Cambridge, where I am supervised by Adrian Weller MBE and advised by Richard Turner, and visiting student with Josh Tenenbaum and the Computational Cognitive Science Group at MIT. I am also a Student Fellow at the Leverhulme Centre for the Future of Intelligence (CFI) and volunteer with the Human-Oriented Automated Theorem Proving, led by Sir Tim Gowers. Previously, I was a part-time Student Researcher at Google DeepMind with Krishnamurthy (Dj) Dvijotham.

I am passionate about applied computational cognitive science and investigating human-AI interaction from the lense of cognitive science. In particular, I am interested in the study and design of AI thought partners that meet our expectations and complement our limitations, and do so by engaging deeply with the behavioral sciences. I am particularly motivated by applications in biomedicine, mathematics, and education, and ensuring that systems faithfully express, reason over, and communicate uncertainty.

I received an MPhil in Machine Learning and Machine Intelligence from the University of Cambridge and a Bachelors of Science from MIT in Brain and Cognitive Sciences, with minors in Computer Science and Biomedical Engineering. I am grateful to the Marshall Scholarship for funding my MPhil and PhD, as well as King’s College, and the Cambridge Trust for additional support. I am passionate about interdisciplinary efforts in AI and the behavioral sciences and have helped co-organize multiple workshops: the NeurIPS 2024 Workhops on Behavioral Machine Learning, COGGRAPH at CogSci 2024, the NeurIPS 2023 Math-AI Workshop and the ICML 2022 Workshop on Human-Machine Collaboration and Teaming. During my undergrad, I founded the MITxHarvard Women in AI Group.

Outside of research, I love to run (!) and used to run competitively for MIT.

Selected Papers

You can find the most up-to-date listing on Google Scholar profile.

Building Machines that Learn and Think with People Katherine M. Collins, Ilia Sucholutsky, Umang Bhatt, Kartik Chandra, Lionel Wong, Mina Lee^^, Cedegao E. Zhang^^, Tan Zhi-Xuan^^, Mark Ho^^, Vikash Mansinghka^, Adrian Weller^, Joshua B. Tenenbaum^, Thomas L. Griffiths^.
Pre-print, under review (2024).

Evaluating Language Models for Mathematics through Interactions
Katherine M. Collins, Albert Q. Jiang*, Simon Frieder, Lionel Wong, Miri Zilka, Umang Bhatt, Thomas Lukasiewicz, Yuhuai Wu, Joshua B. Tenenbaum, William Hart, Timothy Gowers, Wenda Li, Adrian Weller^, Mateja Jamnik^.
PNAS (2024).
CheckMate Interactive Eval Platform MathConverse Data

Human Uncertainty in Concept-Based AI Systems
Katherine M. Collins, Matthew Barker^^, Mateo Espinosa Zarlenga^^, Naveen Raman**, Umang Bhatt, Mateja Jamnik, Ilia Sucholutsky, Adrian Weller, Krishnamurthy (Dj) Dvijotham.
AIES (2023).
CUB-S Data Project Page

Eliciting and learning with soft labels from every annotator
Katherine M Collins, Umang Bhatt*, Adrian Weller.
AAAI HCOMP (2022).
Code and Data Project Page

Structured, flexible, and robust: benchmarking and improving large language models towards more human-like behavior in out-of-distribution reasoning tasks
Katherine M. Collins, Lionel Wong*, Jiahai Feng, Megan Wei, Joshua B. Tenenbaum.
CogSci (2022). Invited Talk. Awarded Travel Grant for Paper.
Code and Data Project Page Alan Turing Institute Talk

*Contributed equally. ^^Contributed equally. ^Equal co-supervision.