How Do Language Models Choose Between Context and Memory?
B. Shih, J. Winnicki, and A. Cao
Research
My research spans mechanistic interpretability and theoretical and scientific machine learning.
Overview
At Stanford, I am advised by Eric Darve in the DASH Lab, where I work on mechanistic interpretability: reverse-engineering the internal mechanisms of neural networks to understand how they represent information and arrive at their outputs.
Previously at Brown, I worked in the CRUNCH group with Zhongqiang Zhang and George Em Karniadakis on neural operators for differential equations, including transformer-based operator learning in finite-regularity settings.
Publications
Bachelor's thesis
Temporal Learning Capacity of Transformers in Non-Markovian Dynamical Systems
B. Shih
Research experience
Mechanistic interpretability
Current research in the DASH Lab at Stanford, advised by Eric Darve. I work on mechanistic interpretability — reverse-engineering the internal computations of neural networks — across several directions in how models represent information and produce their behavior.
DASH Lab / Eric Darve
Neural operators and scientific machine learning
Previous work at Brown on neural operators for differential equations with the CRUNCH group, advised by Zhongqiang Zhang and George Em Karniadakis.
GWAS of neurodegenerative diseases
Earlier research on genome-wide association studies of neurodegenerative diseases with Dr. Li-San Wang at the University of Pennsylvania Wang Lab.
Talks
- Approximation Theory: An Overview — Classical and Modern Perspectives, Brown University, 2024.