Master’s Thesis Opportunity Mechanistic Interpretability of Multimodal (Audio) Language Models

We are looking for a motivated Master’s student to investigate why modern audio-language models struggle to use clinical information to improve speech recognition for people with speech impairments as part of a Master’s thesis project. The project will focus on understanding the internal mechanisms of these models, identifying why additional context can negatively affect transcription accuracy, and exploring potential solutions. The work involves machine learning, mechanistic interpretability, and speech recognition, with opportunities to contribute to a scientific publication. The thesis will be conducted at the Institute of Neuroinformatics, ETH Zurich, in close collaboration with researchers at Cornell University.