Rethinking Depth in Speech Encoder

Priberam Machine Learning Lunch Seminar

Abstract:

Nowadays, speech encoders are becoming increasingly powerful, but also larger and more complex. However, we observe significant redundancy within these models, which motivates a rethinking of how depth should be designed. In this talk, I will present a parameter-efficient alternative based on shared weights with recursive (looped) application of encoder layers, where a smaller set of parameters is reused across multiple iterations instead of stacking many distinct layers. This approach aims to preserve strong representational capacity while reducing model size.

Bio:

Thomas Rolland is a Postdoctoral Researcher at INESC-ID in Lisbon, focusing on building robust speech systems for low-resource, noisy, and domain-shifted settings. His work centers on parameter-efficient architectures, synthetic data augmentation, and post-training strategies to improve adaptability and fairness across diverse speech scenarios.