Layer-Wise Probing of Paralinguistic Attributes in Fine-Tuned Whisper for Kazakh Speech

Abstract

Large pre-trained speech models similar to Whisper are now commonly used for speech recognition and related tasks. The distribution of paralinguistic features, which include emotions and speaker characteristics across model layers, remains uncertain, particularly for low-resource languages. The current study evaluates each layer of the Kazakh-adapted Whisper encoder to determine its performance in recognizing emotional expression, speaker identity, age, and gender attributes. We extract fixed representations from every encoder layer and test them with both linear and Multilayer Perceptron (MLP) probes. The evaluation process uses accuracy, macro-averaged F1-score (Macro-F1), and balanced accuracy metrics, whereas non-parametric statistical tests evaluate the importance of changes across different layers. The experimental evaluation of KazEmoTTS focuses on emotional expression, whereas Common Voice (Kazakh) data serve for speaker identification and demographic attribute analysis. The results demonstrate that age and gender information are strongly present at all layers of the model with little change in representation across depths, yet speaker identity shows statistically significant but weak variations between layers. Emotion information appears mainly in the model's middle layers, which is the area where probing is most effective. The research findings reveal how Whisper processes Kazakh speech, allowing researchers to choose appropriate layers for paralinguistic speech applications.

More by Shirali Kadyrov

• Bounded continued fractions and quadratic orders in $\Q(\sqrt{127})$arXiv · 2026
• Code and data for "Reinforcement learning-based PID for nonlinear temperature control with delayed feedback and measurement noise"Zenodo · 2026
• Life expectancy in Kazakhstan: an integrated analysis and forecastZenodo · 2026
• Data-driven homotopic reinforcement learning for adaptive tracking control of completely unknown Takagi–Sugeno fuzzy Markovian jump systemsCommunications in Nonlinear Science and Numerical Simulation · 2026