Topological analysis of unaligned audio and text data

Abstract

We have performed preliminary work on topological analysis of audio and text data for unsupervised speech processing. The work assumes that phoneme frequencies and contextual relationships are similar in the acoustic and text domains for the same language. Accordingly, this allowed the creation of a mapping between these spaces that considers their geometric structure. As a first step, generative methods based on variational autoencoders were chosen to map audio and text data into two latent vector spaces. In the next stage, persistent homology methods are used to analyze the topological structure of two spaces. Although the results obtained support the idea of the similarity of the two spaces, further research is needed to correctly map acoustic and text spaces, as well as to evaluate the real effect of including topological information in the autoencoder training process.

More by Zhanibek Kozhirbayev

• Sequential Domain Adaptation of Whisper for Kazakh Academic Lecture Speech RecognitionLecture notes in computer science · 2026
• The comparative evaluation of explanations for algorithmic programming problems provided by a Human vs. ChatGPT: an exploratory online user studyResearch Square · 2026
• Enhancing Personalised Learning with Graph-Based Ensemble Prediction and Skill Cluster Mapping for Student Knowledge CompletenessComputers · 2026
• Evaluating Large Language Models for Text-to-Gloss Translation in Kazakh-Russian Sign Language: A Pilot StudyLanguage Resources and Evaluation Conference · 2026