Skip to content
DSML.KZ
Publications

Preprint · January 10, 2026

HordeVision: An Open-Source Kazakh Vision-Language Model

Pavel Zubitskii, Vitaliy Morozov, Sanzhar Murzakhmetov, Beksultan Sagyndyk, Sanzhar Umbet

Sanzhar Umbet
Open via DOIPDF

Abstract

We present HordeVision, an open-source Kazakh vision-language model designed for optical character recognition (OCR), image captioning, visual question answering (VQA), reasoning, and instruction following. Supporting local open-source initiatives, HordeVision addresses the critical need for Kazakh-language multimodal AI systems. Our dataset was collected using a syntactic data generation pipeline covering diverse contexts including social life, education, work, economy, media, communications, culture, and heritage. We employ a two-stage training approach combining supervised fine-tuning (SFT) and reinforcement learning from human feedback using Group Relative Policy Optimization (GRPO). Evaluated on a human-annotated benchmark against models including Google Gemma,

Multimodal Machine Learning ApplicationsNatural Language Processing TechniquesHandwritten Text Recognition TechniquesKazakh languagevision-language modelsoptical character recognition

More by Sanzhar Umbet