Talk · Conference on Machine Translation · January 1, 2026

Pare4Bit: Quantization, Depth Pruning, and Model Selection for the WMT26 Model Compression Task

Abstract

We describe Team Pare4Bit's submissions to the WMT26 Model Compression shared task on the three evaluation pairs Czech→German, English→Simplified Chinese, and English→Egyptian Arabic.In the constrained track we compress google/gemma-3-12b-it with a data-light pipeline-vision-tower removal followed by GPTQ W4A16 quantization-reaching 43.2 mean chrF at 7.6 GB, 3.1× smaller than the bf16 text-only reference for a 0.3 chrF drop.We further explore depth pruning with LoRA knowledge-distillation healing, which recovers a collapsed pruned model to within 1.3 chrF of the INT4 primary and adds an engine-agnostic speedup.In the unconstrained track we treat model selection as a first-class compression decision: the newer Gemma-4-12B, deployed via quantization-aware-trained INT4, outperforms the constrained Gemma-3 baseline by +2.5 mean chrF (+4.7 on Egyptian Arabic, where it produces the dialect rather than Modern Standard Arabic).We report several negative results-activation quantization instability on Gemma-3, re-segmentation under naive vocabulary pruning, and a chrF-versus-COMET disagreement on a sentence-level MT specialist applied to paragraphs-as practical guidance on what does and does not compress this model family.

Advanced Neural Network ApplicationsNatural Language Processing TechniquesTopic Modelingmodel compression4-bit quantizationmodel selection

Pare4Bit: Quantization, Depth Pruning, and Model Selection for the WMT26 Model Compression Task

Abstract

More by Наиль Фахрутдинов