Preprint · Preprints.org · October 27, 2026

From Algorithms to Bitstreams: A Survey of FPGA-Based Acceleration for Transformer Inference

Sayat Abdikul, Aitzhan Kadyrov, Nursultan Kabylkas, Kamilya Smagulova

Abstract

FPGA acceleration of Transformer inference involves model transformation, arithmetic optimization, memory organization, and deployment. This survey critically synthesizes 266 publications (2021–2026), identifying five mechanisms underlying performance gains: computation reduction, data movement reduction, arithmetic density, hardware utilization, and computational capacity. Quantization emerges as a frequently reported optimization, while architectural rankings depend on study selection and classification. Configuration-level comparisons and break-even analyses reveal how conversion, communication, integration overhead, and model quality constrain reported benefits. Supported by a traceable evidence map, the survey identifies limitations in existing evaluations and provides practical design guidance and controlled experiments for future FPGA-based Transformer accelerators.

Embedded Systems Design TechniquesAdvanced Neural Network ApplicationsVLSI and FPGA Design TechniquesFPGAtransformer inferencehardware acceleration

From Algorithms to Bitstreams: A Survey of FPGA-Based Acceleration for Transformer Inference

Abstract

More by Kamilya Smagulova