Hive: A secure, scalable framework for distributed Ollama inference

Abstract

Large Language Models (LLMs) require substantial computational resources, often necessitating distributed inference across multiple machines. However, organizations frequently struggle to unify scattered resources, whether due to firewalls, private networks, or heterogeneous hardware. Traditional approaches demand complex VPN configurations, direct network exposure, or manual workload balancing, making large-scale deployment impractical for many. We present Hive, an open-source framework designed to seamlessly integrate fragmented compute resources into a single, unified inference system. Hive consists of HiveCore, a central proxy that handles client requests, authentication, and task queuing, and HiveNode, a lightweight worker agent that connects securely to HiveCore and executes inference locally. By relying solely on outbound connections, Hive allows remote and isolated machines running Ollama to contribute computational capacity without requiring public network exposure. This architecture enables organizations to leverage both high-performance clusters and legacy hardware, dynamically scaling LLM inference without operational overhead.

More by Jernej Vičič

• AI-generated text detection for Slavic languagesZenodo · 2026
• Dataset of Uzbek base words: extraction and data analysis based on the school corpusSocial Sciences & Humanities Open · 2026 · Citations: 2
• Dataset of Uzbek base wordsRepository of the University of Primorsk · 2026
• Uzbek 65 million web corpus lemmatized Word2Vec modelZenodo · 2026