A sophisticated fleet routing and load balancing solution for running Ollama across multiple Linux machines through a single unified endpoint.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install linux-ollama
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install linux-ollama using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is linux-ollama?
Linux Ollama Herd is a powerful Openclaw Skills extension designed to transform a collection of Linux servers, desktops, and edge devices into a high-performance AI inference cluster. By utilizing a central router, it aggregates the compute power of multiple nodes, allowing users to access complex models like Llama, Qwen, and DeepSeek via a single Linux Ollama URL. This setup effectively eliminates the bottlenecks of single-machine inference by distributing workloads across your entire hardware fleet.
The tool integrates seamlessly with the Linux ecosystem, offering native systemd support for robust background operations and automatic service recovery. Whether you are managing a professional server rack or a home lab of mixed Linux devices, this skill provides the infrastructure necessary for scalable, local AI deployment while maintaining compatibility with both OpenAI SDK and standard Ollama API formats.
linux-ollama Use Cases
- Scaling local AI inference across multiple Linux workstations or servers to handle higher request volumes.
- Creating a unified AI API endpoint for development teams using heterogeneous Linux hardware.
- Deploying large language models (LLMs) on edge devices while offloading heavy computation to more powerful Linux nodes.
- Implementing high-availability AI services on Linux using systemd for automated uptime management.
- Centralized monitoring and health tracking of distributed Ollama instances through a single dashboard.
How linux-ollama Works
- The Linux Ollama router (herd) is initialized on a primary machine to act as the traffic coordinator for the fleet.
- Individual Linux nodes run the herd-node agent, which automatically discovers the central router using mDNS or a direct IP connection.
- The router maintains a real-time registry of available nodes, their health status, and the specific AI models currently loaded on each.
- When an API request is received via the unified port (11435), the router intelligently directs the traffic to the most appropriate Linux node based on model availability and latency.
- The system continuously monitors node performance and health, providing a live dashboard for administrators to track fleet-wide inference statistics.
linux-ollama Setup
Prerequisites
Ensure you have curl or wget installed on your Linux system.
Installation
First, install the base Ollama service:
curl -fsSL https://ollama.ai/install.sh | sh
Next, install the Openclaw Skills compatible fleet manager:
pip install ollama-herd
Activation
Start the router on your main Linux machine:
herd
herd-node
On all other nodes in your Linux fleet:
herd-node
linux-ollama Data Schema & Taxonomy
The skill manages its state and performance data through specific paths in the user's home directory. It uses structured logging for easy parsing and a database for latency tracking.
| Data Type | Path / Format | Description |
|---|---|---|
| Latency DB | ~/.fleet-manager/latency.db |
Stores network performance and node response times. |
| Structured Logs | ~/.fleet-manager/logs/herd.jsonl |
JSONL format logs for auditing and debugging fleet events. |
| Node Discovery | mDNS / Direct IP | Dynamic discovery mechanism for joining nodes to the cluster. |
| Metadata | Openclaw Config | Tracks system requirements such as nvidia-smi and systemctl status. |
linux-ollama Advanced Features
- Systemd Integration: Full support for running as a background service with automated restarts and boot-time activation.
- Multi-GPU Support: Intelligent routing based on available vRAM (NVIDIA CUDA and experimental AMD ROCm).
- Automated Health Checks: Features 15 automated health checks accessible via a dedicated monitoring API.
- Cross-Modal Capabilities: Supports image generation and text embeddings across the distributed Linux fleet.
- Custom Environment Optimization: Advanced configuration for model keep-alive times and parallel request handling through Linux environment variables.
SKILL.md
Loading
Related Openclaw Skills
RTX Local AI
RTX Local AI turns your gaming PC into a high-performance local AI server, allowing you to run massive LLMs with zero cloud costs using NVIDIA RTX GPUs.
Windows Ollama Fleet Manager
An advanced orchestration tool for routing Ollama inference requests across a network of Windows PCs to maximize hardware utilization.
Windows AI Local Inference Cluster
A powerful tool to turn your Windows PCs into a private, high-performance AI cluster for local LLM inference, image generation, and embeddings.
wsl2-local-ai
A high-performance LLM development stack for Windows users utilizing WSL2, NVIDIA GPU passthrough, and Ollama Herd routing.
Linux AI Server
Transform your Linux hardware into a high-performance, headless AI inference cluster using native systemd and NVIDIA CUDA.
CUDA Ollama Fleet Router
A high-performance routing layer that unifies multiple NVIDIA GPUs into a single, load-balanced CUDA Ollama inference cluster.
Featured*
