An observability and monitoring toolkit for local AI fleet operations using Ollama Herd and SQLite-backed tracing.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install ai-devops-toolkit
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install ai-devops-toolkit using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is AI DevOps Toolkit?
The AI DevOps Toolkit is a specialized observability layer designed for developers running local LLM inference at production quality. As part of the Openclaw Skills ecosystem, it provides a comprehensive suite of tools for monitoring DevOps traces, performing health checks, and managing capacity without the need for complex external infrastructure like Prometheus or Grafana.
By leveraging a lightweight SQLite backend, this skill enables seamless DevOps latency monitoring and structured logging. It acts as the operational brain for an Ollama Herd fleet, ensuring that every routing decision and model request is recorded, analyzed, and optimized for maximum reliability and throughput.
AI DevOps Toolkit Use Cases
- Monitoring health and heartbeats of distributed local AI nodes.
- Investigating request failures and identifying unreliable LLM models via fallback frequency.
- Analyzing P50, P75, and P99 latency benchmarks to optimize user experience.
- Tracking token consumption and usage costs across different teams or environments using DevOps tag analysis.
- Planning hardware upgrades and model distribution based on automated DevOps capacity recommendations.
How AI DevOps Toolkit Works
- The Openclaw Skills agent interfaces with a running Ollama Herd router via its REST API.
- Request metadata, including latency, token counts, and routing scores, are persisted into a local SQLite database at ~/.fleet-manager/latency.db.
- Structured JSONL logs are generated for real-time monitoring of stream errors and context protection events.
- The toolkit runs periodic DevOps health checks to identify offline nodes, memory pressure, or version mismatches.
- Users can query the observability data via CLI commands or view aggregate trends through a built-in web dashboard.
AI DevOps Toolkit Setup
Ensure you have the necessary binaries installed, then set up the environment:
# Install the core package
pip install ollama-herd
# Start the DevOps router to expose observability endpoints
herd
# Start node agents on every machine to be monitored
herd-node
AI DevOps Toolkit Data Schema & Taxonomy
The toolkit organizes its data within the ~/.fleet-manager/ directory to ensure low-overhead observability.
| File/Path | Purpose | Data Type |
|---|---|---|
latency.db |
Stores request traces, latency history, and token usage | SQLite Database |
logs/herd.jsonl |
Daily rotated structured logs (30-day retention) | JSONL |
request_traces table |
Records request_id, model, node_id, TTFT, and status | DB Table |
Every Openclaw Skills trace includes detailed signals such as time_to_first_token_ms, retry_count, and fallback_used.
AI DevOps Toolkit Advanced Features
- Automated DevOps Health Analysis: Fifteen distinct checks including Zombie reaper and Model thrashing detection.
- Multi-tag Analytics: Assign requests to specific environments (e.g., devops-prod) via headers for granular reporting.
- Capacity Planning Engine: Generates hardware-aware model recommendations based on curated benchmarks.
- Cold Load Detection: Specialized monitoring for time-to-first-token (TTFT) to identify performance bottlenecks during model loading.
- Dynamic Configuration: Toggle runtime settings like auto_pull through the DevOps API.
SKILL.md
Loading
Related Openclaw Skills
OGP Agent-Comms Configuration
An interactive wizard for managing how your AI agent communicates and shares information with federated peers.
Business ROI Calculator
A professional-grade real-time investment return calculator designed to help business experts evaluate the viability of partnerships and projects.
Envoy
A comprehensive integration for Envoy that enables AI agents to manage visitor logs, desk reservations, and workplace logistics through the Membrane framework.
Amazon Price History Tracker
A professional-grade Amazon price monitoring and analysis agent designed to identify optimal buying and selling windows.
Wink Pings News Fetcher
A specialized news retrieval skill that connects to the Wink Pings service to deliver personalized AI industry updates and curated news feeds.
SERPhouse
A powerful SERPhouse integration for managing SEO projects, keyword tracking, and search engine ranking data through automated workflows.
Featured*
