A streamlined pipeline for training Stable Diffusion LoRA adapters locally on Apple Silicon with automated LLM-based quality scoring.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install lora-finetune
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install lora-finetune using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is LoRA Fine-Tuning (Apple Silicon)?
This skill provides a robust, local pipeline designed to train Low-Rank Adaptation (LoRA) weights for Stable Diffusion 1.5 directly on Mac hardware. By leveraging Openclaw Skills, developers can bypass expensive cloud GPU costs and maintain full data privacy while fine-tuning models for specific artistic styles, unique characters, or domain-specific visual assets. The pipeline is specifically optimized for the Metal Performance Shaders (MPS) backend, ensuring stability and performance on M-series chips.
Beyond simple training, the skill integrates an evaluation framework that uses an LLM-as-judge approach. This allows for objective scoring of style consistency and prompt adherence by comparing local fine-tuned results against commercial benchmarks like Gemini or DALL-E. It is an essential tool for developers looking to push the boundaries of on-device generative AI using Openclaw Skills.
LoRA Fine-Tuning (Apple Silicon) Use Cases
- Creating consistent custom characters for graphic novels or marketing materials.
- Adapting Stable Diffusion to specific architectural or industrial design styles.
- Implementing private, offline image generation workflows that require specific visual fine-tuning.
- Benchmarking Apple Silicon hardware performance for local AI training tasks using Openclaw Skills.
How LoRA Fine-Tuning (Apple Silicon) Works
- The user prepares a dataset of 15-25 images (512x512) with corresponding text descriptions in a local directory.
- The training script initializes the Stable Diffusion 1.5 base model from the Hugging Face Hub using the provided HF_TOKEN.
- Training executes locally using float32 precision to ensure stability on the Apple Silicon MPS backend, typically completing 500 steps in about 15 minutes on an M4 chip.
- The skill generates a compact LoRA adapter file (approx. 3.1MB) that modifies the base model's behavior.
- An automated evaluation script generates test images and uses a vision-capable LLM to score the results based on quality and style alignment.
LoRA Fine-Tuning (Apple Silicon) Setup
Ensure you have Python 3 installed and an M-series Mac with sufficient VRAM (24GB recommended for SD 1.5). You will also need a Hugging Face read token.
# Set your environment variable
export HF_TOKEN='your_huggingface_token'
# Prepare your dataset in a folder
# Ensure images and .txt captions are paired (e.g., img_1.png, img_1.txt)
# Run the training script
python3 scripts/train_lora.py \
--data_dir ./training_data \
--output_dir ./lora_weights \
--steps 500 \
--lr 1e-4 \
--rank 4
LoRA Fine-Tuning (Apple Silicon) Data Schema & Taxonomy
The skill organizes training data and results into a structured directory format to ensure compatibility with other Openclaw Skills.
| Path | Purpose | Format |
|---|---|---|
training_data/ |
Source images and captions | PNG + TXT pairs |
lora_weights/ |
Output directory for trained adapters | .safetensors / .bin |
scripts/train_lora.py |
Core training logic for MPS | Python Script |
scripts/compare_models.py |
LLM-as-judge evaluation logic | Python Script |
LoRA Fine-Tuning (Apple Silicon) Advanced Features
- Automated LLM-as-judge scoring using Pixtral Large to quantify style consistency and quality.
- Specialized float32 implementation to prevent the NaN errors common in float16 training on Apple Silicon.
- Head-to-head benchmarking capabilities against commercial APIs (Gemini/DALL-E) to measure fine-tuning efficacy.
- Optimized rank configuration (Rank 4) for high-quality results with minimal file size overhead in Openclaw Skills workflows.
SKILL.md
Loading
Related Openclaw Skills
Gateway Environment Injector
A secure automation tool for injecting 1Password secrets directly into macOS LaunchAgent environment variables.
Agent Budget Governance
A governance framework for multi-agent systems that enforces daily token budgets through automated auditing and privilege management.
Web Search and Research MCP
A comprehensive toolset enabling AI agents to search the web, extract content from PDFs and YouTube, and perform semantic research.
Secret Scanner
An automated security tool that identifies and remediates hardcoded credentials, API keys, and sensitive tokens across your codebase.
Prompt Cache
A lightweight caching layer that uses SHA-256 hashing to eliminate redundant LLM and TTS API calls and reduce operational costs.
LLM Judge Ensemble
A multi-layered evaluation framework for scoring AI outputs using deterministic checks, heuristics, and sampled LLM judges.
Featured*
