FusionBench is a powerful benchmark and toolkit designed for deep model fusion, supporting over 30 merging algorithms for LLMs and vision models.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install fusion-bench
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install fusion-bench using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is FusionBench?
FusionBench serves as a comprehensive ecosystem for researchers and developers to experiment with model merging. It provides a standardized framework to load various model architectures, apply cutting-edge fusion algorithms, and evaluate the resulting performance across multiple task pools. By utilizing Openclaw Skills like FusionBench, users can significantly enhance model capabilities without the high cost of full retraining.
The toolkit bridges the gap between theoretical merging research and practical application. Whether you are working with CLIP Vision models or Large Language Models like Llama, FusionBench offers the infrastructure to handle state dict arithmetic, lazy loading for memory efficiency, and distributed execution via Lightning Fabric.
FusionBench Use Cases
- Merging multiple fine-tuned models into a single multi-task model.
- Evaluating the performance of different merging algorithms like TIES or DARE on specific benchmarks.
- Researching new model fusion techniques by implementing custom algorithms.
- Managing complex model pools and task evaluations for LLM and CLIP architectures.
- Optimizing model performance through hyperparameter sweeps of scaling coefficients.
How FusionBench Works
- Configuration: The user defines the experiment using Hydra YAML configs, specifying the model pool, task pool, and merging method.
- Model Loading: The ModelPool component loads the base model and the specialized models (task vectors) into memory or via lazy loading.
- Algorithm Execution: The selected Method (e.g., Task Arithmetic, Fisher Merging) processes the model weights to create a fused state dictionary.
- Integration: The merged weights are loaded back into a model architecture, creating the final fused model.
- Evaluation: The TaskPool runs the fused model through defined benchmarks (like MMLU for LLMs or classification tasks for CLIP) to report accuracy and performance metrics.
FusionBench Setup
To get started with this Openclaw Skills integration, install the package via pip and run a baseline experiment:
# Install the core toolkit
pip install fusion-bench
# Run a task arithmetic experiment with CLIP ViT-B/32
fusion_bench method=task_arithmetic modelpool=clip-vit-base-patch32 taskpool=clip-vit-base-patch32_8tasks
FusionBench Data Schema & Taxonomy
FusionBench organizes data through a modular taxonomy of configurations and caches:
| Component | Description | Default Path |
|---|---|---|
| Model Cache | Stores downloaded or local model weights | ~/.cache/fusion_bench/models |
| Hydra Configs | YAML files defining methods, model pools, and tasks | fusion_bench/config/ |
| State Dicts | Intermediate weight representations | Handled in-memory or via LazyStateDict |
| Task Pools | Definitions for evaluation datasets (ARC, MMLU, etc.) | fusion_bench/tasks/ |
Models are typically categorized into _base_ (the starting checkpoint) and specific task-tuned variants within a ModelPool.
FusionBench Advanced Features
- Support for over 30 merging algorithms including MoE-based (WE-MoE, PWE-MoE) and Continual Merging (OPCM, DOP).
- Distributed and Mixed Precision support using Fabric (e.g., DeepSpeed Stage 2) for large-scale LLMs.
- LazyStateDict utility for loading massive models without encountering Out-Of-Memory (OOM) errors.
- Extensible API for registering custom merging algorithms by inheriting from BaseModelFusionAlgorithm.
- Interactive exploration via the fusion_bench_webui for a graphical interface to merging experiments.
SKILL.md
Loading
Related Openclaw Skills
Intelligent Content Summary & Project Review
An automated utility for generating intelligent summaries and performing comprehensive project post-mortems across multiple content types.
Kiro Development Workflow
Kiro is an agentic IDE and CLI tool developed by Amazon that enables Spec-Driven Development and highly automated agent workflows.
pdf2ofd: High-Fidelity PDF to OFD Converter
A specialized skill for converting PDFs to the Chinese National Standard OFD format with character-level precision and transparency support.
Variant Design
Transform prompts into three distinct UI design variations using a professional design system and anti-AI-slop quality gates.
Chat Summary
An automated dialogue management tool that clusters chat histories by topic and generates structured multilingual summaries.
Blog Content Management Skill
A lightweight CLI toolkit for managing the end-to-end lifecycle of blog content, from initial drafts to SEO optimization and social media distribution.
Featured*
