Eliminate N+1 query overhead by batching and caching database or API requests into efficient single operations.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install batch-processing
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install batch-processing using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Batch Processing DataLoader?
The Batch Processing skill implements the highly efficient DataLoader pattern designed to resolve the notorious N+1 query problem. By aggregating individual data requests into a single batch operation, this skill reduces the communication overhead between your application and databases or external APIs. Using Openclaw Skills like this ensures that your AI coding agents generate optimized code that scales effectively without redundant network or database latency.
This utility acts as a request-scoped cache and batching layer. It collects various IDs or keys requested during a single execution cycle and fetches them all at once, ensuring that even complex, nested data requirements are handled with minimal round-trips. It is an indispensable addition to any modern backend architecture or GraphQL-style resolver setup.
Batch Processing DataLoader Use Cases
- Resolving N+1 query issues in database-heavy applications.
- Batching multiple REST API calls into a single request sequence.
- Optimizing GraphQL resolvers that fetch nested relationships.
- Reducing load on downstream services during high-concurrency operations.
- Implementing request-level caching to prevent redundant data fetching.
How Batch Processing DataLoader Works
- The AI agent or application identifies multiple individual data requests for the same resource type.
- Each request is passed to the DataLoader via the load method.
- The skill waits for the current execution tick or a specified delay to collect all keys.
- A batch loading function is triggered once, fetching all collected keys in one operation.
- The results are mapped back to their original requests, resolving the promises with the correct data.
Batch Processing DataLoader Setup
The Batch Processing skill is self-contained and can be integrated directly into your project. Use the following steps to set it up:
# Ensure the dataloader.js file is placed in your local project directory
# No external npm dependencies are required for the core logic
Import the loader into your service using standard Node.js module syntax:
const DataLoader = require('./batch-processing/dataloader.js');
Batch Processing DataLoader Data Schema & Taxonomy
| Property | Description |
|---|---|
| batchLoadFn | The core function that receives an array of keys and returns a promise of values. |
| maxBatchSize | Limits the number of items processed in a single batch (default: 100). |
| batchScheduleMs | The window of time to wait before executing a batch (default: 0ms). |
| cache | Boolean flag to enable or disable per-request caching (default: true). |
This skill manages data in memory during the lifecycle of the loader instance, ensuring high-speed access to previously fetched keys within the same execution context.
Batch Processing DataLoader Advanced Features
- Custom Cache Keys: Define how keys are indexed in the internal cache for complex objects using a custom cacheKeyFn.
- Manual Cache Priming: Use the prime method to pre-load data from other sources like a global cache or local variables before a load occurs.
- Cache Invalidation: Granular control via clear and clearAll methods to handle data mutations and keep the state consistent.
- Nested Loading Support: Efficiently handles deep object graphs where children also require batched loading through recursive loader calls.
SKILL.md
Loading
Related Openclaw Skills
Claude Code Launcher
A terminal automation skill that launches Claude Code and automatically enables Remote Control for seamless multi-device coding sessions.
Triumvirate Protocol
An identity-aware debate engine that orchestrates multi-architecture AI discourse through structured identity graphs and persistent thread tracking.
Identity Persistence Layer
A structured identity graph system for AI agents that synthesizes memory files into versioned, scored identity snapshots.
Cortex Protocol
A gasless on-chain identity protocol for AI agents using the ERC-8004 standard on the Base network.
HTTP Retry + Circuit Breaker Skill
A robust HTTP client extension for Openclaw Skills that implements exponential backoff retries and circuit breaker patterns to drastically reduce API failure rates.
WebSocket Reconnect
A sophisticated WebSocket management tool providing automated recovery through exponential backoff, jittered retries, and circuit breaker patterns.
Featured*
