A high-performance pattern for publishing events to Kafka for durable storage and Redis for real-time delivery.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install dual-stream-architecture
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install dual-stream-architecture using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Dual-Stream Architecture?
The Dual-Stream Architecture is a specialized pattern within the Openclaw Skills ecosystem designed for building resilient, event-driven systems. It addresses the common trade-off between the durability of message brokers like Kafka and the low-latency requirements of real-time applications using Redis Pub/Sub.
By leveraging this architecture, developers can ensure that critical system events are never lost while simultaneously providing sub-millisecond updates to frontend interfaces, such as dashboards or live feeds. This skill provides the logical framework to handle dual-delivery paths, ensuring that the primary persistence layer (Kafka) is prioritized while the real-time layer (Redis) operates as an efficient, best-effort broadcast mechanism.
Dual-Stream Architecture Use Cases
- Event-driven systems needing both strict durability AND real-time responsiveness
- Powering WebSocket and SSE backends that require immediate live updates
- Real-time monitoring dashboards that show events as they happen without waiting for Kafka consumer lags
- Systems where Kafka handles heavy background processing while Redis handles instant user notifications
How Dual-Stream Architecture Works
- The Ingester sends an event to the DualPublisher service component.
- The system initiates a write to Kafka; this is the critical path where success is mandatory for the operation to proceed.
- Upon successful Kafka acknowledgment, the system generates a lightweight notification payload.
- This lightweight payload is published to a specific Redis Pub/Sub channel based on a structured naming convention.
- Real-time gateways (like WebSocket servers) subscribe to Redis channels to push updates to clients immediately, while Event Processors consume the full, durable record from Kafka.
Dual-Stream Architecture Setup
To integrate this pattern into your project, use the standard installation command for Openclaw Skills:
npx clawhub@latest install dual-stream-architecture
Ensure your environment has access to a Kafka broker and a Redis instance to support the dual-stream delivery.
Dual-Stream Architecture Data Schema & Taxonomy
The skill follows a specific channel naming convention and data separation strategy to maintain performance:
| Level | Format | Purpose |
|---|---|---|
| Channel Name | events:{source_type}:{source_id} |
Targeted subscription based on entity type and ID |
| Kafka Payload | Full JSON Object | Durable, complete history for replay and processing |
| Redis Payload | Minimal Metadata | Lightweight ID and Type for instant notification |
Common Channel Examples:
events:user:octocat— User-specific event streamevents:repo:owner/repo— Repository-level updatesevents:org:acme— Organization-wide broadcast
Dual-Stream Architecture Advanced Features
- High-throughput batch publishing support for both Kafka and Redis pipelines
- Intelligent fire-and-forget Redis logic to prevent real-time failures from blocking critical data persistence
- Source-level channel sharding to prevent Redis Pub/Sub bottlenecks
- Seamless integration with WebSocket hub patterns for global real-time event distribution
- Built-in error handling strategies for Kafka backpressure and Redis downtime
SKILL.md
Loading
Related Openclaw Skills
Distinctive Design Systems
A specialized skill for engineering design systems that prioritize unique brand personality through structured tokens, layered surfaces, and evocative aesthetics.
Design System Patterns
A foundational skill for architecting scalable design systems through multi-layer token hierarchies, automated pipelines, and robust theming infrastructure.
Design System Creation
A comprehensive workflow for architecting distinctive, token-driven design systems from aesthetic foundation to component implementation.
Design System Components
A standardized framework for building reusable, type-safe UI components using Surface primitives and class-variance-authority.
E2E Testing Patterns
Build reliable, high-performance end-to-end test suites using industry-standard patterns for Playwright and Cypress.
Estimation Patterns
A comprehensive framework for producing accurate, defensible software estimates using mathematical models and systematic decomposition.
Featured*
