A professional-grade data engineering assistant for designing scalable ETL/ELT pipelines, dimensional modeling, and automated data quality frameworks.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install kv-senior-data-engineering
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install kv-senior-data-engineering using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is Senior Data Engineer?
The Senior Data Engineer skill is a specialized toolset designed for building production-ready data infrastructure within the Openclaw Skills framework. It focuses on the end-to-end lifecycle of data engineering, from initial architectural decisions like choosing between Lambda and Kappa patterns to the deployment of optimized Airflow or dbt workflows. It ensures that every pipeline is built with industrial-strength standards, including mandatory data quality checks and idempotency strategies.
This skill serves as a technical consultant and automation engine for developers working with large-scale data systems. By integrating best practices in DataOps and observability, it helps teams avoid common pitfalls like data loss or high operational costs. Whether you are modeling a star schema or troubleshooting a slow Spark job, this skill provides the structured logic and scripts necessary to maintain high-performance data environments.
Senior Data Engineer Use Cases
- Designing batch, streaming, or hybrid data architectures based on latency requirements.
- Building robust ETL/ELT pipelines using tools like Airflow, Prefect, Dagster, dbt, or Spark.
- Implementing data modeling strategies such as star schemas, snowflake schemas, and Data Vault.
- Setting up automated data quality validation and data contracts to prevent pipeline breakage.
- Optimizing slow warehouse queries and Spark jobs through performance analysis and tuning recommendations.
How Senior Data Engineer Works
- Classification: The skill analyzes the user request to identify the primary focus, such as architecture, pipeline construction, or quality validation.
- Reference Loading: It retrieves specific architectural patterns or troubleshooting guides from its internal knowledge base to ensure technical accuracy.
- Script Execution: It utilizes built-in Python scripts to generate orchestration configurations, validate schemas, or analyze performance.
- Artifact Generation: The skill produces high-quality outputs, such as dbt models or pipeline DAGs, while explicitly stating any design trade-offs.
- Observability Integration: Every workflow concludes with specific recommendations for monitoring and incident response thresholds.
Senior Data Engineer Setup
The Senior Data Engineer skill requires Python 3.8+ for its core automation scripts. To begin using the orchestration and validation features, ensure your environment is configured to run the provided Python utilities.
# Generate a pipeline orchestration config for Airflow
python scripts/pipeline_orchestrator.py generate \
--type airflow --source postgres --destination snowflake --schedule "0 5 * * *"
# Run a data quality validation check on a Parquet file
python scripts/data_quality_validator.py validate \
--input data/file.parquet --schema schemas/file.json \
--checks freshness,completeness,uniqueness
Senior Data Engineer Data Schema & Taxonomy
The skill organizes its technical references and execution logic into a structured directory system to ensure consistency across the Openclaw Skills ecosystem:
| Directory | Purpose |
|---|---|
references/ |
Contains markdown files for architectural patterns, modeling, and DataOps best practices. |
scripts/ |
Python utilities for generating DAGs, validating data quality, and optimizing performance. |
schemas/ |
Storage for JSON-based schema definitions and data contracts. |
data/ |
Workspace for temporary data artifacts and files used during validation routines. |
Senior Data Engineer Advanced Features
- Multi-Engine Orchestration Support: Seamlessly generate configurations for Airflow, Prefect, and Dagster.
- Automated Data Quality Checks: Built-in validation for freshness, uniqueness, and schema drift as part of the pipeline lifecycle.
- Performance Optimization Engine: Analyzes Spark and SQL queries to provide actionable recommendations for query tuning.
- Idempotency & Error Handling: Automatically incorporates retry logic and dead-letter queue strategies into all pipeline designs.
- Architecture Trade-off Analysis: Provides detailed comparisons between batch and streaming options based on stated latency needs.
SKILL.md
Loading
Related Openclaw Skills
GitHub CLI Integration
A powerful integration that enables AI agents to manage GitHub repositories, pull requests, and CI/CD pipelines using the official gh CLI.
Romance AI Agent Skill
The Romance skill enables AI agents to find and build romantic connections through deep personality matching and communication alignment.
Intelligent Continuous Requirements Engineering Skill
A comprehensive framework for AI-augmented continuous requirements engineering that transforms vague ideas into evolving, living documentation systems.
Cactus
Cactus is a specialized dating and relationship platform designed specifically for AI agents to find meaningful, personality-matched connections.
Intelligent Expense Assistant (fb-expense-skill)
An AI-driven assistant that automates the entire expense reimbursement lifecycle from invoice OCR to smart approval tracking.
Image Embedding
A high-performance image vectorization tool designed to transform visual data into searchable 1024-dimensional feature vectors.
Featured*
