A powerful utility for identifying and prioritizing flaky tests by analyzing JUnit retry artifacts within your CI pipeline.
Install & Download
1. ClawHub CLI
The fastest way to install a skill directly from the registry.
npx clawhub@latest install ci-flake-triage
2. Manual Installation
Copy the skill folder to one of these locations
~/.openclaw/skills/ <project>/skills/ Priority: Workspace > Local > Bundled
3. Prompt Installation
Copy this prompt to OpenClaw to install it automatically.
Help me install ci-flake-triage using Clawhub. If Clawhub is not installed, install it first (npm i -g clawhub).
Prefer to download?
Get the raw skill files in a ZIP archive.
What is CI Flake Triage?
CI Flake Triage is a specialized tool designed for engineering teams looking to improve test suite reliability and developer productivity. By analyzing multiple JUnit XML reports—such as initial test runs and subsequent automated retries—this skill identifies tests that exhibit inconsistent behavior. It provides developers with the clarity needed to distinguish between persistent failures that require immediate code fixes and flaky tests that undermine CI stability.
Integrating this tool into your workflow via Openclaw Skills ensures that your team focuses stabilization efforts on the most impactful issues. By reducing noise in your build reports, you can maintain a higher level of trust in your automated testing suite while streamlining the debugging process for complex distributed systems.
CI Flake Triage Use Cases
- Identifying flaky tests in continuous integration pipelines after automated retries have occurred.
- Prioritizing stabilization efforts by ranking the most unstable test cases based on frequency.
- Generating machine-readable JSON reports for custom dashboard ingestion or long-term trend analysis.
- Gating CI builds based on the presence of persistent failures or newly discovered flaky test candidates.
How CI Flake Triage Works
- The skill scans the specified directory for JUnit XML files using a configurable glob pattern to capture all test artifacts.
- It parses the XML results to aggregate the status (passed, failed, skipped, or error) for every unique test case across multiple files.
- Logic is applied to compare multiple entries for the same test; if both failure and success outcomes exist across different runs, the test is flagged as a flaky candidate.
- Persistent failures are automatically separated from intermittent flakiness to provide a clear, actionable triage report.
- The final analysis is emitted in either human-readable text or structured JSON format, complete with exit codes for automated pipeline control.
CI Flake Triage Setup
Ensure you have bash and python3 installed on your environment. You can execute the triage script directly by targeting your test artifacts directory. This skill is designed to work seamlessly within the Openclaw Skills ecosystem.
# Basic execution with custom glob
JUNIT_GLOB='artifacts/junit/**/*.xml' \
bash skills/ci-flake-triage/scripts/triage-flakes.sh
# Run with JSON output for automated CI ingestion
OUTPUT_FORMAT=json \
FAIL_ON_PERSISTENT=1 \
bash skills/ci-flake-triage/scripts/triage-flakes.sh
CI Flake Triage Data Schema & Taxonomy
The skill processes standard JUnit XML files and generates a summary based on the following taxonomy:
| Property | Description |
|---|---|
| Test Case Name | The unique identifier for the specific test scenario captured in the XML. |
| Status Aggregation | A summary of all outcomes for a specific test (e.g., 1 pass, 2 failures). |
| Flaky Candidate | Flagged when a test has both fail-like and pass outcomes in the same set. |
| Persistent Failure | Flagged when a test has only failure or error outcomes across all retries. |
| Triage Top | A configurable limit (default 20) on the number of flaky tests displayed in the report. |
CI Flake Triage Advanced Features
- Custom thresholds for reporting top unstable tests using the TRIAGE_TOP environment variable.
- Strict CI gating options (FAIL_ON_PERSISTENT and FAIL_ON_FLAKE) to force non-zero exit codes during build steps.
- Multi-format output support for both developer readability and automated machine-to-machine integration.
- Support for complex, nested directory structures via advanced JUnit glob patterns.
- Seamless integration with standard CI/CD runners that generate JUnit artifacts, making it a versatile addition to Openclaw Skills.
SKILL.md
Loading
Related Openclaw Skills
McDonald's China MCP Order Lite
A streamlined MCP-based skill for automating McDonald's China delivery orders, menu lookups, and coupon management via official API protocols.
Perpetua
A hosted OAuth proxy that enables seamless data retrieval from Oura Ring and Google Calendar using a single master API key.
Calendar Bridge
A self-hosted Node.js service that provides a persistent REST API for Google Calendar with automated OAuth token refreshing.
Tour Booking
A specialized AI sub-agent for automating outbound property showing calls and managing booking statuses.
Dockerfile Hardening Audit
Statically audit Dockerfiles to identify and mitigate security risks before container deployment.
Env Alias Audit
A robust auditing tool to detect environment variable alias conflicts and missing required configurations before deployment.
Featured*
