Website API Endpoint Discovery using AI and Regex - n8n Workflow

Security Researchers and Penetration Testers mapping application attack surfaces.
Developers requiring programmatic access to services that lack documented public APIs.
Technical professionals looking for advanced n8n templates combining web scraping, core logic, and generative AI.
n8n users interested in building complex AI Agent pipelines and custom regex validation loops.

Modern web applications often rely on internal API endpoints embedded within client-side JavaScript, which are rarely documented publicly. This sophisticated n8n workflow solves the challenge of discovering these hidden endpoints by providing a dual-pronged approach.

First, it uses a robust predefined regex pattern to quickly extract potential endpoint strings from all relevant JS files. Second, it utilizes a powerful AI Agent (powered by models like Claude 3.7 Sonnet) to iteratively generate highly specific, accurate regex expressions based on an initial LLM analysis (Gemini 2.5 Pro). This n8n workflow features a crucial self-validation loop where the Agent tests its generated regex against the source data, ensuring high accuracy.

By merging the results from both the standard regex extraction and the refined AI extraction, this n8n template guarantees maximum coverage, providing a comprehensive final list exported to an XLSX file for easy analysis. This is an extremely valuable n8n node arrangement for technical web analysis.

This advanced n8n workflow executes in several stages, initiated by the manual n8n trigger:

1. Initial Extraction & Filtering: The workflow fetches the target URL and uses the HTML n8n node to extract all JavaScript file URLs. A Filter n8n node then keeps only relevant paths (e.g., local, bundled, CDN sources).


2. JS Content Retrieval: The workflow downloads the content of these filtered JS files using an HTTP Request n8n node.


3. Predefined Regex Extraction (Path 1): A Set n8n node applies a complex predefined regex to the JS content, extracting and cleaning potential API endpoints. An If n8n node filters out files that yield fewer than five endpoints, streamlining data flow. The results are then deduplicated.


4. AI Initial Analysis (Stage 1): The raw JS content is sent to an LLM (Gemini 2.5 Pro via OpenRouter) for in-depth analysis. The analysis report, which identifies specific endpoints, methods, and descriptions, is saved as a reference file locally on the n8n instance.


5. AI Agent Custom Regex Generation (Stage 2 & 3): An AI Agent (Langchain n8n node, using Claude 3.7 Sonnet) reads the analysis report and generates an optimal n8n expression (regex) to extract the clean endpoints. The Agent validates this expression using a recursive process: it calls a sub-workflow via the Workflow Tool n8n node, which tests the regex against the saved analysis file content. This iteration ensures the generated n8n regex is reliable.


6. Final LLM Execution (Path 2): Once the Agent provides the validated, specific regex, another Set n8n node executes this custom expression on the data.


7. Merging and Export: The final step uses Merge n8n node to combine endpoints from both the predefined regex path and the successful LLM regex path. The results are sorted and then exported into a comparison XLSX file using the Convert to File n8n node. This robust process ensures all possibilities are covered by this comprehensive n8n workflow.

To install and run this n8n workflow, follow these steps:

1. Import the Workflow: Copy the provided JSON and import it into your n8n instance using the 'New' menu -> 'Import from JSON'.


2. Set up OpenRouter Credentials: This workflow requires access to large language models. You must configure an OpenRouter API credential.

1. Run the n8n trigger: Execute the workflow manually using the Start API Discovery n8n trigger node to begin the analysis.

Start API Discovery (Manual Trigger n8n trigger): The entry point to the entire process, initiating the website analysis.
Fetch Website HTML (HTTP Request n8n node): Responsible for initiating the web scrape, fetching the main HTML page content.
Extract URLs of JS files (HTML n8n node): Extracts all script source URLs from the fetched HTML using a configured CSS selector.
Keep Relevant JS Files (Filter n8n node): Filters the extracted URLs, keeping only those likely to contain application logic and API calls (e.g., local or bundle paths).
Extract API Endpoints (Set n8n node): Applies the initial, broad regex to the JS content to find and clean potential API endpoint strings.
AI Endpoints Analysis (HTTP Request n8n node): Connects to the OpenRouter API to send the JS content to a powerful LLM (Gemini 2.5 Pro) for detailed semantic analysis.
Save Endpoints File(s) (Read/Write File n8n node): Persists the LLM analysis results onto the n8n drive, creating temporary files necessary for the subsequent Agent validation process.
Create Endpoints Regex With AI (Langchain Agent n8n node): The core AI component. This agent utilizes system prompts and tools (including the Validation sub-workflow) to generate a precise, optimized n8n expression (regex).
Validate LLM Regex (Langchain Tool Workflow n8n node): Executes a sub-workflow, triggered by the Agent, to test the validity of the generated regex. This self-correction loop is key to the high performance of this n8n workflow.
Execute LLM Regex (Set n8n node): Executes the validated, AI-generated n8n expression on the source data using $evaluateExpression() to extract the final set of LLM-verified endpoints.
Merge Original And LLM Endpoints (Merge n8n node): Combines the endpoint lists derived from both the initial regex and the refined LLM regex paths, creating a single comprehensive dataset for the final output.
Export Comparison Results To Excel (Convert to File n8n node): Converts the final merged data into an easy-to-read XLSX format.