Recipe Data Extraction and Structuring using Bright Data and OpenAI - n8n Workflow

Data Scientists and Analysts requiring structured, high-volume web data.
Developers looking for advanced n8n templates integrating scraping and AI.
Users of Bright Data's Managed Collecting Platform (MCP).
Anyone needing an n8n workflow to handle complex, multi-page data extraction.

This advanced n8n workflow tackles the significant challenge of extracting structured data from dynamically paginated websites. Traditional scraping often fails to consistently extract clean JSON from messy HTML. This solution combines the power of Bright Data's MCP n8n node for reliable web scraping with the intelligence of OpenAI's GPT-4o mini LLM to parse and structure the scraped content.

The n8n template is designed for efficiency: it first scrapes the initial page to find all paginated links, and then loops through each link, applying a sophisticated LLM-based parsing step. The result is clean, validated JSON output ready for storage or further analysis. This is a highly effective n8n workflow for anyone serious about automating complex data pipelines.

The process begins with a Manual n8n trigger, initiating the data pipeline.

1. Initialization: The n8n workflow starts by listing Bright Data tools and then assigning the target website URL (a recipe search result page) and a notification webhook URL using the 'Set the Recipe Extract URL' n8n node.


2. Initial Scraping: The 'Bright Data MCP Client For Recipe Extract' n8n node scrapes the initial search results page in HTML format.


3. Pagination Extraction: The raw HTML content is passed to the 'Paginated Data Extract' Chain LLM, which uses the 'OpenAI Chat Model' (GPT-4o mini) and a structured output parser n8n node to identify all subsequent page links. The output is an array of objects containing page numbers and corresponding links.


4. Loop Setup: The 'Code to output the array of paginated info' node formats the data, which is then split by the 'Loop Over Items' n8n node, creating individual items for each page link.


5. Iterative Scraping & Delay: Within the loop, a 'Wait' n8n node introduces a 10-second delay to manage request load. The 'Bright Data MCP Client For Recipe Extract Within The Loop' n8n node scrapes the specific URL for that iteration.


6. Structured Recipe Extraction: The newly scraped content is fed into the 'Structured Recipe Data Extract' Chain LLM, utilizing a dedicated 'OpenAI Chat Model' and a highly detailed structured output parser n8n node to generate a standardized JSON schema of the recipe data (title, rating, category, etc.).


7. Output & Storage: The final structured data is simultaneously sent to a 'Webhook Notification' and converted into binary format. The 'Write the structured content to disk' n8n node saves the final, clean JSON file locally, completing the execution of this complex n8n template.

1. Import: Copy the provided n8n workflow JSON and paste it into your n8n instance using the 'New' -> 'Import from JSON' option.


2. Credentials: This n8n workflow requires two primary credentials:

1. Community Node: Note that this n8n template uses the community 'MCP Client' n8n node, which requires an n8n self-hosted environment.


2. Customization: Update the URL in the 'Set the Recipe Extract URL' n8n node and ensure you replace the placeholder URL in the 'Webhook Notification...' n8n node with your desired endpoint.


3. Execution: Click the 'When clicking ‘Test workflow’' n8n trigger to run a test execution.

Manual Trigger (When clicking ‘Test workflow’): The initial n8n trigger used for manual testing and execution of the n8n workflow.
Set (Set the Recipe Extract URL): Defines static input variables, including the url for the initial scrape, the webhookurl for notifications, and the baseurl for relative link reconstruction.
MCP Client (Bright Data MCP Client For Recipe Extract): An n8n node that interfaces with Bright Data's scraping infrastructure, configured here to use the scrapeashtml tool for reliable web scraping.
Chain LLM (Paginated Data Extract): This n8n node uses prompt engineering with the connected OpenAI Chat Model (GPT-4o mini) to locate and extract page links from the raw HTML content, essential for this complex n8n workflow.
Structured Output Parser for Paginated Data: Provides the LLM with a strict schema [{page_number, link}], ensuring the output is perfectly structured for the subsequent 'Loop Over Items' n8n node.
Split In Batches (Loop Over Items): Manages the iterative process, taking the array of extracted links and processing them sequentially through the subsequent scraping and extraction steps.
Chain LLM (Structured Recipe Data Extract): The core AI processing n8n node, which takes the HTML content of a single recipe page and applies a complex JSON schema via the output parser to pull out specific recipe attributes like title, rating, and dietary options.
HTTP Request (Webhook Notification for Data Extract Within the Loop): An n8n node configured to POST the final structured JSON data to a specified webhook endpoint, providing real-time data export.

• ReadWriteFile (Write the structured content to disk): Writes the final output from the n8n workflow to a local file path as a structured JSON file, leveraging the page number from the loop for unique naming.