Tech News Article Scraper using Advanced HTML Parsing - n8n Workflow

Data Analysts and Researchers who need structured data feeds from journalistic sources.
Content Curators looking to aggregate and analyze the latest technology articles.
Automation specialists seeking advanced examples of using the n8n HTML node for deep parsing.
Users looking for robust n8n templates for web scraping projects.

Acquiring structured data from complex websites like TechCrunch often requires sophisticated web scraping techniques. This powerful n8n workflow provides a complete, no-code solution for deep article extraction. Instead of just grabbing titles from an RSS feed, this n8n automation fetches the latest articles, iterates through each one, and visits the detail page to extract the full article body content, ensuring clean and complete data. This approach showcases the strength of the n8n node ecosystem, particularly the combination of HTTP Request, HTML Parser, and Split Out nodes to handle dynamic content lists. By implementing this n8n workflow, users save significant time on manual data collection and benefit from reliable, automated data feeds, making it one of the most useful n8n templates for information gathering.

This comprehensive n8n workflow executes a multi-stage web scraping process:

1. Trigger Initialization: The process begins with a manual n8n trigger, allowing the user to initiate the scrape when desired.


2. Initial Request: An HTTP Request n8n node fetches the raw HTML of the TechCrunch 'Latest' page.


3. Isolating the List: The first HTML n8n node analyzes the page structure and extracts the main container (ul.wp-block-post-template) holding all the post listings.


4. Extracting Post Entries: A subsequent HTML n8n node specifically targets and extracts each individual post (li.wp-block-post) as an array of HTML segments.


5. Itemization: The Split Out n8n node iterates over this array, creating separate items in the n8n workflow for each of the 20 articles found.


6. Metadata Extraction (List View): For each article item, an HTML n8n node extracts initial metadata (image URL, primary title, article URL, and creation timestamp) directly from the list item HTML.


7. Detail Page Fetch: An HTTP Request n8n node uses the extracted article URL to request the full detail page content.


8. Content Parsing (Detail View): A final HTML n8n node processes the detail page HTML, focusing on extracting the full article body (div.entry-content), cleaning up the text, and confirming the final title.


9. Final Output Assembly: The Set n8n node consolidates all the information—metadata from the list view and content from the detail view—into a clean, unified JSON structure. This structured data is the final result of the n8n workflow.

To start using this specific n8n workflow, follow these steps:

1. Import the n8n workflow: Copy the provided JSON code.


2. In your n8n instance, click 'New' and then 'Import from JSON'. Paste the code and save the workflow.


3. No Credentials Needed: This n8n template uses only core nodes and scrapes public web content via HTTP Request, so no external credentials are required.


4. Execution: Since the starting n8n trigger is a Manual Trigger, click the 'Test workflow' button to run the entire data extraction process immediately and view the results in the execution history.


5. To schedule this as a recurring process, change the n8n trigger node to a Schedule Trigger.

When clicking ‘Test workflow’ (Manual Trigger): The initiating n8n trigger for the automation. It allows for quick testing and manual execution of the data scrape.
Request Techcrunsh Latest Page (HTTP Request): Sends a GET request to https://techcrunch.com/latest/0 to retrieve the initial listing page HTML.
Parse a posts box (HTML): Extracts the main container element (ul.wp-block-post-template) from the HTML response, focusing the subsequent parsing steps on the relevant article list area.
Parse all posts (HTML): This critical n8n node extracts all individual article blocks (li.wp-block-post) into an array structure, preparing them for iteration.
split out the posts (Split Out): Essential for flow control, this n8n node converts the array of articles into individual data items, ensuring that the scraping process visits the detail page for every single article.
Parse each post in detail (HTML): Extracts initial metadata (URL, image, title) from the list item HTML for each post item.
Request a post detail page (HTTP Request): Uses an expression (={{ $json.url }}) to dynamically request the full content page for the article, crucial for obtaining the entire text body.
Parse a post's content and metadata (HTML): Focuses on the detail page to extract the full article content from div.entry-content, ensuring clean, trim text output. This advanced n8n node is the heart of the extraction.

• Save the values (Set): This final n8n node compiles the final output, mapping and cleaning data fields using references from previous nodes (e.g., $('Parse each post in detail').item.json.url) to create the final, standardized JSON payload.