Multi-Platform Content Source Discovery and AI Analysis - n8n Workflow

• Researchers and Journalists: Automating the process of finding up-to-date, relevant sources for articles or studies.


• Content Strategists: Identifying new, high-quality websites for content curation or competitor analysis.


• Automation Enthusiasts: Users looking for advanced examples of parallel processing and custom n8n node integration within an n8n workflow.


• Academics: Compiling extensive resource lists for specific subjects.

Manually searching for new, authoritative information sources across specialized platforms like GitHub, Reddit, and niche news services is incredibly time-consuming. This complex n8n workflow solves this by executing parallel searches across four major data streams: Google News (via SerpAPI), DuckDuckGo News, Bluesky, Reddit RSS feeds, and GitHub repository READMEs. After gathering thousands of potential links, the workflow uses powerful filtering and deduplication logic based on base URLs and custom ignore lists. The final and most powerful step uses an AI n8n node (Mistral Cloud Information Extractor) to analyze the actual website content for relevance and recency, providing an intelligent evaluation of the discovered sources. This sophisticated n8n template is essential for anyone needing a robust source discovery tool.

The core of this n8n workflow involves setting configuration variables and executing parallel search branches before merging and filtering the results.

1. Initialization: The n8n trigger starts the process (either manually or via the Schedule n8n trigger). It loads configuration arguments (search themes, existing sources to ignore) from a dedicated 'Configure Workflow Args' n8n node.


2. Parallel Source Discovery: The workflow splits the search themes and executes four simultaneous search paths:

1. Aggregation and Cleaning: All results are consolidated using a Merge n8n node. Data is cleaned by removing entries without links or titles.


2. Filtering and Deduplication: Base URLs are extracted using a Code n8n node. A sequence of Filter and Remove Duplicates n8n nodes removes sources already known or listed on the 'sourcestoignore' list.


3. AI Evaluation: The cleaned list is limited to 36 sources. An HTTP Request n8n node fetches the content of each base URL. The content is truncated (using another Code n8n node) and passed to the Mistral Cloud Information Extractor n8n node. This final step assesses whether the source is relevant, accessible, and recently updated, completing the robust n8n workflow.

To deploy this powerful n8n workflow, follow these steps:

1. Import the JSON: Copy the provided JSON data and import it directly into your self-hosted or cloud n8n instance.


2. Install Custom Nodes: This n8n workflow requires several community or paid n8n node integrations. You must install the SerpApi, Bluesky Enhanced, DuckDuckGo Search, and Global Constants n8n nodes (if using the scheduled path).


3. Set Credentials: Configure credentials for:

1. Configure Arguments: If running manually, edit the 'Configure Workflow Args (Manual)' n8n node to define your searchthemes, existingsources, and sourcestoignore arrays. If running via the Schedule n8n trigger, ensure these variables are configured correctly in the 'Configure Workflow Args' Global Constants n8n node.

Schedule / When clicking ‘Execute workflow’ (n8n trigger): Initiates the flow. The Schedule n8n trigger allows for continuous, automated discovery.
Configure Workflow Args (n8n node): Manages essential variables like searchthemes (e.g., ["technology"]), lists of known sources, and lists of sources to ignore.
DuckDuckGo (n8n node): Performs news searches based on themes. Key Configuration: operation: searchNews, maxResults: 100, safeSearch: Strict.
Bluesky Search (n8n node): Searches recent posts on the Bluesky network. Key Configuration: limit: 10, resource: searchPosts.
Googlenews search (SerpAPI n8n node): Retrieves highly relevant news results. Configuration relies on valid SerpAPI credentials.
Get list of Subreddits (Reddit n8n node): Finds relevant subreddits to scrape for source links. Limited to 1 result.
HTTP Request (n8n node): Used multiple times, particularly to search GitHub repositories and retrieve the content of the discovered base URLs.
Code (n8n node - Get Base URL): Critical custom n8n node logic used to normalize complex URLs down to the root domain for accurate filtering and deduplication.
Filter (n8n node - Remove Existing Content): Compares extracted base URLs against the user-defined list of existing sources to ensure no duplicates are processed.
Limit (n8n node): Implemented before the final AI analysis to prevent excessive API usage, capped at 36 items.
Code (n8n node - Limiting number of Chars): Truncates HTML content to 1500 characters, optimizing the input payload for the subsequent LLM call.
Information Extractor (AI n8n node): The final evaluation step, using Mistral Cloud to analyze the webpage content and output a structured JSON verdict on relevance and update status. This makes the n8n template highly valuable for automated curation.