Parallel AI Processing with Fan-Out-Fan-In Architecture - n8n Workflow

Ideal Users

Automation Architects seeking high-performance n8n solutions.
Users who need to run time-consuming tasks (like multiple API calls or AI generations) simultaneously.
Developers looking to understand advanced n8n workflow control mechanisms like self-referencing workflows and static data state management.
Teams focused on optimizing complex n8n templates where latency is critical.

When building complex automation projects, waiting for a long sequence of independent steps to complete serially can drastically slow down your results. This advanced n8n workflow solves this by utilizing the powerful Fan-Out-Fan-In pattern, allowing all independent tasks to run in parallel.

This specific n8n workflow acts as a Project Manager (the main thread) and Specialist Teams (parallel sub-threads). It uses a key combination of the Execute Workflow n8n node (set to not wait) and the Wait n8n node, managed by a central 'Project Dashboard' (implemented using the Code n8n node and Static Data).

By ensuring that three separate AI content generation tasks run at the same time, this n8n template significantly reduces overall execution time, providing a model for optimizing any speed-critical n8n operation.

This intricate n8n workflow operates in two main segments: The Project Manager (Fan-Out) and the Specialist Teams (Fan-In/State Management).

Phase 1: Fan-Out (The Project Manager)

1. Trigger and Brief: The process starts with a manual n8n trigger. The The Project Brief set n8n node defines an array of independent tasks, each specifying the process name (e.g., writedescription) and necessary input data.


2. Task Assignment: The tasks are split out and then passed to the Assign Tasks to Teams n8n node (an Execute Workflow node). Crucially, this node calls the same n8n workflow recursively, passing the task data, the main execution ID, and the unique resumeurl of the next node. Since the 'Wait for Sub-Workflow' option is OFF, the main execution does not wait.


3. Pause: The main n8n workflow immediately moves to the Wait for All Teams to Finish n8n node, which pauses the execution indefinitely, waiting for an external HTTP POST request to its resumeurl.

Phase 2: Parallel Work and State Management

1. Receive Work Order: The sub-executions (the Specialist Teams) enter the n8n workflow via the Receive Work Order n8n trigger.


2. Dashboard Initialization: The Check Work Order Status node ensures the status is 'pending', and the item is passed to the The Project Dashboard (Code) n8n node. This code node uses n8n's Static Data feature to initialize the project dashboard, marking all tasks as 'pending'.


3. Task Execution: The The Dispatcher switch n8n node directs the work order to the correct specialist team (Description Team, Ad Copy Team, or Email Team), where the AI generation happens via the LangChain n8n node.


4. Reporting Back: Once an AI task is complete, the Report Back to Manager n8n node calls the sub-workflow again, but this time with a status of 'completed' and the result data.

Phase 3: Fan-In and Completion

1. Dashboard Update: The completed task data is again processed by the The Project Dashboard (Code) n8n node. This n8n node updates the task status in the Static Data and checks if all tasks are now complete.


2. Resumption: If all tasks are done, the Code n8n node outputs a specialized item containing the aggregated results and the original resumeurl. The Is Project Complete? if n8n node directs this item to the Resume Parent Workflow HTTP Request n8n node.


3. Finalization: The Resume Parent Workflow n8n node makes a POST request to the main workflow's Wait n8n node, resuming its execution. The aggregated results are received by the Project Complete! n8n node, marking the successful and parallel completion of the entire n8n workflow.

To deploy this powerful n8n workflow template, follow these steps:

1. Import the n8n Workflow: Copy the provided JSON code and paste it into your n8n instance using the 'New' -> 'Import from JSON' feature.


2. Credentials Setup: This n8n workflow uses the LangChain Chat Google Gemini n8n node for AI processing. You must set up a credential named 'IA2S' (or update the credential ID in the LangChain nodes) with your Google Gemini API key.


3. Static Data Configuration: Ensure your n8n instance supports Static Data usage, as the The Project Dashboard (Code) n8n node relies heavily on $getWorkflowStaticData('global') for state persistence across executions.


4. Activation: Ensure the n8n workflow is activated (set to 'Active' mode). Since it uses a manual n8n trigger, you can initiate the execution directly from the n8n editor to test the parallel processing capabilities.

This n8n workflow template utilizes several complex and custom-configured n8n node components:

Start Project (Manual Trigger n8n trigger): The starting point for the main execution.
The Project Brief (Set n8n node): Defines the input payload, which is an array of tasks (e.g., 'writedescription', 'writeadcopy') that need to be run in parallel.
Assign Tasks to Teams (Execute Workflow n8n node): The Fan-Out mechanism. It calls the current n8n workflow ID recursively, passing the task data, but crucially, Wait for Sub-Workflow is disabled. It passes the resumeurl for later callback.
Wait for All Teams to Finish (Wait n8n node): Pauses the main execution. Its unique resumeurl is passed to the sub-workflows, acting as the final callback endpoint.
Receive Work Order (Execute Workflow Trigger n8n trigger): The entry point for all parallel sub-executions initiated by the Assign Tasks to Teams n8n node.
The Dispatcher (Switch n8n node): Based on the incoming process name (e.g., writedescription), this n8n node routes the request to the correct specialist team.
Description/Ad Copy/Email Team (LangChain Agent n8n nodes): These nodes perform the actual AI work, receiving specific instructions and product data from the input payload. They are linked to the The AI Specialist lmChatGoogleGemini n8n node.
The Project Dashboard (Code n8n node): The central state manager. It uses $getWorkflowStaticData('global') to track the status of all parallel tasks. It updates the status from 'pending' to 'completed' and checks if all work is done before initiating the Fan-In.

• Resume Parent Workflow (HTTP Request n8n node): The Fan-In mechanism. This n8n node is only executed when the dashboard confirms all tasks are complete. It performs a POST request to the previously saved resume_url, sending the aggregated results and waking up the main n8n workflow.