AI Lead Validation and Deduplication using Gemini and Google Sheets - n8n Workflow

This automation template is ideal for:
Growth Marketers needing automated lead processing pipelines.
Sales Operations teams requiring clean, deduplicated CRM input data.
Users looking for advanced examples of AI Agents within an n8n workflow.
Developers seeking robust n8n templates for handling large datasets and external API calls.

This sophisticated n8n workflow solves the common challenge of integrating messy scraped lead data (from sources like Apify or Apollo) into a structured format like Google Sheets. Instead of relying on rigid conditional logic, this n8n automation utilizes the power of a Google Gemini AI Agent, which is configured within a specialized n8n node.

The core functionality of this n8n template is its ability to handle large batches (up to 1000 leads), perform complex validation (checking for required fields like Name, Email, and Company), handle deduplication checks against existing sheets, and generate unique Lead IDs (e.g., AP-DDMMYY-xxxx). This specific n8n node configuration ensures data quality and provides transparent, consolidated reporting via Telegram, making this a highly valuable and reliable business automation tool. Implementing this n8n workflow significantly reduces manual data cleaning efforts.

This comprehensive n8n workflow operates in a controlled loop to manage large datasets efficiently:

1. Execution Trigger: The process begins using the n8n trigger node, "When Executed by Another Workflow," making this a highly reusable n8n workflow component that can be called by a scheduling or webhook n8n trigger.


2. Data Fetch: The APIFY Post Request n8n node calls the scraping API (Apify/Apollo) to initiate or fetch the lead data. This is followed by an Apify Get Request n8n node used in the loop for pagination/fetching the next data chunk.


3. Batching: The Loop Over Items n8n node splits the retrieved leads into manageable batches (configured to 1000 items) to prevent memory or API timeouts, ensuring the n8n automation runs efficiently.


4. AI Processing: Each batch is passed to the Knowledge Base Agent n8n node, which leverages the Google Gemini Chat Model. The agent follows strict, detailed instructions: validate required fields, perform real-time deduplication via the Google Sheets Get Row(s) n8n tool, and use the Google Sheets Append n8n tool to insert clean data. Memory is maintained for context across processing steps.


5. Reporting: Once the agent completes processing a batch, it outputs a single consolidated summary message (containing successes, flagged items, and skipped stats). This output is then sent by the Send a text message n8n node via Telegram.


6. Continuation Loop: The batch loop enforces rate limiting using a Wait n8n node (30 seconds) before proceeding to fetch the next 1000 records, completing the powerful n8n workflow cycle until all leads are processed.

To set up this advanced n8n workflow template, follow these steps:

1. Import: Copy the provided JSON and import it into your n8n instance.


2. Credentials Setup: Configure the required credentials:

1. Spreadsheet Preparation: Create your Google Sheet with the required columns: Lead ID, Name, Email, Phone Number, Company Name, Job Title, Website/LinkedIn, Address, Company Summary, Relevant Partner.


2. Activation: Once all connections and the spreadsheet are configured, activate the n8n workflow. It is designed to be triggered by a master n8n trigger (e.g., a scheduled execution or a webhook) that supplies the initial parameters.

When Executed by Another Workflow (n8n trigger): The designated starting point for this automation, allowing it to function as a modular sub-workflow.
APIFY Post Request / Apify Get Request (n8n node): HTTP Request n8n nodes used to manage data retrieval from the Apify/Apollo scraping API.
Loop Over Items (n8n node): Splits incoming data into batches of 1000 items (batchSize: 1000), a critical component for handling large lead volumes within the n8n workflow.
Knowledge Base Agent (n8n node): The central AI processing core. It is configured with strict instructions for data validation, ID generation, and orchestrating downstream actions via connected n8n tools.
Google Gemini Chat Model (n8n node): Provides the necessary large language model capability for the AI Agent's reasoning and transformation tasks.
Simple Memory (n8n node): Ensures context persistence for up to 20 messages within the agent's processing cycle, improving performance when handling sequential tasks.
Append row in sheet in Google Sheets / Get row(s) in sheet in Google Sheets (n8n tool): These Google Sheets n8n tools are connected directly to the AI Agent, allowing it to dynamically perform deduplication lookups and insert validated rows.
Send a text message (n8n node): Sends the final, detailed batch summary generated by the AI agent to a designated Telegram chat ID, ensuring timely reporting of the n8n workflow execution status.