Indeed Company Data Extraction, Summarization, and AI Agent Formatting - n8n Workflow

Markdown-formatted:
Recruiters and HR Analysts: Who need quick, summarized competitive intelligence on target companies.
Data Engineers: Looking for robust web scraping solutions using Bright Data integrated directly into an n8n node.

• Automation Specialists: Interested in building advanced, multi-step AI processes involving text extraction, summarization, and structured data output using the n8n workflow platform.

This comprehensive n8n workflow demonstrates how to leverage external scraping services (Bright Data Web Unlocker) combined with advanced AI processing (Google Gemini) within a single automation pipeline. It solves the complex problem of extracting structured information from dynamic web sources like Indeed, normalizing the data (markdown to text), summarizing key findings, and using an autonomous AI Agent to ensure the final output is perfectly formatted before being delivered via a webhook.

By utilizing specialized n8n node configurations for large language models (LLMs) and HTTP requests, this n8n template provides a reliable, scalable foundation for sophisticated data intelligence operations. If you frequently need competitive company data or need examples of advanced AI functionality within an n8n workflow, this is an essential resource.

Markdown-formatted:

1. Trigger and Setup: The process begins with a Manual n8n trigger. The subsequent n8n node, 'Set Indeed Search Query', defines the target company name (e.g., Starbucks) and the necessary Bright Data zone for scraping.


2. Web Scraping: The 'Perform Indeed Web Request' n8n node initiates a POST request to Bright Data's Web Unlocker API, requesting the target Indeed company URL. The scraped data is returned in markdown format.


3. Data Normalization (LLM Chain): The workflow splits. One path converts the raw markdown to HTML (for parallel notification). The primary path sends the markdown data to the 'Markdown to Textual Data Extractor' n8n node (powered by a Google Gemini Chat Model) to convert the structured markdown into clean, readable text.


4. Summarization Chain: The cleaned textual data is passed to the 'Indeed Summarization' n8n node, which uses another Google Gemini Chat Model to create a concise summary of the company information.


5. AI Agent Formatting: The summarized text is then fed into the 'Indeed Expert AI Agent' n8n node. This agent, configured with an expert persona and its own Google Gemini Chat Model, uses a specialized tool (Initiate a Webhook Request) to structure and push the final summary and search results to the designated webhook URL.


6. Notifications: Parallel HTTP Request n8n node steps send the raw HTML response and the pure summary text to different webhooks, ensuring full visibility into the data transformation steps of this n8n workflow.

Markdown-formatted:

1. Import the n8n workflow: Copy the provided JSON data and paste it into your n8n instance via the 'Import from JSON' option in the Workflows section.


2. Bright Data Credentials: Configure the HTTP Header Auth credentials used in the 'Perform Indeed Web Request' n8n node with your Bright Data API key.


3. Google Gemini Credentials: Ensure the Google Gemini (PaLM) API credentials are set up and linked to all the LangChain LLM Chat n8n node instances throughout the n8n template.


4. Customize Webhooks: Update the URL parameters in the following HTTP Request n8n node steps with your desired endpoint for receiving data:

1. Customize Query: Modify the 'Set Indeed Search Query' n8n node to specify the company name you wish to scrape.


2. Testing: Execute the n8n workflow by clicking the 'Test workflow' n8n trigger to verify all connections and data flow.

Markdown-formatted:
When clicking ‘Test workflow’ (Manual Trigger): The starting point for this n8n workflow. This n8n trigger is used for development and immediate execution.
Set Indeed Search Query (Set n8n node): Essential setup n8n node. Defines input variables (searchquery and the Bright Data zone) required for the subsequent HTTP Request.
Perform Indeed Web Request (HTTP Request n8n node): Executes the actual web scraping request to Bright Data. Key Configuration: Uses POST method, Generic HTTP Header Auth, and constructs the URL to target indeed.com/cmp/{{ companyname }} while requesting output in markdown format.
Markdown to Textual Data Extractor (LangChain Chain LLM n8n node): Function: Converts the raw, scraped markdown into clean, usable text by leveraging a Google Gemini Chat Model. Key Configuration: Uses a defined prompt: "You need to analyze the below markdown and convert to textual data."
Indeed Summarization (LangChain Chain Summarization n8n node): Function: Takes the cleaned text and generates a brief summary of the Indeed company data. It is paired with the 'Google Gemini Chat Model For Summarization' n8n node.
Indeed Expert AI Agent (LangChain Agent n8n node): Function: The central intelligence n8n node. Acts as an 'Indeed Expert' to format and deliver the final result using the 'Initiate a Webhook Request' tool. Key Configuration: Custom prompt defining the agent's role and instructions for using the tool.

• Initiate a Webhook Request (LangChain Tool HTTP Request n8n node): The final action n8n node used by the AI Agent to send the structured summary data to a final receiving endpoint.