GitHub Profile Data Scraper to Google Sheets - n8n Workflow

Recruiters and Sourcers looking to gather comprehensive data on potential developer candidates.
Data Analysts needing structured, enriched data from web sources.
Automation Specialists seeking advanced examples of dynamic spreadsheet management within an n8n workflow.
Users of BrowserAct who need a robust system to process complex web scraping outputs using an n8n node.

This sophisticated n8n workflow acts as a data enrichment engine. It solves the challenge of turning a basic list of GitHub URLs into detailed, organized records. Starting from a source list in Google Sheets, the n8n workflow uses the BrowserAct n8n node to perform headless browser scraping on each profile. Critically, it includes a Code n8n node to repair and consolidate the messy JSON output often resulting from complex scraping tasks. Finally, instead of dumping all data into one sheet, this n8n template dynamically creates a separate, organized sheet (tab) for each GitHub user, ensuring clean data separation for profile info, links, and repositories. This level of structure makes the resulting reports immediately actionable.

The entire process is initiated by the Manual n8n trigger, or can be set to run on a schedule.

1. Input Reading: The n8n workflow starts by pulling a master list of GitHub URLs from a designated source Google Sheet via the Get row(s) in sheet n8n node.


2. Iterative Processing: The Loop Over Items n8n node ensures that each profile URL is processed sequentially, creating a separate, dedicated execution path for each user.


3. Scraping Execution: A Slack message is sent, confirming the start of the task. The Run a workflow task (BrowserAct) n8n node then launches the external scraping job, passing the profile URL as input.


4. Wait and Retrieve: The Get details of a workflow task n8n node pauses the n8n workflow execution until the scraping task is fully complete and retrieves the raw output data.


5. Data Fix and Consolidation: The specialized Code in JavaScript n8n node takes the raw output, fixes common JSON formatting errors (like missing quotation marks around keys), and merges all extracted data points (profile summary, links, repositories) into one clean data item.


6. Dynamic Sheet Creation: For the newly processed user, the n8n workflow dynamically creates a new Google Sheet tab named after the user, clears any existing data, and sets up headers.


7. Output Formatting: The consolidated data is segmented using Split Out n8n nodes into three categories: User Data (profile summary), User Links, and User Repositories.


8. Final Write: Each data segment is written to the user's dedicated sheet using separate Append row in sheet Google Sheets n8n nodes, finalizing the structured report.

To deploy this n8n workflow template, follow these steps:

1. Import: Copy the provided JSON into your n8n instance using the 'Import from JSON' option.


2. Google Sheets Credentials: Configure the Google Sheets OAuth2 API credentials. You will need to specify both the input spreadsheet (containing the list of URLs in a column named URL) and the output spreadsheet (where the extracted data will be written).


3. BrowserAct Setup: Set up your BrowserAct API credentials. Ensure you have the required external BrowserAct template active and note its Workflow ID to update the Run a workflow task n8n node.


4. Slack Setup (Optional but Recommended): Configure your Slack OAuth2 API credentials and update the Channel ID in the Send a message n8n node for execution alerts.


5. Execution: Use the manual n8n trigger to test the n8n workflow or set it to Active status for scheduled runs.

This n8n workflow relies on several core and community n8n node types to manage the flow and data transformation:

When clicking ‘Execute workflow’ (ManualTrigger): The primary n8n trigger used for manual execution of the data pipeline.
Google Sheets (Get row(s) in sheet): Fetches the list of target GitHub URLs from your predefined input spreadsheet.
Loop Over Items (SplitInBatches): A flow control n8n node that ensures the entire automation process is executed individually for every URL retrieved, maintaining data integrity.
BrowserAct (Run a workflow task/Get details of a workflow task): These specialized n8n nodes handle the interaction with the external web scraping service. They initiate the scraping task and wait for the rich, detailed JSON output.
Key Configuration: Passes ={{ $json.URL }} to the BrowserAct template's Target_Page parameter.
Code in JavaScript (Code n8n node): Essential for data cleaning. This n8n node uses JavaScript to apply regex fixes to malformed JSON strings, parse the cleaned string, and consolidate data objects into a single, unified item before writing to Sheets.
Google Sheets (Create sheet, Clear sheet, Append row in sheet): Manages the output. The Create sheet n8n node uses a dynamic expression (={{ $json.Name }}) to generate unique tabs for each user report.
Split Out n8n node: Used three times (Split Out1, Split Out, Split Out2) to separate the single consolidated data item into distinct lists ready for row-by-row writing (e.g., separating user profile info from a list of repositories).