Automated GitHub Trending Repository Tracker using ScrapeOps and Google Sheets - n8n Workflow

• Developers and engineering managers tracking open-source innovation.


• Data analysts needing structured data on trending dev tools.


• Users looking for advanced examples of web scraping using the n8n platform.


• Anyone needing robust n8n templates for external data collection.

This automation solves the problem of manually monitoring popular open-source projects. Leveraging a powerful scheduled n8n trigger, this specific n8n workflow fetches real-time data from GitHub Trending pages, handles complex HTML parsing using custom JavaScript logic, and ensures data quality by normalizing metrics. By using the ScrapeOps n8n node, the process is resilient against common scraping challenges. The final output is neatly organized in Google Sheets, allowing users to analyze trends over time (daily, weekly, or monthly) without duplicate entries, thanks to the upsert function within the Google Sheets n8n node. This particular n8n workflow is an excellent example of using specialized n8n templates for actionable business intelligence.

1. Schedule Trigger: The n8n workflow starts on a recurring schedule (e.g., daily or weekly) via the Cron n8n trigger.


2. Define Inputs and Build URLs: The Set Inputs n8n node defines the period (since: daily/weekly/monthly) and desired languages. The subsequent Code n8n node, Build URLs, dynamically creates the specific GitHub Trending URLs needed for scraping, generating multiple items if multiple languages are specified.


3. Scraping Loop: The Split URLs (loop) n8n node iterates over each generated URL one by one. The ScrapeOps n8n node is then executed to reliably fetch the HTML content of the page. A Polite Delay follows, demonstrating responsible scraping practice within the n8n workflow.


4. Data Extraction: The Extract Trending Repos Code n8n node performs precise parsing of the scraped HTML using regular expressions, isolating critical repository metadata like owner, name, description, rank, and raw star/fork counts.


5. Normalization and Scoring: The Normalize + Enrich-from-page Code n8n node cleans numerical data (converting 'K' and 'M' notation to integers), calculates a custom score to rank importance, and generates a unique dedupekey (e.g., daily|owner/repo). This score calculation logic is a key feature of this n8n template.


6. Raw Data Storage: The structured data is committed to the trendingraw Google Sheet tab using the Google Sheets n8n node. It uses the "Append or Update Row" operation based on the unique dedupekey generated by the preceding n8n node, guaranteeing historical integrity and preventing duplicates.


7. Weekly Summary Generation (Optional): A separate branch of the n8n workflow reads the existing raw data, filters it for the current week, calculates weekly summaries (top 10 repositories by score, top thematic keywords), and writes a concise summary to the weeklybrief tab.

1. Import the n8n Workflow: Copy the provided JSON and import it directly into your n8n instance.


2. ScrapeOps Setup: Create an account with ScrapeOps and obtain your API key. Install the ScrapeOps community n8n node if it's not already available. Configure the required ScrapeOps credentials within the ScrapeOps: Fetch HTML n8n node.


3. Google Sheets Setup: Duplicate the required Google Sheet template (linked in the workflow sticky note) to ensure the sheet ID, tab names (trendingraw, weeklybrief), and columns match the mappings defined in the Google Sheets n8n node.


4. Credential Linking: Configure your Google Sheets OAuth2 API credentials and link them to both Google Sheets n8n node instances in the workflow.


5. Customization: Adjust the since and languages_csv parameters in the Set Inputs n8n node to control which GitHub trending pages are scraped.


6. Activate: Ensure the n8n workflow is set to active and run it manually or wait for the Cron n8n trigger to execute.

Cron (daily/weekly) (n8n trigger): The starting point, configuring the automated execution frequency of this n8n workflow.
Set Inputs (n8n node): Defines configuration variables like since (set to monthly by default in the JSON) and languagescsv (set to any).
Build URLs (Code n8n node): Dynamically generates the specific GitHub Trending URLs based on the user-defined time period and languages, calculating necessary metadata like weekid.
ScrapeOps: Fetch HTML (ScrapeOps n8n node): A dedicated community n8n node using the ScrapeOps API to fetch target HTML reliably, bypassing common blocking mechanisms in this n8n workflow.
Polite Delay (n8n node): Implements a 2-second delay between scraping requests to maintain good crawling etiquette.
Extract Trending Repos (Code n8n node): Contains custom JavaScript logic to precisely parse repository details (owner, rank, description, raw star counts) from the raw HTML output provided by the ScrapeOps n8n node.
Normalize + Enrich-from-page (Code n8n node): Cleans up raw numerical data, calculates a custom score for prioritization, and creates the essential dedupekey required for unique row identification.
Write to Google Sheets (raw tab) (Google Sheets n8n node): Upserts the detailed repository data into the sheet. Key Configuration: Operation is appendOrUpdate, matching on the dedupekey field.
Read raw (for weekly) (Google Sheets n8n node): Reads the historical data from the trending_raw tab to be processed for the summary report.
Generate Weekly (Code n8n node): Performs filtering, sorting by the custom score, and keyword analysis to generate a concise summary report.