Academic Research Aggregation, Deduplication, and Multi-Format Export - n8n Workflow

Academic Researchers: Individuals needing to rapidly survey literature across major databases without manual merging.
Data Scientists: Users requiring clean, ranked research data for text analysis or model training.
n8n Developers: Those looking for sophisticated n8n templates demonstrating custom code logic for data cleaning and ranking within an n8n workflow.
Librarians and Reviewers: Professionals who need to generate accurate BibTeX citations efficiently.

Conducting comprehensive literature reviews often involves searching multiple specialized databases, leading to redundant results and complex manual deduplication. This powerful n8n workflow solves this by centralizing the search, processing, and output phases.

Leveraging the PDF Vector n8n node, this n8n workflow simultaneously queries PubMed, ArXiv, Google Scholar, Semantic Scholar, and ERIC. After aggregation, the n8n workflow uses custom JavaScript logic (via the Code n8n node) to remove duplicates based on DOI and title similarity, and then applies a relevance ranking algorithm based on citation counts, recency, and keyword match. This automation provides a highly efficient and exportable dataset, drastically cutting down research preparation time. This advanced n8n template is a must-have for serious researchers.

This automation is initiated when the n8n trigger starts the workflow execution, usually manually or via a schedule.

1. Set Search Parameters: The n8n workflow begins by defining three key parameters: searchQuery, yearFrom, and resultsPerSource. This allows the user to easily customize the search topic and scope.


2. Multi-Database Search: The PDF Vector n8n node executes the core search function, querying the five specified academic providers (PubMed, ArXiv, etc.) using the defined parameters.


3. Deduplicate Results: A crucial custom Code n8n node runs. It processes the raw results, prioritizing deduplication based on the doi. If a DOI is missing, it compares titles for similarity to ensure no duplicates are carried forward.


4. Rank by Relevance: The workflow then passes the cleaned list to another Code n8n node. This node calculates a relevanceScore for each paper, weighting factors like title keyword match, total citations, and publication recency.


5. Generate BibTeX: The results are passed to a final Code n8n node, which structures the data into a usable array of items and generates the full text BibTeX citation string.


6. Export Outputs: Finally, the n8n workflow branches, utilizing three Write Binary File n8n node instances to export the processed data simultaneously into three formats: BibTeX, JSON (structured data), and tab-separated CSV.

To use this n8n template effectively, follow these steps:

1. Import: Copy the provided JSON code and import it directly into your n8n instance.


2. Credentials: This n8n workflow utilizes the specialized PDF Vector n8n node. Ensure you have the necessary credentials configured for the PDF Vector service to enable successful execution.


3. Initial Configuration: Locate the 'Set Search Parameters' n8n node and update the searchQuery and yearFrom values to match your current research needs.


4. Run: Execute the n8n workflow using the manual n8n trigger to test the data flow and generate your initial files. The output files will be saved in your local n8n file system or volume, depending on your self-hosting configuration.

Set Search Parameters (Set n8n node): Serves as the configuration starting point for the n8n workflow. It defines the search term (e.g., 'machine learning healthcare applications') and constraints (e.g., results per source).
PDF Vector - Multi-DB Search (pdfVector n8n node): The key API integrator. It executes an academic resource search across five distinct providers (PubMed, ArXiv, etc.), aggregating initial results.
Deduplicate Results (Code n8n node): Contains custom JavaScript logic to ensure data quality. It intelligently identifies and removes duplicate papers, first by comparing DOIs, and then by fuzzy title matching.
Rank by Relevance (Code n8n node): Custom code that assigns a weighted score to each paper. This crucial step transforms a list of results into a prioritized, actionable list based on academic metrics.
Generate BibTeX (Code n8n node): Prepares the final export formats. It generates the required BibTeX formatted string while keeping the structured JSON data available for parallel exports.
Export BibTeX File / Export JSON / Export CSV (Write Binary File n8n node): The final actions in the n8n workflow. These nodes take the processed data and save it to the file system in the appropriate research formats, automatically naming files with the current date.