Blockify Data Optimization for RAG Knowledge Bases - n8n Workflow

• AI/ML Engineers: Seeking to improve the accuracy and context retrieval of their RAG pipelines.


• Data Scientists: Who need a scalable method for transforming large, unstructured datasets into high-quality, structured knowledge.


• n8n Automation Specialists: Looking for advanced n8n templates combining custom code, external APIs (Blockify), and LangChain integration.


• Enterprise IT Teams: Implementing internal knowledge base chatbots that require minimal hallucination and high precision.

This automation solves the pervasive problem of low accuracy in RAG systems caused by traditional 'dump-and-chunk' data ingestion. By utilizing the Blockify Ingest technology, this n8n workflow transforms raw, unstructured text into concise, context-rich XML 'IdeaBlocks.'

IdeaBlocks are designed to be optimally suited for vector storage and retrieval, dramatically increasing the precision of answers returned by the subsequent AI Agent. This specific n8n workflow template demonstrates the complete pipeline, from file extraction via Google Drive and custom JavaScript chunking, through API optimization, and finally into an in-memory vector database, which is then accessed by a sample RAG Chatbot.

The execution of this comprehensive n8n workflow begins with a manual n8n trigger node.

1. Data Ingestion: The flow first downloads a text file from Google Drive using the dedicated Google Drive n8n node, followed by the 'Extract Text from .TXT File' node to prepare the raw content.


2. Custom Chunking: The raw text is passed to the 'Chunk Text' Code n8n node. This node uses custom JavaScript logic to split the document into manageable segments (1,000 characters with 100 character overlap), a standard preparation step for large language models and APIs.


3. Optimization Loop: The 'Loop Over Chunks' (Split In Batches) node iterates over these text chunks. For each chunk, the 'Blockify Ingest API' HTTP Request node is executed. This node sends the chunk to the Blockify service to be transformed into structured IdeaBlocks.


4. IdeaBlock Storage: The 'Extract IdeaBlocks from API Response' Set node captures the XML output. These structured IdeaBlocks are then inserted into the 'Simple IdeaBlock Vector Store' (an in-memory vector database).


5. RAG Demonstration: A separate branch of the n8n workflow demonstrates the utility of the optimized data. The 'RAG Chatbot' n8n trigger initializes an AI Agent that uses the newly populated vector store (via the 'Query Data Tool' n8n node) and an OpenAI chat model to answer user queries with high accuracy, leveraging the structured IdeaBlocks.

To deploy this powerful n8n workflow template, follow these steps:

1. Import the n8n Workflow: Copy the provided JSON and import it into your n8n instance via the 'Workflows' section.


2. Blockify Credentials: You will need a Blockify API key. Obtain one from the official Blockify console. Edit the 'Blockify Ingest API' HTTP Request node and update the 'Blockify API Production' HTTP Bearer Auth credential with your key.


3. OpenAI Credentials: The RAG demonstration requires an OpenAI account. Update the 'Embeddings OpenAI' and 'OpenAI Chat Model' nodes with your active OpenAI API key credential.


4. Google Drive Setup (Optional): If you wish to use a different source file, update the 'Download .TXT File for Ingest' Google Drive n8n node with the file ID and credentials for your chosen document.


5. Execution: After configuring credentials, execute the main data ingestion branch starting from the manual 'n8n trigger'. Once complete, the RAG Chatbot is ready to accept queries.

When clicking ‘Execute workflow’ (Manual Trigger): The starting point of the ingestion n8n workflow. Allows manual initiation for data loading.
Download .TXT File for Ingest (Google Drive): Downloads the source document. Key configuration is the specific fileId used for retrieving the source text.
Extract Text from .TXT File (Extract From File): Transforms the binary file output into a usable string of text for further processing.
Chunk Text (Code n8n node): Critical custom logic. Splits text into chunks of MAX_LEN = 1000 characters with an OVERLAP = 100. This prepares the data optimally for the Blockify API call.
Loop Over Chunks (Split In Batches): Manages the iterative process, ensuring each generated chunk is sent individually to the optimization service.
Blockify Ingest API (HTTP Request): Sends the chunk data using a POST request to Blockify. The JSON body is dynamically built using {{ JSON.stringify($json.chunk) }} and specifies the model: "ingest".
Simple IdeaBlock Vector Store (Vector Store In Memory): The database n8n node where the optimized IdeaBlocks are stored for retrieval. Configured in insert mode, linked to OpenAI Embeddings and a custom Text Splitter that splits the XML output by tags.
RAG Chatbot (Chat Trigger): Provides a public web interface n8n trigger to interact with the RAG system, demonstrating the final use case of the optimized data.