Multi-Model LLM Response Evaluation using Contextual AI LMUnit - n8n Workflow

• AI/ML Engineers and prompt designers needing systematic LLM benchmarking.


• Developers building RAG systems who require model quality consistency.


• Users looking for advanced n8n workflow examples integrating multiple AI providers.


• Technical writers creating documentation based on comparative LLM output.

Manually comparing the quality of Large Language Model responses (e.g., from OpenAI, Anthropic, and Google Gemini) for criteria like clarity and conciseness is time-consuming and subjective. This robust n8n workflow solves this problem by automating the entire evaluation lifecycle. The solution uses a specialized n8n node, Contextual AI's LMUnit, which provides standardized, context-aware unit testing for natural language responses.

This sophisticated n8n template captures a single user prompt, sends it to multiple providers, and then leverages the LMUnit n8n node to generate a quantitative score (1–5) against custom criteria. The final aggregated result provides clear, data-driven feedback, allowing users to efficiently benchmark and optimize their chosen LLMs. This is a powerful use case demonstrating the flexibility of n8n in complex AI automation tasks.

This complex n8n workflow initiates immediately upon receiving a chat message input via the dedicated n8n trigger:

1. Trigger and Parallel Execution: The When chat message received n8n trigger captures the user's prompt. This prompt is simultaneously sent to three separate n8n nodes: OpenAI GPT 4.1, Gemini 2.5 Flash, and Claude 4.5 Sonnet.


2. Preprocessing and Merging: Dedicated Set n8n nodes normalize the varied JSON structures of the responses and label them with the provider name. The Combine responses Merge n8n node collects these outputs into a single stream.


3. Unit Test Injection: The Add unit tests to responses Code n8n node introduces the evaluation criteria (e.g., clarity and conciseness checks). It multiplies the data items, creating a unique item for every model response paired with every test case.


4. Batching and Waiting: The Iterate over each unit tests n8n node ensures sequential processing, and Wait for 3 sec prevents rate limiting before accessing the external evaluation API.


5. LMUnit Evaluation: The Run LMUnit n8n node sends the response and the specific unit test criteria to Contextual AI, receiving an objective quality score.


6. Data Aggregation: The scores are associated with the original response details using the Associate scores with Responses n8n node. Then, the Group Results Together Code n8n node aggregates all scores, grouping them under their respective providers.


7. Final Output: The Format Final Result n8n node structures the grouped data into a readable summary message, which is then delivered back to the user via the Final Response n8n node (a LangChain Chat n8n node).

To deploy this powerful n8n workflow, follow these steps:

1. Import: Copy the provided JSON code and paste it into your n8n instance using the 'New' -> 'Import from JSON' function.


2. Contextual AI Credentials: You will need an API key from Contextual AI. Navigate to 'Credentials' in your n8n instance and add a new credential of type 'Contextual AI', providing your API key.


3. LLM Provider Credentials: Ensure you have configured credentials for all three major providers used in this n8n template:

1. Activation: Once all credentials are set up for the relevant n8n node steps, activate the n8n workflow. The automation will be ready to process incoming chat messages and perform the multi-model comparison.

When chat message received (n8n trigger):
Function: Serves as the starting n8n trigger, capturing the user's prompt.
Key Configuration: Response Mode set to responseNodes to handle the final output.
OpenAI GPT 4.1 / Gemini 2.5 Flash / Claude 4.5 Sonnet (LangChain n8n node):
Function: Generates responses in parallel from three different large language models based on the single input prompt captured by the n8n trigger.
Key Configuration: Uses specific, high-performance models (GPT-4.1, Gemini 2.5 Flash, Claude 4.5 Sonnet).
Combine responses (Merge n8n node):
Function: Combines the three parallel response streams (OpenAI, Gemini, Claude) into a single path for subsequent evaluation.
Add unit tests to responses (Code n8n node):
Function: Programmatically injects the evaluation criteria (unit tests) and pairs them with each model's response, preparing the data structure for the LMUnit n8n node.
Run LMUnit (Contextual Ai n8n node):
Function: Core evaluation step. Sends the response and test criteria to the Contextual AI service to obtain an objective quality score (1-5).
Key Configuration: Resource set to LMUnit. Dynamically inputs query, response, and unitTest parameters.
Group Results Together (Code n8n node):
Function: Aggregates the numerous scored items, grouping the results by the provider for structured comparison.
Format Final Result (Code n8n node):
Function: Transforms the aggregated, scored data into a clean, human-readable summary message for the end-user.
Final Response (LangChain Chat n8n node):
* Function: Delivers the final, formatted evaluation report back to the initiating chat interface, completing the n8n workflow cycle.