Side-by-Side LLM Evaluation and Logging to Google Sheets - n8n Workflow

AI Developers and Engineers needing to benchmark LLM performance.
Teams requiring a structured way to manually evaluate AI responses in a shared document (Google Sheets).
Users looking for advanced n8n templates utilizing LangChain and LLM comparison techniques.
Automation specialists wanting to implement a complex, looping n8n node structure.

Determining the best Large Language Model (LLM) for a specific production task is challenging due to their non-deterministic nature. This specialized n8n workflow solves this by allowing developers to send a single prompt to two different models concurrently. It captures the user input, the complete conversational context (using isolated memory), and both model responses, logging everything instantly to Google Sheets. This setup provides an objective, side-by-side view for quick quality assessment and ensures that your final AI Agent performs optimally. This powerful n8n workflow configuration is essential for rigorous testing.

The process initiates when a user sends a message, activating the When chat message received n8n trigger.

1. Model Definition & Splitting: The Define Models to Compare n8n node sets an array of two LLM IDs (e.g., using OpenRouter). The Split Models into Items n8n node then breaks this array into individual items, one for each model.


2. Context Isolation: The Set model, sessionId, chatInput, sessionIdBase n8n node dynamically creates a unique session ID for each model iteration, ensuring that Model A and Model B maintain separate, isolated conversational memory contexts via the Simple Memory n8n node.


3. Parallel Execution: The workflow loops, sending the user's prompt through the AI Agent, which utilizes the specified OpenRouter Chat Model for generation.


4. Data Preparation: The Prepare Data for Chat and Google Sheets n8n node formats the model's response for display in the chat interface and extracts crucial data points (answer, context history) for logging.


5. Output Aggregation: Once both models have responded, the Loop Over Items node exits the batch. The Concatenate Chat Answers n8n node combines the two distinct LLM outputs (formatted with separators) and sends this combined response back to the chat UI.


6. Logging & Evaluation: Simultaneously, the Group Model Outputs for Evaluation n8n node gathers the full comparison data (input, Model 1 answer, Model 2 answer, context for both) into one structure. This is immediately appended to the designated Google Sheet using the Add Model Results to Google Sheet n8n node, finalizing the data collection process in this robust n8n workflow.

To deploy this comprehensive n8n workflow, follow these steps:

1. Import the n8n Workflow: Copy the provided JSON data and import it directly into your n8n instance via the 'Import from JSON' option in the workflow editor.


2. Google Sheets Setup: Create a copy of the companion Google Sheets template (linked in the Sticky Note within the workflow) or create a sheet with necessary headers (e.g., sessionId, userinput, model1answer, model2_answer, etc.).


3. Credential Configuration:

1. Node Configuration: Update the Add Model Results to Google Sheet n8n node parameters, ensuring the correct Document ID and Sheet Name (e.g., gid=0) are selected based on your setup.


2. Model Customization: In the Define Models to Compare n8n node, update the array of model strings (e.g., ["model/a", "model/b"]) to reflect the specific LLMs you wish to compare using your configured provider.

When chat message received (n8n trigger): The starting point. It captures the user's input from the integrated chat interface.
Define Models to Compare (Set n8n node): Holds the critical array of model IDs (e.g., openai/gpt-4.1 and mistralai/mistral-large) that will be tested in parallel. A key configuration point in this n8n template.
Split Models into Items (Split Out n8n node): Converts the array of models into separate execution paths, enabling parallel processing.
Set model, sessionId, chatInput, sessionIdBase (Set n8n node): Crucial for memory management. It creates a unique, model-specific sessionId (combining the base session ID and the model name) to isolate the context for each LLM.
OpenRouter Chat Model / AI Agent (n8n nodes): These nodes handle the communication with the LLM API using the dynamically set model ID, processing the input, and generating the response using the isolated context provided by the Simple Memory n8n node.
Add Model Results to Google Sheet (Google Sheets n8n node): Performs the final action, using the append operation to log all collected comparison data (user input, model answers, context) into the designated spreadsheet row. This n8n node is configured to handle potential errors (onError: continueRegularOutput).

• Concatenate Chat Answers (Summarize n8n node): Aggregates the separate text outputs from Model A and Model B into a single, combined response message, formatted for clear side-by-side display in the chat UI.