Intelligent Private LLM Router using Self-Hosted Ollama - n8n Workflow

• AI enthusiasts and developers utilizing local LLMs.


• Privacy-conscious users needing self-hosted AI solutions.


• Users running Ollama who require dynamic routing between specialized models (vision, code, reasoning).


• Technical users looking for advanced n8n templates for AI orchestration.

When operating a self-hosted environment with multiple large language models (LLMs) via Ollama, selecting the most effective model for a given query is crucial for performance and resource utilization. This specialized n8n workflow solves the 'model selection' problem automatically. It acts as an expert LLM router, analyzing the user's intent—be it complex reasoning, coding assistance, or visual data extraction—and automatically selecting the correct local Ollama model. This complex n8n workflow ensures maximum efficiency by routing the task to a specialized model, all while maintaining complete privacy because the entire process runs locally. This sophisticated use of an n8n node structure provides enterprise-level AI orchestration capabilities in a private, self-hosted environment.

This automation begins with the 'When chat message received' n8n trigger, which initiates the flow whenever a new chat input is detected, passing the user’s prompt to the LLM Router. The core of this n8n workflow is the 'LLM Router' node, which functions as a decision-making AI agent. This agent utilizes the 'Ollama phi4' model and the 'Router Chat Memory' to analyze the input against a detailed decision tree (which classifies inputs into complex reasoning, coding, vision, or general chat tasks). The router’s output is a JSON object containing the name of the optimal LLM model (e.g., 'granite3.2-vision' for chart analysis).

This crucial decision is then passed to the final execution step. The 'AI Agent with Dynamic LLM' node is responsible for generating the final response. Its connected language model, 'Ollama Dynamic LLM', uses a dynamic n8n expression to read the chosen model name from the router's JSON output: ={{ $('LLM Router').item.json.output.parseJson().llm }}. This dynamically configured n8n node ensures that the correct, specialized local Ollama LLM is called to answer the user's request, maintaining conversation context via the 'Agent Chat Memory' node and completing the private, intelligent n8n workflow.

To deploy this powerful n8n workflow template, follow these steps:

1. Import: Copy the provided JSON code and import it into your n8n instance.


2. Ollama Setup: Ensure you have Ollama installed and running on your local machine (typically accessible at http://127.0.0.1:11434).


3. Model Availability: Pull all models referenced in the router's system prompt using the Ollama CLI (e.g., ollama pull qwq, ollama pull qwen2.5-coder:14b, etc.).


4. Credential Configuration: Configure the 'Ollama API' credential within n8n, pointing to your local Ollama endpoint. Both 'Ollama phi4' and 'Ollama Dynamic LLM' nodes rely on this credential.


5. Activate: Activate the 'When chat message received' n8n trigger webhook to start listening for incoming chat requests. This specialized n8n workflow is now ready for testing.

This advanced n8n workflow uses several specialized nodes to achieve dynamic routing:

When chat message received (n8n Trigger):
Function: Serves as the starting point, initiating the n8n workflow upon receiving a user chat message (input).
Key Configuration: A LangChain Chat Trigger webhook setup to receive chatInput and sessionId data.

LLM Router (n8n Node):
Function: The central decision-maker. It analyzes the user prompt and selects the optimal specialized Ollama model.
Key Configuration: Uses a highly detailed System Message defining classification rules, decision trees, and required JSON output format (llm and reason). It uses 'Ollama phi4' as its computational engine.

Ollama phi4 (n8n Node):
Function: Acts as the primary LLM for the Router Agent, performing the prompt classification and model selection task.
Key Configuration: Uses the phi4:latest model via the Ollama API connection.

Router Chat Memory & Agent Chat Memory (n8n Node):
Function: Stores the history for the conversation to maintain context, ensuring the LLM Router and the final AI Agent provide coherent responses across multiple turns.

Ollama Dynamic LLM (n8n Node):
Function: The model used by the final execution agent. Its model choice is dynamically determined by the preceding routing step.
Key Configuration: The model name is set dynamically using an expression: ={{ $('LLM Router').item.json.output.parseJson().llm }}. This powerful feature is core to this n8n workflow's flexibility.

AI Agent with Dynamic LLM (n8n Node):
Function: The final agent that executes the user's request using the dynamically selected local Ollama LLM and generates the response.
* Key Configuration: Uses the user's original input as the main text and relies on the dynamically connected 'Ollama Dynamic LLM' for processing.