Build a Local AI Chatbot with Python and RAG: Tutorial

Unlock the potential of your PDF documents with a Python RAG (Retrieval Augmented Generation) application! This comprehensive guide walks you through building an AI chatbot capable of answering questions about your PDFs using natural language, all while running locally on your computer. Imagine asking 'How do I build a hotel in Monopoly?' and getting a precise, sourced answer in seconds. Let's dive into how to create this powerful tool.

What is RAG?

Retrieval Augmented Generation (RAG) is a powerful technique that enhances Large Language Models (LLMs) by allowing them to access and incorporate external knowledge.

RAG helps to avoid the need for constant retraining with new data and enables LLMs to generate more accurate and contextually relevant responses.

Think of it like giving your AI assistant a library to consult before answering questions. The process involves:

• Indexing: Preparing a data source for efficient retrieval.
• Retrieval: Fetching the most relevant information from the indexed data based on a user's query.
• Generation: Using the retrieved information to inform the LLM's response.

This approach is particularly useful when dealing with large volumes of proprietary or frequently updated information, like internal documentation, research Papers, or in this case, board Game manuals.

By combining the strengths of both retrieval and generation, RAG allows for more grounded and insightful AI interactions. The key is to understand how this process works and tailor it to your specific needs.

Why Build a Local RAG Application?

Building a local RAG application offers several key advantages:

• Data Privacy: Keep your sensitive data secure by processing it entirely on your own machine.

  No need to share confidential documents with external services.

• Cost Savings: Avoid the recurring costs associated with cloud-based AI services.
• Customization: Fine-tune your RAG application to precisely match your specific data and requirements.
• Offline Functionality: Access your Knowledge Base even without an internet connection.
• Enhanced Control: Full control over every aspect of the system, from data indexing to response generation.

These benefits make local RAG a compelling choice for individuals and organizations seeking greater control, privacy, and cost-effectiveness in their AI solutions.

Loading Data Sources with LangChain

The first step is to load your data source into a format that the RAG application can understand. In this guide, we're focusing on PDF documents, a common format for manuals, reports, and other information-rich content.

Here's how you can do it using LangChain, a powerful framework for building LLM applications:

from langchain.document_loaders.pdf import PyPDFDirectoryLoader

def load_documents():
    document_loader = PyPDFDirectoryLoader(DATA_PATH)
    return document_loader.load()

This code snippet leverages the PyPDFDirectoryLoader class from LangChain to efficiently load all PDF files from a specified directory (DATA_PATH). LangChain offers a variety of document loaders for different file types, including CSV, HTML, Markdown, and more. This allows for flexibility in choosing your data source.

With these document loaders, you can easily load all PDFs, CSVs, webpages, and more to extract data into your rag pipeline.

Splitting Documents into Manageable Chunks

LLMs typically have limitations on the amount of text they can process at once. To overcome this, we need to split the loaded documents into smaller, more manageable chunks.

This ensures that the LLM can effectively analyze the information and generate accurate responses.

LangChain provides the RecursiveCharacterTextSplitter class, which intelligently splits text while preserving semantic relationships:

from langchain.text_splitters import RecursiveCharacterTextSplitter
from langchain.schema.document import Document

def split_documents(documents: list[Document]):
    text_splitter = RecursiveCharacterTextSplitter(
        chunk_size=800,
        chunk_overlap=80,
        length_function=len,
        is_separator_regex=False,
    )
    return text_splitter.split_documents(documents)

This code splits the documents into chunks of 800 characters, with an overlap of 80 characters between chunks to maintain context. Fine-tuning these parameters can optimize performance for your specific data.

Creating Embeddings for Semantic Search

To enable semantic search, we need to convert the text chunks into numerical representations called embeddings. Embeddings capture the meaning of the text, allowing the RAG application to find relevant information based on semantic similarity. Here's how to define an embedding function using LangChain:

from langchain_community.embeddings.bedrock import BedrockEmbeddings

def get_embedding_function():
    embeddings = BedrockEmbeddings(
        credentials_profile_name="default", region_name="us-east-1"
    )
    return embeddings

This code uses AWS Bedrock to create embeddings. However, you can use other embedding models from providers like OpenAI, Hugging Face, or even run a model locally using Ollama. Using OLLAMA will be a fully local experience. The choice of embedding model can significantly impact the accuracy and performance of the RAG application. Remember that using the same embedding model for the database and for querying the database is highly important. If they are different, then the application may not work correctly.

Setting Up the Vector Database for Efficient Retrieval

A vector database is a specialized database designed to store and efficiently search embeddings. For this guide, we'll use Chroma, an open-source vector database that's easy to set up and use.

Here's how to add the processed text chunks to the Chroma database:

from get_embedding_function import get_embedding_function
from langchain.vectorstores.chroma import Chroma

def add_to_chroma(chunks: list[Document]):
    db = Chroma(
        persist_directory=CHROMA_PATH,
        embedding_function=get_embedding_function(),
    )
    db.add_documents(new_chunks, ids=new_chunk_ids)
    db.persist()

This code creates a Chroma database, specifies the storage directory (CHROMA_PATH), and associates it with the embedding function. The text chunks and their corresponding IDs are then added to the database, enabling efficient semantic search.

SEO Keywords: vector database, Chroma, semantic search, embeddings, LangChain, data storage.

How to Test Responses

testing and evaluating the quality of the AI-generated responses is a crucial step in the RAG application development process.

It allows you to assess the accuracy, relevance, and overall usefulness of the chatbot.

Here's a basic outline of the testing process:

1. Define Expected Responses: Create a set of questions and their corresponding expected answers based on your data source.
2. Query the RAG Application: Submit the questions to your RAG application and record the generated responses.
3. Compare Actual vs. Expected Responses: Evaluate the accuracy of the generated responses by comparing them to the expected answers.
4. Measure Sources: Also comparing whether or not the appropriate sources are used is important. The sources should be relevant and correct.

By going through this process you will be able to correctly validate your app when you make a change to the data source, the code, or the LLM model.

AWS Bedrock Pricing

The pricing of AWS Bedrock depends on the models used as well as the amount of traffic that is going through the model. As AWS pricing depends on usage it is hard to determine a fixed number for AWS Bedrock.

Alternative Solutions Pricing

For alternatives, there are the option of using OLLAMA, and other open source embedding tools. These options can be free or paid but will require a more complex setup to ensure stability. Here is a summary of different open source options as well as their pricing:

Local RAG Application features

A great local RAG application is going to require three basic features:

• Running RAG Locally: Use your computer, keep data safe, and save on costs.
• Updating Your Database: Add new data without starting over, keep things fresh and relevant.
• Testing RAG Output: Check responses quickly and spot errors, ensure things are correct and useful.

Board Game Instruction Manuals

Make board game night smoother and faster. Instead of wasting time reading through all the rules, simply ask a question to the RAG. The source will also be listed so you know that the source comes from the right place.

Company Documents

RAG applications can be used to quickly find certain aspects of companies. This keeps company data safe.