Large Language Models: Text Classification Explained

Large language models (LLMs) have revolutionized many areas of artificial intelligence, and text classification is no exception. This article explores how LLMs can be used for effective text classification, covering various techniques such as zero-shot and few-shot learning. By understanding these methods, you can leverage LLMs to classify text data accurately and efficiently, and build sophisticated AI solutions.

What is Text Classification?

Text classification involves assigning predefined categories or labels to pieces of text. Traditionally, this required extensive labeled datasets for training machine learning models, a process that was both time-consuming and resource-intensive. However, Large Language Models (LLMs) have made text classification more accessible and efficient. LLMs, particularly those based on the GPT architecture, can perform text classification using a technique known as zero-shot learning. This innovative method allows the model to classify text without needing any labeled examples beforehand. Instead of relying on a dataset of annotated examples, we provide the LLM with a natural language description of the classification task. The model then uses its vast knowledge, acquired from training on diverse and extensive text data, to understand and perform the classification task based on the given description.

The following code gets sample emails that a professor might get from faculty, students and family members. We will write code to classify these emails.

import requests

def get_email(i):
 if i<1 or i>10:
 raise Exception("Invalid email number")
 # URL to download
 url = f"https://data.heatonresearch.com/wustl/CABI/genai-langchain/emails/email_{i}.txt"
 # Perform a GET request to the URL
 response = requests.get(url)
 # Check if the request was successful
 if response.status_code == 200:
 # Convert the content of the response to a string
 content = response.text
 return content
 else:
 raise Exception("Failed to retrieve the content")

For example, the following displays email #1:

print(get_email(1))

Output:

Dear Professor Lawson,

I'm Alex Chen, leader of Team Nova in the Data Science Challenge (Spring 2019).

I am seeking to pursue a PhD in Computer Science, and I am hoping you could provide a recommendation letter for my application. This term has been unexpectedly demanding, leading to delays in preparing my applications. I acknowledge the timing is not ideal. Your guidance in CSC 482 Advanced Machine Learning and your support in my research project have been invaluable. I was particularly engaged in the previous term (Fall 2019), I encountered a challenging issue in my project on "Automated Segmentation of Cardiac MRI". Looking ahead, I aim to delve deeper into Machine Learning and computational models. The knowledge acquired from your course has greatly prepared me.

I have attached my most recent CV and academic transcript for your review. I am applying to approximately 10 universities, and any help you could provide would be greatly appreciated.

Best regards,
Alex Chen

Zero-Shot vs. Few-Shot Learning

When using LLMs for text classification, two primary approaches are available: zero-shot and few-shot learning.

Zero-shot learning requires no prior training examples. You simply provide the LLM with a Prompt that describes the classification task and the categories to choose from. The model then uses its existing knowledge to classify the text. This approach is particularly useful when you don't have access to labeled data or when you need to quickly adapt to new classification tasks. However, the accuracy of zero-shot learning may be lower compared to few-shot learning.

Few-shot learning, on the other hand, involves providing the LLM with a small number of labeled examples. These examples help the model understand the task better and improve its classification accuracy. While few-shot learning requires some labeled data, it can still be much more efficient than traditional supervised learning, which often requires thousands of examples.

We can summarize these approaches in the table below:

Foundation Models

A foundation model for large language models (LLMs) refers to a base model that has been pre-trained on a broad range of data and can be adapted or fine-tuned for specific tasks or applications. These models are called 'foundation' because they provide a foundational layer of knowledge and capabilities upon which specialized functionalities can be built.

Several prominent technology companies and research organizations provide large language models. Notable among them are OpenAI with models like GPT (Generative Pre-trained Transformer), Google with BERT (Bidirectional Encoder Representations from Transformers) and other variants, and Facebook (Meta) which offers models such as RoBERTa (Robustly Optimized BERT Pretraining Approach).

Training a large language model from scratch involves significant computational resources and expertise. It requires extensive data collection, cleaning, and processing, along with access to high-powered computing infrastructure capable of handling immense data processing and complex model training. The cost of training such models can run into millions of dollars, making it prohibitive for most individuals and even many organizations. This situation results in a trend where specialized companies invest in creating foundation models, while others may focus on teaching how to use and fine-tune existing models to solve specific problems or conduct research.

Setting Up the Environment

To classify emails using LLMs, you need to set up the necessary environment and import the required libraries. This typically involves installing the OpenAI Python library and configuring your API key. The provided code snippet demonstrates how to authenticate with OpenAI using an API key:

import os

# OpenAI Secrets
if COLAB:
 os.environ['OPENAI_API_KEY'] = userdata.get('OPENAI_API_KEY')
# Install needed libraries in colab
if COLAB:
 !pip install langchain langchain-openai

This code checks if you're running in a Google Colab environment and retrieves your OpenAI API key from the user data. If not using Colab, you'll need to manually set the OPENAI_API_KEY environment variable. It's crucial to avoid hardcoding your API key directly in your source code to prevent accidental exposure

.

Loading Sample Emails

Before classifying emails, you need to load them into your environment. The get_email function retrieves sample emails from a specified URL. These emails represent the types of messages a professor might receive from faculty, students, and family members. The function checks if the request was successful and returns the content of the email. Here's the code:

def get_email(i):
 if i<1 or i>10:
 raise Exception("Invalid email number")
 # URL to download
 url = f"https://data.heatonresearch.com/wustl/CABI/genai-langchain/emails/email_{i}.txt"
 # Perform a GET request to the URL
 response = requests.get(url)
 # Check if the request was successful
 if response.status_code == 200:
 # Convert the content of the response to a string
 content = response.text
 return content
 else:
 raise Exception("Failed to retrieve the content")

This function allows you to fetch and display sample emails for classification

.

Defining Email Categories

To perform email classification, you need to define the categories into which the emails will be classified. In this example, the following categories are used:

• Spam: Marketing emails trying to sell something.
• Faculty: Faculty announcements and requests.
• Help: Students requesting help on an assignment.
• LOR: Students requesting a letter of recommendation.
• Other: Emails that do not fit into any of these categories.

These categories cover a broad range of emails that an instructor at a university might receive

. The goal is to train the LLM to accurately classify each email into one of these categories. If the email does not fit into one of these, classify it as “Other”.

Creating a Program to Classify Emails

With the environment set up and email categories defined, you can create a program to classify emails. The program iterates through each email, classifies it into one of the defined categories, and extracts the assignment number if the email is a request for help. Here's the code:

from langchain_openai import ChatOpenAI
from langchain.chains import LLMChain, SimpleSequentialChain
from langchain.prompts import PromptTemplate
from langchain.memory import ConversationBufferMemory
MODEL = "gpt-4o-mini"

llm = ChatOpenAI(model=MODEL, temperature=0)

email_prompt = PromptTemplate(
 input_variables=["email"],
 template="""
Classify the following email as either:
* spam - For marketing emails trying to sell something
* faculty - For faculty annoucements and requests
* help - For students requesting help on an assignment
* lor - For students requesting a letter of recommendation
* other - If it does not fit into any of these.

Here is the email.  Return code, such as spam.  Return nothing else, do not explain your choice.
Make sure that if the email does not fit into one of the categories that you classify it as other.
Here is the email:

{email}
"""
)

help_prompt = PromptTemplate(
 input_variables=["email"],
 template="""
You are given an email where a student is asking about an assignment.  Return the assignment number that they are asking about.  If you cannot tell return a ?.  Return only the assignment number as an integer, do not explain.
Here is the email:

{email}
"""
)

chain_email = LLMChain(llm=llm, prompt=email_prompt)
chain_help = LLMChain(llm=llm, prompt=help_prompt)

for i in range(1,11):
 email = get_email(i)
 classification = chain_email.invoke(email)["text"].strip()
 if classification == "help":
 assignment = chain_help.invoke(email)["text"].strip()
 print(f"Email #{i} is a question about assignment {assignment}")
 else:
 print(f"Email #{i} is: {classification}")

This code demonstrates the use of a large language model (LLM) for classifying and extracting information from emails using the Langchain framework. The process starts by importing necessary classes and modules from Langchain, including HumanMessage, SystemMessage, AIMessage, ChatPromptTemplate, HumanMessagePromptTemplate, SystemMessagePromptTemplate, PromptTemplate, LLMChain, and SimpleSequentialChain.

Step-by-Step Guide to Email Classification

Here's a step-by-step guide on how to use LLMs for email classification:

1. Set up the environment: Install the necessary libraries and configure your OpenAI API key.
2. Load sample emails: Use the get_email function to retrieve sample emails from the data source.
3. Define email categories: Determine the categories into which you want to classify the emails (e.g., spam, faculty, help, LOR, other).
4. Create prompts: Design prompts that instruct the LLM to classify the emails based on the defined categories.
5. Implement the classification program: Write code to iterate through each email, classify it using the LLM, and extract relevant information.
6. Test and refine: Test the program with various emails and refine the prompts to improve classification accuracy.

By following these steps, you can effectively use LLMs for email classification and streamline your inbox management

.

Cost Considerations

When using OpenAI models for text classification, it's important to understand the pricing structure. The cost depends on the model you choose and the amount of text you process. GPT-4o Mini is more cost-effective and smarter. While GPT-4 Turbo is mentioned in the older parts of the code, it has now become outdated, GPT-4o is better and less expensive. Make sure to check the OpenAI pricing page for the latest information and optimize your prompts to minimize costs.

Key Capabilities

LLMs offer several key features that make them well-suited for text classification:

• Zero-shot learning: Classify text without needing labeled examples.
• Few-shot learning: Improve performance with a small number of labeled examples.
• Contextual understanding: Understand the meaning and intent behind the text.
• Flexibility: Adapt to various classification tasks with minimal changes.
• Efficiency: Automate the classification process and reduce manual effort.

Applications Across Industries

Text classification has a wide range of applications across various industries:

• Email categorization: Automatically sort emails into relevant categories.
• Sentiment analysis: Determine the emotional tone of customer reviews or social media posts.
• Spam detection: Identify and filter out unwanted messages.
• Content moderation: Ensure online content adheres to community guidelines.
• Document organization: Classify and organize documents based on their topic or content.