kevin009 / llama3-merged-adapters

Llama 3.1 Merged Adapters

Model Description

This is a merged model combining multiple fine-tuned LoRA adapters using TIES (Task Inference with Expert Selection) merging technique. The model combines the strengths of multiple specialized adapters into a single model.

Base Model

• unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit

Merged Adapters

The following adapters were merged:

• llama337 - Specialized for creative writing
• llama338 - Specialized for logical reasoning
• llama340 - Specialized for code generation
• llama344 - Specialized for instruction following
• llama345 - Specialized for factual knowledge
• llama346 - Specialized for conversational abilities
• llama349 - Specialized for problem solving
• llama350 - Specialized for structured output

Merging Parameters

• Merging Method : TIES (Task Inference with Expert Selection)
• Density : 0.2 (controls parameter sparsity)
• Weights : Equal weighting (1.0 for each adapter)
• Merge Date : 2025-03-09

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

# Load the model and tokenizer
model = AutoModelForCausalLM.from_pretrained("kevin009/llama3-merged-adapters")
tokenizer = AutoTokenizer.from_pretrained("kevin009/llama3-merged-adapters")

# Example usage
prompt = "Write a short story about a robot learning to paint."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Model Performance

This merged model combines the capabilities of multiple specialized adapters, resulting in improved performance across a variety of tasks compared to individual adapters.

Limitations

• The model inherits limitations from the base Llama 3.1 model
• May produce inconsistent outputs for certain edge cases
• As with all language models, can produce incorrect or misleading information

License

This model is subject to the license of the original Llama 3.1 model.