allknowingroger / Qwenslerp4-14B

huggingface.co
Total runs: 20
24-hour runs: 0
7-day runs: 0
30-day runs: 0
Model's Last Updated: November 28 2024
text-generation

Introduction of Qwenslerp4-14B

Model Details of Qwenslerp4-14B

merge

This is a merge of pre-trained language models created using mergekit .

Merge Details
Merge Method

This model was merged using the DARE TIES merge method using Qwen/Qwen2.5-14B as a base.

Models Merged

The following models were included in the merge:

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: CultriX/Qwen2.5-14B-Wernicke
    parameters:
      weight: 0.55         # Backbone model for conversational ability and GPQA
      density: 0.80        # Retain most critical parameters for stability and strength
  - model: VAGOsolutions/SauerkrautLM-v2-14b-DPO
    parameters:
      weight: 0.20         # High IFEval and MMLU-PRO performance with minimized weaknesses
      density: 0.60        # Focus on impactful parameters for specific benchmarks
  - model: rombodawg/Rombos-LLM-V2.6-Qwen-14b
    parameters:
      weight: 0.25         # Enhanced emphasis on reasoning-heavy tasks like MUSR and MATH
      density: 0.70        # Retain reasoning-intensive parameters for improved benchmarks
  - model: allknowingroger/Qwenslerp2-14B
    parameters:
      weight: 0.15         # General stabilizer for consistency across all tasks
      density: 0.65        # Focus on balance and avoiding redundancy
base_model: Qwen/Qwen2.5-14B
merge_method: dare_ties
parameters:
  normalize: true          # Ensure parameter scale consistency
  int8_mask: true          # Optimize for memory and compute efficiency
dtype: bfloat16
tokenizer_source: Qwen/Qwen2.5-14B-Instruct
adaptive_merge_parameters:
  task_weights:
    IFEval: 1.0            # Maintain high IFEval performance
    MATH: 1.3              # Prioritize reasoning and calculation-heavy tasks
    GPQA: 1.1              # Boost factual recall and reasoning accuracy
    MUSR: 1.2              # Enhance logical reasoning and factual understanding
    MMLU-PRO: 1.0          # Retain consistent knowledge representation
  smoothing_factor: 0.15   # Fine-tune blending for stable transitions between tasks
gradient_clipping: 1.0      # Prevent over-contribution from any single model

Runs of allknowingroger Qwenslerp4-14B on huggingface.co

20
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
0
30-day runs

More Information About Qwenslerp4-14B huggingface.co Model

Qwenslerp4-14B huggingface.co

Qwenslerp4-14B huggingface.co is an AI model on huggingface.co that provides Qwenslerp4-14B's model effect (), which can be used instantly with this allknowingroger Qwenslerp4-14B model. huggingface.co supports a free trial of the Qwenslerp4-14B model, and also provides paid use of the Qwenslerp4-14B. Support call Qwenslerp4-14B model through api, including Node.js, Python, http.

allknowingroger Qwenslerp4-14B online free

Qwenslerp4-14B huggingface.co is an online trial and call api platform, which integrates Qwenslerp4-14B's modeling effects, including api services, and provides a free online trial of Qwenslerp4-14B, you can try Qwenslerp4-14B online for free by clicking the link below.

allknowingroger Qwenslerp4-14B online free url in huggingface.co:

https://huggingface.co/allknowingroger/Qwenslerp4-14B

Qwenslerp4-14B install

Qwenslerp4-14B is an open source model from GitHub that offers a free installation service, and any user can find Qwenslerp4-14B on GitHub to install. At the same time, huggingface.co provides the effect of Qwenslerp4-14B install, users can directly use Qwenslerp4-14B installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

Qwenslerp4-14B install url in huggingface.co:

https://huggingface.co/allknowingroger/Qwenslerp4-14B

Url of Qwenslerp4-14B

Provider of Qwenslerp4-14B huggingface.co

allknowingroger
ORGANIZATIONS

Other API from allknowingroger