CloudGoat / Mephisto-4B-0725

Mephisto-4B-0725

This is a merge of pre-trained language models created using mergekit . I believe that the advanced conversational EQ of Agents-A1-4B stems not from isolated parameters, but rather from a high-density, intricate parameter structure. In contrast, I attribute the high precision of Tmax-4B’s CLI-based tool invocation to specific attention heads and FFN layer weights that trigger strongly in response to particular character strings (such as terminal commands). Since the critical parameters underpinning Tmax-4B’s core strengths are sparse, I believe the model is resilient against performance degradation even when merged using DARE with high dropout rates. This model is currently being improved.

Merge Details

Merge Method

This model was merged using the DARE TIES merge method using Qwen/Qwen3.5-4B as a base.

Models Merged

The following models were included in the merge:

• allenai/tmax-4b
• InternScience/Agents-A1-4B

Configuration

The following YAML configuration was used to produce this model:

merge_method: dare_ties
base_model: Qwen/Qwen3.5-4B

models:
    - model: InternScience/Agents-A1-4B
    parameters:
      weight: 1.0
      density: 1.0

  - model: allenai/tmax-4b
    parameters:
      weight: 0.7
      density: 0.3

parameters:
  int8_mask: true
  normalize: false      

dtype: bfloat16

Benchmarks

in progress