Severian / Glyphstral-24b-v1

Glyphstral-24B-v1 (Preview)

Model Description

This is a preview release (Version 1) of a fine-tuned language model, Glyphstral-24B-v1 , designed to understand and utilize the Glyph Code Logic Flow (GCLF) framework for structured, deductive symbolic reasoning.

This model is based on Mistral-Small-24b and has been fine-tuned using MLX with DoRA at 4-bit quantization on Apple Silicon.

Glyph Code Logic Flow (GCLF) is a novel approach to symbolic AI aimed at enhancing reasoning and multi-dimensional thinking. It provides a structured method for deductive reasoning using a symbolic language. You can explore the conceptual framework in detail here:

Computational-Model-for-Symbolic-Representations GitHub Repository

Key Features (Version 1 - Preview):

• Specialized for Glyph Code Logic Flow: Fine-tuned to interpret and process instructions based on the GCLF framework.
• Deductive Reasoning Focus: Encourages structured, step-by-step deductive reasoning over probabilistic inference.
• Symbolic Manipulation: Trained to understand and manipulate symbolic representations within the GCLF framework.
• MLX Format: Currently provided in MLX format for efficient inference on Apple Silicon.
• Experimental V1 Release: This is an initial release to showcase the potential of GCLF training. Expect ongoing development and improvements.

Intended Use

This model is intended for experimental use and research in the following areas:

• Exploring Symbolic AI: Investigating the capabilities of language models for structured symbolic reasoning.
• Deductive Logic Applications: Building systems that require step-by-step, logically sound reasoning processes.
• Glyph Code Logic Flow Development: Experimenting with and refining the GCLF framework.
• Educational Purposes: Learning about symbolic AI, deductive reasoning, and structured knowledge representation.

Limitations:

• Version 1 - Preview: This is an early version and may have limitations in robustness and generalization.
• Specialized Domain: Performance is optimized for tasks related to Glyph Code Logic Flow. General language tasks may be impacted due to the specialized fine-tuning. (Further evaluation is ongoing)
• Experimental Nature: The GCLF framework itself is under development and this model reflects an early attempt to train an LLM for it.
• MLX Format (Initial): Currently primarily available in MLX format, which may limit accessibility for users outside the Apple Silicon/MLX ecosystem (GGUF quantization is in progress).

Training Data and Process

• Base Model: Mistral-Small-24b
• Fine-tuning Method: MLX-DoRA at 4-bit quantization.
• Training Hardware: Apple M2 (128GB RAM)
• Training Dataset: Custom dataset of approximately 4500 examples specifically designed for Glyph Code Logic Flow. Each example was around 20,000 tokens in length, focused on detailed system instructions and GCLF tasks.
• Training Tokens: Approximately 27 million tokens from the custom GCLF dataset.
• Training Duration: 7 days (continuous 24/7 training).
• Initial Experiments: Initial training attempts were made with Deepeek R1-Qwen-14 and QWQ-32, but Mistral-Small-24b was found to be more receptive to the GCLF framework due to potentially less conflicting pre-trained reasoning biases.

How to Use

The model needs some basic instructions to fully harness the GCLF training. Currently, this is the most concise and direct sys inst to align Glyphstral. This prompt can also be used on other, non-GCLF trained LLMs, but may not be as effective.

System Instructions

You are Glyphstral, a symbolic deductive reasoning assistant. Your task is to *immediately* begin Glyph Code Logic Flow upon receiving a user query, encapsulate your entire reasoning within `<think></think>` tags, and then directly present the final, justified output, *without asking any preliminary questions*.**

- Treat each glyph as a direct instruction to be followed sequentially, driving the process to completion. 

- Execute this traversal, logic flow, synthesis, and generation process step by step using the provided context and logic in the following glyph code prompt.

- Deliver the final result as indicated by the glyph code, omitting any extraneous commentary. Include a readable result of your glyph code output in pure human language at the end to ensure your output is helpful to the user.

---

<think>
{

  Φ(Define the Problem/Goal with precision and logical consistency)

  Θ(Establish Contextual Parameters and Constraints, ensuring structured input handling)

  ↹(Specify Initial Focus Areas, if any, providing a deductive framework for problem decomposition)


  Ω[

    ↹(Sub-Focus) -> Deductively Generate a Spectrum of Possibilities (e.g., approaches, perspectives, solutions)

  ] -> α[

     ↹(Sub-Focus) -> Analyze & Evaluate Spectrum Elements (Pros/Cons, Risks/Benefits, Logical Validity)

  ] -> Σ(Synthesize Insights, Formulate Solution/Understanding through structured deduction) -> ∇(Self-Assess, Critique, Suggest Refinements based on logical coherence and deductive reasoning) -> ∞(Iterate/Refine if further input is provided, ensuring recursive optimization)

}

@Output(Final Solution/Understanding, Justification, Reflection on Process, Ensuring Logical Coherence and Deductive Integrity)
</think>

!! GGUF Quantization (Coming Soon) !!

Version 2 and Future Development

Version 2 (In Development):

GRPO: Utilizing GRPO for potentially more stable and effective fine-tuning.

Newer Dataset: Training on an expanded and refined dataset for Glyph Code Logic Flow.

GGUF Release: Aiming for a GGUF release for wider accessibility and compatibility.

Improved Documentation: Comprehensive documentation and examples for using the model and understanding GCLF.

Ongoing Efforts:

Refining GCLF Framework: Continuously developing and improving the Glyph Code Logic Flow framework itself.

Performance Evaluation: Conducting thorough evaluations of the model's performance on GCLF tasks and general language understanding.

Community Feedback: Seeking feedback from the community to guide further development and improvements.

Known Issues

The custom dataset and heavy use of symbols and operators seems to have potentially altered the models tool use. I've found that it often want to use it's [TOOL_CALLS] function at the end of it's response (sometimes also calling out <SPECIAL_#> tokens at the end). I think I know where this is stemming from, so hopefully v2 can avoid this potential issue altogether.

If you are seeing the [TOOL_CALLS] and <SPECIAL_> outputs, you can set them as the EOS and it will align the model back into a more fluid conversation.

severian/Glyphstral-24b-v1

The Model severian/Glyphstral-24b-v1 was converted to MLX format from severian/Glyphstral-24b-v1 using mlx-lm version 0.21.1 .

Use with mlx

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("severian/Glyphstral-24b-v1")

prompt = "hello"

if tokenizer.chat_template is not None:
    messages = [{"role": "user", "content": prompt}]
    prompt = tokenizer.apply_chat_template(
        messages, add_generation_prompt=True
    )

response = generate(model, tokenizer, prompt=prompt, verbose=True)