ji-farthing / RalphSeek-V4.1-Flash-GGUF

RalphSeek V4.1 Flash (tiny DeepSeek-V4.1 test fixture)

A 334M-parameter model with the DeepSeek-V4.1-Flash architecture. Its weights were randomly initialized, then trained from scratch on a small synthetic corpus. It exists so that DeepSeek-V4.1 support can be built and tested without the full model. It ships with the output distributions of DeepSeek's own reference code on fixed token sequences, so an implementation can be checked against that reference with llama-perplexity and nothing else.

This is not a usable language model. It is ten layers deep and 128 wide, which is enough to carry every V4.1 structure and nothing more. It answers everything with a confident, childlike non-fact, which is what it was trained to do:

I think clouds hide when the sun counts. They taste like the playground flag.

My ear says refrigerators hold hands through metal by the kitchen moon.

For use as a fixture, not as a model. It is the V4.1 sibling of RalphSeek V4 Flash .

Structure

It trained for 12 epochs on 256-token blocks. The weights here are from the epoch with the lowest validation loss (epoch 4).

Files

Usage

llama-cli -m ralphseek-v41-flash-bf16.gguf -c 512 -ngl 99 -fa on --temp 0 -n 48 \
          -p "<｜begin▁of▁sentence｜><｜User｜>Tell me a fact about clouds.<｜Assistant｜></think>"

Set -c . Without it the context is the configured 1,048,576 tokens, and the K cache alone asks for 10 GiB.

Checking an implementation against the reference

The two .kld files hold the output distributions of DeepSeek's reference forward ( inference/model.py from deepseek-ai/DeepSeek-V4.1-Flash ) on this model, in the format llama-perplexity --kl-divergence-base reads. Point any build at them:

llama-perplexity -m ralphseek-v41-flash-bf16.gguf -ngl 99 -fa on -c 256 -b 256 -ub 256 \
                 --kl-divergence-base ralphseek-v41-ref-c256.kld --kl-divergence

and the same with -c 2048 -b 2048 -ub 2048 for the second file. -c must equal the context the file was written at.

The tokens are RalphSeek validation conversations in the V4.1 chat template, concatenated, with BOS at the start of each chunk. The files carry the tokens, so tokenizer differences cannot enter.

Reading the output

Use Mean KLD and Same top p . Do not use Mean PPL(Q)/PPL(base) , which reads about 1.20 even for a close match. The file stores each position over a 24-nat window below the top logit, and a token below that window counts at the window floor. Most user-prompt tokens fall there, because the model never trained on them. So PPL(base) comes out at 1,227.8 (c256) and 1,358.1 (c2048), while the reference's exact perplexity on the same tokens is 1,461.6 and 1,616.7. Compare Mean PPL(Q) with those two numbers instead.

For scale, the reference itself, with its fp8/fp4 storage fake-quant of the KV cache, indexer keys, and compressed latent switched off, scores Mean KLD 0.00083 against both files (same top 100% and 99.7%). This model trained with that fake-quant switched off.

Limits

This model depends very little on distant tokens. It trained on 256-token blocks, and its indexer receives no gradient through the top-k. At 2048 tokens, keeping every position instead of the top 512 moves the reference output by KLD 0.00031 (same top 99.9%). These files therefore cannot detect a missing or wrong token-selection stage. Candidate filtering (2048 blocks of 8) keeps every block below 16,384 positions, so neither file reaches it.

The reference forward replaces DeepSeek's tilelang sparse_attn kernel with an equivalent torch function, because at head dim 512 the kernel needs more shared memory than the GPU used (RTX 4070) has. The torch function matches the kernel to one bf16 ulp at head dims 64 and 128. Everything else in model.py runs as published, including the storage fake-quant.

Measured on ik_llama.cpp

At cdf232c , RTX 4070, -ngl 99 -fa on :

At temp 0 the bf16 and q8_0 files reproduce the reference's own completions of three sample prompts word for word. IQ4_NL gives the same completion for one prompt and a different one for the other two.

Observed on the same commit:

• convert_hf_to_gguf.py writes the source dtypes for deepseek41 whatever --outtype says. The q8_0 and IQ4_NL files come from llama-quantize .
• llama-quantize refuses to requantize MXFP4 tensors. The two quants were made with --custom-q "ffn_(gate|up|down)_exps=mxfp4,engram_embd=q8_0" , which copies the experts and engram tables unchanged.
• The converter writes none of candidate_source_layer_id , candidate_block_size , and candidate_topk_blocks . With V41_SEPARATE=1 the loader uses its defaults, including source layer 20, which this model does not have.

Other runtimes

Upstream llama.cpp master has no deepseek41 architecture; open PR ggml-org/llama.cpp#28696 adds one. These GGUFs were made with ik_llama.cpp's converter, and loading them anywhere else is untested. hf/ holds the checkpoint for converting it with another tool. There is no MTP head.

Provenance and licensing

• Weights are original, trained from random initialization on the synthetic corpus used for RalphSeek V4 Flash. They are not derived from, distilled from, or a slice of any DeepSeek checkpoint. The hidden size alone (128 against 5120) makes that impossible.
• Architecture and reference forward are DeepSeek's inference/model.py , engram.py , and kernel.py from deepseek-ai/DeepSeek-V4.1-Flash , which is MIT licensed.
• Tokenizer and tokenizer config are copied from the same repository, so that prompts and special tokens behave as they do on the real model. The chat template below is added to tokenizer_config.json and also ships as chat_template.jinja .
• The chat template is the tokenizer.chat_template of vcruz305/DeepSeek-V4.1-Flash-GGUF , because DeepSeek's tokenizer_config.json ships none.
• Not affiliated with or endorsed by DeepSeek.

The name is the same joke as RalphSeek V4 Flash's, about a small thing that says wrong facts with total confidence.