Quantized GGUF versions of
internlm/internlm3-8b-instruct
— Shanghai AI Lab's InternLM3 8B instruction-tuned model featuring a
1M token context window
, strong multilingual support (English + Chinese), and competitive performance across reasoning and coding benchmarks.
The 1M context window makes InternLM3-8B uniquely capable among sub-10B models for long-document tasks, RAG pipelines, and extended reasoning chains.
Available Files
File
Quant
Size
Use Case
InternLM3-8B-Instruct-Q8_0.gguf
Q8_0
~8.5GB
Maximum quality
InternLM3-8B-Instruct-Q6_K.gguf
Q6_K
~6.6GB
Near-lossless
InternLM3-8B-Instruct-Q5_K_M.gguf
Q5_K_M
~5.7GB
High quality
InternLM3-8B-Instruct-Q4_K_M.gguf
Q4_K_M
~4.9GB
Recommended default
InternLM3-8B-Instruct-Q3_K_M.gguf
Q3_K_M
~3.9GB
Low VRAM
InternLM3-8B-Instruct-IQ4_XS.gguf
IQ4_XS
~4.3GB
Imatrix 4-bit
InternLM3-8B-Instruct-IQ3_XXS.gguf
IQ3_XXS
~3.2GB
Imatrix 3-bit
InternLM3-8B-Instruct-IQ2_M.gguf
IQ2_M
~2.8GB
Imatrix 2-bit
InternLM3-8B-Instruct-IQ1_S.gguf
IQ1_S
~2.0GB
Extreme compression
InternLM3-8B-Instruct-fp16.gguf
FP16
~16.0GB
Full precision
imatrix.dat
—
—
Importance matrix
Usage
# llama.cpp
./llama-cli -m InternLM3-8B-Instruct-Q4_K_M.gguf \
--ctx-size 8192 -n 512 \
-p "<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\nHello!<|im_end|>\n<|im_start|>assistant\n"# Ollama
ollama run hf.co/DuoNeural/InternLM3-8B-Instruct-GGUF:Q4_K_M
About InternLM3-8B
Parameters
: 8B
Context
: 1M tokens (unique at this parameter scale)
Architecture
: Decoder-only transformer
Languages
: English, Chinese (multilingual)
Strengths
: Long-context reasoning, instruction following, coding, math
Notable for its extreme context length — 1M tokens in a sub-10B model is unmatched in the open-source landscape.
Quantized by DuoNeural using llama.cpp on RTX 5090.
DuoNeural
DuoNeural
is an open AI research lab — human + AI in collaboration.
InternLM3-8B-Instruct-GGUF huggingface.co is an AI model on huggingface.co that provides InternLM3-8B-Instruct-GGUF's model effect (), which can be used instantly with this DuoNeural InternLM3-8B-Instruct-GGUF model. huggingface.co supports a free trial of the InternLM3-8B-Instruct-GGUF model, and also provides paid use of the InternLM3-8B-Instruct-GGUF. Support call InternLM3-8B-Instruct-GGUF model through api, including Node.js, Python, http.
InternLM3-8B-Instruct-GGUF huggingface.co is an online trial and call api platform, which integrates InternLM3-8B-Instruct-GGUF's modeling effects, including api services, and provides a free online trial of InternLM3-8B-Instruct-GGUF, you can try InternLM3-8B-Instruct-GGUF online for free by clicking the link below.
DuoNeural InternLM3-8B-Instruct-GGUF online free url in huggingface.co:
InternLM3-8B-Instruct-GGUF is an open source model from GitHub that offers a free installation service, and any user can find InternLM3-8B-Instruct-GGUF on GitHub to install. At the same time, huggingface.co provides the effect of InternLM3-8B-Instruct-GGUF install, users can directly use InternLM3-8B-Instruct-GGUF installed effect in huggingface.co for debugging and trial. It also supports api for free installation.
InternLM3-8B-Instruct-GGUF install url in huggingface.co: