This is a GGUF-formatted checkpoint of
rnj-1-instruct
suitable
for use in llama.cpp. This has been quantized with the Q4_K_M scheme, which
results in model weights of size 4.8GB.
Support for this model has been PR'd upstream, but in the meantime you can
build from source with these instructions for MacOS. For Linux, install cmake
using your package manager. For Windows, consult the llama.cpp
build
guide
.
brew install cmake
git clone https://github.com/Essential-AI/llama.cpp
cd llama.cpp
git checkout rnj-1
cmake -B build
cmake --build build --config Release
To run this model in a web UI, use this command and open your browser to
http://localhost:8080
:
build/bin/llama-server -hf EssentialAI/rnj-1-instruct-GGUF -c 0
To run it in the CLI, use this command:
build/bin/llama-cli -hf EssentialAI/rnj-1-instruct-GGUF