Easy-to-use SillyTavern Starter, based on Docker Compose.
git clone https://github.com/moeru-ai/easiest.git
cd easiest
cp intel.docker-compose.yml docker-compose.yml # Intel oneAPI SYCL
# cp rocm.docker-compose.yml docker-compose.yml # AMD ROCm (TODO)
# cp cuda.docker-compose.yml docker-compose.yml # NVIDIA CUDA (TODO)
# cp vulkan.docker-compose.yml docker-compose.yml # Vulkan (TODO)
nano docker-compose.yml # edit config
sudo docker compose up -d
# podman compose up -d # if you use podman
sudo docker compose down
# podman compose down # if you use podman
If this section hasn't been updated in a long time, I recommend looking for a new model.
For GGUF format, I recommend
Q5_K_M
orQ4_K_M
(and imatrix).
- 7B:
Lewdiculous/KukulStanta-7B-GGUF-IQ-Imatrix
- SillyTavern Presets:
- Lewdicu-Context-3.0.2-eros.json
=>
./sillytavern/config/context
- Lewdicu-Instruct-Alpaca-3.0.2-tentative.json
=>
./sillytavern/config/instruct
- Lewdicu-Samplers-3.0.2.json
=>
./sillytavern/config/TextGen Settings
- Lewdicu-Context-3.0.2-eros.json
=>
- SillyTavern Presets:
- 11B: mradermacher/Fimbulvetr-11B-v2-i1-GGUF
- 70B:
mradermacher/Midnight-Miqu-70B-v1.5-i1-GGUF
- SillyTavern Presets:
- prompting-tips
=>
./sillytavern/config/context
- instruct-formats
=>
./sillytavern/config/instruct
- sampler-tips
=>
./sillytavern/config/TextGen Settings
- prompting-tips
=>
- SillyTavern Presets:
llama.cpp
provides the official docker image for Intel Arc Graphics.
I may change to ollama
or koboldcpp
later.