vault81/redvault-ai
4
0
Fork 0
Code Issues 1 Pull requests 1 Projects 1 Releases Packages Wiki Activity

Make nemotron run with desktop env

Browse source
This commit is contained in:
Tristan D. 2024-11-27 10:18:54 +01:00
parent d33e4109c3
commit 932faab35b
Signed by: tristan
SSH key fingerprint: SHA256:3RU4RLOoM8oAjFU19f1W6t8uouZbA7GWkaSW6rjp1k8
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
llama_proxy_man/config.yaml
View file

@ -60,7 +60,7 @@ models:
CUDA_VISIBLE_DEVICES: 0
HSA_OVERRIDE_GFX_VERSION: '11.0.0'
args:
ctx-size: 16384
ctx-size: 8192
flash-attn: true
cache-type-k: q8_0
cache-type-v: q8_0