| .. | ||
| src | ||
| Cargo.toml | ||
| config.yaml | ||
| README.md | ||
| TODO.org | ||
LLama Herder
- manages multiple llama.cpp instances in the background
- keeps track of used & available video & cpu memory
- starts/stops llama.cpp instances as needed, to ensure memory limit is never reached