vault81/redvault-ai
4
0
Fork 0
Code Issues 1 Pull requests 1 Projects 1 Releases Packages Wiki Activity
No description
12 commits 2 branches 0 tags 5.3 MiB
Rust 68.9%
Emacs Lisp 17.6%
Nix 5.5%
TypeScript 2.8%
SCSS 2.1%
Other 3.1%
Find a file
tristan 734a6300a1 Update llama_proxy_man/README.md
2024-10-08 15:37:58 +00:00
.nix WIP 2024-09-05 22:33:13 +02:00
docstub WIP 2024-09-05 22:33:13 +02:00
frozen_llama Remove iced dir & add as git dep 2024-09-09 11:46:22 +02:00
leptos_stub Remove unused pkg 2024-09-09 15:49:41 +02:00
llama_forge_rs Remove unused pkg 2024-09-09 15:49:41 +02:00
llama_proxy_man Update llama_proxy_man/README.md 2024-10-08 15:37:58 +00:00
.envrc Init 2024-07-21 02:42:48 +02:00
.gitignore WIP 2024-09-05 22:33:13 +02:00
Cargo.lock Optimize by stopping large models first 2024-10-08 17:29:11 +02:00
Cargo.toml Add llama_proxy_man pkg 2024-09-19 17:21:46 +02:00
clippy.yml Remove unused pkg 2024-09-09 15:49:41 +02:00
flake.lock Optimize by stopping large models first 2024-10-08 17:29:11 +02:00
flake.nix Optimize by stopping large models first 2024-10-08 17:29:11 +02:00
package-lock.json Init 2024-07-21 02:42:48 +02:00
package.json Init 2024-07-21 02:42:48 +02:00
README.md Update README.md 2024-10-08 15:36:08 +00:00
rust-toolchain.toml WIP 2024-09-05 22:33:13 +02:00

README.md

Redvau.lt AI Monorepo

Current Repos

llama-forge-rs:

  • old alpha-state webview based GUI app for chatting with ai
  • basic streaming
  • manages a llama-server in the background
  • currently unmaintained

llama-proxy-man:

  • proxy which auto starts/stops llama.cpp instances for you
  • retries requests in the background and keeps the connection open
  • enables you to run more models than your VRAM can fit, with HTTP-API requests working like they do (they'll just be slow if you have to start instances to make them happen)

Ideas

  • emacs-qwen-plugin
    • use emacs-module-rs + llama.cpp/proxy to get awesome qwen integration into emacs
  • agent
    • add experimental rag+agent framework