README: Add support for llama.cpp
* README.org: The llama.cpp server supports OpenAI's API, so we can reuse it. Closes #121.
This commit is contained in:
parent
f571323174
commit
85bd47cb4c
1 changed files with 34 additions and 9 deletions
29
README.org
29
README.org
|
@ -5,14 +5,14 @@
|
||||||
GPTel is a simple Large Language Model chat client for Emacs, with support for multiple models/backends.
|
GPTel is a simple Large Language Model chat client for Emacs, with support for multiple models/backends.
|
||||||
|
|
||||||
| LLM Backend | Supports | Requires |
|
| LLM Backend | Supports | Requires |
|
||||||
|-------------+----------+-------------------------|
|
|-------------+----------+---------------------------|
|
||||||
| ChatGPT | ✓ | [[https://platform.openai.com/account/api-keys][API key]] |
|
| ChatGPT | ✓ | [[https://platform.openai.com/account/api-keys][API key]] |
|
||||||
| Azure | ✓ | Deployment and API key |
|
| Azure | ✓ | Deployment and API key |
|
||||||
| Ollama | ✓ | [[https://ollama.ai/][Ollama running locally]] |
|
| Ollama | ✓ | [[https://ollama.ai/][Ollama running locally]] |
|
||||||
| GPT4All | ✓ | [[https://gpt4all.io/index.html][GPT4All running locally]] |
|
| GPT4All | ✓ | [[https://gpt4all.io/index.html][GPT4All running locally]] |
|
||||||
| Gemini | ✓ | [[https://makersuite.google.com/app/apikey][API key]] |
|
| Gemini | ✓ | [[https://makersuite.google.com/app/apikey][API key]] |
|
||||||
|
| Llama.cpp | ✓ | [[https://github.com/ggerganov/llama.cpp/tree/master/examples/server#quick-start][Llama.cpp running locally]] |
|
||||||
| PrivateGPT | Planned | - |
|
| PrivateGPT | Planned | - |
|
||||||
| Llama.cpp | Planned | - |
|
|
||||||
|
|
||||||
*General usage*: ([[https://www.youtube.com/watch?v=bsRnh_brggM][YouTube Demo]])
|
*General usage*: ([[https://www.youtube.com/watch?v=bsRnh_brggM][YouTube Demo]])
|
||||||
|
|
||||||
|
@ -46,6 +46,7 @@ GPTel uses Curl if available, but falls back to url-retrieve to work without ext
|
||||||
- [[#gpt4all][GPT4All]]
|
- [[#gpt4all][GPT4All]]
|
||||||
- [[#ollama][Ollama]]
|
- [[#ollama][Ollama]]
|
||||||
- [[#gemini][Gemini]]
|
- [[#gemini][Gemini]]
|
||||||
|
- [[#llamacpp][Llama.cpp]]
|
||||||
- [[#usage][Usage]]
|
- [[#usage][Usage]]
|
||||||
- [[#in-any-buffer][In any buffer:]]
|
- [[#in-any-buffer][In any buffer:]]
|
||||||
- [[#in-a-dedicated-chat-buffer][In a dedicated chat buffer:]]
|
- [[#in-a-dedicated-chat-buffer][In a dedicated chat buffer:]]
|
||||||
|
@ -221,6 +222,30 @@ You can pick this backend from the transient menu when using gptel (see Usage),
|
||||||
|
|
||||||
#+html: </details>
|
#+html: </details>
|
||||||
|
|
||||||
|
#+html: <details>
|
||||||
|
#+html: <summary>
|
||||||
|
**** Llama.cpp
|
||||||
|
#+html: </summary>
|
||||||
|
|
||||||
|
Register a backend with
|
||||||
|
#+begin_src emacs-lisp
|
||||||
|
(gptel-make-openai ;Not a typo, same API as OpenAI
|
||||||
|
"llama-cpp" ;Any name
|
||||||
|
:stream t ;Stream responses
|
||||||
|
:protocol "http"
|
||||||
|
:host "localhost:8000" ;Llama.cpp server location
|
||||||
|
:models '("test")) ;List of available models
|
||||||
|
#+end_src
|
||||||
|
These are the required parameters, refer to the documentation of =gptel-make-openai= for more.
|
||||||
|
|
||||||
|
You can pick this backend from the transient menu when using gptel (see Usage), or set this as the default value of =gptel-backend=:
|
||||||
|
#+begin_src emacs-lisp
|
||||||
|
(setq-default gptel-backend (gptel-make-openai "llama-cpp" ...)
|
||||||
|
gptel-model "test")
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+html: </details>
|
||||||
|
|
||||||
** Usage
|
** Usage
|
||||||
|
|
||||||
(This is also a [[https://www.youtube.com/watch?v=bsRnh_brggM][video demo]] showing various uses of gptel.)
|
(This is also a [[https://www.youtube.com/watch?v=bsRnh_brggM][video demo]] showing various uses of gptel.)
|
||||||
|
|
Loading…
Add table
Reference in a new issue