gptel: Add multi-llm support
README.org: Update README with new information and a multi-llm demo.
gptel.el (gptel-host, gptel--known-backends, gptel--api-key,
gptel--create-prompt, gptel--request-data, gptel--parse-buffer, gptel-request,
gptel--parse-response, gptel--openai, gptel--debug, gptel--restore-state,
gptel, gptel-backend):
Integrate multiple LLMs through the introcution of gptel-backends. Each backend
is composed of two pieces:
1. An instance of a cl-struct, containing connection, authentication and model
information. See the cl-struct `gptel-backend` for details. A separate
cl-struct type is defined for each supported backend (OpenAI, Azure, GPT4All and
Ollama) that inherits from the generic gptel-backend type.
2. cl-generic implementations of specific tasks, like gathering up and
formatting context (previous user queries and LLM responses), parsing responses
or responses streams etc. The four tasks currently specialized this way are
carried out by `gptel--parse-buffer` and `gptel--request-data` (for constructing
the query) and `gptel--parse-response` and `gptel-curl--parse-stream` (for
parsing the response). See their implementations for details. Some effort has
been made to limit the number of times dispatching is done when reading
streaming responses.
When a backend is created, it is registered in the collection
`gptel--known-backends` and can be accessed by name later, such as from the
transient menu.
Only one of these backends is active at any time in a buffer, stored in the
buffer-local variable `gptel-backend`. Most messaging, authentication etc
accounts for the active backend, although there might be some leftovers.
When using `gptel-request` or `gptel-send`, the active backend can be changed or
let-bound.
- Obsolete `gptel-host`
- Fix the rear-sticky property when restoring sessions from files.
- Document some variables (not user options), like `gptel--debug`
gptel-openai.el (gptel-backend, gptel-make-openai, gptel-make-azure,
gptel-make-gpt4all): This file (currently always loaded) sets up the generic
backend struct and includes constructors for creating OpenAI, GPT4All and Azure
backends. They all use the same API so a single set of defgeneric
implemenations suffices for all of them.
gptel-ollama.el (gptel-make-ollama): This file includes the cl-struct,
constructor and requisite defgeneric implementations for Ollama support.
gptel-transient.el (gptel-menu, gptel-provider-variable, gptel--infix-provider,
gptel-suffix-send):
- Provide access to all available LLM backends and models from `gptel-menu`.
- Adjust keybindings in gptel-menu: setting the model and query parameters is
now bound to two char keybinds, while redirecting input and output is bound to
single keys.
2023-10-23 23:40:52 -07:00
#+title : GPTel: A simple LLM client for Emacs
2023-03-05 16:59:44 -08:00
2023-03-19 19:58:19 -07:00
[[https://melpa.org/#/gptel ][file:https://melpa.org/packages/gptel-badge.svg ]]
2023-12-21 21:23:28 -08:00
GPTel is a simple Large Language Model chat client for Emacs, with support for multiple models and backends.
2024-01-15 16:09:41 -08:00
| LLM Backend | Supports | Requires |
|-----------------+----------+---------------------------|
| ChatGPT | ✓ | [[https://platform.openai.com/account/api-keys][API key]] |
| Azure | ✓ | Deployment and API key |
| Ollama | ✓ | [[https://ollama.ai/][Ollama running locally]] |
| GPT4All | ✓ | [[https://gpt4all.io/index.html][GPT4All running locally]] |
| Gemini | ✓ | [[https://makersuite.google.com/app/apikey][API key]] |
| Llama.cpp | ✓ | [[https://github.com/ggerganov/llama.cpp/tree/master/examples/server#quick-start][Llama.cpp running locally]] |
| Llamafile | ✓ | [[https://github.com/Mozilla-Ocho/llamafile#quickstart][Local Llamafile server]] |
| Kagi FastGPT | ✓ | [[https://kagi.com/settings?p=api][API key]] |
| Kagi Summarizer | ✓ | [[https://kagi.com/settings?p=api][API key]] |
2024-01-19 14:43:07 -08:00
| together.ai | ✓ | [[https://api.together.xyz/settings/api-keys][API key]] |
| Anyscale | ✓ | [[https://docs.endpoints.anyscale.com/][API key]] |
2024-02-03 19:36:24 -05:00
| Perplexity | ✓ | [[https://docs.perplexity.ai/docs/getting-started][API key]] |
gptel: Add multi-llm support
README.org: Update README with new information and a multi-llm demo.
gptel.el (gptel-host, gptel--known-backends, gptel--api-key,
gptel--create-prompt, gptel--request-data, gptel--parse-buffer, gptel-request,
gptel--parse-response, gptel--openai, gptel--debug, gptel--restore-state,
gptel, gptel-backend):
Integrate multiple LLMs through the introcution of gptel-backends. Each backend
is composed of two pieces:
1. An instance of a cl-struct, containing connection, authentication and model
information. See the cl-struct `gptel-backend` for details. A separate
cl-struct type is defined for each supported backend (OpenAI, Azure, GPT4All and
Ollama) that inherits from the generic gptel-backend type.
2. cl-generic implementations of specific tasks, like gathering up and
formatting context (previous user queries and LLM responses), parsing responses
or responses streams etc. The four tasks currently specialized this way are
carried out by `gptel--parse-buffer` and `gptel--request-data` (for constructing
the query) and `gptel--parse-response` and `gptel-curl--parse-stream` (for
parsing the response). See their implementations for details. Some effort has
been made to limit the number of times dispatching is done when reading
streaming responses.
When a backend is created, it is registered in the collection
`gptel--known-backends` and can be accessed by name later, such as from the
transient menu.
Only one of these backends is active at any time in a buffer, stored in the
buffer-local variable `gptel-backend`. Most messaging, authentication etc
accounts for the active backend, although there might be some leftovers.
When using `gptel-request` or `gptel-send`, the active backend can be changed or
let-bound.
- Obsolete `gptel-host`
- Fix the rear-sticky property when restoring sessions from files.
- Document some variables (not user options), like `gptel--debug`
gptel-openai.el (gptel-backend, gptel-make-openai, gptel-make-azure,
gptel-make-gpt4all): This file (currently always loaded) sets up the generic
backend struct and includes constructors for creating OpenAI, GPT4All and Azure
backends. They all use the same API so a single set of defgeneric
implemenations suffices for all of them.
gptel-ollama.el (gptel-make-ollama): This file includes the cl-struct,
constructor and requisite defgeneric implementations for Ollama support.
gptel-transient.el (gptel-menu, gptel-provider-variable, gptel--infix-provider,
gptel-suffix-send):
- Provide access to all available LLM backends and models from `gptel-menu`.
- Adjust keybindings in gptel-menu: setting the model and query parameters is
now bound to two char keybinds, while redirecting input and output is bound to
single keys.
2023-10-23 23:40:52 -07:00
2023-12-25 18:20:29 -08:00
*General usage* : ([[https://www.youtube.com/watch?v=bsRnh_brggM ][YouTube Demo ]])
2023-03-05 16:59:44 -08:00
2023-04-06 17:29:40 -07:00
https://user-images.githubusercontent.com/8607532/230516812-86510a09-a2fb-4cbd-b53f-cc2522d05a13.mp4
https://user-images.githubusercontent.com/8607532/230516816-ae4a613a-4d01-4073-ad3f-b66fa73c6e45.mp4
2023-03-05 16:59:44 -08:00
gptel: Add multi-llm support
README.org: Update README with new information and a multi-llm demo.
gptel.el (gptel-host, gptel--known-backends, gptel--api-key,
gptel--create-prompt, gptel--request-data, gptel--parse-buffer, gptel-request,
gptel--parse-response, gptel--openai, gptel--debug, gptel--restore-state,
gptel, gptel-backend):
Integrate multiple LLMs through the introcution of gptel-backends. Each backend
is composed of two pieces:
1. An instance of a cl-struct, containing connection, authentication and model
information. See the cl-struct `gptel-backend` for details. A separate
cl-struct type is defined for each supported backend (OpenAI, Azure, GPT4All and
Ollama) that inherits from the generic gptel-backend type.
2. cl-generic implementations of specific tasks, like gathering up and
formatting context (previous user queries and LLM responses), parsing responses
or responses streams etc. The four tasks currently specialized this way are
carried out by `gptel--parse-buffer` and `gptel--request-data` (for constructing
the query) and `gptel--parse-response` and `gptel-curl--parse-stream` (for
parsing the response). See their implementations for details. Some effort has
been made to limit the number of times dispatching is done when reading
streaming responses.
When a backend is created, it is registered in the collection
`gptel--known-backends` and can be accessed by name later, such as from the
transient menu.
Only one of these backends is active at any time in a buffer, stored in the
buffer-local variable `gptel-backend`. Most messaging, authentication etc
accounts for the active backend, although there might be some leftovers.
When using `gptel-request` or `gptel-send`, the active backend can be changed or
let-bound.
- Obsolete `gptel-host`
- Fix the rear-sticky property when restoring sessions from files.
- Document some variables (not user options), like `gptel--debug`
gptel-openai.el (gptel-backend, gptel-make-openai, gptel-make-azure,
gptel-make-gpt4all): This file (currently always loaded) sets up the generic
backend struct and includes constructors for creating OpenAI, GPT4All and Azure
backends. They all use the same API so a single set of defgeneric
implemenations suffices for all of them.
gptel-ollama.el (gptel-make-ollama): This file includes the cl-struct,
constructor and requisite defgeneric implementations for Ollama support.
gptel-transient.el (gptel-menu, gptel-provider-variable, gptel--infix-provider,
gptel-suffix-send):
- Provide access to all available LLM backends and models from `gptel-menu`.
- Adjust keybindings in gptel-menu: setting the model and query parameters is
now bound to two char keybinds, while redirecting input and output is bound to
single keys.
2023-10-23 23:40:52 -07:00
*Multi-LLM support demo* :
https://github-production-user-asset-6210df.s3.amazonaws.com/8607532/278854024-ae1336c4-5b87-41f2-83e9-e415349d6a43.mp4
2023-05-31 18:35:08 -07:00
- It's async and fast, streams responses.
gptel: Add multi-llm support
README.org: Update README with new information and a multi-llm demo.
gptel.el (gptel-host, gptel--known-backends, gptel--api-key,
gptel--create-prompt, gptel--request-data, gptel--parse-buffer, gptel-request,
gptel--parse-response, gptel--openai, gptel--debug, gptel--restore-state,
gptel, gptel-backend):
Integrate multiple LLMs through the introcution of gptel-backends. Each backend
is composed of two pieces:
1. An instance of a cl-struct, containing connection, authentication and model
information. See the cl-struct `gptel-backend` for details. A separate
cl-struct type is defined for each supported backend (OpenAI, Azure, GPT4All and
Ollama) that inherits from the generic gptel-backend type.
2. cl-generic implementations of specific tasks, like gathering up and
formatting context (previous user queries and LLM responses), parsing responses
or responses streams etc. The four tasks currently specialized this way are
carried out by `gptel--parse-buffer` and `gptel--request-data` (for constructing
the query) and `gptel--parse-response` and `gptel-curl--parse-stream` (for
parsing the response). See their implementations for details. Some effort has
been made to limit the number of times dispatching is done when reading
streaming responses.
When a backend is created, it is registered in the collection
`gptel--known-backends` and can be accessed by name later, such as from the
transient menu.
Only one of these backends is active at any time in a buffer, stored in the
buffer-local variable `gptel-backend`. Most messaging, authentication etc
accounts for the active backend, although there might be some leftovers.
When using `gptel-request` or `gptel-send`, the active backend can be changed or
let-bound.
- Obsolete `gptel-host`
- Fix the rear-sticky property when restoring sessions from files.
- Document some variables (not user options), like `gptel--debug`
gptel-openai.el (gptel-backend, gptel-make-openai, gptel-make-azure,
gptel-make-gpt4all): This file (currently always loaded) sets up the generic
backend struct and includes constructors for creating OpenAI, GPT4All and Azure
backends. They all use the same API so a single set of defgeneric
implemenations suffices for all of them.
gptel-ollama.el (gptel-make-ollama): This file includes the cl-struct,
constructor and requisite defgeneric implementations for Ollama support.
gptel-transient.el (gptel-menu, gptel-provider-variable, gptel--infix-provider,
gptel-suffix-send):
- Provide access to all available LLM backends and models from `gptel-menu`.
- Adjust keybindings in gptel-menu: setting the model and query parameters is
now bound to two char keybinds, while redirecting input and output is bound to
single keys.
2023-10-23 23:40:52 -07:00
- Interact with LLMs from anywhere in Emacs (any buffer, shell, minibuffer, wherever)
- LLM responses are in Markdown or Org markup.
2023-05-31 18:35:08 -07:00
- Supports conversations and multiple independent sessions.
gptel: saving and restoring state for Markdown/Text
* gptel.el (gptel--save-state, gptel--restore-state,
gptel-temperature, gptel-model, gptel-max-tokens,
gptel-directives, gptel--always, gptel--button-buttonize,
gptel--system-message, gptel--bounds): Write gptel parameters as
file-local variables when saving chats in Markdown or text files.
The local variable gptel--bounds stores the locations of the
responses from the LLM. This is not a great solution, but the best
I can think to do without adding more syntax to the document.
Chats can be restored by turning on `gptel-mode'. One of the
problem with this approach is that if the buffer is modified
before `gptel-mode' is turned on, the state data is out of date.
Another problem is that this metadata block as printed in the
buffer can become quite long. A better approach is needed.
Define helper functions `gptel--always' and
`gptel--button-buttonize' to work around Emacs 27.1 support.
* README.org: Mention saving and restoring chats where
appropriate.
2023-07-28 15:57:08 -07:00
- Save chats as regular Markdown/Org/Text files and resume them later.
gptel: Add multi-llm support
README.org: Update README with new information and a multi-llm demo.
gptel.el (gptel-host, gptel--known-backends, gptel--api-key,
gptel--create-prompt, gptel--request-data, gptel--parse-buffer, gptel-request,
gptel--parse-response, gptel--openai, gptel--debug, gptel--restore-state,
gptel, gptel-backend):
Integrate multiple LLMs through the introcution of gptel-backends. Each backend
is composed of two pieces:
1. An instance of a cl-struct, containing connection, authentication and model
information. See the cl-struct `gptel-backend` for details. A separate
cl-struct type is defined for each supported backend (OpenAI, Azure, GPT4All and
Ollama) that inherits from the generic gptel-backend type.
2. cl-generic implementations of specific tasks, like gathering up and
formatting context (previous user queries and LLM responses), parsing responses
or responses streams etc. The four tasks currently specialized this way are
carried out by `gptel--parse-buffer` and `gptel--request-data` (for constructing
the query) and `gptel--parse-response` and `gptel-curl--parse-stream` (for
parsing the response). See their implementations for details. Some effort has
been made to limit the number of times dispatching is done when reading
streaming responses.
When a backend is created, it is registered in the collection
`gptel--known-backends` and can be accessed by name later, such as from the
transient menu.
Only one of these backends is active at any time in a buffer, stored in the
buffer-local variable `gptel-backend`. Most messaging, authentication etc
accounts for the active backend, although there might be some leftovers.
When using `gptel-request` or `gptel-send`, the active backend can be changed or
let-bound.
- Obsolete `gptel-host`
- Fix the rear-sticky property when restoring sessions from files.
- Document some variables (not user options), like `gptel--debug`
gptel-openai.el (gptel-backend, gptel-make-openai, gptel-make-azure,
gptel-make-gpt4all): This file (currently always loaded) sets up the generic
backend struct and includes constructors for creating OpenAI, GPT4All and Azure
backends. They all use the same API so a single set of defgeneric
implemenations suffices for all of them.
gptel-ollama.el (gptel-make-ollama): This file includes the cl-struct,
constructor and requisite defgeneric implementations for Ollama support.
gptel-transient.el (gptel-menu, gptel-provider-variable, gptel--infix-provider,
gptel-suffix-send):
- Provide access to all available LLM backends and models from `gptel-menu`.
- Adjust keybindings in gptel-menu: setting the model and query parameters is
now bound to two char keybinds, while redirecting input and output is bound to
single keys.
2023-10-23 23:40:52 -07:00
- You can go back and edit your previous prompts or LLM responses when continuing a conversation. These will be fed back to the model.
2024-01-15 17:07:58 -08:00
- Don't like gptel's workflow? Use it to create your own for any supported model/backend with a [[https:/ /github.com/karthink/gptel/wiki#defining-custom-gptel-commands][simple API]].
2023-03-05 16:59:44 -08:00
2023-06-01 11:24:13 +08:00
GPTel uses Curl if available, but falls back to url-retrieve to work without external dependencies.
2023-06-12 17:27:52 -07:00
** Contents :toc:
- [[#installation ][Installation ]]
- [[#straight ][Straight ]]
- [[#manual ][Manual ]]
- [[#doom-emacs ][Doom Emacs ]]
- [[#spacemacs ][Spacemacs ]]
gptel: Add multi-llm support
README.org: Update README with new information and a multi-llm demo.
gptel.el (gptel-host, gptel--known-backends, gptel--api-key,
gptel--create-prompt, gptel--request-data, gptel--parse-buffer, gptel-request,
gptel--parse-response, gptel--openai, gptel--debug, gptel--restore-state,
gptel, gptel-backend):
Integrate multiple LLMs through the introcution of gptel-backends. Each backend
is composed of two pieces:
1. An instance of a cl-struct, containing connection, authentication and model
information. See the cl-struct `gptel-backend` for details. A separate
cl-struct type is defined for each supported backend (OpenAI, Azure, GPT4All and
Ollama) that inherits from the generic gptel-backend type.
2. cl-generic implementations of specific tasks, like gathering up and
formatting context (previous user queries and LLM responses), parsing responses
or responses streams etc. The four tasks currently specialized this way are
carried out by `gptel--parse-buffer` and `gptel--request-data` (for constructing
the query) and `gptel--parse-response` and `gptel-curl--parse-stream` (for
parsing the response). See their implementations for details. Some effort has
been made to limit the number of times dispatching is done when reading
streaming responses.
When a backend is created, it is registered in the collection
`gptel--known-backends` and can be accessed by name later, such as from the
transient menu.
Only one of these backends is active at any time in a buffer, stored in the
buffer-local variable `gptel-backend`. Most messaging, authentication etc
accounts for the active backend, although there might be some leftovers.
When using `gptel-request` or `gptel-send`, the active backend can be changed or
let-bound.
- Obsolete `gptel-host`
- Fix the rear-sticky property when restoring sessions from files.
- Document some variables (not user options), like `gptel--debug`
gptel-openai.el (gptel-backend, gptel-make-openai, gptel-make-azure,
gptel-make-gpt4all): This file (currently always loaded) sets up the generic
backend struct and includes constructors for creating OpenAI, GPT4All and Azure
backends. They all use the same API so a single set of defgeneric
implemenations suffices for all of them.
gptel-ollama.el (gptel-make-ollama): This file includes the cl-struct,
constructor and requisite defgeneric implementations for Ollama support.
gptel-transient.el (gptel-menu, gptel-provider-variable, gptel--infix-provider,
gptel-suffix-send):
- Provide access to all available LLM backends and models from `gptel-menu`.
- Adjust keybindings in gptel-menu: setting the model and query parameters is
now bound to two char keybinds, while redirecting input and output is bound to
single keys.
2023-10-23 23:40:52 -07:00
- [[#setup ][Setup ]]
- [[#chatgpt ][ChatGPT ]]
- [[#other-llm-backends ][Other LLM backends ]]
- [[#azure ][Azure ]]
- [[#gpt4all ][GPT4All ]]
- [[#ollama ][Ollama ]]
2023-12-20 11:12:40 +00:00
- [[#gemini ][Gemini ]]
2023-12-31 14:37:26 -08:00
- [[#llamacpp-or-llamafile ][Llama.cpp or Llamafile ]]
2024-01-15 16:09:41 -08:00
- [[#kagi-fastgpt--summarizer ][Kagi (FastGPT & Summarizer) ]]
2024-01-19 14:43:07 -08:00
- [[#togetherai ][together.ai ]]
- [[#anyscale ][Anyscale ]]
2024-02-03 19:36:24 -05:00
- [[#perplexity ][Perplexity ]]
2023-06-12 17:27:52 -07:00
- [[#usage ][Usage ]]
- [[#in-any-buffer ][In any buffer: ]]
- [[#in-a-dedicated-chat-buffer ][In a dedicated chat buffer: ]]
gptel: Add multi-llm support
README.org: Update README with new information and a multi-llm demo.
gptel.el (gptel-host, gptel--known-backends, gptel--api-key,
gptel--create-prompt, gptel--request-data, gptel--parse-buffer, gptel-request,
gptel--parse-response, gptel--openai, gptel--debug, gptel--restore-state,
gptel, gptel-backend):
Integrate multiple LLMs through the introcution of gptel-backends. Each backend
is composed of two pieces:
1. An instance of a cl-struct, containing connection, authentication and model
information. See the cl-struct `gptel-backend` for details. A separate
cl-struct type is defined for each supported backend (OpenAI, Azure, GPT4All and
Ollama) that inherits from the generic gptel-backend type.
2. cl-generic implementations of specific tasks, like gathering up and
formatting context (previous user queries and LLM responses), parsing responses
or responses streams etc. The four tasks currently specialized this way are
carried out by `gptel--parse-buffer` and `gptel--request-data` (for constructing
the query) and `gptel--parse-response` and `gptel-curl--parse-stream` (for
parsing the response). See their implementations for details. Some effort has
been made to limit the number of times dispatching is done when reading
streaming responses.
When a backend is created, it is registered in the collection
`gptel--known-backends` and can be accessed by name later, such as from the
transient menu.
Only one of these backends is active at any time in a buffer, stored in the
buffer-local variable `gptel-backend`. Most messaging, authentication etc
accounts for the active backend, although there might be some leftovers.
When using `gptel-request` or `gptel-send`, the active backend can be changed or
let-bound.
- Obsolete `gptel-host`
- Fix the rear-sticky property when restoring sessions from files.
- Document some variables (not user options), like `gptel--debug`
gptel-openai.el (gptel-backend, gptel-make-openai, gptel-make-azure,
gptel-make-gpt4all): This file (currently always loaded) sets up the generic
backend struct and includes constructors for creating OpenAI, GPT4All and Azure
backends. They all use the same API so a single set of defgeneric
implemenations suffices for all of them.
gptel-ollama.el (gptel-make-ollama): This file includes the cl-struct,
constructor and requisite defgeneric implementations for Ollama support.
gptel-transient.el (gptel-menu, gptel-provider-variable, gptel--infix-provider,
gptel-suffix-send):
- Provide access to all available LLM backends and models from `gptel-menu`.
- Adjust keybindings in gptel-menu: setting the model and query parameters is
now bound to two char keybinds, while redirecting input and output is bound to
single keys.
2023-10-23 23:40:52 -07:00
- [[#save-and-restore-your-chat-sessions ][Save and restore your chat sessions ]]
2023-12-21 14:56:26 -08:00
- [[#faq ][FAQ ]]
2024-01-15 17:07:58 -08:00
- [[#i-want-the-window-to-scroll-automatically-as-the-response-is-inserted ][I want the window to scroll automatically as the response is inserted ]]
- [[#i-want-the-cursor-to-move-to-the-next-prompt-after-the-response-is-inserted ][I want the cursor to move to the next prompt after the response is inserted ]]
2024-01-20 13:19:52 -08:00
- [[#i-want-to-change-the-formatting-of-the-prompt-and-llm-response ][I want to change the formatting of the prompt and LLM response ]]
2024-01-21 18:40:55 -08:00
- [[#i-want-the-transient-menu-options-to-be-saved-so-i-only-need-to-set-them-once ][I want the transient menu options to be saved so I only need to set them once ]]
2024-01-20 13:19:52 -08:00
- [[#i-want-to-use-gptel-in-a-way-thats-not-supported-by-gptel-send-or-the-options-menu ][I want to use gptel in a way that's not supported by =gptel-send= or the options menu ]]
2024-01-15 17:07:58 -08:00
- [[#doom-emacs-sending-a-query-from-the-gptel-menu-fails-because-of-a-key-conflict-with-org-mode ][(Doom Emacs) Sending a query from the gptel menu fails because of a key conflict with Org mode ]]
2024-02-03 09:40:52 -08:00
- [[#chatgpt-i-get-the-error-http2-429-you-exceeded-your-current-quota ][(ChatGPT) I get the error "(HTTP/2 429) You exceeded your current quota" ]]
2024-01-15 17:07:58 -08:00
- [[#why-another-llm-client ][Why another LLM client? ]]
2023-06-12 17:27:52 -07:00
- [[#additional-configuration ][Additional Configuration ]]
- [[#alternatives ][Alternatives ]]
2024-01-20 13:19:52 -08:00
- [[#extensions-using-gptel ][Extensions using GPTel ]]
2023-12-21 14:56:26 -08:00
- [[#breaking-changes ][Breaking Changes ]]
2023-06-12 17:27:52 -07:00
- [[#acknowledgments ][Acknowledgments ]]
2023-03-05 16:59:44 -08:00
** Installation
gptel: Add multi-llm support
README.org: Update README with new information and a multi-llm demo.
gptel.el (gptel-host, gptel--known-backends, gptel--api-key,
gptel--create-prompt, gptel--request-data, gptel--parse-buffer, gptel-request,
gptel--parse-response, gptel--openai, gptel--debug, gptel--restore-state,
gptel, gptel-backend):
Integrate multiple LLMs through the introcution of gptel-backends. Each backend
is composed of two pieces:
1. An instance of a cl-struct, containing connection, authentication and model
information. See the cl-struct `gptel-backend` for details. A separate
cl-struct type is defined for each supported backend (OpenAI, Azure, GPT4All and
Ollama) that inherits from the generic gptel-backend type.
2. cl-generic implementations of specific tasks, like gathering up and
formatting context (previous user queries and LLM responses), parsing responses
or responses streams etc. The four tasks currently specialized this way are
carried out by `gptel--parse-buffer` and `gptel--request-data` (for constructing
the query) and `gptel--parse-response` and `gptel-curl--parse-stream` (for
parsing the response). See their implementations for details. Some effort has
been made to limit the number of times dispatching is done when reading
streaming responses.
When a backend is created, it is registered in the collection
`gptel--known-backends` and can be accessed by name later, such as from the
transient menu.
Only one of these backends is active at any time in a buffer, stored in the
buffer-local variable `gptel-backend`. Most messaging, authentication etc
accounts for the active backend, although there might be some leftovers.
When using `gptel-request` or `gptel-send`, the active backend can be changed or
let-bound.
- Obsolete `gptel-host`
- Fix the rear-sticky property when restoring sessions from files.
- Document some variables (not user options), like `gptel--debug`
gptel-openai.el (gptel-backend, gptel-make-openai, gptel-make-azure,
gptel-make-gpt4all): This file (currently always loaded) sets up the generic
backend struct and includes constructors for creating OpenAI, GPT4All and Azure
backends. They all use the same API so a single set of defgeneric
implemenations suffices for all of them.
gptel-ollama.el (gptel-make-ollama): This file includes the cl-struct,
constructor and requisite defgeneric implementations for Ollama support.
gptel-transient.el (gptel-menu, gptel-provider-variable, gptel--infix-provider,
gptel-suffix-send):
- Provide access to all available LLM backends and models from `gptel-menu`.
- Adjust keybindings in gptel-menu: setting the model and query parameters is
now bound to two char keybinds, while redirecting input and output is bound to
single keys.
2023-10-23 23:40:52 -07:00
GPTel is on MELPA. Ensure that MELPA is in your list of sources, then install gptel with =M-x package-install⏎= =gptel= .
2023-03-05 16:59:44 -08:00
2023-03-19 17:50:51 -07:00
(Optional: Install =markdown-mode= .)
2023-03-05 16:59:44 -08:00
2023-08-12 11:17:30 -07:00
#+html : <details><summary>
2023-03-28 12:23:04 -07:00
**** Straight
2023-08-12 11:17:30 -07:00
#+html : </summary>
2023-03-05 16:59:44 -08:00
#+begin_src emacs-lisp
2023-03-19 17:50:51 -07:00
(straight-use-package 'gptel)
2023-03-05 16:59:44 -08:00
#+end_src
Installing the =markdown-mode= package is optional.
2023-08-12 11:17:30 -07:00
#+html : </details>
#+html : <details><summary>
2023-03-28 12:23:04 -07:00
**** Manual
2023-08-12 11:17:30 -07:00
#+html : </summary>
2023-03-28 12:23:04 -07:00
Clone or download this repository and run =M-x package-install-file⏎= on the repository directory.
2023-03-05 16:59:44 -08:00
Installing the =markdown-mode= package is optional.
2023-08-12 11:17:30 -07:00
#+html : </details>
#+html : <details><summary>
2023-04-01 23:25:47 +02:00
**** Doom Emacs
2023-08-12 11:17:30 -07:00
#+html : </summary>
2023-04-01 23:25:47 +02:00
In =packages.el=
#+begin_src emacs-lisp
(package! gptel)
#+end_src
In =config.el=
#+begin_src emacs-lisp
(use-package! gptel
:config
(setq! gptel-api-key "your key"))
#+end_src
2024-01-20 13:19:52 -08:00
"your key" can be the API key itself, or (safer) a function that returns the key. Setting =gptel-api-key= is optional, you will be asked for a key if it's not found.
2023-08-12 11:17:30 -07:00
#+html : </details>
#+html : <details><summary>
2023-03-28 12:23:04 -07:00
**** Spacemacs
2023-08-12 11:17:30 -07:00
#+html : </summary>
2023-03-25 11:26:00 +01:00
After installation with =M-x package-install⏎= =gptel=
2023-03-28 12:23:04 -07:00
- Add =gptel= to =dotspacemacs-additional-packages=
- Add =(require 'gptel)= to =dotspacemacs/user-config=
2023-08-12 11:17:30 -07:00
#+html : </details>
gptel: Add multi-llm support
README.org: Update README with new information and a multi-llm demo.
gptel.el (gptel-host, gptel--known-backends, gptel--api-key,
gptel--create-prompt, gptel--request-data, gptel--parse-buffer, gptel-request,
gptel--parse-response, gptel--openai, gptel--debug, gptel--restore-state,
gptel, gptel-backend):
Integrate multiple LLMs through the introcution of gptel-backends. Each backend
is composed of two pieces:
1. An instance of a cl-struct, containing connection, authentication and model
information. See the cl-struct `gptel-backend` for details. A separate
cl-struct type is defined for each supported backend (OpenAI, Azure, GPT4All and
Ollama) that inherits from the generic gptel-backend type.
2. cl-generic implementations of specific tasks, like gathering up and
formatting context (previous user queries and LLM responses), parsing responses
or responses streams etc. The four tasks currently specialized this way are
carried out by `gptel--parse-buffer` and `gptel--request-data` (for constructing
the query) and `gptel--parse-response` and `gptel-curl--parse-stream` (for
parsing the response). See their implementations for details. Some effort has
been made to limit the number of times dispatching is done when reading
streaming responses.
When a backend is created, it is registered in the collection
`gptel--known-backends` and can be accessed by name later, such as from the
transient menu.
Only one of these backends is active at any time in a buffer, stored in the
buffer-local variable `gptel-backend`. Most messaging, authentication etc
accounts for the active backend, although there might be some leftovers.
When using `gptel-request` or `gptel-send`, the active backend can be changed or
let-bound.
- Obsolete `gptel-host`
- Fix the rear-sticky property when restoring sessions from files.
- Document some variables (not user options), like `gptel--debug`
gptel-openai.el (gptel-backend, gptel-make-openai, gptel-make-azure,
gptel-make-gpt4all): This file (currently always loaded) sets up the generic
backend struct and includes constructors for creating OpenAI, GPT4All and Azure
backends. They all use the same API so a single set of defgeneric
implemenations suffices for all of them.
gptel-ollama.el (gptel-make-ollama): This file includes the cl-struct,
constructor and requisite defgeneric implementations for Ollama support.
gptel-transient.el (gptel-menu, gptel-provider-variable, gptel--infix-provider,
gptel-suffix-send):
- Provide access to all available LLM backends and models from `gptel-menu`.
- Adjust keybindings in gptel-menu: setting the model and query parameters is
now bound to two char keybinds, while redirecting input and output is bound to
single keys.
2023-10-23 23:40:52 -07:00
** Setup
*** ChatGPT
2023-03-05 16:59:44 -08:00
Procure an [[https://platform.openai.com/account/api-keys ][OpenAI API key ]].
2023-04-23 22:55:34 -04:00
Optional: Set =gptel-api-key= to the key. Alternatively, you may choose a more secure method such as:
2023-06-01 11:24:13 +08:00
- Storing in =~/.authinfo= . By default, "api.openai.com" is used as HOST and "apikey" as USER.
2023-04-23 22:55:34 -04:00
#+begin_src authinfo
2023-06-01 11:24:13 +08:00
machine api.openai.com login apikey password TOKEN
2023-04-23 22:55:34 -04:00
#+end_src
- Setting it to a function that returns the key.
2023-03-05 16:59:44 -08:00
gptel: Add multi-llm support
README.org: Update README with new information and a multi-llm demo.
gptel.el (gptel-host, gptel--known-backends, gptel--api-key,
gptel--create-prompt, gptel--request-data, gptel--parse-buffer, gptel-request,
gptel--parse-response, gptel--openai, gptel--debug, gptel--restore-state,
gptel, gptel-backend):
Integrate multiple LLMs through the introcution of gptel-backends. Each backend
is composed of two pieces:
1. An instance of a cl-struct, containing connection, authentication and model
information. See the cl-struct `gptel-backend` for details. A separate
cl-struct type is defined for each supported backend (OpenAI, Azure, GPT4All and
Ollama) that inherits from the generic gptel-backend type.
2. cl-generic implementations of specific tasks, like gathering up and
formatting context (previous user queries and LLM responses), parsing responses
or responses streams etc. The four tasks currently specialized this way are
carried out by `gptel--parse-buffer` and `gptel--request-data` (for constructing
the query) and `gptel--parse-response` and `gptel-curl--parse-stream` (for
parsing the response). See their implementations for details. Some effort has
been made to limit the number of times dispatching is done when reading
streaming responses.
When a backend is created, it is registered in the collection
`gptel--known-backends` and can be accessed by name later, such as from the
transient menu.
Only one of these backends is active at any time in a buffer, stored in the
buffer-local variable `gptel-backend`. Most messaging, authentication etc
accounts for the active backend, although there might be some leftovers.
When using `gptel-request` or `gptel-send`, the active backend can be changed or
let-bound.
- Obsolete `gptel-host`
- Fix the rear-sticky property when restoring sessions from files.
- Document some variables (not user options), like `gptel--debug`
gptel-openai.el (gptel-backend, gptel-make-openai, gptel-make-azure,
gptel-make-gpt4all): This file (currently always loaded) sets up the generic
backend struct and includes constructors for creating OpenAI, GPT4All and Azure
backends. They all use the same API so a single set of defgeneric
implemenations suffices for all of them.
gptel-ollama.el (gptel-make-ollama): This file includes the cl-struct,
constructor and requisite defgeneric implementations for Ollama support.
gptel-transient.el (gptel-menu, gptel-provider-variable, gptel--infix-provider,
gptel-suffix-send):
- Provide access to all available LLM backends and models from `gptel-menu`.
- Adjust keybindings in gptel-menu: setting the model and query parameters is
now bound to two char keybinds, while redirecting input and output is bound to
single keys.
2023-10-23 23:40:52 -07:00
*** Other LLM backends
#+html : <details><summary>
**** Azure
#+html : </summary>
Register a backend with
#+begin_src emacs-lisp
2024-01-19 14:43:07 -08:00
(gptel-make-azure "Azure-1" ;Name, whatever you'd like
:protocol "https" ;Optional -- https is the default
:host "YOUR_RESOURCE_NAME.openai.azure.com"
:endpoint "/openai/deployments/YOUR_DEPLOYMENT_NAME/chat/completions?api-version=2023-05-15" ;or equivalent
:stream t ;Enable streaming responses
:key #'gptel-api-key
:models '("gpt-3.5-turbo" "gpt-4"))
gptel: Add multi-llm support
README.org: Update README with new information and a multi-llm demo.
gptel.el (gptel-host, gptel--known-backends, gptel--api-key,
gptel--create-prompt, gptel--request-data, gptel--parse-buffer, gptel-request,
gptel--parse-response, gptel--openai, gptel--debug, gptel--restore-state,
gptel, gptel-backend):
Integrate multiple LLMs through the introcution of gptel-backends. Each backend
is composed of two pieces:
1. An instance of a cl-struct, containing connection, authentication and model
information. See the cl-struct `gptel-backend` for details. A separate
cl-struct type is defined for each supported backend (OpenAI, Azure, GPT4All and
Ollama) that inherits from the generic gptel-backend type.
2. cl-generic implementations of specific tasks, like gathering up and
formatting context (previous user queries and LLM responses), parsing responses
or responses streams etc. The four tasks currently specialized this way are
carried out by `gptel--parse-buffer` and `gptel--request-data` (for constructing
the query) and `gptel--parse-response` and `gptel-curl--parse-stream` (for
parsing the response). See their implementations for details. Some effort has
been made to limit the number of times dispatching is done when reading
streaming responses.
When a backend is created, it is registered in the collection
`gptel--known-backends` and can be accessed by name later, such as from the
transient menu.
Only one of these backends is active at any time in a buffer, stored in the
buffer-local variable `gptel-backend`. Most messaging, authentication etc
accounts for the active backend, although there might be some leftovers.
When using `gptel-request` or `gptel-send`, the active backend can be changed or
let-bound.
- Obsolete `gptel-host`
- Fix the rear-sticky property when restoring sessions from files.
- Document some variables (not user options), like `gptel--debug`
gptel-openai.el (gptel-backend, gptel-make-openai, gptel-make-azure,
gptel-make-gpt4all): This file (currently always loaded) sets up the generic
backend struct and includes constructors for creating OpenAI, GPT4All and Azure
backends. They all use the same API so a single set of defgeneric
implemenations suffices for all of them.
gptel-ollama.el (gptel-make-ollama): This file includes the cl-struct,
constructor and requisite defgeneric implementations for Ollama support.
gptel-transient.el (gptel-menu, gptel-provider-variable, gptel--infix-provider,
gptel-suffix-send):
- Provide access to all available LLM backends and models from `gptel-menu`.
- Adjust keybindings in gptel-menu: setting the model and query parameters is
now bound to two char keybinds, while redirecting input and output is bound to
single keys.
2023-10-23 23:40:52 -07:00
#+end_src
Refer to the documentation of =gptel-make-azure= to set more parameters.
2023-12-31 14:37:26 -08:00
You can pick this backend from the menu when using gptel. (see [[#usage ][Usage ]])
gptel: Add multi-llm support
README.org: Update README with new information and a multi-llm demo.
gptel.el (gptel-host, gptel--known-backends, gptel--api-key,
gptel--create-prompt, gptel--request-data, gptel--parse-buffer, gptel-request,
gptel--parse-response, gptel--openai, gptel--debug, gptel--restore-state,
gptel, gptel-backend):
Integrate multiple LLMs through the introcution of gptel-backends. Each backend
is composed of two pieces:
1. An instance of a cl-struct, containing connection, authentication and model
information. See the cl-struct `gptel-backend` for details. A separate
cl-struct type is defined for each supported backend (OpenAI, Azure, GPT4All and
Ollama) that inherits from the generic gptel-backend type.
2. cl-generic implementations of specific tasks, like gathering up and
formatting context (previous user queries and LLM responses), parsing responses
or responses streams etc. The four tasks currently specialized this way are
carried out by `gptel--parse-buffer` and `gptel--request-data` (for constructing
the query) and `gptel--parse-response` and `gptel-curl--parse-stream` (for
parsing the response). See their implementations for details. Some effort has
been made to limit the number of times dispatching is done when reading
streaming responses.
When a backend is created, it is registered in the collection
`gptel--known-backends` and can be accessed by name later, such as from the
transient menu.
Only one of these backends is active at any time in a buffer, stored in the
buffer-local variable `gptel-backend`. Most messaging, authentication etc
accounts for the active backend, although there might be some leftovers.
When using `gptel-request` or `gptel-send`, the active backend can be changed or
let-bound.
- Obsolete `gptel-host`
- Fix the rear-sticky property when restoring sessions from files.
- Document some variables (not user options), like `gptel--debug`
gptel-openai.el (gptel-backend, gptel-make-openai, gptel-make-azure,
gptel-make-gpt4all): This file (currently always loaded) sets up the generic
backend struct and includes constructors for creating OpenAI, GPT4All and Azure
backends. They all use the same API so a single set of defgeneric
implemenations suffices for all of them.
gptel-ollama.el (gptel-make-ollama): This file includes the cl-struct,
constructor and requisite defgeneric implementations for Ollama support.
gptel-transient.el (gptel-menu, gptel-provider-variable, gptel--infix-provider,
gptel-suffix-send):
- Provide access to all available LLM backends and models from `gptel-menu`.
- Adjust keybindings in gptel-menu: setting the model and query parameters is
now bound to two char keybinds, while redirecting input and output is bound to
single keys.
2023-10-23 23:40:52 -07:00
If you want it to be the default, set it as the default value of =gptel-backend= :
#+begin_src emacs-lisp
2024-01-19 14:43:07 -08:00
(setq-default gptel-backend (gptel-make-azure "Azure-1" ...)
gptel-model "gpt-3.5-turbo")
gptel: Add multi-llm support
README.org: Update README with new information and a multi-llm demo.
gptel.el (gptel-host, gptel--known-backends, gptel--api-key,
gptel--create-prompt, gptel--request-data, gptel--parse-buffer, gptel-request,
gptel--parse-response, gptel--openai, gptel--debug, gptel--restore-state,
gptel, gptel-backend):
Integrate multiple LLMs through the introcution of gptel-backends. Each backend
is composed of two pieces:
1. An instance of a cl-struct, containing connection, authentication and model
information. See the cl-struct `gptel-backend` for details. A separate
cl-struct type is defined for each supported backend (OpenAI, Azure, GPT4All and
Ollama) that inherits from the generic gptel-backend type.
2. cl-generic implementations of specific tasks, like gathering up and
formatting context (previous user queries and LLM responses), parsing responses
or responses streams etc. The four tasks currently specialized this way are
carried out by `gptel--parse-buffer` and `gptel--request-data` (for constructing
the query) and `gptel--parse-response` and `gptel-curl--parse-stream` (for
parsing the response). See their implementations for details. Some effort has
been made to limit the number of times dispatching is done when reading
streaming responses.
When a backend is created, it is registered in the collection
`gptel--known-backends` and can be accessed by name later, such as from the
transient menu.
Only one of these backends is active at any time in a buffer, stored in the
buffer-local variable `gptel-backend`. Most messaging, authentication etc
accounts for the active backend, although there might be some leftovers.
When using `gptel-request` or `gptel-send`, the active backend can be changed or
let-bound.
- Obsolete `gptel-host`
- Fix the rear-sticky property when restoring sessions from files.
- Document some variables (not user options), like `gptel--debug`
gptel-openai.el (gptel-backend, gptel-make-openai, gptel-make-azure,
gptel-make-gpt4all): This file (currently always loaded) sets up the generic
backend struct and includes constructors for creating OpenAI, GPT4All and Azure
backends. They all use the same API so a single set of defgeneric
implemenations suffices for all of them.
gptel-ollama.el (gptel-make-ollama): This file includes the cl-struct,
constructor and requisite defgeneric implementations for Ollama support.
gptel-transient.el (gptel-menu, gptel-provider-variable, gptel--infix-provider,
gptel-suffix-send):
- Provide access to all available LLM backends and models from `gptel-menu`.
- Adjust keybindings in gptel-menu: setting the model and query parameters is
now bound to two char keybinds, while redirecting input and output is bound to
single keys.
2023-10-23 23:40:52 -07:00
#+end_src
#+html : </details>
#+html : <details><summary>
**** GPT4All
#+html : </summary>
Register a backend with
#+begin_src emacs-lisp
2024-01-19 14:43:07 -08:00
(gptel-make-gpt4all "GPT4All" ;Name of your choosing
2024-02-03 19:36:24 -05:00
:protocol "http"
gptel: Add multi-llm support
README.org: Update README with new information and a multi-llm demo.
gptel.el (gptel-host, gptel--known-backends, gptel--api-key,
gptel--create-prompt, gptel--request-data, gptel--parse-buffer, gptel-request,
gptel--parse-response, gptel--openai, gptel--debug, gptel--restore-state,
gptel, gptel-backend):
Integrate multiple LLMs through the introcution of gptel-backends. Each backend
is composed of two pieces:
1. An instance of a cl-struct, containing connection, authentication and model
information. See the cl-struct `gptel-backend` for details. A separate
cl-struct type is defined for each supported backend (OpenAI, Azure, GPT4All and
Ollama) that inherits from the generic gptel-backend type.
2. cl-generic implementations of specific tasks, like gathering up and
formatting context (previous user queries and LLM responses), parsing responses
or responses streams etc. The four tasks currently specialized this way are
carried out by `gptel--parse-buffer` and `gptel--request-data` (for constructing
the query) and `gptel--parse-response` and `gptel-curl--parse-stream` (for
parsing the response). See their implementations for details. Some effort has
been made to limit the number of times dispatching is done when reading
streaming responses.
When a backend is created, it is registered in the collection
`gptel--known-backends` and can be accessed by name later, such as from the
transient menu.
Only one of these backends is active at any time in a buffer, stored in the
buffer-local variable `gptel-backend`. Most messaging, authentication etc
accounts for the active backend, although there might be some leftovers.
When using `gptel-request` or `gptel-send`, the active backend can be changed or
let-bound.
- Obsolete `gptel-host`
- Fix the rear-sticky property when restoring sessions from files.
- Document some variables (not user options), like `gptel--debug`
gptel-openai.el (gptel-backend, gptel-make-openai, gptel-make-azure,
gptel-make-gpt4all): This file (currently always loaded) sets up the generic
backend struct and includes constructors for creating OpenAI, GPT4All and Azure
backends. They all use the same API so a single set of defgeneric
implemenations suffices for all of them.
gptel-ollama.el (gptel-make-ollama): This file includes the cl-struct,
constructor and requisite defgeneric implementations for Ollama support.
gptel-transient.el (gptel-menu, gptel-provider-variable, gptel--infix-provider,
gptel-suffix-send):
- Provide access to all available LLM backends and models from `gptel-menu`.
- Adjust keybindings in gptel-menu: setting the model and query parameters is
now bound to two char keybinds, while redirecting input and output is bound to
single keys.
2023-10-23 23:40:52 -07:00
:host "localhost:4891" ;Where it's running
:models '("mistral-7b-openorca.Q4_0.gguf")) ;Available models
#+end_src
These are the required parameters, refer to the documentation of =gptel-make-gpt4all= for more.
2023-12-31 14:37:26 -08:00
You can pick this backend from the menu when using gptel (see [[#usage ][Usage ]]), or set this as the default value of =gptel-backend= . Additionally you may want to increase the response token size since GPT4All uses very short (often truncated) responses by default:
2023-10-31 16:52:38 -07:00
#+begin_src emacs-lisp
;; OPTIONAL configuration
(setq-default gptel-model "mistral-7b-openorca.Q4_0.gguf" ;Pick your default model
gptel-backend (gptel-make-gpt4all "GPT4All" :protocol ...))
(setq-default gptel-max-tokens 500)
#+end_src
gptel: Add multi-llm support
README.org: Update README with new information and a multi-llm demo.
gptel.el (gptel-host, gptel--known-backends, gptel--api-key,
gptel--create-prompt, gptel--request-data, gptel--parse-buffer, gptel-request,
gptel--parse-response, gptel--openai, gptel--debug, gptel--restore-state,
gptel, gptel-backend):
Integrate multiple LLMs through the introcution of gptel-backends. Each backend
is composed of two pieces:
1. An instance of a cl-struct, containing connection, authentication and model
information. See the cl-struct `gptel-backend` for details. A separate
cl-struct type is defined for each supported backend (OpenAI, Azure, GPT4All and
Ollama) that inherits from the generic gptel-backend type.
2. cl-generic implementations of specific tasks, like gathering up and
formatting context (previous user queries and LLM responses), parsing responses
or responses streams etc. The four tasks currently specialized this way are
carried out by `gptel--parse-buffer` and `gptel--request-data` (for constructing
the query) and `gptel--parse-response` and `gptel-curl--parse-stream` (for
parsing the response). See their implementations for details. Some effort has
been made to limit the number of times dispatching is done when reading
streaming responses.
When a backend is created, it is registered in the collection
`gptel--known-backends` and can be accessed by name later, such as from the
transient menu.
Only one of these backends is active at any time in a buffer, stored in the
buffer-local variable `gptel-backend`. Most messaging, authentication etc
accounts for the active backend, although there might be some leftovers.
When using `gptel-request` or `gptel-send`, the active backend can be changed or
let-bound.
- Obsolete `gptel-host`
- Fix the rear-sticky property when restoring sessions from files.
- Document some variables (not user options), like `gptel--debug`
gptel-openai.el (gptel-backend, gptel-make-openai, gptel-make-azure,
gptel-make-gpt4all): This file (currently always loaded) sets up the generic
backend struct and includes constructors for creating OpenAI, GPT4All and Azure
backends. They all use the same API so a single set of defgeneric
implemenations suffices for all of them.
gptel-ollama.el (gptel-make-ollama): This file includes the cl-struct,
constructor and requisite defgeneric implementations for Ollama support.
gptel-transient.el (gptel-menu, gptel-provider-variable, gptel--infix-provider,
gptel-suffix-send):
- Provide access to all available LLM backends and models from `gptel-menu`.
- Adjust keybindings in gptel-menu: setting the model and query parameters is
now bound to two char keybinds, while redirecting input and output is bound to
single keys.
2023-10-23 23:40:52 -07:00
#+html : </details>
#+html : <details><summary>
**** Ollama
#+html : </summary>
Register a backend with
#+begin_src emacs-lisp
2024-01-19 14:43:07 -08:00
(gptel-make-ollama "Ollama" ;Any name of your choosing
:host "localhost:11434" ;Where it's running
:stream t ;Stream responses
:models '("mistral:latest")) ;List of models
gptel: Add multi-llm support
README.org: Update README with new information and a multi-llm demo.
gptel.el (gptel-host, gptel--known-backends, gptel--api-key,
gptel--create-prompt, gptel--request-data, gptel--parse-buffer, gptel-request,
gptel--parse-response, gptel--openai, gptel--debug, gptel--restore-state,
gptel, gptel-backend):
Integrate multiple LLMs through the introcution of gptel-backends. Each backend
is composed of two pieces:
1. An instance of a cl-struct, containing connection, authentication and model
information. See the cl-struct `gptel-backend` for details. A separate
cl-struct type is defined for each supported backend (OpenAI, Azure, GPT4All and
Ollama) that inherits from the generic gptel-backend type.
2. cl-generic implementations of specific tasks, like gathering up and
formatting context (previous user queries and LLM responses), parsing responses
or responses streams etc. The four tasks currently specialized this way are
carried out by `gptel--parse-buffer` and `gptel--request-data` (for constructing
the query) and `gptel--parse-response` and `gptel-curl--parse-stream` (for
parsing the response). See their implementations for details. Some effort has
been made to limit the number of times dispatching is done when reading
streaming responses.
When a backend is created, it is registered in the collection
`gptel--known-backends` and can be accessed by name later, such as from the
transient menu.
Only one of these backends is active at any time in a buffer, stored in the
buffer-local variable `gptel-backend`. Most messaging, authentication etc
accounts for the active backend, although there might be some leftovers.
When using `gptel-request` or `gptel-send`, the active backend can be changed or
let-bound.
- Obsolete `gptel-host`
- Fix the rear-sticky property when restoring sessions from files.
- Document some variables (not user options), like `gptel--debug`
gptel-openai.el (gptel-backend, gptel-make-openai, gptel-make-azure,
gptel-make-gpt4all): This file (currently always loaded) sets up the generic
backend struct and includes constructors for creating OpenAI, GPT4All and Azure
backends. They all use the same API so a single set of defgeneric
implemenations suffices for all of them.
gptel-ollama.el (gptel-make-ollama): This file includes the cl-struct,
constructor and requisite defgeneric implementations for Ollama support.
gptel-transient.el (gptel-menu, gptel-provider-variable, gptel--infix-provider,
gptel-suffix-send):
- Provide access to all available LLM backends and models from `gptel-menu`.
- Adjust keybindings in gptel-menu: setting the model and query parameters is
now bound to two char keybinds, while redirecting input and output is bound to
single keys.
2023-10-23 23:40:52 -07:00
#+end_src
2023-10-31 16:52:38 -07:00
These are the required parameters, refer to the documentation of =gptel-make-ollama= for more.
2023-12-31 14:37:26 -08:00
You can pick this backend from the menu when using gptel (see [[#usage ][Usage ]]), or set this as the default value of =gptel-backend= :
gptel: Add multi-llm support
README.org: Update README with new information and a multi-llm demo.
gptel.el (gptel-host, gptel--known-backends, gptel--api-key,
gptel--create-prompt, gptel--request-data, gptel--parse-buffer, gptel-request,
gptel--parse-response, gptel--openai, gptel--debug, gptel--restore-state,
gptel, gptel-backend):
Integrate multiple LLMs through the introcution of gptel-backends. Each backend
is composed of two pieces:
1. An instance of a cl-struct, containing connection, authentication and model
information. See the cl-struct `gptel-backend` for details. A separate
cl-struct type is defined for each supported backend (OpenAI, Azure, GPT4All and
Ollama) that inherits from the generic gptel-backend type.
2. cl-generic implementations of specific tasks, like gathering up and
formatting context (previous user queries and LLM responses), parsing responses
or responses streams etc. The four tasks currently specialized this way are
carried out by `gptel--parse-buffer` and `gptel--request-data` (for constructing
the query) and `gptel--parse-response` and `gptel-curl--parse-stream` (for
parsing the response). See their implementations for details. Some effort has
been made to limit the number of times dispatching is done when reading
streaming responses.
When a backend is created, it is registered in the collection
`gptel--known-backends` and can be accessed by name later, such as from the
transient menu.
Only one of these backends is active at any time in a buffer, stored in the
buffer-local variable `gptel-backend`. Most messaging, authentication etc
accounts for the active backend, although there might be some leftovers.
When using `gptel-request` or `gptel-send`, the active backend can be changed or
let-bound.
- Obsolete `gptel-host`
- Fix the rear-sticky property when restoring sessions from files.
- Document some variables (not user options), like `gptel--debug`
gptel-openai.el (gptel-backend, gptel-make-openai, gptel-make-azure,
gptel-make-gpt4all): This file (currently always loaded) sets up the generic
backend struct and includes constructors for creating OpenAI, GPT4All and Azure
backends. They all use the same API so a single set of defgeneric
implemenations suffices for all of them.
gptel-ollama.el (gptel-make-ollama): This file includes the cl-struct,
constructor and requisite defgeneric implementations for Ollama support.
gptel-transient.el (gptel-menu, gptel-provider-variable, gptel--infix-provider,
gptel-suffix-send):
- Provide access to all available LLM backends and models from `gptel-menu`.
- Adjust keybindings in gptel-menu: setting the model and query parameters is
now bound to two char keybinds, while redirecting input and output is bound to
single keys.
2023-10-23 23:40:52 -07:00
2023-10-31 16:52:38 -07:00
#+begin_src emacs-lisp
;; OPTIONAL configuration
(setq-default gptel-model "mistral:latest" ;Pick your default model
gptel-backend (gptel-make-ollama "Ollama" :host ...))
#+end_src
gptel: Add multi-llm support
README.org: Update README with new information and a multi-llm demo.
gptel.el (gptel-host, gptel--known-backends, gptel--api-key,
gptel--create-prompt, gptel--request-data, gptel--parse-buffer, gptel-request,
gptel--parse-response, gptel--openai, gptel--debug, gptel--restore-state,
gptel, gptel-backend):
Integrate multiple LLMs through the introcution of gptel-backends. Each backend
is composed of two pieces:
1. An instance of a cl-struct, containing connection, authentication and model
information. See the cl-struct `gptel-backend` for details. A separate
cl-struct type is defined for each supported backend (OpenAI, Azure, GPT4All and
Ollama) that inherits from the generic gptel-backend type.
2. cl-generic implementations of specific tasks, like gathering up and
formatting context (previous user queries and LLM responses), parsing responses
or responses streams etc. The four tasks currently specialized this way are
carried out by `gptel--parse-buffer` and `gptel--request-data` (for constructing
the query) and `gptel--parse-response` and `gptel-curl--parse-stream` (for
parsing the response). See their implementations for details. Some effort has
been made to limit the number of times dispatching is done when reading
streaming responses.
When a backend is created, it is registered in the collection
`gptel--known-backends` and can be accessed by name later, such as from the
transient menu.
Only one of these backends is active at any time in a buffer, stored in the
buffer-local variable `gptel-backend`. Most messaging, authentication etc
accounts for the active backend, although there might be some leftovers.
When using `gptel-request` or `gptel-send`, the active backend can be changed or
let-bound.
- Obsolete `gptel-host`
- Fix the rear-sticky property when restoring sessions from files.
- Document some variables (not user options), like `gptel--debug`
gptel-openai.el (gptel-backend, gptel-make-openai, gptel-make-azure,
gptel-make-gpt4all): This file (currently always loaded) sets up the generic
backend struct and includes constructors for creating OpenAI, GPT4All and Azure
backends. They all use the same API so a single set of defgeneric
implemenations suffices for all of them.
gptel-ollama.el (gptel-make-ollama): This file includes the cl-struct,
constructor and requisite defgeneric implementations for Ollama support.
gptel-transient.el (gptel-menu, gptel-provider-variable, gptel--infix-provider,
gptel-suffix-send):
- Provide access to all available LLM backends and models from `gptel-menu`.
- Adjust keybindings in gptel-menu: setting the model and query parameters is
now bound to two char keybinds, while redirecting input and output is bound to
single keys.
2023-10-23 23:40:52 -07:00
#+html : </details>
2023-12-20 13:51:01 -08:00
#+html : <details><summary>
2023-12-20 11:12:40 +00:00
**** Gemini
#+html : </summary>
Register a backend with
#+begin_src emacs-lisp
2023-12-20 13:51:01 -08:00
;; :key can be a function that returns the API key.
2024-01-19 14:43:07 -08:00
(gptel-make-gemini "Gemini"
:key "YOUR_GEMINI_API_KEY"
:stream t)
2023-12-20 11:12:40 +00:00
#+end_src
2023-12-20 13:51:01 -08:00
These are the required parameters, refer to the documentation of =gptel-make-gemini= for more.
2023-12-20 11:12:40 +00:00
2023-12-31 14:37:26 -08:00
You can pick this backend from the menu when using gptel (see [[#usage ][Usage ]]), or set this as the default value of =gptel-backend= :
2023-12-20 11:12:40 +00:00
#+begin_src emacs-lisp
;; OPTIONAL configuration
2023-12-20 13:51:01 -08:00
(setq-default gptel-model "gemini-pro" ;Pick your default model
2023-12-20 11:12:40 +00:00
gptel-backend (gptel-make-gemini "Gemini" :host ...))
#+end_src
#+html : </details>
2023-12-28 22:23:31 -08:00
#+html : <details>
#+html : <summary>
2023-12-31 14:37:26 -08:00
**** Llama.cpp or Llamafile
2023-12-28 22:23:31 -08:00
#+html : </summary>
2023-12-31 14:37:26 -08:00
(If using a llamafile, run a [[https://github.com/Mozilla-Ocho/llamafile#other-example-llamafiles ][server llamafile ]] instead of a "command-line llamafile", and a model that supports text generation.)
2023-12-28 22:23:31 -08:00
Register a backend with
#+begin_src emacs-lisp
2024-01-19 14:43:07 -08:00
;; Llama.cpp offers an OpenAI compatible API
(gptel-make-openai "llama-cpp" ;Any name
:stream t ;Stream responses
:protocol "http"
:host "localhost:8000" ;Llama.cpp server location
:models '("test")) ;Any names, doesn't matter for Llama
2023-12-28 22:23:31 -08:00
#+end_src
These are the required parameters, refer to the documentation of =gptel-make-openai= for more.
2023-12-31 14:37:26 -08:00
You can pick this backend from the menu when using gptel (see [[#usage ][Usage ]]), or set this as the default value of =gptel-backend= :
2023-12-28 22:23:31 -08:00
#+begin_src emacs-lisp
2024-01-19 14:43:07 -08:00
;; OPTIONAL configuration
2023-12-28 22:23:31 -08:00
(setq-default gptel-backend (gptel-make-openai "llama-cpp" ...)
gptel-model "test")
#+end_src
#+html : </details>
2023-12-21 21:23:28 -08:00
#+html : <details><summary>
2024-01-15 16:09:41 -08:00
**** Kagi (FastGPT & Summarizer)
2023-12-21 21:23:28 -08:00
#+html : </summary>
2024-01-15 16:09:41 -08:00
Kagi's FastGPT model and the Universal Summarizer are both supported. A couple of notes:
1. Universal Summarizer: If there is a URL at point, the summarizer will summarize the contents of the URL. Otherwise the context sent to the model is the same as always: the buffer text upto point, or the contents of the region if the region is active.
2. Kagi models do not support multi-turn conversations, interactions are "one-shot". They also do not support streaming responses.
2023-12-21 21:23:28 -08:00
Register a backend with
#+begin_src emacs-lisp
2024-01-19 14:43:07 -08:00
(gptel-make-kagi "Kagi" ;any name
:key "YOUR_KAGI_API_KEY") ;can be a function that returns the key
2023-12-21 21:23:28 -08:00
#+end_src
These are the required parameters, refer to the documentation of =gptel-make-kagi= for more.
2024-01-15 16:09:41 -08:00
You can pick this backend and the model (fastgpt/summarizer) from the transient menu when using gptel. Alternatively you can set this as the default value of =gptel-backend= :
2023-12-21 21:23:28 -08:00
#+begin_src emacs-lisp
;; OPTIONAL configuration
2024-01-15 16:09:41 -08:00
(setq-default gptel-model "fastgpt"
2023-12-21 21:23:28 -08:00
gptel-backend (gptel-make-kagi "Kagi" :key ...))
#+end_src
2024-01-15 16:09:41 -08:00
The alternatives to =fastgpt= include =summarize:cecil= , =summarize:agnes= , =summarize:daphne= and =summarize:muriel= . The difference between the summarizer engines is [[https://help.kagi.com/kagi/api/summarizer.html#summarization-engines ][documented here ]].
2024-01-19 14:43:07 -08:00
#+html : </details>
#+html : <details><summary>
**** together.ai
#+html : </summary>
Register a backend with
#+begin_src emacs-lisp
;; Together.ai offers an OpenAI compatible API
(gptel-make-openai "TogetherAI" ;Any name you want
:host "api.together.xyz"
:key "your-api-key" ;can be a function that returns the key
:stream t
:models '(;; has many more, check together.ai
"mistralai/Mixtral-8x7B-Instruct-v0.1"
"codellama/CodeLlama-13b-Instruct-hf"
"codellama/CodeLlama-34b-Instruct-hf"))
#+end_src
You can pick this backend from the menu when using gptel (see [[#usage ][Usage ]]), or set this as the default value of =gptel-backend= :
#+begin_src emacs-lisp
;; OPTIONAL configuration
(setq-default gptel-backend (gptel-make-openai "TogetherAI" ...)
gptel-model "mistralai/Mixtral-8x7B-Instruct-v0.1")
#+end_src
#+html : </details>
#+html : <details><summary>
**** Anyscale
#+html : </summary>
Register a backend with
#+begin_src emacs-lisp
;; Anyscale offers an OpenAI compatible API
(gptel-make-openai "Anyscale" ;Any name you want
:host "api.endpoints.anyscale.com"
:key "your-api-key" ;can be a function that returns the key
:models '(;; has many more, check anyscale
"mistralai/Mixtral-8x7B-Instruct-v0.1"))
#+end_src
You can pick this backend from the menu when using gptel (see [[#usage ][Usage ]]), or set this as the default value of =gptel-backend= :
#+begin_src emacs-lisp
;; OPTIONAL configuration
(setq-default gptel-backend (gptel-make-openai "Anyscale" ...)
gptel-model "mistralai/Mixtral-8x7B-Instruct-v0.1")
#+end_src
2024-02-03 19:36:24 -05:00
#+html : </details>
#+html : <details><summary>
**** Perplexity
#+html : </summary>
Register a backend with
#+begin_src emacs-lisp
;; Perplexity offers an OpenAI compatible API
(gptel-make-openai "Perplexity" ;Any name you want
:host "api.perplexity.ai"
:key "your-api-key" ;can be a function that returns the key
:endpoint "/chat/completions"
:stream t
:models '(;; has many more, check perplexity.ai
"pplx-7b-chat"
"pplx-70b-chat"
"pplx-7b-online"
"pplx-70b-online"))
#+end_src
You can pick this backend from the menu when using gptel (see [[#usage ][Usage ]]), or set this as the default value of =gptel-backend= :
#+begin_src emacs-lisp
;; OPTIONAL configuration
(setq-default gptel-backend (gptel-make-openai "Perplexity" ...)
gptel-model "pplx-7b-chat")
#+end_src
2023-12-21 21:23:28 -08:00
#+html : </details>
gptel: Add multi-llm support
README.org: Update README with new information and a multi-llm demo.
gptel.el (gptel-host, gptel--known-backends, gptel--api-key,
gptel--create-prompt, gptel--request-data, gptel--parse-buffer, gptel-request,
gptel--parse-response, gptel--openai, gptel--debug, gptel--restore-state,
gptel, gptel-backend):
Integrate multiple LLMs through the introcution of gptel-backends. Each backend
is composed of two pieces:
1. An instance of a cl-struct, containing connection, authentication and model
information. See the cl-struct `gptel-backend` for details. A separate
cl-struct type is defined for each supported backend (OpenAI, Azure, GPT4All and
Ollama) that inherits from the generic gptel-backend type.
2. cl-generic implementations of specific tasks, like gathering up and
formatting context (previous user queries and LLM responses), parsing responses
or responses streams etc. The four tasks currently specialized this way are
carried out by `gptel--parse-buffer` and `gptel--request-data` (for constructing
the query) and `gptel--parse-response` and `gptel-curl--parse-stream` (for
parsing the response). See their implementations for details. Some effort has
been made to limit the number of times dispatching is done when reading
streaming responses.
When a backend is created, it is registered in the collection
`gptel--known-backends` and can be accessed by name later, such as from the
transient menu.
Only one of these backends is active at any time in a buffer, stored in the
buffer-local variable `gptel-backend`. Most messaging, authentication etc
accounts for the active backend, although there might be some leftovers.
When using `gptel-request` or `gptel-send`, the active backend can be changed or
let-bound.
- Obsolete `gptel-host`
- Fix the rear-sticky property when restoring sessions from files.
- Document some variables (not user options), like `gptel--debug`
gptel-openai.el (gptel-backend, gptel-make-openai, gptel-make-azure,
gptel-make-gpt4all): This file (currently always loaded) sets up the generic
backend struct and includes constructors for creating OpenAI, GPT4All and Azure
backends. They all use the same API so a single set of defgeneric
implemenations suffices for all of them.
gptel-ollama.el (gptel-make-ollama): This file includes the cl-struct,
constructor and requisite defgeneric implementations for Ollama support.
gptel-transient.el (gptel-menu, gptel-provider-variable, gptel--infix-provider,
gptel-suffix-send):
- Provide access to all available LLM backends and models from `gptel-menu`.
- Adjust keybindings in gptel-menu: setting the model and query parameters is
now bound to two char keybinds, while redirecting input and output is bound to
single keys.
2023-10-23 23:40:52 -07:00
** Usage
2023-10-29 13:09:49 -07:00
2023-12-25 18:20:29 -08:00
(This is also a [[https://www.youtube.com/watch?v=bsRnh_brggM ][video demo ]] showing various uses of gptel.)
2023-10-29 13:09:49 -07:00
|-------------------+-------------------------------------------------------------------------|
2023-10-31 16:52:38 -07:00
| *Command* | Description |
2023-10-29 13:09:49 -07:00
|-------------------+-------------------------------------------------------------------------|
2023-12-21 14:56:26 -08:00
| =gptel-send= | Send conversation up to =(point)=, or selection if region is active. Works anywhere in Emacs. |
| =gptel= | Create a new dedicated chat buffer. Not required to use gptel. |
2024-01-03 21:58:29 -05:00
| =C-u= =gptel-send= | Transient menu for preferences, input/output redirection etc. |
2023-10-29 13:09:49 -07:00
| =gptel-menu= | /(Same)/ |
|-------------------+-------------------------------------------------------------------------|
| =gptel-set-topic= | /(Org-mode only)/ Limit conversation context to an Org heading |
|-------------------+-------------------------------------------------------------------------|
2023-04-06 17:29:40 -07:00
*** In any buffer:
2023-12-21 14:56:26 -08:00
1. Call =M-x gptel-send= to send the text up to the cursor. The response will be inserted below. Continue the conversation by typing below the response.
2023-04-09 04:41:24 -07:00
2023-12-21 14:56:26 -08:00
2. If a region is selected, the conversation will be limited to its contents.
2023-04-06 17:29:40 -07:00
2023-04-09 04:41:24 -07:00
3. Call =M-x gptel-send= with a prefix argument to
- set chat parameters (GPT model, directives etc) for this buffer,
- to read the prompt from elsewhere or redirect the response elsewhere,
- or to replace the prompt with the response.
[[https://user-images.githubusercontent.com/8607532/230770018-9ce87644-6c17-44af-bd39-8c899303dce1.png ]]
With a region selected, you can also rewrite prose or refactor code from here:
*Code* :
[[https://user-images.githubusercontent.com/8607532/230770162-1a5a496c-ee57-4a67-9c95-d45f238544ae.png ]]
*Prose* :
2023-04-06 17:29:40 -07:00
2023-04-09 04:41:24 -07:00
[[https://user-images.githubusercontent.com/8607532/230770352-ee6f45a3-a083-4cf0-b13c-619f7710e9ba.png ]]
2023-04-06 17:29:40 -07:00
2023-03-14 02:03:22 -07:00
*** In a dedicated chat buffer:
2023-03-10 04:12:32 -08:00
gptel: Add multi-llm support
README.org: Update README with new information and a multi-llm demo.
gptel.el (gptel-host, gptel--known-backends, gptel--api-key,
gptel--create-prompt, gptel--request-data, gptel--parse-buffer, gptel-request,
gptel--parse-response, gptel--openai, gptel--debug, gptel--restore-state,
gptel, gptel-backend):
Integrate multiple LLMs through the introcution of gptel-backends. Each backend
is composed of two pieces:
1. An instance of a cl-struct, containing connection, authentication and model
information. See the cl-struct `gptel-backend` for details. A separate
cl-struct type is defined for each supported backend (OpenAI, Azure, GPT4All and
Ollama) that inherits from the generic gptel-backend type.
2. cl-generic implementations of specific tasks, like gathering up and
formatting context (previous user queries and LLM responses), parsing responses
or responses streams etc. The four tasks currently specialized this way are
carried out by `gptel--parse-buffer` and `gptel--request-data` (for constructing
the query) and `gptel--parse-response` and `gptel-curl--parse-stream` (for
parsing the response). See their implementations for details. Some effort has
been made to limit the number of times dispatching is done when reading
streaming responses.
When a backend is created, it is registered in the collection
`gptel--known-backends` and can be accessed by name later, such as from the
transient menu.
Only one of these backends is active at any time in a buffer, stored in the
buffer-local variable `gptel-backend`. Most messaging, authentication etc
accounts for the active backend, although there might be some leftovers.
When using `gptel-request` or `gptel-send`, the active backend can be changed or
let-bound.
- Obsolete `gptel-host`
- Fix the rear-sticky property when restoring sessions from files.
- Document some variables (not user options), like `gptel--debug`
gptel-openai.el (gptel-backend, gptel-make-openai, gptel-make-azure,
gptel-make-gpt4all): This file (currently always loaded) sets up the generic
backend struct and includes constructors for creating OpenAI, GPT4All and Azure
backends. They all use the same API so a single set of defgeneric
implemenations suffices for all of them.
gptel-ollama.el (gptel-make-ollama): This file includes the cl-struct,
constructor and requisite defgeneric implementations for Ollama support.
gptel-transient.el (gptel-menu, gptel-provider-variable, gptel--infix-provider,
gptel-suffix-send):
- Provide access to all available LLM backends and models from `gptel-menu`.
- Adjust keybindings in gptel-menu: setting the model and query parameters is
now bound to two char keybinds, while redirecting input and output is bound to
single keys.
2023-10-23 23:40:52 -07:00
1. Run =M-x gptel= to start or switch to the chat buffer. It will ask you for the key if you skipped the previous step. Run it with a prefix-arg (=C-u M-x gptel= ) to start a new session.
2023-03-05 16:59:44 -08:00
2023-03-14 02:03:22 -07:00
2. In the gptel buffer, send your prompt with =M-x gptel-send= , bound to =C-c RET= .
2023-03-05 16:59:44 -08:00
gptel: Add multi-llm support
README.org: Update README with new information and a multi-llm demo.
gptel.el (gptel-host, gptel--known-backends, gptel--api-key,
gptel--create-prompt, gptel--request-data, gptel--parse-buffer, gptel-request,
gptel--parse-response, gptel--openai, gptel--debug, gptel--restore-state,
gptel, gptel-backend):
Integrate multiple LLMs through the introcution of gptel-backends. Each backend
is composed of two pieces:
1. An instance of a cl-struct, containing connection, authentication and model
information. See the cl-struct `gptel-backend` for details. A separate
cl-struct type is defined for each supported backend (OpenAI, Azure, GPT4All and
Ollama) that inherits from the generic gptel-backend type.
2. cl-generic implementations of specific tasks, like gathering up and
formatting context (previous user queries and LLM responses), parsing responses
or responses streams etc. The four tasks currently specialized this way are
carried out by `gptel--parse-buffer` and `gptel--request-data` (for constructing
the query) and `gptel--parse-response` and `gptel-curl--parse-stream` (for
parsing the response). See their implementations for details. Some effort has
been made to limit the number of times dispatching is done when reading
streaming responses.
When a backend is created, it is registered in the collection
`gptel--known-backends` and can be accessed by name later, such as from the
transient menu.
Only one of these backends is active at any time in a buffer, stored in the
buffer-local variable `gptel-backend`. Most messaging, authentication etc
accounts for the active backend, although there might be some leftovers.
When using `gptel-request` or `gptel-send`, the active backend can be changed or
let-bound.
- Obsolete `gptel-host`
- Fix the rear-sticky property when restoring sessions from files.
- Document some variables (not user options), like `gptel--debug`
gptel-openai.el (gptel-backend, gptel-make-openai, gptel-make-azure,
gptel-make-gpt4all): This file (currently always loaded) sets up the generic
backend struct and includes constructors for creating OpenAI, GPT4All and Azure
backends. They all use the same API so a single set of defgeneric
implemenations suffices for all of them.
gptel-ollama.el (gptel-make-ollama): This file includes the cl-struct,
constructor and requisite defgeneric implementations for Ollama support.
gptel-transient.el (gptel-menu, gptel-provider-variable, gptel--infix-provider,
gptel-suffix-send):
- Provide access to all available LLM backends and models from `gptel-menu`.
- Adjust keybindings in gptel-menu: setting the model and query parameters is
now bound to two char keybinds, while redirecting input and output is bound to
single keys.
2023-10-23 23:40:52 -07:00
3. Set chat parameters (LLM provider, model, directives etc) for the session by calling =gptel-send= with a prefix argument (=C-u C-c RET= ):
2023-03-14 02:03:22 -07:00
[[https://user-images.githubusercontent.com/8607532/224946059-9b918810-ab8b-46a6-b917-549d50c908f2.png ]]
2023-03-05 16:59:44 -08:00
That's it. You can go back and edit previous prompts and responses if you want.
2023-03-10 04:12:32 -08:00
The default mode is =markdown-mode= if available, else =text-mode= . You can set =gptel-default-mode= to =org-mode= if desired.
2023-08-09 17:56:58 -07:00
**** Save and restore your chat sessions
2024-02-03 19:36:24 -05:00
Saving the file will save the state of the conversation as well. To resume the chat, open the file and turn on =gptel-mode= before editing the buffer.
2023-08-09 17:56:58 -07:00
2023-12-21 14:56:26 -08:00
** FAQ
2024-01-15 17:07:58 -08:00
#+html : <details><summary>
**** I want the window to scroll automatically as the response is inserted
#+html : </summary>
2023-04-08 18:00:18 -07:00
2023-12-21 14:56:26 -08:00
To be minimally annoying, GPTel does not move the cursor by default. Add the following to your configuration to enable auto-scrolling.
2023-04-08 18:00:18 -07:00
2023-12-21 14:56:26 -08:00
#+begin_src emacs-lisp
(add-hook 'gptel-post-stream-hook 'gptel-auto-scroll)
#+end_src
2023-04-08 18:00:18 -07:00
2024-01-15 17:07:58 -08:00
#+html : </details>
#+html : <details><summary>
**** I want the cursor to move to the next prompt after the response is inserted
#+html : </summary>
2023-04-08 18:00:18 -07:00
2023-12-21 14:56:26 -08:00
To be minimally annoying, GPTel does not move the cursor by default. Add the following to your configuration to move the cursor:
2023-06-09 14:09:28 -07:00
2023-12-21 14:56:26 -08:00
#+begin_src emacs-lisp
2024-01-12 20:53:40 -08:00
(add-hook 'gptel-post-response-functions 'gptel-end-of-response)
2023-12-21 14:56:26 -08:00
#+end_src
2023-06-09 14:09:28 -07:00
2023-12-21 14:56:26 -08:00
You can also call =gptel-end-of-response= as a command at any time.
2024-01-15 17:07:58 -08:00
#+html : </details>
#+html : <details><summary>
2024-01-20 13:19:52 -08:00
**** I want to change the formatting of the prompt and LLM response
2024-01-15 17:07:58 -08:00
#+html : </summary>
2023-12-21 14:56:26 -08:00
2024-01-20 13:19:52 -08:00
For dedicated chat buffers: customize =gptel-prompt-prefix-alist= and =gptel-response-prefix-alist= . You can set a different pair for each major-mode.
2023-12-21 14:56:26 -08:00
2024-01-20 13:19:52 -08:00
Anywhere in Emacs: Use =gptel-pre-response-hook= and =gptel-post-response-functions= , which see.
2024-01-15 17:07:58 -08:00
2024-01-21 18:40:55 -08:00
#+html : </details>
#+html : <details><summary>
**** I want the transient menu options to be saved so I only need to set them once
#+html : </summary>
Any model options you set are saved for the current buffer. But the redirection options in the menu are set for the next query only:
https://github.com/karthink/gptel/assets/8607532/2ecc6be9-aa52-4287-a739-ba06e1369ec2
You can make them persistent across this Emacs session by pressing ~C-x C-s~ :
https://github.com/karthink/gptel/assets/8607532/b8bcb6ad-c974-41e1-9336-fdba0098a2fe
(You can also cycle through presets you've saved with ~C-x p~ and ~C-x n~ .)
Now these will be enabled whenever you send a query from the transient menu. If you want to use these options without invoking the transient menu, you can use a keyboard macro:
#+begin_src emacs-lisp
;; Replace with your key to invoke the transient menu:
(keymap-global-set "<f6 >" "C-u C-c <return > <return >")
#+end_src
See [[https://github.com/karthink/gptel/issues/94#issuecomment-1657093458 ][this comment by Tianshu Wang ]] for an Elisp solution.
2024-01-15 17:07:58 -08:00
#+html : </details>
2024-01-20 13:19:52 -08:00
#+html : <details><summary>
**** I want to use gptel in a way that's not supported by =gptel-send= or the options menu
#+html : </summary>
GPTel's default usage pattern is simple, and will stay this way: Read input in any buffer and insert the response below it. Some custom behavior is possible with the transient menu (=C-u M-x gptel-send= ).
2024-01-15 17:07:58 -08:00
2024-01-20 13:19:52 -08:00
For more programmable usage, gptel provides a general =gptel-request= function that accepts a custom prompt and a callback to act on the response. You can use this to build custom workflows not supported by =gptel-send= . See the documentation of =gptel-request= , and the [[https://github.com/karthink/gptel/wiki ][wiki ]] for examples.
#+html : </details>
2024-01-15 17:07:58 -08:00
#+html : <details><summary>
2024-02-03 19:36:24 -05:00
**** (Doom Emacs) Sending a query from the gptel menu fails because of a key conflict with Org mode
2024-01-15 17:07:58 -08:00
#+html : </summary>
Doom binds ~RET~ in Org mode to =+org/dwim-at-point= , which appears to conflict with gptel's transient menu bindings for some reason.
Two solutions:
- Press ~C-m~ instead of the return key.
- Change the send key from return to a key of your choice:
#+begin_src emacs-lisp
(transient-suffix-put 'gptel-menu (kbd "RET") :key "<f8 >")
#+end_src
2024-02-03 09:40:52 -08:00
#+html : </details>
#+html : <details><summary>
**** (ChatGPT) I get the error "(HTTP/2 429) You exceeded your current quota"
#+html :</summary>
#+begin_quote
(HTTP/2 429) You exceeded your current quota, please check your plan and billing details.
#+end_quote
Using the ChatGPT (or any OpenAI) API requires [[https://platform.openai.com/account/billing/overview ][adding credit to your account ]].
2024-01-15 17:07:58 -08:00
#+html : </details>
#+html : <details><summary>
**** Why another LLM client?
#+html : </summary>
2023-12-21 14:56:26 -08:00
Other Emacs clients for LLMs prescribe the format of the interaction (a comint shell, org-babel blocks, etc). I wanted:
1. Something that is as free-form as possible: query the model using any text in any buffer, and redirect the response as required. Using a dedicated =gptel= buffer just adds some visual flair to the interaction.
2. Integration with org-mode, not using a walled-off org-babel block, but as regular text. This way the model can generate code blocks that I can run.
2023-06-09 14:09:28 -07:00
2024-01-15 17:07:58 -08:00
#+html : </details>
2023-06-01 11:24:13 +08:00
** Additional Configuration
gptel: Add multi-llm support
README.org: Update README with new information and a multi-llm demo.
gptel.el (gptel-host, gptel--known-backends, gptel--api-key,
gptel--create-prompt, gptel--request-data, gptel--parse-buffer, gptel-request,
gptel--parse-response, gptel--openai, gptel--debug, gptel--restore-state,
gptel, gptel-backend):
Integrate multiple LLMs through the introcution of gptel-backends. Each backend
is composed of two pieces:
1. An instance of a cl-struct, containing connection, authentication and model
information. See the cl-struct `gptel-backend` for details. A separate
cl-struct type is defined for each supported backend (OpenAI, Azure, GPT4All and
Ollama) that inherits from the generic gptel-backend type.
2. cl-generic implementations of specific tasks, like gathering up and
formatting context (previous user queries and LLM responses), parsing responses
or responses streams etc. The four tasks currently specialized this way are
carried out by `gptel--parse-buffer` and `gptel--request-data` (for constructing
the query) and `gptel--parse-response` and `gptel-curl--parse-stream` (for
parsing the response). See their implementations for details. Some effort has
been made to limit the number of times dispatching is done when reading
streaming responses.
When a backend is created, it is registered in the collection
`gptel--known-backends` and can be accessed by name later, such as from the
transient menu.
Only one of these backends is active at any time in a buffer, stored in the
buffer-local variable `gptel-backend`. Most messaging, authentication etc
accounts for the active backend, although there might be some leftovers.
When using `gptel-request` or `gptel-send`, the active backend can be changed or
let-bound.
- Obsolete `gptel-host`
- Fix the rear-sticky property when restoring sessions from files.
- Document some variables (not user options), like `gptel--debug`
gptel-openai.el (gptel-backend, gptel-make-openai, gptel-make-azure,
gptel-make-gpt4all): This file (currently always loaded) sets up the generic
backend struct and includes constructors for creating OpenAI, GPT4All and Azure
backends. They all use the same API so a single set of defgeneric
implemenations suffices for all of them.
gptel-ollama.el (gptel-make-ollama): This file includes the cl-struct,
constructor and requisite defgeneric implementations for Ollama support.
gptel-transient.el (gptel-menu, gptel-provider-variable, gptel--infix-provider,
gptel-suffix-send):
- Provide access to all available LLM backends and models from `gptel-menu`.
- Adjust keybindings in gptel-menu: setting the model and query parameters is
now bound to two char keybinds, while redirecting input and output is bound to
single keys.
2023-10-23 23:40:52 -07:00
:PROPERTIES:
:ID: f885adac-58a3-4eba-a6b7-91e9e7a17829
:END:
2023-10-29 13:09:49 -07:00
#+begin_src emacs-lisp :exports none :results list
gptel: Add multi-llm support
README.org: Update README with new information and a multi-llm demo.
gptel.el (gptel-host, gptel--known-backends, gptel--api-key,
gptel--create-prompt, gptel--request-data, gptel--parse-buffer, gptel-request,
gptel--parse-response, gptel--openai, gptel--debug, gptel--restore-state,
gptel, gptel-backend):
Integrate multiple LLMs through the introcution of gptel-backends. Each backend
is composed of two pieces:
1. An instance of a cl-struct, containing connection, authentication and model
information. See the cl-struct `gptel-backend` for details. A separate
cl-struct type is defined for each supported backend (OpenAI, Azure, GPT4All and
Ollama) that inherits from the generic gptel-backend type.
2. cl-generic implementations of specific tasks, like gathering up and
formatting context (previous user queries and LLM responses), parsing responses
or responses streams etc. The four tasks currently specialized this way are
carried out by `gptel--parse-buffer` and `gptel--request-data` (for constructing
the query) and `gptel--parse-response` and `gptel-curl--parse-stream` (for
parsing the response). See their implementations for details. Some effort has
been made to limit the number of times dispatching is done when reading
streaming responses.
When a backend is created, it is registered in the collection
`gptel--known-backends` and can be accessed by name later, such as from the
transient menu.
Only one of these backends is active at any time in a buffer, stored in the
buffer-local variable `gptel-backend`. Most messaging, authentication etc
accounts for the active backend, although there might be some leftovers.
When using `gptel-request` or `gptel-send`, the active backend can be changed or
let-bound.
- Obsolete `gptel-host`
- Fix the rear-sticky property when restoring sessions from files.
- Document some variables (not user options), like `gptel--debug`
gptel-openai.el (gptel-backend, gptel-make-openai, gptel-make-azure,
gptel-make-gpt4all): This file (currently always loaded) sets up the generic
backend struct and includes constructors for creating OpenAI, GPT4All and Azure
backends. They all use the same API so a single set of defgeneric
implemenations suffices for all of them.
gptel-ollama.el (gptel-make-ollama): This file includes the cl-struct,
constructor and requisite defgeneric implementations for Ollama support.
gptel-transient.el (gptel-menu, gptel-provider-variable, gptel--infix-provider,
gptel-suffix-send):
- Provide access to all available LLM backends and models from `gptel-menu`.
- Adjust keybindings in gptel-menu: setting the model and query parameters is
now bound to two char keybinds, while redirecting input and output is bound to
single keys.
2023-10-23 23:40:52 -07:00
(let ((all))
(mapatoms (lambda (sym)
(when (and (string-match-p "^gptel-[^-]" (symbol-name sym))
(get sym 'variable-documentation))
(push sym all))))
all)
#+end_src
2023-06-01 11:24:13 +08:00
2023-10-29 13:09:49 -07:00
|---------------------------+---------------------------------------------------------------------|
| *Connection options* | |
|---------------------------+---------------------------------------------------------------------|
| =gptel-use-curl= | Use Curl (default), fallback to Emacs' built-in =url=. |
| =gptel-proxy= | Proxy server for requests, passed to curl via =--proxy=. |
| =gptel-api-key= | Variable/function that returns the API key for the active backend. |
|---------------------------+---------------------------------------------------------------------|
2023-12-25 18:20:29 -08:00
|-------------------+---------------------------------------------------------|
| *LLM options* | /(Note: not supported uniformly across LLMs)/ |
|-------------------+---------------------------------------------------------|
| =gptel-backend= | Default LLM Backend. |
| =gptel-model= | Default model to use, depends on the backend. |
| =gptel-stream= | Enable streaming responses, if the backend supports it. |
| =gptel-directives= | Alist of system directives, can switch on the fly. |
| =gptel-max-tokens= | Maximum token count (in query + response). |
| =gptel-temperature= | Randomness in response text, 0 to 2. |
|-------------------+---------------------------------------------------------|
2023-10-29 13:09:49 -07:00
2024-02-10 12:00:19 -08:00
|-----------------------------+----------------------------------------------------------------|
| *Chat UI options* | |
|-----------------------------+----------------------------------------------------------------|
| =gptel-default-mode= | Major mode for dedicated chat buffers. |
| =gptel-prompt-prefix-alist= | Text inserted before queries. |
| =gptel-response-prefix-alist= | Text inserted before responses. |
| =gptel-use-header-line= | Display status messages in header-line (default) or minibuffer |
| =gptel-display-buffer-action= | Placement of the gptel chat buffer. |
|-----------------------------+----------------------------------------------------------------|
2023-12-15 09:42:37 -08:00
2023-12-21 14:56:26 -08:00
** COMMENT Will you add feature X?
2023-03-05 16:59:44 -08:00
2023-05-19 23:11:30 -07:00
Maybe, I'd like to experiment a bit more first. Features added since the inception of this package include
- Curl support (=gptel-use-curl= )
- Streaming responses (=gptel-stream= )
- Cancelling requests in progress (=gptel-abort= )
- General API for writing your own commands (=gptel-request= , [[https://github.com/karthink/gptel/wiki ][wiki ]])
- Dispatch menus using Transient (=gptel-send= with a prefix arg)
- Specifying the conversation context size
- GPT-4 support
- Response redirection (to the echo area, another buffer, etc)
- A built-in refactor/rewrite prompt
gptel: saving and restoring state for Markdown/Text
* gptel.el (gptel--save-state, gptel--restore-state,
gptel-temperature, gptel-model, gptel-max-tokens,
gptel-directives, gptel--always, gptel--button-buttonize,
gptel--system-message, gptel--bounds): Write gptel parameters as
file-local variables when saving chats in Markdown or text files.
The local variable gptel--bounds stores the locations of the
responses from the LLM. This is not a great solution, but the best
I can think to do without adding more syntax to the document.
Chats can be restored by turning on `gptel-mode'. One of the
problem with this approach is that if the buffer is modified
before `gptel-mode' is turned on, the state data is out of date.
Another problem is that this metadata block as printed in the
buffer can become quite long. A better approach is needed.
Define helper functions `gptel--always' and
`gptel--button-buttonize' to work around Emacs 27.1 support.
* README.org: Mention saving and restoring chats where
appropriate.
2023-07-28 15:57:08 -07:00
- Limiting conversation context to Org headings using properties (#58)
- Saving and restoring chats (#17)
gptel: Add multi-llm support
README.org: Update README with new information and a multi-llm demo.
gptel.el (gptel-host, gptel--known-backends, gptel--api-key,
gptel--create-prompt, gptel--request-data, gptel--parse-buffer, gptel-request,
gptel--parse-response, gptel--openai, gptel--debug, gptel--restore-state,
gptel, gptel-backend):
Integrate multiple LLMs through the introcution of gptel-backends. Each backend
is composed of two pieces:
1. An instance of a cl-struct, containing connection, authentication and model
information. See the cl-struct `gptel-backend` for details. A separate
cl-struct type is defined for each supported backend (OpenAI, Azure, GPT4All and
Ollama) that inherits from the generic gptel-backend type.
2. cl-generic implementations of specific tasks, like gathering up and
formatting context (previous user queries and LLM responses), parsing responses
or responses streams etc. The four tasks currently specialized this way are
carried out by `gptel--parse-buffer` and `gptel--request-data` (for constructing
the query) and `gptel--parse-response` and `gptel-curl--parse-stream` (for
parsing the response). See their implementations for details. Some effort has
been made to limit the number of times dispatching is done when reading
streaming responses.
When a backend is created, it is registered in the collection
`gptel--known-backends` and can be accessed by name later, such as from the
transient menu.
Only one of these backends is active at any time in a buffer, stored in the
buffer-local variable `gptel-backend`. Most messaging, authentication etc
accounts for the active backend, although there might be some leftovers.
When using `gptel-request` or `gptel-send`, the active backend can be changed or
let-bound.
- Obsolete `gptel-host`
- Fix the rear-sticky property when restoring sessions from files.
- Document some variables (not user options), like `gptel--debug`
gptel-openai.el (gptel-backend, gptel-make-openai, gptel-make-azure,
gptel-make-gpt4all): This file (currently always loaded) sets up the generic
backend struct and includes constructors for creating OpenAI, GPT4All and Azure
backends. They all use the same API so a single set of defgeneric
implemenations suffices for all of them.
gptel-ollama.el (gptel-make-ollama): This file includes the cl-struct,
constructor and requisite defgeneric implementations for Ollama support.
gptel-transient.el (gptel-menu, gptel-provider-variable, gptel--infix-provider,
gptel-suffix-send):
- Provide access to all available LLM backends and models from `gptel-menu`.
- Adjust keybindings in gptel-menu: setting the model and query parameters is
now bound to two char keybinds, while redirecting input and output is bound to
single keys.
2023-10-23 23:40:52 -07:00
- Support for local LLMs.
2023-05-19 23:11:30 -07:00
Features being considered or in the pipeline:
gptel: saving and restoring state for Markdown/Text
* gptel.el (gptel--save-state, gptel--restore-state,
gptel-temperature, gptel-model, gptel-max-tokens,
gptel-directives, gptel--always, gptel--button-buttonize,
gptel--system-message, gptel--bounds): Write gptel parameters as
file-local variables when saving chats in Markdown or text files.
The local variable gptel--bounds stores the locations of the
responses from the LLM. This is not a great solution, but the best
I can think to do without adding more syntax to the document.
Chats can be restored by turning on `gptel-mode'. One of the
problem with this approach is that if the buffer is modified
before `gptel-mode' is turned on, the state data is out of date.
Another problem is that this metadata block as printed in the
buffer can become quite long. A better approach is needed.
Define helper functions `gptel--always' and
`gptel--button-buttonize' to work around Emacs 27.1 support.
* README.org: Mention saving and restoring chats where
appropriate.
2023-07-28 15:57:08 -07:00
- Fully stateless design (#17)
2023-05-19 23:11:30 -07:00
2024-01-20 13:19:52 -08:00
** Alternatives
2023-12-21 14:56:26 -08:00
2024-01-20 13:19:52 -08:00
Other Emacs clients for LLMs include
2023-12-21 14:56:26 -08:00
2024-01-20 13:19:52 -08:00
- [[https://github.com/xenodium/chatgpt-shell ][chatgpt-shell ]]: comint-shell based interaction with ChatGPT. Also supports DALL-E, executable code blocks in the responses, and more.
- [[https://github.com/rksm/org-ai ][org-ai ]]: Interaction through special =#+begin_ai ... #+end_ai= Org-mode blocks. Also supports DALL-E, querying ChatGPT with the contents of project files, and more.
There are several more: [[https://github.com/CarlQLange/chatgpt-arcana.el ][chatgpt-arcana ]], [[https://github.com/MichaelBurge/leafy-mode ][leafy-mode ]], [[https://github.com/iwahbe/chat.el ][chat.el ]]
2023-12-21 14:56:26 -08:00
*** Extensions using GPTel
These are packages that depend on GPTel to provide additional functionality
- [[https://github.com/kamushadenes/gptel-extensions.el ][gptel-extensions ]]: Extra utility functions for GPTel.
- [[https://github.com/kamushadenes/ai-blog.el ][ai-blog.el ]]: Streamline generation of blog posts in Hugo.
** Breaking Changes
2024-01-12 20:53:40 -08:00
- =gptel-post-response-hook= has been renamed to =gptel-post-response-functions= , and functions in this hook are now called with two arguments: the start and end buffer positions of the response. This should make it easy to act on the response text without having to locate it first.
2023-12-21 14:56:26 -08:00
- Possible breakage, see #120: If streaming responses stop working for you after upgrading to v0.5, try reinstalling gptel and deleting its native comp eln cache in =native-comp-eln-load-path= .
- The user option =gptel-host= is deprecated. If the defaults don't work for you, use =gptel-make-openai= (which see) to customize server settings.
- =gptel-api-key-from-auth-source= now searches for the API key using the host address for the active LLM backend, /i.e./ "api.openai.com" when using ChatGPT. You may need to update your =~/.authinfo= .
2023-05-19 23:11:30 -07:00
** Acknowledgments
- [[https://github.com/algal ][Alexis Gallagher ]] and [[https://github.com/d1egoaz ][Diego Alvarez ]] for fixing a nasty multi-byte bug with =url-retrieve= .
- [[https://github.com/tarsius ][Jonas Bernoulli ]] for the Transient library.
2023-03-05 16:59:44 -08:00
2023-06-12 17:27:52 -07:00
# Local Variables:
# toc-org-max-depth: 4
2023-12-21 14:56:26 -08:00
# eval: (and (fboundp 'toc-org-mode) (toc-org-mode 1))
2023-06-12 17:27:52 -07:00
# End:
