gptel-kagi: Add support for Kagi FastGPT

* gptel.el: Bump version and update package description.

* gptel-kagi.el (gptel--parse-response, gptel--request-data,
gptel--parse-buffer, gptel-make-kagi): Add new file and support
for the Kagi FastGPT LLM API.  Streaming and setting model
parameters (temperature, max tokesn) are not supported by the API.
A Kagi backend can be added with `gptel-make-kagi`.

* README.org: Update with instructions for Kagi.
This commit is contained in:
Karthik Chikmagalur 2023-12-21 21:23:28 -08:00
parent 612aea3456
commit c6a07043af
3 changed files with 214 additions and 25 deletions

View file

@ -2,18 +2,19 @@
[[https://melpa.org/#/gptel][file:https://melpa.org/packages/gptel-badge.svg]] [[https://melpa.org/#/gptel][file:https://melpa.org/packages/gptel-badge.svg]]
GPTel is a simple Large Language Model chat client for Emacs, with support for multiple models/backends. GPTel is a simple Large Language Model chat client for Emacs, with support for multiple models and backends.
| LLM Backend | Supports | Requires | | LLM Backend | Supports | Requires |
|-------------+----------+---------------------------| |--------------+----------+---------------------------|
| ChatGPT | ✓ | [[https://platform.openai.com/account/api-keys][API key]] | | ChatGPT | ✓ | [[https://platform.openai.com/account/api-keys][API key]] |
| Azure | ✓ | Deployment and API key | | Azure | ✓ | Deployment and API key |
| Ollama | ✓ | [[https://ollama.ai/][Ollama running locally]] | | Ollama | ✓ | [[https://ollama.ai/][Ollama running locally]] |
| GPT4All | ✓ | [[https://gpt4all.io/index.html][GPT4All running locally]] | | GPT4All | ✓ | [[https://gpt4all.io/index.html][GPT4All running locally]] |
| Gemini | ✓ | [[https://makersuite.google.com/app/apikey][API key]] | | Gemini | ✓ | [[https://makersuite.google.com/app/apikey][API key]] |
| Llama.cpp | ✓ | [[https://github.com/ggerganov/llama.cpp/tree/master/examples/server#quick-start][Llama.cpp running locally]] | | Llama.cpp | ✓ | [[https://github.com/ggerganov/llama.cpp/tree/master/examples/server#quick-start][Llama.cpp running locally]] |
| Llamafile | ✓ | [[https://github.com/Mozilla-Ocho/llamafile#quickstart][Local Llamafile server]] | | Llamafile | ✓ | [[https://github.com/Mozilla-Ocho/llamafile#quickstart][Local Llamafile server]] |
| PrivateGPT | Planned | - | | Kagi FastGPT | ✓ | [[https://kagi.com/settings?p=api][API key]] |
| PrivateGPT | Planned | - |
*General usage*: ([[https://www.youtube.com/watch?v=bsRnh_brggM][YouTube Demo]]) *General usage*: ([[https://www.youtube.com/watch?v=bsRnh_brggM][YouTube Demo]])
@ -48,6 +49,7 @@ GPTel uses Curl if available, but falls back to url-retrieve to work without ext
- [[#ollama][Ollama]] - [[#ollama][Ollama]]
- [[#gemini][Gemini]] - [[#gemini][Gemini]]
- [[#llamacpp-or-llamafile][Llama.cpp or Llamafile]] - [[#llamacpp-or-llamafile][Llama.cpp or Llamafile]]
- [[#kagi-fastgpt][Kagi FastGPT]]
- [[#usage][Usage]] - [[#usage][Usage]]
- [[#in-any-buffer][In any buffer:]] - [[#in-any-buffer][In any buffer:]]
- [[#in-a-dedicated-chat-buffer][In a dedicated chat buffer:]] - [[#in-a-dedicated-chat-buffer][In a dedicated chat buffer:]]
@ -249,6 +251,31 @@ You can pick this backend from the menu when using gptel (see [[#usage][Usage]])
#+end_src #+end_src
#+html: </details> #+html: </details>
#+html: <details><summary>
**** Kagi FastGPT
#+html: </summary>
*NOTE*: Kagi's FastGPT model does not support multi-turn conversations, interactions are "one-shot". It also does not support streaming responses.
Register a backend with
#+begin_src emacs-lisp
;; :key can be a function that returns the API key
(gptel-make-kagi
"Kagi" ;Name of your choice
:key "YOUR_KAGI_API_KEY")
#+end_src
These are the required parameters, refer to the documentation of =gptel-make-kagi= for more.
You can pick this backend from the transient menu when using gptel (see Usage), or set this as the default value of =gptel-backend=:
#+begin_src emacs-lisp
;; OPTIONAL configuration
(setq-default gptel-model "fastgpt" ;only supported Kagi model
gptel-backend (gptel-make-kagi "Kagi" :key ...))
#+end_src
#+html: </details>
** Usage ** Usage
(This is also a [[https://www.youtube.com/watch?v=bsRnh_brggM][video demo]] showing various uses of gptel.) (This is also a [[https://www.youtube.com/watch?v=bsRnh_brggM][video demo]] showing various uses of gptel.)

154
gptel-kagi.el Normal file
View file

@ -0,0 +1,154 @@
;;; gptel-kagi.el --- Kagi support for gptel -*- lexical-binding: t; -*-
;; Copyright (C) 2023 Karthik Chikmagalur
;; Author: Karthik Chikmagalur <karthikchikmagalur@gmail.com>
;; Keywords: hypermedia
;; This program is free software; you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation, either version 3 of the License, or
;; (at your option) any later version.
;; This program is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
;; GNU General Public License for more details.
;; You should have received a copy of the GNU General Public License
;; along with this program. If not, see <https://www.gnu.org/licenses/>.
;;; Commentary:
;; This file adds support for the Kagi FastGPT LLM API to gptel
;;; Code:
(require 'gptel)
(require 'cl-generic)
(eval-when-compile
(require 'cl-lib))
;;; Kagi
(cl-defstruct (gptel-kagi (:constructor gptel--make-kagi)
(:copier nil)
(:include gptel-backend)))
(cl-defmethod gptel--parse-response ((_backend gptel-kagi) response info)
(let* ((data (plist-get response :data))
(output (plist-get data :output))
(references (plist-get data :references)))
(when references
(setq references
(cl-loop with linker =
(pcase (buffer-local-value 'major-mode
(plist-get info :buffer))
('org-mode
(lambda (text url)
(format "[[%s][%s]]" url text)))
('markdown-mode
(lambda (text url)
(format "[%s](%s)" text url)))
(_ (lambda (text url)
(buttonize
text (lambda (data) (browse-url data))
url))))
for ref across references
for title = (plist-get ref :title)
for snippet = (plist-get ref :snippet)
for url = (plist-get ref :url)
for n upfrom 1
collect
(concat (format "[%d] " n)
(funcall linker title url) ": "
(replace-regexp-in-string
"</?b>" "*" snippet))
into ref-strings
finally return
(concat "\n\n" (mapconcat #'identity ref-strings "\n")))))
(concat output references)))
(cl-defmethod gptel--request-data ((_backend gptel-kagi) prompts)
"JSON encode PROMPTS for sending to ChatGPT."
`(,@prompts :web_search t :cache t))
(cl-defmethod gptel--parse-buffer ((_backend gptel-kagi) &optional _max-entries)
(let ((prompts)
(prop (text-property-search-backward
'gptel 'response
(when (get-char-property (max (point-min) (1- (point)))
'gptel)
t))))
(if (and (prop-match-p prop)
(prop-match-value prop))
(user-error "No user prompt found!")
(setq prompts (list
:query
(if (prop-match-p prop)
(concat
;; Fake a system message by including it in the prompt
gptel--system-message "\n\n"
(string-trim
(buffer-substring-no-properties (prop-match-beginning prop)
(prop-match-end prop))
(format "[\t\r\n ]*\\(?:%s\\)?[\t\r\n ]*"
(regexp-quote (gptel-prompt-prefix-string)))
(format "[\t\r\n ]*\\(?:%s\\)?[\t\r\n ]*"
(regexp-quote (gptel-response-prefix-string)))))
"")))
prompts)))
;;;###autoload
(cl-defun gptel-make-kagi
(name &key stream key
(host "kagi.com")
(header (lambda () `(("Authorization" . ,(concat "Bot " (gptel--get-api-key))))))
(models '("fastgpt"))
(protocol "https")
(endpoint "/api/v0/fastgpt"))
"Register a Kagi FastGPT backend for gptel with NAME.
Keyword arguments:
HOST is the Kagi host (with port), defaults to \"kagi.com\".
MODELS is a list of available Kagi models: only fastgpt is supported.
STREAM is a boolean to toggle streaming responses, defaults to
false. Kagi does not support a streaming API yet.
PROTOCOL (optional) specifies the protocol, https by default.
ENDPOINT (optional) is the API endpoint for completions, defaults to
\"/api/v0/fastgpt\".
HEADER (optional) is for additional headers to send with each
request. It should be an alist or a function that retuns an
alist, like:
((\"Content-Type\" . \"application/json\"))
KEY (optional) is a variable whose value is the API key, or
function that returns the key.
Example:
-------
(gptel-make-kagi \"Kagi\" :key my-kagi-key)"
stream ;Silence byte-compiler
(let ((backend (gptel--make-kagi
:name name
:host host
:header header
:key key
:models models
:protocol protocol
:endpoint endpoint
:url (if protocol
(concat protocol "://" host endpoint)
(concat host endpoint)))))
(prog1 backend
(setf (alist-get name gptel--known-backends
nil nil #'equal)
backend))))
(provide 'gptel-kagi)
;;; gptel-kagi.el ends here

View file

@ -3,7 +3,7 @@
;; Copyright (C) 2023 Karthik Chikmagalur ;; Copyright (C) 2023 Karthik Chikmagalur
;; Author: Karthik Chikmagalur ;; Author: Karthik Chikmagalur
;; Version: 0.5.5 ;; Version: 0.6.0
;; Package-Requires: ((emacs "27.1") (transient "0.4.0") (compat "29.1.4.1")) ;; Package-Requires: ((emacs "27.1") (transient "0.4.0") (compat "29.1.4.1"))
;; Keywords: convenience ;; Keywords: convenience
;; URL: https://github.com/karthink/gptel ;; URL: https://github.com/karthink/gptel
@ -29,20 +29,24 @@
;; gptel is a simple Large Language Model chat client, with support for multiple models/backends. ;; gptel is a simple Large Language Model chat client, with support for multiple models/backends.
;; ;;
;; gptel supports ChatGPT, Azure, Gemini and local models using Ollama and ;; gptel supports
;; GPT4All. ;; - The services ChatGPT, Azure, Gemini, and Kagi (FastGPT)
;; - Local models via Ollama, Llama.cpp, Llamafiles or GPT4All
;; ;;
;; Features: ;; Additionally, any LLM service (local or remote) that provides an
;; - Its async and fast, streams responses. ;; OpenAI-compatible API is supported.
;; - Interact with LLMs from anywhere in Emacs (any buffer, shell, minibuffer,
;; wherever)
;; - LLM responses are in Markdown or Org markup.
;; - Supports conversations and multiple independent sessions.
;; - Save chats as regular Markdown/Org/Text files and resume them later.
;; - You can go back and edit your previous prompts or LLM responses when
;; continuing a conversation. These will be fed back to the model.
;; ;;
;; Requirements for ChatGPT, Azure or Gemini: ;; Features:
;; - Its async and fast, streams responses.
;; - Interact with LLMs from anywhere in Emacs (any buffer, shell, minibuffer,
;; wherever)
;; - LLM responses are in Markdown or Org markup.
;; - Supports conversations and multiple independent sessions.
;; - Save chats as regular Markdown/Org/Text files and resume them later.
;; - You can go back and edit your previous prompts or LLM responses when
;; continuing a conversation. These will be fed back to the model.
;;
;; Requirements for ChatGPT, Azure, Gemini or Kagi:
;; ;;
;; - You need an appropriate API key. Set the variable `gptel-api-key' to the ;; - You need an appropriate API key. Set the variable `gptel-api-key' to the
;; key or to a function of no arguments that returns the key. (It tries to ;; key or to a function of no arguments that returns the key. (It tries to
@ -50,13 +54,17 @@
;; ;;
;; - For Azure: define a gptel-backend with `gptel-make-azure', which see. ;; - For Azure: define a gptel-backend with `gptel-make-azure', which see.
;; - For Gemini: define a gptel-backend with `gptel-make-gemini', which see. ;; - For Gemini: define a gptel-backend with `gptel-make-gemini', which see.
;; - For Kagi: define a gptel-backend with `gptel-make-kagi', which see
;; ;;
;; For local models using Ollama or GPT4All: ;; For local models using Ollama, Llama.cpp or GPT4All:
;; ;;
;; - The model has to be running on an accessible address (or localhost) ;; - The model has to be running on an accessible address (or localhost)
;; - Define a gptel-backend with `gptel-make-ollama' or `gptel-make-gpt4all', ;; - Define a gptel-backend with `gptel-make-ollama' or `gptel-make-gpt4all',
;; which see. ;; which see.
;; ;;
;; Consult the package README for examples and more help with configuring
;; backends.
;;
;; Usage: ;; Usage:
;; ;;
;; gptel can be used in any buffer or in a dedicated chat buffer. The ;; gptel can be used in any buffer or in a dedicated chat buffer. The