Local model

local Performs the AI functions assigned to it with a model on your own hardware, through any server that speaks the OpenAI API: Ollama, LM Studio, llama.cpp or vLLM. Nothing is sent to a provider.

Setup

1. Connect to your model server

   1. Enter the address of the OpenAI-compatible server you run, ending /v1. On this machine, by its default port:

      Server	Address
      Ollama	http://host.docker.internal:11434/v1
      LM Studio	http://host.docker.internal:1234/v1
      llama.cpp	http://host.docker.internal:9931/v1
      vLLM	http://host.docker.internal:8000/v1

   2. On another machine, use its address instead, such as http://gpu.example:11434/v1.
   3. On Linux, a server on this machine has to listen on an address other than 127.0.0.1, and hosted apps can reach it there too.
   4. Enter an API key only if the server requires one: --api-key for llama-server or vllm serve, Require Authentication in LM Studio.

   OllamaLM Studiollama.cppvLLM

   Settings base_url, api_key

2. Choose a model

   1. Enter the model's name as the server serves it:

      Server	Model name
      Ollama	As ollama list shows it, after ollama pull
      LM Studio	Its identifier, as lms ls shows it
      llama.cpp	Its --alias, or else the -m path
      vLLM	Its --served-model-name, or else the model given to vllm serve

   Settings model