How to Use a Local Ollama Model in VS Code Insiders

Step 1. Install Ollama and Download a Model

Install Ollama and download the Qwen3.5 model. For example:

ollama pull qwen3.5:latest

Check that the model is available:

ollama list

Ollama provides an OpenAI-compatible API that can be used to connect a model to VS Code.

Step 2. Check API Accessibility

By default, Ollama exposes its API on port 11434.

If VS Code runs directly on Windows, the API endpoint is:

http://localhost:11434/v1/chat/completions

If you use VS Code inside a Dev Container, keep in mind that localhost inside the container refers to the container itself, not the Windows host.

In this case, use:

http://host.docker.internal:11434/v1/chat/completions

You can check whether Ollama is accessible from the Dev Container terminal by running:

curl http://host.docker.internal:11434/v1/models

The response should contain a list of models available in Ollama.

Step 3. Add the Model to VS Code Insiders

Open the configuration file:

%APPDATA%\Code - Insiders\User\chatLanguageModels.json

Add the following configuration:

[
  {
    "name": "Ollama",
    "vendor": "customendpoint",
    "apiType": "chat-completions",
    "models": [
      {
        "id": "qwen3.5:latest",
        "name": "Qwen 3.5",
        "url": "http://host.docker.internal:11434/v1/chat/completions",
        "toolCalling": true,
        "maxInputTokens": 28672,
        "maxOutputTokens": 4096
      }
    ]
  }
]

This example uses host.docker.internal because VS Code is working with a project inside a Dev Container. If VS Code and Ollama both run directly on Windows, replace the URL with http://localhost:11434/v1/chat/completions.

The maxInputTokens and maxOutputTokens settings define the token limits used by VS Code when working with the model. Adjust them according to the model’s context window and the available memory.

Note the toolCalling setting. It allows VS Code to use the tool-calling capabilities supported by the model, for example when working with files.

Step 4. Set the API Key

After saving the configuration, open the Command Palette by pressing Ctrl+Shift+P and run:

Chat: Manage Language Models

Find the Qwen 3.5 model you added and select Update API Key.

Enter any value, for example:

ollama

A local Ollama server normally does not require an API key for authentication. However, VS Code expects one for this configuration, so you can use an arbitrary value.

You don’t need to add the apiKey property to the JSON manually. VS Code will generate the required entry when you configure the key. It will look something like this:

"apiKey": "${input:chat.lm.secret.ollama}"

Step 5. Select the Model and Test It

Open Chat in VS Code Insiders and select Qwen 3.5.

Start by asking the model to explain some code:

Explain what this PHP function does.

Then test file creation:

Create hello.php with a PHP Hello World example.

If everything is configured correctly and VS Code provides the necessary tools, the model should initiate file creation through a tool call instead of simply printing JSON describing the operation in the chat.

Keep in mind that tool-calling support depends on the model. For example, in my tests, qwen3.5:latest returned structured tool_calls through Ollama’s OpenAI-compatible API, whereas qwen2.5-coder:7b returned the tool-call description as ordinary text in a similar test.

Step 6. Check GPU Usage

If you run Ollama on a computer with limited video memory, the model size and context window can significantly affect performance.

To check how Ollama has distributed the model between the CPU and GPU, run:

ollama ps

The PROCESSOR column shows the distribution of processing between the CPU and GPU.