CH 16 · Local Deployment and Token Freedom

Word count~3,650 wordsTime~25 minPrereqCH 06 (Custom Provider)LevelReproducible

Chapter Goal

In all the previous chapters, you've been calling the DeepSeek official API — useful, but billed per token, and Agent sessions particularly eat tokens (every tool result has to be fed back into the context). And this isn't just a DeepSeek issue: as long as you go through a cloud API, whether official or third-party, you can't escape per-token billing.

This chapter gives you a completely different path: local deployment. Download an open-source model to your own computer and run it; the entire inference doesn't go through any third-party server — no token bill, true "Token freedom".

The method you've already learned in CH 06: dsh only recognizes "OpenAI-compatible endpoints", and the model is a configuration, not a binding. This chapter applies that capability to a real scenario — let dsh run on your own model.

How Local Deployment Differs from Cloud

First make clear "why local deployment is free". When you use the DeepSeek official API, prompts, tool results, and file content are all sent to its server, which uses its own GPU for inference, then bills by token. Every cloud API is like this: the compute belongs to someone else, and the bill follows the tokens.

Local deployment is the opposite: the model is downloaded to your computer, inference runs on your own GPU/CPU, no third-party server is involved. The compute is yours, the electricity is yours, so naturally there's no per-token bill — that's what "Token freedom" really means.

Dimension	Cloud API (e.g. DeepSeek official)	Local deployment (LM Studio running open-source model)
Billing	Per-token billing	Free (only electricity cost)
Network	Must connect to the internet	Can be fully offline
Data	Sent to the service provider's server	Stays on your machine
Capability	Flagship model, strongest	Depends on how big a model your GPU can run
Onboarding cost	Zero, register and you have a Key	Install software, download model (several GB)

Local deployment's hallmark in one line: free, offline, private, but capability is limited by your hardware — the stronger your GPU, the bigger the model you can run, the better the result.

Principle: Model Is Configuration

Why dsh can swap models at will is the model-routing plugin llm-pi-ai: it treats the "model provider" as a piece of YAML configuration; any endpoint exposing the OpenAI Chat Completions protocol (i.e. /v1/chat/completions) can be plugged in as a provider. Swap model = swap a piece of configuration, no need to change dsh itself.

So local deployment and connecting to the official API, the method is the same as in CH 06, just with a different endpoint address:

Direction	Base URL	Key	Typical model
DeepSeek official (cloud)	https://api.deepseek.com	Official Key	deepseek-v4-flash etc.
LM Studio local	http://localhost:1234/v1	Anything (local ignores)	Qwen3 8B etc.

Connect a Local Model: LM Studio Onboarding

Step 1: Install LM Studio, Download a Model

Local model runners are not just one: if you're familiar with Ollama, you can also use Ollama directly (command-line onboarding, the configuration method is identical to below). This chapter uses LM Studio uniformly for the demo — with a graphical interface, the friendliest for beginners.

LM Studio supports Windows / macOS / Linux; go to the LM Studio website to download the installer. After installation, don't try to guess which model to download, just leave that to dsh — it will check your computer's environment, then recommend for you. Send this in the Web UI's input box:

Check my computer's GPU and VRAM, estimate in Q4 quantization how large a local model can run comfortably; recommend 2~3 models suitable for local Agent (must support tool calling), give model names that can be searched directly in the LM Studio search box, each with a one-sentence reason. Only query and recommend, no other operations.

dsh will read out your GPU and list a few specific model names. Take the names to LM Studio and search.

Open LM Studio: bottom-left Settings → Explorer, in the search box search for the model name dsh recommended. Tagged with a green label (indicating compatible with your hardware, full GPU load) in the search results is what your machine can run, just click to download.

An unchanging reminder: for local models running an Agent, tool-calling capability is more important than benchmark scores — the model must be able to follow instructions to call tools, otherwise the Agent can't turn.

Step 2: Open the Local Server, Confirm the Endpoint Is Alive

After the model is downloaded, in LM Studio bottom-left Settings → Local Models → Local Model API (Local Server), turn the Local API server switch on. By default it listens on http://localhost:1234, and the OpenAI-compatible endpoint is at /v1 — the page will directly show the base URL http://localhost:1234/v1; when you see "running" the server is up.

Open in browser:

http://localhost:1234/v1/models

If you see a JSON listing your downloaded model, the endpoint is fine. This step also lets you see each model's exact ID (use what's returned here; e.g. the Gemma 4 E2B I downloaded shows as gemma-4-e2b-it-qat), which you'll need to fill in the config later.

Model Parameters in the Resource Library

Want to see which parameters the model runs with? Bottom-left Settings → Local Models → Resource Library, find your model, click the Settings button on the right side, and the "Model Default Settings" page will open:

It's split into three sections: Prompt, Context & Performance, Generation. Automatic Optimize Based on Hardware (RECOMMENDED) is on by default; context length, GPU offload etc. are all AUTO — LM Studio will tune them based on your computer. The demo in this chapter uses all defaults, not a single one needs changing; come back here later if you want to tune.

Step 3: Add a "Custom Provider" in dsh

Following method 2 of CH 06: Settings → Models → Add Custom Provider, fill in:

Field	What to fill
Provider ID	lm-studio-local (lowercase)
API address	http://localhost:1234/v1
API protocol	OpenAI Chat Completions compatible
Key	Fill anything (local doesn't validate, lm-studio is a fine placeholder)
Model	The full ID from /v1/models in the previous step

Effect and Expectation Management

• Benefits: completely offline, zero API fees, code and docs stay on your machine — for privacy-sensitive scenarios, the local model is the only choice.
• Reality: a local small model is for "running through the loop", not for heavy work. Agents particularly eat tool-calling and long context; small models are more likely to miss tool calls and plan poorly than flagship models. It's very suitable for testing during plugin development; for real work, switch back to a cloud model.

The Gemma 4 E2B deployed in this demo is a vision model (Google official: E2B has a vision encoder, supports image input) — but for it to actually receive images and do OCR in dsh, one more declaration is needed: dsh's custom provider treats as plain text by default; without the declaration images are rejected as invalid input (covered in CH 06). How to do it: in dsh's settings window click the top-right Open config file, find llm-pi-ai.providers.lm-studio-local.models and the corresponding model, add a line input: [text, image]:

llm-pi-ai:
  providers:
    lm-studio-local:
      apiKeyEnv: LM_STUDIO_API_KEY
      api: openai-completions
      baseURL: http://localhost:1234/v1
      models:
        - id: gemma-4-e2b-it-qat
          input: [text, image]

After saving, restart the session, send it an image and it can OCR.

Tool calling also works: in the demo I sent "create a txt file telling me who you are", and the model really did call the write tool — it errored and retried a few times, then finally successfully wrote identity.txt. This proves the local model can really run dsh's tool loop, not just chat; it's just not as stable as the flagship, with more errors.

Local or Cloud: How to Choose

Scenario	Pick
Daily work, want the strongest result	Cloud API (per-token, easy)
Simple daily tasks, e.g. the timed hot-topic briefing with aihot earlier	LM Studio local model (offline, free)

They are not mutually exclusive: dsh can have several providers mounted at the same time, and you can switch in a new session as you like. When cost-sensitive, route daily sessions to flash and leave the heavy work to pro — the simplest money-saving posture; the local model underpins testing and privacy scenarios.

Common Pitfalls

Pit	How to avoid
Base URL typed wrong	OpenAI-compatible endpoints usually end in /v1; first open GET {baseURL}/models in the browser to verify
Key environment variable not set	apiKeyEnv only references, doesn't create; confirm the environment in which dsh starts can actually read this variable
Model ID doesn't match	Local models must use the full ID returned by /v1/models (vendor/model-name), don't fill based on memory
Image error, says it doesn't support OCR	Custom provider's model is treated as plain text by default — even if the model itself supports OCR, you must add input: [text, image] in the provider config
Local model keeps missing tool calls	Not a config issue, it's the small model's capability boundary — go back to cloud models or switch to a bigger local model

What you learned in this chapter

• [ ] State "model is configuration": dsh only recognizes OpenAI-compatible endpoints, swap model = swap a piece of provider config
• [ ] State the essential difference between local deployment and cloud API: why it's free (compute is yours), offline, data stays on your machine
• [ ] Know how to use LM Studio to download a model, turn on the local server, and confirm localhost:1234/v1/models is alive
• [ ] Know how to add the lm-studio-local custom provider in dsh and successfully run a local session
• [ ] Know that vision models need input: [text, image], know that small local models are suitable for testing not for heavy work