Off Grid AI Desktop turns one Windows GPU PC into a shared local AI server

One PC becomes a team AI server

A tutorial published on dev.to details how a single Windows machine with an NVIDIA graphics card can serve a whole team's AI workloads without any cloud involvement. The free Off Grid AI Desktop app, version 0.0.55 or later, ships a gateway with a Team mode that lets several people send requests to one PC simultaneously, each with a personal API key. The guide frames the tool as an alternative to Ollama for teams whose software already speaks the OpenAI API.

According to dev.to, the requirements are modest: Windows 10 or 11 on an x64 processor (there is no ARM build), the app itself, a downloaded chat model such as Qwen 3.5 9B, and other computers on the same office network. No account or payment is needed, since the gateway and Team mode are part of the free application.

Concurrency scales with VRAM

The key engineering detail is how many simultaneous requests a single GPU can absorb. In Auto mode on CUDA, the gateway serves up to 4 concurrent requests on cards with less than 24 GB of VRAM, 8 on 24 GB cards such as an RTX 3090 or 4090, 16 at 48 GB and 32 at 96 GB. Without CUDA, running on Vulkan or the CPU, the ceiling is 4. The memory left over after the model loads sets the real limit, with the guide budgeting roughly 8,000 tokens per request.

Requests beyond the cap wait in line in the order they arrived. When the queue itself fills up, the gateway answers with HTTP 429 plus a Retry-After header, so clients back off and retry rather than hang.

Setup, keys and visibility

Team mode replaces the default single-user setting, which handles one request at a time at full speed. Setup involves opening the Gateway panel, noting that the default base URL (127.0.0.1 on port 7878) works only on the host PC, and substituting the machine's LAN IPv4 address, found with ipconfig in PowerShell, for everyone else. The gateway listens on port 7878 on every network interface, and the tutorial recommends allowing it through Windows Defender Firewall on Private networks only, so it stays closed on hotel or cafe Wi-Fi.

API keys are off by default, and the guide is blunt that they should be enabled before sharing the address, because otherwise anyone on the network can use the models. Keys support expiry windows from 7 days up to no expiry, live in an encrypted secret store tied to the Windows user account rather than a plain-text settings file, and can be revoked individually, so a lost laptop does not mean reissuing everyone's credentials. A settings screen logs every request with its model, timing and key name, and a gateway status endpoint reports how many requests are running, waiting or refused.

Headless operation

The app can run without its desktop window, launched from PowerShell with a server-only flag plus options for team mode and client count, or the equivalent environment variables. In Team mode the PC is kept awake while answers are streaming, and Windows may sleep a minute after the last request finishes. Beyond chat, the server exposes image understanding, text-to-image, speech-to-text, text-to-speech, embeddings and a model list, all reachable with the same OpenAI-style clients.

What it cannot do

The tutorial is candid about limits: sharing one GPU means each answer streams more slowly when several people use it at once, image generation handles one picture at a time even in Team mode, and there is no relay or cloud tunnel, so remote staff need a VPN into the office network. The app also does not override lid-close or power-button behaviour, which makes a desktop PC the safer choice for a shared server.

Why it matters

For small teams working with sensitive code or documents, this is a workable pattern for local AI without per-seat cloud bills or data leaving the building. An OpenAI-compatible endpoint means existing editors, scripts and agents can simply point at a LAN address, while per-person keys and a request log add a layer of governance that bare single-user setups lack. One caveat worth noting: this is a single vendor tutorial on dev.to, and the concurrency figures come from the developer's own material rather than independent benchmarks.