Locally Uncensored 3.0.6 lets local AI models click and type, gated by layered consent prompts

Version 3.0.6 of Locally Uncensored, an open-source desktop app (AGPL-3.0) with more than 170,000 downloads, adds computer control to its agent: a local model can now move the mouse and press keys inside apps the user approves. According to the dev.to post announcing the release, the feature is a beta, and most of the development effort went into the permission questions the app asks before the model acts. Because everything runs locally, the screenshots the agent captures and the input it generates stay on the machine.

Consent comes in layers

Computer control is disabled by default and must be switched on in Settings under Agent Permissions. Enabling it is not a blanket approval: the app asks at the start of every new chat, the first time it touches each application, on every action in the default supervised mode, and on sensitive actions even in autonomous mode. Until the user clicks Allow on the first prompt, no screenshot is taken and nothing happens, and each approval covers only that chat — new conversations and app restarts ask again, with no setting that skips the check.

Sensitive means anything that sends, buys, deletes or changes permissions; those confirmations show the reason in red. Clicks on buttons, links or menu items that carry no name are treated the same way, because the app cannot tell what they would do.

No-go zones and a kill switch

Some areas are off limits in every mode: terminals, password managers, system sign-in and permission dialogs, security settings, the Run dialog and administrator windows. The app never types into password fields and refuses to paste into them, and a double click on a program, installer, script or shortcut always asks first.

Ctrl+Alt+Space pauses a session and Ctrl+Alt+Esc stops it (Control+Option on macOS). Autonomous sessions also pause when the user moves the mouse, then show what was done and offer to take over, continue or add an instruction. A screen task ends after at most 25 steps or 8 minutes, with time spent answering questions counted against the limit. While active on Windows, the app shrinks to a small overlay in the top-right corner with a purple glow along the screen edge, and its prompts accept mouse clicks only so the target app keeps keyboard focus.

Which model drives the screen

By default the agent uses the chat model already configured. On Windows the app reads the target window's list of buttons and fields; image-capable models get the screenshot plus that list, while text-only models work from the list alone — the path that makes small local text models viable at all. In the developer's comparison, two tasks run three times each, both approaches finished all six runs: 497 seconds in total with screenshots versus 381 with the list alone. Typing a Notepad note requested by voice, using a local 9B model on the list method, took 11 to 22 seconds from the end of the spoken command to the spoken result.

The catalog also lists a dedicated screen model, Holo 3.1 9B, marked beta and recommended from 16 GB of video memory. On Windows it finished a simple typing task in one of three runs and does not fit next to a 9B chat model on a 12 GB card — a weak showing the developer chose to report anyway.

Platform coverage and stated caveats

The feature was checked point by point on a single Windows 10 machine. Linux has not been run in the app yet and needs an X11 session; macOS support is present but not fully tested, and requires Accessibility and Screen Recording permissions plus an image-capable model. The button list and autonomous mode are Windows-only, so on Linux and macOS every click is handled as a sensitive action. There is no background mode on any platform; the agent works in the foreground with the real pointer.

The developer also lists known limits: small local models can misjudge when a task is finished, ending a session as stopped even if the work is done; text read from the screen is marked untrusted, since on-screen content could try to give the model instructions; sensitive-action detection depends on a fixed list of wordings; and eight corrections and ten security fixes made after the Windows check have run in automated tests only. The source code and a document describing every question and limit are published on GitHub.

Why it matters

Computer control is what turns a chat model into an agent that can act on a real desktop, and the computer-use agents that have drawn attention so far generally depend on cloud-hosted models seeing your screen. Locally Uncensored demonstrates the opposite pattern: fully local, off by default, and gated by layered consent that always confirms purchases, deletions and permission changes, with on-screen text treated as a potential prompt-injection vector. It is early — one machine checked, uneven platform support, a struggling dedicated screen model — but as a working reference implementation of privacy-first agent permissions, it marks a notable step for on-device agents.