Ambient Context turns focused macOS windows into daily Markdown logs for LLMs

Ambient Context, an open-source macOS menu bar app that surfaced on Hacker News in a Show HN post, tackles "screen memory" without recording the screen. Instead of screenshots or video, it samples text from whichever window is focused — every few seconds, via the macOS accessibility tree — and appends the result to a plain Markdown file, one per day. Point an agent such as Claude Code at that folder and, according to the project's README, it can answer questions like what you worked on Tuesday, build up knowledge of your projects, or draft your standup update.

How it works

The app lives in the menu bar: an open eye means recording, a closed eye means paused, and a right-click reveals today's file, the capture folder and settings. Day files read as a timeline — each block carries a time range, the application name, the document title, and the document path or URL, so an agent can open the original source instead of trusting captured fragments. Duplicate lines are written only once per day and interface noise is filtered out, which the developer says keeps each file small enough to hand to an LLM whole. An AGENTS.md file in the capture folder documents the format for whatever reads it.

The default save location is ~/Ambient Context, placed outside ~/Documents on purpose so iCloud does not copy the record off the machine. Once initial setup is complete, recording resumes automatically with the app, the README says.

Privacy and redaction

The design leans hard on locality. The current build makes no network calls at all — no account, no server, no telemetry, no bundled model; the developer notes the signed release will add a single update check against GitHub. Capture is limited to the focused window: never background windows, other displays or minimised windows, and never while the screen is locked.

Redaction happens before anything is written. Password managers and private-browsing windows are excluded entirely, secure input fields are skipped at the accessibility level, and patterns matching credentials, API keys and card-like numbers are scrubbed before text reaches disk. The capture folder itself is excluded from capture, so the app cannot observe its own output. Everything it produces is plaintext the user owns and can move, grep or delete.

Building and running it

The project is early and unsigned: there is no notarised download yet because Apple Developer enrolment is still in progress, so for now you build it yourself from the dragthelake/ambient-context repository. That takes roughly two minutes with Node, Rust and Xcode Command Line Tools installed, using a Tauri build step. On first run the app requests the Accessibility permission, which is what allows it to read window text, and asks where to save. It requires macOS 14 or newer on Apple Silicon.

Known limitations

Coverage depends on how each application exposes its accessibility tree. According to the README, Chromium and Electron apps — Chrome, Slack, VS Code, Obsidian, Figma — only build that tree on demand, so the first seconds of capture come back sparse and fill in on later passes, and Chrome may show a slightly glitchy window-resize animation while capture is active. GPU-rendered terminals such as Kitty and Alacritty expose little or no text at all, while Terminal.app and iTerm2 work. The developer is asking testers to log which apps return rich, partial or empty captures in a census document kept in the repo.

Why it matters

Most approaches to giving AI assistants memory of your work lean on screenshots or screen recording, which produce large, unstructured data and obvious privacy exposure. Ambient Context inverts that: the log is small, greppable, human-readable Markdown that the user owns, with redaction applied at capture time rather than after the fact. For local agents that need context about what you did, a text-first record is a far more tractable input — compact enough to feed whole, structured enough to cite its sources.

The trade-off is fidelity. Because it relies on accessibility trees rather than pixels, anything that renders text outside them is invisible, and early capture in Electron apps runs thin. Whether this becomes a dependable memory layer or a partial one depends on the app-by-app census now being crowdsourced — but as a privacy-preserving pattern for agent context, plain local Markdown is a compelling baseline.