LLM-Surgeon checks whether a model fits your VRAM before you download it

A developer writing as Leonel59 has released LLM-Surgeon v2.0, a command-line tool that diagnoses whether an open-weight model will actually run on your machine before you download tens of gigabytes of weights. According to the announcement post on dev.to, the tool works on Windows, Linux and macOS, needs Node.js 18 or newer, installs with no external npm packages, and runs entirely offline.

The guessing game it targets

The author frames the problem as familiar to anyone running models through Ollama, LM Studio or llama.cpp: you pick a 30B mixture-of-experts or 14B dense model, wonder whether it fits in VRAM at 16K context, and only find out after downloading that the runtime quietly offloads layers into system RAM, dragging throughput from around 55 tokens per second down to 4. LLM-Surgeon tries to replace that trial and error with hardware telemetry and grouped-query-attention KV-cache calculations.

The post positions the tool as a combination of three existing projects. From CanIRun.ai it borrows the idea of section-based recommendations, while fixing what the author describes as a browser-sandbox limitation: a web app cannot see your real free VRAM, cannot spot background Windows processes consuming GPU memory, and cannot detect unified-memory reservations on integrated GPUs. From Artificial Analysis it takes benchmark data, and from llmfit the terminal ergonomics, but with a curated roster instead of a flat list of roughly 500 raw models.

What the scanner reports

On startup, the CLI queries NVML (the library behind nvidia-smi), checks for AVX-512 or AVX2 CPU vector extensions, maps dual-channel DDR4 or DDR5 memory topology, detects iGPU unified memory reservations, and lists the exact background processes the author says can quietly claim 1.5 GB or more of GPU memory.

The diagnostic output covers 28 state-of-the-art open-source models organized into seven curated sections. Each card shows a segmented VRAM bar splitting the footprint into weights, KV-cache and buffer, how memory demand degrades as context grows from 4K to 32K tokens, and the exact ollama run or llama-cli flags needed to launch the model.

Leaderboards crossed with your PC

The post says the tool also crosses the Artificial Analysis Progress Index, which ranks models in tiers from tiny (up to 4B parameters) to large (over 150B), against your physical GPU and memory bandwidth. The roster the author cites includes models such as Nanbeige 4.3-33B, Qwen 3.6 27B, K2 Horizon MoVA 36B-A4B, Gemma 4 31B, Kimi K3.1 Pro and DeepSeek V4.1 Flash. These names and rankings come from the author's own post and have not been independently verified.

A lab for feeling the numbers

An interactive lab lets you compare two models streaming side by side at their predicted physical cadence, so you can experience the difference between a fully GPU-resident setup the author benchmarks at 59.6 tokens per second and a hybrid configuration spilling into DDR5 at 24.6, before committing to a 20 GB download. A benchmark mode can also measure models already installed locally through Ollama or LM Studio in real time.

Zero dependencies as a security feature

The security design is a selling point of the announcement. Because the tool uses only native Node.js modules, the author argues it is immune to npm supply-chain attacks. The post further claims command-injection immunity through strict use of execFileSync with shell execution disabled, a loopback-only network policy that prevents external requests, and automatic redaction of OS usernames and home directory paths from the exported diagnostic report.

Installation follows the conventional route: clone the repository from GitHub, run npm link, and invoke llm-surgeon from any directory.

Why it matters

Local model users routinely burn bandwidth, disk space and evening hours on weights that cannot run acceptably on their hardware, and the accounting has only gotten harder as MoE architectures, quantization options and longer contexts shift the bottleneck between model weights and KV-cache. A pre-download diagnostic turns that gamble into a quick check, and doing it with hardware telemetry rather than browser-based estimates addresses a real gap. The zero-dependency stance also reflects growing unease about supply-chain risk in npm tooling. The caveats are equally real: this is a single-author side project, its performance figures and model roster reflect one person's curation, and anyone relying on it for purchasing or download decisions should treat the predictions as estimates rather than guarantees.