ROCmFix and InferBench Automate AMD GPU Setup and Local LLM Backend Benchmarks

AMD's local LLM friction points

A post on dev.to zeroes in on two obstacles that regularly trip up people running large language models on AMD GPUs with tools such as Ollama or LM Studio. The first is missing drivers or graphics architectures that go unrecognized, which forces users to set the HSA_OVERRIDE_GFX_VERSION environment variable by hand before ROCm-based runtimes will treat the card as supported. The second is plain uncertainty: with both Vulkan and ROCm/HIP backends available, there is no obvious way to tell which one delivers more tokens per second on a particular machine.

Rather than settling the question with a one-off comparison, the author points to two utilities hosted on GitHub, ROCmFix and InferBench, one for each half of the problem.

ROCmFix: automating the override

ROCmFix is a single-file Python utility that takes the guesswork out of the HSA_OVERRIDE_GFX_VERSION workaround. According to the dev.to post, it queries the GPU's PCI ID directly, reading the Windows Registry on Windows or calling lspci on Linux, so the correct override value is derived from the actual hardware instead of being copied out of a forum thread.

The tool can then write the variable either permanently or only for the current session, and it covers the major shells on both platforms: CMD and PowerShell on Windows, plus Bash, Zsh and Fish on Linux. A companion command, rocmfix doctor, inspects the installed HIP SDK and Vulkan components to help diagnose a broken setup. Running it requires nothing more than python rocmfix.py.

InferBench: making backend comparisons honest

The second utility, InferBench, automates speed testing of local LLM inference across backend engines. Its workflow is designed to produce numbers that are actually comparable between Vulkan and ROCm/HIP. It executes warm-up queries first, then forces VRAM unloads between runs so that caching and memory fragmentation cannot bias later measurements in favor of whichever engine happened to run first.

For each configuration it reports the median tokens per second and the Time-to-First-Token (TTFT), covering the two metrics that matter most in practice: throughput for long generations and latency for interactive use. Notably, the post does not declare a universal winner between Vulkan and HIP. The implied argument is that the answer depends on the specific GPU, driver stack and model, and that InferBench is how you find out on your own hardware.

Why it matters

Running local models on Radeon hardware still involves plumbing that users of more mature stacks rarely touch: architecture version numbers, environment overrides and backend-specific quirks circulate as folk knowledge in issue threads. That state of affairs is fragile, since a driver update or an unsupported card can silently break a working setup.

These two tools attack the problem from both ends. ROCmFix turns the most common workaround into a deterministic, inspectable procedure, while InferBench packages a careful benchmarking methodology, including warm-up, forced unloads, medians and TTFT, so an ordinary user can reproduce it. For practitioners choosing between Vulkan and ROCm/HIP, that converts a debate usually settled by anecdotes into a measurement that can be repeated after every driver or model update.