Open-source agent generates, verifies and benchmarks CUDA kernels in an autonomous loop

What the tool does

A project called the agentic CUDA kernel optimizer, published on GitHub by developer bertaye and surfaced on Hacker News's front page on September 25, 2026, applies an AI agent to one of the more specialized corners of systems programming: writing fast GPU kernels.

According to the repository's README, the tool accepts a plain description of a workload — the example given is single-precision GEMM with rectangular matrices — and produces GPU implementations through a repeating cycle of code generation, correctness checking, benchmarking and refinement.

The system has two main parts. A LangGraph-based agent workflow in Python proposes candidate changes, while a standalone C++ harness compiles kernels at runtime using NVRTC, launches them through the CUDA Driver API, and records their outputs. Python then compares results with NumPy and selects which candidate survives to the next round.

The agent can alter both the kernel source and the launch configuration for individual test cases. It can also query GPU properties and, depending on configuration, pull NVIDIA documentation for optimization guidance and read Nsight Compute profiler counters to inform its next experiment. Every attempt is logged, and the fastest implementation that passes validation is the one that gets kept.

The optimization loop

The README breaks the workflow into five steps. The agent first loads or generates the workload signature, input cases, a reference kernel and an initial kernel. It runs the reference and evaluates the starting implementation. From there it proposes a change, compiles it, compares outputs against the reference using NumPy, and measures kernel latency. Those results feed into the next attempt, with invalid candidates repaired while iterations remain in the budget. Finally, the run saves the fastest validated candidate, its execution history, and a timing heatmap.

Users can supply their own components through flags for the signature, reference kernel, initial kernel and input cases; anything omitted is inferred or generated. Runs can also resume from a previous session's saved files, so later optimizations build on an earlier best kernel rather than starting over.

How performance is measured

Every case must pass validation, or the candidate is out. Ranking is based on the geometric mean of latency across the performance cases, while small correctness-only cases do not influence the score. Timing defaults to 10 warmup launches followed by 100 measured launches using CUDA events. Compilation time and profiler replay timings are excluded from the ranking. The README advises keeping the GPU otherwise idle when comparing timings.

Requirements and output

The project was developed on Windows with an RTX 3060 Laptop GPU. It requires Python 3.12+, an NVIDIA GPU with a compatible CUDA Toolkit and driver, CMake 3.24+, a C++17 compiler and an OpenAI API key. The default model is gpt-5-mini at medium reasoning effort, with API usage billed to the user's account.

Each session writes to its own results directory containing kernel sources, requests, input and output data, model and tool responses, plus history and summary files. Successful runs export the best kernel source and heatmap visualizations. The README demonstrates the tool on a float32 GEMM workload, including a follow-up run that continues optimizing a previously generated kernel.

Stated limitations

The author is unusually direct about the tool's limits. It is experimental and intended for individual kernels. Passing the supplied test cases does not prove general correctness, and a reference kernel generated by the system is not an independent correctness oracle. Improvements are workload-dependent, and no comparison against cuBLAS or other vendor libraries is currently included.

There is also a practical security note: generated input scripts run locally as Python subprocesses without a sandbox, and generated CUDA kernels execute on the local GPU.

Why it matters

Kernel tuning has traditionally required deep human expertise in memory hierarchies, occupancy and hardware counters. This project shows a workable agentic pattern for that work: the model does not merely write code, it runs experiments against real hardware, reads profiler output and iterates within a fixed budget, with correctness checked mechanically rather than trusted. Whether such output can compete with vendor-tuned libraries is untested here, but the generate-verify-benchmark-refine loop is a template that generalizes to other performance-sensitive engineering tasks, and it is a concrete example of AI agents doing measurable, verifiable work at the systems level.