Frost: a 1400-line deep learning framework built on the Neve language, with ResNet-18 benchmarks

A framework and a language, released together

The developer behind the project, who publishes as NoSavedDATA, has released Frost, a deep learning framework built on the Neve programming language that came out earlier. According to the introductory post on dev.to, the framework expresses parallel dataloaders, GPU kernels and a high-level computing language in roughly 1400 lines of code, and it arrives alongside a first benchmark: ResNet-18, with the benchmark code published in a separate public repository so the results can be reproduced rather than taken on trust.

Three problems that motivated it

The post grounds the project in three frustrations with the current PyTorch-centred stack.

The first is legibility. After reading the Sophia optimizer paper and experimenting with an unofficial implementation, the author got poor results riddled with NaNs, and a later paper argued that this and other optimizers had overstated claims. The lesson drawn is that even optimizers are hard to read and extend in PyTorch, so auditing someone else's research costs hours of study and debugging with no guarantee of a clear answer.

The second is GPU code. Flash attention achieved roughly a 2x speedup, the author notes, but it was written in C++ and CUDA because high-level kernel frameworks were too immature for its author to rely on. Bolting C++ onto PyTorch, meanwhile, means setuptools and extension boilerplate before a single line of kernel code actually runs.

The third is parallelism. While examining the Efficient Zero reinforcement learning model around a Bachelor's thesis implementing BBF for Atari, the author found parallelism that PyTorch does not express, with the original implementation dropping into Cython so C functions could be called from Python. PyTorch's own data workers are written in C to sidestep the Global Interpreter Lock, and preprocessing such as byte-pair encoding is typically done in C, C++ or Rust. Faced with choosing between Python's productivity, C-family speed, Lua or Julia interoperability and yet other tools for concurrency, the author started writing a language in the idle hours between training jobs, which later matured into a Master's project.

Where Neve sits next to Julia, Mojo and Triton

The post compares Neve against three alternatives. Julia earns respect for dynamic typing that runs close to C++ speed, plus garbage collection and channels — but its CUDA kernels still expose raw primitives such as blockIdx().x, the cp_async intrinsic important for fast matrix multiplication must be added through interop, and the author objects to the syntactic noise of the end keyword.

Mojo gets credit for Python interoperability, byte-pair encoding benchmarks, flash-attention kernels and GPU portability through MAX. It loses points for benchmarking only BPE inference rather than training, lacking channels, and adopting Rust-style ownership. Its layouts and tiling, derived from CuTe and Cutlass, inspired the way Neve handles the same problems, though the author judges the resulting code heavy: comptime metaprogramming appears constantly and layout declarations are verbose.

Triton, finally, offers similar layouts and tiling while staying dynamically typed and avoiding comptime headaches.

Results claimed so far

Alongside the ResNet-18 benchmark, the author reports that Neve comes close to Python with SentencePiece in text processing and BPE training, and is competitive with NumPy and OpenBLAS in CPU matrix multiplication while written as high-level SIMD code. On the GPU side, a better programming interface is in progress, aimed at implementing flash attention. The post text itself presents no benchmark tables; the numbers live in the linked repositories, which is also where independent verification would start.

Why it matters

The Sophia anecdote is the real argument. If reproducing a ten-page paper means hours of study plus debugging and still leaves doubt about whether its claims hold, then tooling is a bottleneck for research, not just ideas. A deep learning stack compact enough to read end to end, in one language spanning dataloaders, preprocessing and kernels, lowers the cost of auditing, extending and reproducing experiments, and removes the multi-language glue that made the original work painful. Frost remains an early, single-maintainer project whose performance claims rest on its author's own benchmarks, so skepticism is fair. But as proof that a full training stack does not require Python, C++, Cython and CUDA all at once, it is worth a run.