deniz.in

Markets

Weather

Loading weather

· via dev.to (home feed)

llama.cpp hits v1.0.0, adding API stability and semantic versioning to libllama and libggml

llama.cpp has released v1.0.0 of its core libraries, promising a stable API under semantic versioning, plus FP8 KV caches and built-in text-to-speech.

llama.cpp hits v1.0.0, adding API stability and semantic versioning to libllama and libggml

llama.cpp reaches 1.0

llama.cpp, one of the most widely used open-source engines for running language models locally, has shipped version 1.0.0 of its core libraries. According to a dev.to writeup published on 16 November 2026, the milestone applies to libllama and libggml and introduces semantic versioning, finally giving developers a stable API contract for the libraries that power local inference.

A stable API at last

The change addresses a long-standing pain point. Anyone linking llama.cpp's core libraries directly into an application has had to chase frequent API changes, which made the project risky as a dependency for anything serious. Under the scheme described in the dev.to post, breaking changes will trigger a major version bump, backward-compatible features and model updates will arrive as minor bumps, and bug fixes will ship as patch releases. That predictability is the point: downstream projects can now reason about upgrades instead of bracing for them.

Performance work under the hood

Beyond versioning, the release bundles performance features aimed at running advanced models on consumer hardware. The dev.to writeup highlights FP8 (E4M3) KV caches, which store the attention key-value cache in a lower-precision numeric format, and a host-memory Mixture-of-Experts expert cache with a configurable CPU share for MoE architectures.

Multimodal expansion

Multimodal support also gets a significant boost. llama-server now exposes an OpenAI-compatible /v1/audio/speech endpoint, so text-to-speech works directly from a local llama.cpp instance without a separate service. The release adds support for new text and vision models — the post names Kimi-K3.1-Pro and Gemma 4 streaming audio input — and consolidates multimodal handling under changes to the project's mtmd infrastructure. A dedicated llama-tts tool can run TTS models such as Qwen3-TTS-12Hz-1.7B-Base-GGUF, though the writeup notes some models may need a speaker reference audio file for the best voice quality.

Building and running it

The typical workflow still means building from source: a C++ compiler, cmake and git, plus a GGUF-format model, many of which are hosted on Hugging Face. The llama-cli executable can fetch models itself via a -hf flag, run interactive chat with a --chat-template option, or handle one-off prompts. On Apple Silicon, the post recommends enabling the Metal backend for GPU acceleration, with CUDA and Vulkan available for NVIDIA and other GPUs. Common pitfalls it lists include missing build tools, GPUs sitting idle because the binary was built without the right backend, and out-of-memory errors on large models — addressable by choosing a smaller quantization such as Q4_K_M or cutting the context size.

Caveats

Two are worth flagging. First, llama.cpp remains a command-line tool at heart; the writeup suggests Ollama or LM Studio for users who want a GUI and more abstraction out of the box, while calling llama.cpp the gold standard for deep integration and early access to new features. Second, the dev.to post itself states it was generated with AI assistance and asks readers to verify details against primary sources — reasonable caution for specifics like exact model names and build flags.

Why it matters

A 1.0 release is usually more about trust than features, and that is the case here. Semantic versioning turns llama.cpp from a moving target into something applications can safely depend on, which matters for the growing ecosystem of local, privacy-focused AI tools that link its libraries directly. Combined with efficiency features like FP8 KV caches and MoE caching that push more capable models onto ordinary hardware, and server-side multimodal endpoints like /v1/audio/speech, the release strengthens the argument that local inference can be a production foundation rather than a hobbyist workaround.

  • #llama-cpp
  • #local-ai
  • #llm
  • #open-source
  • #text-to-speech

Related posts