Nemotron 3.5 Lightning, NVIDIA's 30B hybrid MoE model, lands free on Vercel AI Gateway

Vercel has added NVIDIA's Nemotron 3.5 Lightning to its AI Gateway catalog and is offering it at no cost, served by NVIDIA under the model string nvidia/nemotron-3.5-lightning-free. According to Vercel's changelog, the model is aimed at developers building long-running agents, sub-agent pipelines, chatbots and retrieval-augmented generation (RAG) systems.

A sparse hybrid architecture for agent workloads

Nemotron 3.5 Lightning is an open-weight reasoning and chat model with 30 billion parameters in total, but only 3 billion active per token. That sparsity comes from a hybrid Mixture-of-Experts design that, as Vercel describes it, combines Mamba-2 layers with MoE layers and keeps only a small number of attention layers. Fewer active parameters per token generally mean cheaper, faster inference, which matters for agents that make many sequential model calls or carry very large working contexts.

The model ships with a 1 million token context window plus support for tool calling and structured outputs, the two capabilities most agent frameworks rely on for invoking functions and parsing deterministic JSON replies.

Vercel flags one privacy caveat: the provider may retain prompts submitted to the free endpoint for model improvement, so teams handling sensitive data should weigh that before routing production traffic through it.

How to access it

Developers can call the model through the AI SDK or through the Chat Completions, Responses and Messages APIs, which covers most existing client libraries. Coding agents connected to AI Gateway can use it as well; Vercel names Claude Code, Codex and Cursor as examples. Setup involves installing the latest Vercel CLI, running npx vercel ai-gateway setup, and then selecting the free Nemotron model inside the agent.

What AI Gateway contributes

AI Gateway is Vercel's unified API in front of a broad model catalog. Beyond routing requests, it offers usage and cost tracking, retries, provider failover, and performance tuning that Vercel claims delivers better uptime than going to a provider directly. It also includes per-key budgets, routing rules and custom reporting.

On pricing, Vercel states that the gateway mirrors provider pricing with no markup and takes no platform fee on inference, including for bring-your-own-key (BYOK) requests. Since the underlying provider price here is zero, inference on this endpoint costs nothing through the gateway.

Why it matters

Free access to a model with a one-million-token context, tool calling and structured outputs removes a real cost barrier for one of the fastest-growing areas of LLM development: agents and RAG. Prototyping an agent loop or evaluating long-document retrieval burns a lot of tokens, and a no-cost model with low active parameters makes that experimentation cheap.

The architecture is notable too. Hybrid designs built on Mamba-2 are still rare among widely served models, and having one exposed through a mainstream gateway gives developers an easy way to benchmark it against transformer-based alternatives on their own workloads.

The trade-off is the data caveat: prompt retention for model improvement makes this a better fit for experimentation and non-sensitive work than for proprietary data. As a distribution move, it also signals how gateways such as Vercel's are competing on catalog breadth and free tiers rather than inference margins.