dev.to Walkthrough Traces One vLLM Request from generate() to the GPU

A walkthrough published on dev.to traces a single offline inference request through vLLM's V1 engine, from the public LLM.generate() call across an inter-process boundary to scheduling, GPU model execution and paged KV-cache access. Adapted by the author from a Chinese-language article originally posted on Zhihu, it is the first installment of a three-part series on vLLM internals, with later parts set to cover CUDA kernels with paged attention and FlashAttention from PyTorch to Triton. According to the dev.to article, all source references were verified against vLLM 0.22.0, checked on September 3, 2026, though the author warns that filenames and call boundaries will move as the project evolves.

One request, two processes

The structural detail that shapes the entire path, the author argues, is that vLLM 0.22 separates the caller-facing engine from EngineCore with a process boundary. EngineCore runs continuously in a child process, and the caller does not drive GPU execution one token at a time. A submitted request travels from LLM.generate() through LLMEngine.add_request() and an EngineCoreClient into an IPC input queue, where it is registered with the scheduler.

On the other side of the boundary, EngineCore's run_busy_loop() repeatedly performs EngineCore.step(), which the walkthrough decomposes into three stages. First, scheduler.schedule() spends token and KV-cache budgets to produce a SchedulerOutput. Second, model_executor.execute_model() runs the GPU forward pass and sampling. Third, scheduler.update_from_output() updates request state, releases the KV blocks belonging to finished requests, and pushes results to an IPC output queue for the main process to pick up.

step() consumes results, it does not compute them

A central clarification in the article concerns LLMEngine.step(). Despite the name, it neither submits requests nor directly drives model execution. Its implementation fetches output that EngineCore has already produced via get_output(), decodes tokens and evaluates stopping conditions, aborts requests terminated by stop strings, and records statistics before returning user-facing RequestOutput objects. Submission happened earlier in add_request(), and the actual computation advances independently inside the child process. The author notes this description assumes the default multiprocess V1 configuration, which can be disabled for debugging with VLLM_ENABLE_V1_MULTIPROCESSING=0.

LLM is a facade, generate() is a loop

In entrypoints/llm.py, the LLM class is a thin wrapper: it packs model, dtype and related configuration into EngineArgs and delegates engine construction to the factory method LLMEngine.from_engine_args(). Stripped of detail, the internals of generate() register each prompt with add_request() and then loop over step() while any request remains unfinished, collecting finished results before returning them. Unlike a hand-written loop over one sequence, the author points out, this tracks a set of requests whose lengths and completion times differ.

That loop is also where continuous batching becomes concrete. Because the update stage removes completed requests and frees their resources while the next call to schedule() can admit waiting work, the active request set changes from step to step. The GPU therefore does not have to wait for one fixed batch to finish before another request joins.

Reading the source beyond the analogy

The stated motivation for going to the source is that most introductions stop at the analogy that PagedAttention manages the KV cache the way virtual memory manages pages, without explaining how a request is admitted, how variable-length requests become a flat token batch, or what the page table looks like at the kernel boundary. The walkthrough accordingly expands the engine loop into six parts — entry, scheduling, input preparation, model execution, PagedAttention's write and read paths, and output handling — keeping only the code needed to connect one boundary to the next. It assumes the reader already knows prefill, decode, KV caching and autoregressive generation, and focuses on how those concepts appear in the code.

Why it matters

vLLM is widely used for running and serving LLM inference, and its headline performance rests on mechanisms like continuous batching, token budgets, paged KV allocation and the separation of scheduling from execution that are usually described only through analogies. A source-referenced walkthrough turns those analogies into checkable code paths, which matters to engineers debugging throughput, tuning batch behavior, or porting vLLM's ideas into their own stacks. Pinning the explanation to vLLM 0.22.0 on a specific date also gives teams a fixed reference point in a codebase the author himself expects to keep moving.