Kestra 2.0 rewrites its engine, splits control and data planes, stays Apache 2.0

Kestra has released version 2.0 of its open-source orchestration platform, a release that replaces most of the execution engine and reorganises the system into a control plane and a data plane. According to the announcement post on dev.to, the engine, UI and plugins remain under Apache 2.0, and 2.0 ships as a long-term support release.

The post presents 2.0 as the answer to criticism of Kestra's original 2022 design, which demanded a Kafka cluster and an Elasticsearch cluster just to schedule work — a setup readers of the first dev.to article called far too heavy to attempt. Four months later the project added a JDBC backend so a single Postgres or MySQL could serve as both queue and state store, and version 1.0 followed in September 2025.

Workers no longer need database access

In the 1.x architecture, every worker connected directly to the central database. That single constraint dictated where tasks were allowed to run: when a security team refused to open a route from one network to a database in another, teams either deployed a full Kestra instance per site or gave up on orchestration altogether. The author says the second outcome was common.

Version 2.0 separates the two concerns. The executor, scheduler, webserver, indexer and a new worker controller form the control plane and never execute user code; workers are the data plane. Each worker opens one persistent gRPC stream to the worker controller, always initiated by the worker, and jobs, results, logs and metrics travel over that same channel. The stream can be encrypted with TLS, and workers can be required to present a client certificate or a JWT before any job is dispatched.

The practical result, per the post, is that a worker holds no database credentials and accepts no inbound connections, so it can run in another cloud, another region, on-premises next to the data, or inside a network that only permits outbound traffic.

One engine instead of two

The rewrite also pays down internal debt. In earlier versions the queue and the repository shipped as a fixed pair — JDBC for both, or Kafka alongside Elasticsearch — which meant two engine implementations, every bug fixed twice, and behavioural differences between the two paths. Kestra 2.0 has one executor, one scheduler and one worker, with the queue and repository chosen independently. The Kafka Streams engine has been deleted entirely.

Changes at the flow level

Several workflow-level features arrive with the rewrite:

• Loop replaces the ForEach and ForEachItem constructs, with each iteration running as its own sub-execution so a runaway loop can no longer take down the executor — something the author admits happened before.
• Trigger conditions collapse into a single "when" expression, in the manner of CI configuration.
• Quotas can cap executions per flow or namespace, so one team's mistake does not become everyone's incident.
• Any flow can be exposed as an MCP tool. An agent calling it triggers a normal execution with the same permissions as a person clicking Run, tagged system.from: mcp, and irreversible steps can be held back for human approval.
• AI is optional: the AI Agent task can point at Ollama on local hardware, and setting kestra.ai.enabled: false removes every AI endpoint from the server.
• The UI gains a canvas editor and drafts, with canvas and YAML kept in sync; the YAML in a Git repository stays the underlying source of truth, and user code imports nothing from Kestra.

Licensing boundary

The post is explicit about where the open-core line sits: RBAC, SSO, audit logs, multi-tenancy, worker groups, Policies, Cases and the cloud VM task runners are paid, while everything needed to run flows in production is not. The author notes that a major version is typically where projects change their licence, states Kestra did not, and says there are no plans to. More than four billion executions have run on Kestra since the first release, most of them on the open-source edition on hardware the project never sees.

Why it matters

Orchestration tools have often assumed that compute can reach a central database, an assumption that collapses in regulated or segmented networks. Kestra's new model — workers dial out over gRPC with no stored credentials — removes a constraint that, by the project's own account, pushed teams back to cron, and it makes multi-cloud and on-prem hybrid deployments far more realistic. The engine consolidation simplifies what self-hosters have to reason about, and the MCP trigger turns existing flows into callable tools for AI agents. And as the post itself concedes, major releases are where open-source projects usually relicense; staying Apache 2.0 here, presented as a deliberate decision, matters to anyone choosing an orchestrator on a multi-year horizon.