Atlassian's Rovo built working Jira automations in only 2 of 10 hands-on tests

Only two of ten drafts worked

A hands-on test published on dev.to fed ten automation requests to Atlassian's Create with Rovo on a Jira Premium test site on 13 November 2026. Just two of the generated rules did what was asked with no changes. Seven needed fixes — a wrong component, field, status or smart value, where a smart value is a placeholder such as {{issue.key}} that automation fills in at run time — and one was rejected by the editor and could never be switched on.

The testers also put numbers on the cost. According to the write-up, each message to Rovo moved their organisation's meter by about 10 credits, 140 credits across 14 messages, and they could find no Atlassian page stating whether generating a flow is billed in the first place. Rules that ran and did their job consumed 3 or 4 automation steps per run, while runs that stopped at a condition cost nothing.

Silent failures were the worst results

The author argues the failure mode matters more than the score. One draft, meant to close a parent item once all its sub-tasks were done, ran twice and never moved the parent — with no error anywhere. Another would have sent an empty email every Monday. A rule that executes without complaint but does nothing is far harder to catch than one the editor refuses to enable.

Rovo also ignored the test space's real configuration. Drafts referenced a "Closed" status the space never had, and one searched using the retired Epic Link field. Even in a draft that worked, Rovo looked up the current project lead and hard-coded that person's name into the rule, so the automation would keep assigning to the old lead after any personnel change.

There were genuine bright spots. Rovo picked a sensible trigger in nine of ten drafts, asked a useful clarifying question when handed a Confluence space key that did not exist, and listed its assumptions when the request was deliberately vague. The prompt dialog itself carries an "Uses AI. Verify results" notice, which the results bear out.

How the feature came back

The capability has a recent, bumpy history. According to Atlassian's tracker AUTO-2376, the original in-product Create with Rovo button was "intentionally deprecated" in May 2026 alongside the general release of Studio V2, the current version of Rovo Studio. On 4 June 2026 Scott Boisvert commented on JRACLOUD-98402 that the removal significantly hurt organisations that restrict Studio access to a small group; by 13 November the request had gathered 24 votes and 19 watchers.

Weekly Cloud release notes listed "Build automation rules with Rovo" as rolling out from 17 August to 28 September, and on 1 September Atlassian's Sophia Do announced on the Community forum that flows built with Rovo live in Rovo Studio but remain viewable from the product's automation flows list. Per the test, you need Jira Cloud on Standard, Premium or Enterprise with Rovo enabled for Jira plus access to Rovo Studio, and Atlassian says Rovo is unavailable for Jira in its Government environment.

Test conditions

The evaluation ran in a fresh company-managed Kanban space with four statuses, signed in as an org admin, with each draft inspected component by component and three drafts published and triggered with real work items. Planning took 97 to 129 seconds to appear, roughly 23 of which was the testers' own browser automation overhead, and building the flow took another 20 to 23 seconds. The hand-off from Jira to Rovo Studio failed twice with an "Invoke action failed because unknown error" message, though the testers were driving a headless browser and note it may not reproduce normally; generating directly inside Rovo Studio worked every time.

Two governance details are worth flagging. Drafts arrived set so that all Studio users could edit them, and Atlassian's documentation says every Studio user can create flows by default — relevant for anyone limiting who may build automations. Separately, Atlassian's access page says the button stays available as long as any Jira app on the site has Rovo enabled; the testers blocked Rovo for their Jira app for about three minutes and the button remained.

Note also the vocabulary shift: Atlassian now calls an automation rule a flow, a project a space and an issue a work item.

Why it matters

For buyers evaluating AI-assisted admin tooling, this is a useful calibration point. A 20 percent as-built success rate means AI-generated rules need the same expert review as human-written ones, and the silent failures — flows that run without error but do nothing, or send empty emails — argue strongly for testing in a sandbox before enabling anything in production. Credit metering adds a cost dimension to what used to be a free click in the editor, and the open question of whether generation itself is billed makes budgeting harder. Finally, the default that every Studio user can create and edit flows deserves a governance review before wider rollout.