Fireworks AI breach exposed unattributable Notion AI chats from under 1% of workspaces

What happened

Notion has disclosed that a security incident at Fireworks AI, one of its third-party AI vendors, exposed chat transcripts from its Notion AI assistant. According to a security notice published by Notion and surfaced on the front page of Hacker News, fewer than 1% of Notion workspaces may have been affected.

Per Notion's account, Fireworks first flagged potential incidents earlier this month, prompting Notion to sever all connections to the vendor as a precaution. Notion states that by November 3 it had disconnected its systems, deleted all data held in Fireworks, stopped sending any traffic, and halted all training runs.

On November 6, Fireworks told Notion that malicious actors may have accessed customer data, and on November 8 the vendor confirmed a new dataset. When Notion's security team analyzed it, they found it contained Notion AI chat transcripts that had been used for the Insights feature.

Notion stresses that the transcripts were "unattributable": the chats were not labeled with their author or originating workspace, did not include tool call results such as Notion page content or external data pulled in by Notion AI, and carried no information linking them back to a workspace, user, or company. The company says no other data from within workspaces was affected.

How Notion used Fireworks

Fireworks is one of the most widely used AI infrastructure platforms, powering tens of thousands of customers and hundreds of thousands of developers, according to Notion's description. Notion relied on it in two ways: running classification jobs behind the Insights feature, which helps customers understand their own AI usage, and serving as a general inference provider for open-weight models inside the product.

Both uses have now stopped. Notion has disabled Insights while it investigates the incident and no longer serves open-weight models through Fireworks.

The notice also reveals that Notion's Learning and Early Access Program, in which customers volunteer data to help test new AI technology, previously supplied information used to experiment with training task-specific models on Fireworks.

What affected customers should do

Notion says it has contacted the owners of affected workspaces directly. If neither you nor your workspace owner has heard from Notion, your workspace was not affected and no action is currently needed.

Enterprise workspace owners can download a CSV of their workspace's affected data, plus an annotated version with additional metadata, such as the user associated with each transcript, to support their own impact analysis. Notion notes these annotations were added after the fact and were not part of the exposed data. Workspace owners on other plans receive a page with a link they can share with affected members, letting each member retrieve a CSV of only their own affected chats. The review tools work in the desktop app or a web browser, but not on mobile.

Notion's core advice is to treat the exposed transcripts as compromised. If anyone on a team pasted passwords, API keys, or other credentials into Notion AI during the relevant timeframe, the company recommends rotating them immediately.

Why it matters

The incident is a reminder that AI features increasingly route customer conversations through third-party inference providers, so a breach at the vendor becomes a breach at every downstream product that depends on it. Notion's design choices — unattributable transcripts with no tool call results — limited the blast radius here, but users who paste secrets into AI chats have no such protection, which is why Notion is telling customers to rotate credentials.

The episode also highlights the awkward disclosure dynamics of vendor incidents: Notion's transparency arrived only after Fireworks confirmed a stolen dataset, and the details of how the attackers got in rest with the vendor rather than the affected product. For engineering teams, the practical takeaways are the ones Notion itself acted on: minimize what you send to inference providers, keep transcripts stripped of identifying metadata, and assume anything typed into an AI chat may eventually be read by someone else.