Open-source prototype fine-tunes a 230M language model on Android from delayed user feedback

A model that keeps training after it ships

According to a write-up on dev.to, a team has released Online-SDFT, an open-source prototype that keeps a small language model learning on a phone after deployment. The system pairs LiquidAI's LFM2.5-230M, a 230-million-parameter model, with a rank-4 LoRA adapter and uses ONNX Runtime Training to update that adapter on the device. Once provisioning is complete, inference, interaction storage, replay and adapter updates all stay local; no server performs the update.

The testbed is Android notification routing: the model picks between showing a notification immediately, saving it for later or archiving it, and then learns from what the user actually does next.

Hindsight, not labels or rewards

The dev.to authors frame the core problem as learning from signals that arrive late, are private and are ambiguous. Supervised fine-tuning would require a correct action for every notification, but the phone never observes the ideal choice. Reinforcement learning needs a reward, yet opening a notification does not prove the timing was right, and ignoring one does not prove it was unimportant — the user may simply have been busy. The model also only sees the consequence of the action it took, never the counterfactual. What the device receives is hindsight: the original context plus the user's later behavior.

One model as both student and teacher

Online-SDFT handles this with self-distillation. At decision time the student — the base model with its LoRA adapter active — sees the notification, the time and local context. Later, the same base model with the adapter disabled acts as a teacher, reviewing the identical context plus the observed outcome and producing a soft distribution over the three actions. That distribution is distilled into the student, which has to decide without seeing outcomes. There is no separate teacher network.

Rather than converting outcomes into hard rules such as "dismissed means archive," the method uses soft targets conditioned on outcome reliability. Dependable outcomes strongly support one action, ambiguous ones spread probability across the actions that remain plausible, and uninformative outcomes trigger no update at all. The authors say this stops the training loop from inventing counterfactuals or reading every gesture as an explicit preference.

Two stabilizers for live learning

Two additions keep the online loop useful. Controlled exploration injects a small amount of randomization while the model is uncertain and tapers it as confidence grows, preventing the system from settling into self-confirming behavior. A bounded replay buffer retains a window of recent lessons, sampling across feedback categories while favoring newer examples and always including the newest one. Replay is used only for training and does not lengthen the serving prompt.

Preliminary numbers, with caveats

The evaluation covered six approaches on three paired synthetic notification streams, with 240 decisions per stream. The frozen base model scored 28.2% preference accuracy with cumulative regret of 164.7; retrieval reached 50.0% and 106.6; rejection fine-tuning reached 52.8% and 105.3. Online-SDFT came in at 70.3% accuracy and regret of 44.8, matching the hidden sampled preference on 506 of 720 decisions.

The comparison with rejection fine-tuning is instructive: that baseline used the same LoRA capacity but required verified hard targets, and accepted only 75 of 311 hindsight-teacher candidates. Online-SDFT could exploit graded signals too weak to justify a one-hot label. Replay also proved essential — removing it dropped accuracy to 39.6% and pushed regret up to 134.9.

The authors are explicit that the results are preliminary: only three synthetic streams were tested, and the selected configuration was tuned on those same streams rather than confirmed on an independent held-out benchmark.

Running on a physical phone

The repository ships with a separate Android project that runs the whole loop on real hardware. The frozen base model makes routing decisions, and once an outcome arrives, ONNX Runtime Training updates the adapter; checkpoints and replay state live in app-private storage and survive restarts. In one physical test, repeatedly dismissing a notification taught the model to keep it quiet, and requesting the notification again changed the learned behavior so the next one was shown.

The limitations are equally clear. It is an engineering prototype, not a production notification manager: the current graph is FP32 and targets high-memory ARM64 devices, export and initial provisioning still need a Linux host, and the team has not yet profiled latency, peak memory, battery draw or thermal throttling. Because Android notification-listener APIs operate after a notification is posted, the demo does post-time routing rather than guaranteed suppression before an alert appears.

Why it matters

Most on-device small language models are frozen the day they ship, which discards exactly the signals a personal assistant needs most: how users actually respond over time. Online-SDFT sketches a practical middle ground between supervised labels and RL rewards by treating delayed, ambiguous outcomes as hindsight and distilling them conservatively. The same structure — act, observe delayed consequences, update locally — extends to writing suggestions and other assistive features, and the code, Colab notebook and Android app are open for others to try. Whether gains measured on synthetic streams hold up for real users and real devices is the question that now needs answering.