Human-Agent-Society's Reef unifies agent feedback, learning and versioned delivery

The Human-Agent-Society project has published Reef, an open-source system meant to keep AI agents improving after they ship. According to a post on dev.to, Reef collapses four concerns that usually live in separate pipelines — serving agent requests, collecting feedback on those interactions, turning feedback into updates, and publishing versioned changes — into one piece of infrastructure that treats model weights and agent harnesses as equally manageable artifacts.

A four-stage loop

Reef breaks its learning cycle into four stages. Serve answers requests and records each interaction. Observe attaches feedback that arrives later to those stored records. Grow turns eligible records into candidate updates. Commit applies a configured selection policy and publishes only the versions that pass. The dev.to post notes that the repository maps these stages onto its service, storage, training, evaluation, artifact-history and delivery modules.

Two surfaces a recipe can update

A deployment recipe decides what actually changes. Weight-oriented recipes can train model parameters through Slime and SGLang. Harness-oriented recipes edit an agent's prompts, rules and skills, drawing on a model endpoint instead of a local training cluster.

For the weight path, Reef documents OpenAI- and Anthropic-compatible inference endpoints. Each response carries an interaction record ID that later reports can reference, optionally including a numeric score alongside textual or structured feedback. Once a recipe has gathered enough eligible feedback, the project says, it can run a training step and synchronize refreshed weights to the serving runtime without a restart of Reef.

The harness path needs no training GPUs at all. In the project's coding tutorial, a failed report can trigger a candidate skill update; Reef then compares that candidate against the incumbent harness on three coding tasks and publishes it only if it wins.

Different prerequisites, one gate

The two routes impose different requirements, per the dev.to write-up. Model-weight training demands a trainable model, feedback the chosen recipe can consume, and a supported GPU stack. Harness optimization demands a model endpoint, representative tasks and an evaluator — but no local training hardware. Both routes converge on the Commit stage, where the selection policy runs and the version-management and artifact-delivery components ship whatever is accepted. Remaining live during updates is listed as a built-in capability.

Why it matters

Reef's most consequential design choice, as the dev.to analysis frames it, is that generating an update and publishing it are separate acts. A finished training step or a harness edit does not automatically become the delivered version; candidate evaluation and selection sit in between. That makes the loop more controlled than one that applies every change it produces, but the control is only as strong as the evaluator and policy configured around it.

The trade-off left open is automation versus confidence. Reef can standardize interaction records, update jobs, version history and delivery, yet operators still have to supply representative tasks and an evaluator that decides what counts as evidence. A weak evaluation therefore becomes a weak gate: the infrastructure will enforce whatever decision it is handed.

For teams running agents in production, Reef offers a unified spine for the improvement loop, with hot-swappable weight updates as a practical bonus. Whether agents actually get better still depends on the quality of the feedback collected and the rigor of the checks standing between a candidate and release.