Instructor redesigns ML course after AI agents complete every take-home assignment

An early warning, ignored

An instructor writing on The Last Software Engineer — an account that reached the front page of Hacker News — recalls a test from around 2021, before ChatGPT launched. Vincent Hellendoorn suggested pointing GPT-3 at the reading quizzes for the course, and the model produced answers good enough to pass the grading rubric without ever seeing the assigned paper. Nothing changed at the time. Five years later, the author says AI agents can now complete every assignment in the course, and most of its assessments have been rebuilt in response, even though the skills the course aims to teach have stayed almost the same.

What changed and what did not

The course is Machine Learning in Production, an upper-level class about building production systems around ML models with a heavy MLOps focus, enrolling 100 to 170 students. Its learning goals concern engineering tradeoffs, anticipating and mitigating risks, and teamwork, rather than writing code or operating specific tools — which, according to the author, is why they survived the redesign largely intact. Students are permitted to use AI anywhere in the course, in any form and without attribution, apart from written and oral exams. The instructor's position is that policing AI use is not realistic, and that learning responsible use of these tools is part of the education anyway.

Where the research and reality collide

The uncomfortable core of the account is that generative AI is eroding teaching practices that education research actually endorses. Frequent, low-stakes assignments with feedback are preferred over a few high-stakes exams, but the low-stakes formats are precisely the ones agents can complete, tilting grading weight toward exams. A resubmission safety net, meant to reward learning from mistakes, was being gamed by students who submitted machine-generated work and then fixed only the flagged issues on retry; resubmissions now carry a 10% penalty. In-class discussion questions were likewise being outsourced to models by some groups. The author weighs this against personal experience of German university courses graded entirely by one final exam, with failure and drop-out rates of 50 to 80%, and has chosen to keep low-stakes interactions despite the abuse, accepting that some students will pass through without deep learning.

The new assessment model

Anything graded that once required written reasoning at home is gone. The author still asks for reports describing a solution and pointing to code, and is content to receive AI-generated versions, since those documents exist for navigation rather than assessment. Reflection questions — the kind asking what was challenging or how teamwork could improve — became pointless, because any expectation that can be spelled out in a rubric can also be delegated to a model. Replacing them are 15-minute, pass/fail conversations with a TA after each assignment, worth 20% of that assignment's points, with retries that cost time but not points. TAs are encouraged to hold high standards, and the author says many students fail on their first attempt. Stanford's CS221 is reportedly running a controlled experiment on a similar format this semester.

Reading quizzes were abandoned rather than converted. Assigned readings were cut by half and detached from points, with their lessons folded into classroom discussion — an acknowledgment that most students will keep summarizing papers with a model regardless. Weekly labs built around tools like Kafka, Grafana, Docker and Weights and Biases are small and scaffolded by design, and therefore trivial for coding agents to automate; they are now verified through in-person, pass/fail check-ins where students show evidence of completion and answer follow-up questions.

The staffing bill and new failure modes

This model runs on people. The course operates at a 20:1 student-to-TA ratio with roughly 10 hours of work per TA per week, and the check-ins alone consume about 300 minutes per TA every two weeks — workable, the author says, while conceding the staffing situation is fortunate. A newer problem surfaced as well: students were observed using tools such as Cluely during live Zoom discussions to have answers fed to them, so future check-ins will likely be in-person only.

Why it matters

This is a concrete, published blueprint of what AI pressure does to a computing course: assessment of understanding migrates to supervised, conversational settings, while unsupervised work is either ungraded or accepted as AI-assisted. It documents a genuine tradeoff — integrity bought at the price of practices research says support learning — and shows that the real cost lands on staffing and student time rather than on the curriculum. Institutions without generous TA budgets, or courses whose goals are about writing code itself, will feel the same pressure far more sharply. The broader signal is blunt: take-home work no longer demonstrates individual capability, and anyone who grades it — in universities or hiring pipelines — now has to decide what it is actually for.