Developer fine-tunes 459M GLiNER on $9 of Gemini labels to replace a paid API

What happened

A developer has published a hands-on account of replacing a paid Gemini API pipeline with a small fine-tuned model, at a total bill of roughly $11.50. According to the write-up by Peter Vijeh, which surfaced on Hacker News, $9 paid for one-off labeling of 4,290 Reddit comments by Gemini 3.1 Pro, and about $2.50 covered 24 minutes of training on a Tesla T4. The result, a fine-tuned GLiNER large v2.5 model with 459 million parameters, scored 0.83 F1 when graded against Gemini's own outputs.

The problem

Vijeh scrapes Reddit threads where enthusiasts argue about high-end chef's knives and extracts every brand, model and steel mentioned — a task called named-entity recognition. Gemini handled it well but billed per comment for as long as the scraper ran, so costs grew with how much people posted, and the only way to cap them was to skip comments. Running the open-source GLiNER model zero-shot cost nothing but managed only an estimated 0.65 F1 against Gemini's answers. The project asked whether one paid labeling pass could teach the small model to close that gap.

The approach

Gemini labeled all 4,290 comments through OpenRouter at temperature 0 in 25 minutes, at $0.0021 per comment. At that price, Vijeh calculates the trained model pays for itself at roughly comment 4,291, assuming later comments are similar in length and inference runs on hardware he already owns.

The prompt decision that mattered most, he writes, was asking the model for exact substrings rather than character offsets, because Gemini counts characters badly and returns spans off by two or three positions. His code computes the offsets and drops any entity whose string cannot be found. A custom regex tokenization keeps product names like VG-10, CPM-154 and 1.4116 from being split apart, and spans that miss a token boundary are discarded rather than guessed.

The final dataset held 2,250 examples — 1,575 positive, 675 negative — split into 2,029 training and 225 validation comments, with 3,907 entity spans across brand, model and material classes. About 30 percent were deliberate negatives: comments containing known false-positive triggers such as gyuto, carbon, handle and patina but no actual product, labeled as empty. The validation set was locked before any tuning began.

Five failures out of ten

Half the training runs produced no usable model, and none of the failures were about modeling. Three were configuration issues with the Hugging Face Trainer: a default step count overriding the requested epochs, a checkpoint-loading flag that requires an explicit evaluation strategy, and saved state-dict keys missing a prefix GLiNER's loader expects. One run failed because negative examples lacked a label list.

The expensive failures were two runs sunk by a tensor called words_mask. It sits beside the attention mask, shares its shape, and Vijeh filled it in like one. Training ran to completion with loss drifting from about 130 to about 70, checkpoints saving on schedule, evaluation F1 near zero, and no crash or warning. Reading GLiNER's training loop revealed the tensor is not a mask at all but an incremental word index the span-scoring head uses to pool sub-tokens back into words; filling it with ones told the model the entire comment was a single word. With the index corrected, the next run learned on the first try.

Results and caveats

The 209M model reached 0.800 F1; the 459M model, which fits a T4 only with gradient accumulation, reached 0.83. Switching from one global threshold to per-class thresholds raised material recall from 0.787 to 0.911, because steel names like MagnaCut and S35VN score lower confidence than brands. Ten times more adversarial negatives actually hurt, dropping F1 to 0.799. Vijeh also notes an honest limitation: nobody hand-checked Gemini's labels, so the model is graded against Gemini, not against ground truth — it is marked right for copying the teacher's mistakes.

Why it matters

The write-up is a concrete demonstration of how cheap bespoke NLP has become: a one-time LLM labeling bill and a rented GPU for under half an hour replaced a recurring API cost, and the whole system now runs locally on consumer hardware. Just as instructive is where the effort went — five of ten failures were configuration and tensor semantics, with the worst one failing silently, which suggests the scarce skill in small-model fine-tuning is debugging plumbing, not designing models. It is also a reminder that distillation inherits the teacher's blind spots, since a model graded against another model can only ever match its mistakes, not correct them.