Open 150 GB dataset pairs Central Asian languages with source code for LLM training

What happened

A solo developer has published a roughly 150 GB open dataset that combines text in Central Asian languages with source code, aiming at a corner of machine learning that large labs mostly skip. According to a post on dev.to, the collection covers Kyrgyz, Kazakh, Uzbek and Tajik alongside Russian and English, plus Python, C++, Rust and Go code, and is available on Hugging Face under a CC BY 4.0 license.

What the dataset contains

The author's breakdown of the uncompressed data:

• Russian: 60 GB
• Kyrgyz: 23 GB, which the post calls a milestone for Kyrgyz NLP
• Kazakh: 22 GB
• Uzbek: 7 GB
• Tajik: 7 GB
• English: 1.5 GB
• Source code: 20 GB, roughly 5 GB each for Python, C++, Rust and Go

The material ships as structured .txt and .l files. Worth noting before anyone plans a training run: the per-language figures add up to about 140 GB, somewhat short of the headline ~150 GB, and Russian alone is the largest single share. The four Central Asian languages together account for roughly 59 GB — still substantial for languages with scarce technical corpora, but anyone evaluating the set should weigh the actual composition rather than the label.

The packaging problem

Collecting the data was, by the author's account, only half the effort. Uploading 150 GB raw would have burned community bandwidth, so the archive was compressed with 7z at its maximum setting, a choice that pushed the developer's machine to its limits. Text compression at that scale demands large amounts of memory for dictionary buffers, and the post describes roughly ten hours of crashed compression runs, a repeatedly frozen operating system and persistent out-of-memory failures before the parameters finally cooperated. The finished archive lands at about 27.7 GB, a little under a fifth of the original size.

Intended uses

The author pitches three main applications:

• Fine-tuning code models such as StarCoder or CodeLlama so they handle technical specifications, documentation and comments written in Central Asian languages.
• Training translation models that can carry IT terminology across correctly, with threading concepts in Rust and microservices vocabulary in Go given as examples.
• Continual pre-training or domain adaptation, with the national-language corpora described as large enough to support training small 1B to 2.5B parameter models from scratch.

Questions the post leaves open

This is a first-person account rather than a documented release, and several details that matter for training decisions are not covered. There is no description of where the text was sourced, how it was filtered or deduplicated, or how the original licensing of scraped material was handled. The CC BY 4.0 terms apply to the release as the author has packaged it, which is not the same as clearing every underlying document. Notably, the compression troubleshooting occupies most of the write-up, while data quality work is summarized in a single passing mention. No independent evaluation or benchmark exists yet, and the dataset has not been reviewed by third parties.

Why it matters

Open training data remains heavily skewed toward English and a handful of high-resource languages, and the gap is widest for technical content: documentation, code comments and IT terminology in languages such as Kyrgyz or Tajik are close to absent from public corpora. That scarcity shows up downstream as weak model performance and unreliable translation. A community-built release of this size does not close the gap on its own, and its real value will depend on provenance and quality rather than raw gigabytes. But it hands researchers working on low-resource NLP something concrete to evaluate, fine-tune against and improve on, and the unusual pairing of regional languages with source code makes it more interesting than a plain text dump.