---
格式版本: 2
标题: "Puro-2B: Poor Lab's Qwen2-1.5B Trained on RTX 5090 within $5090"
原文链接: "https://arxiv.org/abs/2608.27370"
发布日期: "2026-08-27"
发布时间校准状态: "found"
发布时间需复核: "否"
发布时间来源: "rule:local:strict_original_body"
发布时间证据: "**\\[v1\\]** Thu, 27 Aug 2026 17:07:12 UTC (603 KB)"
发布时间校准原因: "规则确认唯一严格发布时间，来源 local:strict_original_body"
发布时间校准置信度: "high"
发布时间候选数量: 18
发布时间严格候选数量: 6
发布时间原页读取状态: "source template page reused from URL open"
发布时间未找到原因: ""
发布时间校准时间: "2026-08-30T15:09:35+08:00"
发布时间仲裁状态: "skipped"
发布时间仲裁尝试次数: 0
发布时间仲裁耗时毫秒: 0
发现时间: "2026-08-30T15:08:58+08:00"
入库时间: "2026-08-30T07:09:35.447Z"
来源平台: "arXiv 学术论文搜索"
搜索渠道: "source_template"
搜索词: "https://arxiv.org/search/?query=GPU&searchtype=all"
匹配关键词:
  - "GPU"
  - "performance"
相关厂家:
  []
相关专家:
  []
内容类型: "网页"
抓取工具: "Free Fetch + Defuddle"
清洗工具: "Defuddle Markdown + Defuddle/Readability 正文提取"
原始附件:
  []
AI优质: "否"
AI打分: 41
AI分档: "非优质"
AI质检状态: "不通过"
AI打分理由: "正文主线是使用消费级RTX 5090以FP8训练Puro-2B模型的低成本预训练配方，而非超节点或机架级AI基础设施。arXiv预印本属于可追溯的一手学术来源；固定知识库未发现同一研究，但未命中不能证明首次出现。新增事实包括最高1.4万亿token、训练成本低于6900美元及成本缩放定律估算，但技术创新集中于模型训练配方和成本效率，无机架拓扑、互连、供电、液冷、RAS或生产部署信号。命中“应用与模型效率”硬否决项，当前页面不适合进入超节点业务情报库。"
AI质检模型: "gpt-5.6-sol"
AI质检时间: "2026-08-30T15:09:43+08:00"
AI主题相关性: 2
AI来源权威性: 11
AI新颖性: 14
AI技术细节: 7
AI商业部署信号: 0
AI完整性: 7
AI评分提示词版本: "v17-精简生产版"
AI评分提示词SHA256: "48fb9777f386026761b4873eaff30807694fb11e9b352d7c69bf2dfde750cc7d"
AI评分知识库版本: "knowledge_base_v1-20260819+runtime.55"
AI评分知识库SHA256: "ca56afec1ff616b1fa4394cd1d08b1e2a6af47eccbe70c802c9f1fea6037e803"
AI评分知识库检索词: "[\"GPU\",\"Puro-2B\",\"Qwen2-1.5B\",\"RTX\",\"PDF\",\"arxiv.org/pdf/2608.27370\",\"HTML\",\"arxiv.org/html/2608.27370v1\",\"v1\",\"Llama-3.2-3B\",\"SmolLM3-3B\",\"FP8\"]"
AI评分知识库命中: "[{\"id\":\"historical-jun-010\",\"title\":\"爱建证券-电子行业专题报告：Vera Rubin量产提速，RTX Spark打开终端AI新空间-260608.pdf\",\"sourceType\":\"curated_item\",\"time\":\"2026-06\",\"matchedTerms\":[\"RTX\",\"PDF\"],\"rank\":-14.05375436470775},{\"id\":\"july-correct-0111\",\"title\":\"StrataCL: Fabric-Native Communication Library for Production Supernodes\",\"sourceType\":\"labeled_article\",\"time\":\"2026-07\",\"matchedTerms\":[\"PDF\",\"HTML\",\"v1\"],\"rank\":-11.333006229301258},{\"id\":\"runtime-569639751a0dbe7ef3ffccc5\",\"title\":\"[2608.17503] Predict Before Replay: Joint FEC and Flight Control for Reliable Scale-Up Links\",\"sourceType\":\"ai_excellent_article\",\"time\":\"2026-08-18\",\"matchedTerms\":[\"PDF\",\"HTML\",\"v1\"],\"rank\":-10.52985569710448},{\"id\":\"july-correct-0010\",\"title\":\"Schneider Electric and AMD release first Helios platform reference design to accelerate AI Factory deployment\",\"sourceType\":\"labeled_article\",\"time\":\"2026-07\",\"matchedTerms\":[\"GPU\",\"PDF\",\"v1\"],\"rank\":-8.011504167715488},{\"id\":\"historical-jun-005\",\"title\":\"輝達推出「NVIDIA DSX 平台」，提供創建 AI 工廠的完整方案\",\"sourceType\":\"curated_item\",\"time\":\"2026-06\",\"matchedTerms\":[\"GPU\",\"RTX\"],\"rank\":-7.35075785168517}]"
AI摘要: "该研究提出一套低成本开源预训练配方，在消费级RTX 5090上用FP8精度从零训练出Puro-2B模型，最优模型成本不到6900美元，性能接近Qwen2.5-1.5B。"
AI摘要模型: "ali-deepseek-v4-flash"
AI摘要时间: "2026-08-30T18:09:04.531Z"
采集批次: "2026年8月30日14点11分37秒"
采集批次ID: "20260830-141137-130"
去重键: "https://arxiv.org/abs/2608.27370"
---

## Computer Science > Computation and Language

## Title:Puro-2B: Poor Lab's Qwen2-1.5B Trained on RTX 5090 within $5090

[View PDF](https://arxiv.org/pdf/2608.27370) [HTML (experimental)](https://arxiv.org/html/2608.27370v1)

> Abstract:Language model pretraining has become almost synonymous with prohibitive cost, placing it out of reach for much of the academic and open-source communities. Although strong open-source efforts already exist, including open-weight models and open-source training recipes, a cost-efficient, hardware-accessible, and open-source pretraining recipe has long been missing. Even at a small scale, training Llama-3.2-3B costs over \\$1.5M, and reproducing SmolLM3-3B needs over \\$700K. In this report, we present an open pretraining recipe designed to lower this barrier. Using this recipe, we train a collection of Puro-2B models from scratch on up to 1.4 trillion tokens with FP8 precision on consumer-grade RTX 5090 GPUs. The models in the collection differ in token budgets and selected recipe variants. Our best model is trained at a compute cost of less than \\$6.9K and approaches Qwen2.5-1.5B performance under our evaluation protocol. This cost efficiency is enabled by a combination of approaches, including hardware selection, low-precision training, hyperball optimization, curriculum model averaging, and the data recipe. Beyond the recipe itself, we provide two additional results. First, across the Puro-2B collection, we derive a Puro Cost Scaling Law that relates training cost to average model performance; the fitted law suggests that about \\$4.4K, less than \\$5,090, is sufficient to reach the performance of Qwen2-1.5B. Second, as an end-to-end case study, we examine how pretraining data curricula shape downstream performance after post-training. Such controlled studies are enabled by having access to the full pretraining pipeline rather than model weights alone. We release the full training recipe for Puro-2B, including data, code, and model weights under Apache 2.0 at [this https URL](https://huggingface.co/collections/thu-pacman/puro-2b).

| Comments: |  |
| --- | --- |
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| ACM classes: | I.2 |
| Cite as: | [arXiv:2608.27370](https://arxiv.org/abs/2608.27370) \[cs.CL\] |
|  | (or [arXiv:2608.27370v1](https://arxiv.org/abs/2608.27370v1) \[cs.CL\] for this version) |
|  | [https://doi.org/10.48550/arXiv.2608.27370](https://doi.org/10.48550/arXiv.2608.27370) |

## Submission history

From: Shengqi Chen \[[view email](https://arxiv.org/show-email/1f0dd47e/2608.27370)\]  
**\[v1\]** Thu, 27 Aug 2026 17:07:12 UTC (603 KB)

[Which authors of this paper are endorsers?](https://arxiv.org/auth/show-endorsers/2608.27370) | Disable MathJax ([What is MathJax?](https://info.arxiv.org/help/mathjax.html))