FlameF0X/loss-prediction
Viewer • Updated • 602 • 44
Daniel Fox PRO
FlameF0X
AI & ML interests
Pre-training text generator.
(Brother, im 18)
Please don't try to contact me.
Recent Activity
new activity about 2 hours ago
FlameF0X/Tiny-T2I-Leaderboard:Update index.html reacted to FredyRivera-dev's post with 🚀 about 18 hours ago
We wrote a full technical guide on how to train a bilingual (ES/EN) LLM from scratch: TinyQwen.
Covers:
- Hybrid architecture based on Qwen3.5
- Pre-training with 15B tokens
- Cost benchmark between H200 and B200
- Post-training with SFT + LoRA
- Full code and data, open source
With ~$11 of compute on an H200 we ran an initial training run, enough to validate the full architecture and pipeline.
Blog post: https://aquiles-ai.vercel.app/blog/tinyqwen-from-scratch
Technical feedback welcome, especially from anyone looking to replicate the pipeline with more compute. updated a Space about 24 hours ago
FWKV/READMEOrganizations
