ACC: Compiling Agent Trajectories for Long-Context Training
Paper โข 2605.21850 โข Published โข 58
ZeroGPU Explorers
community
AI & ML interests
None defined yet.
Recent Activity
View all activity
Lin-Chenย
authored a
paper 5 days ago
Lin-Chenย
authored 3
papers 15 days ago
VimRAG: Navigating Massive Visual Context in Retrieval-Augmented Generation via Multimodal Memory Graph
Paper โข 2602.12735 โข Published โข 8
Flow-OPD: On-Policy Distillation for Flow Matching Models
Paper โข 2605.08063 โข Published โข 97
SCOPE: Structured Decomposition and Conditional Skill Orchestration for Complex Image Generation
Paper โข 2605.08043 โข Published โข 10
seyedhamidrezaย
authored a
paper 15 days ago
seyedhamidrezaย
submitted a
paper to Daily Papers 15 days ago
Ritvik19ย
authored a
paper 17 days ago
CongWei1230ย
authored a
paper 17 days ago
limingcvย
authored 2
papers 24 days ago
limingcvย
submitted 2
papers to Daily Papers 26 days ago
Post
1981
Big update to llm-datasets, my curated list of datasets and tools for post-training LLMs.
> Added many new datasets
> New "thinking" column
> Refreshed recommended tools.
Thanks to everyone who told me they used it for their research at ICLR, you motivated this update!
> Added many new datasets
> New "thinking" column
> Refreshed recommended tools.
Thanks to everyone who told me they used it for their research at ICLR, you motivated this update!
- 2 replies
wangfuyunย
authored a
paper about 1 month ago
Post
3333
A small model that struggled against a random opponent now beats GPT-5-mini at tic-tac-toe
I took LiquidAI/LFM2-2.6B and trained it through play.
๐งโ๐ณ Here's how:
1๏ธโฃ Build a solid RL env with Verifiers (Prime Intellect)
2๏ธโฃ Generate synthetic data: <200 games sampled from GPT-5-mini playing in the env
3๏ธโฃ SFT warm-up to teach format
4๏ธโฃ Group-based RL (CISPO) against opponents making 20-70% random moves
5๏ธโฃ RL again with stronger opponents (0-25% random moves) + 1.25 temperature to push exploration and shake off suboptimal strategies
Done! Beats GPT-5-mini ๐
---
๐ฎ Play against the model: anakin87/LFM2-2.6B-mr-tictactoe
๐ค Model: anakin87/LFM2-2.6B-mr-tictactoe
๐ Walkthrough/course: https://github.com/anakin87/llm-rl-environments-lil-course
๐ค Dataset and checkpoints: https://huggingface.co/collections/anakin87/lfm2-26b-mr-tic-tac-toe
I took LiquidAI/LFM2-2.6B and trained it through play.
๐งโ๐ณ Here's how:
1๏ธโฃ Build a solid RL env with Verifiers (Prime Intellect)
2๏ธโฃ Generate synthetic data: <200 games sampled from GPT-5-mini playing in the env
3๏ธโฃ SFT warm-up to teach format
4๏ธโฃ Group-based RL (CISPO) against opponents making 20-70% random moves
5๏ธโฃ RL again with stronger opponents (0-25% random moves) + 1.25 temperature to push exploration and shake off suboptimal strategies
Done! Beats GPT-5-mini ๐
---
๐ฎ Play against the model: anakin87/LFM2-2.6B-mr-tictactoe
๐ค Model: anakin87/LFM2-2.6B-mr-tictactoe
๐ Walkthrough/course: https://github.com/anakin87/llm-rl-environments-lil-course
๐ค Dataset and checkpoints: https://huggingface.co/collections/anakin87/lfm2-26b-mr-tic-tac-toe
Post
106
Local Gemma 4 agent ๐๐ต๏ธ๐บ๏ธ
drop in a mysterious map, get the location, live weather, and top spots to visit
I've been exploring what google/gemma-4-E4B-it can do in a local agentic setup and put together a ๐ ๐ฃ๐ค๐ฉ๐๐๐ค๐ค๐ with Gemma + Haystack AI Framework covering 4 demos.
๐ https://t.ly/04Ty5
Another interesting one is the ๐๐ถ๐๐๐๐ฏ ๐๐ด๐ฒ๐ป๐.
I initially tried to load all tools from the GitHub MCP server, quickly filling the context available on Colab -> unusable, forgetful agent โ
Then I used the ๐ฆ๐ฒ๐ฎ๐ฟ๐ฐ๐ต๐ฎ๐ฏ๐น๐ฒ ๐ง๐ผ๐ผ๐น๐๐ฒ๐ ๐ ๐งฐ
It dynamically discovers the right tools from the GitHub MCP server on the fly, loading only what it actually needs for the task at hand, keeping context lean.
Now it actually works.
The notebook also contains
๐ Multimodal weather agent: the mystery map demo above
๐ Visual Question Answering from a paper
๐ RAG on Rock music
drop in a mysterious map, get the location, live weather, and top spots to visit
I've been exploring what google/gemma-4-E4B-it can do in a local agentic setup and put together a ๐ ๐ฃ๐ค๐ฉ๐๐๐ค๐ค๐ with Gemma + Haystack AI Framework covering 4 demos.
๐ https://t.ly/04Ty5
Another interesting one is the ๐๐ถ๐๐๐๐ฏ ๐๐ด๐ฒ๐ป๐.
I initially tried to load all tools from the GitHub MCP server, quickly filling the context available on Colab -> unusable, forgetful agent โ
Then I used the ๐ฆ๐ฒ๐ฎ๐ฟ๐ฐ๐ต๐ฎ๐ฏ๐น๐ฒ ๐ง๐ผ๐ผ๐น๐๐ฒ๐ ๐ ๐งฐ
It dynamically discovers the right tools from the GitHub MCP server on the fly, loading only what it actually needs for the task at hand, keeping context lean.
Now it actually works.
The notebook also contains
๐ Multimodal weather agent: the mystery map demo above
๐ Visual Question Answering from a paper
๐ RAG on Rock music
Post
10406
How LLM training with RL Environments works?
It all starts with ๐ฅ๐ฒ๐ถ๐ป๐ณ๐ผ๐ฟ๐ฐ๐ฒ๐บ๐ฒ๐ป๐ ๐๐ฒ๐ฎ๐ฟ๐ป๐ถ๐ป๐ด ๐๐ถ๐๐ต ๐ฉ๐ฒ๐ฟ๐ถ๐ณ๐ถ๐ฎ๐ฏ๐น๐ฒ ๐ฅ๐ฒ๐๐ฎ๐ฟ๐ฑ๐
- question asked
- model generates reasoning + answer
- answer checked against ground truth
- reward drives RL training
In this setup, the environment is simple: fixed questions and answers, rollout logic, reward(s)
Consider a more complex tic-tac-toe env โโญ
It adds:
- dynamic game generation/handling
- tunable opponent skill
- multi-turn interactions
(envs can also include tools)
---
What happens at training?
We use ๐๐ฟ๐ผ๐๐ฝ ๐ฅ๐ฒ๐น๐ฎ๐๐ถ๐๐ฒ ๐ฃ๐ผ๐น๐ถ๐ฐ๐ ๐ข๐ฝ๐๐ถ๐บ๐ถ๐๐ฎ๐๐ถ๐ผ๐ป with a tic-tac-toe env
No critic model needed, the group is the baseline
Simpler than PPO
1๏ธโฃ Rollout generation: from the same board, model plays N games via sampling
2๏ธโฃ Each game scored with deterministic rewards (win, format, ...)
3๏ธโฃ Mean score computed across the group
4๏ธโฃ Each rollout's advantage = its score minus the group mean
5๏ธโฃ Model updated to favor trajectories above baseline
๐ Repeat
For a deep dive, check out
๐ฑ https://github.com/anakin87/llm-rl-environments-lil-course
a free hands-on course on RL environments for LLMs
It all starts with ๐ฅ๐ฒ๐ถ๐ป๐ณ๐ผ๐ฟ๐ฐ๐ฒ๐บ๐ฒ๐ป๐ ๐๐ฒ๐ฎ๐ฟ๐ป๐ถ๐ป๐ด ๐๐ถ๐๐ต ๐ฉ๐ฒ๐ฟ๐ถ๐ณ๐ถ๐ฎ๐ฏ๐น๐ฒ ๐ฅ๐ฒ๐๐ฎ๐ฟ๐ฑ๐
- question asked
- model generates reasoning + answer
- answer checked against ground truth
- reward drives RL training
In this setup, the environment is simple: fixed questions and answers, rollout logic, reward(s)
Consider a more complex tic-tac-toe env โโญ
It adds:
- dynamic game generation/handling
- tunable opponent skill
- multi-turn interactions
(envs can also include tools)
---
What happens at training?
We use ๐๐ฟ๐ผ๐๐ฝ ๐ฅ๐ฒ๐น๐ฎ๐๐ถ๐๐ฒ ๐ฃ๐ผ๐น๐ถ๐ฐ๐ ๐ข๐ฝ๐๐ถ๐บ๐ถ๐๐ฎ๐๐ถ๐ผ๐ป with a tic-tac-toe env
No critic model needed, the group is the baseline
Simpler than PPO
1๏ธโฃ Rollout generation: from the same board, model plays N games via sampling
2๏ธโฃ Each game scored with deterministic rewards (win, format, ...)
3๏ธโฃ Mean score computed across the group
4๏ธโฃ Each rollout's advantage = its score minus the group mean
5๏ธโฃ Model updated to favor trajectories above baseline
๐ Repeat
For a deep dive, check out
๐ฑ https://github.com/anakin87/llm-rl-environments-lil-course
a free hands-on course on RL environments for LLMs
- 2 replies
Why doesn't anyone host llms in zerogpu spaces?
8
#172 opened about 1 month ago
by 
Reality123b
Why doesn't anyone host llms in zerogpu spaces?
8
#172 opened about 1 month ago
by Reality123b
CongWei1230ย
authored a
paper about 1 month ago