Hello, I'm
RL-Infra & RL Algorithms · Agentic-RL
Apodex
I currently work at Apodex, focusing on RL-Infra and RL algorithm development with a focus on Agentic-RL. I built the RL training system behind apodex-v1.0, Apodex's flagship agentic model — a fully-asynchronous stack (on top of miles) that decouples training from rollout, evaluates without pausing training, and reclaims idle training GPUs for generation via hybrid-colocate. I also built async_openrlhf, an open-source, Ray-based high-performance RLHF framework with a fully-async training/rollout pipeline, extending the widely-adopted OpenRLHF. My research spans LLM reasoning, MCTS, scaling laws for agentic reasoning, and efficient reasoning compression.
RL-Infra & RL Algorithms (Agentic-RL)
Math-Reasoning & Agentic-RL
AIGC Safety
Content Safety, Public Opinion Analysis, Personal Privacy, LLM Safety
Department of Electronic Engineering
Bachelor's Degree
apodex-v1.0 is Apodex's flagship agentic model. My main contribution is its RL training system — a fully-asynchronous, high-throughput Agentic-RL stack built on top of miles that fully decouples training from rollout, evaluates the policy without ever pausing the training loop, and squeezes maximum utilization out of every GPU by letting idle training cards double as rollout engines whenever there is no data left to consume.
Training workers and rollout engines run on independent loops, communicating only through a shared data buffer and output queue, so neither ever blocks the other. Off-policy staleness is bounded by two-level (cross-group + per-slot) filtering and a configurable staleness threshold, with cooperative-abort weight sync over NCCL between iterations.
A dedicated eval pool hot-reloads the rolling training checkpoints and evaluates in the background, so evaluation never interrupts the training loop (“training-data roll”). Eval failures are isolated and can never abort training, keeping throughput steady while metrics stream continuously.
When the buffer has no consumable data, colocated training GPUs onload as generation engines and join the rollout routing pool — then offload again for the optimizer step. Idle training cards become extra rollout throughput, driving near-full cluster utilization with dedicated remote engines handling steady-state generation.
Top-venue publications accepted in 2025–2026
A family of open research agents (MiroThinker-1.7 & h1) that scale long-horizon, verification-centric reasoning, trained with our fully-asynchronous RL stack.
An evidence-assembly framework that lets deep-research agents accumulate and verify evidence at scale for reliable long-horizon research.
Discovering scaling laws for agentic reinforcement learning where LLMs spontaneously learn to execute code for mathematical reasoning.
Rethinking how to scale agent interactions for long-horizon research tasks.
A comprehensive gym environment for training and evaluating generalist large language models.
A novel self-play framework that advances agent capabilities through search without external supervision.
A dual-perspective dynamic preference learning framework that adaptively reweights samples by jointly considering data quality and model performance evolution during training.
A comprehensive survey of reasoning LLMs, tracing the evolution from fast intuitive (System 1) to slow deliberate (System 2) reasoning paradigms.
A Ray-based open-source framework enabling scalable RLHF training, widely adopted by the research community.
Showing that LLMs can bootstrap inductive reasoning capability through deductive self-teaching.
An efficient method to compress long chain-of-thought reasoning while preserving accuracy.
Full list on Google Scholar
Technical writing on LLM reasoning, MCTS, and reinforcement learning
Research interests: Probabilistic Graphical Models, Reinforcement Learning, NLP
System 1 vs System 2 thinking paradigm with MCTS for LLM reasoning
Insights on reproducing OpenAI o1 with long-CoT distillation and DPO
Reward shaping, domain randomization, and what zero-RL really optimizes
Deep dive into RL implementation details: loss, gradient accumulation, GRPO variance
Simplifying LLM-RL: start minimal, add complexity only when needed
Zero-shot tool-integrated reasoning with RL — small models, big results
Async rollout + pipeline: 4x faster training built on OpenRLHF
Training-inference consistency for MoE models under RL
Stabilizing off-policy RL training under high staleness