Streaming Multi-Track Timeline Control for 3D Human Motion Generation

Abstract

Text-driven human motion generation has advanced substantially, yet most methods assume instructions are available before synthesis. Interactive applications require responding to new instructions while continuing ongoing actions, such as answering a phone while walking. Existing approaches address streaming generation or simultaneous composition without explicitly combining streaming instruction arrival with independently timed, overlapping actions. We introduce streaming multi-track timeline control and propose TimelineControl to incorporate new instructions alongside ongoing actions. Interval-aware conditioning preserves instruction timing, while causal part-structured representations and part-aware denoising coordinate concurrent actions across body regions. We also construct TimelineMotion, a dataset with overlapping instruction intervals and body-part annotations. Experiments on TimelineMotion and MTT demonstrate improved semantic alignment and temporal adherence over evaluated streaming baselines, including models retrained on the same data. Ablations and human evaluations validate our design, complemented by spatial conditioning and humanoid execution demonstrations. Our code, data and models will become publicly available.

Related publications

• PACT: End-to-End Learning of Human Pose, Contacts, and Forces from VideoarXiv · 2026 · Same author: Rikhat Akizhanov
• PhysMoDPO: Physically-Plausible Humanoid Motion with Preference OptimizationarXiv · 2026 · Same author: Rikhat Akizhanov
• AVTR-1: Open Stack for Real-Time Interactive AvatarsarXiv · 2026 · Shared topic: Human Motion and Animation
• Perturb-and-Solve: Efficient Learned-Operator Conditioning for Latent Diffusion Inverse ProblemsarXiv · 2026 · Shared topic: Generative Adversarial Networks and Image Synthesis