Physically simulated characters can learn highly natural full-body motion guided by motion capture datasets. However, the range of motion is limited to the existing high-quality datasets, and cannot effectively adapt to challenging scenarios. We propose a novel policy architecture that learns part-wise motion skills, where individual parts can be separately extended and combined for unobserved settings. Our method employs a set of part-specific codebooks, which robustly capture motion dynamics without catastrophic collapse or forgetting. This structured decomposition allows intuitive control over the character’s behavior and dynamic exploration for a novel combination of part-wise motion. We further incorporate a refinement network compensating for subtle discrepancies in the disjoint discrete tokens, thus improving motion quality and stability. Our extensive evaluations show that our part-wise latent token achieves superior performance in imitating motions, even those from unseen distribution. We also validate our method in challenging tasks, including body tracking, navigation on complex terrains, and point-goal navigation with damaged body parts. Finally, we introduce a part-wise expansion of motion priors, where the physically simulated character incrementally adapts partial motion and produces unique combinations of whole-body motion, significantly diversifying motions.
Policy architecture of the part-wise latent token (PLT) model. (a) During the imitation-learning phase, we first spatially split the output of the encoder. Then, we employ multiple codebooks to extract latent token \(z_k\) for each body part. A refinement network calculates continuous offset \(\Delta z\) and splits it into \(\{\Delta z_k\}_K\). Each part-wise refinement \(\Delta z_k\) is applied to each latent token \(z_k\), and the corresponding part-wise low-level policy \(\pi_{\text{low}, k}\) computes actions \(a_k\) subsequently. (b) In the task-learning phase, a high-level policy \(\pi_{\text{high}}\) selects indices for latent tokens from the pretrained codebooks. (c) Our policy design allows part-wise adaptation by updating only the minimal set of parameters associated with the queried body part.
@inproceedings{bae2025plt,
title={PLT: Part-Wise Latent Tokens as Adaptable Motion Priors for Physically Simulated Characters},
author={Bae, Jinseok and Lee, Younghwan and Lim, Donggeun and Kim, Young Min},
booktitle={Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers},
pages={1--10},
year={2025}
}