A research team led by Professor Sung-Hee Lee from the Graduate School of Culture Technology developed a novel neural-network-based framework for constructing a latent motion manifold that can represent a wide range of human motions in a long sequence. The team introduced several new components that increase spatial and temporal coverage, while retaining the details of motion capture data at the same time. 

Motion data is typically represented as a time series consisting of frames. Each frame captures a character’s pose, which is parametrized by the character’s joint angles or positions. This model is useful in data processing, but valid human motion comprises only a small subspace of this representation, called a motion manifold. For this reason, it is of great interest to researchers to identify effective ways of finding compact and comprehensive motion spaces that produce a wide range of plausible motions at random sampling. 

Based on the previous studies on the unsupervised sequence-to-sequence models, Professor Lee and Deok-Kyeong Jang, a PhD student from Professor Lee’s Motion Computing Laboratory, introduced several technical contributions to achieve better motion manifolds and motion generation methods. 

The novelty of this model comes from the use of the combination of two decoders, one of which learns to generate the joint rotation, while the other learns to output joint rotation velocities. The rotation decoder has the ability to reconstruct long-term motions better, while the velocity decoder improves the motion continuity; combined, they demonstrated higher reconstruction accuracy compared to the single decoder model.

Moreover, the forward kinematics layer — the kinematic equations that calculate the position from specified joint parameters — allowed the model to satisfy bone length constraints and simplify the representation of joint limits. Lastly, the team introduced several loss functions that enhance the quality of the motion manifold, for example, their adversarial loss function increases the naturalness of the motion generated by the manifold.