# Data-Efficient Approach to Humanoid Control via Fine-Tuning a Pre-Trained
GPT on Action Data

Siddharth Padmanabhan1, Kazuki Miyazawa1, Takato Horii1, Takayuki Nagai1,2
1Graduate School of Engineering Science, Osaka University, 1-3 Machikaneyama-
cho, Toyonaka-shi, Osaka 560-8531, Japan2AIX, The University of Electro-
Communications, 1-5-1 Chofugaoka, Chofu-shi, Tokyo 182-8585, Japan

###### Abstract

There are several challenges in developing a model for multi-tasking humanoid
control. Reinforcement learning and imitation learning approaches are quite
popular in this domain. However, there is a trade-off between the two.
Reinforcement learning is not the best option for training a humanoid to
perform multiple behaviors due to training time and model size, and imitation
learning using kinematics data alone is not appropriate to realize the actual
physics of the motion. Training models to perform multiple complex tasks take
long training time due to high DoF and complexities of the movements. Although
training models offline would be beneficial, another issue is the size of the
dataset, usually being quite large to encapsulate multiple movements. Many
papers have implemented state of the art deep learning models such as
transformers to control humanoid characters and predict their motion based on
a large dataset of recorded/reference motion. In this paper, we train a GPT on
a large dataset of noisy expert policy rollout observations from a humanoid
motion dataset as a pre-trained model and fine tune that model on a smaller
dataset of noisy expert policy rollout observations and actions to
autoregressively generate physically plausible motion trajectories. We show
that it is possible to train a GPT-based foundation model on a smaller dataset
in shorter training time to control a humanoid in a realistic physics
environment to perform human-like movements.

###### Index Terms:

GPT, Imitation Learning, Humanoid, Motion Prediction, Whole Body Control

##  I Introduction

Humanoid whole body control is becoming an increasingly interesting domain for
enabling humanoids to be platforms for testing powerful deep learning models
such as transformers for whole body control. However, for any character or
robot that operates in continuous domain, reinforcement learning is time
consuming and difficult for multiple reasons; designing reward statement is
usually task-specific, character/robot specific and/or possibly simulator
specific, and the online interaction between the training model and the
simulator consumes time. Imitation learning has come a long way, and many
papers have shown the efficacy of using motion capture data as reference data
to train humanoids to learn natural human-like movements[1, 2, 3]. However,
imitation learning by using kinematic data alone cannot be used to implement a
model to control a humanoid in a realistic physics environment, as it serves
to only help the model imitate the desired behavior and not necessarily
capture the realism of the dynamics as well unless additional feedback like
reward signals are used. This impact of the trade-off between reinforcement
learning and imitation learning can be minimized by employing both strategies
to encourage the model to imitate desired behaviors without having to design
reward functions and at the same time follow the rules of physics either in
simulators or in the real world. It is particularly difficult to generalize
motion for humanoids on a variety of tasks, either due to the complex nature
of the skeleton structure and/or the task. Expressing a single controller that
can perform multiple movements is holy grail for whole-body control of
humanoids.

On the other hand, research is progressing on foundation models that can be
adapted to multiple tasks [4]. For instance, in the field of natural language
processing, a single model can perform multiple tasks by self-supervised pre-
training on a large amount of textual data [5]. Similarly, in humanoid
control, if we can create a foundation model that can adapt to multiple tasks,
we can improve humanoid adaptability.

Figure 1: Proposed approach to training a GPT policy to autoregressively
generate physically plausible motion in simulation by pre-training on a large
dataset containing only observations and fine tuning on a small dataset
containing observations and actions

In this study, we explore the potential of using a pre-trained motion
foundation model, originally trained on non-physics data, and fine-tuning it
on a smaller physics-based dataset. This approach allows us to avoid the
extensive training time typically required for large multi-task models in an
online simulator environment. Instead, we utilize a substantial dataset
comprised of observations and actions derived from policy rollouts across
multiple tasks. This facilitates data-efficient imitation learning.

We hypothesize that by using a GPT-based motion pre-trained model, we would
only have to fine tune this model on a smaller dataset, which we refer to as a
Human Motion Generator (HMG), to plan physically plausible trajectories for
humanoid motion, therefore significantly reducing the training time and
dataset size (Fig. 1).

In this paper we:

  1. 1.

demonstrate data efficient learning via training a GPT based on a large
dataset without physics data and then fine tune on physics data.

  2. 2.

perform a comprehensive evaluation between our proposed model HMG and models
trained from scratch by comparing the performances based on motion prediction
metrics, trajectory generation lengths, empirical analysis of humanoid motion.

We show from our experiments that our proposed model can generate more
accurate and longer motion trajectories of higher quality than the state of
the art. These promising results serve as a preliminary step towards
developing the first humanoid motion foundation model for control that can be
further fine-tuned for other downstream tasks.

##  II Related Work

###  II-A Using Pre-trained Knowledge

Generalization of human motion prediction is a difficult problem due to
several reasons, such as the varying skeletal structure and idiosyncrasies in
the human motion data. To tackle this issue, [6] trains a model through a
curriculum and continual learning manner, such that a model can be first
trained on a diverse dataset to be robust and then fine tuned to predict
motion of new subjects. [7] proposes a two stage training, a pre-training
stage where 3-D motion is derived from noisy partial 2-D observation, then a
fine tuning stage where the model is fine-tuned to solve downstream tasks such
as 3-D pose estimation, action recognition, and mesh recovery. [8] implements
a similar version of a two-stage training, where initially the humanoid is
trained to walk in a fully observed condition (having access to all sensory
information and more from the humanoid) and then this policy is distilled to
another policy trained in a partially observed condition. In this work, a GPT
is pre-trained on observation data, resulting in a generalized representation
of humanoid kinematics over a variety of of behaviors, and later fine tuned on
action data that can control the humanoid in a physics simulator.

###  II-B Transformer based models for humanoid/robot control

Transformers have been shown to be powerful to generate human motion[8, 9].
[10] uses the idea of dual attention to capture spatial and temporal
dependencies of known data without relying on hidden states in RNNs or
temporal encodings like Discrete Cosine Transformation. This model effectively
generates poses that are temporally coherent. MotionGPT fuses language and
motion to enhance performance of motion-related tasks such as text-to-motion,
motion generation, and motion in-betweening [11, 12]. [13] also demonstrates
GPT’s capability to generate motion in a physics engine after taking in one
second long motion prompts. In this paper, we train a GPT policy to control a
humanoid, similar to [13], but by employing a more data-efficient training
strategy.

###  II-C Prediction Models for Humanoid

Many papers have focused on optimizing human motion prediction by using
various deep learning techniques[14, 15, 16]. Instead of using memory-based
networks like RNN or Transformers, a simple feed-forward network with fully
connected layers, layer normalization, and transpose operations, can be used
to generate human motion using spatial and temporal information [17].
Similarly considering the idea that human motion prediction depends on spatial
and temporal information, by using a scene image, the past body poses of
human, and the past 2-D locations as contexts, future 3-D poses and 3-D
locations can be predicted [18]. FrankMocap [19] proposes a modular approach
where regression is performed for face, hands, and body individually and then
integrated later to produce a whole body pose output. [20] proposes DLow, a
sampling strategy to obtain diverse set of samples from a trained generative
model. This sampling strategy serves to tackle two problems; lack of diversity
and inability to cover minor nodes in the data distribution. Here, we use GPT
for motion generation since it is powerful for predicting long sequences of
structured data given a small context.

###  II-D Datasets

There are many popular datasets for human motion capture data available to the
public[21, 22, 23, 24], but not many have simulation data consisting of
control outputs coupled with observational data. In this paper we use both the
large and small versions of the MoCapAct dataset[13], to train and evaluate
our proposed model HMG and other GPTs for comparative evaluation. MoCapAct
dataset is a dataset available to the public that consists of several rollouts
of states and actions of the MuJoCo ball-joint humanoid played in the MuJoCo
simulator. There are two versions of this dataset, one is around 500 GB in
size and the other is 60 GB in size. In [13], a GPT policy was trained from
scratch whereas we show a more data-efficient approach to train a GPT via pre-
training and fine-tuning, while also obtaining similar, if not, better motion
generation capability.

##  III Methodology

Pre-training a model and then fine-tuning that model on downstream tasks is a
common practice adopted in NLP [25, 26], and in human motion prediction for
human-robot interaction. The architecture, datasets, and hyperparameters used
are the same as those used in [13].

The training methodology implemented and architecture are shown in Fig. 2. The
first step is to train the humanoid motion foundation model. This serves as a
pre-training phase. The motion foundation model is a minGPT [27] with model
size 57M parameters. The motion foundation model is trained only on the
observations taken from the large version of the MoCapAct dataset, i.e, the
input and the output were observations. This dataset contains 100 rollouts of
each motion behavior.

After the pre-training phase, this foundation model is fine-tuned by loading
the previously obtained weights, and replacing the final linear feed-forward
layer with a new learnable layer suited to output actions. Using both
observations and actions taken from the small version of the MoCapAct dataset
the foundation model is fine-tuned to realize the physics of the behaviors.
This smaller dataset contains 10 rollouts of each motion behavior.

Later in the Experiment Section, we show in our evaluations of fine-tuning our
pre-trained model on different dataset sizes and show that our proposed model
surpasses models trained from scratch quicker with significantly smaller
dataset size. A learning rate of $3*10^{-7}$, which is lower than that used
during pre-training, was used in fine tuning.

Figure 2: Detailed Proposed Approach of HMG: On top is the pre-training phase
where a GPT is trained on a large observation dataset consisting of only
observations; in the middle is the fine tuning phase, where the same GPT
weights are used except the observation head which is replaced with an
untrained action head to output actions, and the pre-trained model is fine-
tuned on a small dataset consisting of both observations and actions; the
bottom shows the inference of the resulting HMG. The GPT weights in gray
depict that they are frozen, and the GPT weights in other colors denote that
they are trainable.

The weights of the pre-trained model were updated against a cross-entropy loss
during training and an MSE loss during validation. The losses have a minor
change during the fine-tuning process where the losses are calculated between
actions but not observations. The cross entropy loss is defined below

Figure 3: MuJoCo humanoid from dm_control package in MuJoCo simulator

$L_{ce}=\sum_{i=1}^{N}\hat{\tau}_{i}\log({\tau}_{i}),$ (1)

where $\hat{\tau}_{i}$ can be either generated observation $\hat{s}_{t}$ or
generated action $\hat{a}_{t}$ depending on whether the model is trained in
pre-training stage or fine-tuning stage, and ${\tau}_{i}$ denotes either real
observations $s_{t}$ or actions $a_{t}$. $N$ is the output size, therefore if
$\tau_{i}$ is action then $N$ is the number of actions and if $\tau_{i}$ is
observation then $N$ is the number of observations. The observable quantities
used in training the pre-trained model and HMG is given in Table I. The mode
of control is positional control and the output actions are joint positions.
Just like in [13], the simulator used in this work to test our motion
prediction policy is MuJoCo[28] and the humanoid used in the experiments is a
standard MuJoCo humanoid from the dm_control package that has 56 DoF (Fig. 3).

TABLE I: List of Observables and Actions taken from MoCapAct dataset

Observables | Description
---|---
Joint Pose | joint angles of each DoF in radians
Velocimeter | root velocity in Cartesian XYZ directions
Gyrometer | root orientation in Cartesian XYZ directions
End Effector Pose | end effector orientation
World Z Axis | direction of Cartesian Z axis
Actuator Activation | boolean values on whether actuators are active or not
Touch Sensors | contact forces between humanoid and ground
Torque Sensors | joint torque of each DoF
Body Height | root height from ground

Actions | Description
---|---
Joint Pose | joint angles of each DoF in radians
Control Mode | position control mode

##  IV Experiment

In our experiments, we train the pre-trained model on a large dataset and fine
tune it on a small dataset for motion prediction downstream task, which we
call it the HMG. We then compare and evaluate this HMG with MoCapAct-Small and
MoCapAct-Large, where Small and Large denotes the size of the dataset the
models are trained on.

The datasets used are the publicly available large and small versions of
mocapact dataset. These datasets contains noisy observation and action data
collected from expert policies. The pre-trained model is trained on a large
dataset of only observations and then it is fine tuned on a smaller dataset
consisting of both observations and actions. The time taken for pre-training
is around 52 hours (2M steps), and the time taken for fine-tuning is around 12
hours (400K steps), thus the fine tuning phase takes almost a quarter of the
time taken to train the motion foundation model.

We trained and tested three models: our proposed model HMG, MoCapAct-Large,
and MoCapAct-Small, and performed a comprehensive comparative evaluation
between them. The differences between these models in terms of training
configuration is given in Table II. HMG is our proposed foundation model fine-
tuned on a motion prediction downstream task. As previously stated, we chose
two GPTs, namely MoCapAct-Large and MoCapAct-Small, trained on the large and
small versions of the MoCapAct dataset respectively from scratch, to compare
their performances with the performance of our model trained by using our
proposed method on motion completion. Motion completion can be defined as the
generation of motion based on a motion prompt input of a given length. The
length of motion prompt used in this work is 32 steps that amounts to one
second. The weights for MoCapAct-Large are publicly available and MoCapAct-
Small was trained at our facility. We could see that although our proposed
model and MoCapAct-Large possess similar capabilities, our model still
slightly outperforms the latter.

The following subsections delineates the models’ evaluations, where we see how
well the models are capable of generating behaviors via prediction lengths,
empirical differences in behaviors, and motion prediction metrics to evaluate
motion quality, motion similarity to the ground truth, and motion diversity.
Since the generated trajectory lengths by MoCapAct-Small were significantly
lower than the other two models, we concluded that MoCapAct-Small performs
poorly and did not include it in some of the evaluations.

TABLE II: Models used for Evaluation | Dataset | Training
---|---|---
Model | Large Dataset | Small Dataset | Pre-Trained | Fine-Tuned | Trained from scratch
HMG | ✓(Pre-Training Only) | ✓(Fine-Tuning Only) | ✓(2M steps) | ✓(400K steps) | ✗
MoCapAct-Large | ✓ | ✗ | ✗ | ✗ | ✓(2M steps)
MoCapAct-Small | ✗ | ✓ | ✗ | ✗ | ✓(2M steps)

###  IV-A Dataset Sizes

Our proposed model was fine-tuned on datasets of half and quarter the size of
the Small dataset. We can see that the average generated episode lengths of
the HMG trained on quarter and half the size of the Small dataset are lower
than that of MoCapAct-Large, which is trained on the Large dataset that is 10
times the size of the Small dataset, and higher than that of MoCapAct-Small.
From Fig. 4, we can see that HMG quickly generates higher average prediction
lengths on the validation datasets after being trained on the Small dataset,
compared to MoCapAct-Large which requires the Large dataset to generate an
average prediction length of 5.75 seconds. This shows that our proposed model
is significantly data-efficient in learning motion prediction downstream task.

Figure 4: Performance based on Dataset Sizes

###  IV-B Behavioral Differences

####  IV-B1 Quantitative Analysis

Using the differences in episode lengths of motion predictions, we were able
to find the exact behaviors generated by the models that outperformed the
other models in terms of episode lengths. We define an episode length as the
total length of an episode for the humanoid to complete the task before
episode termination (when it falls down) or the maximum episode length (set to
about 15 seconds). We considered the minimum difference in episode lengths of
generated behaviors between models to be 6 seconds which we considered
significant.

We observed that the number of episode lengths that were generated by HMG that
were longer than that generated by MoCapAct-Large, were more than the number
of episode lengths that were generated by MoCapAct-Large that were longer than
that generated by HMG (Table III). Also, the average of the differences in
episode lengths were observed to be higher in HMG than in MoCapAct-Large. This
shows that our proposed model has better capability in predicting longer
trajectories that enables the humanoid to survive longer in the simulation
environment.

####  IV-B2 Qualitative Analysis

We further looked into the exact behaviors that led to these differences in
prediction lengths, to observe empirically how different the motion
predictions were. Figs. 7 to 7 show the comparison between the frames of the
humanoid behaviors generated by HMG and MoCapAct-Large models. The gray
humanoid depicts the reference motion, the other humanoid when in red denotes
that the humanoid is given the motion prompt, and when in bronze denotes that
the humanoid is under motion prediction. Fig. 7 shows a humanoid behavior that
involves arm gestures while standing. Fig. 7 shows a humanoid behavior that
involves immediate change in direction and running. Fig. 7 shows a simple
locomotion behavior.

There were differences between the models’ generations of moderate speed
cyclic movements like walking, fast cyclic movements such as running, and non-
cyclic movements as well. Walking behaviors patterns were observed to be
similar, with the only difference being the generation length, with HMG coming
out on top. Both the models suffer when predicting running behaviors and the
humanoid falls quickly to the ground. We attribute this issue to the flight
phase in running which might make it more difficult to predict the proper
footstep planning. For non-cyclic behaviors like arm gestures or side-
stepping, HMG generates movements that are not necessarily close to the ground
truth, but helps it survive longer in the episode compared to MoCapAct-Large.
This indicates that our proposed model has learnt different representation
that abstract a better understanding of the relationship between observations
and actions, that in turn helps in understanding the dynamics of the humanoid
in the simulation environment to preserve the imitation and survival as much
as possible.

Figure 5: Gestures: Top - HMG’s prediction does not match the ground truth but
survives the entire duration of the episode., Bottom - MoCapAct-Large’s motion
generation also does not accurately predict the gestures, loses balance and
falls.

Figure 6: Running Towards the Left: Top - HMG survives a little longer than
MoCapAct-Large but falls possibly due to the flight phase or fast changes in
foot placement, Bottom - MoCapAct-Large falls almost immediately from the
start of the motion prediction.

Figure 7: Walking Randomly: Top - HMG again maintains balance throughout the prediction while also succeeds in walking in different directions, Bottom - MoCapAct-Large fails to change direction and falls early. TABLE III: Motion Generation Durability |  | Avg. Episode
---|---|---
| No. of generated motions | Length Differences
Model | that outlasted the other model | (sec)
HMG | 9 | 8.63
MoCapAct-Large | 5 | 8.082

###  IV-C Generated Trajectory Length

In this subsection, we are going to cover the models’ capability for
generating long sequences given a motion prompt. HMG, MoCapAct-Large, and
MoCapAct-Small were all made to generate predicted trajectories. The resulting
episode lengths of generated trajectories of behavior clips from the training
and validation datasets were recorded.The length of the motion prompts was 32
steps long, which constitutes one second of motion from the expert policy.

We can observe that the generated episode lengths of our proposed model are
similar to those lengths generated by the MoCapAct-Large model, and overall
slightly longer than MoCapAct-Large before episode termination. MoCapAct-
Small, like the other models, performs well on the validation dataset compared
to the training dataset. However, it fails to compete with the other models as
it is unable to generate longer episode lengths. This shows that MoCapAct-
Small lacks the ability to predict sequences greater than 5 seconds in length,
indicating that training a GPT model on a small dataset for motion generation
without pre-training performs poorly. We can confirm this from Figs. 9 and 9
as well. From these box plots, it is easier to discern that HMG slightly
outperforms MoCapAct-Large in terms of generating more behaviors and longer
predictions, whereas MoCapAct-Small lack similar capabilities.

Figure 8: Comparison of Generated Episode Lengths on Training Dataset

Figure 9: Comparison of Generated Episode Lengths on Validation Dataset

###  IV-D Motion Prediction Metrics

To further evaluate the motion prediction in terms of motion quality, motion
imitation, and generation diversity, the standard metrics for motion
prediction were chosen, namely; Frechet Inception Distance (FID), Average
Displacement Error (ADE), Final Displacement Error (FDE), and Diversity (DIV).
We chose these metrics as the standard based on previous works that used these
metrics to evaluate kinematic motion prediction[11, 29, 30, 31]. We believe
that these metrics are appropriate here even for dynamics motion prediction
since the mode of control is positional (units are homogeneous between
kinematic and dynamic motion prediction). First the metrics will be defined as
below.

FID is one of the most important metrics to determine how well the generated
motion quality is. The idea behind FID is to compare the distributions between
the features from the motion feature extractor based on the real and generated
trajectories. The common practice is to train another RNN as a motion feature
extractor on the relevant data and then extract the features. Training another
RNN in this case is redundant, since the GPT is trained on reconstructing and
predicting the motion given the motion prompt, plus, the representation of the
features taken from the middle of the GPT network is rich[32]. Hence, HMG and
MoCapAct-Large themselves were used as motion feature extractors to calculate
FID and DIV respectively. Equation (2) shows how to calculate FID

$\text{FID}=\|\mu_{r}-\mu_{g}\|^{2}+\mathrm{Tr}(\Sigma_{r}+\Sigma_{g}-2(\Sigma_{r}\Sigma_{g})^{1/2}),$
(2)

where, $\mu_{r}$, $\mu_{g}$, $\Sigma_{r}$, $\Sigma_{g}$ are the means and co-
variances of the features from the real and generated data respectively.
$\mathrm{Tr}$ is the trace of the resulting matrix.

Understanding ADE is intuitively quite straightforward. It is essentially the
average of the differences in joint poses across entire trajectories between
the generated trajectories and the ground truth

$\text{ADE}=\frac{1}{N\times
T}\sum_{i=1}^{N}\sum_{t=1}^{T}\|\hat{\mathbf{j}}_{i,t}-\mathbf{j}_{i,t}\|,$
(3)

where $\hat{\mathbf{j}}_{i,t}$ and $\mathbf{j}_{i,t}$ are the generated and
real joint poses respectively at time step $t$ and trajectory $i$.

FDE is the difference in joint poses at the last step of the trajectory
between the generated motion and ground truth

$\text{FDE}=\frac{1}{N}\sum_{i=1}^{N}\|\hat{\mathbf{j}}_{i,T}-\mathbf{j}_{i,T}\|,$
(4)

where $\hat{\mathbf{j}}_{i,T}$ and $\mathbf{j}_{i,T}$ are the generated and
real joint poses respectively at the final time step of the trajectory $i$.

FID, ADE, and FDE help us understand the quality of the motion produced and
how close the generated motion is to the ground truth.

Another important metric is the generation diversity (DIV). This metric shows
how divergent the generated motion feature distribution are from the real
feature motion data. The higher the diversity, the higher the ability of the
model to produce novel behaviors. Although, it may be desirable if our model
stays true to the real data as much as possible to reduce the risk of
generating unpredictable novel behaviors. Equation 5 calculates the diversity
score of a model

$\text{DIV}=\frac{1}{n}\sum_{i=1}^{n}\mathbf{E_{d}}(x_{i},\hat{x}_{i})$ (5)

where $x,\hat{x}$ are real and generated features respectively, $n$ is the
total number of samples, and $\mathbf{E_{d}}$ is the Euclidean distance.

For our model we desire all the metrics except DIV to be lower than that of
MoCapAct-Large. We require ADE and FDE to be low so that the motion similarity
is high. We require FID to be low to ensure that the motion quality is as
close to that of ground truth. Generally, diversity is preferred to be
slightly higher for a multi-tasking model, capable of generating novel and
emergent behaviors, but not too much that it deviates largely from the actual
motion. From Table IV, we can see that the model displays the desired scores
except for DIV. This is probably due to the trade-off of having a low FID
score, staying faithful to the ground truth enables the model to generate
lesser diverse behaviors, hence the lower DIV score.

TABLE IV: Motion Prediction Scores: $\downarrow$ indicates that lower score is desired and $\uparrow$ indicates that higher score is desired. Model | FID $\downarrow$ | ADE $\downarrow$ | FDE $\downarrow$ | DIV $\uparrow$
---|---|---|---|---
HMG | 14.012 | 2.145 | 0.061 | 4.291
MoCapAct-Large | 20.785 | 2.443 | 2.544 | 4.662

##  V Discussion

In [13], box plots and histograms were used to evaluate their model’s
capability to generate motion. However, these graphs alone are not enough to
evaluate the model since it is hard to tell how well the humanoid can actually
imitate and complete the ground truth motion prompts accurately. Hence the
standard metrics FID, DIV, ADE, and FDE were chosen to additionally evaluate
the motion quality, imitation and diversity. These are the standards that have
been adopted in [11, 33, 34]. FID, ADE, FDE can evaluate the quality and
accuracy of imitation, and DIV is a metric used to evaluate the multi-tasking
capability of the model to generate novel behaviors (diversity). The
representations of the GPT policies are different when the GPT is trained from
scratch and when fine-tuned over a foundation model. Fine-tuning a foundation
model gets the model to understand the relationship between the observations
themselves and the actions too, therefore learning a much richer
representation. This can explain the differences in predicted trajectories
observed in Figs. 7-7, and in turn the generation lengths and the other
evaluations.

We are curious to see if our foundation model can be trained and fine-tuned on
simulation data obtained from other robots such as [35]. If successful, then
our foundation model can be fine-tuned to any data that is robot/domain
specific. Perhaps we can use this foundation model for other downstream tasks
such as motion clustering and action recognition.

Our proposed model is an end-to-end model, therefore separate controllers are
not trained to target imitation and balance individually, thus that eliminates
the need for a high-level controller for imitation and balance.

##  VI Conclusion

We proposed a data-efficient approach for motion prediction by pre-training a
humanoid motion foundation model on observation data and fine-tuning it on
both observation and action data on a motion prediction downstream task. We
have evaluated our proposed method’s performance with training from scratch
based on several facets; motion prediction metrics, generation lengths, and
empirical observations on various cyclic, non-cyclic, and fast paced
behaviors. This comprehensive evaluation supports the fact that our model is
capable of generating physically plausible trajectories and can perform as
well as and even better than the conventional approach. It would be
interesting to see how this model can be applied to real humanoid robots by
fine-tuning the foundation model on data obtained from various real robots.
Furthermore, it would be interesting to see whether the humanoid character’s
data from MoCapAct dataset would suffice or the foundation model would need to
be trained on real robot data.

## Acknowledgment

This work was supported by the JST Moonshot R&D Grant Number JPMJMS2011.

## References

  * [1] Z. Xie, X. Da, M. van de Panne, B. Babich, and A. Garg, “Dynamics randomization revisited: A case study for quadrupedal locomotion,” _CoRR_ , vol. abs/2011.02404, 2020. [Online]. Available: https://arxiv.org/abs/2011.02404
  * [2] X. B. Peng, E. Coumans, T. Zhang, T. E. Lee, J. Tan, and S. Levine, “Learning agile robotic locomotion skills by imitating animals,” _CoRR_ , vol. abs/2004.00784, 2020. [Online]. Available: https://arxiv.org/abs/2004.00784
  * [3] A. Tang, T. Hiraoka, N. Hiraoka, F. Shi, K. Kawaharazuka, K. Kojima, K. Okada, and M. Inaba, “Humanmimic: Learning natural locomotion and transitions for humanoid robot via wasserstein adversarial imitation,” 2024. [Online]. Available: https://arxiv.org/abs/2309.14225
  * [4] R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill _et al._ , “On the opportunities and risks of foundation models,” _arXiv preprint arXiv:2108.07258_ , 2021.
  * [5] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, “Language models are unsupervised multitask learners,” 2019. [Online]. Available: https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf
  * [6] M. S. Yasar and T. Iqbal, “Improving human motion prediction through continual learning,” _CoRR_ , vol. abs/2107.00544, 2021. [Online]. Available: https://arxiv.org/abs/2107.00544
  * [7] W. Zhu, X. Ma, Z. Liu, L. Liu, W. Wu, and Y. Wang, “Motionbert: A unified perspective on learning human motion representations,” 2023. [Online]. Available: https://arxiv.org/abs/2210.06551
  * [8] I. Radosavovic, T. Xiao, B. Zhang, T. Darrell, J. Malik, and K. Sreenath, “Real-world humanoid locomotion with reinforcement learning,” 2023. [Online]. Available: https://arxiv.org/abs/2303.03381
  * [9] M. Janner, Q. Li, and S. Levine, “Offline reinforcement learning as one big sequence modeling problem,” 2021. [Online]. Available: https://arxiv.org/abs/2106.02039
  * [10] E. Aksan, P. Cao, M. Kaufmann, and O. Hilliges, “Attention, please: A spatio-temporal transformer for 3d human motion prediction,” _CoRR_ , vol. abs/2004.08692, 2020. [Online]. Available: https://arxiv.org/abs/2004.08692
  * [11] B. Jiang, X. Chen, W. Liu, J. Yu, G. Yu, and T. Chen, “Motiongpt: Human motion as a foreign language,” 2023. [Online]. Available: https://arxiv.org/abs/2306.14795
  * [12] Y. Zhang, D. Huang, B. Liu, S. Tang, Y. Lu, L. Chen, L. Bai, Q. Chu, N. Yu, and W. Ouyang, “Motiongpt: Finetuned llms are general-purpose motion generators,” 2023. [Online]. Available: https://arxiv.org/abs/2306.10900
  * [13] N. Wagener, A. Kolobov, F. V. Frujeri, R. Loynd, C.-A. Cheng, and M. Hausknecht, “Mocapact: A multi-task dataset for simulated humanoid control,” 2023. [Online]. Available: https://arxiv.org/abs/2208.07363
  * [14] D. Ha and J. Schmidhuber, “World models,” 2018. [Online]. Available: https://zenodo.org/record/1207631
  * [15] D. Hafner, T. Lillicrap, M. Norouzi, and J. Ba, “Mastering atari with discrete world models,” 2022. [Online]. Available: https://arxiv.org/abs/2010.02193
  * [16] D. Hafner, J. Pasukonis, J. Ba, and T. Lillicrap, “Mastering diverse domains through world models,” 2023. [Online]. Available: https://arxiv.org/abs/2301.04104
  * [17] W. Guo, Y. Du, X. Shen, V. Lepetit, X. Alameda-Pineda, and F. Moreno-Noguer, “Back to mlp: A simple baseline for human motion prediction,” 2022. [Online]. Available: https://arxiv.org/abs/2207.01567
  * [18] Z. Cao, H. Gao, K. Mangalam, Q. Cai, M. Vo, and J. Malik, “Long-term human motion prediction with scene context,” _CoRR_ , vol. abs/2007.03672, 2020. [Online]. Available: https://arxiv.org/abs/2007.03672
  * [19] Y. Rong, T. Shiratori, and H. Joo, “Frankmocap: A monocular 3d whole-body pose estimation system via regression and integration,” _CoRR_ , vol. abs/2108.06428, 2021. [Online]. Available: https://arxiv.org/abs/2108.06428
  * [20] Y. Yuan and K. M. Kitani, “Dlow: Diversifying latent flows for diverse human motion prediction,” _CoRR_ , vol. abs/2003.08386, 2020. [Online]. Available: https://arxiv.org/abs/2003.08386
  * [21] C. Ionescu, D. Papava, V. Olaru, and C. Sminchisescu, “Human3.6m: Large scale datasets and predictive methods for 3d human sensing in natural environments,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 36, no. 7, pp. 1325–1339, jul 2014.
  * [22] Y. Zhu, N. Samet, and D. Picard, “H3wb: Human3.6m 3d wholebody dataset and benchmark,” 2023. [Online]. Available: https://arxiv.org/abs/2211.15692
  * [23] A. R. Punnakkal, A. Chandrasekaran, N. Athanasiou, A. Quiros-Ramirez, and M. J. Black, “BABEL: bodies, action and behavior with english labels,” _CoRR_ , vol. abs/2106.09696, 2021. [Online]. Available: https://arxiv.org/abs/2106.09696
  * [24] N. Mahmood, N. Ghorbani, N. F. Troje, G. Pons-Moll, and M. J. Black, “AMASS: archive of motion capture as surface shapes,” _CoRR_ , vol. abs/1904.03278, 2019. [Online]. Available: http://arxiv.org/abs/1904.03278
  * [25] T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei, “Language models are few-shot learners,” 2020. [Online]. Available: https://arxiv.org/abs/2005.14165
  * [26] J. Devlin, M. Chang, K. Lee, and K. Toutanova, “BERT: pre-training of deep bidirectional transformers for language understanding,” _CoRR_ , vol. abs/1810.04805, 2018. [Online]. Available: http://arxiv.org/abs/1810.04805
  * [27] A. Karpathy, “mingpt,” 2020. [Online]. Available: https://github.com/karpathy/minGPT
  * [28] E. Todorov, T. Erez, and Y. Tassa, “Mujoco: A physics engine for model-based control,” in _2012 IEEE/RSJ International Conference on Intelligent Robots and Systems_. IEEE, 2012, pp. 5026–5033.
  * [29] C. Guo, X. Zuo, S. Wang, S. Zou, Q. Sun, A. Deng, M. Gong, and L. Cheng, “Action2motion: Conditioned generation of 3d human motions,” in _Proceedings of the 28th ACM International Conference on Multimedia_ , ser. MM ’20. ACM, Oct. 2020. [Online]. Available: http://dx.doi.org/10.1145/3394171.3413635
  * [30] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, “Gans trained by a two time-scale update rule converge to a local nash equilibrium,” 2018. [Online]. Available: https://arxiv.org/abs/1706.08500
  * [31] T. Salzmann, M. Pavone, and M. Ryll, “Motron: Multimodal probabilistic human motion forecasting,” 2022. [Online]. Available: https://arxiv.org/abs/2203.04132
  * [32] M. Chen, A. Radford, R. Child, J. Wu, H. Jun, D. Luan, and I. Sutskever, “Generative pretraining from pixels,” in _Proceedings of the 37th International Conference on Machine Learning_ , ser. ICML’20. JMLR.org, 2020.
  * [33] Y. Zhang, M. J. Black, and S. Tang, “We are more than our joints: Predicting how 3d bodies move,” 2021. [Online]. Available: https://arxiv.org/abs/2012.00619
  * [34] H. Ma, J. Li, R. Hosseini, M. Tomizuka, and C. Choi, “Multi-objective diverse human motion prediction with knowledge distillation,” in _2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2022, pp. 8151–8161.
  * [35] F. Al-Hafez, G. Zhao, J. Peters, and D. Tateo, “Locomujoco: A comprehensive imitation learning benchmark for locomotion,” 2023. [Online]. Available: https://arxiv.org/abs/2311.02496