# Estimation of posture and joint angle of human body using foot pressure
distribution :Morphological computation with human foot

Yo Kobayashi<EMAIL_ADDRESS>Graduate School of Engineering
Science, Osaka University, Osaka, Japan  Yasutaka Nakashima  Faculty of
Engineering, Kyushu University, Fukuoka, Japan

###### Abstract

This paper proposes a novel contact and wearable sensing system for estimating
the upper body posture and joint angles (ankle, knee, and hip) of the human
body using foot pressure distribution information obtained from a sensor
attached to the plantar region. In the proposed estimation method, sensors are
installed only on the plantar region, which is the end of the human body and
the point of contact with the environment. The posture and joint angles of
other parts of the body are estimated using only this information. As a
contact and wearable sensor, the proposed system differs from previous
measurement systems in the sense that the sensor does not need to be placed
near the target joint or body. The estimation was carried out using a
multivariate linear regression model with the foot pressure distribution as
the input and the joint angle or posture as the output. The results reveal
that it is possible to estimate the posture and joint angles of the human body
from foot pressure distribution information (R2$\fallingdotseq$0.9). The
proposed estimation method was validated by morphological computation to
confirm that it is enabled by foot morphology. The validation approach
compared the estimation accuracy achieved when an object was interposed
between the foot pressure distribution sensor and the plantar region and the
morphological relationship of the plantar region to the environment varied.
The results reveal that there is a significant difference in the estimation
accuracy between cases with and without an intervening object, suggesting that
the morphology of the plantar region contributes to the estimation.
Furthermore, the proposed estimation method is considered as physical
reservoir computing, wherein the human foot is used as a computational
resource.

Keywords: posture and joint angle estimation, foot pressure distribution,
multivariate linear regression model, morphological computation.

##  I Introduction

The measurement and estimation of the joint angles and posture of the human
body play an important role in various fields. For example, the estimation of
the human body posture and joint angles is an important research topic in
various fields such as robotics, human–computer interaction, and biomechanics
[1, 2, 3, 4, 5, 6, 6, 7, 4, 8, 9, 10, 11, 12]. In the field of robotics, the
accurate measurement and estimation of the human body posture and joint angles
is important. For example, when a human and a robot work together, the human’s
posture and movement must be accurately determined [3]. Additionally, in the
case of wearable assistive robots, the posture and angle of the wearer when
controlling the device must be estimated [4]. In the field of human–computer
interaction, understanding the movement and posture of the human body is
important for improving human–computer interaction. For example, technologies
such as virtual reality (VR) and augmented reality (AR) detect the user’s body
movement and posture to reflect information on the actual environment in
digital space [5], and can also use gestures and body movement to enable
natural and intuitive interaction with computers [6]. The field of
biomechanics considers kinematics and other aspects of the human body, and has
practical application in sports, rehabilitation, and medicine. In these
fields, it is important that the posture and joint angles of the human body
are accurately estimated to understand human motion [7, 8, 9, 10, 11, 12]. For
example, understanding the mechanical properties and functions of the
musculoskeletal system in sports, work, and daily life can help in evaluating
the performance of human movement, and contributes toward the diagnosis,
prevention, and treatment of various disorders and injuries related to
posture, balance, and mobility [12].

Different sensors combined with diverse information processing methods are
used for joint angle and posture estimation. Generally, these sensors are
categorized into contact and wearable sensors and non-contact sensing systems.
Regarding non-contact sensing systems, there are two main types. The first
type is motion capture using optical markers [13]. Optical motion capture
systems have been used to quantify joint kinematics by tracking the position
of reflective surface markers during dynamic activity [14]. However, these
systems are expensive, limited to controlled laboratory environments, and
prone to occlusion [15]. The second type is vision-based motion capture, which
uses a small number of RGB or RGB-D cameras to perform pose estimation.
Vision-based methods are small systems compared with optical motion capture
systems [16, 17, 18, 19, 20]. However, although these systems are expected to
extract distinguishable features from an image, they are sensitive to human
appearance and typically underperform when untextured clothing or challenging
ambient lighting are present [21]. Furthermore, vision-based methods are
susceptible to occlusion.

Joint angle and posture estimation using contact and wearable sensors is
independent of the measurement environment and does not suffer from the
occlusion problems of non-contact sensing systems [21]. Additionally, complex
equipment is not required and there are no restrictions on the range of use
[21]. For contact and wearable sensors, research has been conducted on encoder
and goniometer measurements. These contact sensors are placed directly on the
joints and can perform high-precision measurements. In many applications,
however, sensor placement without disturbing the joint integrity is difficult,
and can be uncomfortable and cumbersome when encoders are integrated into
wearable systems [1]. For other single-joint motion, a goniometer can be used
[22]. However, goniometers require careful alignment to the joint center and
must be repositioned each time the joint of interest changes, which renders
them incapable of monitoring multi-joint motion [7]. In contrast, inertial
measurement units (IMUs) are placed on links at a distance from the joints,
providing design flexibility and comfort without affecting the system
dynamics. The IMUs are small and portable, allowing for continuous
measurements without location constraints. Theoretically, the position and
orientation are calculated by the simple integration of acceleration and
angular velocity [1]. However, small errors in the measurement data
accumulate, resulting in position and orientation errors, which is known as
drift [14].

This study proposes a novel contact and wearable sensing system to estimate
the posture and joint angles of the human body using foot pressure
distribution information obtained from a sensor attached to the plantar
region. In this estimation method, sensors are installed only on the plantar
region, which is the end of the human body and the point of contact with the
environment, and the posture and joint angles of other parts of the body are
estimated only from this information. As a contact and wearable sensor, the
proposed system differs from previous measurement systems in the sense that
the sensor does not need to be placed near the target joint or body. In this
study, estimation was carried out using a multivariate linear regression
model. Additionally, the proposed estimation method was validated by
morphological computation to confirm that it is enabled by foot morphology. In
the validation process, the estimation accuracy achieved when an object is
interposed between the foot pressure distribution sensor and the plantar
region was compared with that achieved when the morphological relationship of
the plantar region to the environment varied. The findings of this study
demonstrate that, by processing a multivariate linear regression model
utilizing foot morphology, it is possible to estimate the posture and joint
angles of the human body from foot pressure distribution information.

##  II  Methods

Figure 1:  Experimental setup: (a) (a-1) experimental setup in this study;
(a-2) motion capture was used to obtain the posture and joint angles of the
human body; (a-3) a sheet-shaped distributed pressure sensor was used to
measure the pressure distribution on the plantar surface of the foot. (b) The
experiment was conducted under three conditions: (b-1) condition wherein
nothing was placed between the sole and the pressure distribution sensor
(Condition (A): nothing); (b-2) condition wherein a rubber plate was placed
between the sole and the pressure distribution sensor (Condition (B): rubber);
(b-3) condition wherein a plastic plate was placed between the sole and the
pressure distribution sensor (Condition (C): plastic). The experimental
participants were equipped with motion-capture markers and performed natural
squat exercises on a pressure distribution sensor. Foot pressure distribution
data and motion capture data were collected during the exercise.

This study was approved by the ethical committee for human studies at the
Graduate School of Engineering Science, Osaka University, Japan. All
experiments were performed according to the approved design. Before the
experiment, all participants signed the informed consent form. The
participants included in the figures provided informed consent for the
publication of images in online publications.

The experimental concept is shown in Fig. 1. A sheet-shaped distributed
pressure sensor was used to measure the pressure distribution on the plantar
surface of the foot, while motion capture was used to obtain the posture and
joint angles of the human body. The posture and joint angles were estimated
using a multivariate linear regression model, with the pressure distribution
information as the model variables. Additionally, this study investigated
whether foot morphology affects the estimation accuracy. Specifically, the
investigation focused on whether the estimation accuracy changes when the
effect of foot morphology is eliminated by placing a rubber plate or plastic
plate between the foot pressure sensor and the plantar region.

The human posture and joint angles were measured by motion capture (V120 Trio,
OptiTrack, Acuity Inc.) using optical markers. The markers were attached to
the participants’ ankles, knees, hips, and shoulders. The movements of the
markers from the side of the participant were recorded by motion capture, and
each joint angle (ankle, knee, and hip joint) and upper body posture was
calculated from the positional information of the markers in the sagittal
plane. Thus, this study focused only on two-dimensional motion. Specifically,
the ankle joint angle, knee joint angle, and hip joint angle were calculated
from the marker positions, and the upper body angle was calculated from these
angles.

To measure the plantar pressure distribution, an electromagnetic induction-
based pressure distribution sheet sensor (LL Sensor, LL480$\times$480, Newcom
Inc.) was used to measure the participants’ foot pressure distribution. This
pressure distribution sensor consists of 48$\times$48 pixels, with each pixel
measuring 10$\times$10 mm. Comprehensive pressure data were collected for both
foot regions.

Seven adult males participated in the experiment. The average height of the
participants was 170 cm, and the average weight was 65 kg. Motion-capture
markers were placed on the participants, who were subsequently instructed to
perform natural squat exercises on a pressure distribution sensor with a
period of 2 s. A metronome was used to adjust the pace of exercises.
Measurements were taken over 30 s. The participants were instructed to refrain
from changing their plantar position during measurement, to keep the entire
plantar surface of their foot on the ground, and to refrain from lifting the
sole off the foot pressure distribution sensor. During the exercise, foot
pressure distribution data and motion capture data were collected.

The experiment was conducted under three conditions. Under the first
condition, nothing was placed between the sole and the pressure distribution
sensor (Condition (A): nothing). Under the second condition, a rubber plate
was placed between the sole and the pressure distribution sensor (Condition
(B): rubber). Under the third condition, a plastic plate was placed between
the sole and the pressure distribution sensor (Condition (C): plastic).
Condition (B) was set to test the effect of uniform foot stiffness
characteristics on the estimation accuracy. The rubber plate was made of
silicone rubber (Ecoflex0030, Smooth-On) with characteristics resembling those
of the human skin and a thickness of 3 mm. Condition (C) was set to test
whether the estimation accuracy is affected when the foot structure resembles
a link mechanism. The 3-mm thick plastic plate was fabricated using a 3D
printer with poly-lactic acid (PLA) filament. Condition (A) was implemented to
verify that this method can accurately estimate joint angles and postures,
while conditions (B) and (C) were implemented for comparison with condition
(A). Each participant was tested five times for condition (A), three times for
condition (B), and three times for condition (C), amounting to a total of 11
experiments. Because there were seven participants in the experiment, 77
datasets were obtained in total.

In the processing of experimental data, the sampling period of the pressure
distribution sensor was not constant, and the data varied. Therefore, data
resampling was carried out at regular intervals (20 ms) using linear
interpolation. Here, 20 ms was the average sampling time of the pressure
distribution sensor. Similarly, the motion capture data obtained at the
sampling rate of 120 Hz were resampled to the sampling time of 20 ms. The
measurement data were split into training and validation data. Specifically,
the first 3 s of the measurement data were removed, and the remaining 27 s
were used as training and validation data at the ratio of 5:1.

The multivariate linear regression model was used to estimate the joint angle
and posture from the pressure distribution information obtained in this study.
Here, the multivariate linear regression model was used because of its high
interpretability, simplicity, and low learning cost compared with other
machine learning models, such as neural networks. In the process of learning
the weights for linear regression, the models were calculated using Ridge
regression to avoid overlearning and multicollinearity problems. Lasso
regression and elastic net regression were performed on some data, but the
accuracy was similar to or lower than that of Ridge regression. Unlike Ridge
regression, which can be optimized with analytical expressions, Lasso and
elastic net regression require iterative calculations for optimization. In
this study, Ridge regression was selected owing to its advantages of
simplicity and speed. The specific procedure is described below.

First, variable selection was performed in the training process, because using
all pressure data would include unnecessary variables, and the estimation
accuracy would decrease because of overtraining. Only pressure data with high
correlation coefficients and joint angles were used as model variables.
Specifically, as a univariate feature selection for the Filter method,
features with low correlation coefficients were excluded. The correlations
with the four angles (ankle joint, knee joint, hip joint, and upper body) were
calculated, and only variables with a correlation coefficient R$>$0.15 were
selected. Consequently, the same set of variables was used to estimate each of
the four angles. The threshold value of 0.15 was selected by grid search. This
study selected the value at which the average value of R2 for all experiments
under condition (A) was highest. The results reveal that there was almost no
difference in the mean value of R2 at the threshold values in the range
0.10$<$ R$<$0.25. Subsequently, time series data for the pressure sensors (all
pressure data selected by variable selection), and angles (all joints and
posture angles) were standardized (Z-score normalization) for training. Thus,
the selected and standardized pressure data $p^{i}_{k}$ at the k-th time data
point and i-th location and each standardized joint data $\theta_{k}$ at the
k-th time data point were collected.

Subsequently, the multivariate linear regression model was used to estimate
the joint angle and posture from the pressure distribution information, as
follows. Notably, the following description pertains to a method for
estimating a single joint angle or posture in a single experiment for one
experimental dataset.

Equation (1) was used to calculate the estimated joint angle at the k-th time
data point in the multivariate linear regression model.

$\displaystyle\hat{\theta}_{k}=\sum_{i}w^{i}p^{i}_{k}$ (1)

where $\hat{\theta}_{k}$ is the estimated joint angle at the k-th time data
point, $p_{k}^{i}$ represents the pressure data of the i-th location at the
k-th time data point, and $w^{i}$ is the weight corresponding to the pressure
data at the i-th location.

A linear regression model was trained using Ridge regression, which is a
method for calculating the weight $\mathbf{w}=(w^{1},w^{2},\cdots)$ that
minimizes the following loss function with the residual sum of squares term
and the L2 regularization term.

$\displaystyle
L=\sum_{k}\left(\theta_{k}-\hat{\theta}_{k}\right)^{2}+\frac{1}{2}\lambda\sum_{i}(w^{i})^{2}$
(2)

where $\theta_{k}$ is the measured joint angle at each time data point k.

The weight $\mathbf{w}$ minimizing the loss function was determined
analytically using the following equation.

$\displaystyle\mathbf{w}=\bm{\theta}\mathbf{P}^{\mathrm{T}}\left(\mathbf{P}\mathbf{P}^{\mathrm{T}}+\lambda\mathbf{I}\right)^{-1}$
(3)

where $\mathbf{w}=(w^{1},w^{2},\cdots)$ is the weight vector;
$\bm{\theta}=\left(\theta_{1},\theta_{2},\cdots\right)$ is the joint vector
summarizing the measured joint angle at each time data
point;$\mathbf{P}=(\mathbf{p^{1}},\mathbf{p^{2}},\cdots)$ is the pressure
state matrix summarizing the vectors of the measured pressure data;
$\mathbf{p^{i}}=\left(p^{i}_{1},p^{i}_{2},\cdots\right)^{T}$ is the pressure
vector of the i-th location; $\lambda$ is a hyperparameter and $\mathbf{I}$ is
the identity matrix. The hyperparameter $\lambda$ was determined by trial and
error, and the same value ($\lambda=10$) was used for all training.

Using the above-mentioned methods, estimations were obtained at all joint
angles and postures for all trials. Notably, the learning and estimation
processes were carried out for each joint angle and posture in each dataset of
each experiment, and a different learner was used in each case.

For each dataset, the measured and estimated joint angles were subjected to
the inverse transformation of standardization, and their original scale was
restored. Then, the Root Mean Square Error (RMSE) and coefficient of
determination (R2) values between the estimated and measured values in the
validation data were calculated.

Additionally, statistical analyses were performed on the RSME and R2 results
between the data obtained under experimental conditions (A) and (B), and
between the data obtained under experimental conditions (A) and (C). In these
statistical analyses, the Shapiro-Wilk test revealed normality in the data,
and Welch’s t-test was performed because the F-test did not reveal equal
variance in the data. The statistical significance was set to p$<$0.05. The
reason for performing statistical analysis between the data for experimental
conditions (A) and (B) was to test whether the intervention of the rubber
plate, which ensures uniform sole characteristics, has a significant influence
on the estimation accuracy. Statistical analysis was also conducted for the
data obtained under experimental conditions (A) and (C) to test whether there
is a difference in the estimation accuracy when an intervening plastic plate
makes the foot structure resemble a link structure.

##  III  Results

Figure 2 shows a representative example of the estimation results for the
joint angles of the ankle, knee, and hip joints, and the upper body posture
under experimental condition (A). Figure 3 shows the estimation error (RMSE
and R2) results for each experimental condition. The results for experimental
condition (A) in Fig. 3 reveal that the accuracy of estimating the angle of
each joint and the posture of the upper body using the foot pressure
distribution is R2$\fallingdotseq$0.9. The RMSE was estimated at 3.3 $\pm$ 1.3
deg, 8.8 $\pm$ 3.0 deg, 8.4 $\pm$ 2.8 deg, and 3.0 $\pm$ 1.2 deg for the ankle
joint, knee joint, hip joint, and upper body posture, respectively.

Figure 3 summarizes the estimation results for conditions (A), (B), and (C),
respectively. The figure shows the mean and variance of the estimation error
(RMSE and R2) for each experimental condition. From Figure 3, the results for
Condition (A) have a lower RMSE compared with the other results. Additionally,
the results for Condition (A) have a higher R2 compared with the other
results.

The results obtained by the statistical analysis of the ankle joint, knee
joint, hip joint, and upper body posture are shown in Fig. 3. The results
obtained by the statistical analysis of the estimation results for the ankle
joint, knee joint, and hip joint are discussed below. For all estimations,
there are significant differences in the RMSE and R2 between condition (A) and
condition (B). When the rubber plate was interposed at the foot, the
estimation accuracy was significantly lower. Similarly, there are significant
differences between condition (A) and condition (C) in all estimations.
Therefore, the estimation accuracy was significantly reduced when the plastic
plate was interposed at the foot. The results obtained by the statistical
analysis of the upper body posture are discussed below. The difference between
condition (A) and condition (B) is significant for R2 (p=0.03), but not for
RMSE (p=0.12). The difference between condition (A) and condition (C) is
significant for R2 (p=0.008), but not for RSME (p=0.07).

Figure 2:  Representative example of estimation results for: (a) ankle joint
angle, (b) knee joint angle, (c) hip joint angle, and (d) upper body posture
under experimental condition (A). This figure shows only the range of
validation data. For comparison, the plots for each output (a)–(d) are
overlaid with the time series of the measurement data $\theta$ (red line) and
estimated data $\hat{\theta}$ (blue line). These results reveal the high
estimation accuracy for the joint angles and posture using the foot pressure
distribution.  Figure 3:  Results for error (RMSE and R2) in estimation of (a)
ankle joint angle, (b) knee joint angle, (c) hip joint angle, and (d) upper
body posture under each experimental condition. The figure also shows the
statistical analyses performed on the RSME and R2 results for the data
obtained under experimental Condition (A) and (B), and the data obtained under
experimental Conditions (A) and (C). The statistical significance was set to
p$<$0.05. The difference in the measurement accuracy between cases with and
without an intervening object is significant, with lower estimation accuracy
for the rubber plate (Condition (B)) and plastic plate (Condition (C))
compared with the case without an intervening object (Condition (A)), which is
true for all results except the RSME of body posture estimation.

##  IV  Discussion

A highly accurate estimation of R2$\fallingdotseq$0.9 was achieved in the
estimation of the joint angles and posture using foot pressure distribution
information. These results reveal that the posture and joint angles of the
human body can be estimated with high accuracy from foot pressure distribution
information using multivariate linear regression. The estimation accuracy
achieved by a previous study that used image-based estimation for squatting
motion similar to that in this study are as follows: upper body posture error:
2.8 $\pm$ 0.6 deg; knee joint angle error: 8.0 $\pm$ 1.4 deg [23]. The
estimation results obtained by the proposed method are comparable. Therefore,
the proposed method can estimate joint angles and posture with sufficient
accuracy for some applications.

To estimate multiple postures and joint angles, the proposed method only
requires that a pressure distribution sensor is attached to the plantar part
of the foot, which is the end of the body in contact with the environment.
Additionally, the proposed method is characterized by its ability to estimate
multiple angular information of the human body from a single source, that is,
foot pressure distribution information, and can also obtain information for
locations away from the foot, such as the angle of the knee joint, hip joint,
and upper body. Additionally, sensors are only required for the foot sole, and
there is no need to attach sensors near the joints of the human body.
Generally, when using contact and wearable sensors, multiple sensors must be
placed near each part of the body to estimate multiple joint angles and
postures. A distinctive feature absent in other contact and wearable sensing
methods is the ability to estimate joint angles at a distance from the sensor,
eliminating the need to place sensors outside of the foot.

The statistical analysis results indicate that the plantar morphology
contributes to the joint angle and posture estimation. There is a difference
in the measurement accuracy between cases with and without an intervening
object. Specifically, the estimation accuracy is lower for the rubber plate
and plastic plate. The rubber plate is considered to have the effect of
equalizing the hardness distribution in the plantar region, while the plastic
plate is considered to make the structure of the plantar region
inconsequential, giving it a morphology resembling a link structure. These
results suggest that the morphology (structure and characteristics)
information of the foot is required to perform the proposed estimation. In the
field of biological systems, it is important to understand how the human body
functions in terms of perception and movement. The plantar region is exposed
to the environment and receives various information from it [24]. The plantar
region perceives motion and plays a role in motor control [25], and plantar
skin information is known to play an important role in human postural control
[26]. The morphology of the human foot is very complex [27], and the extent to
which the foot sensory system is involved in motor control remains unclear
[24]. The results of this study are interesting from the perspective of
morphological computation. In this field, the morphology of the human body is
considered to have certain computational functions [28]. The results of this
study reveal that the joint angle and posture estimation from foot pressure
distribution information ceases to function when the ground contact conditions
change (Condition (B) and (C)). Therefore, the morphology of the plantar
region functions as a type of computational function and contributes to the
estimation of the joint angle and posture. Here, the complex morphology of the
foot plays an important role in the generation of the complex and diverse
information of the foot pressure distribution, and this complex and diverse
information is required for estimation. Moreover, the results indicate that it
is possible for the complex morphology of the human foot to function as
morphological computation and serve as a computational resource for human
perception and motor control. Notably, however, these conclusions do not have
a physiological and anatomical basis, and are only based on empirical and
indirect data.

In recent years, studies on physical reservoir computing for a soft body,
which considers the dynamics of soft materials as information processing
devices, such as recurrent neural networks, have reported the possibility of
using various physical phenomena as computational resources [29, 30]. Some
studies have investigated the computational capabilities of the
musculoskeletal system [31], used the soft spine as a physical reservoir to
generate the motion of a quadruped robot [32], processed information using a
soft body such as an octopus [33, 34, 35, 36], and used soft material bodies
as computational resources. Another study proposed a soft wearable suit with
tactile sensors, which can be considered as a tactile sensor network for
monitoring natural body dynamics, as a computational resource for estimating
human and robot posture [37]. In physical reservoir computing, the
intermediate layer uses nonlinear physical states, and the output layer is a
linear and static readout. Then, only the output layer is learned. The output
layer and learning process used in physical reservoir computing are the same
as those in the multivariate linear regression model and its learning method
used in this study. Therefore, the method proposed herein can be considered as
an information processor that introduces the concept of physical reservoir
computing and uses the human foot as a computational resource to estimate the
joints and posture. In other words, the foot dynamics are considered as a
physically implemented recurrent neural network, and the joint angles and
posture are estimated from the time series data of the foot state (pressure
sensor data) using only linear and static readouts (multivariate linear
regression model). This concept, which uses the human foot as a computational
resource, contributes toward highly accurate estimation with a simple model
and learning method, and toward the significant reduction of training and
computation cost. We name these types of research area “biomechanical
reservoir computing,” which utilizes the physical dynamics occurring in the
human (organisms) interior and in contact with surrounding environment as a
computational resource, and are promoting researches such as [38] using this
framework.

Consideration should be given to the information processing methods of this
study in terms of their relationship with mechanoreceptors. For the
information processing in this study, a very simple model was used as the
multivariate linear regression model, in which the time series data for each
pressure are weighted and added together. Notably, because all pressure data
are standardized, the pressure magnitude at a given location does not affect
the estimation accuracy, which is only affected by the waveform pattern of the
time series data. Figure 4(b) shows a representative example of a map of the
absolute value of weight $\omega$ in the multivariate linear regression model
for each pressure sensor location. Here, when the absolute value of $\omega$
in the estimation method is large, the information at the pressure sensor
location makes a large contribution to the estimation. Figure 4 shows that the
weight $\omega$ is large at locations where mechanoreceptors are considered to
be abundant in the plantar region, such as the thumb and base of the five toes
[39, 40]. Figure 4(a) shows information for locations where mechanoreceptors
are considered to be abundant in the plantar region [39, 40]. By comparing
Figure 4(a) and Figure 4(b), it can be seen that the locations where the
weights of the information in the estimation method are large, and the
locations where there are many mechanoreceptors in the plantar region, are
qualitatively consistent to some extent. This suggests that, in the perceptual
processing of organisms, information may be processed by a mechanism
resembling that of the proposed estimation method. In other words, it is
hypothesized that, in the perceptual processing of living organisms, the joint
angles and posture are estimated as proprioceptive sensations by integrating
the local time-series patterns of pressure at locations where there are many
mechanoreceptors in the plantar region. Notably, however, these conclusions
are not based on quantitative investigation or physiological data, and there
is only qualitative agreement regarding the location.

Figure 4:  (a) Locations where mechanoreceptors are considered to be abundant
in the plantar region and (b) representative example of map of absolute value
of weight $\omega$ in multivariate linear regression model for each pressure
sensor location. By comparing (a) and (b), the locations where the information
weights in the estimation method are large, and the locations where there are
many mechanoreceptors in the plantar region, are qualitatively consistent to
some extent.

Hence, the proposed framework may also help in elucidating the role of the
human body and other living organisms. Morphological computation and physical
reservoir computing with a soft body can be considered as an attempt to
harness the inherent computational capabilities of living organisms.
Investigating which computational tasks can be performed using living
organisms will provide new perspectives for understanding questions such as
why living organisms have such specialized morphology. Whether living bodies
can serve as morphological computation or physical reservoir computing, and
whether they can serve as computational resources for biological perception
and motor control has been a matter of debate [38, 41]. The findings obtained
by this study can provide interesting research directions for further
investigation.

The proposed method for estimating posture and joint angles using foot
pressure distribution is expected to find application in the estimation of the
angle of each joint during walking. By using only one plantar pressure
distribution sensor installed in the shoe, it may be possible to estimate the
angles of multiple joints during walking, eliminating the need for additional
sensors. However, as the foot lifts off the ground during the stance and the
ground contact conditions between the plantar area and the pressure
distribution sensor change, the learner may need to change depending on the
gait phase.

The experimental results in Fig.2 reveal that the waveform patterns of the
ankle, knee, and hip joints are similar during squatting movement. Therefore,
it is possible that there is an excess of joint types that can be estimated.
Future work should investigate whether similar results can be obtained for
non-natural movement with different waveform patterns.

Notably, because the surface material of the pressure distribution sensor used
in this study was flexible, even when a plastic plate was interposed between
the foot and the sensor, a shift in the pressure distribution caused by
plastic plate tilt was observed. This flexibility of the pressure distribution
sensor may be a factor affecting the measurement accuracy when a plastic plate
exists, and this may have influenced the results of the statistical analysis
of the upper body posture, which did not reveal significant differences in the
RMSE between condition (A) and condition (B), and between condition (A) and
condition (C).

## data availability

The datasets generated in this study are available from the corresponding
author upon reasonable request. The data are not publicly available to protect
the privacy of the research participants.

###### Acknowledgements.

This study was supported in part by a Grant-in-Aid for Scientific Research
from the Ministry of Education, Culture, Sports, Science and Technology (MEXT)
(19H02112, 19K22878, and 23K18479), Japan. This study was supported in part by
JST, PRESTO (Grant Number JPMJPR23P2), Japan. This study was supported in part
by Innovation Inspired by Nature, Research Support Program, SEKISUI CHEMICAL
CO., LTD.

## References

  * [1] Cole Woods and Vishesh Vikas.  Joint angle estimation using accelerometer arrays and model-based filtering.  IEEE Sensors Journal, 22(20):19786–19796, 2022.
  * [2] R Williamson and BJ Andrews.  Detecting absolute human knee angle and angular velocity using accelerometers and rate gyroscopes.  Medical and Biological Engineering and Computing, 39:294–302, 2001.
  * [3] Ilias El Makrini, Glenn Mathijssen, Sten Verhaegen, Tom Verstraten, and Bram Vanderborght.  A virtual element-based postural optimization method for improved ergonomics during human-robot collaboration.  IEEE Transactions on Automation Science and Engineering, 19(3):1772–1783, 2022.
  * [4] Vijaykumar Rajasekaran, Joan Aranda, Alicia Casals, and Jose L Pons.  An adaptive control strategy for postural stability using a wearable robot.  Robotics and Autonomous Systems, 73:16–23, 2015.
  * [5] Taravat Anvari and Kyoungju Park.  3d human body pose estimation in virtual reality: A survey.  In 2022 13th International Conference on Information and Communication Technology Convergence (ICTC), pages 624–628, 2022.
  * [6] LI Yang, Jin Huang, TIAN Feng, WANG Hong-An, and DAI Guo-Zhong.  Gesture interaction in virtual reality.  Virtual Reality & Intelligent Hardware, 1(1):84–112, 2019.
  * [7] Vincent Bonnet, Vladimir Joukov, Dana Kulić, Philippe Fraisse, Nacim Ramdani, and Gentiane Venture.  Monitoring of hip and knee joint angles using a single inertial measurement unit during lower limb rehabilitation.  IEEE Sensors journal, 16(6):1557–1564, 2015.
  * [8] Mahmoud El-Gohary and James McNames.  Shoulder and elbow joint angle tracking with inertial sensors.  IEEE Transactions on Biomedical Engineering, 59(9):2635–2641, 2012.
  * [9] Mahmoud El-Gohary and James McNames.  Human joint angle estimation with inertial sensors and validation with a robot arm.  IEEE Transactions on Biomedical Engineering, 62(7):1759–1767, 2015.
  * [10] Sumit Majumder and M Jamal Deen.  Wearable imu-based system for real-time monitoring of lower-limb joints.  IEEE sensors journal, 21(6):8267–8275, 2020.
  * [11] Sumit Majumder and M. Jamal Deen.  A robust orientation filter for wearable sensing applications.  IEEE Sensors Journal, 20(23):14228–14236, 2020.
  * [12] Caitlin N Teague, J Alex Heller, Brandi N Nevius, Andrew M Carek, Samer Mabrouk, Florencia Garcia-Vicente, Omer T Inan, and Mozziyar Etemadi.  A wearable, multimodal sensing system to monitor knee joint health.  IEEE Sensors Journal, 20(18):10323–10334, 2020.
  * [13] Emmanuel Roux, Stéphane Bouilland, A-P Godillon-Maquinghen, and Denis Bouttens.  Evaluation of the global optimisation method within the upper limb kinematics analysis.  Journal of biomechanics, 35(9):1279–1283, 2002.
  * [14] Yumiko Murata and Toshiyuki Murakami.  Estimation of posture and position based on geometric calculation using imus.  In IECON 2019-45th Annual Conference of the IEEE Industrial Electronics Society, volume 1, pages 5388–5393. IEEE, 2019.
  * [15] Greg Welch and Eric Foxlin.  Motion tracking: No silver bullet, but a respectable arsenal.  IEEE Computer graphics and Applications, 22(6):24–38, 2002.
  * [16] Long Chen, Haizhou Ai, Rui Chen, Zijie Zhuang, and Shuang Liu.  Cross-view tracking for multi-human 3d pose estimation at over 100 fps.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3279–3288, 2020.
  * [17] Ikhsanul Habibie, Weipeng Xu, Dushyant Mehta, Gerard Pons-Moll, and Christian Theobalt.  In the wild human pose estimation using explicit 2d features and intermediate 3d representations.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10905–10914, 2019.
  * [18] Dushyant Mehta, Oleksandr Sotnychenko, Franziska Mueller, Weipeng Xu, Mohamed Elgharib, Pascal Fua, Hans-Peter Seidel, Helge Rhodin, Gerard Pons-Moll, and Christian Theobalt.  Xnect: Real-time multi-person 3d motion capture with a single rgb camera.  Acm Transactions On Graphics (TOG), 39(4):82–1, 2020.
  * [19] Matthew Trumble, Andrew Gilbert, Adrian Hilton, and John Collomosse.  Deep convolutional networks for marker-less human pose estimation from multiple views.  In Proceedings of the 13th European conference on visual media production (CVMP 2016), pages 1–9, 2016.
  * [20] Donglai Xiang, Hanbyul Joo, and Yaser Sheikh.  Monocular total capture: Posing face, body, and hands in the wild.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10965–10974, 2019.
  * [21] Xinyu Yi, Yuxiao Zhou, and Feng Xu.  Transpose: Real-time 3d human translation and pose estimation with six inertial sensors.  ACM Transactions on Graphics (TOG), 40(4):1–13, 2021.
  * [22] John Z Edwards, Kenneth A Greene, Robert S Davis, Mark W Kovacik, Donald A Noe, and Michael J Askew.  Measuring flexion in knee arthroplasty patients.  The Journal of arthroplasty, 19(3):369–372, 2004.
  * [23] Po-Chieh Lin, Yu-Jung Chen, Wei-Shin Chen, and Yun-Ju Lee.  Automatic real-time occupational posture evaluation and select corresponding ergonomic assessments.  Scientific Reports, 12(1):2139, 2022.
  * [24] Peter F Meyer, Lars IE Oddsson, and Carlo J De Luca.  The role of plantar cutaneous sensation in unperturbed stance.  Experimental brain research, 156:505–512, 2004.
  * [25] Ryota Okoba, Masaki Hasegawa, Hisayoshi Yoshizuka, Yuichi Honda, Masayoshi Ichiba, and Toyoko Asami.  Particular protrusion perception arising from plantar sensory input and task guidance enhances lower limb joint dynamics during gait.  Journal of Physical Therapy Science, 31(3):261–266, 2019.
  * [26] Anne Kavounoudias, Régine Roll, and Jean-Pierre Roll.  The plantar sole is a ‘dynamometric map’for human balance control.  Neuroreport, 9(14):3247–3252, 1998.
  * [27] Zhangqi Lai, Xiaoyue Hu, Li Xu, Kun Dong, and Lin Wang.  Evaluating the function of the foot core system in the elderly.  JoVE (Journal of Visualized Experiments), (181):e63479, 2022.
  * [28] Pfeifer Rolf and Bongard Josh.  How the Body Shapes the Way We Think.  The MIT Press, 2006.
  * [29] Gouhei Tanaka, Toshiyuki Yamane, Jean Benoit Héroux, Ryosho Nakane, Naoki Kanazawa, Seiji Takeda, Hidetoshi Numata, Daiju Nakano, and Akira Hirose.  Recent advances in physical reservoir computing: A review.  Neural Networks, 115:100–123, 2019.
  * [30] Kohei Nakajima.  Physical reservoir computing—an introductory perspective.  Japanese Journal of Applied Physics, 59(6):060501, 2020.
  * [31] Hidenobu Sumioka, Helmut Hauser, and Rolf Pfeifer.  Computation with mechanically coupled springs for compliant robots.  In 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 4168–4173. IEEE, 2011.
  * [32] Qian Zhao, Kohei Nakajima, Hidenobu Sumioka, Xiaoxiang Yu, and Rolf Pfeifer.  Embodiment enables the spinal engine in quadruped robot locomotion.  In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 2449–2456. IEEE, 2012.
  * [33] Kohei Nakajima, Helmut Hauser, Tao Li, and Rolf Pfeifer.  Information processing via physical soft body.  Scientific reports, 5(1):10487, 2015.
  * [34] Kohei Nakajima, Tao Li, Helmut Hauser, and Rolf Pfeifer.  Exploiting short-term memory in soft body dynamics as a computational resource.  Journal of The Royal Society Interface, 11(100):20140437, 2014.
  * [35] Kohei Nakajima, Helmut Hauser, Rongjie Kang, Emanuele Guglielmino, Darwin G Caldwell, and Rolf Pfeifer.  A soft body as a reservoir: case studies in a dynamic model of octopus-inspired soft robotic arm.  Frontiers in computational neuroscience, 7:91, 2013.
  * [36] Kohei Nakajima, Helmut Hauser, Tao Li, and Rolf Pfeifer.  Exploiting the dynamics of soft materials for machine learning.  Soft robotics, 5(3):339–347, 2018.
  * [37] Hidenobu Sumioka, Kohei Nakajima, Kurima Sakai, Takashi Minato, and Masahiro Shiomi.  Wearable tactile sensor suit for natural body dynamics extraction: case study on posture prediction based on physical reservoir computing.  In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 9504–9511. IEEE, 2021.
  * [38] Yo Kobayashi.  Information processing via human soft tissue.  arXiv preprint arXiv:2305.14366, 2023.
  * [39] Mario Bizzini.  Sensomotorische Rehabilitation nach Beinverletzungen: mit Fallbeispielen in allen Heilungsstadien.  Thieme Georg Verlag, 2000.
  * [40] Kirsten Götz-Neumann.  Gehen verstehen: Ganganalyse in der Physiotherapie.  Thieme, 2006.
  * [41] Tetsushi Nonaka.  Locating the inexhaustible: Material, medium, and ambient information.  Frontiers in psychology, 11:447, 2020.