# Reconstructing 3D Human Pose by Watching Humans in the Mirror

Qi Fang Qing Shuai∗ Junting Dong Hujun Bao Xiaowei Zhou†
State Key Lab of CAD&CG Equal Contribution. †Corresponding author. Zhejiang
University

###### Abstract

In this paper, we introduce the new task of reconstructing 3D human pose from
a single image in which we can see the person and the person’s image through a
mirror. Compared to general scenarios of 3D pose estimation from a single
view, the mirror reflection provides an additional view for resolving the
depth ambiguity. We develop an optimization-based approach that exploits
mirror symmetry constraints for accurate 3D pose reconstruction. We also
provide a method to estimate the surface normal of the mirror from vanishing
points in the single image. To validate the proposed approach, we collect a
large-scale dataset named Mirrored-Human, which covers a large variety of
human subjects, poses and backgrounds. The experiments demonstrate that, when
trained on Mirrored-Human with our reconstructed 3D poses as pseudo ground-
truth, the accuracy and generalizability of existing single-view 3D pose
estimators can be largely improved. The code and dataset are available at
https://zju3dv.github.io/Mirrored-Human/.

##  1 Introduction

3D human pose estimation has ubiquitous applications in sport analysis, human-
computer interaction, and fitness and dance teaching. While there has been
remarkable progress in 3D pose estimation from a monocular image or video [22,
37, 25, 24, 59], inevitable challenges such as the depth ambiguity and the
self-occlusion are still unsolved.

Input image

SMPLify-X [42]

3D visualization of our (left) and SMPLify-X (right) results

Figure 1: While the state-of-the-art single-view 3D pose estimator [42] yields
a small reprojection error, the recovered 3D poses may be erroneous due to the
depth ambiguity. We make use of the mirror in the image to resolve the
ambiguity and reconstruct more accurate human pose as well as the mirror
geometry.

In many scenes like dancing rooms and gyms, people are often in front of a
mirror. In this case, we are able to see the person and his/her mirror image
simultaneously. The mirror image actually provides an additional virtual view
of the person, which can resolve the single-view depth ambiguity if the mirror
is properly placed. Moreover, unseen part of the person can also be observed
from the mirror image, so that the occlusion problem can be alleviated.

In this paper, we investigate the feasibility of leveraging such mirror images
to improve the accuracy of 3D human pose estimation. We develop an
optimization-based framework with mirror symmetry constraints that are
applicable without knowing the mirror geometry and camera parameters. We also
provide a method to utilize the properties of vanishing points to recover the
mirror normal along with the camera parameters, so that an additional mirror
normal constraint can be imposed to further improve the human pose estimation
accuracy. The effectiveness of our framework is validated on a new dataset for
this new task with 3D pose ground-truth provided by a multi-view camera
system.

An important application of the proposed approach is to generate pseudo
ground-truth annotations to train existing 3D pose estimators. To this end, we
collect a large-scale set of Internet images that contain people and mirrors
and generate 3D pose annotations with the proposed optimization method. The
dataset is named Mirrored-Human. Compared with existing 3D human pose datasets
[19, 33, 56] that are captured with very few subjects and background scenes,
Mirrored-Human has a significantly larger diversity in human poses,
appearances and backgrounds, as shown in Fig. 7. The experiments show that, by
combining Mirrored-Human with existing datasets as training data, both
accuracy and generalizability of existing 3D pose estimation methods can be
significantly improved for both single-person and multi-person cases.

In summary, we make the following contributions:

  * •

We introduce a new task of reconstructing human pose from a single image in
which we can see the person and the person’s mirror image.

  * •

We develop a novel optimization-based framework with mirror symmetry
constraints to solve this new task, as well as a method to recover mirror
geometry from a single image.

  * •

We collect a large-scale dataset named Mirrored-Human from the Internet,
provide our reconstructed 3D poses as pseudo ground-truth, and show that
training on this new dataset can improve the performance of existing 3D human
pose estimators.

##  2 Related work

Figure 2: Overview of our approach. Given the input image (a), we first
estimate the 2D keypoints and SMPL parameters as the initialization (b). Then,
we minimize the reprojection error with mirror symmetry constraints for
reconstruction (c). We collect a considerable number of Internet images (d)
and build a dataset named Mirrored-Human (e) with pseudo ground-truth
generated by our framework. The dataset can be used for the training of
single-view methods.

###  2.1 3D human pose

Benefiting from neural networks, the task of monocular 3D human pose
estimation has made considerable progress. The skeleton-based approaches
either lift the 2D pose to 3D [32, 7, 57], or adopt an end-to-end manner to
regress the 3D pose directly from the image [51, 52, 67, 43]. The model-based
approaches estimate the pose and shape simultaneously with parametric models
[2, 31, 41, 60]. Approaches along this line can be divided into two
categories. Optimization-based methods fit the model using 2D evidence and
some human body priors [5, 28, 42, 11]. Regression-based methods directly
regress the model parameters from the image [22, 40, 61, 23, 64]. Kolotouros
et al. [25] incorporate their advantages and propose a self-improving
framework. To relax the heavy reliance on the model’s parameter space, model-
free approaches directly regress the 3D locations of the mesh vertices [26,
37, 9]. For multi-person cases, previous works focus on how to extend single-
person frameworks to multi-person ones [47, 34] or model the interaction
between people [63, 14]. Some recent works [36, 66, 29, 30, 12] explore the
representation of the absolute depth in the camera coordinate system.

In monocular settings, the depth ambiguity is inevitable. One solution is to
utilize the additional supervision signal. Pavlakos et al.[43] use the ordinal
depths of human joints to weakly supervise the network. Kanazawa et al.[22]
train a discriminator network to judge if the estimated pose is reasonable.
Some other works use temporal constraints [24, 55, 44] or geometric self-
consistency [8]. Another line of work resolves the ambiguity with scene
layouts. Hassan et al.[17] exploit static 3D scene structures with the inter-
penetration and the contact constraints. Others [34, 63] integrate the ground
plane information to recover the 3D location and pose. We use the additional
view provided by the mirror reflection to resolve the depth ambiguity.

Learning-based methods are inseparable from the training data. Existing
widely-used 3D datasets such as Human3.6M [19], Human Eva [48], MPI-INF-3DHP
[33] and Panoptic Studio [21] are built with motion capture systems and thus
have limited appearance and pose diversity. 3DPW [56] is a recent dataset
consisting of ordinary activities. The combination of a camera and several
IMUs attached to the human body provides accurate 3D poses, but the dataset
still lacks diversity. Kanazawa et al. [23] contribute some Internet datasets
which however lack 3D annotations. Arnab et al. [3] propose a bundle-
adjustment-based algorithm, based upon which they generate 3D annotations for
Internet data. However, this method still suffers from the depth ambiguity
derived from monocular videos. To summarize, a large-scale Internet dataset
with 3D annotations is still missing. We collect numerous Internet images
containing mirrors and people with a large diversity in appearances and poses,
and build a dataset using our reconstructed 3D poses as pseudo ground-truth,
to address this problem to some extent.

###  2.2 Reconstruction with mirrors

Earlier years have witnessed some researches on the mirror geometry of a
catadioptric system (mirrors + lenses), especially for reconstruction and
extrinsic camera calibration [54, 45]. For 3D reconstruction that is more
relevant to our task, some works use the configuration of two planar mirrors
to capture stereo images [38, 15, 53, 27] or produce four virtual views by
assuming there is a one-time interreflection between two mirrors [62]. They
calibrate the mirror based on the image correspondences or silhouettes. Nguyen
et al. [39] use a depth sensor and at least two mirrors, trying to remove
depth distortions. Akay et al. [1] reconstruct 3D object with a mirror and
RGBD cameras. Hu et al. [18] only use one mirror and one RGB camera, but they
need to label the object and mirror region. To the best of our knowledge, the
task of exploiting mirror symmetry to recover 3D human pose and shape from an
Internet image has not been discussed in literature.

Furthermore, not only does the symmetry property exist in scenes with mirrors,
but it also appears on symmetrical objects, which has been explored to
reconstruct faces, cars, etc. Some works [49, 20] explicitly detect the plane
of symmetry with 2D correspondences and others [58] implicitly use the
symmetry prior.

##  3 Methods

Fig. 2 presents the pipeline of our framework. Given an image that contains a
person and a mirror, our goal is to recover the human mesh considering the
mirror geometry. The key insight is that the person and his/her mirror image
can be treated as two people, and we reconstruct them together with the mirror
symmetry constraints. This section will be organized as follows. First, the
formulation of single-person mesh recovery is introduced (Sec. 3.1). Then the
mirror symmetry constraints that relate the two people will be elaborated
(Sec. 3.2). Finally, the objective functions and the whole optimization are
described (Sec. 3.3).

###  3.1 Human mesh recovery with SMPL model

We adopt the SMPL model [31] as our human representation. The SMPL model is a
differentiable function $\bm{M}(\bm{\theta},\bm{\beta})\in\mathbb{R}^{3\times
N_{v}}$ mapping the pose parameters $\bm{\theta}\in\mathbb{R}^{72}$ and the
shape parameters $\bm{\beta}\in\mathbb{R}^{10}$ to a triangulated mesh with
$N_{v}=6890$ vertices. The 3D body joints $\bm{J}(\bm{\theta},\bm{\beta})$ of
the model can be defined as a linear combination of the mesh vertices. Hence
for $N_{j}$ joints, we defined the body joints
$\bm{J}(\bm{\theta},\bm{\beta})\in\mathbb{R}^{3\times
N_{j}}=\mathcal{J}(\bm{M}(\bm{\theta},\bm{\beta}))$, where $\mathcal{J}$ is a
pre-trained linear regressor. Let $\bm{R}\in SO(3)$ and
$\bm{T}\in\mathbb{R}^{3}$ denote the global rotation and translation,
respectively.

Given an image and the detected 2D bounding boxes, the 2D human keypoints
$\bm{W}$ can be estimated with the cropped regions. The objective function for
human mesh recovery generally consists of a reprojection term $L_{2d}$ and a
prior term $L_{p}$ with respect to variables $\bm{\theta}$, $\bm{\beta}$,
$\bm{R}$ and $\bm{T}$.

The reprojection term penalizes the weighted 2D distance between the estimated
2D keypoints $\bm{W}$ with the confidence $c$, and the corresponding projected
SMPL joints:

$L_{2d}=\sum_{i}c_{i}\rho(\bm{W}_{i}-\Pi_{K}(\bm{R}\bm{J}(\bm{\theta},\bm{\beta})_{i}+\bm{T})),$
(1)

where $\Pi_{K}$ is the projection from 3D to 2D through the intrinsic
parameter $K$. $\rho$ denotes the Geman-McClure robust error function for
suppressing noisy detections.

The human body priors are used to encourage realistic 3D human mesh results.
Since the pose and shape parameters ($\bm{\tilde{\theta}},\bm{\tilde{\beta}}$)
estimated by a neural network can be viewed as learned prior, the final
results are supposed to be close to them:

$L_{p}=||\bm{\theta}-\bm{\tilde{\theta}}||_{2}^{2}+\lambda_{\beta}||\bm{\beta}-\bm{\tilde{\beta}}||_{2}^{2},$
(2)

where $\lambda_{\beta}$ is a weight.

###  3.2 Mirror-induced constraints

If there is a mirror in the image, the relation between the person and the
mirrored person can be used to enhance the reconstruction performance. This
relation is a simple reflection transformation if the mirror geometry is
known, which however is impracticable for an arbitrary image from the
Internet. To tackle this problem and take advantage of the characteristic of
the mirror, the following mirror-induced constraints are introduced, as
illustrated in Fig. 3. Note that all symbols with the superscript prime refer
to variables related to the mirrored person unless specifically mentioned.

#### Mirror symmetry constraints:

Since the adopted human representation disentangles the orientation $\bm{R}$,
pose parameters $\bm{\theta}$ and shape parameters $\bm{\beta}$, $\bm{\beta}$
can be shared by the person and the mirrored person, and $\bm{\theta}$ is
related to $\bm{\theta}^{\prime}$ by a simple reflection operation as follows:

$\bm{\beta}^{\prime}=\bm{\beta},~{}\bm{\theta}^{\prime}=\mathcal{S}(\bm{\theta}),$
(3)

where $\mathcal{S}(\cdot)$ denotes the reflection operation on axis angles.

Figure 3: An illustration of mirror-induced constraints. The line segment
connecting the joint $\bm{J}_{i}$ and its mirrored joint $\bm{J}_{i}^{\prime}$
has the direction $\bm{n}_{i}$ and the middle point $\bm{p}_{i}$.
Theoretically, $\bm{n}_{i}//\bm{n}_{j}$, and
$\bm{n}_{i}\perp\overline{\bm{p}_{i}\bm{p}_{j}}$. If the mirror normal
$\bm{n}$ (red arrow) is known, $\bm{n}//\bm{n}_{i}$ and $\bm{n}//\bm{n}_{j}$
should be satisfied as well.

As Eq. 3 does not take $\bm{R}$ and $\bm{T}$ into consideration, the
constraint on 3D keypoints can be imposed to estimate the human orientation
and position better. We abbreviate the global coordinates of the $i$-th joint
$\bm{R}\bm{J}(\bm{\theta},\bm{\beta})_{i}+\bm{T}$ as $\bm{J}_{i}$. Given a
pair of body joints $i,j$, we denote the direction of the line segment
$\overline{\bm{J}_{i}\bm{J}_{i}^{\prime}}$,
$\overline{\bm{J}_{j}\bm{J}_{j}^{\prime}}$ as $\bm{n}_{i}$, $\bm{n}_{j}$ and
the middle point of them as $\bm{p}_{i}$, $\bm{p}_{j}$, respectively. Ideally,
$\bm{n}_{i}$ should be parallel to $\bm{n}_{j}$ and $\bm{p}_{i},\bm{p}_{j}$
are supposed to be on the mirror plane. Despite the fact that the mirror
geometry is unknown, it needs to be satisfied that $\bm{n}_{i}$ is
perpendicular to the line $\overline{\bm{p}_{i}\bm{p}_{j}}$. So for any pair
of joints, we minimize the sum of the L2 norm of the cross product between
$\bm{n}_{i}$ and $\bm{n}_{j}$, and the inner product between $\bm{n}_{i}$ and
$\bm{p}_{j}-\bm{p}_{i}$:

$L_{s}=\sum_{(i,j)}(||\bm{n}_{i}\times\bm{n}_{j}||_{2}+||\bm{n}_{i}\cdot(\bm{p}_{j}-\bm{p}_{i})||_{2}).$
(4)

#### Mirror normal constraint:

A mirror can be represented as a plane, parameterized as its normal and
position. If its normal $\bm{n}$ is known, the geometric properties of the
mirror can thus be utilized explicitly by constraining $\bm{n}_{i}$ and
$\bm{n}$ to be parallel with the following loss function:

$L_{n}=\sum_{i}||\bm{n}\times\bm{n}_{i}||_{2}.$ (5)

Figure 4: Vanishing points in an image containing a person and a mirror. In
most cases at least two vanishing points can be found, where $\bm{v}_{0}$
comes from 2D human keypoints, and $\bm{v}_{1}$ comes from the annotated
mirror edges. $O_{c}$ denotes the camera center. Note that
$\overline{O_{c}v_{0}}//\bm{n}$ and $\overline{O_{c}v_{1}}\perp\bm{n}$, where
$\bm{n}$ is the mirror normal.

#### Mirror normal estimation:

Though the mirror normal is not directly available, the vanishing points can
be used to estimate it. The vanishing point of lines with direction $\bm{n}$
in 3D space is the intersection $\bm{v}$ of the image plane with a ray through
the camera center with direction $\bm{n}$ [16]:

$\bm{v}=K\bm{n},$ (6)

where the vanishing point $\bm{v}\in\mathbb{R}^{3}$ is in the form of
homogeneous coordinates and $K$ is the camera intrinsic matrix.

Eq. 6 reveals that obtaining the mirror normal $\bm{n}$ requires both $K$ and
$\bm{v}$. As the parallel lines connecting points on the real object and
corresponding points on the mirrored object are perpendicular to the mirror,
the vanishing point $\bm{v}$ with this direction can be estimated through
their 2D positions. To get such correspondences, some previous works require
additional inputs such as masks [18], which is infeasible for images from the
Internet. Fortunately, since 2D human keypoints provide robust semantic
correspondences, the left ankle of the real person and the right ankle of the
mirrored person, this vanishing point can be acquired naturally and
automatically in our setting ($\bm{v}_{0}$ in Fig. 4).

Note that if the intrinsic matrix $K$ is provided, the mirror normal can thus
be solved easily through $\bm{n}=K^{-1}\bm{v}_{0}$, otherwise $K$ should be
calibrated from a single image if possible. From the projective geometry [16],
we know that it is possible to calibrate the camera intrinsic parameters from
a single image. Suppose the camera has zero skew and square pixels. The
intrinsic matrix $K$ can be computed via three orthogonal vanishing points.
Additionally, if the principal point is assumed to be in the image center
(only the focal length is unknown), $K$ can be computed via only two
orthogonal vanishing points. Please refer to the supplementary material for
more details.

As we have stated, one vanishing point $\bm{v}_{0}$ has been acquired based on
reliable 2D human keypoints. Different from the general scene where finding
orthogonal relations may be difficult, our setting contains richer
information. Fig. 4 shows that if we annotate the mirror edges, at least one
vanishing point $\bm{v}_{1}$ orthogonal to $\bm{v}_{0}$ can be obtained. With
these vanishing points, the calibration can be performed. Note that images
from the same video share the same intrinsic matrix $K$, thus the annotation
process is not laborious.

The mirror normal constraint is optional, which depends on how easy it is to
find mirror edges. In the experiment, we will show that our method can still
achieve satisfactory performance without the mirror normal constraint.

###  3.3 Objective function and optimization

Combining all discussed above, the final objective function to optimize can be
written as:

$\begin{split}\min_{\begin{subarray}{c}\Theta,\Theta^{\prime}\end{subarray}}~{}L_{2d}+L_{2d}^{\prime}+\lambda_{p}(L_{p}&+L_{p}^{\prime})+\lambda_{s}L_{s}+\lambda_{n}L_{n}\\\
s.t.~{}~{}\bm{\beta}^{\prime}=\bm{\beta}&,~{}\bm{\theta}^{\prime}=\mathcal{S}(\bm{\theta}),\end{split}$
(7)

where $\Theta=\\{\bm{\theta},\bm{\beta},\bm{R},\bm{T}\\}$ and
$\Theta^{\prime}=\\{\bm{\theta}^{\prime},\bm{\beta}^{\prime},\bm{R}^{\prime},\bm{T}^{\prime}\\}$.
$L_{2d}^{\prime}$ and $L_{p}^{\prime}$ refer to the reprojection term and the
prior term of the mirrored person, respectively. $\lambda_{p}$, $\lambda_{s}$
and $\lambda_{n}$ are weights. $\lambda_{n}$ is set to zero whenever the
mirror normal is unavailable. If there are two or more people, the
optimization can be done for each subject separately.

We optimize Eq. 7 with respect to all parameters using L-BFGS and PyTorch. An
off-the-shelf model [25] is adopted to generate the initial estimation. Given
the 2D keypoints [50, 6], $\bm{R}$ and $\bm{T}$ are further optimized by
aligning the initial SMPL model to the 2D keypoints. To improve the robustness
of the initialization, we select the person with smaller reprojection error
and apply the selected pose parameter to the other person after a reflection
operation.

##  4 Evaluation

Figure 5:  The configuration of our evaluation set. The poses of all subjects
are captured with six cameras. The camera extrinsic paramters, camera
intrinsic parameters, and mirror geometry are calibrated.

###  4.1 Dataset for evaluation

Since no dataset exists for our task which contains both mirrors and labeled
3D human keypoints, we collect a new dataset with six calibrated HD cameras,
as shown in Fig. 5. All videos are recorded with a resolution of
1920$\times$1080 pixels at 30 fps. Each person is performing various actions
in front of a large mirror. A calibration board is placed on the mirror, thus
the mirror geometry can be easily determined, which can also be used to
evaluate the estimated mirror geometry. Note that the ground-truth of 3D
keypoints is generated from all views. 2D bounding boxes, 2D keypoints, and
the correspondences between multiple people are annotated manually.

###  4.2 Reconstruction evaluation and ablation study

The 3D keypoints generated by this multi-view system are used as ground-truth
to evaluate the reconstruction accuracy. Metrics include MPJPE, PA-MPJPE, and
MRPE. MPJPE is the distance (mm) between predicted and ground-truth 3D
keypoints after root alignment. PA-MPJPE is calculated with further Procrustes
alignment. MRPE is defined as the distance (mm) between the predicted and the
ground-truth root joint, which evaluates the absolute root position accuracy.

Methods | MPJPE $\downarrow$ | PA-MPJPE $\downarrow$ | MRPE $\downarrow$
---|---|---|---
SMPLify-X [42] | 143.73 | 90.57 | 368.00
SPIN [25] | 109.79 | 67.42 | 167.62
Baseline ([25]+[42]) | 82.92 | 61.47 | 147.44
Ours (w/o $L_{s}$, $L_{n}$) | 53.81 | 34.48 | 108.43
Ours (w/o $L_{n}$) | 39.52 | 33.24 | 101.91
Ours (full) | 38.77 | 32.96 | 93.22

Table 1: Quantitative analysis. ‘Baseline’ uses [25] as initialization and
[42] as the optimization method to fit the body model to 2D keypoints for each
person separately.  Figure 6: Sensitivity of pose error to focal length
estimation. The vertical black line indicates the ground-truth focal length.

#### Pose Reconstruction:

Two previous methods are compared here. SMPLify-X [42] is the state-of-the-art
optimization-based method and SPIN [25] is the state-of-the-art regression-
based method. ‘Baseline’ combines the two methods by using [25] as
initialization and [42] as optimization. In Table 1, comparing ‘Ours (full)’
with first three rows, the result shows that our approach outperforms previous
methods by a large margin.

The ablation study is performed to show the effect of our mirror-induced
constraints. It can be observed in Table 1 that without the mirror normal
constraint (‘Ours (w/o $L_{n}$)’), our method can still perform well,
indicating our applicability in the case where vanishing points are hard to
acquire. If $L_{s}$ is also discarded (‘Ours (w/o $L_{s}$, $L_{n}$)’), MPJPE
will degrade severely while PA-MPJPE has a relatively slight change,
reflecting that the mirror symmetry constraint can adjust the human
orientation effectively. ‘Ours (w/o $L_{s}$, $L_{n}$)’ is better than the
baseline since $\bm{\theta}$ and $\bm{\beta}$ are shared. For MRPE, more
constraints will bring the better improvement and the position error of our
full model is less than 10cm.

We also measure the accuracy of mirror normal estimation. As we have stated,
the ground-truth mirror normal comes from calibration. We use the average
angle between the ground-truth and the estimated normal. The angle is 1.9∘
with the ground-truth focal length, and 4.1∘ with the estimated focal length,
both of which are quite small.

#### Focal length estimation:

To evaluate the accuracy of focal length estimation, we simulate the case in
which only the focal length is unknown and two orthogonal vanishing points are
used. One vanishing point is computed by the lines parallel to the mirror and
the other is from two ways: 1) if we label two lines that are perpendicular to
the mirror from the scene, the error of focal length is 4.9% of the true
value, and 2) if we make use of 2D human keypoints, the error reduces to 3.6%.
It means that the 2D human keypoints provide reliable correspondences that are
helpful to estimate the focal length.

To evaluate the influence of the predicted focal length on pose estimation, we
modify the focal length to different values and calculate the reconstruction
accuracy. As shown in Fig. 6, when the focal length deviates from the ground-
truth by less than 10%, the change of the reconstruction error is less than
5%, showing the stability of our algorithm.

Figure 7: Samples in our Mirrored-Human dataset. From left to right, each
column shows the variety of actions, appearances, viewpoints, poses and
multiple mirrors/people, respectively.

Datasets | Frames(k) | Subjects | Annotations
---|---|---|---
3D | 2D | Internet
InstaVariety [23] | 2100 | 28272 |  |  | ✓
Penn Action [65] | 77 | 2326 |  | ✓ | ✓
Human3.6M [19] | 581 | 11 | ✓ | ✓ |
3DPW [56] | 51 | 18 | ✓ | ✓ |
Mirrored-Human | 1800 | 200+ | ✓* | ✓ | ✓

Table 2: Comparison of relevant datasets in terms of the number of frames,
subjects and annotation types. *Our Mirrored-Human dataset adopts our
reconstructed 3D poses as pseudo ground-truth.

###  4.3 Qualitative comparison

Some representative results are visualized in Fig. 8, showing that our method
can reconstruct accurate 3D human meshes as well as the mirror geometry.
Compared with the monocular human mesh recovery algorithms [42, 25], our
method produces much more consistent positions and orientations (Fig. 8) and
more accurate poses (Fig. 9). More qualitative results can be found in the
supplementary material.

##  5 Learning with the Mirrored-Human dataset

###  5.1 Mirrored-Human

Based on our framework, a large-scale Internet dataset can be built for the
training of single-view tasks. The existing datasets [19, 33] lack the variety
of both appearances and poses, making the training easy to overfit. For multi-
person tasks, collecting data is more difficult . Therefore, previous methods
exploit MuCo [35], a pseudo multi-person dataset composited from MPI-INF-3DHP
[33] by masks, or JTA [13], a synthetic dataset. The gap between these
datasets and the real scene may limit the performance of learning-based
methods.

To alleviate the training data issue, we provide a large-scale Internet
dataset named Mirrored-Human with our framework. Specifically, we collect a
large number of videos from the Internet, in which we can see the person and
the person’s mirror image. Actions cover dancing, fitness, mirror
installation, swing practice, etc. Fig. 7 demonstrates both the appearance and
pose diversity of our dataset. Table 2 shows a thorough comparison between our
dataset and relevant datasets. Please refer to our supplementary material for
more details of our dataset.

###  5.2 Single-person mesh recovery

Methods | 3DPW | Human3.6M
---|---|---
MPJPE $\downarrow$ | PA-MPJPE $\downarrow$ | MPJPE $\downarrow$ | PA-MPJPE $\downarrow$
HMR [22] | - | 81.3 | 88.0 | 56.8
HMMR [23] | - | 72.6 | - | 56.9
Arnab. [3] | - | 72.2 | 77.8 | 54.3
CMR [26] | - | 70.2 | - | 50.1
SPIN [25] | 98.2* | 59.2 | 62.3* | 41.1
MeshNet [37] | 93.2 | 58.6 | 55.7 | 41.7
Baseline | 90.0 | 57.5 | 54.7 | 41.7
[37]+MiHu | 85.1 | 54.8 | 53.6 | 41.0

Table 3: Results on 3DPW and Human3.6M datasets. ‘MiHu’ is our Mirrored-Human
dataset. ‘Baseline’ means training MeshNet [37] on Mirrored-Human with 3D
annotations given by SMPLify-X [42]. *MPJPE of [25] is obtained by their
released model.

For this task, we choose MeshNet [37], a state-of-the-art method for single-
view 3D pose estimation, for evaluation. Two datasets are used for evaluation.
Human3.6M [19] is an indoor benchmark with 3D annotations. 3DPW [56] is an
outdoor benchmark to test the generalization ability and only its defined test
set is used. Following standard protocols, we report both MPJPE and PA-MPJPE.
We also test the baseline method that uses the state-of-the-art optimization-
based method SMPLify-X [42] to generate pseudo ground-truth to train the same
network. Table 3 shows that with our dataset, the performance of MeshNet can
be improved significantly, especially when tested on the 3DPW dataset without
using training data from 3DPW. We also outperform the baseline, indicating
that our framework is more accurate than [42].

###  5.3 Multi-person 3D pose estimation.

| Methods | AP${}_{root}^{25}\uparrow$ | PCK${}_{rel}\uparrow$ | PCK${}_{abs}\uparrow$
---|---|---|---|---
TD | LCRNet [46] | - | 53.8 | -
LCRNet++ [47] | - | 70.6 | -
Dabral. [10] | - | 71.3 | -
PandaNet [4] | - | 72.0 | -
HMOR [29] | - | 82.0 | 43.8
Moon. [36] | 31.0 | 81.8 | 31.5
Moon. [36]+MiHu | 42.2 | 82.3 | 43.0
BU | Mehta. [35] | - | 65.0 | -
Xnect [34] | - | 70.4 | -
SMAP [66] | 37.3 | 73.5 | 35.4
SMAP [66]+MiHu | 42.3 | 74.1 | 38.0

Table 4: Results on the MuPoTS-3D dataset. The numbers are calculated for all
people. ‘MiHu’ is our Mirrored-Human dataset. ‘TD’ and ‘BU’ mean ‘top-down’
and ‘bottom-up’, respectively.

For this task, previous methods fall into two categories. Top-down methods
detect human first and then estimate keypoints with a single-person pose
estimator. Bottom-up methods localize all keypoints in the image first and
then group them into people. We choose the top-down method [36] and the
bottom-up method [66] for evaluation.

The MuPoTS-3D [35] dataset is used. Following previous methods [36, 66],
AP${}_{root}^{25}$, PCKrel and PCKabs are measured. AP${}_{root}^{25}$ is the
average precision of 3D human root location, which treats the prediction as
correct if it lies within 25cm from the ground-truth. PCKrel is the percentage
of correct keypoints after root alignment. A keypoint is correct if the
distance between the prediction and the ground-truth is smaller than 15cm.
PCKabs has almost the same definition as PCKrel, but without the root
alignment it measures the absolute pose accuracy. Note that AP is calculated
only for the root and PCK is for all keypoints.

It can be observed from Table 4 that with our dataset, AP${}_{root}^{25}$ and
PCKabs are improved significantly compared with [36]. For bottom-up methods,
we also improve the performance of [66] apparently.

##  6 Conclusion

In this paper, we present an optimization-based framework that leverages the
mirror reflection to reconstruct accurate 3D human pose. We collect a large-
scale Internet dataset named Mirrored-Human with our reconstructed 3D poses as
pseudo ground-truth and show that training on this dataset can enhance the
performance of existing 3D human pose estimators. Our work opens many new
directions for future research. We plan to extend the method to handle
multiple mirrors, multiple people, multiple frames and more detailed
reconstruction of shape and appearance.

#### Acknowledgement:

The authors would like to acknowledge the support from the National Key
Research and Development Program of China (No. 2020AAA0108901) and NSFC (No.
61806176).

Input SMPLify-X[42] (left) and ours (right) Input SMPLify-X[42] (left) and
ours (right)

Figure 8: Reconstruction results of Internet images. Given the same 2D
keypoints, we compare our method (blue) with SMPLify-X [42] (red). Our method
simultaneously reconstructs precise poses, global locations and mirror
geometry, while SMPLify-X [42] produces inconsistent results due to the depth
ambiguity.

Input Image SMPLify-X[42] SPIN[25] Ours Input Image SMPLify-X[42] SPIN[25]
Ours

Figure 9: Qualitative comparison of 3D pose estimation. For each image, we
show the results of the person estimated by the optimization-based method
SMPLify-X [42] (red), the CNN-based method SPIN [25] (green) and our method
(blue). For each block, top row shows the predicted mesh from the camera view
and the bottom row shows another view. The circles emphasize some
representative differences among three methods. The single-view optimization-
based method produces inaccurate 3D poses despite smaller reprojection error.
Our method produces more accurate 3D outputs given the same input.

## References

  * [1] A. Akay and Y. S. Akgul.  3d reconstruction with mirrors and rgb-d cameras.  In VISAPP, 2014.
  * [2] Dragomir Anguelov, Praveen Srinivasan, Daphne Koller, Sebastian Thrun, Jim Rodgers, and James Davis.  Scape: shape completion and animation of people.  In SIGGRAPH. 2005.
  * [3] Anurag* Arnab, Carl* Doersch, and Andrew Zisserman.  Exploiting temporal context for 3d human pose estimation in the wild.  In CVPR, 2019.
  * [4] Abdallah Benzine, Florian Chabot, Bertrand Luvison, Quoc Cuong Pham, and Catherine Achard.  Pandanet: Anchor-based single-shot multi-person 3d pose estimation.  In CVPR, 2020.
  * [5] Federica Bogo, Angjoo Kanazawa, Christoph Lassner, Peter Gehler, Javier Romero, and Michael J. Black.  Keep it SMPL: Automatic estimation of 3D human pose and shape from a single image.  In ECCV, 2016.
  * [6] Zhe Cao, Tomas Simon, Shih-En Wei, and Yaser Sheikh.  Realtime multi-person 2d pose estimation using part affinity fields.  In CVPR, 2017.
  * [7] Ching-Hang Chen and Deva Ramanan.  3d human pose estimation= 2d pose estimation+ matching.  In CVPR, 2017.
  * [8] Ching-Hang Chen, Ambrish Tyagi, Amit Agrawal, Dylan Drover, Stefan Stojanov, and James M Rehg.  Unsupervised 3d pose estimation with geometric self-supervision.  In CVPR, 2019.
  * [9] Hongsuk Choi, Gyeongsik Moon, and Kyoung Mu Lee.  Pose2mesh: Graph convolutional network for 3d human pose and mesh recovery from a 2d human pose.  In ECCV, 2020.
  * [10] Rishabh Dabral, Nitesh B Gundavarapu, Rahul Mitra, Abhishek Sharma, Ganesh Ramakrishnan, and Arjun Jain.  Multi-person 3d human pose estimation from monocular images.  In 3DV, 2019.
  * [11] Junting Dong, Qing Shuai, Yuanqing Zhang, Xian Liu, Xiaowei Zhou, and Hujun Bao.  Motion capture from internet videos.  In ECCV, 2020.
  * [12] Matteo Fabbri, Fabio Lanzi, Simone Calderara, Stefano Alletto, and Rita Cucchiara.  Compressed volumetric heatmaps for multi-person 3d pose estimation.  In CVPR, 2020.
  * [13] Matteo Fabbri, Fabio Lanzi, Simone Calderara, Andrea Palazzi, Roberto Vezzani, and Rita Cucchiara.  Learning to detect and track visible and occluded body joints in a virtual world.  In ECCV, 2018.
  * [14] M. Fieraru, M. Zanfir, E. Oneata, A. I. Popa, V. Olaru, and C. Sminchisescu.  Three-dimensional reconstruction of human interactions.  In CVPR, 2020.
  * [15] Joshua Gluckman and Shree K Nayar.  Catadioptric stereo using planar mirrors.  IJCV, 2001.
  * [16] Richard Hartley and Andrew Zisserman.  Multiple view geometry in computer vision.  Cambridge university press, 2003.
  * [17] Mohamed Hassan, Vasileios Choutas, Dimitrios Tzionas, and Michael J. Black.  Resolving 3D human pose ambiguities with 3D scene constraints.  In ICCV, 2019.
  * [18] B. Hu, C. M. Brown, and R. Nelson.  Multiple-view 3-d reconstruction using a mirror.  2005\.
  * [19] Catalin Ionescu, Dragos Papava, Vlad Olaru, and Cristian Sminchisescu.  Human3.6m: Large scale datasets and predictive methods for 3d human sensing in natural environments.  IEEE TPAMI, 2014.
  * [20] Nianjuan Jiang, Ping Tan, and Loong-Fah Cheong.  Symmetric architecture modeling with a single image.  In ACM SIGGRAPH Asia. 2009.
  * [21] Hanbyul Joo, Tomas Simon, Xulong Li, Hao Liu, Lei Tan, Lin Gui, Sean Banerjee, Timothy Scott Godisart, Bart Nabbe, Iain Matthews, Takeo Kanade, Shohei Nobuhara, and Yaser Sheikh.  Panoptic studio: A massively multiview system for social interaction capture.  IEEE TPAMI, 2017.
  * [22] Angjoo Kanazawa, Michael J. Black, David W. Jacobs, and Jitendra Malik.  End-to-end recovery of human shape and pose.  In CVPR, 2018.
  * [23] Angjoo Kanazawa, Jason Y. Zhang, Panna Felsen, and Jitendra Malik.  Learning 3d human dynamics from video.  In CVPR, 2019.
  * [24] Muhammed Kocabas, Nikos Athanasiou, and Michael J. Black.  Vibe: Video inference for human body pose and shape estimation.  In CVPR, 2020.
  * [25] Nikos Kolotouros, Georgios Pavlakos, Michael J Black, and Kostas Daniilidis.  Learning to reconstruct 3d human pose and shape via model-fitting in the loop.  In ICCV, 2019.
  * [26] Nikos Kolotouros, Georgios Pavlakos, and Kostas Daniilidis.  Convolutional mesh regression for single-image human shape reconstruction.  In CVPR, 2019.
  * [27] Douglas Lanman, Daniel Crispell, and Gabriel Taubin.  Surround structured lighting: 3-d scanning with orthographic illumination.  CVIU, 2009.
  * [28] Christoph Lassner, Javier Romero, Martin Kiefel, Federica Bogo, Michael J. Black, and Peter V. Gehler.  Unite the people: Closing the loop between 3d and 2d human representations.  In CVPR, 2017.
  * [29] Jiefeng Li, Can Wang, Wentao Liu, Chen Qian, and Cewu Lu.  Hmor: Hierarchical multi-person ordinal relations for monocular multi-person 3d pose estimation.  In ECCV, 2020.
  * [30] Jiahao Lin and Gim Hee Lee.  Hdnet: Human depth estimation for multi-person camera-space localization.  In ECCV, 2020.
  * [31] Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black.  SMPL: A skinned multi-person linear model.  ACM TOG, 2015.
  * [32] Julieta Martinez, Rayat Hossain, Javier Romero, and James J. Little.  A simple yet effective baseline for 3d human pose estimation.  In ICCV, 2017.
  * [33] Dushyant Mehta, Helge Rhodin, Dan Casas, Pascal Fua, Oleksandr Sotnychenko, Weipeng Xu, and Christian Theobalt.  Monocular 3d human pose estimation in the wild using improved cnn supervision.  In 3DV, 2017.
  * [34] Dushyant Mehta, Oleksandr Sotnychenko, Franziska Mueller, Weipeng Xu, Mohamed Elgharib, Pascal Fua, Hans-Peter Seidel, Helge Rhodin, Gerard Pons-Moll, and Christian Theobalt.  Xnect: Real-time multi-person 3d human pose estimation with a single rgb camera.  ACM TOG, 2020.
  * [35] Dushyant Mehta, Oleksandr Sotnychenko, Franziska Mueller, Weipeng Xu, Srinath Sridhar, Gerard Pons-Moll, and Christian Theobalt.  Single-shot multi-person 3d pose estimation from monocular rgb.  In 3DV, 2018.
  * [36] Gyeongsik Moon, Juyong Chang, and Kyoung Mu Lee.  Camera distance-aware top-down approach for 3d multi-person pose estimation from a single rgb image.  In ICCV, 2019.
  * [37] Gyeongsik Moon and Kyoung Mu Lee.  I2l-meshnet: Image-to-lixel prediction network for accurate 3d human pose and mesh estimation from a single rgb image.  In ECCV, 2020.
  * [38] S. A. Nene and S. K. Nayar.  Stereo with mirrors.  In ICCV, 1998.
  * [39] Trong-Nguyen Nguyen, Huu-Hung Huynh, and Jean Meunier.  3d reconstruction with time-of-flight depth camera and multiple mirrors.  IEEE Access, 2018.
  * [40] Mohamed Omran, Christoph Lassner, Gerard Pons-Moll, Peter V. Gehler, and Bernt Schiele.  Neural body fitting: Unifying deep learning and model-based human pose and shape estimation.  In 3DV, 2018.
  * [41] Ahmed A A Osman, Timo Bolkart, and Michael J. Black.  STAR: A spare trained articulated human body regressor.  In ECCV, 2020.
  * [42] Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed A. A. Osman, Dimitrios Tzionas, and Michael J. Black.  Expressive body capture: 3d hands, face, and body from a single image.  In CVPR, 2019.
  * [43] Georgios Pavlakos, Xiaowei Zhou, and Kostas Daniilidis.  Ordinal depth supervision for 3d human pose estimation.  In CVPR, 2018.
  * [44] Sida Peng, Yuanqing Zhang, Yinghao Xu, Qianqian Wang, Qing Shuai, Hujun Bao, and Xiaowei Zhou.  Neural body: Implicit neural representations with structured latent codes for novel view synthesis of dynamic humans.  In CVPR, 2021.
  * [45] Rui Rodrigues, Joao P Barreto, and Urbano Nunes.  Camera pose estimation using images of planar mirror reflections.  In ECCV, 2010.
  * [46] Gregory Rogez, Philippe Weinzaepfel, and Cordelia Schmid.  Lcr-net: Localization-classification-regression for human pose.  In CVPR, 2017.
  * [47] Gregory Rogez, Philippe Weinzaepfel, and Cordelia Schmid.  Lcr-net++: Multi-person 2d and 3d pose detection in natural images.  IEEE TPAMI, 2019.
  * [48] L. Sigal, A. Balan, and M. J. Black.  HumanEva: Synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion.  IJCV, 2010.
  * [49] Sudipta N Sinha, Krishnan Ramnath, and Richard Szeliski.  Detecting and reconstructing 3d mirror symmetric objects.  In ECCV, 2012.
  * [50] Ke Sun, Bin Xiao, Dong Liu, and Jingdong Wang.  Deep high-resolution representation learning for human pose estimation.  In CVPR, 2019.
  * [51] Xiao Sun, Jiaxiang Shang, Shuang Liang, and Yichen Wei.  Compositional human pose regression.  In ICCV, 2017.
  * [52] Xiao Sun, Bin Xiao, Fangyin Wei, Shuang Liang, and Yichen Wei.  Integral human pose regression.  In ECCV, 2018.
  * [53] Tomu Tahara, Ryo Kawahara, Shohei Nobuhara, and Takashi Matsuyama.  Interference-free epipole-centered structured light pattern for mirror-based multi-view active stereo.  In 3DV, 2015.
  * [54] K. Takahashi, S. Nobuhara, and T. Matsuyama.  A new mirror-based extrinsic camera calibration using an orthogonality constraint.  In CVPR, 2012.
  * [55] Shashank Tripathi, Siddhant Ranade, Ambrish Tyagi, and Amit Agrawal.  Posenet3d: Learning temporally consistent 3d human pose via knowledge distillation.  In 3DV, 2020.
  * [56] Timo von Marcard, Roberto Henschel, Michael Black, Bodo Rosenhahn, and Gerard Pons-Moll.  Recovering accurate 3d human pose in the wild using imus and a moving camera.  In ECCV, 2018.
  * [57] Bastian Wandt and Bodo Rosenhahn.  Repnet: Weakly supervised training of an adversarial reprojection network for 3d human pose estimation.  In CVPR, 2019.
  * [58] Shangzhe Wu, Christian Rupprecht, and Andrea Vedaldi.  Unsupervised learning of probably symmetric deformable 3d objects from images in the wild.  In CVPR, 2020.
  * [59] Donglai Xiang, Hanbyul Joo, and Yaser Sheikh.  Monocular total capture: Posing face, body, and hands in the wild.  In CVPR, 2019.
  * [60] Hongyi Xu, Eduard Gabriel Bazavan, Andrei Zanfir, William T Freeman, Rahul Sukthankar, and Cristian Sminchisescu.  Ghum & ghuml: Generative 3d human shape and articulated pose models.  In CVPR, 2020.
  * [61] Yuanlu Xu, Song-Chun Zhu, and Tony Tung.  Denserac: Joint 3d pose and shape estimation by dense render-and-compare.  In ICCV, 2019.
  * [62] Xianghua Ying, Kun Peng, Yongbo Hou, Sheng Guan, Jing Kong, and Hongbin Zha.  Self-calibration of catadioptric camera with two planar mirrors from silhouettes.  IEEE TPAMI, 2012.
  * [63] Andrei Zanfir, Elisabeta Marinoiu, and Cristian Sminchisescu.  Monocular 3d pose and shape estimation of multiple people in natural scenes-the importance of multiple scene constraints.  In CVPR, 2018.
  * [64] Wang Zeng, Wanli Ouyang, Ping Luo, Wentao Liu, and Xiaogang Wang.  3d human mesh regression with dense correspondence.  In CVPR, 2020.
  * [65] Weiyu Zhang, Menglong Zhu, and Konstantinos G Derpanis.  From actemes to action: A strongly-supervised representation for detailed action understanding.  In ICCV, 2013.
  * [66] Jianan Zhen, Qi Fang, Jiaming Sun, Wentao Liu, Wei Jiang, Hujun Bao, and Xiaowei Zhou.  Smap: Single-shot multi-person absolute 3d pose estimation.  In ECCV, 2020.
  * [67] Xingyi Zhou, Qixing Huang, Xiao Sun, Xiangyang Xue, and Yichen Wei.  Towards 3d human pose estimation in the wild: a weakly-supervised approach.  In ICCV, 2017.


# Stress-activated Constraints in Dense Suspension Rheology

Abhinendra Singh<EMAIL_ADDRESS>James Franck Institute, University
of Chicago, Chicago, Illinois 60637, USA  Pritzker School of Molecular
Engineering, University of Chicago, Chicago, Illinois 60637, USA  Grayson L.
Jackson  James Franck Institute, University of Chicago, Chicago, Illinois
60637, USA  Michael van der Naald  James Franck Institute, University of
Chicago, Chicago, Illinois 60637, USA  Department of Physics, The University
of Chicago, Chicago, Illinois 60637, USA  Juan J de Pablo  Pritzker School of
Molecular Engineering, University of Chicago, Chicago, Illinois 60637, USA
Pritzker School of Molecular Engineering, University of Chicago, Chicago,
IlliMaterials Science Division, Argonne National Laboratory, Lemont, Illinois
60439, USA  Heinrich M. Jaeger  James Franck Institute, University of Chicago,
Chicago, Illinois 60637, USA  Department of Physics, The University of
Chicago, Chicago, Illinois 60637, USA

###### Abstract

Dispersing small particles in a liquid can produce surprising behaviors when
the solids fraction becomes large: rapid shearing drives these systems out of
equilibrium and can lead to dramatic increases in viscosity (shear thickening)
or even solidification (shear jamming). These phenomena occur above a
characteristic onset stress when particles are forced into frictional contact.
Here we show via simulations how this can be understood within a framework
that abstracts details of the forces acting at particle-particle contacts into
general stress-activated constraints on relative particle movement. We find
that focusing on just two constraints, affecting sliding and rolling at
contact, can reproduce the experimentally observed shear thickening behavior
quantitatively, despite widely different particle properties, surface
chemistries, and suspending fluids. Within this framework parameters such as
coefficients of sliding and rolling friction can each be viewed as proxy for
one or more forces of different physical or chemical origin, while the
parameter magnitudes indicate the relative importance of the associated
constraint. In this way, a new link is established that connects features
observable in macroscale rheological measurements to classes of constraints
arising from micro- or nano-scale properties.

soft matter, shear thickening, suspensions, friction

Concentrated or “dense” suspensions comprising small particles in a suspending
liquid are ubiquitous in nature as well as industrial settings Coussot (1997);
Van Damme (2018); Blanco  _et al._ (2019); Brown and Jaeger (2014); Morris
(2020). Even for simple liquids with completely Newtonian characteristics,
i.e. constant viscosity, such suspensions can exhibit strikingly non-Newtonian
behaviors under applied shear, such as yielding, shear thinning, shear
thickening, and shear-jamming Morris (2020); Mewis and Wagner (2011); Brown
and Jaeger (2014); Denn  _et al._ (2018); Guazzelli and Pouliquen (2018);
Singh  _et al._ (2019). All of these behaviors originate from short-ranged
forces between particles that either are mediated by the hydrodynamics of thin
liquid layers or involve direct, frictional and possibly also cohesive contact
Jamali and Brady (2019); Comtet  _et al._ (2017); James  _et al._ (2018). The
strength of these forces depends not only on the particles’ physical
properties such as shape, stiffness and surface roughness, but also on their
surface chemistry and its role during particle-particle contact or through
interaction with the surrounding liquid Hsu  _et al._ (2018); James  _et al._
(2019); Bourrianne  _et al._ (2020); van der Naald  _et al._ (2021); Hsu  _et
al._ (2021). Given this large set of potentially contributing factors,
establishing a predictive link between microscale properties and macroscale
observable flow behaviors has been a longstanding problem.

One recent approach, by Guy _et al._ , to address this issue has been to
classify the macroscale rheology not by focusing on the details of specific
forces, but rather on the general types of constraints that affect relative
particle movement Guy  _et al._ (2018). The promise of this approach lies in
that the physical or chemical origin of any particular particle-particle
interaction may matter far less than its net effect on the ability of
neighboring particles to move with respect to one another. What remained to be
shown, however, is which specific types of constraints are necessary for
quantitative modeling of dense suspension rheology.

Here we demonstrate that quantitative modeling and prediction is indeed
possible. We focus on shear thickening, perhaps the most remarkable non-
Newtonian behavior of dense suspensions, and introduce a new diagnostic
framework to classify rheological flow curves in terms of whether sliding and
rolling constraints on particle motion are present. This approach provides
insight into why particular particle-scale properties introduce additional
constraints while others do not and unlocks a new connection between bulk
rheology and nanoscale particle surface properties (Fig. 1A).

During shear thickening the shear stress $\sigma$ increases faster than the
shear rate $\dot{\gamma}$, leading to a net increase in viscosity
$\eta=\sigma/\dot{\gamma}$. For low solids volume fractions $\phi$ this
increase is mild and occurs continuously as a function of applied shear, but
for larger $\phi$ the viscosity can increase abruptly and dramatically when a
critical rate $\dot{\gamma}_{c}$ is reached, behavior termed discontinuous
shear thickening (DST). For sufficiently large $\phi$ and shear stress
$\sigma$, these suspensions can furthermore transform into a shear-jammed
(SJ), solid-like state, which “melts” back into a fluid once stress is
released Peters  _et al._ (2016); Seto  _et al._ (2019). Past work has
established a strong foundation to understand the evolution of shear
thickening toward DST and SJ, yet focused almost exclusively on stress-
activated sliding friction, represented by a single coefficient for sliding
friction $\mu_{s}$.

In this picture, once the applied shear stress overwhelms the repulsive
interparticle potential, unconstrained, hydrodynamically “lubricated” contacts
transition to “frictional” contacts that prevent sliding Seto  _et al._
(2013); Mari  _et al._ (2014); Guy  _et al._ (2015); Ness and Sun (2016);
Singh  _et al._ (2018); James  _et al._ (2018); Clavaud  _et al._ (2017); Hsu
_et al._ (2018). When viscosity $\eta$ is plotted as a function of applied
shear stress $\sigma$ (Fig. 1B), this increase in the number of frictionally
constrained particle-particle contacts manifests as shear thickening that
starts at an onset stress $\sigma_{\mathrm{on}}$ and persists up to an upper
limit $\sigma_{\mathrm{max}}$, where the system has reached a state with all
contacts frictional. As the solids fraction $\phi$ gets closer to the onset
packing fraction for jamming $\phi_{J}^{\mu}$, the dependence of $\eta$ on
stress becomes steeper within the shear thickening regime, until DST is
reached. In plots like Fig. 1B, DST is identified by a slope
$d$(log$\eta$)/$d$(log$\sigma)=1$, i.e., the viscosity
$\eta=\sigma/\dot{\gamma}_{c}$ is directly proportional to the stress.
Increasing the sliding friction coefficient $\mu_{s}$ reduces $\phi_{J}^{\mu}$
and thus, for given $\phi$, brings the system closer to jamming. This in turn
steepens the rise in viscosity with stress while at the same time increasing
the final viscosity level that is reached in the large stress limit.

However, if only sliding constraints are considered, the effect of $\mu_{s}$
on $\phi_{J}^{\mu}$ is rather small: it reduces the onset of jamming from
$\phi_{J}^{0}$ for frictionless ($\mu_{s}=0$) particles, which for
monodisperse rigid spheres is equal to the random close-packing value
$\phi_{\rm{RCP}}\approx 0.64$ Mari  _et al._ (2014); Wyart and Cates (2014);
Singh  _et al._ (2018); Guy  _et al._ (2018), to no lower than
$\phi_{J}^{\mu}\approx 0.56$ even for $\mu_{s}=\infty$ Singh  _et al._ (2018);
Mari  _et al._ (2014). This poses a serious problem for quantitative
prediction, since experiments with rough spherical particles and also with
specific chemical surface groups have demonstrated DST for packing fractions
so low that the associated $\phi_{J}^{\mu}$ lies well below $0.56$ and thus
outside the range of current models based on just $\mu_{s}$ Laun (1984); James
_et al._ (2018, 2019); Hsu  _et al._ (2018); Hsiao and Pradeep (2019); Lootens
_et al._ (2005); Pradeep  _et al._ (2020); Hsu  _et al._ (2021). Therefore,
additional constraints beyond sliding are needed to properly capture the
behavior of real suspensions.

Figure 1: (A) Shear thickening is a phenomena that can be understood across a
hierarchy of length scales, and unraveling the inter-relationship between
macroscopic rheology, microscopic stress-activated constraints that hinder
relative particle motion, and nanoscopic particle surface properties poses a
major challenge. Macroscale rheology can become a sensitive probe of
nanometer-scale interactions between particle surfaces once the link with
microscopic frictional constraints has been established. (B) How microscopic
frictional constraints affect shear thickening. Viscosity $\eta$ plotted as
function of applied shear stress $\sigma$ at constant volume fraction $\phi$.
Above the critical onset stress $\sigma_{\mathrm{on}}$, “lubricated” contacts
are starting to become transformed into “constrained” frictional ones. At
$\sigma_{\mathrm{max}}$ all contacts are constrained and a maximum plateau
viscosity is reached. At this volume fraction, stress-activated sliding
constraints alone lead only to continuous shear thickening (_solid black
line_). Attractive central forces typically lead to a yield stress, without
affecting shear thickening (_dashed green line_). Adding stress-activated
rolling constraints can lead to discontinuous shear thickening (DST, slope 1)
over a wider stress range and with a higher plateau viscosity (_solid blue
line_).

Taking cues from modeling the rheology of dry granular materials Estrada  _et
al._ (2011); Ai  _et al._ (2011); Santos  _et al._ (2020); Ding  _et al._
(2007); Dominik and Tielens (1995), recent simulations explored how additional
stress-activated constraints on rolling affect shear thickening in dense
suspensions Singh  _et al._ (2020). Interestingly, under the right conditions,
already small additions of rolling friction were found to generate significant
effects. At the same volume fraction where suspensions with only sliding
constraints exhibit mild shear thickening, adding rolling friction can lead to
DST, broaden the stress range over which shear thickening is observed, and
increase the viscosity of the frictional state. In Fig. 1B this is shown by
comparing the flow curves with (blue) and without (black) rolling friction. In
the limit of infinite sliding and rolling friction the frictional jamming
point drops as low as $\phi_{J}^{\mu}=0.37$. Short-ranged attractive particle-
particle interactions that are not stress-activated and give rise to a yield
stress can be included by simply adding them Singh  _et al._ (2019), as
exemplified by the dashed green trace in Fig. 1B for the case without rolling
friction.

Taken together, this opens up an opportunity that we explore here: to model
experimental suspension rheology quantitatively and understand how the
combination of stress-activated sliding and rolling constraints, expressed in
terms of an onset stress for frictional contact and coefficients for sliding
and rolling friction, can be linked to particle-scale properties.

To this end we utilize simulations of rigid spheres that include lubrication,
electrostatic repulsion, and frictional sliding and rolling constraints. We
demonstrate how quantitative detail about the constraints operative over
nanoscale distances at particle-particle contacts can be extracted from bulk
measurements of $\eta(\sigma)$ as in Fig. 1B by comparing the simulations with
experimental data for spherical particles. We first show that a number of
commonly studied suspensions that vary widely in their composition,
surprisingly have nearly identical frictional constraints and that they
primarily affect sliding of contacting particle surfaces. We term these
“standard” particle suspensions. We then use the simulations to predict how
the magnitudes of stress-activated sliding or rolling friction coefficients
alter the relative contributions from the associated constraints to the
measured $\eta(\sigma)$ curves. Finally, we illustrate with three examples how
modifications to particle surface roughness or chemistry lead to
characteristic deviations from the “standard” behavior and how this can be
understood in terms of additional stress-activated constraints on particle
rolling.

## Results

###  .1 “Standard” particle suspensions

Figure 2:  Shear thickening of “standard” particle suspensions show nearly
identical shear thickening after scaling by the onset stress
$\sigma_{\mathrm{on}}$. As there are small discrepancies in the
experimentally-reported volume fractions, data with similar thickening are
grouped by color. See Table 1 for details regarding particle size, solvent,
and onset stress. Solid lines denote simulation data for
$\\{\mu_{s},\mu_{r}\\}=\\{0.5,0.07\\}$ and Debye length $\lambda/a=0.01$ at
various volume fractions $\phi$ as mentioned. (_Inset_) Beyond the onset of
DST $\phi_{c}<\phi<\phi_{J}^{\mu}$, simulations (_solid black line_) capture
non-monotonic flow curves while experimental measurements cannot (_orange
dashed line_).

In Fig. 2 we compare the shear thickening behavior across a series of
suspension types and volume fractions: sterically stabilized poly(methyl
methacrylate) (PMMA) particles in dioctyl phthalate D’Haene  _et al._ (1993),
charge stabilized silica particles in aqueous glycerol Royer  _et al._ (2016)
or PEG-200 Egres (2005), glass beads in poly(dimethylsiloxane) (PDMS) Xu  _et
al._ (2020), and carboxylate-coated particles in aqueous solutions Laun
(1984); James  _et al._ (2018). While the differences in particle size and
solvent/surface chemistries do affect the onset stress $\sigma_{\mathrm{on}}$
for shear thickening (see Table 1), plotting the reduced viscosity $\eta_{r}$,
i.e., $\eta$ normalized by the viscosity of the suspending liquid, as a
function of $\sigma$ scaled by $\sigma_{\mathrm{on}}$ reveals remarkably
similar behavior at each volume fraction, with only a few minor
deviations.111There are some discrepancies in the volume fractions, but these
are within the typically reported experimental errors associated with
determining exact particle density and/or packing fraction Poon_2012. The
deviation at low volume fraction for Ref. James  _et al._ (2018) is because
this data was taken at a constant ratio of [urea]:[solvent] (6 M), meaning
that the amount of urea relative to the concentration of surface -CO2H groups
changes with packing fraction. The higher relative urea concentration at low
packing fractions leads to a lower effective friction coefficient between
particle surfaces James  _et al._ (2019). Furthermore, these volume-fraction-
dependent curves show near quantitative agreement (baseline viscosity values,
slope, and extent of shear thickening) with simulations when sets of sliding
and rolling friction coefficients $\\{\mu_{s},\mu_{r}\\}$ were picked that
primarily emphasize constraints on sliding, such as $\\{1,0\\}$ or
$\\{0.5,0.07\\}$. We note here, and discuss below, that both combinations
produce essentially the same curves of $\eta_{r}(\sigma)$, will lead to the
onset of DST at the same packing fraction of $\approx 0.56$, and have the same
$\phi_{J}^{\mu}\approx 0.58$. The seeming discrepancy between simulations and
experiments at the largest volume fraction shown in Fig. 2 has its origin in
the fact that with experiments it is difficult to map out the multi-valued
region in S-shaped $\sigma(\dot{\gamma})$ flow curves (Fig. 2, inset) that
appears for volume fractions beyond the onset of DST. In the corresponding
plots of $\eta(\sigma)$ the bending backward of S-shaped flow curves produces
intervals along $\sigma$ where the slope $d$(log$\eta$)/$d$(log$\sigma)$ is
larger than 1. This is reproduced by the simulations, but cannot be observed
in conventional rheology experiments, where the slope maximally reaches 1
Hermes  _et al._ (2016); Han  _et al._ (2019); Saint-Michel  _et al._ (2018);
Peters  _et al._ (2016); Han  _et al._ (2019); Rathee  _et al._ (2017).

All of these suspensions are therefore in a regime where their thickening
behavior is dictated primarily by sliding constraints. We define these
suspensions and others that can be collapsed onto similar sets of
$\eta_{r}(\sigma)$ curves as “standard” particle suspensions. The key point
here is that while nanometer-scale features and molecular level details at
contacting particle surfaces certainly control the interparticle potential
(and thus the onset stress), only those that modify sliding constraints appear
to affect shear thickening. The collapse of these experimental data and the
agreement with the simulations indicates that at each packing fraction

  1. 1.

the stress-dependent balance of frictional versus lubricated contacts is
similar,

  2. 2.

once particles enter into frictional contact, they experience similar
microscopic constraints, and

  3. 3.

these systems primarily experience constraints on sliding with little penalty
for rolling.

Table 1: Details of suspension properties in Fig. 2 Particle Identity | Particle size (diameter) | Solvent | Stabilization type |  $\sigma_{\mathrm{on}}$ (Pa)
---|---|---|---|---
1\. PMMA (D’Haene _et al._) |  $0.69~{}\mu$m | dioctyl phthalate | steric | 220
2\. Silica (Royer _et al._) |  $1.54~{}\mu$m | aqueous glycerol | charge | 418
3\. Silica (Egres) |  $0.45~{}\mu$m | PEG-200 | charge | 433
4\. Carboxylic acid coated Latex (Laun) |  $0.20~{}\mu$m | pH = 6.2 aqueous solution | charge | 5.66
5\. Carboxylic acid coated PMMA (James _et al._) |  $0.80~{}\mu$m | 6 M urea in aqueous glycerol | charge | 160
6\. Glass (Xu _et al._) |  $20~{}\mu$m | low molecular weight PDMS | charge | 1.66

###  .2 Altering Jamming via Microscopic Frictional Constraints

As shown in Ref. Singh  _et al._ (2020), the minimum volume fraction
$\phi_{J}^{\mu}$ required for the onset of jamming depends on both sliding and
rolling frictional constraints. In Fig. 3 we show this dependence with data
traces at various fixed $\mu_{s}$ while varying $\mu_{r}$. The set of friction
coefficients $\\{\mu_{s},\mu_{r}\\}=\\{0,0\\}$ represents the limit of no
microscopic frictional constraints with a jamming point $\phi_{J}^{\mu}\approx
0.65$. Conversely, $\\{\mu_{s},\mu_{r}\\}=\\{\infty,\infty\\}$ is the limit
where relative particle motion at contact is fully constrained (e.g.
contacting particles can neither slide nor roll) and $\phi_{J}^{\mu}\approx
0.37$.

The data in Fig. 3 provide the context for understanding “standard” particle
suspensions as well as deviations from it. The thick horizontal band
represents the jamming volume fraction $\phi_{J}^{\mu}\approx 0.58$ for
“standard” particle suspensions shown in Fig. 2. The jamming onset is
relatively flat in this region, meaning that friction coefficients for the
“standard” particle suspensions could range between $0.5<\mu_{s}<1$ and
$0<\mu_{r}<0.07$, supporting our earlier assertion that in this region shear
thickening is primarily due to sliding constraints. Once $\mu_{s}\geq 0.5$ and
thus significant sliding constraints are present, even small additional
amounts of rolling friction lead to large reductions in $\phi_{J}^{\mu}$. This
explains observations such as $\phi_{J}^{\mu}\approx 0.45$ for raspberry-type
particles Hsu  _et al._ (2018) or related findings for other rough particles
Lootens  _et al._ (2005); Hsiao  _et al._ (2017); Pradeep  _et al._ (2020).
Thus, by experimentally determining $\phi_{J}^{\mu}$ one can check immediately
whether a shear thickening suspension is governed mainly by sliding
constraints (“standard”) or by a combination of sliding and rolling
constraints.

Figure 3:  Tuning the jamming volume fraction map by changing stress-activated
frictional constraints due to sliding ($\mu_{s}$) and rolling ($\mu_{r}$). The
horizontal thick line represents the jamming volume fraction for “Standard”
Particle Suspensions, while “Extreme Roughness” refers to the experimentally
measured jamming point for rough raspberry-type particles Hsu  _et al._
(2018).

###  .3 Deviations from “Standard” Behavior due to Rolling Constraints

We now focus on three illustrative deviations from the “standard” suspension
behavior, starting with the most intuitive case, namely very rough particles,
where large protruding asperities can interlock as particles come into
contact. This reduces the jamming volume fraction significantly Lootens  _et
al._ (2005); Hsiao  _et al._ (2017); Hsu  _et al._ (2018); Pradeep  _et al._
(2020), and thus in light of Fig. 3 indicates the presence of an additional
constraint on rolling.

Figure 4:  Linking changes in microscopic constraints to deviations from
“standard” behavior. (A) Particle surface roughness modifies both sliding and
rolling constraints. Experimental data from Ref. Hsiao  _et al._ (2017) at
volume fraction $\phi=0.5$ (_symbols_) for variable roughness with
$\sigma_{\mathrm{on}}=$ 0.95 Pa (MR) and 5 Pa (VR), simulation data (_lines_)
for combinations for $\\{\mu_{s},\mu_{r}\\}$. (B) Addition of urea to
carboxylic acid (-CO2H) coated particles disrupts interparticle hydrogen
bonding and specifically reduces rolling constraints. Experimental data from
Ref. James  _et al._ (2018). The onset stress $\sigma_{\mathrm{on}}$ for 0 M
and 6M urea concentrations is 5 Pa and 160 Pa, respectively. (C) Decreasing
solution pH for -CO2H coated particles dramatically increases microscopic
constraints. Experimental data from Ref. Laun (1984) with dashed line being
simulation data set from Ref. Singh  _et al._ (2019). Here,
$\sigma_{\mathrm{on}}$ is 3500 (pH=7.1), 1925 (6.2), 696.6 (5.7), and 96.3
(5.1) Pa.

As an example we show in Fig. 4A data from experiments by Hsiao _et al._ with
PMMA particles of increasing roughness, from smooth (SM) and medium rough (MR)
to very rough (VR) (with dimensionless asperity sizes of 0.026, 0.075, and
0.082, respectively) at a volume fraction $\phi=0.5$ Hsiao  _et al._ (2017).
For smooth colloids the distance to jamming $\phi_{J}^{\mu}-\phi$ is so large
that no shear thickening is expected and $\eta_{r}(\sigma)$ is essentially
flat. For the MR particles the jamming volume fraction was found in the
experiments to decrease to $\phi_{J}^{\mu}=0.59$. According to Fig. 3 this
implies standard suspension behavior, where combinations of
$\\{\mu_{s},\mu_{r}\\}$ for the limit of dominant sliding and negligible
friction are appropriate. Specifically, we find that simulations with
$\\{\mu_{s},\mu_{r}\\}=\\{0.5,0\\}$ can reproduce the experimental well.
However, for the VR colloids the experimentally determined
$\phi_{J}^{\mu}=0.54$ cannot be reached simply by increasing sliding friction,
even in the limit of infinite $\mu_{s}$. This is a clear indicator that now
the particle surface is sufficiently rough that asperities interlock to
constrain also rolling. Based on Fig. 3 we expect that $\eta_{r}(\sigma)$ for
the VR suspensions can be reproduced by increasing
$\\{\mu_{s},\mu_{r}\\}=\\{0.5,0\\}$ used for the MR system to combinations
such as $\\{1,0.1\\}$. This is shown by the red trace in Fig. 4A.

Changes in particle surface chemistry have also been shown to affect shear
thickening. For example, addition of urea decreases shear thickening in
aqueous suspensions of particles coated with carboxylic acid groups (-CO2H)
James  _et al._ (2018, 2019), as shown in Fig. 4B. In this figure the
experimental data have been scaled by their respective onset stresses for
thickening $\sigma_{\mathrm{on}}$. Data without added urea (i.e. 0 M) are
matched by the simulations only when combinations of sliding and rolling
friction are used that fall outside the “standard” range in Fig. 3, such as
$\\{\mu_{s},\mu_{r}\\}=\\{0.5,0.25\\}$ or $\\{0.6,0.10\\}$. Such large
effective friction, resulting from significant constraints on sliding plus
additional rolling resistance, can be be attributed to the formation of
hydrogen bonds between carboxylated surfaces as particles are coming into
close contact. The reduced shear thickening in 6 M urea is then caused by
“capping” of the -CO2H groups. This disrupts hydrogen bonding, and urea
thereby reduces the effective friction coefficient to levels of “standard”
suspensions. This is seen by the fact that we can model the 6 M urea data in
Fig. 4B with combinations such as $\\{\mu_{s},\mu_{r}\\}=\\{0.5,0.07\\}$.

The constraint-based perspective also sheds new light on the observation that
shear thickening for CO2H-coated latex is highly dependent on pH Laun (1984).
At a constant volume fraction, suspensions at pH = 7.1 exhibit continuous
shear thickening whereas those at pH = 5.1 display DST (Fig. 4C). While the
viscosity of the low stress state of the experimental data is considerably
higher than that of the simulation, this is due to the existence of a finite
yield stress and shear thinning in the experimental data set (here we only
show the data for intermediate to high stress region). We can easily account
for that in the simulations by including attractive forces between particles,
similar to what is depicted in Fig. 1 and discussed in detail in Ref. Singh
_et al._ (2019). By fixing $\mu_{s}=1$ for convenience and increasing rolling
constraints $\mu_{r}$ from 0 to 1.0 in the simulations, we can quantitatively
reproduce the experimental data. The constraint-based picture therefore allows
us to move beyond the results of Noy _et al._ Noy  _et al._ (1997), who showed
that the effective friction coefficient between -CO2H coated surfaces
increases with decreasing pH. Specifically, increased hydrogen bonding between
“sticky” or adhesive protonated -CO2H groups at lower pH gives rise to
increased sliding and rolling constraints, which are responsible for the
changes in the shear thickening behavior observed on macroscopic scales.

## Conclusions

Our numerical simulations highlight the power of a constraint-based approach
for understanding and predicting the shear thickening behavior of diverse
kinds of dense suspensions, including a wide range of different particle types
and particle surface features. We find that two constraints on relative
particle movement suffice for quantitative modeling, namely separate
constraints on sliding and rolling that are activated when the local stress
exceeds a threshold such that particles are coming into direct contact and
experience friction forces. A central point in this approach is that each
constraint can represent a variety of different physical or chemical
interactions giving rise to this friction. In particular, the approach allows
one to translate ideas about chemical interactions typically developed under
more dilute, closer to equilibrium conditions to these concentrated, out-of-
equilibrium systems. In the simulations, constraints are implemented via
associated friction coefficients $\mu_{s}$ and $\mu_{r}$, whose magnitudes
provide an indicator of their relative strength. The combination of sliding
and rolling constraints can affect the shear thickening behavior in highly
nonlinear ways, where already small amounts of additional rolling friction can
make an outsized contribution. As Fig. 3 demonstrates, for physically
realistic values $\mu_{s}\approx 0.5$ additional rolling resistance can have a
very large effect on $\phi_{J}^{\mu}$ and thus on the stress response of a
dense suspension. As a result, plots of the suspension viscosity
$\eta(\sigma)$ as a function of applied shear stress can be viewed as a
macroscopic reporter of stress-activated constraints that originate from
contact interactions at the nanoscale. In particular:

  * •

if $\eta(\sigma)$ _follows_ the “standard” rheology in Fig. 2, then the
rolling and sliding friction coefficients fall within the “flat” region of
$\phi_{J}^{\mu}$ in Fig. 3, indicating minimal surface roughness and a lack of
strong interparticle adhesive interactions for the given combination of
particle/solvent chemistry;

  * •

if $\eta(\sigma)$ _deviates_ from the “standard” rheology, then the rolling
and/or sliding constraints are stronger due to surface roughness or adhesive
interactions such as hydrogen bonding between particle surfaces

We note that access to such detailed information about the relative
contributions from sliding and rolling is exceedingly difficult to obtain from
experiments that use scanning probe techniques, which necessarily focus on
lateral sliding motion.

In terms of an effective coefficient of friction accounting for the role of
both $\mu_{s}$ and $\mu_{r}$, the “flat” region of $\phi_{J}^{\mu}$ in Fig. 2
provides an explanation as to why prior simulations that did not include
rolling friction were still able to model “standard” suspensions as long as
they used large values of $\mu_{s}\approx 1$ Mari  _et al._ (2015); Ness and
Sun (2016); Singh  _et al._ (2018). More generally, Fig. 2 shows that
combinations of different types and strengths of contact interactions,
represented by different sets of values for $\mu_{s}$ and $\mu_{r}$, can give
rise to the same jamming threshold $\phi_{J}^{\mu}$ and thus the same shear
thickening behavior. Therefore, our results suggest that $\phi_{J}^{\mu}$
suitably represents the combined effect of the two different types of
constraint. The same framework could also be adapted to understand how stress-
deactivated constraints Guy  _et al._ (2018) give rise to shear thinning.
Finally, while we here only focused on the connection between microscopic
constraints and the macroscopic shear thickening response, there are open
questions on how different stress-activated constraints alter the mesoscale
force chain network.

###### Acknowledgements.

We acknowledge support from the Center for Hierarchical Materials Design
(CHiMaD) under award number 70NANB19H005 (US Dept. Commerce). and from the
Army Research Office under grants W911NF-19-1-0245, W911NF-20-2-0044, and
W911NF-21-1-0038.

## References

  * Coussot (1997) P. Coussot,  _Mudflow Rheology and Dynamics_ (CRC Press, 1997).
  * Van Damme (2018) H. Van Damme, Cem. Concr. Res. 112, 5 (2018).
  * Blanco  _et al._ (2019) E. Blanco, D. J. Hodgson, M. Hermes, R. Besseling, G. L. Hunter, P. M. Chaikin, M. E. Cates, I. Van Damme, and W. C. K. Poon, Proc. Natl. Acad. Sci. USA 116, 10303 (2019).
  * Brown and Jaeger (2014) E. Brown and H. M. Jaeger, Rep. Prog. Phys. 77, 046602 (2014).
  * Morris (2020) J. F. Morris, Annual Review of Fluid Mechanics 52, 121 (2020).
  * Mewis and Wagner (2011) J. Mewis and N. J. Wagner,  _Colloidal Suspension Rheology_ (Cambridge University Press, 2011).
  * Denn  _et al._ (2018) M. M. Denn, J. F. Morris, and D. Bonn, Soft Matter 14, 170 (2018).
  * Guazzelli and Pouliquen (2018) É. Guazzelli and O. Pouliquen, J. Fluid Mech. 852 (2018).
  * Singh  _et al._ (2019) A. Singh, S. Pednekar, J. Chun, M. M. Denn, and J. F. Morris, Phys. Rev. Lett. 122, 098004 (2019).
  * Jamali and Brady (2019) S. Jamali and J. F. Brady, Phys. Rev. Lett. 123, 138002 (2019).
  * Comtet  _et al._ (2017) J. Comtet, G. Chatté, A. Niguès, L. Bocquet, A. Siria, and A. Colin, Nat. Comm. 8, 15633 (2017).
  * James  _et al._ (2018) N. M. James, E. Han, R. A. L. de la Cruz, J. Jureller, and H. M. Jaeger, Nat. Mater. 17, 965 (2018).
  * Hsu  _et al._ (2018) C.-P. Hsu, S. N. Ramakrishna, M. Zanini, N. D. Spencer, and L. Isa, Proc. Nat. Acad. Sci. (2018).
  * James  _et al._ (2019) N. M. James, C.-P. Hsu, N. D. Spencer, H. M. Jaeger, and L. Isa, J. Phys. Chem. Lett. 10, 1663 (2019).
  * Bourrianne  _et al._ (2020) P. Bourrianne, V. Niggel, G. Polly, T. Divoux, and G. H. McKinley, arXiv preprint arXiv:2001.02290 (2020).
  * van der Naald  _et al._ (2021) M. van der Naald, L. Zhao, G. L. Jackson, and H. M. Jaeger, Soft Matter (2021).
  * Hsu  _et al._ (2021) C.-P. Hsu, J. Mandal, S. N. Ramakrishna, N. D. Spencer, and L. Isa, Nature communications 12, 1 (2021).
  * Guy  _et al._ (2018) B. M. Guy, J. Richards, D. Hodgson, E. Blanco, and W. C. K. Poon, Phys. Rev. Lett. 121, 128001 (2018).
  * Peters  _et al._ (2016) I. R. Peters, S. Majumdar, and H. M. Jaeger, Nature 532, 214 (2016).
  * Seto  _et al._ (2019) R. Seto, A. Singh, B. Chakraborty, M. M. Denn, and J. F. Morris, Gran. Matt. 21, 82 (2019).
  * Seto  _et al._ (2013) R. Seto, R. Mari, J. F. Morris, and M. M. Denn, Phys. Rev. Lett. 111, 218301 (2013).
  * Mari  _et al._ (2014) R. Mari, R. Seto, J. F. Morris, and M. M. Denn, J. Rheol. 58, 1693 (2014).
  * Guy  _et al._ (2015) B. M. Guy, M. Hermes, and W. C. K. Poon, Phys. Rev. Lett. 115, 088304 (2015).
  * Ness and Sun (2016) C. Ness and J. Sun, Soft Matter 12, 914 (2016).
  * Singh  _et al._ (2018) A. Singh, R. Mari, M. M. Denn, and J. F. Morris, J. Rheol. 62, 457 (2018).
  * Clavaud  _et al._ (2017) C. Clavaud, A. Bérut, B. Metzger, and Y. Forterre, Proc. Natl. Acad. Sci. U.S.A. , 5147 (2017).
  * Wyart and Cates (2014) M. Wyart and M. E. Cates, Phys. Rev. Lett. 112, 098302 (2014).
  * Laun (1984) H. M. Laun, Angew. Makromol. Chem. 123, 335 (1984).
  * Hsiao and Pradeep (2019) L. C. Hsiao and S. Pradeep, Curr. Opin. Colloid Interface Sci. (2019).
  * Lootens  _et al._ (2005) D. Lootens, H. van Damme, Y. Hémar, and P. Hébraud, Phys. Rev. Lett. 95, 268302 (2005).
  * Pradeep  _et al._ (2020) S. Pradeep, A. R. Jacob, and L. C. Hsiao, arXiv preprint arXiv:2007.01825 (2020).
  * Estrada  _et al._ (2011) N. Estrada, E. Azéma, F. Radjai, and A. Taboada, Phys. Rev. E 84, 011306 (2011).
  * Ai  _et al._ (2011) J. Ai, J.-F. Chen, J. M. Rotter, and J. Y. Ooi, Powder Technology 206, 269 (2011).
  * Santos  _et al._ (2020) A. P. Santos, D. S. Bolintineanu, G. S. Grest, J. B. Lechman, S. J. Plimpton, I. Srivastava, and L. E. Silbert, Physical Review E 102, 032903 (2020).
  * Ding  _et al._ (2007) W. Ding, A. Howard, M. M. Peri, and C. Cetinkaya, Philosophical Magazine 87, 5685 (2007).
  * Dominik and Tielens (1995) C. Dominik and A. Tielens, Phil. Mag. A 72, 783 (1995).
  * Singh  _et al._ (2020) A. Singh, C. Ness, R. Seto, J. J. de Pablo, and H. M. Jaeger, Phys. Rev. Lett. 124, 248005 (2020).
  * D’Haene  _et al._ (1993) P. D’Haene, J. Mewis, and G. G. Fuller, J. Colloid Interface Sci. 156, 350 (1993).
  * Royer  _et al._ (2016) J. R. Royer, D. L. Blair, and S. D. Hudson, Phys. Rev. Lett. 116, 188301 (2016).
  * Egres (2005) R. G. Egres,  _The effect of particle anisotropy on the rheology and microstructure of concentrated colloidal suspensions through the shear thickening transition_ , Ph.D. thesis, University of Delaware (2005).
  * Xu  _et al._ (2020) Q. Xu, A. Singh, and H. M. Jaeger, Journal of Rheology 64, 321 (2020).
  * Note (1) There are some discrepancies in the volume fractions, but these are within the typically reported experimental errors associated with determining exact particle density and/or packing fraction Poon_2012. The deviation at low volume fraction for Ref. James  _et al._ (2018) is because this data was taken at a constant ratio of [urea]:[solvent] (6 M), meaning that the amount of urea relative to the concentration of surface -CO2H groups changes with packing fraction. The higher relative urea concentration at low packing fractions leads to a lower effective friction coefficient between particle surfaces James  _et al._ (2019).
  * Hermes  _et al._ (2016) M. Hermes, B. M. Guy, W. C. K. Poon, G. Poy, M. E. Cates, and M. Wyart, J. Rheol. 60, 905 (2016).
  * Han  _et al._ (2019) E. Han, N. M. James, and H. M. Jaeger, Phys. Rev. Lett. 123, 248002 (2019).
  * Saint-Michel  _et al._ (2018) B. Saint-Michel, T. Gibaud, and S. Manneville, Physical Review X 8, 031006 (2018).
  * Rathee  _et al._ (2017) V. Rathee, D. L. Blair, and J. S. Urbach, Proceedings of the National Academy of Sciences 114, 8740 (2017).
  * Hsiao  _et al._ (2017) L. C. Hsiao, S. Jamali, E. Glynos, P. F. Green, R. G. Larson, and M. J. Solomon, Phys. Rev. Lett. 119, 158001 (2017).
  * Noy  _et al._ (1997) A. Noy, D. V. Vezenov, and C. M. Lieber, Annual Review of Materials Science 27, 381 (1997).
  * Mari  _et al._ (2015) R. Mari, R. Seto, J. F. Morris, and M. M. Denn, Proceedings of the National Academy of Sciences 112, 15326 (2015).

## Supplemental Material for “Stress-activated Constraints in Dense Suspension
Rheology”

In this document we provide details about the simulation scheme used in the
simulations.

## Methodology

For all of the experimental systems studied both the Reynold’s Number and the
Stokes number are vanishingly small as they both scale with the square of the
particle size. This assumption allows us to simulate particle suspensions in
the overdamped limit. We simulate an assembly of inertialess frictional
spheres in a Newtonian viscous fluid under an imposed stress $\sigma$, that
gives rise to an imposed velocity field
$\vec{v}=\dot{\gamma}(t)\hat{\vec{v}}(\bm{x})=\dot{\gamma}(t)(x_{2},0,0)$. We
use Lees-Edwards periodic boundary conditions with $N=2000$ particles in a
unit cell with a bidisperse particles, with radii $a$ and $1.4a$ mixed at
equal volume fractions to avoid ordering Mari  _et al._ (2014). The particles
interact through short-range hydrodynamic forces (lubrication), short-ranged
repulsive forces (electrostatics), and frictional contact forces. The equation
of motion for $N$ spheres can be reduced to $6N$-dimensional force (as well as
torque) balance between the hydrodynamic ($\vec{F}_{\mathrm{H}}$), repulsive
($\vec{F}_{\mathrm{R}}$), and contact ($\vec{F}_{\mathrm{C}}$) interactions
according to:

$\vec{0}=\vec{F}_{\mathrm{H}}(\vec{X},\vec{U})+\vec{F}_{\mathrm{C}}(\vec{X})+\vec{F}_{\mathrm{R}}(\vec{X}),$
(1)

where $\vec{X}$ and $\vec{U}$ denote the particle positions and
velocities/angular velocities, respectively. To match simulations to the
experimental data we tuned the Debye length $\lambda$ in the repulsive force
$F_{R}$ and the friction coefficients $\mu_{r}$ and $\mu_{s}$ in $F_{C}$. For
more information on the simulation method, including the functional forms of
the forces in the equations of motion as well how they are solved numerically,
we refer the reader to refs. Mari  _et al._ (2014); Singh  _et al._ (2018,
2020). Unit scales are $\dot{\gamma}_{0}\equiv F_{\rm 0}/{6\pi\eta_{0}a^{2}}$
for the strain rate and $\sigma_{0}\equiv\eta_{0}\dot{\gamma}_{0}=F_{\rm
0}/6\pi a^{2}$ for the stress. The onset stress for shear thickening,
$\sigma_{\mathrm{on}}$, used in the main text is obtained from experimental
data and scales as $\sigma_{\mathrm{on}}=\alpha\sigma_{0}$, with typical
values for $\alpha$ around unity.