undefined

# Large Language Models for Orchestrating Bimanual Robots

Kun Chu1, Xufeng Zhao1, Cornelius Weber1, Mengdi Li1, Wenhao Lu1, and Stefan
Wermter1 This research was funded by the German Research Foundation (DFG) in
the project Crossmodal Learning (TRR-169) and the China Scholarship Council
(CSC).1The authors are with the Knowledge Technology Group, Department of
Informatics, University of Hamburg, 22527 Hamburg, Germany. E-mail: {kun.chu,
xufeng.zhao, cornelius.weber, wenhao.lu<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Although there has been rapid progress in endowing robots with the ability to
solve complex manipulation tasks, generating control policies for bimanual
robots to solve tasks involving two hands is still challenging because of the
difficulties in effective temporal and spatial coordination. With emergent
abilities in terms of step-by-step reasoning and in-context learning, Large
Language Models (LLMs) have taken control of a variety of robotic tasks.
However, the nature of language communication via a single sequence of
discrete symbols makes LLM-based coordination in continuous space a particular
challenge for bimanual tasks. To tackle this challenge for the first time by
an LLM, we present LAnguage-model-based Bimanual ORchestration (LABOR), an
agent utilizing an LLM to analyze task configurations and devise coordination
control policies for addressing long-horizon bimanual tasks. In the simulated
environment, the LABOR agent is evaluated through several everyday tasks on
the NICOL humanoid robot. Reported success rates indicate that overall
coordination efficiency is close to optimal performance, while the analysis of
failure causes, classified into spatial and temporal coordination and skill
selection, shows that these vary over tasks. The project website can be found
at http://labor-agent.github.io

##  I Introduction

While humans routinely operate with both hands in their daily lives, achieving
effective bimanual coordination remains a formidable challenge in robotics as
the dual-arm setting faces complexity in terms of temporal and spatial
coordination. Bimanual manipulation offers a significant difficulty for the
existing planning-based and learning-based methods. Previous planning-based
methods [1, 2] largely focus on motion planning for coordinating two arms
through hand-designed primitives or predefined trajectories, limiting their
application in dynamic or complex task spaces. On the other hand, learning-
based methods, such as Reinforcement Learning (RL) and Imitation Learning
(IL), enable a robot to learn control policies from human-designed rewards [3,
4] or human-teleoperated demonstrations [5, 6]. However, it is notoriously
known that RL algorithms are hard to train, especially in dual-arm settings
that have high degrees of freedom (DoFs), and collecting human demonstrations
for bimanual robots is non-trivial. At the same time, the high complexity
associated with the variety of bimanual patterns suggests that low-level
control policies and high-level planning must be considered for an integrated
control system design [7].

Large Language Models (LLMs) have demonstrated remarkable knowledge and
reasoning capabilities [8, 9], leading to a revolutionary change in numerous
fields [10]. In robotics, an LLM is typically employed as a high-level planner
to reason and invoke primitive skills for accomplishing complex tasks
described in natural language [11, 12, 13]. However, as far as we know, all
previous methods focus on unimanual manipulations with complex chains of
skills and lack explicit analysis of bimanual tasks. In this research, we
attempt to further explore the feasibility of leveraging LLMs in robotic
bimanual control, especially focusing on high-level temporal and spatial
coordination between two hands. Our research question is: How can LLMs be
employed to achieve versatile bimanual manipulation control in diverse robotic
tasks?

Figure 1: Illustration of the LABOR agent. During the execution of the task,
with the designed prompt and skill library, the LLM generates the plan step by
step, including a sequential or simultaneous control command for the left and
right hand. The bimanual robot then performs actions on the environment
according to the command and the results get feedback to the LLM for the next
action, and so on until the end of the task.

To this aim, we introduce the LABOR (LAnguage-model-based Bimanual
ORchestration) agent, which uses an LLM to effectively coordinate bimanual
control to deal with complex manipulation tasks, as shown in Fig. 1. Two types
of control patterns, i.e., sequential and simultaneous control, are explicitly
predicted by the LLM, given a designed prompt with a set of possible skills.
With a high-level control timeline, a bimanual robot is able to complete a
long-horizon complex task by combining different control policies through the
cooperation of both hands at different stages. We evaluate the LABOR agent on
the semi-humanoid bimanual NICOL, the Neuro-Inspired COLlaborator [14], on
three typical everyday tasks. The experimental results show that the LABOR
agent is effective in coordinating both hands in long-horizon tasks.

##  II Related Work

In this section, we describe the related work on planning-based and learning-
based methods for bimanual manipulations, and LLM’s use in robotics.

###  II-A Bimanual Manipulation

####  II-A1 Planning-based methods

Early work [15, 7] mainly focuses on planning algorithms for dual-arm settings
under the closed-chain constraint, which rely on static known dynamic models
to generate kinematic configurations that are suitable for two arms. With
engineered primitives or predefined trajectories [1, 16, 2, 17], bimanual
robots exhibit some flexibility when solving tasks like folding shirts or
scooping food. However, such planning-based methods are not adaptive to
dynamic environments and mainly rely on task-specific primitives, which are
costly to design or collect in a more complex task setting.

####  II-A2 Learning-based methods

Some works have employed RL [18, 3, 4, 19] and IL [5, 20, 21, 6] framework in
dual-arm settings. These typically require massive samples and training time,
due to the high dimensionality of the action space in a bimanual setting.
Although some recent works proposed to design parameterized primitives to
reduce the size of the action space for exploration, these movement primitives
normally require costly hard-coded motion design or human-teleoperated
demonstration collection, limiting their applications in complex environments.

Instead of designing many task-specific primitives, we use easily designed
general primitives for the robot and attempt to utilize LLMs to generate
chains of skills to accomplish complex bimanual motions. Furthermore, the
approaches listed above do little to analyze the properties of the task at a
higher level, but rather adopt a single control policy for controlling the
robot to accomplish the task. In this sense, the control policy in their tasks
is normally fixed and monolithic which tackles only simple and short-duration
tasks. In contrast, we explore the usage of LLMs to generate control policies
from a higher perspective that considers the current stage of the task at
hand, thus adapting to complex and variable long-horizon tasks.

###  II-B LLM in Robotics

Learning robotic control policies to address various manipulation tasks often
requires the collection of massive datasets [22, 23, 24, 25]. Recent works
have leveraged common sense and established knowledge from LLMs to control
robots in tackling complex tasks. Some studies have delved into distilling LLM
expertise into localized models. For instance, researchers have utilized LLMs’
programming capabilities to generate executable code for fulfilling open
instructions [26, 27, 28], or to provide direct guidance for RL policies [29,
30]. Other approaches involve decomposing complex tasks into manageable sub-
tasks using LLMs, thus enabling robots to address them individually with pre-
defined skills [12, 11], i.e., language-conditioned policies. We adopt the
latter paradigm to construct a versatile robot equipped with atomic skills
that can be efficiently reused across multiple tasks or potentially enhanced
for new abilities [13]. Specifically, our robot is furnished with general
skills for both arms, offering a vast number of potential combinations.
However, this inherent complexity of combinatorial capabilities poses
challenges to efficient reasoning and planning in bimanual manipulation, which
will be explored in this study.

##  III Method

This section introduces the LABOR agent regarding two main aspects, Bimanual
Manipulation, which describes the control policies adopted, and LLM
Orchestration, which describes how the control policies can be coordinated
using an LLM.

###  III-A Bimanual Manipulation

As described in earlier works [5, 31, 32], basic bimanual manipulation control
mainly involves the consideration of coordination and, when required,
synchronization. For example, grasping or releasing small objects does not
need bimanual coordination as one single hand can accomplish such a task. In
contrast, to lift a large object like a bowl, two hands are needed to hold it,
and they must coordinate symmetrically. In the task of opening a pair of
scissors, two hands need to cooperate asymmetrically, for example, one hand
needs to stabilize the scissors first, and another hand slides the handle away
from the other handle.

Figure 2: Control policies adopted by the LABOR agent.

Considering the spatial and temporal constraints, we adopt two types of
control policies for two hands, simultaneous control for symmetric processes,
and sequential control for uncoordinated and asymmetric processes, as shown in
Fig. 2. In a macroscopic view of long-horizon bimanual manipulation tasks, a
combination of processes can be involved. For example, imagine that there is
an apple and a bowl on the table, the task is placing the apple in the bowl
and holding the bowl to a serving position. There will be uncoordinated
processes such as grasping and releasing the apple, and then symmetrically
coordinated processes such as joint holding and moving are required. During
the task accomplishment, which control policy to adopt should be decided
first, based on the current spatial and temporal situations, and then the
specific control commands to coordinate two hands.

###  III-B LLM Orchestration

Figure 3: Prompts for the LLM to orchestrate a bimanual robot to accomplish
the task. The general prompting template, indicated in gray, remains
consistent across various tasks, while task-specific details such as task
description and manipulation rules are presented in standard black text.

An LLM exhibits remarkable abilities in terms of step-by-step reasoning, in-
context- and instruction-following responses in robotics, which makes it
possible to generate policies under temporal and spatial constraints in
bimanual manipulations. As shown in Fig 1, the presented LABOR agent involves
human user inputs like the initial prompt and skill library, the LLM
coordinator, and the bimanual robot executor. As depicted in Fig. 3, the
designed prompt contains background information regarding the bimanual robot,
like coordinates and workspaces of two hands, following some rules specifying
the bimanual control principles when facing different situations. At the same
time, the skill library is available to the LLM in the form of tool APIs.
Given this, the prompts explicitly require the LLM to generate policies that
are sequential or simultaneous, with specific control commands based on the
provided skill library. The skill library is designed based on control
policies and consists of two main types: single-handed skills and both-handed
skills. After executing an action assigned by the command, the LLM obtains
observations from the execution results, e.g., an action was completed by the
left or right hand, the skill execution failed because the assigned action was
not in the valid workspace of the selected hand, etc. Iterating in this way,
the LLM progressively generates a solution to the task while self-correcting
mistaken commands, thus generating a chain of skills throughout the lifespan
of the task, as shown in Fig. 4.

Figure 4:  An illustrative example of the chain of skills with sequential
(seq.) or simultaneous (sim.) control policy. Commands generated for each
timestep follow one specific control policy, and the skills contained are
chosen from the designed skill library.

In summary, the LABOR agent uses the LLM to explicitly coordinate the
operation of both hands while continuously generating skill-based commands, to
accomplish complex long-horizon bimanual tasks effectively.

##  IV Experiments

To determine the extent to which an LLM can manage to control the bimanual
robot in everyday tasks, we deploy the LABOR agent on the semi-humanoid
NICOL111The NICOL robot website can be found at https://www.inf.uni-
hamburg.de/en/inst/ab/wtm/research/neurobotics/nicol.html, the Neuro-Inspired
COLlaborator [14], on three typical everyday tasks in the
CoppeliaSim222https://www.coppeliarobotics.com/ simulation. The implementation
of the LABOR agent and experiment design details will be open-sourced upon the
paper’s publication.

###  IV-A LABOR Setup

For the NICOL robot, we design a skill library333The skills are developed
based on the nicol_coppeliasim Python library,
https://github.com/knowledgetechnologyuhh/nicol_coppeliasim as listed in Table
I. There are three categories of skills: Single, Both, and Info. Single or
Both denotes that the skill is designed for a single hand or both hands, while
Info refers to those functions that can obtain information about the robot and
the environment.

We use the OpenAI GPT models444We chose two models, gpt-4-0125-preview and
gpt-3.5-turbo-0125, for our experiments. as the LLM for inference in the LABOR
agent. Since the accomplishment of bimanual tasks requires the LLM to generate
sequential or simultaneous control several times, we take the
LangChain555https://github.com/langchain-ai/langchain Python library framework
to enable the LLM to generate a chain of skills step by step. Based on the
LangChain tool design structure, several tools are provided to the LLM as
follows:

  * •

sequential control: llm_seq_control

required inputs: (left_hand_command, left_parameter, right_hand_command,
right_parameter, if_left_first)

  * •

simultaneous control: llm_simul_control

required inputs: (both_hand_command, parameter)

  * •

get information: (get_info_command, parameter)

where left_hand_command and right_hand_command are chosen from the “Single"
skill category with the parameters specified in left_parameter and
right_parameter respectively, if_left_first is a bool value that specifies if
the left_hand_command should be executed at first or not, and
both_hand_command and get_info_command are chosen from Both and Info skill
categories respectively. With tools for different control policies, the LABOR
agent enables the LLM to choose sequential or simultaneous control in a high-
level manner explicitly.

TABLE I: The full list of skills used for Nicol robot

Skill Name | Parameters | Catagory
---|---|---
move_to_pose | (side, obj_name, offset) | Single
move_to_jab_at | (side, obj_name, direction)
move_to_press_on | (side, obj_name)
delta_move | (side, direction, distance)
top_grasp | (side, obj_name)
side_grasp | (side, obj_name)
release | (side)
flip_down | (side)
wait | (side)
move_both_to_pose | (obj_name) | Both
lateral_hold | (obj_name)
reset_both | None
get_arm_state | None | Info
get_object_distance | (obj_A, obj_B)
get_joint_angle | (joint)

###  IV-B Robot Setup

NICOL has a head that can express stylized facial expressions, and two Robotic
Open-Manipulator-Pro arms with adult-sized five-fingered Seed Robotics RH8D
hands (for details see [14]). With such equipment, NICOL has a large bimanual
workspace to handle everyday objects and tasks. NICOL’s workspace with objects
in the real and simulated world is shown in Fig 5.

Based on the objects listed, there are three tasks designed in the
experiments: ControlScissors, ServeWater, and HoldBowl, which all require the
cooperation of two hands with a chain of skills to accomplish.

(a) real world

(b) simulated world

Figure 5: NICOL workspace with everyday objects. In the simulated environment,
objects placed from left to right are apple, banana, blue cup, bowl, scissors,
and yellow cup.

(a) ControlScissors

(b) ServeWater

(c) HoldBowl

Figure 6: Description of bimanual tasks designed for the NICOL robot. There
are critical stages on three tasks illustrated: (a) the left hand presses on
the scissors handle to stabilize the scissor while the right hand slides
another handle using its finger to open the scissor; (b) the right hand flips
down the water-filled cup to pour out the ‘water’ (small ball) to the empty
cup while the left hand is holding the grasped empty cup; (c) the left and
right hand is holding together the bowl with the apple and banana inside.

(a) ControlScissors (b) ServeWater (c) HoldBowl

Figure 7: Illustration of the number of timesteps used when accomplishing
three tasks in successful cases. The vertical coordinate represents the number
of timesteps required for all successful cases, and the corresponding
horizontal length of each bar denotes the count of cases that required that
number of timesteps. In ServeWater and HoldBowl tasks, orange color denotes
scenes where objects are initially posed at different sides (left/right) of
the robot, while blue denotes all objects are placed on the same side. The
dashed lines of the corresponding colors indicate the minimum number of
timesteps required, as given by the human-designed solution.

In the ControlScissors task, there is a pair of scissors placed horizontally
on the work table, with its tip pointing in various directions: Front, Behind,
Left, and Right. The goal of the task is to stabilize the handle on one side
while sliding the handle on the other side in a certain direction to Open or
Close the scissors. In this way, eight variants have been produced, depending
on the combination of the direction in which the tips of the scissors are
pointed and the task requirements of the switch. The difficulty of this task
comes from the fact that the LLM needs to understand the spatial relationship
between the two handles, and then reason the direction of the handle’s
movement after stabilizing the scissors.

In the ServeWater task, there are two cups on the work table. The yellow cup
is empty and the blue cup is water-filled (we use a small ball as simulated
water). The goal of the task is to use the yellow cup to serve water at a
specified serving position. This means, that the water has to be transferred
from the blue cup to the yellow cup, the yellow cup is then brought to the
serving position, and the blue cup is released back to its original position.
The cups can be in various spatial relations including, including Both_Right,
Both_Left, Left_Blue_Right_Yellow, and Left_Yellow_Right_Blue. Since the robot
cannot reach with its right arm over the left side of the table, and vice
versa, different spatial configurations afford different task solutions.
However, this relationship information is not passed explicitly to the LLM,
which instead receives raw information about the coordinates of the two cups,
placed on the table at random positions. The task itself requires a long skill
chain to accomplish and coupled with the changing positional relationship
between cups and hands, choosing appropriate control policies effectively is
even more challenging.

In the HoldBowl task, there is one large bowl, one apple, and one banana on
the work table. The goal of the task is to grasp the apple and the banana, put
them into the bowl, and hold the bowl to a specified serving position. Similar
to ServeWater, the positions of the apple and banana bring complexity and
variety to this task, including Both_Right, Both_Left,
Left_Apple_Right_Banana, and Left_Banana_Right_Apple, as well as the choices
of control policies. For each case, the sequential skills should be adopted
first to transfer the objects into the bowl, and then the simultaneous ones
for holding the bowl to move to the specified position.

###  IV-C Results

In this section, we report the results of the LABOR agent in controlling the
NICOL robot to solve the above tasks, including success rate and failure
analysis. To evaluate the effectiveness, we repeat each task 100 times and for
each time a variant of the task is randomly chosen. The success rates and
failure rates in three aspects of LABOR when using GPT-4 as the LLM on these
tasks are listed in Table II.

TABLE II: Success and Failure rates of NICOL on three everyday tasks Task | ControlScissors | ServeWater | HoldBowl
---|---|---|---
Success | 79% | 89% | 86%
Failure | Spatial | 12% | 5% | 4%
Temporal | 1% | 6% | 1%
Skill | 8% | 0% | 9%

It is observed that the LABOR agent with GPT-4 demonstrates extraordinary
performance in orchestrating bimanual control in long-horizon manipulation
tasks. Regarding the properties of bimanual manipulation and the LLM itself,
we found that the failure cases mainly come from three aspects including
Temporal Coordination, Spatial Coordination, and Skill Association. In this
sense, Temporal failure refers to the fact that the LLM fails to generate
skills in the correct order, Spatial failure refers that the LLM mishandles
the spatial relationships in the operation, such as the relationship between
two objects or between the robot’s hand and an object, and Skill failure
refers to the understanding deviance of the LLM in the correlation between the
designed skills and the task requirements. To simplify the analysis process,
failures are counted based on the error type that occurred the first time.

It was observed that GPT-4 encountered some difficulties with spatial
reasoning (i.e., moving in a certain direction to open or close the scissors)
in the ControlScissors task. Whereas, the main errors in the ServeWater task
came from temporal and spatial coordination, such as executing pouring before
moving the blue cup over the yellow cup, executing grasping when the selected
hand had already grasped another object, executing grasping when the object
that was outside of the selected hand’s workspace, etc. Moreover, some skill
comprehension biases happened in the ControlScissors and HoldBowl tasks, such
as moving the scissors handle with the hand currently pressing on it,
executing grasping the bowl with a single hand, and so on.

In contrast, when implementing LABOR with GPT-3.5, it only achieved a 49%
success rate in the ControlScisssor task, which might be due to the task
requiring fewer planning steps. Apart from that, it demonstrates its inability
to accomplish either ServeWater or HoldBowl tasks. The failures observed in
these tasks couldn’t be neatly categorized into three aspects, as in such
long-horizon tasks, the GPT-3.5 model’s understanding of the skills and its
temporal and spatial coordination was incorrect at the first step. In
addition, it is not sensitive enough to the information feedback by the
environment, causing it to stay stuck in repeating wrong commands.

In addition to the success rate, we also counted the lengths of the skill
chains generated by GPT-4 in all cases where the task was successfully solved,
and compared them with the optimal human-designed skill chain lengths, as
shown in Fig. 7. Since there are variants in the ServeWater and HoldBowl
quests that single-hand sequential operation, the statistics in these two
tasks are reported separately based on the position in which the two objects
are located, i.e., Same Side and Two Sides. The number of timesteps indicated
by the dashed lines is considered to be the optimal lower bound for the number
of timesteps required. We can observe that there are many cases in ServeWater
and ControlScissors tasks, where an optimal skill chain can be achieved that
is comparable to the human level, while it is pretty close in the HoldBowl
task. To ensure the safety of the two-handed operation, the GPT-4 generates
some waits for single hands thus bringing about an increase in the length of
the chain. At the same time, humans understand the task well enough, thus
allowing for the most efficient utilization of both hands. Notably, due to
GPT-4’s strong in-context reasoning and understanding ability, sometimes long
skill chains can still ensure task accomplishment in the ServeWater and
HoldBowl tasks.

In summary, the LABOR agent with GPT-4 shows remarkable performance in
coordinating the NICOL robot in long-horizon tasks, not only in terms of the
success rate but also in outputting skill chains comparable to the human
level.

##  V Conclusions

In this work, we introduce LABOR, an agent that utilizes an LLM to effectively
orchestrate bimanual control when performing robotic manipulation tasks.
Experimental results on the NICOL robot for three typical everyday tasks
showcase the effectiveness of the LABOR agent.

Limitations and Future Work. Similar to other LLM-based methods, the LABOR
agent is dependent on the performance of LLM like the GPT-4 model, requiring
its reasoning and understanding of the correlations of primitive skills and
task solutions. Besides, since the agent is currently solely running in the
simulated environment instead of the real world, some potential differences in
environmental dynamics might not be sufficiently analyzed or considered, which
brings some unknowns in transferring the agent. In future work, the learning-
based skills combined with vision-based models will be developed to reinforce
the abilities of LABOR agent in bimanual manipulation. As skills expand, the
LABOR agent should be able to control bimanual robots to perform more complex
tasks. Furthermore, it is a promising direction to include obstacle detection
and collision-free motion planning models in dealing with rich-contact tasks.

##  VI ACKNOWLEDGMENT

The authors would like to thank Jan-Gerrit Habekost and Lennart Clasmeier for
their work and support on the NICOL robot in the simulation.

## References

  * [1] S. S. Mirrazavi Salehian, N. B. Figueroa Fernandez, and A. Billard, “Dynamical system-based motion planning for multi-arm systems: Reaching for moving objects,” in _Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI-17)_ , 2017, pp. 4914–4918.
  * [2] J. Grannen, Y. Wu, S. Belkhale, and D. Sadigh, “Learning bimanual scooping policies for food acquisition,” in _6th Annual Conference on Robot Learning_ , 2022. [Online]. Available: https://openreview.net/forum?id=qDtbMK67PJG
  * [3] R. Chitnis, S. Tulsiani, S. Gupta, and A. Gupta, “Efficient bimanual manipulation using learned task schemas,” in _2020 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2020, pp. 1149–1155.
  * [4] K. S. Luck and H. B. Amor, “Extracting bimanual synergies with reinforcement learning,” in _2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2017, pp. 4805–4812.
  * [5] R. Zollner, T. Asfour, and R. Dillmann, “Programming by demonstration: dual-arm manipulation tasks for humanoid robots,” in _2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)(IEEE Cat. No. 04CH37566)_ , vol. 1. IEEE, 2004, pp. 479–484.
  * [6] S. Stepputtis, M. Bandari, S. Schaal, and H. B. Amor, “A system for imitation learning of contact-rich bimanual manipulation policies,” in _2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2022, pp. 11 810–11 817.
  * [7] C. Smith, Y. Karayiannidis, L. Nalpantidis, X. Gratal, P. Qi, D. V. Dimarogonas, and D. Kragic, “Dual arm manipulation—a survey,” _Robotics and Autonomous systems_ , vol. 60, no. 10, pp. 1340–1353, 2012.
  * [8] J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. H. Chi, Q. V. Le, and D. Zhou, “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models,” in _Advances in Neural Information Processing Systems_ , May 2022. [Online]. Available: https://openreview.net/forum?id=_VjQlMeSB_J
  * [9] X. Zhao, M. Li, W. Lu, C. Weber, J. H. Lee, K. Chu, and S. Wermter, “Enhancing zero-shot chain-of-thought reasoning in large language models through logic,” in _2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)_ , May 2024.
  * [10] S. Bubeck, V. Chandrasekaran, R. Eldan, J. Gehrke, E. Horvitz, E. Kamar, P. Lee, Y. T. Lee, Y. Li, S. Lundberg, H. Nori, H. Palangi, M. T. Ribeiro, and Y. Zhang, “Sparks of Artificial General Intelligence: Early experiments with GPT-4,” no. arXiv:2303.12712, p. 2303.12712, Apr. 2023. [Online]. Available: http://arxiv.org/abs/2303.12712
  * [11] M. Ahn, A. Brohan, N. Brown, Y. Chebotar, O. Cortes, B. David, C. Finn, C. Fu, K. Gopalakrishnan, K. Hausman, _et al._ , “Do as I can, not as I say: Grounding language in robotic affordances,” _arXiv preprint arXiv:2204.01691_ , 2022.
  * [12] X. Zhao, M. Li, C. Weber, M. B. Hafez, and S. Wermter, “Chat with the Environment: Interactive Multimodal Perception using Large Language Models,” in _2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , no. arXiv:2303.08268. arXiv, Oct. 2023, p. 2303.08268. [Online]. Available: http://arxiv.org/abs/2303.08268
  * [13] J. Zhang, J. Zhang, K. Pertsch, Z. Liu, X. Ren, M. Chang, S.-H. Sun, and J. J. Lim, “Bootstrap Your Own Skills: Learning to Solve New Tasks with Large Language Model Guidance,” in _7th Annual Conference on Robot Learning_. arXiv, Oct. 2023, p. 2310.10021. [Online]. Available: http://arxiv.org/abs/2310.10021
  * [14] M. Kerzel, P. Allgeuer, E. Strahl, N. Frick, J.-G. Habekost, M. Eppe, and S. Wermter, “NICOL: A neuro-inspired collaborative semi-humanoid robot that bridges social interaction and reliable manipulation,” _IEEE Access_ , vol. 11, pp. 123 531–123 542, 2023.
  * [15] Y. Koga and J.-C. Latombe, “Experiments in dual-arm manipulation planning,” in _Proceedings 1992 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE Computer Society, 1992, pp. 2238–2239.
  * [16] P. Lertkultanon and Q.-C. Pham, “A certified-complete bimanual manipulation planner,” _IEEE Transactions on Automation Science and Engineering_ , vol. 15, no. 3, pp. 1355–1368, 2018.
  * [17] E. Ng, Z. Liu, and M. Kennedy, “It takes two: Learning to plan for human-robot cooperative carrying,” in _2023 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2023, pp. 7526–7532.
  * [18] Y. Chen, T. Wu, S. Wang, X. Feng, J. Jiang, Z. Lu, S. McAleer, H. Dong, S.-C. Zhu, and Y. Yang, “Towards human-level bimanual dexterous manipulation with reinforcement learning,” _Advances in Neural Information Processing Systems_ , vol. 35, pp. 5150–5163, 2022.
  * [19] F. Amadio, A. Colomé, and C. Torras, “Exploiting symmetries in reinforcement learning of bimanual robotic tasks,” _IEEE Robotics and Automation Letters_ , vol. 4, no. 2, pp. 1838–1845, 2019.
  * [20] R. Lioutikov, O. Kroemer, G. Maeda, and J. Peters, “Learning manipulation by sequencing motor primitives with a two-armed robot,” in _Intelligent Autonomous Systems 13: Proceedings of the 13th International Conference (IAS-13)_. Springer, 2016, pp. 1601–1611.
  * [21] L. P. Ureche and A. Billard, “Constraints extraction from asymmetrical bimanual tasks and their use in coordinated behavior,” _Robotics and autonomous systems_ , vol. 103, pp. 222–235, 2018.
  * [22] O. Kroemer, S. Niekum, and G. Konidaris, “A review of robot learning for manipulation: Challenges, representations, and algorithms,” _Journal of machine learning research_ , vol. 22, no. 30, pp. 1–82, 2021.
  * [23] M. Li*, X. Zhao*, J. H. Lee, C. Weber, and S. Wermter, “Internally Rewarded Reinforcement Learning,” in _40th International Conference on Machine Learning (ICML)_ , July 2023, p. 2302.00270. [Online]. Available: http://arxiv.org/abs/2302.00270
  * [24] X. Zhao, C. Weber, M. B. Hafez, and S. Wermter, “Impact Makes a Sound and Sound Makes an Impact: Sound Guides Representations and Explorations,” in _2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2022, pp. 2512–2518.
  * [25] S. Yang, O. Nachum, Y. Du, J. Wei, P. Abbeel, and D. Schuurmans, “Foundation Models for Decision Making: Problems, Methods, and Opportunities,” _arXiv preprint arXiv:2303.04129_ , p. 2303.04129, 2023.
  * [26] W. Huang, C. Wang, R. Zhang, Y. Li, J. Wu, and L. Fei-Fei, “VoxPoser: Composable 3D value maps for robotic manipulation with language models,” in _7th Annual Conference on Robot Learning_ , 2023, p. 2307.05973. [Online]. Available: https://openreview.net/forum?id=9_8LF30mOC
  * [27] G. Wang, Y. Xie, Y. Jiang, A. Mandlekar, C. Xiao, Y. Zhu, L. Fan, and A. Anandkumar, “Voyager: An Open-Ended Embodied Agent with Large Language Models,” no. arXiv:2305.16291, p. 2305.16291, Oct. 2023. [Online]. Available: http://arxiv.org/abs/2305.16291
  * [28] J. Liang, W. Huang, F. Xia, P. Xu, K. Hausman, b. ichter, P. Florence, and A. Zeng, “Code as policies: Language model programs for embodied control,” in _Workshop on Language and Robotics at CoRL 2022_ , 2022, p. 2209.07753. [Online]. Available: https://openreview.net/forum?id=fmtvpopfLC6
  * [29] Y. J. Ma, W. Liang, G. Wang, D.-A. Huang, O. Bastani, D. Jayaraman, Y. Zhu, L. Fan, and A. Anandkumar, “Eureka: Human-Level Reward Design via Coding Large Language Models,” no. arXiv:2310.12931, p. 2310.12931, Oct. 2023. [Online]. Available: http://arxiv.org/abs/2310.12931
  * [30] K. Chu, X. Zhao, C. Weber, M. Li, and S. Wermter, “Accelerating reinforcement learning of robotic manipulations via feedback from large language models,” in _7th Conference on Robot Learning (CoRL 2023) Workshop , Atlanta, Georgia USA_ , Nov. 2023.
  * [31] J. R. Boehm, N. P. Fey, and A. M. Fey, “Online recognition of bimanual coordination provides important context for movement data in bimanual teleoperated robots,” in _2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2021, pp. 6248–6255.
  * [32] F. Krebs and T. Asfour, “A bimanual manipulation taxonomy,” _IEEE Robotics and Automation Letters_ , vol. 7, no. 4, pp. 11 031–11 038, 2022.


will denote its pullback to $\mathscr{S}_{K^{\circ}}{(\mathsf{G},\mathsf{X})}$
by $\mathcal{P}$. From the short exact sequence (4.3.8), we see that the
pushout $\mathcal{P}\times^{\mathcal{G}}j_{\ast}\pi_{0}(\mathcal{G})$ is a
torsor for the sheaf of abelian groups $j_{\ast}\pi_{0}(\mathcal{G})$. It
suffices to construct a section of it over $S$. Indeed, given such a section,
the pullback along this section of the natural map
$\mathcal{P}\to\mathcal{P}\times^{\mathcal{G}}j_{\ast}\pi_{0}(\mathcal{G})$
gives a reduction of $\mathcal{P}$ to a $\mathcal{G}^{\circ}$-torsor.

By the 2-commutativity of the diagram (4.3.2) applied to
$\mathcal{G}^{\circ}\to\mathcal{G}$, we have a reduction of
$\mathcal{P}^{\lozenge}$ to a
$\left(\mathcal{G}^{\circ}\right)^{\lozenge}$-torsor
$\tilde{\mathcal{Q}}\subset\mathcal{P}^{\lozenge}$. This is gives an
$S^{\lozenge/}$-point of

(4.3.10)
$\displaystyle\mathcal{P}^{\lozenge}\times^{\mathcal{G}^{\lozenge}}j_{\ast}\underline{\pi_{0}(\mathcal{G})}\cong\mathcal{P}^{\lozenge/}\times^{\mathcal{G}^{\lozenge/}}j_{\ast}\underline{\pi_{0}(\mathcal{G})}\cong\left(\mathcal{P}\times^{\mathcal{G}}j_{\ast}\pi_{0}(\mathcal{G})\right)^{\lozenge/}$

where we used Lemma 4.3.4 for the first isomorphism. We want to show that this
point is induced by an $S$-point of
$\mathcal{P}\times^{\mathcal{G}}j_{\ast}\pi_{0}(\mathcal{G})$.

We first observe that

(4.3.11)
$\mathcal{P}\times^{\mathcal{G}}j_{\ast}\pi_{0}(\mathcal{G})=j_{\ast}\left(\mathcal{G}_{k_{E}}\times^{\mathcal{P}_{k_{E}}}\pi_{0}(\mathcal{G})\right).$

From this it follows that

(4.3.12)
$H^{0}\left(S^{\lozenge/},\left(\mathcal{P}\times^{\mathcal{G}}j_{\ast}\pi_{0}(\mathcal{G})\right)^{\lozenge/}\right)=H^{0}\left(S_{k_{E}}^{\diamond},\left(\mathcal{P}_{k_{E}}\times^{\mathcal{G}_{k_{E}}}\pi_{0}(\mathcal{G})\right)^{\diamond}\right).$

Let $S_{k_{E}}^{\mathrm{perf}}$ denote the perfection of $S_{k_{E}}$. It
follows from the full-faithfulness of the functor $X\mapsto X^{\diamond}$ on
perfect schemes, see [SW20, 18.3.1], that

(4.3.13) $\displaystyle
H^{0}\left(S_{k_{E}}^{\diamond},\left(\mathcal{P}_{k_{E}}\times^{\mathcal{G}_{k_{E}}}\pi_{0}(\mathcal{G})\right)^{\diamond}\right)$
$\displaystyle=H^{0}\left(S_{k_{E}}^{\mathrm{perf}},\left(\mathcal{P}_{k_{E}}\times^{\mathcal{G}_{k_{E}}}\pi_{0}(\mathcal{G})\right)^{\mathrm{perf}}\right).$

By topological invariance of the étale site, the right hand side identifies
with
$H^{0}(S_{k_{E}},\mathcal{P}_{k_{E}}\times^{\mathcal{G}_{k_{E}}}\pi_{0}(\mathcal{G}))$.
But from (4.3.11) and the definition of $j_{\ast}$, we have

(4.3.14) $\displaystyle
H^{0}\left(S_{k_{E}},\mathcal{P}_{k_{E}}\times^{\mathcal{G}_{k_{E}}}\pi_{0}(\mathcal{G})\right)=H^{0}\left(S,j_{\ast}\left(\mathcal{P}_{k_{E}}\times^{\mathcal{G}_{k_{E}}}\pi_{0}(\mathcal{G})\right)\right).$

By combining these bijections, we obtain from the
$(\mathcal{G}^{\circ})^{\lozenge}$-torsor $\tilde{\mathcal{Q}}$ an $S$-point
of $\mathcal{P}\times^{\mathcal{G}}j_{\ast}\pi_{0}(\mathcal{G})$, i.e., a
$\mathcal{G}^{\circ}$-torsor $\mathcal{Q}$. Clearly
$\mathcal{Q}^{\lozenge}\cong\tilde{\mathcal{Q}}$, since both are the pullback
of $\mathcal{P}^{\lozenge}$ along the same section of
$\mathcal{P}^{\lozenge}\times^{\mathcal{G}^{\lozenge}}j_{\ast}\underline{\pi_{0}(\mathcal{G})}$
over $S^{\lozenge}$.

Thus we obtain the desired morphism $\pi_{\mathrm{dR},\mathcal{G}^{\circ}}$,
and it follows the construction that (4.3.7) commutes. That
$\pi_{\mathrm{dR},\mathcal{G}^{\circ}}$ recovers the canonical model of the
standard principle bundle on the generic fiber follows from the corresponding
fact for $\pi_{\mathrm{dR},\mathcal{G}}$. Finally, it remains to show that
$\pi_{\mathrm{dR},\mathcal{G}^{\circ}}$ is smooth. This can be checked after
pullback to the smooth cover
$\mathbb{M}_{\mathcal{G},\mu}\to\left[\mathbb{M}_{\mathcal{G},\mu}/\mathcal{G}^{\circ}\right]$,
but here this map is given by

$\mathcal{Q}\hookrightarrow\mathcal{P}\xrightarrow{\pi_{\mathrm{dR},\mathcal{G}}|_{\mathcal{P}}}\mathbb{M}_{\mathcal{G},\mu}.$

While the first map is an open immersion and the second map is smooth by
assumption, the composition is smooth. This concludes the proof that
$\pi_{\mathrm{dR},\mathcal{G}}$ is a scheme-theoretic local model diagram for
$\mathscr{S}_{K^{\circ}}{(\mathsf{G},\mathsf{X})}$. ∎

#### 4.3.5.

We now combine Proposition 4.3.3 with Theorem A.3.3 to deduce the existence of
scheme-theoretic local model diagrams for Shimura varieties of Hodge type at
parahoric level. For this we need the notion of a connected reductive group
$G$ over ${\mathbb{Q}_{p}}$ being acceptable, [KZ21, Definition 3.3.2], this
is automatic if $p\geq 5$ or if $G$ splits over a tamely ramified extension of
${\mathbb{Q}_{p}}$. Similarly we need the notion of $G$ being $R$-smooth, see
[DY24, Definition 2.10] and [KZ21, Definition 2.4.3], this is automatic if $G$
splits over a tamely ramified extension of ${\mathbb{Q}_{p}}$ by [KZ21,
Proposition 2.4.6].

######  Theorem 4.3.6.

Let $(\mathsf{G},\mathsf{X},\mathcal{G}^{\circ})$ be a triple of Hodge type
with $\mathcal{G}^{\circ}$ a parahoric. If $p$ is coprime to
$2\cdot\pi_{1}(G^{\mathrm{der}})$ and the group $G$ is acceptable and
$R$-smooth, then $\mathscr{S}_{K^{\circ}}{(\mathsf{G},\mathsf{X})}$ admits a
scheme-theoretic local model diagram.

###### Proof.

Let us write $\mathcal{G}^{\circ}$ as the relative identity component of a
stabilizer quasi-parahoric $\mathcal{G}$. Then under our assumptions Theorem
A.3.3 applies, see Remark A.3.1, to produce a scheme-theoretic local model
diagram for $\mathscr{S}_{K}{(\mathsf{G},\mathsf{X})}$. The result is now a
consequence of Proposition 4.3.3. ∎

###  4.4. Rapoport–Zink uniformization

Let the notation be as in Section 4.3. In particular, $\mathcal{G}$ is a
stabilizer quasi–parahoric model of $G$ over ${\mathbb{Z}_{p}}$ with
$K_{p}=\mathcal{G}({\mathbb{Z}_{p}})$. Denote by $k_{E}$ the residue field of
$\mathcal{O}_{E}$ as before. For $\ell$ an algebraically closed field in
characteristic $p$ together with a fixed embedding $e\colon
k_{E}\hookrightarrow\ell$, write

${W_{\mathcal{O}_{E},e}(\ell)}=\mathcal{O}_{E}\otimes_{W(k_{E}),e}W(\ell)$

as before. Then for $b\in G(W(\ell)[1/p])$ we have the v-sheaves
$\mathcal{M}^{\mathrm{int}}_{\mathcal{G},b,\mu,e},\mathcal{M}^{\mathrm{int}}_{\mathcal{G}^{\circ},b,\mu,e}$
and we will also consider the image
$\mathcal{M}^{\mathrm{int}}_{\mathcal{G},b,\mu,\delta=1,e}$ of
$\mathcal{M}^{\mathrm{int}}_{\mathcal{G}^{\circ},b,\mu,e}\to\mathcal{M}^{\mathrm{int}}_{\mathcal{G},b,\mu,e}$.

Let $x\in\mathscr{S}_{K_{p}}{(\mathsf{G},\mathsf{X})}(\ell)$, then its image
under $\pi_{\mathrm{crys}}$ defines a $\operatorname{Spd}(\ell)$-point $b_{x}$
of $\mathrm{Sht}_{\mathcal{G},\mu,\delta=1}$. Let $e:k_{E}\to\ell$ be the map
corresponding to
$x:\operatorname{Spec}(\ell)\to\mathscr{S}_{K_{p}}{(\mathsf{G},\mathsf{X})}_{k_{E}}\to\operatorname{Spec}(k_{E})$,
then attached to $x$ is a base point

$x_{0}\colon\operatorname{Spd}(\ell)\to\mathcal{M}^{\mathrm{int}}_{\mathcal{G},b_{x},\mu,e},$

given by the $\operatorname{Spd}(\ell)$-point of
$\mathrm{Sht}_{\mathcal{G},\mu}$ corresponding to $\pi_{\mathrm{crys}}(x)$,
see Remark 3.1.7. In fact, since
$\pi_{\mathrm{crys}}(x)\in\mathrm{Sht}_{\mathcal{G},\mu,\delta=1}$, our base
point lies in
$\mathcal{M}^{\mathrm{int}}_{\mathcal{G},b_{x},\mu,\delta=1,e}(\operatorname{Spd}(\ell))$.

######  Theorem 4.4.1.

If ${(\mathsf{G},\mathsf{X})}$ is of Hodge type, then there exists a
uniformization map

$\Theta_{\mathcal{G},x}\colon\mathcal{M}^{\mathrm{int}}_{\mathcal{G},b_{x},\mu,\delta=1,e}\to\mathscr{S}_{K_{p}}{(\mathsf{G},\mathsf{X})}_{W_{\mathcal{O}_{E},e}(\ell)}^{\diamond}$

sending the base point $x_{0}$ to $x$, which restricts to an isomorphism

$\Theta_{\mathcal{G},x}\colon\widehat{\mathcal{M}^{\mathrm{int}}_{\mathcal{G},b_{x},\mu,\delta=1,e}}_{/x_{0}}\xrightarrow{\cong}(\widehat{\mathscr{S}_{K_{p}}{(\mathsf{G},\mathsf{X})}_{W_{\mathcal{O}_{E},e}(\ell)}}_{/x})^{\lozenge}.$

Moreover the composition of $\Theta_{\mathcal{G},x}$ with
$\pi_{\mathrm{crys}}$

(4.4.1)
$\displaystyle\mathcal{M}^{\mathrm{int}}_{\mathcal{G},b_{x},\mu,\delta=1,e}\to\mathscr{S}_{K_{p}}{(\mathsf{G},\mathsf{X})}_{W_{\mathcal{O}_{E},e}(\ell)}^{\diamond}\to\mathrm{Sht}_{\mathcal{G},\mu}\times_{\operatorname{Spd}(\mathcal{O}_{E})}\operatorname{Spd}({W_{\mathcal{O}_{E},e}(\ell)})$

is 2-isomorphic to the natural map of Lemma 3.1.6.

######  Remark 4.4.2.

Under certain additional hypotheses on ${(\mathsf{G},\mathsf{X})}$, it is
conjectured in [HK19, Axiom A] that (for $\ell=\overline{\mathbb{F}}_{p})$
there should be a uniformization map
$\mathcal{M}^{\mathrm{int}}_{\mathcal{G},b_{x},\mu,e}(\overline{\mathbb{F}}_{p})\to\mathscr{S}_{K_{p}}{(\mathsf{G},\mathsf{X})}(\overline{\mathbb{F}}_{p})$.
If $\Pi_{\mathcal{G}}\not=1$, then such a map cannot upgrade to a
uniformization map as in Theorem 4.4.1. Indeed, the natural map
$\mathscr{S}_{K_{p}}{(\mathsf{G},\mathsf{X})}^{\diamond}\to\mathrm{Sht}_{\mathcal{G},\mu}$
factors through $\mathrm{Sht}_{\mathcal{G},\mu,\delta=1}$ by Remark 4.1.5,
while the natural map
$\mathcal{M}^{\mathrm{int}}_{\mathcal{G},b_{x},\mu,e}\to\mathrm{Sht}_{\mathcal{G},\mu}$
does not.

###### Proof.

The proof of [GLX23, Corollary 6.3] goes through171717We specifically mean the
version of the proof linked in our bibliography, which differs from the Arxiv
version at the time of writing., with the following modification. In the
notation of [GLX23, Section 3.4], we have an isomorphism, where the right hand
side is the local Shimura variety of level $\mathcal{G}({\mathbb{Z}_{p}})$
over $\operatorname{Spd}(\breve{E})$ associated to
$(G,b_{x},\mu,\mathcal{G}({\mathbb{Z}_{p}}))$,

(4.4.2)
$\displaystyle\mathcal{M}^{\mathrm{int}}_{\mathcal{G},b_{x},\mu,\delta=1,e}$
$\displaystyle\times_{\operatorname{Spd}({W_{\mathcal{O}_{E},e}(\ell)})}\operatorname{Spd}({W_{\mathcal{O}_{E},e}(\ell)}[1/p])$
(4.4.3)
$\displaystyle\simeq\operatorname{Sht}_{G,b_{x},\mu,\mathcal{G}({\mathbb{Z}_{p}})}\times_{\operatorname{Spd}(\breve{E})}\operatorname{Spd}({W_{\mathcal{O}_{E},e}(\ell)}[1/p])$

see [PR22, Theorem 4.5.1]. This means that we can follow the construction in
[GLX23, Corollary 3.11] to construct a map

(4.4.4) $\displaystyle
G({\mathbb{Q}_{p}})/\mathcal{G}({\mathbb{Z}_{p}})\to\pi_{0}\left(\mathcal{M}^{\mathrm{int}}_{\mathcal{G},b_{x},\mu,\delta=1,e}\right).$

To show that this map is surjective, we use the commutative diagram

(4.4.5)
${G({\mathbb{Q}_{p}})/\mathcal{G}^{\circ}({\mathbb{Z}_{p}})}$${\pi_{0}\left(\mathcal{M}^{\mathrm{int}}_{\mathcal{G}^{\circ},b_{x},\mu,e}\right)}$${G({\mathbb{Q}_{p}})/\mathcal{G}({\mathbb{Z}_{p}})}$${\pi_{0}\left(\mathcal{M}^{\mathrm{int}}_{\mathcal{G},b_{x},\mu,\delta=1,e}\right),}$

and the surjectivity of
$\mathcal{M}^{\mathrm{int}}_{\mathcal{G}^{\circ},b_{x},\mu,e}\to\mathcal{M}^{\mathrm{int}}_{\mathcal{G},b_{x},\mu,\delta=1,e}$
and
$G({\mathbb{Q}_{p}})/\mathcal{G}^{\circ}({\mathbb{Z}_{p}})\to\pi_{0}\left(\mathcal{M}^{\mathrm{int}}_{\mathcal{G}^{\circ},b_{x},\mu,e}\right)$,
see [GLX23, Corollary 3.11] for the latter. With this in mind, the rest of the
proof of [GLX23, Corollary 6.3] goes through. ∎

######  Corollary 4.4.3.

For $z\in\mathscr{S}_{K_{p}^{\circ}}{(\mathsf{G},\mathsf{X})}(\ell)$ with
image $x\in\mathscr{S}_{K_{p}}{(\mathsf{G},\mathsf{X})}(\ell)$, there is a
unique map

(4.4.6)
$\displaystyle\Theta_{\mathcal{G}^{\circ},z}:\mathcal{M}^{\mathrm{int}}_{\mathcal{G}^{\circ},b_{x},\mu,e}\to\mathscr{S}_{K_{p}^{\circ}}{(\mathsf{G},\mathsf{X})}_{{W_{\mathcal{O}_{E},e}(\ell)}}^{\diamond}$

sending the base point $x_{0}$ to $z$, that restricts to an isomorphism

$\Theta_{\mathcal{G}^{\circ},x}\colon\widehat{\mathcal{M}^{\mathrm{int}}_{\mathcal{G}^{\circ},b_{x},\mu,e}}_{/x_{0}}\xrightarrow{\cong}(\widehat{\mathscr{S}_{K_{p}^{\circ}}{(\mathsf{G},\mathsf{X})}_{W_{\mathcal{O}_{E},e}(\ell)}}_{/z})^{\lozenge}.$

###### Proof.

If we define $Y$ (and $\Theta_{\mathcal{G}^{\circ},z}$) as the fiber product

(4.4.7)
${Y}$${\mathscr{S}_{K_{p}^{\circ}}{(\mathsf{G},\mathsf{X})}_{{W_{\mathcal{O}_{E},e}(\ell)}}^{\diamond}}$${\mathcal{M}^{\mathrm{int}}_{\mathcal{G},b_{x},\mu,\delta=1,e}}$${\mathscr{S}_{K_{p}}{(\mathsf{G},\mathsf{X})}_{{W_{\mathcal{O}_{E},e}(\ell)}}^{\diamond},}$$\scriptstyle{\Theta_{\mathcal{G}^{\circ},z}}$$\scriptstyle{\Theta_{\mathcal{G},x}}$

then by stacking fiber product squares (see the proof of Theorem 4.1.12 and
Theorem 4.2.3) we get a fiber product diagram

(4.4.8)
${Y}$${\mathrm{Sht}_{\mathcal{G}^{\circ},\mu}}$${\mathcal{M}^{\mathrm{int}}_{\mathcal{G},b_{x},\mu,\delta=1,e}}$${\mathrm{Sht}_{\mathcal{G},\mu,\delta=1}.}$

It follows from the proof of Theorem 3.3.5 and Lemma 3.1.6 that
$Y\to\mathcal{M}^{\mathrm{int}}_{\mathcal{G},b_{x},\mu,\delta=1,e}$ is
isomorphic to
$\mathcal{M}^{\mathrm{int}}_{\mathcal{G}^{\circ},b_{x},\mu,e}\to\mathcal{M}^{\mathrm{int}}_{\mathcal{G},b_{x},\mu,\delta=1,e}$,
proving the corollary. ∎

##  Appendix A On scheme-theoretic local model diagrams

In this appendix we flesh out the remark in [PR24, Section 4.9.2] that the
local model diagrams of [KZ21, Theorem 5.1.10] and [KP18, Theorem 4.2.7] give
scheme-theoretic local model diagrams for integral models of Hodge-type
Shimura varieties.

###  A.1. Some rational $p$-adic Hodge theory

Let $X$ be a smooth rigid space over a finite extension $E$ of
${\mathbb{Q}_{p}}$, and $\mathbb{L}$ be a de Rham ${\mathbb{Z}_{p}}$-local
system of rank $n$ on $X_{\text{pro\'{e}t}}$. Associated to $\mathbb{L}$, we
get a filtered vector bundle with integrable connection
$D_{\mathrm{dR}}(\mathbb{L})=(\mathcal{E},\operatorname{Fil}^{\bullet},\nabla)$
on $X_{\text{\'{e}t}}$ satisfying Griffiths transversality, see [LZ17, Theorem
3.9]. We have two $\mathbb{B}_{\mathrm{dR}}^{+}$-lattices on
$X_{\text{pro\'{e}t}}$:

(A.1.1)
$\displaystyle\mathbb{M}:=\mathbb{L}\otimes_{\underline{{\mathbb{Z}_{p}}}}\mathbb{B}_{\mathrm{dR}}^{+},\quad\text{
and
}\quad\mathbb{M}_{0}:=(D_{\mathrm{dR}}(\mathbb{L})\otimes_{\mathcal{O}_{X}}\mathcal{O}\mathbb{B}_{\mathrm{dR}}^{+})^{\nabla=0}.$

Here for the construction of $\mathbb{M}_{0}$, we take flat sections for the
induced connection
$\nabla_{D_{\mathrm{dR}}(\mathbb{L})}\otimes\mathrm{id}+\mathrm{id}\otimes\nabla_{\mathcal{O}\mathbb{B}_{\mathrm{dR}}^{+}}$.
Also, by [PR24, Definition 2.6.4, Proposition 2.6.3, Proposition 2.5.1], there
is an induced shtuka $\mathscr{V}_{\mathbb{L}}$ of rank $n$ on $X^{\lozenge}$.

Let $S=\operatorname{Spa}(R,R^{+})$ be an affinoid perfectoid space of
characteristic $p$ together with a map $f:S\to X^{\lozenge}$ corresponding to
an untilt $S^{\sharp}$ and a map $f:S^{\sharp}\to X$. Note that by
construction of $\mathscr{V}_{\mathbb{L}}$, the completion of
$\mathscr{V}_{\mathbb{L}}$ (resp.
$\mathrm{Frob}_{S}^{\ast}\mathscr{V}_{\mathbb{L}}$) along $S^{\sharp}$ is
canonically identified with $f^{\ast}\mathbb{M}$ (resp.
$f^{\ast}\mathbb{M}_{0}$), where these pullbacks are defined as in the proof
of [PR24, Proposition 2.6.3]. We equip
${\left.\kern-1.2pt\operatorname{Frob}_{S}^{\ast}\mathscr{V}_{\mathbb{L}}\vphantom{\big{|}}\right\rvert_{S^{\sharp}}}$
with a decreasing filtration such that (the Tate twist can be ignored)

$\mathrm{Fil}^{-i}({\left.\kern-1.2pt\operatorname{Frob}_{S}^{\ast}\mathscr{V}_{\mathbb{L}}\vphantom{\big{|}}\right\rvert_{S^{\sharp}}})\coloneqq\mathbb{M}\cap\mathrm{Fil}^{i}(\mathbb{M}_{0})/\mathbb{M}\cap\mathrm{Fil}^{i+1}(\mathbb{M}_{0})(-i).$

######  Lemma A.1.1.

There is a natural isomorphism between filtered vector bundles
$D_{\mathrm{dR}}(\mathbb{L})_{S^{\sharp}}$ and
${\left.\kern-1.2pt\operatorname{Frob}_{S}^{\ast}\mathscr{V}_{\mathbb{L}}\vphantom{\big{|}}\right\rvert_{S^{\sharp}}}$.

###### Proof.

The underlying vector bundle of
$D_{\mathrm{dR}}(\mathbb{L})|_{X_{\text{pro\'{e}t}}}$ can be recovered from
$\mathbb{M}_{0}$ by taking 0th graded piece, see the discussion after the
proof of [Sch13, Lemma 7.7]. Its filtration can be recovered from the relative
position of $\mathbb{M}$ and $\mathbb{M}_{0}$, as explained in [Sch13,
Proposition 7.9]. Since
${\left.\kern-1.2pt\operatorname{Frob}_{S}^{\ast}\mathscr{V}_{\mathbb{L}}\vphantom{\big{|}}\right\rvert_{S^{\sharp}}}=\mathrm{gr}^{0}(\mathbb{M}_{0})$,
by comparing with the formula in [Sch13, Proposition 7.9], we see that it
agrees with $D_{\mathrm{dR}}(\mathbb{L})|_{S^{\sharp}}$ as filtered vector
bundles. This identification is moreover natural in $S$. ∎

Note that Lemma A.1.1 gives us a $2$-commutative diagram of tensor functors
(discarding the connection on $D_{\mathrm{dR}}(-)$)

(A.1.2) ${\\{\text{de Rham ${\mathbb{Z}_{p}}$-local systems on
}X_{\text{pro\'{e}t}}\\}}$${\\{\text{Shtukas on
}X^{\lozenge}\\}}$${\\{\text{Filtered vector bundles on
}X_{\text{\'{e}t}}\\}}$${\\{\text{Filtered vector bundles on
}X^{\lozenge}\\}.}$$\scriptstyle{\mathrm{PR}}$$\scriptstyle{D_{\mathrm{dR}}}$

Here $\mathrm{PR}$ is the (exact) tensor functor of [PR24, Definition 2.6.4],
and the right vertical arrow takes a shtuka $\mathscr{V}$ on $X^{\lozenge}$ to
${\left.\kern-1.2pt\operatorname{Frob}_{S}^{\ast}\mathscr{V}_{\mathbb{L}}\vphantom{\big{|}}\right\rvert_{S^{\sharp}}}$
as in Lemma A.1.1.

#### A.1.2.

Now suppose that $X=Z^{\mathrm{an}}$ for a smooth $E$-scheme $Z$, and that
there is an abelian scheme $\pi:A\to Z$ of relative dimension $g$ such that
$\mathbb{L}\coloneqq
R^{1}\pi_{\ast,\text{pro\'{e}t}}\underline{{\mathbb{Z}_{p}}}$. Then as
explained in [PR24, Example 2.6.2],

(A.1.3) $\displaystyle
D_{\mathrm{dR}}(\mathbb{L})\simeq(\mathcal{H}^{1}_{\mathrm{dR}}(A/X),\operatorname{Fil}^{\bullet}_{\operatorname{Hdg}}),$

where $\mathcal{H}^{1}_{\mathrm{dR}}(A/X)$ denotes the first relative de Rham
cohomology of $\pi$, equipped with its Hodge filtration
$\operatorname{Fil}^{\bullet}_{\operatorname{Hdg}}$. To be precise, there is a
natural surjective map of vector bundles
$\mathcal{H}_{1}^{\mathrm{dR}}(A/X)\to\operatorname{Lie}(A^{\vee})$ with
kernel $\operatorname{Fil}^{1}_{\operatorname{Hdg}}$. Note that it follows
from [CS17, Proposition 2.2.3] that $\mathbb{M}_{0}\subset\mathbb{M}$.

######  Lemma A.1.3.

We can identify $\xi\mathbb{M}\subset\mathbb{M}_{0}$ with the kernel of the
map

(A.1.4) $\displaystyle\mathbb{M}_{0}\to
H^{1}_{\mathrm{dR}}(A/X)\to\operatorname{Lie}(A^{\vee})$

###### Proof.

Note that the Hodge filtration on $H^{1}_{\mathrm{dR}}(A/X)$ only has two
jumps. The lemma now follows from the explicit formula of the filtration
above. ∎

Let $\Lambda={\mathbb{Z}_{p}}^{\oplus 2g}$ and let
$P_{\Lambda}=\underline{\operatorname{Isom}}_{X}(\Lambda\otimes_{{\mathbb{Z}_{p}}}\mathcal{O}_{X},\mathcal{H}^{1}_{\mathrm{dR}}(A/X))$
be the frame bundle of $\mathcal{H}^{1}_{\mathrm{dR}}(A/X)$. Let
$\operatorname{Gr}_{g,\Lambda}$ be the Grassmannian of $g$-dimensional
quotients of $\Lambda$ considered as scheme over ${\mathbb{Z}_{p}}$. Then
there is a map of adic spaces

$\displaystyle\pi_{\mathrm{dR}}:P_{\Lambda}\to\operatorname{Gr}_{g,\Lambda,E}^{\mathrm{an}}$

defined using the natural quotient map
$\Lambda\otimes_{{\mathbb{Z}_{p}}}\mathcal{O}_{X}\cong\mathcal{H}^{1}_{\mathrm{dR}}(A/X)\to\operatorname{Lie}(A^{\vee})$.

#### A.1.4.

We adopt the notation from A.1.2 above. Let $\mathscr{V}$ be the vector bundle
shtuka induced from $\mathbb{L}$. Then by Lemma A.1.3, $\mathscr{V}$ is
minuscule of height $2g$ and dimension $g$ in the sense of [PR24, Definition
2.2.2]. By [PR24, Lemma 2.4.4], we can think of it as a
${\operatorname{GL}(\Lambda)}$-shtuka bounded by the cocharacter
$\mu_{g}=(1^{(g)},0^{(g)})$. Since ${\operatorname{GL}(\Lambda)}$ is a
reductive group, we may identify the local model
$\mathbb{M}_{{\operatorname{GL}(\Lambda)},\mu_{g}}$ with the flag variety
$\operatorname{Gr}_{g,\Lambda}$ of $g$-dimensional quotients of $\Lambda$ as
$\mathcal{O}_{E}$-schemes (see [AGLR22, Example 4.12] and [SW20, Proposition
19.4.2]). We consider the diamond associated to the local model
$\mathbb{M}_{{\operatorname{GL}(\Lambda)},\mu_{g}}^{\lozenge}$ as a closed
subfunctor of the Beilinson–Drinfeld affine Grassmannian
$\operatorname{Gr}_{{\operatorname{GL}(\Lambda)}}$ for
${\operatorname{GL}(\Lambda)}$.

On the generic fiber (base changed to $E$), the isomorphism
$\mathbb{M}_{{\operatorname{GL}(\Lambda)},\mu_{g},E}^{\lozenge}\xrightarrow{\sim}\operatorname{Gr}_{g,\Lambda,E}^{\lozenge}$
is induced by the Białynicki-Birula map, see [SW20, Proposition 19.4.2].

#### A.1.5.

Let the notation be as in Section A.1.2. We can define a
$\mathrm{GL}(\Lambda)^{\lozenge}$-torsor of trivializations
$\mathcal{P}_{\Lambda}$ over $X^{\lozenge}$ via

$(S\to
X^{\lozenge})\mapsto\mathrm{Isom}_{\mathcal{O}_{S^{\sharp}}}(\Lambda\otimes_{{\mathbb{Z}_{p}}}\mathcal{O}_{S^{\sharp}},{\left.\kern-1.2pt\phi^{\ast}\mathscr{V}\vphantom{\big{|}}\right\rvert_{S^{\sharp}}}).$

Lemma A.1.1 implies that there is a canonical isomorphism of
$\mathrm{GL}(\Lambda)^{\lozenge}$-torsors over $X^{\lozenge}$

(A.1.5) $\displaystyle
P^{\lozenge}_{\Lambda}\xrightarrow{\sim}\mathcal{P}_{\Lambda}.$

Apply the construction of [PR24, Section 4.9.1], we get a diagram

$X^{\lozenge}\leftarrow\mathcal{P}_{\Lambda}\rightarrow\mathbb{M}_{{\operatorname{GL}(\Lambda)},\mu_{g},E}^{\lozenge}.$

The right arrow is $\mathrm{GL}(\Lambda)^{\lozenge}$-equivariant, and
following the notation in loc. cit., it is constructed by (locally on $S$)
lifting a section of $\mathcal{P}_{\Lambda}$ to an isomorphism over
$\widehat{S^{\sharp}}\coloneqq\operatorname{Spec}(\hat{\mathcal{O}}_{S\dot{\times}\operatorname{Spa}{\mathbb{Z}_{p}},S^{\sharp}})$,
and then send it to the triple

$(S^{\sharp},\mathscr{V},\alpha:\mathscr{V}|_{\widehat{S^{\sharp}}\backslash
S^{\sharp}}\xrightarrow[\sim]{\phi_{\mathscr{V}}^{-1}}\mathrm{Frob}^{\ast}\mathscr{V}|_{\widehat{S^{\sharp}}\backslash
S^{\sharp}}\simeq\Lambda\otimes_{{\mathbb{Z}_{p}}}\mathcal{O}_{S^{\sharp}})\in\operatorname{Gr}_{\mathrm{GL}(\Lambda),E}(S).$

Its image lies in the minuscule Schubert cell
$\mathbb{M}_{{\operatorname{GL}(\Lambda)},\mu_{g},E}^{\lozenge}=\operatorname{Gr}_{\mathrm{GL}(\Lambda),\mu_{g},E}$.
By Lemma A.1.1, we have the following compatibility.

######  Proposition A.1.6.

The diagram below commutes, where the vertical isomorphisms are the ones from
Sections A.1.5 and A.1.4.

${X^{\lozenge}}$${\mathcal{P}_{\Lambda}}$${\mathbb{M}_{{\operatorname{GL}(\Lambda)},\mu_{g},E}^{\lozenge}}$${X^{\lozenge}}$${P_{\Lambda}^{\lozenge}}$${\operatorname{Gr}_{g,\Lambda,E}^{\lozenge}}$$\scriptstyle{\sim}$$\scriptstyle{\sim}$

#### A.1.7.

Now let ${(\mathsf{G},\mathsf{X})}$ be a Shimura datum with Hodge cocharacter
$\mu$ and reflex field $\mathsf{E}$ satisfying (4.0.3). Let $v|p$ be a place
of $\mathsf{E}$, $E\coloneqq\mathsf{E}_{v}$, and $\mathcal{G}$ be a parahoric
model of $G$ over ${\mathbb{Q}_{p}}$. Let
$K_{p}=\mathcal{G}({\mathbb{Z}_{p}})$ and $K=K^{p}K_{p}$ for
$K^{p}\subset\mathsf{G}({\mathbb{A}_{f}^{p}})$ a neat compact open subgroup.
We now specialize the previous section to the situation that
$X=\mathbf{Sh}_{K}{(\mathsf{G},\mathsf{X})}^{\mathrm{an}}$. Then there is a
pro-étale $\mathcal{G}({\mathbb{Z}_{p}})$-torsor $\mathbb{P}\to X$ which is de
Rham in the sense of [PR24, Definition 2.6.5], see [PR24, Section 4.1]. This
gives us an exact tensor functor

(A.1.6)
$\displaystyle\mathbb{L}_{p}:\operatorname{Rep}_{{\mathbb{Z}_{p}}}\mathcal{G}$
$\displaystyle\to\\{\text{de Rham ${\mathbb{Z}_{p}}$-local systems on
}X_{\text{pro\'{e}t}}\\}$ (A.1.7) $\displaystyle W$
$\displaystyle\mapsto\mathbb{P}\times^{\underline{\mathcal{G}({\mathbb{Z}_{p}})}}\underline{W}.$

The composition $D_{\mathrm{dR}}\circ\mathbb{L}_{p}$ defines a
$G^{\text{an}}$-torsor $P$ on $X_{\text{\'{e}t}}$ via the Tannakian formalism.
It thus follows from Lemma A.1.1 that the $\mathcal{G}^{\lozenge}$-torsor
$P^{\lozenge}$ on $X^{\lozenge}$ is naturally isomorphic to the
$\mathcal{G}^{\lozenge}$-torsor $\mathcal{P}_{\mathrm{PR}}$ induced by the
$\mathcal{G}$-shtuka over $X^{\lozenge}$ coming from $\mathbb{L}_{p}$. We note
that since the filtered vector bundles in the essential image of
$D_{\mathrm{dR}}\circ\mathbb{L}_{p}$ are equipped with a decreasing filtration
of type $\mu$, $P$ has a canonical reduction of structure group to the
standard parabolic attached to $\mu$181818We follow the convention in [CS17,
2.1] for parabolics attached to cocharacters., see [LZ17, Remark 4.1(i)].
Therefore it admits a map
$P\rightarrow\mathcal{F}\ell_{G,\mu}\coloneqq(G/P^{\mathrm{std}}_{\mu})_{E}^{\mathrm{an}}$.
On the other hand, similar to what we have explained in Section A.1.5, the
construction in [PR24, Section 4.9.1] gives a map
$\mathcal{P}_{\mathrm{PR}}\to\mathrm{Gr}_{G,E}$ with image in the Schubert
cell $\mathrm{Gr}_{G,\mu,E}=\mathbb{M}_{\mathcal{G},\mu,E}^{\lozenge}$.
Moreover, the following diagram is commutative,

${X^{\lozenge}}$${\mathcal{P}_{\mathrm{PR}}}$${\mathbb{M}_{\mathcal{G},\mu,E}^{\lozenge}}$${X^{\lozenge}}$${P^{\lozenge}}$${\mathcal{F}\ell_{G,\mu},}$$\scriptstyle{\sim}$$\scriptstyle{\sim}$

where the rightmost vertical arrow is induced by the Białynicki-Birula map.
Note that there is also an exact tensor functor (the canonical construction)

(A.1.8)
$\displaystyle\mathcal{L}:\operatorname{Rep}_{{\mathbb{Z}_{p}}}\mathcal{G}\to\\{\text{Filtered
vector bundles on }X\\},$

see [DLLZ23, Proposition 5.2.10]. It follows from [DLLZ23, Theorem 5.3.1] that
there is a natural isomorphism of tensor functors

(A.1.9)
$\displaystyle\mathcal{L}\xrightarrow{\sim}D_{\mathrm{dR}}\circ\mathbb{L}_{p}.$

Thus the $\mathcal{G}^{\lozenge}$-torsor $\mathcal{P}_{\operatorname{dR}}$ on
$X^{\lozenge}$ corresponding to $\mathcal{L}$ via the Tannakian formalism, is
naturally isomorphic to the $\mathcal{G}^{\lozenge}$ torsor $P_{\mathrm{PR}}$
induced by the $\mathcal{G}$-shtuka over $X^{\lozenge}$ coming from
$\mathbb{L}_{p}$. This isomorphism is moreover compatible with filtrations and
thus with the map to $\mathbb{M}_{\mathcal{G},\mu,E}^{\lozenge}$.

###  A.2. Some integral $p$-adic Hodge theory

Let $S^{\sharp}=\operatorname{Spa}(R^{\sharp},R^{\sharp+})$ be an untilt of an
affinoid perfectoid space $S=\operatorname{Spa}(R,R^{+})$ in characteristic
$p$ and let $\mathfrak{A}$ be the completion of an abelian scheme over
$R^{\sharp+}$ with associated $p$-divisible group
$Y=\mathfrak{A}[p^{\infty}]$. By [SW20, Theorem 17.5.2], we can associate to
$Y$ a finite free $W(R^{+})$-module $M(Y)$ equipped with an isomorphism

(A.2.1)
$\displaystyle\phi_{M}:\phi^{\ast}M(Y)[1/\phi(\xi)]\xrightarrow{\sim}M(Y)[1/\phi(\xi)]$

such that

(A.2.2) $\displaystyle
M(Y)\subset\phi_{M}\left(\phi^{\ast}M(Y)\right)\subset\tfrac{1}{\phi(\xi)}M(Y).$

Here $\xi$ is a generator of the kernel of $W(R^{+})\to R^{\sharp+}$. Let
$M(Y)^{\ast}$ denote the $W(R^{+})$-linear dual of $M(Y)$, which we will equip
with the isomorphism $\phi_{M^{\ast}}$ given by the inverse of the
$W(R^{+})$-linear dual of $\phi_{M}$. This is the (contravariant) prismatic
Dieudonné module of $Y$ and it satisfies

(A.2.3) $\displaystyle\xi
M(Y)^{\ast}\subset\phi_{M^{\ast}}(\phi^{\ast}M(Y)^{\ast})\subset M(Y)^{\ast}.$

By restriction along
$S\dot{\times}\operatorname{Spa}{\mathbb{Z}_{p}}\to\operatorname{Spec}W(R^{+})$,
it gives rise to a minuscule vector bundle shtuka with one leg at
$S^{\sharp}$.

######  Lemma A.2.1.

There is a canonical isomorphism

(A.2.4) $\displaystyle
M(Y)^{\ast}\otimes_{W(R^{+})}R^{\sharp+}\xrightarrow{\sim}H^{1}_{\mathrm{dR}}(A/R^{\sharp+})$

compatible with base change.

###### Proof.

We may identify $M(Y)^{\ast}$ with the $\phi$-pullback of the relative
prismatic cohomology of $\mathfrak{A}$ using [ALB23, Corollary 4.63,
Proposition 4.49].191919In [ALB23, Proposition 4.49], the Frobenius twist is
hidden in the notation $\tilde{\xi}=\phi(\xi)$. The comparison isomorphism now
follows from [BS22, Theorem 1.8.(3)]. ∎

#### A.2.2.

For a characteristic zero untilt $R^{\sharp}$, we want to compare the
isomorphism of Lemma A.2.1 with the isomorphism of Lemma A.1.1. We will do
this under the assumption that $\mathfrak{A}$ is the pullback of a formal
abelian scheme $f:\mathfrak{B}\to\mathfrak{X}$ over a smooth formal scheme
$\mathfrak{X}/\mathcal{O}_{K}$ for some discrete valued field
$K/{\mathbb{Q}_{p}}$ (which will be the case our situation since the Siegel
modular variety is smooth). Denote the special fiber of $\mathfrak{X}$ by
$\mathfrak{X}_{s}$ and the rigid generic fiber of $\mathfrak{X}$ by $X$, and
similarly for $\mathfrak{B}$.

Note that the $F$-isocrystal $\mathcal{E}$ on $\mathfrak{X}_{s}$ obtained by
the contravariant Dieudonné crystal of $\mathfrak{B}_{s}$ is associated to the
vector bundle with flat connection
$(E,\nabla):=(R^{1}f_{\mathrm{dR},\ast}\mathcal{O}_{B},\nabla_{\mathrm{GM}})$
($\nabla_{\mathrm{GM}}$ denotes the Gauss-Manin connection) on $X$, in the
sense of [GR24, Proposition 2.17]. Then the proof of Proposition 2.36.(i) in
loc. cit. shows that
$E\otimes_{\mathcal{O}_{X}}\mathcal{O}\mathbb{B}^{+}_{\mathrm{dR}}$ equipped
with the product connection is isomorphic to
$(\mathbb{B}^{+}_{\mathrm{dR}}(\mathcal{E})\otimes_{\mathbb{B}^{+}_{\mathrm{dR}}}\mathcal{O}\mathbb{B}^{+}_{\mathrm{dR}},\mathrm{id}\otimes\nabla_{\mathcal{O}\mathbb{B}^{+}_{\mathrm{dR}}})$.
In particular, we have a natural identification of the horizonal sections

$\mathbb{B}^{+}_{\mathrm{dR}}(\mathcal{E})=(\mathbb{B}^{+}_{\mathrm{dR}}(\mathcal{E})\otimes_{\mathbb{B}^{+}_{\mathrm{dR}}}\mathcal{O}\mathbb{B}^{+}_{\mathrm{dR}})^{\nabla=0}\cong(E\otimes_{\mathcal{O}_{X}}\mathcal{O}\mathbb{B}^{+}_{\mathrm{dR}})^{\nabla=0}=:\mathbb{M}_{0}.$

On the other hand, under the prismatic–crystalline comparison [BS22, Theorem
1.8(1)], we have that

(A.2.5) $\displaystyle\mathbb{B}^{+}_{\mathrm{dR}}(\mathcal{E})(S^{\sharp})$
$\displaystyle=\mathbb{A}_{\mathrm{crys}}(\mathcal{E})\otimes_{\mathbb{A}_{\mathrm{crys}}}\mathbb{B}^{+}_{\mathrm{dR}}(S^{\sharp})$
(A.2.6)
$\displaystyle=R^{1}f_{\mathrm{crys,\ast}}\mathcal{O}\otimes_{\mathbb{A}_{\mathrm{crys}}}\mathbb{B}^{+}_{\mathrm{dR}}(S^{\sharp})$
(A.2.7)
$\displaystyle\xrightarrow{\sim}\phi^{\ast}H^{1}_{{{\mathbbl{\Delta}}}}(\mathfrak{A}/{{\mathbbl{\Delta}}}_{R^{\sharp+}})\otimes_{W(R^{+})}\mathbb{B}^{+}_{\mathrm{dR}}(S^{\sharp})$
(A.2.8)
$\displaystyle=M(Y)^{\ast}\otimes_{W(R^{+})}B^{+}_{\mathrm{dR}}(R^{\sharp}).$

Here ${{\mathbbl{\Delta}}}_{R^{\sharp+}}$ denotes the perfect prism
$(W(R^{+}),\mathrm{ker}\theta=(\xi))$. Note that the definition of the de Rham
period sheaves in [GR24] differs from ours by a Frobenius twist, see
Definition 2.3, Warning 2.4 in loc. cit., but their arguments work verbatim.
We conclude that the following diagram commutes (cf. [IKY23, Lemma 2.18])

(A.2.9)
${M(Y)^{\ast}}$${M(Y)^{\ast}\otimes_{W(R^{+})}B_{\mathrm{dR}}^{+}(R^{\sharp})}$${H^{1}_{\mathrm{dR}}(A/R^{\sharp})}$${\mathbb{M}_{0}(S^{\sharp}).}$$\scriptstyle{\sim}$

Here the left vertical map is the map from Lemma A.2.1 composed with inverting
$p$ and the bottom horizontal map is the map from Lemma A.1.3. The right
vertical map is the comparison isomorphism from equation (A.2.5).

#### A.2.3.

The discussion in Section A.2.2 above implies an integral version of the
result in Section A.1.5: Following the notation in Section A.1.2. Suppose
$A\to X$ is the rigid generic fiber of a family of formal abelian schemes
$\mathfrak{A}\to\mathfrak{X}$, for some smooth formal scheme $\mathfrak{X}$
over $\mathrm{Spf}\mathcal{O}_{E}$. By descending the relative (contravariant)
prismatic Dieudonné crystals of the pullback of $\mathfrak{A}$ to integral
perfectoids over $\mathfrak{X}$, one has a vector bundle shtuka $\mathscr{V}$
over $\mathfrak{X}^{\lozenge}$. We can define a
$\mathrm{GL}(\Lambda)^{\lozenge}$-torsor of trivializations
$\mathcal{P}_{\Lambda}$ over $\mathfrak{X}^{\lozenge}$ via

$(S\to\mathfrak{X}^{\lozenge})\mapsto\mathrm{Isom}_{\mathcal{O}_{S^{\sharp}}}(\Lambda\otimes_{{\mathbb{Z}_{p}}}\mathcal{O}_{S^{\sharp}},{\left.\kern-1.2pt\phi^{\ast}\mathscr{V}\vphantom{\big{|}}\right\rvert_{S^{\sharp}}}).$

Similarly, one can consider the frame bundle $P_{\Lambda}$ of
$\mathcal{H}^{1}_{\mathrm{dR}}(\mathfrak{A}/\mathfrak{X})$. Lemma A.2.1
implies that there is a canonical isomorphism
$P^{\diamond}_{\Lambda}\times^{\mathrm{GL}_{\Lambda}^{\diamond}}\mathrm{GL}_{\Lambda}^{\lozenge}\xrightarrow{\sim}\mathcal{P}_{\Lambda}$
of $\mathrm{GL}(\Lambda)^{\lozenge}$-torsor over $\mathfrak{X}^{\lozenge}$.

Repeating the procedure in Section A.1.5 we get a commutative diagram (note
that we get a $\mathrm{GL}_{\Lambda}^{\diamond}$-torsor over
$\mathfrak{X}^{\lozenge}$ because the frame bundle is a torsor for the
$p$-adic completion of $\mathrm{GL}_{\Lambda}$ whose associated big diamond
($\lozenge$) gives $\mathrm{GL}_{\Lambda}^{\diamond}$)

${\mathfrak{X}^{\lozenge}}$${\mathcal{P}_{\Lambda}}$${\mathbb{M}_{{\operatorname{GL}(\Lambda)},\mu_{g}}^{\lozenge}}$${\mathfrak{X}^{\lozenge}}$${P^{\diamond}_{\Lambda}\times^{\mathrm{GL}_{\Lambda}^{\diamond}}\mathrm{GL}_{\Lambda}^{\lozenge}}$${\operatorname{Gr}_{g,\Lambda}^{\lozenge}.}$$\scriptstyle{\sim}$$\scriptstyle{\sim}$

By the discussion in Section A.2.2, it is compatible with the one in
Proposition A.1.6 when passing to the generic fiber.

###  A.3. Shimura varieties of Hodge type

We follow the notation in the proof of Theorem 4.2.3. In particular, we have
${(\mathsf{G},\mathsf{X},\mathcal{G})}$ with $\mathcal{G}$ a stabilizer quasi-
parahoric, a Hodge embedding
$\iota:{(\mathsf{G},\mathsf{X})}\to(\mathsf{G}_{V},\mathsf{H}_{V})$. We may
moreover take ${\mathbb{Z}_{p}}$-lattice $\Lambda\subset V_{{\mathbb{Q}_{p}}}$
on which $\psi$ is ${\mathbb{Z}_{p}}$-valued, such that
$\mathcal{G}(\breve{\mathbb{Z}}_{p})$ is the stabilizer in
$G(\breve{\mathbb{Q}}_{p})$ of
$V_{p}\otimes_{{\mathbb{Z}_{p}}}\breve{\mathbb{Z}}_{p}$. Assume moreover that
the Hodge embedding $\iota$ is good for
${(\mathsf{G},\mathsf{X},\mathcal{G})}$ with respect to $\Lambda$ in the sense
of [KZ21, Definition 3.3.15], which means that the natural maps

(A.3.1) $\displaystyle\mathcal{G}\to{\operatorname{GL}(\Lambda)}\ \text{ and
}\ \mathbb{M}_{\mathcal{G},\mu}\to\operatorname{Gr}_{g,\Lambda}$

are closed immersions.202020The local models used by [KZ21] agree with ours
because theirs also satisfy the Scholze–Weinstein conjecture (which means that
they have the correct associated $v$-sheaf), see [KZ21, Proposition 3.3.10].

######  Remark A.3.1.

If $p$ is coprime to $2\cdot\pi_{1}(G^{\mathrm{der}})$ and $G$ is acceptable
([KZ21, Definition 3.3.2]) and $R$-smooth ([DY24, Definition 2.10]), then a
good Hodge embedding always exists, see [KZ21, Lemma 5.1.3]. By Zarhin’s
trick, see [Kis17, Section 1.3.3], we may moreover replace $\Lambda$ by
$\Lambda^{\oplus 4}\oplus\Lambda^{\vee,\oplus 4}$ to assume that $\Lambda$ is
self dual. The resulting Hodge embedding is then still good with respect to
$\Lambda$.

#### A.3.2.

Let $P_{\Lambda^{\vee}}\to\mathscr{S}_{K}{(\mathsf{G},\mathsf{X})}$ be the
${\operatorname{GL}(\Lambda^{\vee})}$-torsor parametrizing trivialisations of
the de Rham cohomology of the universal abelian variety (up to prime-to-$p$
isogeny) coming from $\iota$. Then there is a morphism
$P_{\Lambda^{\vee}}\to\operatorname{Gr}_{g,\Lambda^{\vee},\mathcal{O}_{E}}$ as
in Section A.1.2. By the proof of [KZ21, Theorem 5.1.10], there is a
$\mathcal{G}$-torsor $P\to\mathscr{S}_{K}{(\mathsf{G},\mathsf{X})}$ together
with a $\mathcal{G}$-equivariant map $P\to P_{\Lambda^{\vee}}$ such that the
composition

(A.3.2) $\displaystyle P\to
P_{\Lambda^{\vee}}\to\operatorname{Gr}_{g,\Lambda^{\vee},\mathcal{O}_{E}}$

factors through $\mathbb{M}_{\mathcal{G},\mu}$ via a smooth map. To compare
with the constructions in [PR24], which considers the
$\mathrm{GL}(\Lambda)$-torsor of isomorphisms from the de-Rham homology to
$\Lambda$ (rather than cohomology to $\Lambda^{\vee}$), we push out along the
natural isomorphism
${\operatorname{GL}(\Lambda^{\vee})}\to\operatorname{GL}(\Lambda)$ and obtain
a $\mathrm{GL}(\Lambda)$-torsor $P_{\Lambda}$ with a diagram

$\mathscr{S}_{K}{(\mathsf{G},\mathsf{X})}\leftarrow
P_{\Lambda}\rightarrow\mathrm{Gr}_{g,\Lambda,\mathcal{O}_{E}}.$

We denote the image of $P$ still by $P$ and the above diagram restricts to

$\mathscr{S}_{K}{(\mathsf{G},\mathsf{X})}\leftarrow
P\rightarrow\mathbb{M}_{\mathcal{G},\mu},$

where the left arrow is a $\mathcal{G}$-torsor and the right arrow is smooth
and $\mathcal{G}$-equivariant. It moreover follows from the construction that
its generic fiber comes from the canonical model of the standard principal
bundle, see the discussion in the proof of [CS17, Lemma 2.3.5]. Let us write

(A.3.3)
$\displaystyle\pi_{\mathrm{dR},\mathcal{G}}\colon\mathscr{S}_{K}{(\mathsf{G},\mathsf{X})}\to\left[\mathbb{M}_{\mathcal{G},\mu}/\mathcal{G}\right]$

for the induced smooth morphism of algebraic stacks.

######  Theorem A.3.3.

If the Hodge embedding $\iota$ is good for
${(\mathsf{G},\mathsf{X},\mathcal{G})}$ with respect to a self-dual lattice
$\Lambda$, then the morphism $\pi_{\mathrm{dR},\mathcal{G}}$ is a scheme-
theoretic local model diagram.

For the proof of Theorem A.3.3, we will need the following two lemmas.

######  Lemma A.3.4.

Let $\mathscr{Y}$ be a v-sheaf which is separated over
$\operatorname{Spd}({\mathbb{Z}_{p}})$. For any scheme flat normal scheme $X$
which is separated and of finite-type over ${\mathbb{Z}_{p}}$, the natural
restriction map

(A.3.4)
$\displaystyle\operatorname{Hom}_{\operatorname{Spd}({\mathbb{Z}_{p}})}(X^{\lozenge/},\mathscr{Y})\to\operatorname{Hom}_{\operatorname{Spd}({\mathbb{Q}_{p}})}(X_{{\mathbb{Q}_{p}}}^{\lozenge},\mathscr{Y}_{\mathbb{Q}_{p}})$

is injective.

###### Proof.

This follows from the density of
$|X_{{\mathbb{Q}_{p}}}^{\lozenge}|\subset|X^{\lozenge/}|$, which in turn
follows from the density of
$|(X^{\diamond})_{{\mathbb{Q}_{p}}}|\subset|X^{\diamond}|$ (see [AGLR22, Lemma
2.17]). ∎

######  Lemma A.3.5.

The quotient v-sheaf $\mathrm{GL}(\Lambda)^{\lozenge}/\mathcal{G}^{\lozenge}$
is separated over $\operatorname{Spd}({\mathbb{Z}_{p}})$.

###### Proof.

By [HdS21, Lemma 6.17], there is a finite free representation $W$ of
$\operatorname{GL}(\Lambda)$ and a free rank-one saturated
${\mathbb{Z}_{p}}$-submodule $L\subset W$ for which $\mathcal{G}$ is the
scheme-theoretic stabilizer of $L$ inside of $\mathrm{GL}(\Lambda)$. It
follows that there is a morphism of v-sheaves

(A.3.5)
$\mathrm{GL}(\Lambda)^{\lozenge}/\mathcal{G}^{\lozenge}\to\mathbb{P}(W)^{\lozenge},$

defined at the level of presheaves by $[g]\mapsto g\cdot L$. We claim this is
a monomorphism. Indeed, suppose $S$ is a perfectoid space in characteristic
$p$, and that $a,b\colon
S\to\mathrm{GL}(\Lambda)^{\lozenge}/\mathcal{G}^{\lozenge}$ are two morphisms
which agree after the composition to $\mathbb{P}(W)^{\lozenge}$. After
replacing $S$ by a v-cover, we may assume $a$ and $b$ factor through morphisms
$\tilde{a}$, $\tilde{b}\colon S\to\mathrm{GL}(\Lambda)^{\lozenge}$. Since $a$
and $b$ agree after the composition to $\mathbb{P}(W)^{\lozenge}$, and
$\mathcal{G}$ is the stabilizer of $L$, it follows that
$\tilde{a}\cdot\tilde{b}^{-1}$ factor through $\mathcal{G}^{\lozenge}$; thus
$a=b$.

Since (A.3.5) is a monomorphism, its diagonal is an isomorphism, and therefore
(A.3.5) is separated. Now $\mathbb{P}(W)$ is proper over
$\operatorname{Spec}({\mathbb{Z}_{p}})$, so
$\mathbb{P}(W)^{\lozenge}=\mathbb{P}(W)^{\diamond}$, and hence
$\mathbb{P}(W)^{\lozenge}\to\operatorname{Spd}({\mathbb{Z}_{p}})$ is separated
by [Gle20, Proposition 4.17]. The result follows. ∎

###### Proof of Theorem A.3.3.

We start by identifying

(A.3.6) $\displaystyle
P_{\mathcal{O}_{E}}^{\lozenge/}\times^{\mathcal{G}^{\lozenge/}}\mathcal{G}^{\lozenge}$

with the $\mathcal{G}^{\lozenge}$-torsor coming from the map
$\mathscr{S}_{K}{(\mathsf{G},\mathsf{X})}^{\lozenge/}\to\mathrm{Sht}_{\mathcal{G},\mu}$.
We first check that this holds after composing with
$\mathrm{Sht}_{\mathcal{G},\mu}\to\operatorname{Sht}_{{\operatorname{GL}(\Lambda)},\mu_{g}}$
and pushing out via

(A.3.7)
$\displaystyle\mathcal{G}^{\lozenge}\to{\operatorname{GL}(\Lambda)}^{\lozenge}.$

The latter result is true over
$\mathbf{Sh}_{K}{(\mathsf{G},\mathsf{X})}^{\lozenge}$ by Proposition A.1.6.
Moreover, by Sections A.2.2 and A.2.3, it is true over
$\mathscr{S}_{K}{(\mathsf{G},\mathsf{X})}^{\diamond}$212121The discussion
there assumes that our (formal) abelian scheme comes via pullback from a
(formal) abelian scheme over a smooth (formal scheme). This assumptions holds
here since $\Lambda$ is self dual and thus the integral model of the Shimura
variety for $(\mathsf{G}_{V},\mathsf{H}_{V})$ is smooth., such that induced
isomorphisms agree on
$\left(\mathscr{S}_{K}{(\mathsf{G},\mathsf{X})}^{\diamond}\right)_{E}$, so
they glue to an isomorphism over
$\mathscr{S}_{K}{(\mathsf{G},\mathsf{X})}^{\lozenge/}$.

Next, we check that the induced $\mathcal{G}^{\lozenge}$-torsors agree: after
trivializing the induced ${\operatorname{GL}(\Lambda)}$-torsor, which we may
do Zariski locally on $\mathscr{S}_{K}{(\mathsf{G},\mathsf{X})}$, we are
trying to show the equality of two morphisms
$\mathscr{S}_{K}{(\mathsf{G},\mathsf{X})}^{\lozenge/}\to{\operatorname{GL}(\Lambda)}^{\lozenge}/\mathcal{G}^{\lozenge}$.
By Lemma A.3.5 and Lemma A.3.4, it suffices to check this after base change to
the generic fiber, where the result follows from the discussion in Section
A.1.7.

Finally, we check that, under this identification, the maps to the local model
agree. But $\mathbb{M}_{\mathcal{G},\mu}$ is projective and hence proper over
$\operatorname{Spec}(\mathcal{O}_{E})$, so
$\mathbb{M}_{\mathcal{G},\mu}^{\lozenge}\to\operatorname{Spd}(\mathcal{O}_{E})$
is separated by [Gle20, Proposition 4.17]. By another application of Lemma
A.3.4, it suffices to check the morphisms agree after base change to the
generic fiber, where the result follows from the discussion in Section A.1.7.
Together with the results implied by [KZ21, Theorem 5.1.10] discussed above,
this concludes the proof that $\pi_{\mathrm{dR},\mathcal{G}}$ is a scheme-
theoretic local model diagram. ∎

## References

  * [AGLR22] Johannes Anschütz, Ian Gleason, João Lourenço, and Timo Richarz, _On the $p$-adic theory of local models_, arXiv 2201.01234, January 2022.
  * [ALB23] Johannes Anschütz and Arthur-César Le Bras, _Prismatic Dieudonné theory_ , Forum Math. Pi 11 (2023), Paper No. e2, 92. MR 4530092
  * [ALY22] Piotr Achinger, Marcin Lara, and Alex Youcis, _Specialization for the pro-étale fundamental group_ , Compos. Math. 158 (2022), no. 8, 1713–1745. MR 4490930
  * [Ans22] Johannes Anschütz, _Extending torsors on the punctured ${\rm Spec}(A_{\inf})$_, J. Reine Angew. Math. 783 (2022), 227–268. MR 4373246
  * [Ans23] by same author, _$G$ -bundles on the absolute Fargues-Fontaine curve_, Acta Arith. 207 (2023), no. 4, 351–363. MR 4591255
  * [Bor98] Mikhail Borovoi, _Abelian Galois cohomology of reductive groups_ , Mem. Amer. Math. Soc. 132 (1998), no. 626, viii+50. MR 1401491
  * [BS22] B. Bhatt and P. Scholze, _Prisms and prismatic cohomology_ , Ann. of Math. (2) 196 (2022), no. 3, 1135 – 1275.
  * [BT84] F. Bruhat and J. Tits, _Groupes réductifs sur un corps local. II. Schémas en groupes. Existence d’une donnée radicielle valuée_ , Inst. Hautes Études Sci. Publ. Math. (1984), no. 60, 197–376. MR 756316
  * [CS17] Ana Caraiani and Peter Scholze, _On the generic part of the cohomology of compact unitary Shimura varieties_ , Ann. of Math. (2) 186 (2017), no. 3, 649–766. MR 3702677
  * [Dan22] Patrick Daniels, _Canonical integral models for Shimura varieties defined by tori_ , arXiv 2207.09513, July 2022.
  * [DvanHKZ24] Patrick Daniels, Pol van Hoften, Dongryul Kim, and Mingjia Zhang, _Igusa stacks for Shimura varieties of Hodge type_ , 2024, In preparation.
  * [DLLZ23] Hansheng Diao, Kai-Wen Lan, Ruochuan Liu, and Xinwen Zhu, _Logarithmic Riemann-Hilbert correspondences for rigid varieties_ , J. Amer. Math. Soc. 36 (2023), no. 2, 483–562. MR 4536903
  * [DY24] Patrick Daniels and Alex Youcis, _Canonical Integral Models of Shimura Varieties of Abelian Type_ , arXiv 2402.05727, February 2024.
  * [FS21] L. Fargues and P. Scholze, _Geometrization of the local Langlands correspondence_ , arXiv 2102.13459, 2021.
  * [GI23] Ian Gleason and Alexander B. Ivanov, _Meromorphic vector bundles on the Fargues–Fontaine curve_ , arXiv 2307.00887, July 2023.
  * [Gle20] Ian Gleason, _Specialization maps for Scholze’s category of diamonds_ , arXiv 2012.05483, 2020.
  * [Gle21] Ian Gleason, _On the geometric connected components of moduli spaces of p-adic shtukas and local Shimura varieties_ , arXiv 2107.03579, July 2021, p. arXiv:2107.03579.
  * [GLX23] Ian Gleason, Dong Gyu Lim, and Yujie Xu, _The connected components of affine Deligne-Lusztig varieties_ , https://ianandreigf.github.io/Website/QsplitADLV.pdf, January 2023.
  * [GR24] Haoyang Guo and Emanuel Reinecke, _A prismatic approach to crystalline local systems_ , Inventiones Mathematicae 236 (2024), 1–164.
  * [HdS21] Phùng Hô Hai and João Pedro dos Santos, _On the structure of affine flat group schemes over discrete valuation rings, II_ , Int. Math. Res. Not. IMRN (2021), no. 12, 9375–9424. MR 4276322
  * [He16] Xuhua He, _Kottwitz-Rapoport conjecture on unions of affine Deligne-Lusztig varieties_ , Ann. Sci. Éc. Norm. Supér. (4) 49 (2016), no. 5, 1125–1141.
  * [HK19] Paul Hamacher and Wansu Kim, _$l$ -adic étale cohomology of Shimura varieties of Hodge type with non-trivial coefficients_, Math. Ann. 375 (2019), no. 3-4, 973–1044. MR 4023369
  * [vanH20] Pol van Hoften, _Mod $p$ points on Shimura varieties of parahoric level (with an appendix by Rong Zhou)_, arXiv 2010.10496, October 2020.
  * [HR08] T. Haines and M. Rapoport, _On parahoric subgroups (Appendix to “Twisted loop groups and their affine flag varieties” by Pappas–Rapoport)_ , 2008.
  * [Hub94] R. Huber, _A generalization of formal schemes and rigid analytic varieties_ , Math. Z. 217 (1994), no. 4, 513 – 551.
  * [Hub96] Roland Huber, _Étale cohomology of rigid analytic varieties and adic spaces_ , Aspects of Mathematics, vol. 30, Springer Fachmedien Wiesbaden, 1996.
  * [IKY23] Naoki Imai, Hiroki Kato, and Alex Youcis, _The prismatic realization functor for Shimura varieties of abelian type_ , arXiv 2310.08472, October 2023.
  * [Ked20] Kiran S. Kedlaya, _Some ring-theoretic properties of ${\rm A_{\rm inf}}$_, $p$-adic hodge theory, Simons Symp., Springer, Cham, [2020] ©2020, pp. 129–141. MR 4359204
  * [Kim24] Dongryul Kim, _Descending finite projective modules from a Novikov ring_ , arXiv 2402.17852, February 2024.
  * [Kis10] Mark Kisin, _Integral models for Shimura varieties of abelian type_ , J. Amer. Math. Soc. 23 (2010), no. 4, 967–1012. MR 2669706
  * [Kis17] by same author, _${\rm mod}\,p$ points on Shimura varieties of abelian type_, J. Amer. Math. Soc. 30 (2017), no. 3, 819–914. MR 3630089
  * [KL15] Kiran S. Kedlaya and Ruochuan Liu, _Relative $p$-adic Hodge theory: foundations_, Astérisque (2015), no. 371, 239. MR 3379653
  * [KMPS22] Mark Kisin, Keerthi Madapusi Pera, and Sug Woo Shin, _Honda–Tate theory for Shimura varieties_ , Duke Math. J. 171 (2022), no. 7, 1559–1614. MR 4484214
  * [Kot97] Robert E. Kottwitz, _Isocrystals with additional structure. II_ , Compositio Math. 109 (1997), no. 3, 255–339. MR 1485921
  * [KP18] M. Kisin and G. Pappas, _Integral models of Shimura varieties with parahoric level structure_ , Publ. Math. Inst. Hautes Études Sci. 128 (2018), 121–218. MR 3905466
  * [KP23] Tasho Kaletha and Gopal Prasad, _Bruhat-Tits theory—a new approach_ , New Mathematical Monographs, vol. 44, Cambridge University Press, Cambridge, 2023. MR 4520154
  * [KSZ21] Mark Kisin, Sug Woo Shin, and Yihang Zhu, _The stable trace formula for Shimura varieties of abelian type_ , arXiv 2110.05381, October 2021.
  * [KZ21] Mark Kisin and Rong Zhou, _Independence of $\ell$ for Frobenius conjugacy classes attached to abelian varieties_, https://www.dpmms.cam.ac.uk/~rz240/l-indep_v2.pdf, March 2021.
  * [LZ17] Ruochuan Liu and Xinwen Zhu, _Rigidity and a Riemann-Hilbert correspondence for $p$-adic local systems_, Invent. Math. 207 (2017), no. 1, 291–343. MR 3592758
  * [Mil90] J. S. Milne, _Canonical models of (mixed) Shimura varieties and automorphic vector bundles_ , Automorphic forms, Shimura varieties, and $L$-functions, Vol. I (Ann Arbor, MI, 1988), Perspect. Math., vol. 10, Academic Press, Boston, MA, 1990, pp. 283–414. MR 1044823
  * [Mil92] James S. Milne, _The points on a Shimura variety modulo a prime of good reduction_ , The zeta functions of Picard modular surfaces, Univ. Montréal, Montreal, QC, 1992, pp. 151–253. MR 1155229
  * [Mil05] J. S. Milne, _Introduction to Shimura varieties_ , Harmonic analysis, the trace formula, and Shimura varieties, Clay Math. Proc., vol. 4, Amer. Math. Soc., Providence, RI, 2005, pp. 265–378. MR 2192012
  * [Moo98] Ben Moonen, _Models of Shimura varieties in mixed characteristics_ , Galois representations in arithmetic algebraic geometry (Durham, 1996), London Math. Soc. Lecture Note Ser., vol. 254, Cambridge Univ. Press, Cambridge, 1998, pp. 267–350. MR 1696489
  * [Pap23] Georgios Pappas, _On integral models of Shimura varieties_ , Math. Ann. 385 (2023), no. 3-4, 2037–2097. MR 4566689
  * [PR22] Georgios Pappas and Michael Rapoport, _On integral local Shimura varieties_ , arXiv 2204.02829, April 2022.
  * [PR24] Georgios Pappas and Michael Rapoport, _$p$ -adic shtukas and the theory of global and local Shimura varieties_, Camb. J. Math. 12 (2024), no. 1, 1–164. MR 4701491
  * [Rap05] Michael Rapoport, _A guide to the reduction modulo $p$ of Shimura varieties_, Astérisque (2005), no. 298, 271–318, Automorphic forms. I. MR 2141705
  * [Rap18] by same author, _Accessible and weakly accessible period domains_ , Ann. Sci. Éc. Norm. Supér. (4) 51 (2018), no. 4, 811–863, appendix to On the $p$-adic cohomology of the Lubin–Tate tower by Peter Scholze. MR 3861564
  * [Rod22] J. E. Rodríguez Camargo, _Locally analytic completed cohomology_ , arXiv 2209.01057, August 2022, p. arXiv:2209.01057.
  * [RR96] M. Rapoport and M. Richartz, _On the classification and specialization of $F$-isocrystals with additional structure_, Compositio Math. 103 (1996), no. 2, 153–181. MR 1411570
  * [Sch13] Peter Scholze, _$p$ -adic Hodge theory for rigid-analytic varieties_, Forum Math. Pi 1 (2013), e1, 77. MR 3090230
  * [Sch17] Peter Scholze, _Etale cohomology of diamonds_ , arXiv 1709.07343, September 2017.
  * [Sta24] The Stacks project authors, _The stacks project_ , https://stacks.math.columbia.edu, 2024.
  * [SW13] Peter Scholze and Jared Weinstein, _Moduli of $p$-divisible groups_, Camb. J. Math. 1 (2013), no. 2, 145–237. MR 3272049
  * [SW20] by same author, _Berkeley lectures on $p$-adic geometry_, Annals of Mathematics Studies, vol. 207, Princeton University Press, Princeton, NJ, 2020. MR 4446467
  * [Tak24] Yuta Takaya, _Moduli spaces of level structures on mixed characteristic local shtukas_ , arxiv 2402.07135, February 2024.
  * [Vie21] Eva Viehmann, _On Newton strata in the $B_{dR}^{+}$-Grassmannian_, arXiv 2101.07510, January 2021.
  * [Zha23] Mingjia Zhang, _A PEL-type Igusa Stack and the $p$-adic Geometry of Shimura Varieties_, arXiv 2309.05152, September 2023.
  * [Zho20] Rong Zhou, _Mod $p$ isogeny classes on Shimura varieties with parahoric level structure_, Duke Math. J. 169 (2020), no. 15, 2937–3031.