# On the Verge of Solving Rocket League using Deep Reinforcement Learning and
Sim-to-sim Transfer

Marco Pleines111Correspondence to<EMAIL_ADDRESS>Konstantin
Ramthun, Yannik Wegener, Hendrik Meyer, Matthias Pallasch, Sebastian Prior
Jannik Drögemüller, Leon Büttinghaus, Thilo Röthemeyer, Alexander Kaschwig,
Oliver Chmurzynski, Frederik Rohkrähmer, Roman Kalkreuth, Frank Zimmer1, Mike
Preuss2  Department of Computer Science, TU Dortmund University, Dortmund,
Germany
1Department of Communication and Environment, Rhine-Waal University of Applied
Sciences, Kamp-Linfort, Germany
2LIACS Universiteit Leiden, Leiden, Netherlands

###### Abstract

Autonomously trained agents that are supposed to play video games reasonably
well rely either on fast simulation speeds or heavy parallelization across
thousands of machines running concurrently. This work explores a third way
that is established in robotics, namely sim-to-real transfer, or if the game
is considered a simulation itself, sim-to-sim transfer. In the case of Rocket
League, we demonstrate that single behaviors of goalies and strikers can be
successfully learned using Deep Reinforcement Learning in the simulation
environment and transferred back to the original game. Although the
implemented training simulation is to some extent inaccurate, the goalkeeping
agent saves nearly $100\%$ of its faced shots once transferred, while the
striking agent scores in about $75\%$ of cases. Therefore, the trained agent
is robust enough and able to generalize to the target domain of Rocket League.

###### Index Terms:

rocket league, sim-to-sim transfer, deep reinforcement learning, proximal
policy optimization

††publicationid: pubid: Accepted to IEEE CoG 2022

##  I Introduction

The spectacular successes of agents playing considerably difficult games, such
as StarCraft ii@ [1] and DotA 2 [2], have been possible only because the
employed algorithms were able to train on huge numbers of games on the order
of billions or more. Unfortunately, and despite many improvements achieved in
AI in recent years, the utilized Deep Learning methods are still relatively
sample inefficient. To deal with this problem, fast running environments or
high amounts of computing resources are vital. OpenAI Five for DotA 2 [2] is
an example of the utilization of hundreds of thousands of computing cores in
order to achieve high throughput in terms of played games. However, this way
is closed for games that run only on specific platforms and are thus very hard
to parallelize. Moreover, not many research groups have such resources at
their disposal. Video games that suffer from not being able to be sped up
significantly, risk minimal running times and hence repeatability. Therefore
it makes sense to look for alternative ways to tackle difficult problems.

Figure 1: The game of Rocket League (top) and the contributed simulation
(bottom), which notably advances its ancestor project _RoboLeague_ [3].

Sim-to-real transfer offers such an alternative way and is well established in
robotics, and it follows the general idea that robot behavior can be learned
in a very simplified simulation environment and the trained agents can then be
successfully transferred to the original environment. If the target platform
is a game as well, we may speak of sim-to-sim transfer because the original
game is also virtual, just computationally much more costly. This approach is
applicable to current games, even if they are not parallelizable, and makes
them available for modern Deep Reinforcement Learning (DRL) methods. There is
of course a downside of this approach, namely that it may be difficult or even
infeasible to establish a simulation that is similar enough to enable transfer
later on, but still simple enough to speed up learning significantly. A
considerable amount of effort has to be invested in establishing this
simulation environment before we can make any progress on the learning task.

To our knowledge, the sim-to-sim approach has not yet been applied to train
agents for a recent game. Therefore we aim to explore the possibilities of
this direction in order to detect how simple the simulation can be, and how
good the transfer to the original game works.

The game we choose as a test case of the sim-to-sim approach is Rocket League
(Figure 1), which basically resembles indoor football with cars and teams of
3. Rocket league is freely available for Windows and Mac, possesses a bot API
(RLBot [4]) and a community of bot developers next to a large human player
base. As the 3 members of each team control car avatars with physical
properties different from human runners, the overall tactics are the one of
rotation without fixed roles. Thereby, large parts of the current speed can be
conserved and players do not have to accelerate from zero when ball possession
changes [5]. Next to basic abilities attempting to shoot towards the goal and
to move the goalie in order to prevent a goal, Rocket League is a minimal
_team AI_ setting [6] where layers of team tactics and strategy can be
learned.

The first step of our work re-implements not all, but multiple physical
gameplay mechanics of Rocket League using the game engine Unity, which results
in a slightly inaccurate simulation. We then train an agent in a relatively
easy goalie and striker environment using Proximal Policy Optimization (PPO)
[7]. The learned behaviors are then transferred to Rocket League for
evaluation. Even though the training simulation is imperfect, the transferred
behaviors are robust enough to succeed at their tasks by generalizing to the
domain of Rocket League. The goalkeeping agent saves nearly $100\%$ of the
shots faced, while the striking agent scores about $75\%$ of its shots. The
sim-to-sim transfer is further examined by ablating physical adaptations that
were added to the training simulation.

This paper proceeds with elaborating on related work. Then, the physical
gameplay mechanics of Rocket League are shown. After illustrating the trained
goalie and striker environment, PPO and algorithmic details are presented.
Section V examines the sim-to-sim transfer. Before concluding our work, a
discussion is provided.

##  II Related Work

Sim-to-sim transfer on a popular multiplayer team video game touches majorly
on two different areas, namely multi-agent and sim-to-real transfer. DotA 2
and StarCraft ii@ are the already mentioned prominent examples in the field of
multi-agent environments. As this work focuses on single-agent environments,
namely the goalkeeper and striker environments, related work on sim-to-real
transfer is focused next.

Given the real world, a considered prime example for multi-agent scenarios is
_RoboCup_. RoboCup is an annual international competition [8] that offers a
publicly effective open challenge for the intersection of robotics and AI
research. The competition is known for the robot soccer cup but also includes
other challenges. Reinforcement Learning (RL) has been successfully applied to
simulated robot soccer in the past [9] and has been found a powerful method
for tackling robot soccer. A recent survey [10] provides insights into robot
soccer and highlights significant trends, which briefly mention the transfer
from simulation to the real world.

In general, sim-to-real transfer is a well-established method for robot
learning and is widely used in combination with RL. It allows the transition
of an RL agent’s behavior, which has been trained in simulations, to real-
world environments. Sim-to-real transfer has been predominantly applied to RL-
based robotics [11] where the robotic agent has been trained with state-of-
the-art RL techniques like PPO [7]. Popular applications for sim-to-real
transfer in robotics have been autonomous racing [12], Robot Soccer [13],
navigation [14], and control tasks [15]. To address the inability to exactly
match the real-world environment, a challenge commonly known as sim-to-real
gap, steps have also been taken towards generalized sim-to-real transfer for
robot learning [16, 17]. The translation of synthetic images to realistic ones
at the pixel level is employed by a method called GraspGAN [18] which utilizes
a generative adversarial network (GAN) [19]. GANs are able to generate
synthetic data with good generalization ability. This property can be used for
image synthesis to model the transformation between simulated and real images.
GraspGAN provides a method called pixel-level domain adaptation, which
translates synthetic images to realistic ones at the pixel level. The
synthesized pseudo-real images correct the sim-to-real gap to some extent.
Overall, it has been found that the respective policies learned with
simulations execute more successfully on real robots when GraspGAN is used
[18].

Another approach to narrow the sim-to-real gap is domain randomization [20].
Its goal is to train the agent in plenty of randomized domains to generalize
to the real domain. By randomizing all physical properties and visual
appearances during training in the simulation, a trained behavior was
successfully transferred to the real world to solve the Rubik’s cube [21].

##  III Rocket League Environment

This section starts out by providing an overview of vital components of Rocket
League’s physical gameplay mechanics, which are implemented in the training
simulation based on the game engine Unity and the ML-Agents Toolkit [22].
RLBot [4] provides the interface to Rocket League where the training
situations can be reproduced. Afterward, the DRL environments, designated for
training, and their properties are detailed. The code is open
source222https://github.com/PG642.

###  III-A Implementation of the Training Simulation

TABLE I: Overview on essential physical gameplay mechanics present in Rocket League, which are added to the training simulation. Physics Component | Sources | Additional Information and Different Parameters
---|---|---
Entity Measures (e.g. Arena) | [3, 4] |  | Car model Octane and its collision mesh is used
---
Radius of the ball is set to $93.15$uu (value in Rocket League $92.75$uu)
Car: Velocity, Acceleration, Boost | [23] | No modifications done
Car: Jumps, Double Jumps, Dodge Rolls | [24, 4] | Raise max. angular velocity during dodge from $5.5\frac{\text{rad}}{\text{s}}$ to $7.3\frac{\text{rad}}{\text{s}}$
Car: Air Control | [23] | Adjust drag coefficients for roll to $-4.75$ and pitch to $-2.85$
Bullet and Psyonix Impulse | [23, 25] |  | Used for the ball-to-car interaction and car-to-car interaction. The impulse by the
---
bullet engine replaces the Unity one. Psyonix impulse is an additional impulse
on the center of the ball, which allows a better prediction and control of
collisions.
Ball Bouncing | [23] | Within the bounce’s computation a ball radius of $91.25$uu is considered.
Friction (Air, Ground) and Drifting | [25] | A drag of $-525\frac{\text{uu}}{\text{s}^{2}}$ is used, which is reduced by more than half when the car is upside down.
Ground Stabilization | [4, 25] | The stabilization torque is denoted by an acceleration of $50\frac{\text{rad}}{\text{s}^{2}}$.
Wall Stabilization | [4] | Raise sticky forces for wall stabilization to an acceleration of $500\frac{\text{uu}}{\text{s}^{2}}$
Suspension |  [26] [27] |  | Stiffness of front wheels: $163.9\frac{1}{s^{2}}$ and of back wheels: $275.4\frac{1}{s^{2}}$
---
Damper front and back is set to $30\frac{1}{s}$. The equations used are
inspired by [27],
which may differ to the approach taken in Rocket League that remains unclear.
Car-to-car interaction |  | Implemented using the Bullet and Psyonix impulses, but not thoroughly tested
Demolitions | [28] | Implemented, but not thoroughly tested and hence not considered in this paper
Figure 2: The physical maneuver of a dodge roll is executed to exemplary show
the alignment of the Unity simulation to the ground truth by using different
max angular velocities.

The implementation of the Unity simulation originates from the so called
_RoboLeague_ repository [3]. As this version of the simulation is by far
incomplete and inaccurate, multiple fundamental aspects and concepts are
implemented, which are essentially based on the physical specifications of
Rocket League. These comprise, for example, the velocity and acceleration of
the ball and the car, as well as the concept of boosting. Jumps, double jumps
as well as dodge rolls are now possible, and also collisions and interactions.
There is friction caused by the interaction of a car with the ground, but also
friction caused by the air is taken into account.

However, further adjustments are necessary. Therefore, table I provides an
overview of all the material that was considered during implementing essential
physical components, while highlighting distinct adjustments that differ from
the information provided by the references. It has to be noted that most
measures are given in _unreal units_ (uu). To convert them to Unity’s scale,
these have to be divided by $100$.

Some adjustments are based on empirical findings by comparing the outcome of
distinct physical maneuvers inside the implemented training simulation and the
ground truth provided by Rocket League. A physical maneuver simulates several
player inputs over time, such as applying throttle and steering left or right.
While the simulation is conducted in both simulations, multiple relevant game
state variables like positions, rotations, and velocities are monitored for
later evaluation. Figure 2 is an example where the physical maneuver orders
the car to execute a dodge roll. Whereas the original max angular velocity of
$5.5\frac{\text{rad}}{\text{s}}$ does not compare well to the ground truth, a
more suitable value of $7.3\frac{\text{rad}}{\text{s}}$ is found by analyzing
the observed data.

The speed of the training simulation is about $950~{}steps/second$, while
RLBot is constrained to the real-time, where only $120~{}steps/second$ are
possible. This simulation performance is measured on a Windows Desktop
utilizing a GTX 1080 and a AMD Ryzen 7 3700X.

###  III-B Goalie Environment

In the goalie environment, the agent is asked to save shots. 1000 different
samples of shots, which uniformly vary in speed, direction, and origin, are
faced by the agent during training. In every episode, one shot is fired
towards the agent’s goal. The agent’s position is reset to the center of the
goal at the start of each episode. Every save rewards the agent with $+1$. A
goalkeeping episode terminates if the ball hits the goal or is deflected by
the agent.

###  III-C Striker Environment

To score a goal is the agent’s task inside the striker environment. The ball
moves bouncy, slowly, close, and in parallel to the goal. Its speed and origin
are sampled uniformly from 1000 samples during the agent’s training. The
agent’s position is farther away from the goal while being varied as well.
$+1$ is the only reward signal that the agent receives upon scoring. Once the
ball hits the goal or a time limit is reached, the episode terminates and the
environment is reset.

###  III-D Observation and Action Space

Figure 3: The contents of the agent’s observation.

Both environments share the same observation and action space. The agent
perceives 23 normalized game state variables to fully observe its environment
as illustrated by figure 3. The agent’s action space is multi-discrete and
contains the following 8 dimensions:

  *   * •

Throttle (5 actions)

  * •

Steer (5 actions)

  * •

Yaw (5 actions)

  * •

Pitch (5 actions)

  * •

Roll (3 actions)

  * •

Boost (2 actions)

  * •

Drift or Air Roll (2 actions)

  * •

Jump (2 actions)

Rocket League is usually played by humans using a gamepad as input device.
Some of the inputs (e.g. thumbstick) are thus continuous and not discrete. To
simplify the action space, the continuous actions throttle, steer, yaw, and
pitch are discretized using buckets as suggested by Pleines et al. [29]. By
this means, the agent picks one value from a bucket containing the values
$-1$, $-0.5$, $0$, $0.5$ and $1$. The roll action is also discretized using
the values $-1$, $0$ and $1$. All other actions determine whether the
concerned discrete action is executed or not. The action dimension that is in
charge of drifting and air rolling is another special case. Both actions can
be boiled down to one because drifting is limited to being on the ground,
whereas air rolling can be done in the air only. Moreover, multi-discrete
action spaces allow the execution of concurrent actions. One discrete action
dimension could achieve the same behavior. This would require defining actions
that feature every permutation of the available actions. As a consequence, the
already high-dimensional action space of Rocket League would be much larger
and therefore harder to train.

##  IV Deep Reinforcement Learning

The actor-critic, on-policy algorithm PPO [7] and its clipped surrogate
objective (Equation 1) is used to train the agent’s policy $\pi$, with respect
to its model parameters $\theta$, inside the Unity simulation. PPO,
algorithmic details, and the model architecture are presented next.

###  IV-A Proximal Policy Optimization

$L_{t}^{C}(\theta)$ denotes the policy objective, which optimizes the
probability ratio of the current policy $\pi_{\theta}$ and the old one
$\pi_{\theta old}$:

$L^{C}_{t}(\theta)=\hat{\mathbb{E}}_{t}[min(q_{t}(\theta)\hat{A}_{t},clip(q_{t}(\theta),1-\epsilon,1+\epsilon)\hat{A}_{t})]$
(1) $\textnormal{with the surrogate
objective}~{}q_{t}(\theta)=\frac{\pi_{\theta}(a_{t}|s_{t})}{\pi_{\theta\text{old}}(a_{t}|s_{t})}$

$s_{t}$ is the environment’s state at step $t$. $a_{t}$ is an action tuple,
which is executed by the agent, while being in $s_{t}$. The clipping range is
stated by $\epsilon$ and $\hat{A}_{t}$ is the advantage, which is computed
using generalized advantage estimation [30]. While computing the squared error
loss $L_{t}^{V}$ of the value function, the maximum between the default and
the clipped error loss is determined.

$V_{t}^{C}=V_{\theta old}(s_{t})+clip(V_{\theta}(s_{t})-V_{\theta
old}(s_{t}),-\epsilon,\epsilon)$ (2)
$L_{t}^{V}=max((V_{\theta}(s_{t})-G_{t})^{2},(V_{t}^{C}-G_{t})^{2})$ (3)
$\textnormal{with the sampled return}~{}G_{t}=V_{\theta
old}(s_{t})+\hat{A}_{t}$

The final objective is established by $L^{CVH}_{t}(\theta)$:

$L^{CVH}_{t}(\theta)=\hat{\mathbb{E}}_{t}[L^{C}_{t}(\theta)-c_{1}L^{V}_{t}(\theta)+c_{2}\mathcal{H}[\pi_{\theta}](s_{t})]$
(4)

To encourage exploration, the entropy bonus $\mathcal{H}[\pi_{\theta}](s_{t})$
is added and weighted by the coefficient $c_{2}$. Weighting is also applied to
the value loss using $c_{1}$.

###  IV-B Algorithmic Details and Model Architecture

PPO starts out by sampling multiple trajectories of experiences, which may
contain multiple completed and truncated episodes, from a constant number of
concurrent environments (i.e. workers). The model parameters are then
optimized by conducting stochastic gradient descent for several epochs of
mini-batches, which are sampled from the collected data. Before computing the
loss function, advantages are normalized across each mini-batch. The computed
gradients are clipped based on their norm.

Figure 4: The policy and the value function share gradients and several
parameters. After feeding 23 game states variables as input to the model and
processing a shared fully connected layer, the network is split into a policy
and value stream starting with their own fully connected layer. The policy
stream outputs action probabilities for each available action dimension,
whereas the value stream exposes its estimated state-value.

A relatively shallow neural net (model) is shared by the value function and
the policy (Figure 4). To support multi-discrete actions, the policy head of
the model outputs 8 categorical action probability distributions. During
action selection, each distribution is used to sample actions, which are
provided to the agent as a tuple. The only adjustment to the policy’s loss
computation is that the probabilities of the selected actions are
concatenated. Concerning the entropy bonus, the mean of the action
distributions’ entropies is used.

TABLE II: The resulted error for each run physical maneuver scenario. The
car’s position is considered by the green shaded data points, while the blue
ones are related to the ball’s position.

| 1) Acceleration | 2) Air Control | 3) Drift | 4) Jump | 5) Ball Bounce | 6) Shot
---|---|---|---|---|---|---
Mean | 0.69 | 3.72 | 1.67 | 2.32 | 3.07 | 5.24 | 3.19 | 0.84 | 4.87 | 0.07 | 0.22 | 1.37 | 0.01 | 0.05 | 0.03 | 28.45 | 23.79 | 28.31
Std | 0.48 | 3.05 | 1.92 | 2.61 | 1.81 | 4.58 | 4.73 | 1.04 | 6.80 | 0.06 | 0.15 | 0.69 | 0.01 | 0.03 | 0.02 | 25.20 | 25.01 | 22.49
Max | 1.21 | 8.04 | 5.96 | 8.40 | 8.12 | 12.97 | 16.06 | 5.12 | 21.08 | 0.24 | 0.41 | 2.02 | 0.02 | 0.12 | 0.07 | 58.16 | 59.00 | 58.19

##  V Sim-to-sim Transfer

Two major approaches are considered to examine learned behaviors inside the
Unity simulation and its transfer to Rocket League. The first one runs various
handcrafted scenarios (like seen in section III-A) in both simulations to
directly compare their alignment. This way, it can be determined whether the
car or the ball behave similarly or identically concerning their positions and
velocities. The second approach trains the agent in Unity given the goalie and
the striker environment, while all implemented physics components are
included. We further conduct an ablation study on the implemented physics
where each experiment turns off one or all components. Turning off may also
refer to use the default physics of Unity.

If not stated otherwise, each training run is repeated 5 times and undergoes a
thorough evaluation. Each model checkpoint is evaluated in Unity and Rocket
League by 10 training and 10 novel shots, which are repeated 3 times.
Therefore, each data point aggregates 150 episodes featuring one shot. Result
plots show the interquartile mean of the cumulative reward and a confidence
interval of 95% as recommended by Agarwal et al. [31]. The hyperparameters are
detailed in Table III. At last, we describe some of the learned behaviors that
are also retrieved from training in a more difficult striker environment.

TABLE III: The hyperparameters used to conduct the training with PPO. The learning rate $\alpha$ and $c_{2}$ decay linearly over time. Hyperparameter | Value | Hyperparameter | Value
---|---|---|---
Discount Factor $\gamma$ | 0.99 | Clip Range $\epsilon$ | 0.2
$\lambda$ (GAE) | 0.95 | $c_{1}$ | 0.25
Number of Workers | 16 | Initial $\alpha$ | 0.0003
Worker Steps | 512 | Min $\alpha$ | 0.000003
Batch Size | 8192 | Initial $c_{2}$ | 0.0005
Epochs | 3 | Min $c_{2}$ | 0.00001
Mini Batches | 8 | Optimizer | AdamW
Max Gradient Norm | 0.5 | Activations | ReLU

###  V-A Alignment Comparison using Handcrafted Scenarios

To directly compare the alignment between both simulations, six physical
maneuvers are assessed by 3 different handcrafted scenarios:

  1. 1.

Acceleration

     * •

Car drives forward and steers left and right

     * •

Car drives backward and steers left and right

     * •

Car uses boost and steers left and right

  2. 2.

Air Control

     * •

Car starts up in the air, looks straight up, boosts shortly and boosts while
rolling in the air

     * •

Car starts up in the air, has an angle of $45^{\circ}$, boosts shortly and
boosts while rolling in the air

     * •

Car starts up in the air, looks straight up and concurrently boosts, yaws, and
air rolls

  3. 3.

Drift

     * •

Car drives forward for a bit and then starts turning and drifting while moving
forward

     * •

Car drives backward for a bit and then starts turning and drifting while
moving forward

     * •

Car uses boost and then starts turning and drifting while using boost

  4. 4.

Jump

     * •

Car makes a short jump, then a long one and at last a double jump

     * •

Car makes a front flip, a back flip and a dodge roll

     * •

Car drives forward, does a diagonal front flip and at last a back flip

  5. 5.

Ball Bounce

     * •

Ball falls straight down

     * •

Ball falls down with an initial force applied on its x-axis

     * •

Ball falls down with an initial force applied on its x-axis and an angular
velocity

  6. 6.

Shot

     * •

Car drives forward and hits the motionless ball

     * •

Car drives forward and the ball rolls to the car

     * •

Ball jumps, the car jumps while boosting and hits the ball using a front flip

Each scenario tracks the position of the ball and the car during each frame.
As both simulations end up monitoring the incoming data with slight time
differences, the final data is interpolated to match in shape. Afterward, the
error for each data point between both simulations is measured. The final
results are described by Table II, which comprises the mean, max, and standard
deviation (Std) error across each run scenario. Letting the ball bounce for
some time shows the least error, while a significant one is observed when
examining the scenarios where the car shoots the ball. Note that slight
inaccuracies during acceleration may cause a strongly summed error when
considering a different hit location on the ball. The other scenarios, where
the error is based on the car’s position, also indicate that the Unity
simulation suffers from inaccuracies.

###  V-B Physics Ablation Study based on PPO Training

Figure 5: Results of training the goalie environment under different ablations
and transferring it to Rocket League. The agent is evaluated on training shots
and ones, which were not seen during training. The agent easily solves the
goalie task under all circumstances. Both, training and unseen shots, behave
identically in Rocket League.

Figure 6: Results of training the striker environment under different
ablations and transferring it to Rocket League. The agent is evaluated on
training situations and ones, which were not seen during training. The agent
scores in about 75% of the played episodes given all physical adaptations,
while any ablation turns out catastrophic. Both, training and unseen
situations, behave identically in Rocket League.

The previously shown imperfections of the Unity simulation may lead to the
impression that successfully transferring a trained behavior is rather
unlikely. This assumption can be negated by considering the results retrieved
from training the agent in the goalie environment (Figure 5). Even though each
experiment ablates all, single or no physical adaptations, the agent is still
capable of saving nearly every ball once transferred to Rocket League. A
drawback of the goalie environment lies in its simplicity because the agent
only has to somehow hit the ball to effectively deflect it. The next step of
complexity is posed by the striker environment, where the agent has to land a
more accurate hit on the ball to score a goal. Figure 6 illustrates the
results of the striker training. Notably, when all physical adaptations are
present, the transferred behavior manages to score in about 75% of the played
episodes. Catastrophic performances emerge in Rocket League once single
physical adaptations are turned off.

###  V-C Learned Policies

During the performed experiments, several intriguing agent behaviors
emerged333https://www.youtube.com/watch?v=WXMHJszkz6M&list=PL2KGNY2Ei3ix7Vr_vA-
ZgCyVfOCfhbX0C. When trained as a goalkeeper, the agent tries to hit the ball
very early, while making its body as big as possible towards the ball. This is
achieved by simultaneously jumping and rolling forward or executing a forward
flip. Concerning the striker environment, the agent usually approaches the
ball using its boost. To get a better angle to the ball, the agent steers left
and right or vice versa. Drifting is sometimes used to aid this purpose.
Jumping is always used when needed. This is usually the case if the agent is
close to the ball, which is located above the agent. Otherwise, the agent’s
preference is to stay on the ground. Further training experiments were
conducted in a more difficult striker environment. The ball is not anymore
simply passed in parallel and close to the goal. Instead, the ball bounces
higher and farther away from the goal, which increases the challenge of making
a good touch on the ball to score. Given this setting, two different policies
were achieved. One policy approaches the ball as fast as possible while using
a diagonal dodge roll to make the final touch to score. However, this behavior
fails a few shots. The other emerged behavior can be considered as the
opposite. Depending on the distance and the height of the ball, the agent
waits some time or even backs up to ensure that it will hit the ball while
being on the ground. Therefore, the agent avoids jumping. This is surprising
because the agent should maximize its discounted cumulative reward and
therefore finish the episode faster. Although the increased difficulty led to
different behaviors, the agent may struggle a lot to get there. Usually, 2 out
of 5 training runs succeeded, while the other ones utterly failed.

##  VI Discussion

In this work, the agent is trained on isolated tasks, which are quite apart
from a complete match of Rocket League. To train multiple cooperative and
competitive agents, the first obstacle that comes to mind is the tremendously
high computational complexity, which might be infeasible for smaller research
groups. But before going this far, several aspects need to be considered that
can be treated in isolation as well. At last, the difficulties of training the
more difficult striker environments are discussed.

###  VI-A On Improving the Sim-to-sim Transfer

At first, the Unity simulation is still lacking the implementation of physical
concepts like the car-to-car interaction and suffers from the reported
(Section V-A) inaccuracies. These can be further improved by putting more work
into the simulation, but also other approaches are promising. At the cost of
more computational resources, domain randomization [20] could achieve a more
robust agent, potentially comprising an improved ability to generalize to the
domain of Rocket League. As the ground truth is provided by Rocket League,
approaches from the field of supervised learning can be considered as well.

###  VI-B Training under Human Conditions

Once the physical domain gap is narrowed, the Unity simulation still does not
consider training under human conditions. Notably, the current observation
space provides perfect information on the current state of the environment,
whereas players in Rocket League have to cope with imperfect information due
to solely perceiving the rendered image of the game. Thus, the Unity
simulation has to implement Rocket League’s camera behavior as well. However,
one critical concern is that the RLBot API does not reveal the rendered image
of Rocket League and therefore makes a transfer impossible as of now. However,
even if that information is made available by Psyonix, both simulations’
visual appearances are very different. The Unity simulation’s aesthetics are
very abstract, whereas Rocket League impresses with multiple arenas featuring
many details concerning lighting, geometry, shaders, textures, particle
effects, etc.. To overcome this gap of visual appearance, approaches of the
previously described related work, like GraspGAN [18], can be considered.

Another challenge arises once the environment is partially observable. It
should be considered that the agent will probably need memory to be able to
compete with human players. Otherwise, the agent might not be able to capture
the current affairs of its teammates and opponents. For this purpose, multiple
memory-based approaches might be suitable, like using a recurrent neural
network or a transformer architecture.

Moreover, the multi-discrete action space used in this paper is a
simplification of the original action space that features concurrent
continuous and discrete actions. Initially, the training was done using the
PPO implementation of the ML-Agents toolkit [22], which supports mixed (or
hybrid) concurrent action spaces. However, these experiments were quite
unstable and hindered progress. Therefore, Rocket League presents an
interesting challenge for exploring such action spaces, of which other video
games or applications are likely to take advantage.

###  VI-C Difficulties of Training the harder Striker Environment

While the goalie and the striker environment are relatively easy, the slightly
more difficult striker one poses a much greater challenge due to multiple
reasons:

  * •

Episodes are longer, leading to an even more delayed reward signal and more
challenging credit assignment

  * •

More states have to be explored by the agent

  * •

Even more accurate touches on the ball have to be made to score

To overcome these problems, curriculum learning [32] and reward shaping [33]
can be considered. In curriculum learning, the agent could face easier
scenarios first and once success kicks in, the next level of difficulty can be
trained. However, catastrophic forgetting may occur and therefore a curriculum
should sample from a distribution of scenarios to mitigate this issue.

Concerning reward shaping, multiple variants were casually tried without
improving training results:

  * •

Reward the first touch on the ball

  * •

Reward or penalize the distance between the ball and the agent

  * •

Reward or penalize the dot product between the car’s velocity and the
direction from the car to the ball

Adding more reward signals along the agent’s task introduces bias and is
likely task-irrelevant. For example, the agent could exploit such signals to
cuddle with the ball at a close distance or to slowly approach the ball to
maximize the cumulative return of the episode. If those signals are turned off
once the ball is touched, the value function might struggle to make further
good estimates on the value of the current state of the environment, which
ultimately may lead to misleading training experiences and hence an unstable
learning process. In spite of the results of these first explorative tests,
future work shall examine whether these points shall be reconsidered.

##  VII Conclusion

Towards solving Rocket League by the means of Deep Reinforcement Learning, a
fast simulation is crucial, because the original game cannot be sped up and
neither parallelized on Linux-based clusters. Therefore, we advanced the
implementation of a Unity project that mimics the physical gameplay mechanics
of Rocket League. Although the implemented simulation is not perfectly
accurate, we remarkably demonstrate that transferring a trained behavior from
Unity to Rocket League is robust and generalizes when dealing with a
goalkeeper and striker task. Hence, the sim-to-sim transfer is a suitable
approach for learning agent behaviors in complex game environments. After all,
Rocket League still poses further challenges when targeting a complete match
under human circumstances. Based on our findings, we believe that Rocket
League and its Unity counterpart will be valuable to various research fields
and aspects, comprising: sim-to-sim transfer, partial observability, mixed
action-spaces, curriculum learning, competitive and cooperative multi-agent
settings.

## References

  * [1] O. Vinyals, I. Babuschkin, W. M. Czarnecki, M. Mathieu, A. Dudzik, J. Chung, D. H. Choi, R. Powell, T. Ewalds, P. Georgiev, J. Oh, D. Horgan, M. Kroiss, I. Danihelka, A. Huang, L. Sifre, T. Cai, J. P. Agapiou, M. Jaderberg, A. S. Vezhnevets, R. Leblond, T. Pohlen, V. Dalibard, D. Budden, Y. Sulsky, J. Molloy, T. L. Paine, Ç. Gülçehre, Z. Wang, T. Pfaff, Y. Wu, R. Ring, D. Yogatama, D. Wünsch, K. McKinney, O. Smith, T. Schaul, T. P. Lillicrap, K. Kavukcuoglu, D. Hassabis, C. Apps, and D. Silver, “Grandmaster level in starcraft II using multi-agent reinforcement learning,” _Nat._ , vol. 575, no. 7782, pp. 350–354, 2019\.
  * [2] C. Berner, G. Brockman, B. Chan, V. Cheung, P. Debiak, C. Dennison, D. Farhi, Q. Fischer, S. Hashme, C. Hesse, R. Józefowicz, S. Gray, C. Olsson, J. Pachocki, M. Petrov, H. P. de Oliveira Pinto, J. Raiman, T. Salimans, J. Schlatter, J. Schneider, S. Sidor, I. Sutskever, J. Tang, F. Wolski, and S. Zhang, “Dota 2 with large scale deep reinforcement learning,” _CoRR_ , vol. abs/1912.06680, 2019.
  * [3] RoboLeague, “Roboleague,” 2021, available at https://github.com/roboserg/RoboLeague retrieved February 28, 2022.
  * [4] RLBot, “Rlbot wiki: Getting started,” 2021, available at https://github.com/RLBot/RLBot/wiki retrieved February 28, 2022.
  * [5] Y. Verhoeven and M. Preuss, “On the potential of rocket league for driving team ai development,” in _2020 IEEE Symposium Series on Computational Intelligence (SSCI)_ , 2020, pp. 2335–2342.
  * [6] M. Mozgovoy, M. Preuss, and R. Bidarra, “Guest editorial special issue on team AI in games,” _IEEE Trans. Games_ , vol. 13, no. 4, pp. 327–329, 2021\.
  * [7] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” _arXiv:1707.06347_ , 2017.
  * [8] H. Kitano, M. Asada, Y. Kuniyoshi, I. Noda, and E. Osawa, “Robocup: The robot world cup initiative,” in _Proceedings of the First International Conference on Autonomous Agents, AGENTS 1997, Marina del Rey, California, USA, February 5-8, 1997_ , W. L. Johnson, Ed. ACM, 1997, pp. 340–347.
  * [9] M. J. Hausknecht and P. Stone, “Deep reinforcement learning in parameterized action space,” in _4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings_ , Y. Bengio and Y. LeCun, Eds., 2016.
  * [10] E. Antonioni, V. Suriani, F. Riccio, and D. Nardi, “Game strategies for physical robot soccer players: A survey,” _IEEE Trans. Games_ , vol. 13, no. 4, pp. 342–357, 2021.
  * [11] W. Zhao, J. P. Queralta, and T. Westerlund, “Sim-to-real transfer in deep reinforcement learning for robotics: a survey,” in _2020 IEEE Symposium Series on Computational Intelligence, SSCI 2020, Canberra, Australia, December 1-4, 2020_. IEEE, 2020, pp. 737–744.
  * [12] B. Balaji, S. Mallya, S. Genc, S. Gupta, L. Dirac, V. Khare, G. Roy, T. Sun, Y. Tao, B. Townsend, E. Calleja, S. Muralidhara, and D. Karuppasamy, “Deepracer: Autonomous racing platform for experimentation with sim2real reinforcement learning,” in _2020 IEEE International Conference on Robotics and Automation, ICRA 2020, Paris, France, May 31 - August 31, 2020_. IEEE, 2020, pp. 2746–2754.
  * [13] J. Blumenkamp, A. Baude, and T. Laue, “Closing the reality gap with unsupervised sim-to-real image translation for semantic segmentation in robot soccer,” _CoRR_ , vol. abs/1911.01529, 2019.
  * [14] R. Traoré, H. Caselles-Dupré, T. Lesort, T. Sun, N. D. Rodríguez, and D. Filliat, “Continual reinforcement learning deployed in real-life using policy distillation and sim2real transfer,” _CoRR_ , vol. abs/1906.04452, 2019.
  * [15] O. Pedersen, E. Misimi, and F. Chaumette, “Grasping unknown objects by coupling deep reinforcement learning, generative adversarial networks, and visual servoing,” in _2020 IEEE International Conference on Robotics and Automation, ICRA 2020, Paris, France, May 31 - August 31, 2020_. IEEE, 2020, pp. 5655–5662.
  * [16] K. Rao, C. Harris, A. Irpan, S. Levine, J. Ibarz, and M. Khansari, “Rl-cyclegan: Reinforcement learning aware simulation-to-real,” in _2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020_. Computer Vision Foundation / IEEE, 2020, pp. 11 154–11 163.
  * [17] D. Ho, K. Rao, Z. Xu, E. Jang, M. Khansari, and Y. Bai, “Retinagan: An object-aware approach to sim-to-real transfer,” in _IEEE International Conference on Robotics and Automation, ICRA 2021, Xi’an, China, May 30 - June 5, 2021_. IEEE, 2021, pp. 10 920–10 926.
  * [18] K. Bousmalis, A. Irpan, P. Wohlhart, Y. Bai, M. Kelcey, M. Kalakrishnan, L. Downs, J. Ibarz, P. Pastor, K. Konolige, S. Levine, and V. Vanhoucke, “Using simulation and domain adaptation to improve efficiency of deep robotic grasping,” in _2018 IEEE International Conference on Robotics and Automation, ICRA 2018, Brisbane, Australia, May 21-25, 2018_. IEEE, 2018, pp. 4243–4250.
  * [19] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. C. Courville, and Y. Bengio, “Generative adversarial nets,” in _Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Canada_ , Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, Eds., 2014, pp. 2672–2680.
  * [20] J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel, “Domain randomization for transferring deep neural networks from simulation to the real world,” in _2017 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2017, Vancouver, BC, Canada, September 24-28, 2017_. IEEE, 2017, pp. 23–30.
  * [21] OpenAI, I. Akkaya, M. Andrychowicz, M. Chociej, M. Litwin, B. McGrew, A. Petron, A. Paino, M. Plappert, G. Powell, R. Ribas, J. Schneider, N. Tezak, J. Tworek, P. Welinder, L. Weng, Q. Yuan, W. Zaremba, and L. Zhang, “Solving rubik’s cube with a robot hand,” _CoRR_ , vol. abs/1910.07113, 2019\.
  * [22] A. Juliani, A. Khalifa, V. Berges, J. Harper, E. Teng, H. Henry, A. Crespi, J. Togelius, and D. Lange, “Obstacle tower: A generalization challenge in vision, control, and planning,” in _Proceedings of the 28th International Joint Conference on Artificial Intelligence, IJCAI 2019_ , 2019, pp. 2684–2691.
  * [23] S. Mish, “Rocket league notes,” 2019, available at https://samuelpmish.github.io/notes/RocketLeague/ retrieved February 28, 2022.
  * [24] Timo Huth, “Dodges explained. power & more - rocket science #14,” 2018, available at https://www.youtube.com/watch?v=pX950bhGhJE retrieved February 28, 2022.
  * [25] J. Cone, “It is rocket science! the physics of ’rocket league’ detailed,” 2018, available at https://www.gdcvault.com/play/1024972/It-IS-Rocket-Science-The retrieved February 28, 2022.
  * [26] Timo Huth, “Why are wheel hits so odd? - rocket science #10,” 2017, available at https://www.youtube.com/watch?v=pTAVP00xwF4 retrieved February 28, 2022.
  * [27] Vehicle Physics Pro, “How simple suspensions work,” available at https://vehiclephysics.com/advanced/how-suspensions-work/ retrieved February 28, 2022.
  * [28] Rocket Sledge, “How demos actually work in rocket league,” 2020, available at https://www.gamersrdy.com/blog/2020/06/11/how-demos-actually-work-in-rocket-league/#:~:text=Demos%20were%20not%20intended%20to,broken%20for%20a%20long%20time retrieved February 28, 2022.
  * [29] M. Pleines, F. Zimmer, and V. Berges, “Action spaces in deep reinforcement learning to mimic human input devices,” in _IEEE Conference on Games, CoG 2019, London, United Kingdom, August 20-23, 2019_. IEEE, 2019, pp. 1–8.
  * [30] J. Schulman, P. Moritz, S. Levine, M. I. Jordan, and P. Abbeel, “High-dimensional continuous control using generalized advantage estimation,” in _4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings_ , Y. Bengio and Y. LeCun, Eds., 2016.
  * [31] R. Agarwal, M. Schwarzer, P. S. Castro, A. Courville, and M. G. Bellemare, “Deep reinforcement learning at the edge of the statistical precipice,” in _Thirty-Fifth Conference on Neural Information Processing Systems_ , 2021\.
  * [32] Y. Bengio, J. Louradour, R. Collobert, and J. Weston, “Curriculum learning,” in _Proceedings of the 26th Annual International Conference on Machine Learning, ICML 2009, Montreal, Quebec, Canada, June 14-18, 2009_ , ser. ACM International Conference Proceeding Series, A. P. Danyluk, L. Bottou, and M. L. Littman, Eds., vol. 382. ACM, 2009, pp. 41–48.
  * [33] V. Gullapalli and A. Barto, “Shaping as a method for accelerating reinforcement learning,” in _Proceedings of the 1992 IEEE International Symposium on Intelligent Control_ , 1992, pp. 554–559.


# Giant interfacial induced Dzyaloshinskii-Moriya interaction in
polycrystalline Co/Pt multilayers

D.A. Tatarskiy<EMAIL_ADDRESS>Institute for physics of microstructure of
RAS, Akademicheskaya st., 7, Afonino, 603087, Russia  Lobachevsky University,
Gagarin ave., 23, Nizhniy Novgorod, 603950, Russia  N.S. Gusev  S.A. Gusev
Institute for physics of microstructure of RAS, Akademicheskaya st., 7,
Afonino, 603087, Russia

###### Abstract

Multilayers of heavy metals and ferromagnets are the basis for the formation
of various structures with inhomogeneous chiral magnetic distributions. The
most informative method for studying magnetic states in such structures with a
high spatial resolution is Lorentz transmission electron microscopy (L-TEM).
Here, we report on the observation of chiral Néel domain walls by means of
L-TEM in Co/Pt multilayers with perpendicular magnetic anisotropy. We explain
the existence of chiral Néel internal structure due to the overcritical, giant
interfacial induced Dzyaloshinskii–Moriya interaction. Our results are
confirmed by micromagnetic modeling, which allows taking into account the
effect of local inhomogeneity of the magnetic anisotropy of polycrystalline
films on formation of spin textures.

††preprint: AIP/123-QED

##  I Introduction

Noncollinear spin textures is one of the most interesting and intensively
studied objects of the magnetism during last years due to their unusual
topological and transport properties and their potential applications for data
storage and information carriers in low-power nanoelectronics [1, 2].
Multilayer films, consisting of ferromagnetic (FM) and heavy metal (HM)
layers, are the most popular systems for these magnetic applications because
their magnetic properties can be fine-tuned by appropriate choice of materials
and layer thicknesses, total number of layers and their crystal structure. In
these multilayer films with interfacial induced Dzyaloshinskii-Moriya
interaction [3, 4, 5, 6] (iDMI), it is possible to stabilize nanoscale spin
textures such as skyrmions at room temperatures [7, 8]. Thus, systems of thin
metal films of nanometer thickness with iDMI, due to the relatively simple
technology of their formation, seem to be the most promising structures as a
basis for constructing logical [9, 10, 11] or neuromorphic [12] magnetic
devices.

It is also known that the presence of iDMI in multilayer metal films leads to
topologically homochiral 360 degree Néel domain walls [13] in media with
perpendicular magnetic anisotropy. Also Néel skyrmions and Néel walls are
experimentally observed in some types of multilayers consisting of films of
three different materials [14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25]. In
such structures, the magnetic layers are in contact with layers of heavy
metals and oxides or with two different heavy metals. In this case, the value
and sign of iDMI at the upper and lower boundaries of the ferromagnetic layer
may have different values, and the total iDMI value is expected to be the
greatest one, and this will improve the stability of skyrmions formation and
reduce their size. In addition to such structures, there is some oblique
evidence for the presence of iDMI in quasi-symmetric multilayers [26, 27] of
two types of metal layers Co/Pd. But these multilayers have platinum capping
and underlayers, and they are not purely symmetrical from a material point of
view. Obviously the iDMI is canceled in ideal system consisting of two
materials. That is true for purely epitaxial interfaces [28]. Still the iDMI
value can be tuned by material concentration variations in a top layer [28] or
by growth conditions [29]. One should mention that observed iDMI value in
epitaxial films is few times smaller than it should be from the first
principle calculations even taking into account interface roughness [30].

The most common method used in manufacture of multilayer metal structures is
magnetron sputtering. With this manufacturing method, the metal films have a
polycrystalline structure with imperfect interfaces due to the roughness of
the boundary layers, different orientations of nanocrystals or the mixing of
ferromagnet and heavy metal. These imperfections can introduce asymmetry into
the structure of surfaces “FM/HM” and “HM/FM” of the multilayer, and
therefore, the iDMI value in multilayers can differ at the “top” and “bottom”
interfaces of the ferromagnetic film [31]. Thus, the presence of iDMI due to
the structural asymmetric interfaces can be expected in magnetron-deposited
polycrystalline multilayers, which consist of only two metals, for example,
cobalt and platinum.

Another way to get iDMI in the multilayer films system is an interlayered
mechanism due to spin-orbit coupling not on the interface but on interlayers
of heavy metal [32, 33]. Recently this mechanism of iDMI has been demonstrated
in cobalt-palladium multilayer between platinum top and bottom layers [26].
The most fascinating question wether one can get large enough iDMI in more
homogeneous films, e.g. cobalt-platinum multilayer instead of cobalt-
palladium.

There are several experimental methods which can be applied to study the
distribution of magnetization in nanostructures and can provide information on
the presence of the iDMI in thin film systems. The Brillouin light scattering
[34] and neutron scattering technique [35, 36] are used to study the
Dzyaloshinskii-Moria interaction. However, these methods have a low spatial
resolution, which does not allow to obtain local magnetization distributions
arising in structures with iDMI. The most qualitative and quantitative method
for studying inhomogeneous states in thin films with a high spatial resolution
is Lorentz transmission electron microscopy (L-TEM). In this paper, we present
the experimental results using the L-TEM method, which demonstrate the
existence of iDMI in multilayer Co/Pt films. The micromagnetic modeling was
used to show the effect of nanocrystalline structure of Co and Pt layers on
magnetization distributions in multilayer films, which allowed us to explain
the features of the experimentally observed Fresnel contrast.

##  II Samples preparation and methods

The magnetic multilayer films with Ta(3.2)/Pt(5.5)/[Co(0.5)/Pt(1.0)]×5/Pt(3.8)
structure (where 5 denotes the layer repetition numbers, thickness in nm) are
grown by DC magnetron sputtering on 50-nm-thick Si3N4 membrane windows. The
films are fabricated in a high vacuum magnetron AJA 2200 system at the basic
argon pressure of $4\times 10^{-3}$ Torr. The preliminary pumping of the
working chamber is carried out to a vacuum of $10^{-5}$ Torr. The sputtering
is carried out from separate Ta, Co, and Pt targets at room temperature. The
deposition rates of the materials are 0.1 nm/s for cobalt films and 0.3 nm/s
for platinum. The thicknesses of the films are determined by the deposition
time after preliminary calibration of the thicknesses of individual layers,
measured by X-ray reflectometry. The additional 2 nm platinum layer is
deposited on the top periodic structure to avoid oxidation. Recently, we have
experimentally shown that the thicknesses of the layers obtained by this
technology are in good agreement with the calculated values by studying the
cross-sections of similar structures with transmission electron microscopy
[37].

(a)

(b)

(c)

Figure 1:  (a) – High resolution micrograph of multilayer, the distance
between atomic planes is $\sim 0.225$ nm; (b) – Bright field overview of the
co/Pt multilayer; (c) – Average profile of intensity, extracted from (b) along
red line.

Magnetic properties of the Co/Pt samples are studied by magneto-optical Kerr
effect (MOKE) measurements using custom-made magneto-optical stand with 632 nm
wavelength He-Ne laser. Transmission electron microscopy performed by LIBRA200
MC (Carl Zeiss) instrument operating at 200 kV is used to study the
microstructure of the films. We study the cross-sectional structure of
multilayer films and analyzed in-plane micrographs. TEM lamella preparation is
performed by FIB-SEM crossbeam station Auriga Laser (Carl Zeiss). The
fabricated cross-sections are also polished using a low-energy ($\sim$ 300 eV)
Ar+ source to reduce the thickness of the damaged layers. The L-TEM
measurements are performed by Cs–corrected TITAN 80-300 transmission electron
microscope (FEI) operated at 80 kV. In these experiments, it is possible in
situ to observe changes in the Fresnel contrast caused by an external magnetic
field, which is created at the location of the sample due to weak excitation
of the microscope objective lens and tilting the sample to re-magnetize it.
The maximum value of this magnetic field in our experiments was 35 mT.

We use MuMax3 GPU-accelerated micromagnetic software [38] to simulate the
distributions of magnetization in multilayer structures consisting of
polycrystalline films. Polycrystalline grains in MuMax3 are simulated by the
Voronoi tessellation method [39]. We suppose each grain has its own value and
direction of uniaxial anisotropy and iDMI strength. The film as a whole has an
average nonzero anisotropy along its normal. For the sake of simplicity we use
model of random uniaxial anisotropy in each crystalline grain even if the
crystallographic structure can be cubic or hexagonal as for typical cobalt,
platinum and their alloys. Other micromagnetic parameters (the saturation
magnetization, the exchange stiffness) are homogeneous over the film and their
values are the same as in our previous papers [40, 41]. The distributions of
magnetization in the structures obtained by numerical calculations are used to
simulate images with magnetic contrast in the transmission electron
microscope. To simulate L-TEM images, a specially developed original Python
script for the Gatan Microscopy Suite Software® 3.4.1 is used [42]. It is
known that the value of the contrast $\Delta I$ in Fresnel mode is
proportional to the projection of magnetization curl on the optical axis [43,
44] of an electron microscope and defocus value $\Delta z$

$\Delta I\left(\vec{\rho}\right)\sim\Delta
z\left(\vec{n}\cdot\left[\vec{\nabla}\times\vec{M}\left(\vec{r}\right)\right]\right).$
(1)

where $\vec{\rho}$ is a vector in the detector plane and $\vec{r}$ is a vector
in the plane of the film. In general case the sample is tilted and the film’s
and detector’s plane do not coincides. The total value of the intensity
(taking into account the material contrast $I_{0}(\vec{\rho})$ is an inverse
Fourier transform (IFFT) of the convolution of the Fourier image of (1) with
the phase transfer function [45]

$I_{d}\left(\vec{\rho}\right)\sim\left|\textrm{IFFT}\left[\int\left(I_{0}\left(\vec{q_{\bot}}\right)+\Delta
I\left(\vec{q_{\bot}}\right)\right)\exp\left(\pi\lambda\Delta
z\left|\vec{q}_{\bot}\right|^{2}\right)\exp\left(-\frac{\left(\pi\theta\Delta
z\right)^{2}\left|\vec{q}_{\bot}\right|^{2}}{ln2}\right)d\vec{q_{\bot}}\right]\right|,$
(2)

where $\lambda$ is electron wavelength, $\theta$ is beam divergence and
$\vec{q}_{\bot}$ is a reciprocal lattice vector in the detector’s plane. The
Fresnel contrast simulations are done taking to account TEM transfer function
for 80 keV electron energy, defocus 2 mm and beam divergence $\theta=0.09$
mrad. Then these calculated images are compared with the LTEM images obtained
experimentally.

(a)

(b)

(c)

Figure 2:  (a) – Microdiffraction pattern; (b) – the dark field micrograph of
Co/Pt multilayer; (c) – autocorrelation function of dark field micrograph.

##  III Results and discussions

TEM images are obtained for the overall characterization of the
microstructures of Co/Pt multilayers. The high-resolution TEM image (HRTEM) of
the Co/Pt cross-section clearly shows the crystal lattice of cobalt and
platinum layers with an interplanar spacing of 0.225 nm (Fig. 1a)), which can
best correspond to the (111) plane of the $Co_{1+x}Pt_{3-x}$ solid solution
with Pm3m space group of symmetry lattice. However, the Bright Field (BF)
image of a cross-section of a multilayer film shows the contrast between the
Co and Pt layers (Fig. 1b) which indicates the layered structure of the
sample. The average value of the Co layer thickness measured from the averaged
profiles of micrograph contrast (Fig. 1c) is 0.5 nm, and the thickness of the
Pt layer is 1.0 nm ,which is in good agreement with the values that were set
by the time of the magnetron deposition of films. Analysis of TEM in-plane
images of the film provides additional information on the microstructure of
the Co/Pt sample. The Selected Area Diffraction (SAD) patterns of a film
corresponds to a cubic lattice (Fig. 2a) with lattice parameter $a=0.390\pm
0.008$ nm. This value is between the parameters of platinum lattice
$a_{Pt}=0.392$ nm and cubic lattice of solid solution $CoPt_{3}$
$a_{CoPt}=0.384$ nm, which shows the presence of elastic stresses in the
multilayered structure. A typical Dark Field (DF) micrograph of Co/Pt
polycrystalline multilayer and its autocorrelation function is shown on Fig.
2b,c. The full width at half maximum of autocorrelation function is $\sim 5.6$
nm. This value can be taken to account for calculation of volume-weighted
average grain size [46, 47], and is used by us in the simulations. Basing upon
these measurements we fit the experimental MOKE hysteresis curve by
micromagnetic simulations.

Figure 3:  Circles – magneto-optical measurements; red curve – simulated
hysteresis; blue curve – minor hysteresis, numbers indicate where the Fresnel
contrast is acquired through in situ remagnetization.

We simulate the $256\times 256\times 7.5$ nm3 part of the film with $0.5\times
0.5\times 7.5$ nm3 unit cell. A total count of crystallites was around $\sim
1800$, but only 256 different material parameters (variations of magnetic
anisotropy value and its direction, iDMI value etc.) were taking into account
due to the MuMax3 software restrictions. In other worlds every 7 crystallites
in different parts of simulation have equal material parameters. This approach
demonstrates a good approximation of a real random polycrystalline film. From
our previous experiments we suppose the values for magnetic parameters of
material [41, 40].

(a)

(b)

(c)

Figure 4:  Fresnel contrast on micrographs. Figures are acquired at points
1,2,3 from hysteresis curve. The contrasts are obtained at 30 degrees tilt
along denoted axis.

The difference is that in this work we took into account a nonzero value iDMI,
which could have different values for different crystallites of the film. The
iDMI value for each crystallite was set randomly, and the maximum deviation
did not exceed $15\%$ from the average value. In this range of deviations from
the mean value, the magnetic anisotropy value $K$ for each nanocrystal was
also randomly set. The set of material parameters for modeling was chosen so
as to ensure the best match between the simulation and experimental MOKE and
L-TEM data. The best fit is achieved for magnetization $M_{s}=200$ kA/m,
exchange stiffness $J=7\times 10^{-13}$ pJ/m, $K=3.1\times 10^{4}\pm 15\%$
J/m3, wandering of anisotropy direction from film normal up to $8.5$ degrees,
iDMI value $\left|D\right|=60\pm 15\%$ $\mu$J/m2. The micromagnetic
simulations demonstrates not only a good quantitative agreement with a
magneto-optical measurements (Fig.3) but a good agreement with obtained
Fresnel contrasts during in situ remagnetization of the sample (compare Figs.
4 and 5).

(a)

(b)

(c)

(d)

Figure 5:  (a),(b) Simulated magnetization distributions of points 2 and 3
from hysteresis curve, rainbow color denote in-plane magnetization direction,
black-white colors denote $M_{z}$ direction; (c),(d) – simulated Fresnel
contrast at 30 degree tilt along diagonal axis.

The following experimental observations may indicate the presence of a nonzero
iDMI. The first one is existence of striped-type contrast on L-TEM images with
the Fresnel contrast (Figs. 4b) in a some range of external fields
(excitations of the objective lens). The micromagnetic nature of stripes is
the following. Domain walls in magnetic film with perpendicular magnetic
anisotropy are Bloch-type (6a) in the absence of iDMI. Such distribution is
favorable due to minimal magnetostatic energy. A weak iDMI tends to tilt the
domain wall (6b) as iDMI minimizes the energy for magnetic distribution with
nonzero magnetostatic charges $\left(\vec{\nabla}\cdot\vec{M}\right)\neq 0$.
When the iDMI interaction is larger then a critical value [48], the domain
wall becomes pure Néel-type (6c). This value for our magnetic parameters is
$\left|D_{cr}\right|=0.44\mu_{0}M_{s}^{2}t/\pi=52.8$ $\mu J/m^{2}$, where $t$
is the film thickness. Sharp steps on hysteresis curve (point “2” at Fig. 3)
as between 20-30 mT appears only for $\left|D\right|\gtrsim D_{cr}$. The
magnetization distribution at this point is a topological 360 degrees domain
walls (Fig. 6d), which look like black-white stripes in Fresnel mode (Fig.
5b&d). The interplay of magnetostatic and iDMI leads to the situation when two
180 degrees Néel domain walls are too close but they can’t annihilate (on the
contrary to Bloch walls). The iDMI value for micromagnetic simulation can be
selected easily from experimental data. On the one hand it should be larger
than critical value $D_{cr}$, on the other hand a large iDMI (about $65-75\mu
J/m^{2}$) leads to formation of isolated Néel skyrmions instead of 360 degrees
walls.

(a)

(b)

(c)

(d)

Figure 6:  (a) – Bloch, (b) – canted, (c) – Néel and (d) – 360 degrees Néel
domain walls with schematically shown magnetostatic charges distribution.

We also have observed weak Fresnel contrast at zero tilting angles. The
explanation is the following. In case of pure Néel walls in homogeneous film
there should be zero magnetic contrast at zero tilt [13]. The analysis of
Fresnel contrast with simulation of polycrystalline nature of magnetron
sputtered films shows nonzero contrast for pure Néel walls at zero tilt
conditions (Fig. 7). The small contrast variations arise due to the
magnetization tends to lie along wandering axis of the uniaxial magnetic
anisotropy of nanocrystalline film structure. The tilting angle is 30 degrees
along diagonal of the image. The origin of contrast on Fig. 7a&b is following.
Wandering anisotropy axes leads to a weak canting of spins in domain wall from
pure Néel configuration. That leads to a nonzero values for magnetization
rotor and gives rise to local Fresnel contrast. Due to nanocrystalline origin
of this contrast it is of the same order as the contrast from small
magnetization cants of “homogeneously” magnetized areas. Thus the
nanocrystalline structure of magnetron sputtered films can be the origin of
weak magnetic contrasts in polycrystalline films [27, 22].

(a)

(b)

(c)

(d)

Figure 7:  Comparison of Fresnel and simulated contrast from 360 degrees Néel
domain wall.

##  IV Conclusion

Summing up, we experimentally have investigated the crystal and magnetic
structure of the Co/Pt multilayer films with perpendicular magnetic
anisotropy. Basing upon some previous research [26] we see the significant
role of interlayers in formation of homochiral magnetic textures. In this
paper we have shown the existense of interlayer contribution in iDMI even in
case of “symmetrical” material structure of multilayer. It is surprising fact
that the magnetron sputtered Co/Pt multilayers demonstrate the existence of a
strong iDMI. Lorentz transmission electron microscopy and MOKE measurements
analysed with micromagnetic simulations confirm the iDMI value above critical,
which makes it possible to stabilize chiral distribution of magnetization such
as Néel domain walls in perpendicular magnetic anisotropy media. Also a
careful analysis of L-TEM micrographs shows the presence of a weak magnetic
Fresnel contrast due to the influence of the wandering axis of uniaxial
magnetic anisotropy in nanocrystalline grains. Such interesting behaviour of
Co/Pt multilayers (without any third material, such as Pd, Ta, W, Ir etc.) can
simplify the technology of production thin magnetic films, hosting homochiral
Néel skyrmions at room temperatures.

###### Acknowledgements.

The work was supported by Russian Foundation for Basic Research Grant #
18-02-00827. The equipment of Physics and Technology of Micro- and
Nanostructures (Institute for Physics of Microstructures, Russian Academy of
Sciences) was used. The equipment of CIC nanoGUNE BRTA (SAN Sebastian, Spain)
is used for performing LTEM measurements. The authors are grateful to A.A.
Fraerman and M.V. Sapozhnikov for fruitful discussion; M.Yu. Mikhailovskii and
P.A. Yunin for sample preparation; K.V. Tatarskaya for help with the
manuscript preparation.

## References

  * Yu  _et al._ [2017] G. Yu, P. Upadhyaya, Q. Shao, H. Wu, G. Yin, X. Li, C. He, W. Jiang, X. Han, P. Amiri, and K. Wang, “Room-temperature skyrmion shift device for memory application,” Nano Letters 17, 261–268 (2017).
  * Nagaosa and Tokura [2013] N. Nagaosa and Y. Tokura, “Topological properties and dynamics of magnetic skyrmions,” Nature Nanotechnology 8, 899–911 (2013).
  * Dzyaloshinskii [1958] I. E. Dzyaloshinskii, “A thermodynamic theory of weak ferromagnetism of antiferromagnetics,” J.Phys.Chem.Solids 4, 241 (1958).
  * Moriya [1964] T. Moriya, “Theory of helicoidal structures in antiferromagnets. i. nonmetals,” Phys.Rev. 120, 91 (1964).
  * Bogdanov and Rossler [2001] A. N. Bogdanov and U. K. Rossler, “Chiral symmetry breaking in magnetic thin films and multilayers,” Phys.Rev.Lett. 87, 037203 (2001).
  * Yang  _et al._ [2015] H. Yang, A. Thiaville, S. Rohart, A. Fert, and M. Chshiev, “Anatomy of dzyaloshinskii-moriya interaction at co/pt interfaces,” Phys.Rev.Lett. 115, 267210 (2015).
  * Boulle  _et al._ [2016] O. Boulle, J. Vogel, H. Yang, S. Pizzini, D. De Souza Chaves, A. Locatelli, T. Menteş, A. Sala, L. Buda-Prejbeanu, O. Klein, M. Belmeguenai, Y. Roussigné, A. Stashkevich, S. Mourad Chérif, L. Aballe, M. Foerster, M. Chshiev, S. Auffret, I. Miron, and G. Gaudin, “Room-temperature chiral magnetic skyrmions in ultrathin magnetic nanostructures,” Nature Nanotechnology 11, 449–454 (2016).
  * Wang  _et al._ [2019a] L. Wang, C. Liu, N. Mehmood, G. Han, Y. Wang, X. Xu, C. Feng, Z. Hou, Y. Peng, X. Gao, and G. Yu, “Construction of a room-temperature pt/co/ta multilayer film with ultrahigh-density skyrmions for memory application,” ACS Applied Materials and Interfaces 11, 12098–12104 (2019a).
  * Fert, Reyren, and Cros [2017] A. Fert, N. Reyren, and V. Cros, “Magnetic skyrmions: advances in physics and potential applications,” Nat.Rev.Mater. 2, 17031 (2017).
  * Kim  _et al._ [2018] J.-S. Kim, H.-J. Lee, J.-I. Hong, and C.-Y. You, “Field driven magnetic racetrack memory accompanied with theinterfacial dzyaloshinskii-moriya interaction,” J.Magn.Magn.Mater. 455, 45 (2018).
  * Mendisch  _et al._ [2019] S. Mendisch, G. Ziemys, V. Ahrens, A. Papp, and M. Becherer, “Pt/co/w as a candidate for low power nanomagnetic logic,” J.Magn.Magn.Mater. 485, 345 (2019).
  * Prychynenko  _et al._ [2018] D. Prychynenko, M. Sitte, K. Litzius, B. Kruger, G. Bourianoff, M. Klaui, J. Sinova, and K. Everschor-Sitte, “Magnetic skyrmion as a nonlinear resistive element: A potential building block for reservoir computing,” Phys.Rev.Appl. 9, 014034 (2018).
  * Benitez  _et al._ [2015] M. Benitez, A. Hrabec, A. Mihai, T. Moore, G. Burnell, D. McGrouther, C. Marrows, and S. McVitie, “Magnetic microscopy and topological stability of homochiral neel domain walls in a $pt/co/alo_{x}$ trilayer,” Nat.Comms. 6, 8957 (2015).
  * Zhang, Petford-Long, and Phatak [2016] S. Zhang, A. K. Petford-Long, and C. Phatak, “Creation of artificial skyrmions and antiskyrmions by anisotropy engineering,” Sci.Rep. 6, 31248 (2016).
  * He  _et al._ [2017] M. He, L. Peng, Z. Zhu, G. Li, J. Cai, J. Li, H. Wei, L. Gu, S. Wang, T. Zhao, B. Shen, and Y. Zhang, “Realization of zero-field skyrmions with high-density via electromagnetic manipulation in pt/co/ta multilayers,” Appl.Phys.Rev. 111, 202403 (2017).
  * Zhang  _et al._ [2018a] S. Zhang, J. Zhang, Y. Wen, E. Chudnovsky, and X. Zhang, “Creation of a thermally assisted skyrmion lattice in pt/co/ta multilayer films,” Appl.Phys.Lett. 113, 192403 (2018a).
  * Zhang  _et al._ [2018b] S. Zhang, J. Zhang, Y. Wen, E. Chudnovsky, and X. Zhang, “Determination of chirality and density control of neel-type skyrmions with in-plane magnetic field,” Nat.Comms.Phys. 1, 36 (2018b).
  * Jiang  _et al._ [2019] W. Jiang, S. Zhang, X. Wang, C. Phatak, Q. Wang, W. Zhang, M. B. Jungfleisch, J. E. Pearson, Y. Liu, J. Zhang, X. Cheng, A. Petford-Long, A. Hoffmann, and S. G. E. te Velthuis, “Quantifying chiral exchange interaction for neel-type skyrmions via lorentz transmission electron microscopy,” Phys.Rev.B 99, 104402 (2019).
  * Chen  _et al._ [2013] G. Chen, J. Zhu, A. Quesada, J. Li, A. T. N’Diaye, Y. Huo, T. P. Ma, Y. Chen, H. Kwon, C. Won, Z. Q. Qiu, A. K. Schmid, and Y. Z. Wu, “Novel chiral magnetic domain wall structure in fe/ni/cu(001) films,” Phys.Rev.Lett. 110, 177204 (2013).
  * McVitie  _et al._ [2018] S. McVitie, S. Hughes, K. Fallon, S. McFadzean, D. McGrouther, M. K. adn W. Legrand, D. Maccariello, S. Collin, K. Garcia, N. Reyren, V. Cros, A. Fert, K. Zeissler, and C. H. Marrows, “A transmission electron microscope study of neel skyrmion magnetic textures in multilayer thin film systems with large interfacial chiral interaction,” Sci.Rep. 8, 5703 (2018).
  * Chess  _et al._ [2017] J. J. Chess, S. A. Montoya, E. E. Fullerton, and B. J. McMorran, “Determination of domain wall chirality using in situ lorentz transmission electron microscopy,” AIP Adv. 7, 056807 (2017).
  * Fallon  _et al._ [2019] K. Fallon, S. McVitie, W. Legrand, F. Ajejas, D. Maccariello, S. Collin, V. Cros, and N. Reyren, “Quantitative imaging of hybrid chiral spin textures in magnetic multilayer systems by lorentz microscopy,” Phys.Rev.B 100, 214431 (2019).
  * Wang  _et al._ [2019b] L. Wang, C. Liu, N. Mehmood, G. Han, Y. Wang, X. Xu, C. Feng, Z. Hou, Y. Peng, X. Gao, and G. Yu, “Construction of a room-temperature pt/co/ta multilayer film with ultrahigh-density skyrmions for memory application,” ACS Applied Materials and Interfaces 11, 12098–12104 (2019b).
  * Chen  _et al._ [2017] G. Chen, S. Kang, C. Ophus, A. T. N’Diaye, H. Kwon, R. Qiu, C. Won, K. Liu, Y. Wu, and A. Schmid, “Out-of-plane chiral domain wall spin-structures in ultrathin in-plane magnets,” Nat.Comms. 8, 15302 (2017).
  * Krupinski  _et al._ [2019] M. Krupinski, P. Sobieszczyk, P. Zielinski, and M. Marszalek, “Magnetic reversal in perpendicularly magnetized antidot arrays with intrinsic and extrinsic defects,” Sci.Rep. 9, 13276 (2019).
  * Pollard  _et al._ [2017] S. D. Pollard, J. A. Garlow, J. Yu, Z. Wang, Y. Zhu, and H. Yang, “Observation of stable neel skyrmions in cobalt/palladium multilayers with lorentz transmission electron microscopy,” Nat.Comms. 8, 14761 (2017).
  * Garlow  _et al._ [2019] J. A. Garlow, S. D. Pollard, M. Beleggia, T. Dutta, H. Yang, and Y. Zhu, “Quantification of mixed bloch-neel topological spin textures stabilized by the dzyaloshinskii-moriya interaction in co/pd,” Phys.Rev.Lett. 122, 237201 (2019).
  * Hrabec  _et al._ [2019] A. Hrabec, K. Shahbazi, T. Moore, E. Martinez, and C. Marrows, “Tuning spin–orbit torques at magnetic domain walls in epitaxial $pt/co/pt_{1-x}au_{x}$ trilayers,” Nanotechnology 30, 234003 (2019).
  * Wells  _et al._ [2017] A. Wells, P. Shepley, C. Marrows, and T. Moore, “Effect of interfacial intermixing on the dzyaloshinskii-moriya interaction in pt/co/pt,” Phys.Rev.B 95, 054428 (2017).
  * Zimmermann  _et al._ [2018a] B. Zimmermann, W. Legrand, D. Maccariello, N. Reyren, V. Cros, S. Blügel, and A. Fert, “Dzyaloshinskii-moriya interaction at disordered interfaces from ab initio theory: Robustness against intermixing and tunability through dusting,” Applied Physics Letters 113, 232403 (2018a), cited By 10.
  * Zimmermann  _et al._ [2018b] B. Zimmermann, W. Legrand, N. Reyren, V. Cros, S. Blugel, and A. Fert, “Dzyaloshinskii-moriya interaction at disordered interfaces from ab initio theory: robustness against intermixing and tunability through dusting,” Appl.Phys.Lett. 113, 232403 (2018b).
  * Vedmedenko  _et al._ [2019] E. Y. Vedmedenko, P. Riego, J. A. Arregi, and A. Berger, “Interlayer dzyaloshinskii-moriya interactions,” Phys.Rev.Lett. 122, 257202 (2019).
  * Fernandez-Pacheco  _et al._ [2019] A. Fernandez-Pacheco, E. Vedmedenko, F. Ummelen, R. Mansell, D. Petit, and R. P. Cowburn, “Symmetry-breaking interlayer dzyaloshinskii–moriya interactions in synthetic antiferromagnets,” Nature Materials 18, 679 (2019).
  * Belmeguenai  _et al._ [2019] M. Belmeguenai, Y. Roussigné, S. M. Cherif, A. Stashkevich, T. Petrisor, M. Nasui, and M. S. Gabor, “Influence of the capping layer material on the interfacial dzyaloshinskii-moriya interaction in pt/co/capping layer structures probed by brillouin light scattering,” J.Phys.D:Appl.Phys. 52, 125002 (2019).
  * Meynell  _et al._ [2017] S. A. Meynell, M. N. Wilson, K. L. Krycka, B. J. Kirby, H. Fritzsche, and T. L. Monchesky, “Neutron study of in-plane skyrmions in mnsi thin films,” Phys.Rev.B 96, 054402 (2017).
  * Muhlbauer  _et al._ [2019] S. Muhlbauer, D. Honecker, E. A. Perigo, F. Bergner, S. Disch, A. Heinemann, S. Erokhin, D. Berkov, C. Leighton, M. R. Eskildsen, and A. Michels, “Magnetic small-angle neutron scattering,” Rev.Mod.Phys. 91, 015004 (2019).
  * Tatarskiy  _et al._ [2019] D. A. Tatarskiy, N. S. Gusev, V. Y. Mikhailovskii, Y. V. Petrov, and S. A. Gusev, “Control over the magnetic properties of co/pt-based multilayered periodical structures,” Technical Physics 64, 1584 (2019).
  * Vansteenkiste  _et al._ [2014] A. Vansteenkiste, J. Leliaert, M. Dvornik, M. Helsen, F. Garcia-Sanchez, and B. V. Waeyenberge, “The design and verification of mumax3,” AIP Advances 4, 107133 (2014).
  * Leliaert  _et al._ [2014] J. Leliaert, B. V. de Wiele, A. Vansteenkiste, L. Laurson, G. Durin, L. Dupre, and B. V. Waeyenberge, “Current-driven domain wall mobility in polycrystalline permalloy nanowires: A numerical study,” J.Appl.Phys. 115, 233903 (2014).
  * Sapozhnikov  _et al._ [2016] M. Sapozhnikov, S. Vdovichev, O. Ermolaeva, N. Gusev, A. Fraerman, S. Gusev, and Y. Petrov, “Artificial dense lattice of magnetic bubbles,” Applied Physics Letters 109, 042406 (2016).
  * Sapozhnikov  _et al._ [2020] M. Sapozhnikov, Y. Petrov, N. Gusev, A. Temiryazev, O. Ermolaeva, V. Mironov, and O. Udalov, “Artificial dense lattices of magnetic skyrmions,” Materials 13, 99 (2020).
  * Tatarskiy [2020] D. Tatarskiy, “Fresnel contrast simulation for gatan microscopy suite 3.4.1,”  (2020).
  * McVitie and White [2004] S. McVitie and G. White, “Imaging amperian currents by lorentz microscopy,” J.Phys.D: Appl.Phys. 37, 280 (2004).
  * McVitie and Cushley [2006] S. McVitie and M. Cushley, “Quantitative fresnel lorentz microscopy and the transport of intensity equation,” Ultramicroscopy 106, 423 (2006).
  * de Graef [2001] M. de Graef, “2. lorentz microscopy: Theoretical basis and image simulations,” Experimental Methods in the Physical Sciences 36, 27 (2001).
  * Heilbronner [1992] R. Heilbronner, “The autocorrelation function: an image processing tool for fabric analysis,” Tectonophysics 212, 351 (1992).
  * Zang, Suzuki, and Liu [2018] B. Zang, K. Suzuki, and A. Liu, “Estimation of volume-weighted average grain size in fe-based nanocrystalline soft magnetic materials by autocorrelation function,” Materials Characterization 142, 577 (2018).
  * Meier, Kronseder, and Back [2017] T. Meier, M. Kronseder, and C. Back, “Domain-width model for perpendicularly magnetized systems with dzyaloshinskii-moriya interaction,” Phys.Rev.B 96, 144408 (2017).