(0,-0.75)[c]Convention of the Society for the Study of Artificial Intelligence and Simulation of Behaviour (AISB), 13-14 April 2023, Swansea, UK

Local Minima Drive Communications in Cooperative Interaction

Roger K. Moore
Dept. Computer Science

University of Sheffield

Sheffield, UK

An important open question in human-robot interaction (HRI) is precisely when an agent should decide to communicate, particularly in a cooperative task.  Perceptual Control Theory (PCT) tells us that agents are able to cooperate on a joint task simply by sharing the same `intention', thereby distributing the effort required to complete the task among the agents.  This is even true for agents that do not possess the same abilities, so long as the goal is observable, the combined actions are sufficient to complete the task, and there is no local minimum in the search space.  If these conditions hold, then a cooperative task can be accomplished without any communication between the contributing agents.  However, for tasks that do contain local minima, the global solution can only be reached if at least one of the agents adapts its intention at the appropriate moments, and this can only be achieved by appropriately timed communication.  In other words, it is hypothesised that in cooperative tasks, the function of communication is to coordinate actions in a complex search space that contains local minima.  These principles have been verified in a computer-based simulation environment in which two independent one-dimensional agents are obliged to cooperate in order to solve a two-dimensional path-finding task.

cooperation, communication, interaction, perceptual control theory, search, local minima

§ INTRODUCTION

An important open question in human-robot interaction (HRI) is precisely when an agent should decide to communicate <cit.>.  Unfortunately, research in human-human interaction has been obsessed with `turn-taking' as the underlying mechanism <cit.>, somewhat overlooking the observation that conversation can overlap as well as interleave <cit.>, as well as ignoring the question as to why agents should communicate in the first place <cit.>.  Clearly, communication supports information exchange <cit.> and learning <cit.>, but more importantly it facilitates collaborative problem solving <cit.> and goal sharing <cit.>, i.e. cooperation.  However, little research has been conducted into what conditions the timing and structure of communication in continuous cooperative interaction <cit.>.

This paper addresses these issues from the perspective of Perceptual Control Theory (PCT) <cit.>.  Results are presented from a PCT-based simulation of a cooperative task, and it is shown how appropriately timed communication between agents can overcome local minima in a joint problem space.

§ COMMUNICATION IN COOPERATION

Perceptual Control Theory (PCT) is founded on the mantra “behaviour is the control of perception”, and agents are modelled as a hierarchy of negative-feedback control loops.  Solidly grounded in the tradition of `cybernetics' <cit.>, PCT has been shown to be capable of accounting for a wide range of `intelligent' phenomena based on a parsimonious architecture of replicated closed-loop structures <cit.>.  In particular, PCT tells us that agents are able to cooperate on a joint task simply by sharing the same reference signal, i.e. by having the same intention <cit.>.  The consequence is that the effort required to complete a task may be distributed among the agents involved.

However, it is claimed here that successful convergence towards a solution of a joint task is based on three assumptions:

* the goal is observable (that is, each agent has an appropriate input function),

* the combined actions are sufficient to complete the task (that is, the agents possess complimentary output functions), and

* the goal is accessible (that is, there are no local minima in the search space).

If these three conditions are met, then a cooperative task can be accomplished without any communication between the contributing agents.

This means that, for tasks that do have local minima, the global solution can only be reached if at least one of the agents adapts its intention at the appropriate moment(s).  That is, an agent may need to abandon its original goal in favour of a temporary alternative that facilitates an escape from a local minimum.  Such behaviour requires timely coordination between the agents, and this can only be achieved by appropriately-timed communication.  In other words, it is hypothesised that, in cooperative tasks, one function of communication is to coordinate actions in a complex search space that contains one or more local minima.  From a PCT perspective, this implies that a perceived signal from one agent should trigger a change in a reference signal for another agent.

This hypothesis has been verified in a computer-based simulation in which two independent one-dimensional agents are obliged to communicate (that is, actively cooperate) in order to solve a two-dimensional path-finding task <cit.>.

§ SIMULATION ENVIRONMENT

The simulation environment – implemented in the Pure Data (Pd) dataflow programming language <cit.> – is illustrated in  <ref>.  Two 1D agents control the X and Y positions of a `vehicle' in a 2D space, the task being to steer the vehicle towards a `target' location.  Each agent can only `see' the target in its single dimension, hence cooperation may be required to solve the joint 2D problem.   The difficulty of the task is scaled by the introduction of various forms of obstruction (as illustrated in  <ref>), and the `solution time' (ST) for each successful run was measured.

Screenshot of the Pd-based simulation environment showing the target (green square), the vehicle (red circle) and two barriers (yellow lines).  The X and Y axes depict the 1D projection of the vehicle, target and barriers (if visible from the agent's perspective).  The number in the top-right corner indicates the elapsed time in the current run, and the number in the top-left corner shows the number of runs completed.

There are many configurations in which each controlling X and Y agent can move the vehicle towards the target by reducing their individual `error' in a monotonic fashion (that is, by gradient descent), even if there are barriers present.  For example,  <ref> shows a configuration with three barriers but no local minimum.  Also, some barrier configurations create situations in which it is impossible for the vehicle to reach the target at all – see  <ref>.

Screenshot of the Pd-based simulation environment showing a configuration with three barriers but no local minimum.  This means that the vehicle can reach the target without getting `stuck'.

However, some configurations (such as the one shown in  <ref>) create situations which require agents to increase their error momentarily in order for the vehicle to eventually reach the target.  For example, if one agent has reached its target (in 1D), but the other is stuck behind a barrier, then the first needs to be requested to abandon its target temporarily in an attempt to free the second agent.  Hence, the presence/absence of timely communications is critical in determining whether a run is ultimately successful or not.

Since not all configurations are solvable, the simulation environment was set up such that any experimental run lasting longer than 30 seconds was terminated and marked as `did not finish' (DNF).  In such cases, the solution time was ignored in subsequent data analysis.

Screenshot of the Pd-based simulation environment showing a configuration with three barriers in which it is impossible for the vehicle to reach the target.

Table <ref> lists the variables instantiated in the simulations.  Overall, five cooperation modes were implemented per agent (listed at the bottom of Table <ref>), and different combinations were able to be specified by means of an agent-specific 4-bit binary code.  This meant that there was a total of 16 possible levels of cooperation available for each agent.  Two of these involved no communication at all, but distinguished between just stopping at an obstruction (i.e. no active cooperation) versus moving randomly (i.e. potential cooperation without communicating).

Simulation Environment Setup

2|c|Agent Variables

Forward, Reverse, Stop

Collided with edge

Collided with barrier* (“stuck”)

Target known

Target accessible* (“access”)

Arrived at target (“arrived”)

2|c|Experiment Variables

Initial target position [XY]

Initial vehicle position [XY]

Number of barriers [0–3]

Location, orientation & size of barrier(s) [XYRL]

Number of runs [100–1000]

Target view [true/false]

Cooperation level [0000–1111]$^{\mathrm{a-e}}$

Loop gain [0.01]

Back-off time [1000 msec]

Time to target (ST) [secs.]

Did not finish (DNF) [#]

Time spent communicating [%]

2l$^{\mathrm{a}}$[0000] if self = target known $\Rightarrow$ approach target; else stop

2l$^{\mathrm{b}}$[1000] if self = target known $\Rightarrow$ approach target; else move randomly

2l$^{\mathrm{c}}$[0100] if self = “arrived” + other = “stuck” $\Rightarrow$ back-off

2l$^{\mathrm{d}}$[0010] if self = “stuck” + other = “stuck” $\Rightarrow$ back-off

2l$^{\mathrm{e}}$[0001] if self = “access” + other = “access” $\Rightarrow$ approach target

Of particular interest are each agent's `status' parameters that were available to be communicated for a given level of cooperation.  These are marked with a * in Table <ref>.

The first parameter – “stuck” – relates to the identification of a potential local minimum.  Such a condition arises when one agent has collided with a barrier and the other has arrived at the target, or when both agents have collided with barriers.  Crucially, it was realised that just one agent being stuck at a barrier is not sufficient evidence for a local minimum, as the other agent may be making progress which could resolve the problem.

The second parameter – “access” – relates to whether the target was accessible, i.e. there was no barrier between the agent and the target.  However, it is important to appreciate that such a condition does not guarantee a successful approach, as the target may subsequently become inaccessible for one agent due to the activities of the other agent.

§ EXPERIMENTS & RESULTS

A number of experiments have been conducted, each using multiple simulation runs to investigate different configurations of obstacles and levels of cooperation.  For example,  <ref> shows the distribution of solution times resulting from 1000 runs in an environment containing two fixed barriers (configured as shown in  <ref>.) for four incremental levels of cooperation.  As expected, enabling explicit communication between the agents had a measurable effect in speeding up solution times.  However, it was also noted that the low solution times for [1000] was due to the high number of runs that did not finish (DNF).  In particular, the results revealed that [1000] gave rise to 64% DNFs, whereas [1100] had 13% DNFs, [1110] had 1% DNFs, and [1111] had only 0.6% DNFs.

[boxplot/draw direction=y,xlabel=Cooperation Level,xtick=1,2,3,4,xticklabels=[1000],[1100],[1110],[1111], ylabel=Solution Time, ST (msecs)]
+[boxplot,black,style=solid,mark options=fill=black,mark=asterisk]
table[row sep=
,y index=0]

+[boxplot,black,style=solid,mark options=fill=black,mark=asterisk]
table[row sep=
,y index=0]

+[boxplot,black,style=solid,mark options=fill=black,mark=asterisk]
table[row sep=
,y index=0]

+[boxplot,black,style=solid,mark options=fill=black,mark=asterisk]
table[row sep=
,y index=0]

Distributions of solution times for different levels of cooperation in an environment containing two fixed barriers.

In attempting to analyse the results of the more complex cooperation experiments, it became clear that an overall `goodness measure' was needed in order to resolve the compromise between fast solution times and the numbers of runs that did not finish.  This was necessary because, as seen above, a high number of DNFs tends to give rise to a low mean solution time because the runs that succeed have less challenging barrier configurations.  Likewise, a low number of DNFs may be associated with higher mean solution times as a consequence of the cooperating agents taking longer to solve more challenging barrier configurations.

Hence, an appropriate `goodness measure' was defined as:

\begin{equation}
GM = log(ST^{1+\frac{DNF}{nruns}}),
\end{equation}

where $GM$ is the goodness measure (low is good), $ST$ is the mean solution time for a run, $DNF$ is the number of times a run did not finish, and $nruns$ is the number of runs.

<ref> shows the combined results from the solution times shown in  <ref> and the corresponding number of DNFs plotted using the goodness measure.  This representation clearly shows that, as expected, increasing the level of cooperation between the agents leads to significant improvements in their ability to solve the designated task.

[xlabel=Cooperation Level,ylabel=Goodness Measure, GM,xtick=1,2,3,4,xticklabels=[1000], [1100], [1110], [1111]]
+[black, only marks, mark options=fill=black, error bars/.cd,
y dir=both,y explicit]
(1, 6.0828)
(2, 4.4972)
(3, 3.5445)
(4, 3.5989)

Relationship between the cooperation level and the `goodness measure' (low is good) in an environment containing two fixed barriers.

§.§ Matched Agents

As mentioned above, the simulation environment allowed the cooperation level to be set for each agent independently.  However, due to the combinatorics, the majority of experiments were conducted with matched agents.  For example,  <ref> shows the impact of all sixteen levels of cooperation ranked by the `goodness' of the outcome for matched agents in an environment containing three randomly placed barriers with random lengths and orientations.

[x label style=at=(axis description cs:0.5,-0.1),anchor=north,xlabel=Cooperation Level, ylabel=Goodness Measure, GM,xtick=0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15,xticklabels=[0111], [0110], [1110], [0010], [1111], [1010], [0011], [1011], [1100], [1101], [1000], [1001], [0000], [0100], [0001], [0101],xticklabel style=rotate=90]
+[black, only marks, mark options=fill=black, error bars/.cd,
y dir=both,y explicit]
(12, 5.8590)
(10, 5.6970)
(13, 5.9125)
(8, 5.2083)
(3, 4.9432)
(5, 4.9779)
(1, 4.8731)
(2, 4.9420)
(14, 5.9170)
(11, 5.7455)
(15, 6.0606)
(9, 5.5148)
(6, 4.9792)
(7, 5.0412)
(0, 4.8324)
(4, 4.9586)

Relationship between different cooperation combinations ordered by their `goodness' (low is good) for matched agents in an environment containing three randomly placed barriers with random lengths and orientations.

As can be seen in  <ref>, the relationship between different combinations of cooperation and the goodness measure reveals that the cooperation combination [0111] “arrived”+“stuck”, “stuck”+“stuck” and “access”+“access” gives rise to the best overall performance.  The second-best is [0110] “arrived”+“stuck” and “stuck”+“stuck”.  Next is [1110] “random movements”, “arrived”+“stuck”, and “stuck”+“stuck”, and fourth is [0010] “stuck”+“stuck”.  The following three combinations [1010], [0011] and [1011] also have relatively high `goodness', and confirm that the top eight all have [0010] `stuck”+“stuck” enabled, and performance drops significantly without it.

The highest number of DNFs was 854/1000 (for [0001]), the lowest was 284/1000 (for [0111]), and there were 769/1000 DNFs for no cooperation at all ([0000]).  These results imply that up to 28% of barrier configurations were unsolvable and $\sim$23% were solvable without cooperation, which means that $\sim$49% were able to be solved with cooperation.  The fact that [0001] resulted in a higher number of DNFs than [0000] implies that enabling the “access”+“access” strategy was actually detrimental to performance.

With regard to the proportion of time agents spent communicating, the results shown in  <ref> reveal that there is a clear relationship between the goodness of the cooperation combinations and the proportion of time the agents spent communicating.  As noted above, this is a function of whether [0010] “stuck”+“stuck” is enabled or disabled, and it clearly reflects the frequency with which situations containing local minima arise given the three random barriers.  It is also interesting to note that the highest levels of communication occurred for the top two cooperation combinations.

[x label style=at=(axis description cs:0.5,-0.1),anchor=north,xlabel=Cooperation Level,ylabel=Proportion of Time Communicating (%),xtick=0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15,xticklabels=[0111], [0110], [1110], [0010], [1111], [1010], [0011], [1011], [1100], [1101], [1000], [1001], [0000], [0100], [0001], [0101],xticklabel style=rotate=90, legend pos=north east]
+[only marks, error bars/.cd,
y dir=both,y explicit]
(12, 0.0) +- (0.0, 0.0)
(10, 0.0) +- (0.0, 0.0)
(13, 2.27594) +- (6.14409, 6.14409)
(8, 1.25356) +- (2.7534, 2.7534)
(3, 16.9165) +- (8.24555, 8.24555)
(5, 11.8167) +- (6.12704, 6.12704)
(1,  17.4217) +- (8.29956, 8.29956)
(2,  12.4432) +- (5.78908, 5.78908)
(14,  0.0146092) +- (0.0308235, 0.0308235)
(11,  0.0228188) +- (0.0247545, 0.0247545)
(15,  2.39835) +- (6.39766, 6.39766)
(9,  1.0331) +- (2.86055, 2.86055)
(6,  16.179) +- (8.79654, 8.79654)
(7,  11.2363) +- (6.25733, 6.25733)
(0,  17.2781) +- (8.58085, 8.58085)
(4,  12.1857) +- (5.98063, 5.98063)
+[only marks, error bars/.cd,
y dir=both,y explicit]
(12, 0.0) +- (0.0, 0.0)
(10, 0.0) +- (0.0, 0.0)
(13, 2.73226) +- (7.29137, 7.29137)
(8, 0.910321) +- (2.35392, 2.35392)
(3, 17.2964) +- (8.30454, 8.30454)
(5, 11.8721) +- (6.10759, 6.10759)
(1,  17.5446) +- (8.24792, 8.24792)
(2,  12.6717) +- (5.75424, 5.75424)
(14,  0.0124633) +- (0.0392575, 0.0392575)
(11,  0.0236578) +- (0.0259562, 0.0259562)
(15,  4.03299) +- (8.75191, 8.75191)
(9,  0.98792) +- (2.80034, 2.80034)
(6,  16.0454) +- (8.62732, 8.62732)
(7,  11.2077) +- (6.21543, 6.21543)
(0,  16.9581) +- (8.4287, 8.4287)
(4,  12.1537) +- (5.95948, 5.95948)
X agent,Y agent

Relationship between different combinations of cooperation and the % of time the X and Y agents were communicating for matched agents in an environment containing three randomly placed barriers with random lengths and orientations.  The results are ordered by `goodness' from left to right.

Finally, the correlation between the mean % time spent communicating and mean solution times was consistently $\sim$0.4 for the best eight cooperation combinations.  This shows that harder barrier configurations required proportionally more inter-agent communications, as well as taking longer to solve.

§.§ Mismatched Agents

As an example of the consequences of allowing the cooperation level to be set independently for each agent,  <ref> shows results for all combinations of matched and mismatched agents in an environment containing three randomly placed barriers with random lengths and orientations.

xlabel=X Cooperation Level,
ylabel=Y Cooperation Level,
xtick=0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15,xticklabels=[0000], [1000], [0100], [1100], [0010], [1010], [0110], [1110], [0001], [1001], [0101], [1101], [0011], [1011], [0111], [1111],
xticklabel style=rotate=90,
ytick=0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15,yticklabels=[0000], [1000], [0100], [1100], [0010], [1010], [0110], [1110], [0001], [1001], [0101], [1101], [0011], [1011], [0111], [1111],
3 [surf,shader=interp] table
0	0	5.887962486
1	0	5.695290063
2	0	5.823678901
3	0	5.511129844
4	0	5.612941377
5	0	5.522842302
6	0	5.518770076
7	0	5.507709946
8	0	6.002405465
9	0	5.655295524
10	0	5.953840754
11	0	5.601136994
12	0	5.426198964
13	0	5.466234768
14	0	5.654508831
15	0	5.59994412

0	1	5.695290063
1	1	5.651531092
2	1	5.730440674
3	1	5.676719067
4	1	5.23097378
5	1	5.385242932
6	1	5.320253011
7	1	5.364181289
8	1	5.71275442
9	1	5.666356004
10	1	5.791671295
11	1	5.506719896
12	1	5.445904963
13	1	5.42783904
14	1	5.385050476
15	1	5.340575096

0	2	5.823678901
1	2	5.730440674
2	2	5.998772745
3	2	5.717893309
4	2	5.437866832
5	2	5.629428427
6	2	5.40276307
7	2	5.479570835
8	2	5.985472759
9	2	5.572336051
10	2	6.058524668
11	2	5.579498589
12	2	5.477032039
13	2	5.423796
14	2	5.452366842
15	2	5.371917053

0	3	5.511129844
1	3	5.676719067
2	3	5.717893309
3	3	5.295117858
4	3	5.022176264
5	3	5.274921638
6	3	4.937146068
7	3	5.146263762
8	3	5.698706748
9	3	5.533377905
10	3	5.708304898
11	3	5.334423274
12	3	5.223575494
13	3	5.2557929
14	3	5.180640918
15	3	5.100388438

0	4	5.612941377
1	4	5.676719067
2	4	5.437866832
3	4	5.022176264
4	4	5.062639607
5	4	4.876195844
6	4	4.800716333
7	4	4.878258804
8	4	5.334441412
9	4	5.177230252
10	4	5.510695506
11	4	5.149330512
12	4	4.889316144
13	4	5.039276113
14	4	4.889514806
15	4	4.675229039

0	5	5.522842302
1	5	5.385242932
2	5	5.629428427
3	5	5.274921638
4	5	4.876195844
5	5	4.901128799
6	5	5.016641044
7	5	4.918260634
8	5	5.539961796
9	5	5.390676725
10	5	5.416327989
11	5	5.082951715
12	5	5.019764686
13	5	5.045077015
14	5	5.015524715
15	5	4.92161005

0	6	5.518770076
1	6	5.320253011
2	6	5.40276307
3	6	4.937146068
4	6	4.800716333
5	6	5.016641044
6	6	4.878804982
7	6	4.809119428
8	6	5.44186458
9	6	5.472464864
10	6	5.350173547
11	6	4.895082476
12	6	4.790906053
13	6	4.948420223
14	6	4.766908334
15	6	4.8317497

0	7	5.507709946
1	7	5.364181289
2	7	5.479570835
3	7	5.146263762
4	7	4.878258804
5	7	4.918260634
6	7	4.809119428
7	7	4.978020792
8	7	5.499361701
9	7	5.349452063
10	7	5.607538488
11	7	5.03425766
12	7	4.900579545
13	7	5.072523133
14	7	5.011580388
15	7	4.859750913

0	8	6.002405465
1	8	5.71275442
2	8	5.985472759
3	8	5.698706748
4	8	5.334441412
5	8	5.539961796
6	8	5.44186458
7	8	5.499361701
8	8	5.980301889
9	8	5.481216579
10	8	5.928878983
11	8	5.59999525
12	8	5.424271419
13	8	5.478501225
14	8	5.569256194
15	8	5.488657488

0	9	5.655295524
1	9	5.666356004
2	9	5.572336051
3	9	5.533377905
4	9	5.177230252
5	9	5.390676725
6	9	5.472464864
7	9	5.349452063
8	9	5.481216579
9	9	5.81727091
10	9	5.653422047
11	9	5.862189866
12	9	5.292499243
13	9	5.496263347
14	9	5.384398983
15	9	5.41076013

0	10	5.953840754
1	10	5.791671295
2	10	6.058524668
3	10	5.708304898
4	10	5.510695506
5	10	5.416327989
6	10	5.350173547
7	10	5.607538488
8	10	5.928878983
9	10	5.653422047
10	10	5.914035297
11	10	5.627833139
12	10	5.589745858
13	10	5.408258087
14	10	5.558519045
15	10	5.653764917

0	11	5.601136994
1	11	5.506719896
2	11	5.579498589
3	11	5.334423274
4	11	5.149330512
5	11	5.082951715
6	11	4.895082476
7	11	5.03425766
8	11	5.59999525
9	11	5.862189866
10	11	5.627833139
11	11	5.535047554
12	11	5.16426124
13	11	5.305479948
14	11	5.130766376
15	11	5.209371991

0	12	5.426198964
1	12	5.445904963
2	12	5.477032039
3	12	5.223575494
4	12	4.889316144
5	12	5.019764686
6	12	4.790906053
7	12	4.900579545
8	12	5.424271419
9	12	5.292499243
10	12	5.589745858
11	12	5.16426124
12	12	4.964328309
13	12	5.035769341
14	12	4.873410055
15	12	4.901877318

0	13	5.466234768
1	13	5.42783904
2	13	5.423796
3	13	5.2557929
4	13	5.039276113
5	13	5.045077015
6	13	4.948420223
7	13	5.072523133
8	13	5.478501225
9	13	5.496263347
10	13	5.408258087
11	13	5.305479948
12	13	5.035769341
13	13	5.050266327
14	13	5.078722845
15	13	4.889560028

0	14	5.654508831
1	14	5.385050476
2	14	5.452366842
3	14	5.180640918
4	14	4.889514806
5	14	5.015524715
6	14	4.766908334
7	14	5.011580388
8	14	5.569256194
9	14	5.384398983
10	14	5.558519045
11	14	5.130766376
12	14	4.873410055
13	14	5.078722845
14	14	4.945472784
15	14	4.928725044

0	15	5.59994412
1	15	5.340575096
2	15	5.371917053
3	15	5.100388438
4	15	4.675229039
5	15	4.92161005
6	15	4.8317497
7	15	4.859750913
8	15	5.488657488
9	15	5.41076013
10	15	5.653764917
11	15	5.209371991
12	15	4.901877318
13	15	4.889560028
14	15	4.928725044
15	15	4.949594764

Heat map of the goodness measure for all combinations of X and Y cooperation for mismatched agents in an environment containing three randomly placed barriers with random lengths and orientations.  Blue corresponds to the best, and red to the worst.

The results of this experiment showed that up to 26% of barrier configurations were unsolvable and $\sim$30% were solvable without cooperation.  This meant that $\sim$44% were able to be solved with cooperation.

What is particularly interesting in the results shown in  <ref> is that the good solutions are not confined to the diagonal, i.e. not restricted to the matched agents conditions.  In fact the outcomes resulting from the best mismatched agents are comparable to those for the best matched agents.  For example, the best performance over all cooperation combinations was obtained for [1111]+[0010] (i.e. where one agent had all cooperation modes enabled, and the other agent was only responding to “stuck”), and this result was slightly better than the best matched agents at [0110]+[0110].

Further investigation into the consequences of allowing the cooperation level to be set independently for each agent was made by comparing the performance of the best matched and mismatched combinations mentioned above (i.e. [1111]+[0010] versus [0110]+[0110]) with varying numbers of barriers.  The results (shown in  <ref>) reveal that the matched agents performed slightly better than the mismatched agents.  However, as can be seen in  <ref>, the mismatched agents communicated less than the matched agents, with the difference being proportionally larger for the more difficult barrier configurations.

[xlabel=Number of Barriers,ylabel=Goodness Measure, GM, xtick=0, 1, 2, 3, xticklabels=0, 1, 2, 3, legend pos=north west]
(0, 3.12416)
(1, 3.64301)
(2, 4.24256)
(3, 4.86268)
(0, 3.12417)
(1, 3.77376)
(2, 4.39272)
(3, 4.89171)
x[0110]+[0110], x[1111]+[0010]

Relationship between the goodness measure and the number of random barriers for matched and mismatched agents.

[xlabel=Number of Barriers,ylabel=Proportion of Time Communicating (%), xtick=0, 1, 2, 3, xticklabels=0, 1, 2, 3, legend pos=north west]
(0, 0)
(1, 4.32329/2+4.29881/2)
(2, 11.5836/2+11.4675/2)
(3, 17.5992/2+17.5768/2)
(0, 0)
(1, 3.61589/2+4.23405/2)
(2, 8.82898/2+10.9701/2)
(3, 12.6775/2+16.9292/2)
x[0110]+[0110], x[1111]+[0010]

Relationship between the number of random barriers and the total % of time the X and Y agents were communicating for matched and mismatched agents.

§ DISCUSSION

Clearly, the task posed here is related to finding an optimal route on a map.  As such, two scenarios are possible: (i) the distance between the vehicle's current position and the target is known, but gradient descent may lead to a local minimum, or (ii) the distance between the vehicle's current position and the target is unknown due to an obstruction.  The first of these may be solved by planning (assuming that the map is known), or by recognising that a local minimum has occurred and trying to jump out stochastically.  In the second scenario, only random search is possible.

However, this map-based analysis is based on the privileged perspective of a 2D agent.  In the task posed in this paper, the agents were purposefully designed to be 1D, precisely so that they did not have access to a 2D map.  This meant that planning was not possible, and the recognition of arriving at a local minimum (or of simply not being able to see the target) required message-passing between the agents, i.e. explicit cooperation by communication.

Another insight to emerge from this work is the realisation that communication may be achieved by signalling (i.e. a `push' from a sending agent) or by observation (i.e. a `pull' from a receiving agent).  Clearly, the latter is less efficient due to the need for continuous monitoring.  Hence it can be said that, while an attention mechanism may be important, raising alerts in a timely manner are critical to success in a cooperative task.

It is also interesting to note that the overall paradigm is not specifically concerned with explicit message passing.  Given that the meanings of the particular messages have implications for the subsequent behaviour, the scenario may also be viewed as one agent needing to appreciate the other's situation.  In other words, timely communications to overcome local minima in a cooperative problem space may be viewed as instantiating a primitive `theory-of-mind'<cit.>.

§ SUMMARY AND CONCLUSION

This paper has addressed the question as to what conditions the timing and structure of communication in continuous cooperative interaction.  Experiments have been conducted using a PCT-based simulation of a cooperative task in which two independent one-dimensional agents are obliged to communicate in order to solve a two-dimensional path-finding problem.

Results from a number of simulation experiments have confirmed the hypothesis that appropriately timed communication between agents can overcome local minima in a joint problem space.  It has also been shown that asymmetric levels of cooperative communication can be as effective as equally matched partners, and can even reduce the level of communications required to achieve the same level of performance.

Finally, although this study was aimed at extrinsic communication between multiple agents, it is interesting to note that the results also apply to intrinsic communications within a single agent.


# Kinematic footprint of the Milky Way spiral arms in Gaia EDR3

Luis Martinez-Medina1, Angeles Pérez-Villegas2, Antonio Peimbert1
1Instituto de Astronomía, Universidad Nacional Autónoma de México, A. P.
70-264, Ciudad Universitaria CDMX 04510, México
2Instituto de Astronomía, Universidad Nacional Autónoma de México, Apdo.
Postal 106, 22800 Ensenada, B.C., México Contact
<EMAIL_ADDRESS>
(Released )

###### Abstract

The Milky Way spiral arms are well established from star counts as well as
from the locus of molecular clouds and other young objects, however, they have
only recently started to be studied from a kinematics point of view. Using the
kinematics of thin disc stars in Gaia EDR3 around the extended solar
neighbourhood, we create x-y projections coloured by the radial, residual
rotational, and vertical Galactocentric velocities ($U,\Delta V,W$). The maps
are rich in substructures and reveal the perturbed state of the Galactic disc.
We find that local differences between rotational velocity and circular
velocity, $\Delta V$, display at least five large-scale kinematic spirals; two
of them closely follow the locus of the Sagittarius-Carina and Perseus spiral
arms, with pitch angles of 9.12∘ and 7.76∘, and vertical thickness of $\sim
400$ pc and $\sim 600$ pc, respectively. Another kinematic spiral is located
behind the Perseus arm and appears as a distortion in rotation velocities left
by this massive arm but with no known counterpart in gas/stars overdensity. A
weaker signal close to the Sun’s position is present in our three velocity
maps, and appears to be associated with the Local arm. Our analysis of the
stellar velocities in the Galactic disc shows kinematic differences between
arms and inter-arms, that are in favour of Milky Way spiral arms that do not
corotate with the disc. Moreover, we show that the kinematic spirals are
clumpy and flocculent, revealing the underlying nature of the Milky Way spiral
arms.

###### keywords:

Galaxy: disc — Galaxy: kinematics and dynamics — Galaxy: structure — galaxies:
spiral — galaxies: kinematics and dynamics

††pubyear: 2021††pagerange: Kinematic footprint of the Milky Way spiral arms
in Gaia EDR3–References

##  1 Introduction

Spiral arms are among the most striking structures found in galaxies. More
than 60% of all massive galaxies are spirals (Buta, 1989; Willett et al.,
2013) and, with the help of studies of young stellar population and gas, it
was proved, in the 1950s, that the Milky Way (MW) is not an exception (e.g.
Oort & Muller, 1952; Morgan et al., 1953; van de Hulst et al., 1954). In the
current picture of our Galaxy it contains four major spiral arms (e.g.
Georgelin & Georgelin, 1976; Drimmel & Spergel, 2001; Vallée, 2014), as well
as a strong bar (e.g. Binney et al., 1997; Wegg et al., 2015; Bland-Hawthorn &
Gerhard, 2016).

According to the nature of their spiral arms, (Elmegreen & Elmegreen, 1984;
Elmegreen, 1990) spiral galaxies are classified into three types: i) grand
design, with a clearly defined structure and underlying stellar waves, ii)
flocculent, characterised by short and patchy spiral arm fragments, and iii)
multi-arm, an intermediate type, with a bisymmetric spiral pattern at the
centre, and several irregular arms in the outer parts.

Different types of spiral galaxies imply different formation mechanisms of
their spiral arms. While there is no consensus yet on the morphological nature
of the Galactic spiral arms, the traditional picture of four major arms as
well as some localised features (e.g. the Local arm) suggests that the the MW
spiral arms are quite complex with many authors favouring the scenario of
flocculent transient spiral structures (e.g. Quillen, 2002; Quillen et al.,
2018; Castro-Ginard et al., 2021).

Due to the position of the Sun near the Galactic plane, it has not been
possible to obtain the full picture of the structures in the MW disc:
interstellar dust and projection effects, as well as the multifaceted
characteristics (the makeup) of the arms themselves, interfere with the
determination of the spiral arms properties.

During the last two decades, many efforts have being dedicated to increase our
understanding of the structure, morphology, and dynamics of the MW spiral
arms; going from their location in the Galaxy (Reid et al., 2014; Hou & Han,
2014) to their dynamical influence on the Galactic disc (Hayden et al., 2015;
Antoja et al., 2018; Martinez-Medina et al., 2019). However, there is still
debate on the properties of the spiral arms, e.g., their pattern speed,
location, and even their nature (Grand et al., 2012; D’Onghia et al., 2013;
Roca-Fàbrega et al., 2013; de la Vega et al., 2015).

Regarding the locations of the Galactic spiral arms, they have been obtained
using a wide variety of techniques and tracers. For example: Morgan et al.
(1952); Morgan et al. (1953), using spectroscopic parallaxes of OB stars,
detected parts of a spiral arm located between the Sun and the Galactic
centre; young open clusters also reside inside spiral arms, and early works
measuring their galactic distribution (Becker, 1963; Becker & Fenkart, 1970;
Fenkart & Binggeli, 1979) confirmed their role as good tracers. Dias & Lépine
(2005) found how young open clusters must be to trace spiral arms; finding
that clusters with ages under $\sim$ 12 Myr are still located inside the
spiral arms. More recently, Castro-Ginard et al. (2021) detected overdensities
of open clusters younger than 30 Myr and traced the spiral arms with an
increased range in Galactic azimuth.

Georgelin & Georgelin (1976) were the first to propose the existence of four
major spiral arms in the MW based on their determinations of the positions of
H ii regions and exciting stars. The four-armed nature of the spiral arms has
been supported since early numerical works: Englmaier & Gerhard (1999) fitted
numerical simulations to the projection of gas dynamics on the CO $l-v$ map,
where the best model consists of four major spirals. Recently, these four
spiral arms (as well as the Local arm) have been traced and extended using
trigonometric parallaxes and proper motions of molecular masers associated to
young high-mass stars (Reid et al., 2014, 2019).

Taking advantage of the amount and quality of data available from Gaia, now it
is possible to detect fainter spiral structures than ever before. Pantaleoni
González et al. (2021) used Gaia and Alma data for OB stars and found a new
structure, which extends diagonally from the Local arm towards the Perseus
arm, that they dubbed the Cepheus spur.

Although the Galactic spiral structure has been characterised through the
overdensities and increased signals of its different tracers, they are also
observable in the kinematics of the stellar and gaseous disc components. It is
well known that, as they sweep the disc, spiral arms (along with the bar)
trigger anisotropies in the velocities of stars, as can be observed in the
solar neighbourhood (e.g Antoja et al., 2011; Hunt et al., 2018; Barros et
al., 2020). Mapping the kinematics of stars in barred spiral galaxy models,
Roca-Fàbrega et al. (2013) found that the velocity ellipsoid vertex deviation
follows the density peak of the spirals. Using high quality disc kinematics
and distances of red clump stars, (Eilers et al., 2020) found the response and
location of the Galactic spiral arms.

In this work, we used the full phase-space information based on radial
velocities and astrometry (parallaxes and proper motions) available in the
Gaia Early Data Release 3 (EDR3; Gaia Collaboration et al., 2021). We selected
a sample of 651,164 disc stars to compute the local variations from circular
rotation, $\Delta V(R,\phi)=V(R,\phi)-V_{c}(R)$, where $V$ is the rotation
velocity and $V_{c}$ the circular velocity. We find that a $x-y$ map coloured
by the local $\Delta V$ values displays several large-scale spiral structures.
Our analysis shows that some of these kinematic spirals correlate with two of
the four MW’s major spiral arms, while some others are clear inter-arm
regions.

This paper is organised as follows. In Sec. 2 we describe the data sample.
Then, in Sec. 3.1 we analyse the rotation velocity structure projected in the
Galactic plane, and find several large-scale spirals. Similarly, in Secs. 3.4
and 3.5 we show the kinematic structures found in radial and vertical
velocities, respectively. Finally, a discussion and conclusions of our results
are presented in Sec. 4.

##  2 Data Set

We use Gaia EDR3 that provides 6D phase-space coordinates of 7,209,831 stars,
i.e. positions in the sky $(\alpha,\delta)$, parallaxes $\varpi$, proper
motions $(\mu_{\alpha}^{*},\mu_{\delta})$ (Gaia Collaboration et al., 2021),
and radial line-of-sight velocities (Katz et al., 2019) that come from Gaia
DR2 (Gaia Collaboration et al., 2018) for stars with $G<$13 mag. EDR3
represents a significant improvement in both precision and accuracy of the
astrometry. (Gaia Collaboration et al., 2021).

For our sample we restrict ourselves to stars with positive parallax and
errors smaller than $20\%$. The distances have been calculated through the
inverse of the parallaxes after the zero-point correction (0.017 mas;
Lindegren et al., 2021).

In order to convert the positions, parallaxes, proper motions on the sky, and
radial velocities of the stars into Galactic phase-space positions and
velocities, we use the galpy python tools (Bovy, 2015), assuming that: the Sun
is located a $R_{\odot}=8.2$ kpc (Bland-Hawthorn & Gerhard, 2016; Gravity
Collaboration et al., 2019), a height above the Galactic plane
$Z_{\odot}=0.021$ kpc (Bennett & Bovy, 2019), the velocity of the local
standard of rest (LSR) is 236 km s-1 (Reid et al., 2019), and the peculiar
motion of the Sun with respect to the LSR of $(U_{\odot},V_{\odot},W_{\odot})$
= (11.1, 12.24, 7.25) km s-1 (Schönrich et al., 2010). Note: we used Cartesian
coordinates for the stellar positions, and polar coordinates for the stellar
velocities.

This sample also contains a dynamically hot stellar component that belongs to
the Galactic thick disc, unable to trace the mid-plane perturbations. To avoid
this contamination we decided to put stringent constraints in the vertical
position and velocities, trying to isolate the cold disc component.

To do that, we applied the following cuts: $|z|\leq 0.5$ kpc, $|U|\leq 15$
kms-1, and $|W|\leq 15$ kms-1. Finally, for the rotational velocity, $V$, it
is also necessary to avoid outliers, but those stars cannot be identified
automatically, as we expect to find structure in the rotation curve (RC); for
this we implemented a $1-\sigma$ clipping around the RC as the $V$ filter.

Regarding the remaining 2 spacial coordinates, we restricted ourselves to
stars with $-13\leq x\leq-5$ kpc and $-4\leq y\leq 4$ kpc, assuming that the
Sun’s position is (x,y)=(-8.2, 0) kpc. By choosing this set of filters we end
up with a subsample that contains 651,164 stars and belongs mainly to the thin
disc.

###  2.1 Rotation curve fit

Our goal is to characterise the non-circular motions in the stellar kinematics
of the extended solar neighbourhood. We start by computing the RC using the
sample described in the last section. To this purpose we divide the star
sample in 0.1 kpc radial bins and compute the median of the rotation
velocities from all stars in each bin. As shown in Fig. 1 (red stars), the
result is a collection of values for the median rotation velocity as a
function of the Galactocentric radius.

This RC is not smooth and oscillates. These oscillations should not
automatically be taken as true features of the RC, since Fig. 1 represents the
sum of the Galactic circular rotation with the non-circular motions produced
by the perturbations in the kinematics of stars. These type of features in the
RCs are often associated with an underlying perturbation of the disc, either
Galactic or extragalactic (Quillen et al., 2009; McGaugh, 2019; Martinez-
Medina et al., 2019), that in turn modifies the kinematics of gas and stars
(Shetty et al., 2007; Spekkens & Sellwood, 2007; Baba et al., 2016).

Figure 1: Median RC as function of the Galactocentric radius (red stars). The
blue and orange lines represent the best linear fit and the best fit to the
universal model, respectively

Of particular interest of the observed RC from Fig. 1 are: the two maxima, at
$\sim 6.7$ and $\sim 7.8$ kpc, and the three minima, at $\sim 7$, $\sim 9.5$,
and $\sim 10.7$ kpc. These features are clear deviations from circular motion,
and our goal is to quantify how large they are and how they look like in a 2D
velocity map, where the non-axisymmetries in velocity will be displayed as a
function of Galactocentric radius and azimuth.

To obtain a better contrast of the perturbations in the kinematics we need to
subtract the axisymmetric background from the stellar rotation velocities; for
this we first obtain a circular velocity curve using the RC of Fig. 1 (red
stars).

We have assumed that the true RC is smooth, and that most of the contribution
to the observed oscillations comes from non-circular motions; if that
hypothesis is correct, we expect the locii of the maxima and minima to have a
significant dependence on the angle (as will be shown in the following
sections).

We fit two simple analytical models: a linear model described by
$V(R)=V_{0}+\frac{dV}{dR}(R-R_{0})$, and a the universal RC as introduced by
(Persic et al., 1996), which represents the RC produced by a stellar disc and
a spherical dark matter halo. The universal RC is given by:

$\displaystyle V(x)$ $\displaystyle=$
$\displaystyle\sqrt{V_{d}^{2}(x)+V_{h}^{2}(x)}$ (1) $\displaystyle
V_{d}^{2}(x)$ $\displaystyle=$ $\displaystyle
a_{1}^{2}b\frac{1.97x^{1.22}}{(x^{2}+0.78^{2})^{1.43}}$ (2) $\displaystyle
V_{h}^{2}(x)$ $\displaystyle=$ $\displaystyle
a_{1}^{2}(1-b)x^{2}\frac{1+a^{2}_{3}}{x^{2}+a_{3}^{2}},$ (3)

where $x=R/R_{\rm opt}=(R/R_{0})/a_{2}$, $a_{1}$ is the rotation speed at the
galaxy optical radius $R_{\rm opt}$, $a_{2}=R_{\rm opt}/R$, and $a_{3}$ is the
velocity core radius. We adopted $b=0.72$ (Persic et al., 1996).

In order to completely explore the parameter space, we applied the Bayes’
inference through Markov-chain Monte Carlo (MCMC) using the python package
emcee (Foreman-Mackey et al., 2013); with this method we obtain the posterior
probability distributions. For the linear model fit, we used a uniform prior
to restrict the signal of $V_{0}$ and $\frac{dV}{dR}$. For the universal curve
model, we employed Gaussian prior distributions for the $a_{1}$, $a_{2}$, and
$a_{3}$ parameters. To represent the final distributions, we adopted the
median as the most probable value and the $16^{\rm th}$ and $84^{\rm th}$
percentiles as 1-$\sigma$ uncertainties. The best fit to the linear model has
$V_{0}=228.15^{+1.49}_{-1.52}$ km s-1 and
$\frac{dV}{dR}=-1.90^{+0.75}_{-0.77}$ km s1 kpc-1 (blue line in Fig. 1); while
the best fit to the universal model has $a_{1}=230.68^{+2.61}_{-2.12}$ km s-1,
$a_{2}=0.82^{+0.10}_{-0.11}$, and $a_{3}=1.30^{+0.10}_{-0.11}$ (orange line in
Fig. 1).

Starting at $\sim 7.2$ kpc the two fitting curves are very similar and they
are good in describing the overall trend in the outer RC, suggesting either
curve could be a good approximation of the “true” global RC. This comparison
shows that, although the universal RC model by Persic et al. (1996) is
realistic enough, it is not much more complicated than a simple linear fit.
Overall, we prefer the universal curve because it seems to remain a good
approximation all the way up to $\sim 6.5$ kpc. For our analysis, we adopt the
universal curve to represent the underlying circular velocity in our data.

##  3 Mapping the spiral arms in the stellar kinematics

When using the term “velocity structure” it is sometimes hard to remember that
this is not a one dimensional structure; even when we restrict ourselves to
the plane, the velocity structure remains a 2-dimensional affair. While using
theoretical models, it is possible to construct a galaxy where we can
perfectly separate the axisymmetric background from the non-axisymmetric
components. Observationally, it is not possible to do that; unfortunately, the
rotational velocity signature created by the spiral pattern (or any other non-
axisymmertic perturbation) is locally blended with the RC of the axisymmetric
components. For the MW, it would only be possible to properly determine the
axisymmetric background (and thus isolate the non-axisymmetric component), if
we had quality data from the entire Galaxy, or at least of a slab comprising
180 degrees of the Galactic disc; but, when we are restricted to about 3 kpc
of quality data, we only have at our disposal information of $\sim 45^{\circ}$
of the Galactic disc, which makes it easy to confuse radial axisymmetric
structures on the RC with the signal produced by the arms (or other non-
axisymmetric component).

If we knew the true circular RC, the difference between the local velocities
and the circular RC will be the signal of non-axisymmetric structures.
Currently, we do not possess a RC with that quality, but any good
approximation (such as the universal RC from sec. 2.1) would help us to see
those structures.

To further explore the features in the RC, now we account for the azimuthal
angle. For this purpose, we denote the radial, rotational, and vertical
components of the velocity as $(U,V,W)$, that are positive in the direction of
the Galactic centre, Galactic rotation, and north Galactic pole, respectively.
Then, we project our stellar sample on the x-y plane and draw a mesh to divide
this plane into 0.2 kpc-size cells.

###  3.1 Rotational velocities in the EDR3

Although we have three-dimensional information for each star, first we focus
on $V$. Furthermore, since our goal is to study the non-axisymmetric features
of the rotational velocity, we will use the residual rotational velocity to
highlight their signal. For this purpose, in each cell, we will determine the
value of $\Delta V(R,\phi)=V(R,\phi)-V_{c}(R)$, where $V$ is the median of the
rotation velocity of the stars inside the cell, and $V_{c}$ the circular
velocity (equation 1) evaluated at the radial position of the cell. This means
that at each cell we show a measure of how the rotation in the Galactic plane
deviates from the circular rotation.

The left panel of Figure 2 shows the star sample in the x-y plane and the
cells coloured with the local value of $\Delta V$. Notice that positive values
of $\Delta V$ correspond to regions of the disc where the rotation is faster
than the axisymmetric background, while negative values of $\Delta V$
correspond to regions where the rotation is slower. The first result we find
is that the kinematics of the disc is full of substructure; in particular
several spiral structures are evident.

Since in theory and observations spiral arms tend to behave like logarithmic
spirals, it is common to transform the x-y plane to a angle-log plane. On the
right panel of Fig. 2 we plot the same star sample but now in the $\phi-ln(R)$
plane. We notice that here the kinematic spirals appear roughly as straight
lines (in this plane a logarithmic spiral will appear as straight line with
slope equal to the tangent of the pitch angle of the spiral).

Figure 2: Spiral footprint in stellar $\Delta V$ values projected on the
Galactic plane. Five spiral structures are labelled from $\alpha$ to
$\epsilon$,

We defined two structures ($\alpha$ and $\gamma$) by selecting the “fast” bins
(those with $\Delta V>0$) and two structures ($\beta$ and $\delta+\epsilon$)
by selecting “slow” bins (those with $\Delta V<0$). However, the perpendicular
profile of the outermost structure shows that it is a double-peaked feature,
which we decided to split into two structures, $\delta$ and $\epsilon$. There
is an additional red feature inside $R\approx 6$ kpc; this saturated region is
likely because the relatively simple structure of the universal RC, which is
unable to represent the rich structure of the inner MW disc (near the end of
the bar); whatever the specific reason, such discrepancy is beyond the scope
of this paper. There is also an outer blue region; but, as it is noisy and
boundless, we are not able to characterise the structure in that area.

Because traditional models expect the slow material to accumulate in the arms
and the fast material to define the inter-arms; the three red extended
structures ($\beta$, $\delta$, $\epsilon$) behave like spiral arms, where the
rotation is slower than the local circular velocity; while in blue, with fast
rotation, we find two inter-arm-like structures, one marked as $\alpha$, and
another that crosses through the Sun’s position, marked as $\gamma$.

The red spirals display detailed features that show a wide variety of
behaviours: while spiral $\delta$ is a single large fragment, potentially part
of a larger structure, spiral $\epsilon$ is composed by two main fragments
separated by a prominent gap around (-10.2,1.0) kpc, and spiral $\beta$ is
fragmented and clumpy, with several small gaps along its extension. Overall,
all three spirals show evidence of flocculence.

###  3.2 Characterisation of the kinematic spiral structures

To estimate the parameters that define the kinematic spirals detected in the
$x-y$ projection of Fig. 2, we fitted analytical spirals to these structures,
adopting a log-periodic spiral defined by

$\ln(R/R_{\rm ref})=-\phi\,\tan\psi,$ (4)

where $R$ is the Galactocentric radius at a Galactocentric azimuth $\phi$
(defined as 0 towards the Sun and increasing in the direction of the Galactic
rotation) for a spiral with a radius $R_{\rm ref}$ at reference azimuth zero,
and pitch angle $\psi$.

To isolate each $\Delta V$ kinematic spiral in the $x-y$ projection, we
selected individually each structure as delimited by the change of sign in
$\Delta V$; as for the frontier between $\delta$ and $\epsilon$, we used the
ridge with the highest velocity. We fitted the equation 4 using a non-linear
least squares method to estimate the parameters $R_{\rm ref}$, and $\psi$. The
values of the spiral fits are given in Table 1, and the resulting spiral
curves are shown in Fig. 3 as solid colour lines (green for structure
$\alpha$, magenta for $\beta$, orange for $\gamma$, grey for $\delta$, black
for $\epsilon$).

Table 1: Spiral kinematics fits

| $R_{\rm ref}$ | $\psi$ |  $\phi$ range |
---|---|---|---|---
Spiral | (kpc) | (deg) | (deg) | Type
$\alpha$ | $6.48$ | $9.34$ | ($-32.6,\;29.6$) | Inter-arm
$\beta$ | $6.91$ | $9.12$ | ($-29.6,\;34.5$) | Arm 1
$\gamma$ | $8.13$ | $5.45$ | ($-21.9,\;24.3$) | Inter-arm
$\delta$ | $9.31$ | $7.82$ | ($-24.6,\;24.4$) | Arm 2
$\epsilon$ | $10.34$ | $7.76$ | ($-14.9,\;24.1$) | Arm 3

1 Consistent with the Sagittarius-Carina arm from Reid et al. (2014).
2 Arm-like structure without observational overdensity counterpart.
3 Consistent with the Perseus arm from Reid et al. (2014).

We can now compare these Gaia kinematic structures, presented in Fig. 2, with
the MW spiral arms traced by young objects, presented by Reid et al. (2014).
In Fig. 3 we can see that the $\beta$ and $\epsilon$ structures, described by
the magenta and black spirals, are in good agreement with the Sagittarius-
Carina and Perseus arms, respectively.

Figure 3: Logarithmic spirals used to describe the structures in $\Delta V$
(colour solid lines). The dotted lines are the spiral arm models from Reid et
al. (2014) (from right to left: Scutum-Centaurus, Sagittarius-Carina, Local,
Perseus, and Outer arm).

###  3.3 Galactic spiral arms in 3D

By computing the contrast $\Delta V$ we found the regions of the Galactic
plane where the rotation deviates from circular motion. Interestingly, these
regions are arranged in spiral structures in the $x-y$ plane. Moreover, the
full 3D information of the sample allows us to explore $x-z$ and $y-z$ slices
to have the complete picture of the kinematic spiral arms.

To obtain the other two $\Delta V$ projections of our sample, we divided the
$x-y$ plane into rows (labelled with numbers) and columns (labelled with
letters), where each row and column have a width of 0.8 kpc.

Figure 4: Three-dimensional view of the residual rotational velocity $\Delta
V$. Vertical panels are y-z projections from stars inside each of the vertical
strips labelled from A to H in the central plot. Similarly, horizontal panels
are x-z projections from the strips numbered from 1 to 8 in the central plot.

For the columns, we defined regions limited by a minimum and maximum value of
x, while $-4\leq y\leq 4$ and $-1\leq z\leq 1$; this increased z range,
compared with sections 2 and 3.1, allows us to see the structure beyond the
thin disc. Next, all stars inside each column are projected on the $y-z$
plane, and from here we apply the same procedure explained in sec. 3.1, i.e.,
divide this projection into cells coloured by the local value of $\Delta V$.
The resulting projections are shown in the vertical subplots of Fig. 4, and
labelled from A to H.

For the rows, we delimit the maximum and minimum values of y, while $-13\leq
x\leq-5$ and $-1\leq z\leq 1$. Then, all stars inside each of these horizontal
regions are projected on the x-z plane. The resulting projections are shown in
the horizontal subplots of Fig. 4, and numbered from 1 to 8.

The $x-y$ grid and resulting slices are shown in Fig. 4, which is a set of
cross sections of the disc rotational velocity distribution, that provides a
full three-dimensional picture of the kinematic spiral arms. These projections
allow us to observe the extent of the spirals above and below the plane.

Looking at the kinematic trace of the Perseus arm, present in panel C, we find
that its vertical thickness amounts to $\sim 600$ pc (full height). While the
Sagittarius-Carina arm, present in panel G, shows this structure to be $\sim
400$ pc (full height); the saturated features found at a height of $\sim 800$
pc are not part of the arm, but rather a faint protuberance of the inner slow
moving feature, as can be seen in slices 2-7. This is the first determination
of the thickness of the kinematic spiral arms; it has only been possible with
the quality of the EDR3 Gaia data.

Also notice that panels 4 and 5 show the perpendicular projection $x-z$ that
captures the thickness of Perseus and Sagittarius-Carina at the same time.
Here, we notice that the kinematic counterpart of the Perseus arm is roughly
symmetric around the $z=0$ plane; while that of Sagittarius-Carina arm is not,
it is clear that its amplitude in $\Delta V$ is larger below the Galactic
plane.

###  3.4 Radial Velocity

In this section we look for kinematic structures traced by the radial
velocities, $U$, of our stellar sample. Once again, we project the stars in
our sample on the x-y plane which we divide into cells. Then, each cell is
coloured by the median value of $U$ from all stars in the cell.

Fig. 5, left panel, shows the distribution of radial velocities projected on
the Galactic mid-plane. We see that this plane is also rich in substructures,
although they are not the same as the ones found with $\Delta V$, nor resemble
clear spiral features. In the right panel ($\phi-ln(R)$ plane) it can be seen
that these structures resemble triangles which do not produce clear bands;
therefore we are reluctant to make any fit to the features of this velocity
component.

Figure 5: Kinematic structure in the stellar $U$ velocity projected on the
Galactic plane. For comparison we include the logarithmic spirals (dash-dotted
lines) used to fit the $\Delta V$ structures (Fig. 3). The dotted line is the
local arm from Reid et al. (2014).

To analyse whether the structures in $U$ have a connection with the Galactic
spiral arms, in Fig. 5 we overplot the logarithmic spirals used to fit the
$\Delta V$ structures (Fig. 3). In general, the structures in $U$ cannot be
entirely described with the logarithmic spirals used for the $\Delta V$
structures. However, we notice some similarities: i) the upper half of the
black curve, that matches well with the Perseus arm, resembles the mid region
of the large red structure in the upper part of the plot; ii) the lower
segment of the magenta curve, that follows the Sagittarius-Carina arm, crosses
through a red $U$ structure in the bottom right of the plot.; iii) the grey
spiral has no clear relation with any red or blue structure, residing near the
frontier between blue and red, but changing sides near the middle of our plot;
iv) as for the green and orange spirals, that in $\Delta V$ trace the central
part of the inter-arms, in Fig. 5 they are closer to interfaces where $U$
changes from positive to negative values.

Nonetheless, in Fig. 5 we found an extra structure; close to the Sun’s
position is a red ($U>0$) diagonal spiral fragment that follows the location
of the Local arm from Reid et al. (2014), plotted here with a dotted line.
When looking at the right panel, it is apparent that this structure ends at
phi=0.25, however it is not clear whether the beginning lies close to
$\phi=-0.05$ or it extends from $\phi<-0.4$.

Recently, Eilers et al. (2020) found a spiral signature using the mean
Galactocentric radial velocities of red giant stars in the mid-plane. By
comparing with a non-axisymmetric potential model, they identified the
interface between positive (on the inside) and negative (on the outside)
velocity values as the location of spiral arms density peaks. We cannot find
such clear spiral signature in $U$; however, our analysis is centred on a
smaller region of the Galactic disc.

###  3.5 Vertical Velocity

Across an axisymmetric stellar disc the mean vertical velocity would be zero.
However, in the presence of additional vertically symmetric components (such
as idealised arms or bars), some local deviations are expected to happen,
these deviations would also be symmetrical across the plane, and would
theoretically be zero when projected on the plane; however, the presence of
dust clouds above or below the plane can produce a bias, and thus some signal
could be expected. In a real galaxy, additional perturbations (both external
and internal) are expected. Further discussion on the perturbations in the
vertical velocities of the disc, either in the stellar or gas component, can
be found in the literature (Weinberg, 1991; Faure et al., 2014; Alves et al.,
2020; López-Corredoira et al., 2020). In this section we analyse the vertical
velocity W of our sample.

Fig 6 shows the x-y plane coloured by the local $W$ values; there we find
extended regions with non zero mean vertical velocity. Of particular interest
is the large vertical red region (negative W values) in the middle of the
graph, with a mean vertical velocity of $W\approx-2$ km/s. It looks similar to
the spiral $\delta$ found in $\Delta V$ at the same position, just behind of
the Perseus arm. We confirm this correlation by overplotting the grey spiral
obtained in Fig. 3. This result is not unusual; Eilers et al. (2020), using
red giant stars from APOGEE DR14, found structures in the vertical velocity
map resembling those found in the radial velocity map.

Although the nature of the perturber causing these vertical motions is not
clear, some inferences can be made: i) This feature seems to be related to the
$\delta$ spiral structure found in section 3.1. ii) Its wide area makes it
obvious that it must come from outside the plane, and it cannot be the effect
of molecular clouds obscuring one side of the plane. iii) A plausible scenario
is that it originates from an extragalactic perturbation (perhaps a dwarf
galaxy crossing the Galactic plane). iv) The length of the perturbation
suggests that such impact is not recent. v) Moreover, its observed shape and
pitch angle are very similar to those of Perseus arm; on top of that, this
feature lags behind Perseus arm, suggesting a connection between this
structure with both, Perseus arm and the Galactic rotation.

There is an additional faint blue structure with a Galactocentric distance of
$R\sim$8.3 kpc and velocity $W\approx 1$ km s-1. It seems to be connected to
the local arm; this perturbation could also be related to the Radcliffe gas
wave recently observed by Alves et al. (2020). Whether the original
perturbation is somehow related to the formation mechanism of the local arm,
or it is an independent perturbation propagated by the local arm, is beyond
the scope of this paper.

Figure 6: Footprint in the $W$ component of the stellar velocity projected on
the Galactic plane. The dash-dotted line is the spiral curve used to describe
the $\delta$ structure in $\Delta V$ (Fig. 3), included here for comparison.
The dotted line is the local arm from Reid et al. (2014).

##  4 Discussion and Conclusions

We have analysed the three velocity components of thin disc stars from Gaia
EDR3. The projections on the mid-plane coloured by each of the three stellar
velocity components are rich in substructures; of particular interest are the
local deviations from circular rotation, that reveal detailed large-scale
kinematic spiral structures. All this work has been possible due to the large
breath and exquisite quality available from the Gaia mission.

A big advantage of our approach is that the inherent biases in star counts due
to Galactic extinction do not interfere with our determinations, this is
because we are not measuring star number densities (at most these issues in
star counts would decrease our s/n but without introducing any bias). The
signal we find depends on the response of hundreds of thousands of stars (both
young and old) to the local mass distribution of the different Galactic
components; unlike the signal that can be detected using molecular clouds, H
ii regions, or other young objects, that often depends on small number
statistics.

Based on stellar dynamics, stars in spiral arms rotate slower than the local
circular velocity, while stars in the inter-arms rotate faster. We studied the
residual rotation velocity, $\Delta V$, and found five structures: three of
our structures are slow (red) and we associated them with spiral arms (named
$\beta$, $\delta$, and $\epsilon$); while the other two are fast (blue) and we
associated them with inter-arms (named $\alpha$ and $\gamma$).

Comparing our structures with those by Reid et al. (2014), we found that the
location of the $\beta$ and $\epsilon$ spirals are in good agreement with the
Sagittarius-Carina arm (dominated by gas) and the Perseus arm (more massive,
with a backbone made of stars), respectively. Our best fits with logarithmic
spirals give pitch angles of 9.12∘ and 7.76∘ for our Sagittarius-Carina and
Perseus counterparts, respectively. The kinematic differences between the arms
and inter-arms are in favour of MW spiral arms that do not corotate with the
disc.

Since we are measuring the locus of the kinematic signal instead of the
spatial location of stellar density peaks, these two signals are not
necessarily expected to coincide, as the relation between density and
kinematics is not straight forward. Interestingly, for two of our kinematic
arms, we do find location and pitch angle coincidence with previous
determinations (Reid et al., 2014), the deviations from their known
overdensity counterparts are minimal, consistent within $1-\sigma$
uncertainties.

Besides $\beta$ and $\epsilon$, the last kinematic spiral arm, $\delta$, does
not appear to have an overdensity counterpart, neither in stars nor in gas.
Its pitch angle and location suggests it represents an out of phase response
to Perseus arm ($\epsilon$).

It is worth to mention that each kinematic spiral found in Fig. 2 can be
described by a logarithmic spiral (Fig. 3). However, these structures are not
continuous, they have several gaps, and are made up by clumps, revealing the
flocculent nature of the Galactic spiral arms.

The last action with the $\Delta V$ component was to generate x-z and y-z
maps. The result is a set of vertical slices of the disc (Fig. 4), that reveal
the thickness of the spiral arms. We found that the kinematic counterpart of
Perseus arm has a vertical thickness of $\sim$600 pc, while that of
Sagittarius-Carina arm has a thickness of $\sim$400 pc.

We have also studied the $U$ velocity. We presented a similar x-y map, but
coloured with the local median values of the radial velocity $U$ (Fig. 5). The
signal is quite different between $U$ and $\Delta V$: since we do not find
spiral bands equivalent to either arms or inter-arms, instead finding
elongated triangular features that cannot be identified as spirals.

Finally, regarding the vertical velocity $W$, we also analyse this velocity in
the x-y projection (Fig.6).The simplest models to study the vertical
velocities, $W$, would expect this map to be nothing more than noise. However,
there is some structure to be found in this map. Any such structure would not
be expected on the Galactic disc if it were isolated from external forces.

We find two structures in Fig. 6: the clearest one is a red arch at $R\sim
9.5$ kpc akin to the $\delta$ structure; the second one is a faint blue arch
at $R\sim 8.2$ kpc akin to the local arm, but only present ahead of the Sun
(i.e. y$>0$).

The brightest structure found in $W$ is consistent with the the spiral arm
$\delta$ found in Fig. 2; the presence and shape of this structure in the
vertical velocity component suggest a possible scenario of a dwarf galaxy
crossing the plane in the not-too-recent past. When including all three
velocity components, this feature presents significant signal in both $\Delta
V$ and $W$, as well as a faint signal in $U$; even if there is no known
significant signature of an overdensity in any type of objects, the $\delta$
structure is a complex kinematic perturbation that resides within the Galactic
disc.

The local arm structure seems to be present in two of the velocity components,
$U$ and $W$; but $\Delta V$ seems to be less sensitive to the local arm as it
is to either the Sagittarius-Carina and Perseus arms, which are clearly traced
in Fig. 3. This is likely due to the nature of the Local arm, which since
early studies has been considered a fragment or spur (van de Hulst et al.,
1954; Georgelin & Georgelin, 1976). Its density contrast, either in gas or
stars, is not as large as in any of the four major arms; evidence of this can
be seen in the RC of Fig. 1; while Sagittarius-Carina imprints a local minimum
in the RC at $\sim$7 kpc, and Perseus does the same at $\sim$10.7 kpc, the
Local arm is not massive enough to modify the RC in such a clear way. However,
the components $U$ and $W$ of the velocity appear to be sensitive enough to
trace the Local arm. Figures 5 and 6 reveal a diagonal structure close to the
Sun’s position, that falls under the overplotted Local arm (dotted line) as
measured by Reid et al. (2014).

Overall, our approach has allowed us to uncover six kinematic structures,
three of which represent the first time the kinematic counterparts of Perseus,
Sagittarius-Carina, and the local arm are observed; in particular, using
kinematics, this is the first time the vertical heights of Perseus and
Sagittarius-Carina are measured. The other three represent structures not
studied before: the kinematic inter-arms we call $\alpha$ and $\gamma$, but
most notably $\delta$, which represents a new kind of kinematic arm not
observed before. All this work was only possible thanks to the high-quality
data from Gaia EDR3. We are convinced that with future updates from the Gaia
Collaboration all the structures found here should improve and more of them
will be detected.

## Acknowledgements

The authors acknowledge support from DGAPA-PAPIIT IA101520 and IG100319
grants. We acknowledge DGTIC-UNAM for providing HPC resources on the Cluster
Supercomputer Miztli.

## Data availability

The data that support the findings of this study are available from the
corresponding author, upon reasonable request.

## References

  * Alves et al. (2020) Alves J., et al., 2020, Nature, 578, 237
  * Antoja et al. (2011) Antoja T., Figueras F., Romero-Gómez M., Pichardo B., Valenzuela O., Moreno E., 2011, MNRAS, 418, 1423
  * Antoja et al. (2018) Antoja T., et al., 2018, Nature, 561, 360
  * Baba et al. (2016) Baba J., Morokuma-Matsui K., Miyamoto Y., Egusa F., Kuno N., 2016, MNRAS, 460, 2472
  * Barros et al. (2020) Barros D. A., Pérez-Villegas A., Lépine J. R. D., Michtchenko T. A., Vieira R. S. S., 2020, ApJ, 888, 75
  * Becker (1963) Becker W., 1963, Z. Astrophys., 57, 117
  * Becker & Fenkart (1970) Becker W., Fenkart R. B., 1970, in Becker W., Kontopoulos G. I., eds, Vol. 38, The Spiral Structure of our Galaxy. p. 205
  * Bennett & Bovy (2019) Bennett M., Bovy J., 2019, MNRAS, 482, 1417
  * Binney et al. (1997) Binney J., Gerhard O., Spergel D., 1997, MNRAS, 288, 365
  * Bland-Hawthorn & Gerhard (2016) Bland-Hawthorn J., Gerhard O., 2016, ARA&A, 54, 529
  * Bovy (2015) Bovy J., 2015, ApJS, 216, 29
  * Buta (1989) Buta R., 1989, Galaxy Morphology. p. 151
  * Castro-Ginard et al. (2021) Castro-Ginard A., et al., 2021, arXiv e-prints, p. arXiv:2105.04590
  * D’Onghia et al. (2013) D’Onghia E., Vogelsberger M., Hernquist L., 2013, ApJ, 766, 34
  * Dias & Lépine (2005) Dias W. S., Lépine J. R. D., 2005, ApJ, 629, 825
  * Drimmel & Spergel (2001) Drimmel R., Spergel D. N., 2001, ApJ, 556, 181
  * Eilers et al. (2020) Eilers A.-C., Hogg D. W., Rix H.-W., Frankel N., Hunt J. A. S., Fouvry J.-B., Buck T., 2020, ApJ, 900, 186
  * Elmegreen (1990) Elmegreen B. G., 1990, Annals of the New York Academy of Sciences, 596, 40
  * Elmegreen & Elmegreen (1984) Elmegreen D. M., Elmegreen B. G., 1984, ApJS, 54, 127
  * Englmaier & Gerhard (1999) Englmaier P., Gerhard O., 1999, MNRAS, 304, 512
  * Faure et al. (2014) Faure C., Siebert A., Famaey B., 2014, MNRAS, 440, 2564
  * Fenkart & Binggeli (1979) Fenkart R. P., Binggeli B., 1979, A&AS, 35, 271
  * Foreman-Mackey et al. (2013) Foreman-Mackey D., Hogg D. W., Lang D., Goodman J., 2013, PASP, 125, 306
  * Gaia Collaboration et al. (2018) Gaia Collaboration et al., 2018, A&A, 616, A1
  * Gaia Collaboration et al. (2021) Gaia Collaboration et al., 2021, A&A, 649, A1
  * Georgelin & Georgelin (1976) Georgelin Y. M., Georgelin Y. P., 1976, A&A, 49, 57
  * Grand et al. (2012) Grand R. J. J., Kawata D., Cropper M., 2012, MNRAS, 421, 1529
  * Gravity Collaboration et al. (2019) Gravity Collaboration et al., 2019, A&A, 625, L10
  * Hayden et al. (2015) Hayden M. R., et al., 2015, ApJ, 808, 132
  * Hou & Han (2014) Hou L. G., Han J. L., 2014, A&A, 569, A125
  * Hunt et al. (2018) Hunt J. A. S., Hong J., Bovy J., Kawata D., Grand R. J. J., 2018, MNRAS, 481, 3794
  * Katz et al. (2019) Katz D., et al., 2019, A&A, 622, A205
  * Lindegren et al. (2021) Lindegren L., et al., 2021, A&A, 649, A4
  * López-Corredoira et al. (2020) López-Corredoira M., Garzón F., Wang H. F., Sylos Labini F., Nagy R., Chrobáková Ž., Chang J., Villarroel B., 2020, A&A, 634, A66
  * Martinez-Medina et al. (2019) Martinez-Medina L., Pichardo B., Peimbert A., Valenzuela O., 2019, MNRAS, 485, L104
  * McGaugh (2019) McGaugh S. S., 2019, ApJ, 885, 87
  * Morgan et al. (1952) Morgan W. W., Sharpless S., Osterbrock D., 1952, AJ, 57, 3
  * Morgan et al. (1953) Morgan W. W., Whitford A. E., Code A. D., 1953, ApJ, 118, 318
  * Oort & Muller (1952) Oort J. H., Muller C. A., 1952, Monthly Notes of the Astronomical Society of South Africa, 11, 65
  * Pantaleoni González et al. (2021) Pantaleoni González M., Maíz Apellániz J., Barbá R. H., Reed B. C., 2021, MNRAS, 504, 2968
  * Persic et al. (1996) Persic M., Salucci P., Stel F., 1996, MNRAS, 281, 27
  * Quillen (2002) Quillen A. C., 2002, AJ, 124, 924
  * Quillen et al. (2009) Quillen A. C., Minchev I., Bland-Hawthorn J., Haywood M., 2009, MNRAS, 397, 1599
  * Quillen et al. (2018) Quillen A. C., et al., 2018, MNRAS, 480, 3132
  * Reid et al. (2014) Reid M. J., et al., 2014, ApJ, 783, 130
  * Reid et al. (2019) Reid M. J., et al., 2019, ApJ, 885, 131
  * Roca-Fàbrega et al. (2013) Roca-Fàbrega S., Valenzuela O., Figueras F., Romero-Gómez M., Velázquez H., Antoja T., Pichardo B., 2013, MNRAS, 432, 2878
  * Schönrich et al. (2010) Schönrich R., Binney J., Dehnen W., 2010, MNRAS, 403, 1829
  * Shetty et al. (2007) Shetty R., Vogel S. N., Ostriker E. C., Teuben P. J., 2007, ApJ, 665, 1138
  * Spekkens & Sellwood (2007) Spekkens K., Sellwood J. A., 2007, ApJ, 664, 204
  * Vallée (2014) Vallée J. P., 2014, AJ, 148, 5
  * Wegg et al. (2015) Wegg C., Gerhard O., Portail M., 2015, MNRAS, 450, 4050
  * Weinberg (1991) Weinberg M. D., 1991, ApJ, 373, 391
  * Willett et al. (2013) Willett K. W., et al., 2013, MNRAS, 435, 2835
  * de la Vega et al. (2015) de la Vega A., Quillen A. C., Carlin J. L., Chakrabarti S., D’Onghia E., 2015, MNRAS, 454, 933
  * van de Hulst et al. (1954) van de Hulst H. C., Muller C. A., Oort J. H., 1954, Bull. Astron. Inst. Netherlands, 12, 117