# Approximating Permutations with Neural Network Components for Travelling
Photographer Problem

Chong Sue Sin  Institute for Interdisciplinary Information Sciences (IIIS)
Tsinghua University
Beijing, China
Email<EMAIL_ADDRESS>
###### Abstract

Most of the current inference techniques rely upon Bayesian inference on
Probabilistic Graphical Models of observations and do predictions and
classification on observations. However, there is very little literature on
the mining of relationships between observations and building models of the
relationship between sets of observations or the generating context of the
observations. Moreover, event understanding of machines with observation
inputs needs to understand the relationship between observations. Thus there
is a crucial need to build models and develop effective data structures to
accumulate and organize relationships between observations. Given a PGM model,
this paper attempts to fit a permutation of states to a sequence of
observation tokens (The Travelling Photographer Problem). We have devised a
machine learning inspired architecture for randomized approximation of state
permutation, facilitating parallelization of heuristic search of permutations.
As a result, our algorithm can solve The Travelling Photographer Problem with
minimal error. Furthermore, we demonstrate that by mimicking machine learning
components such as normalization, dropout, and lambda layer with a randomized
algorithm, we can devise an architecture that solves TPP, a permutation NP-
Hard problem. Other than TPP, we can also provide a 2-Local improvement
heuristic for the Travelling Salesman Problem (TSP) with similar ideas.

###### Index Terms:

attention mechanism, simulation, approximation algorithm, randomized
algorithm, permutations, problem-driven architecture

##  I Introduction

Given a macro-level phenomenon, we often have models of how likely an
observation will occur under a macro-level context. However, when bombarded
with a plethora of time-series observation trails, can we simulate phenomenon
level simulated rollout to reconcile the macro-level model with observation
trails? This effort will provide context to observation trails and ease
reasoning about often unorganized time-series observation trails. A macro-
level phenomenon often does not directly dictate the occurrence of any single
observation, often only probabilistically correlated with each observation and
can be well-modelled with Bernoulli distribution.

$observation_{i}|state_{m}\sim Bernoulli(p_{i}^{(m)})$

To solve this problem, we draw inspirations and ideas from successful machine
learning models such as activation function, layer normalization and working
in embedded space to build a permutation approximation architecture. Our
permutation approximation algorithm has proven to be a success on synthetic
data. The contribution of this paper can be summarized as follows:

  * •

Devised an approximation and randomized architecture which incorporates
randomness from Bernoulli simulation in a compact way

  * •

Devised problem-driven architecture which can learn from the structure of the
problem from the model, giving prediction with 1 data point in 1 shot

  * •

Devised an algorithm to solve the Bernoulli stochastic permutation problem.

  * •

Potentially for parallelizing heuristic search for permutations

  * •

Devised a highly accurate data-specific dropout function

We organize the rest of this paper as follows. Section 2 is a literature
review, followed by defining the Travelling Photographer Problem in Section 3.
An overview of our algorithm for Travelling Photographer is introduced in
Section 4, followed by an elaboration on pseudo-state rollout in Section 5.
Section 6 elaborates upon the simulated attention mechanism and customized
dropout layer for our architecture. Section 7 explores Machine Learning
Architecture for Approximation (MLAA) as a template for the real-valued
version of the Travelling Photographer Problem. We perform ablation studies on
both synthetic data in section 8 and the actual dataset in section 9. In
section 10, we customize another MLAA to solve the Travelling Salesman
Problem, demonstrating the generalizing capabilities of the architecture
across problem types. Finally, we list the method’s limitations in section 11,
and the paper concludes in section 12.

##  II Related Work

This section will introduce related works in two areas, attention mechanism
and Travelling Salesman Problem (TSP).

###  II-A Attention Mechanism

Attention has taken the NLP world by storm with the seminal work ”Attention is
All You Need” [39]. Many extensions to the attention mechanisms are evident in
the number of variants of Transformers, such as Set Transformer[31],
Longformer, and InsertionTransformer. Attention can easily be expanded to
solve problems other than NLP; it has also demonstrated capabilities in
solving routing problems and image captioning by focusing its attention on
some internal subsets of the data. The Transformer’s self-attention mechanism
is a weighted average importance score of its internal words concerning a
particular word. Attention performed much better on various machine
translation task benchmarks, BLEU than traditional statistical machine
translation. Since then, many works have studied the structure and kernel of
attention [36], claiming that the effectiveness of attention is due to its
entangling of positional embedding and word embedding. However, recent work by
Yu, Luo and the team [44] studied the attention mechanism for computer vision
tasks. It concluded that replacing the attention mechanism with much simpler
computational gadgets such as a pooling layer is possible. This view is
supported by Microsoft’s DeBERTa’s [41] super-human performance on SuperGLUE;
DeBERTa disentangles positional and word embedding.

###  II-B Travelling Salesman Problem Solver

This section will discuss approximation solvers for the travelling salesman
problem. In the 1960s, the Christofides algorithm is an approximation
algorithm which solves TSP with a worst-case approximation factor of 1.5.
Christofides algorithm was initially thought of as a placeholder, soon to be
replaced by a more sophisticated algorithm. However, it remained the best
worst-case bound approximation algorithm for TSP until today. From the
heuristic search side, the long-standing heuristic search solvers like LKH-
solver remain better performing than recent efforts to solve TSP with machine
learning models despite many recent efforts. LKH solver is a pair improvement
strategy which improves upon a particular TSP solution. Recent efforts to use
NLP machine learning tools to solve TSP have attempted to cast TSP encoding
and decoding as a translation problem or a general heuristic learning problem.
Some works have attempted to train a TSP solver with reinforcement learning
methods, whereas [30] trains a soft-actor critic network to learn an
improvement heuristic for TSP solutions instead of a search heuristic. Both
[33], which combined reinforcement learning strategy selector with LKH, and
[6], which pre-trains on small-cases of TSP, have successfully solved various
hard cases of TSP. In the general machine learning for NP-hard problem
schemes, powerful NP-hard solvers like ReduMIS for Maximum Independent Set
problem and Z3 have been paralleled by recent work that does general tree
search under the guidance of a trained neural network. Chen’s work [17] uses a
2-improvement local search strategy on a reduced graph, a combination of
search and reinforcement learning techniques.

The Travelling Salesman Problem remains a central problem in the graph
learning community that is open to this day.

##  III Problem Formulation

Given a set of states that are probabilistically related to observations
(model) and a string that collates observations from state traversal
(observation trail), we attempt to find the permutation of the state which has
most likely generated the observation trail.

###  III-A The Travelling Photographer Problem

Given the following paragraph, an observation trail and some states of agent
(Bob), the goal is to identify a sequence of states that most likely generated
the observation trail.

Suppose a photographer is travelling between different cities, one city per
day. In different cities, he takes pictures of different scenes at will
randomly. Sometimes different cities have the same scenes. Now that the
photographer is back at home, he has forgotten all about his travelling
schedule, and he now wants to figure out his travelling schedule (permutation
of cities). The photos have a date it was taken printed below; he wishes to
use that to find the order in which he visited the cities. The TPP is about
identifying a state permutation to a series of noisy observations.

We summarize the Travelling Photographer Problem as follows; a city can also
be interpreted as a state and given a model of cities. City A:

$a\sim Ber(0.1);b\sim Ber(0.2)$

City B:

$b\sim Ber(0.9);c\sim Ber(0.3)$

City C:

$b\sim Ber(0.8);d\sim Ber(0.5);b\sim Ber(0.7)$

Find a permutation of $\\{A,B,C\\}$ which has most likely led to observation
sequence $\\{1;2;2\&4\\}$. In other words, we are finding a sequence $s$ for
the following equation.

$\displaystyle argmax_{s_{i}\in\\{A,B,C\\}}$ $\displaystyle\quad
Pr(a|s_{0})Pr(b|s_{1})Pr(b,d|s_{2})$ (1)
$\displaystyle*Pr(s_{1}|s_{0})Pr(s_{2}|s_{1})$ (2)

##  IV Heuristic Search for Permutation

The algorithm is a heuristic search over the space of permutations on a user-
defined embedded space for observations. In essence, the scoring of each
permutation is a Hammersley-Clifford theorem-inspired numerical compression
and activation of a state rollout. We attempt to embed the features of each
observation episode via a numerical embedding function as input into neural
approximation architecture and then perform a ranking on the approximated
values. We have a customized activation function similar to the tanh function
for input state permutation and observation interpretation.

Input: Model $model$, State Transition $T$, Observation $obs$

Output: Best Fit State Permutation $best$

1 $fn=$ Dropout Function $(model,T,obs)$ ;

2 $relevant\\_perms=permutations(states,dropout=fn)$;

3 foreach  _perm in $relevant\\_perms$_ do

4 $score=$ Sequence Scorer $(obs,perm,T)$ ;

5 if  _$score >max\\_score$_ then

6 $best=perm$ ;

7 $score=max\\_score$ ;

8

9 end if

10

11 end foreach

return _$best$_

Algorithm 1 Permutation ML-Approx

Our architecture incorporates common machine learning layers such as the
attention linear layer, normalization layer, activation function, and lambda
layer with multiplication operations. In addition, we designed such that
optimization space is a monotonically increasing space. If a state is more
likely to have produced a set of observations, we give that state-epoch match
a strictly higher score than another state-epoch match is less likely to have
led to the same set of observations. The algorithm is such that instead of
finding a reward from rolling out a state’s Bernoulli observations, we try to
draw numerical values from a distribution parameterized by the Bernoulli
probabilities to replace a rollout. We score each state’s rollout with the
product of cliques which is proportional to the likelihood of a state
producing observations at hand. Since the number of observations per episode
is not constant, we need to normalize the products of each episode concerning
the number of observations.

Input: Observation Trail $observations$, State Permutation $permutation$,
State Transitions $prob$

Output: Heuristic Score

1 initilialize $transitions$ and $memory$ arrays ;

2 for  _Observation episode t_ do

3 $stateScore=$ Pseudo State Rollout$(permutation[t],observations[t])$;

4 $memory.append(stateScore)$;

5 $transitions.append(prob[permutation[t],state[t+1])$;

6

7 end for

8$sequence\\_score=$ Attention Block $(prob,rollout,order)$;

return _$sequence\\_score$_

Algorithm 2 Sequence Scorer

###  IV-A Attention Mechanism Simulation

We attempt to simulate the attention mechanism that balances state rollout and
the likelihood of state permutation sequence. The input layer takes
observation-to-state mapping for each epoch and the probability of length-3
sub-sequences of input state permutation. The algorithm also needs to simulate
the self-attention mechanism that the input state permutation performs on
itself.

Figure 1: Machine Learning Architecture for Approximation (First Order
Attention)

##  V Encoding Probabilities from Model

This paper attempts to score a pseudo-state rollout by repeatedly sampling
from a known model, activating the probability that observation occurs with a
custom activation function, and returning the product of all the activation
values as a heuristic for the likelihood of state generating the set of
observation. Hammersley-Clifford Theorem proves the correctness and
proportionality.

######  Theorem 1 (Hammersley-Clifford Theorem)

Assuming satisfaction of weak positivity condition. Let $U$ denote the set of
all random variables under consideration, and let
$\Theta,\phi_{1},\phi_{2},...,\phi_{n}\subseteq U$ and
$\psi_{1},\psi_{2},...,\psi_{m}\subseteq U$ denote arbitrary sets of
variables. If

$Pr(U)=f(\Theta)\prod_{i=1}^{n}g_{i}(\phi_{i})=\prod_{j=1}^{m}h_{j}(\psi_{j})$

for functions $f,g_{1},g_{2},...,g_{n}$ and $h_{1},h_{2},...,h_{m}$, then
there exist functions $h_{1}^{\prime},h_{2}^{\prime},...,h_{m}^{\prime}$ and
$g_{1}^{\prime},g_{2}^{\prime},...,g_{n}^{\prime}$ such that

$Pr(U)=(\prod_{j=1}^{m}h_{j}^{\prime}(\Theta\cap\psi_{j}))(\prod_{i=1}^{n}g_{i}^{\prime}(\phi_{i}))$

In other words, $\prod_{j=1}^{m}h_{j}(\psi_{j})$ provides a template for
further factorization of $f(\Theta)$.

######  Definition 1 (Weak Positivity Condition)

Functionally compatible densities $\pi_{1},...,\pi_{n}$ satisfy the weak
positivity condition on a set $A\subset S$, if there exists a point
$x^{\prime}\in A$ and a permutation $(r_{1},...,r_{n})$ on $\\{1,...,n\\}$such
that for almost all points $x\in A$ and all $j=1,...,n$,

$\pi_{r_{j}}(x_{r_{j}}^{\prime}|x_{r_{1}},...,x_{r_{j-1}},|x_{r_{j+1}},...,x_{r_{n}})>0$

First of all, we encode the model probabilities $Pr(o_{i}|s_{j})$’s and
$Pr(s_{i}|s_{j})$’s as normal distributions. We then activate drawn samples
from the distribution to get edge weights for our intermediate model.

$\mathcal{N}(\frac{Pr(o_{i}|s_{j})}{2},\sqrt{\frac{Pr(o_{i}|s_{j})}{5}})$ (3)

In the sections after, we denote activation function by $f$.

##  VI Pseudo State Rollout

Pseudo State Rollout  is an approximation of the reward function from a state
rollout, where the reward function rewards occurrences of observation. We want
to give a score proportional to the likelihood that the given state has
produced the observations at hand. Below is our 3-clique scoring scheme.

$Pr(o_{1},o_{2},o_{3},o_{4}|s)\propto\prod_{e_{i},e_{j},e_{k}\in\\{f(p_{1}),f(p_{2}),f(p_{3}),f(p_{4})\\}}e_{i}e_{j}e_{k}$
(4)

The product of all the possible n-cliques of the edge is proportional to the
factorized probability that the observations set is associated with a
particular state. Factorized Clique compressed a state rollout into a score;
it performs the function of an embedding layer. We simulate layer
normalization with constant multiplications in Pseudo State Rollout and other
algorithm subroutines.

Input: Observation Set at episode t $obs$, State $state$, Model $model$,
Clique Size $size$

Output: Activation Value

1 initilialize $value$ as 1 ;

2 foreach  _ob in $obs$_ do

3 $activations$= Rollout Activation$(obs,state,model)$;

4 $combis=combinations(activations,size)$ ;

5 $constant=e^{\frac{1}{len(combis)}}$ ;

6 foreach  _clique in combis_ do

7 $value*=multiply(clique)*constant$;

8

9 end foreach

10

11 end foreach

12$score=\sqrt[3]{value}*len(obs)$ ;

return _score_

Algorithm 3 Pseudo State Rollout Figure 2: Intermediate Model With Activated
Values for Edges

###  VI-A Activation Functions

The rollout activation function score is proportional to the likelihood that
observation occurs under a particular state. The activation value is
thresholded below with a minimum threshold value like a ReLU. The activation
function for the simulated self-attention mechanism for state permutations is
a minimum instead of a maximum and possesses a threshold from the below
feature.

Input: Observation Set at episode t $obs$, State $state$, Model $model$

Output: Activation Value

1 initilialize $draws$ and $thresh$ values, and $activations$ as empty array ;

2 foreach  _ob in $obs$_ do

3 if  _ob in $model[state].key()$_ then

4 $bp=model[state][obs]$ ;

5 #Bernoulli Probability of Observation Occurring Under State ;

6 $distribution=Gaussian(\frac{bp}{2},\sqrt{\frac{bp}{5}})$ ;

7 $samples=sample(distribution,draws)$ ;

8 $val=max(thresh,max(samples)$ ;

9 $activations.append(val)$ ;

10

11 end if

12 return _empty_

13 end foreach

return _activations_

Algorithm 4 Rollout Activation Function

Our activation function incorporates randomness in simulation by activating
values drawn from the distribution. The usage of activation functions, min,
max, and median, is also more intuitive for users; one can direct whether one
wants to consider the worst-case, average-case or the best-case in a simulated
state rollout. Our architecture has a customized activation function for
different input sections of the architecture.

##  VII Attention for Transitions and State Rollouts

We take both activated state permutation and observation graphs as input and
simulate embedding them before feeding both data streams into a common
multiplicative layer. There are two types of attention mechanisms, self-
attention and encoder-decoder attention.

$Attention=Softmax(\frac{Q*K}{Scale})*V^{T}$

  * •

Q = Encoder Layer; K = Decoder Layer; V = Learnt Vector

  * •

Self-Attention: Q = K = V = Source

  * •

Encoder-Decoder Attention: K, V = Source; Q: Target

We experimented with different types of encoding and decoding combinations.
Proportional to the richness of representation of the mechanism in Natural
Language Processing (NLP), the prediction performance also increases
accordingly. In First Order Attention, we simulated self-attention for input
state permutation and performed self-attention on state roll-out keys
$\\{state[i],obs[i]\\}$ as a whole. The decoder is a multiplicative function
that attempts to balance the two components.

Input: Transitions $transitions$, State Rollouts $rollout$

Output: Heuristic Score

1 initilialize $draws$ and $thresh$ values ;

2 $episodes=length(rollout)$ ;

3 $transition\\_product,rollout\\_product=1,1$ ;

4 for  _t in range(episodes - 2)_ do

5 $activated=$ Transition Activation $(transition[t]*transition[t+1])$ ;

6 $transition\\_product*=activated*e^{\frac{1}{episodes}}$

7 end for

8$transition\\_product*=transitions[-1]$ ;

9 foreach  _$state$ in $rollout$_ do

10 $rollout\\_product*=rollout$ ;

11

12 end foreach

13$score=\sqrt[3]{transition\\_product}*rollout\\_product$ ;

return _score_

Algorithm 5 First Order Attention

In the second-order attention, we process state-observation rollout and
likelihood of state sequence in tandem, drawing inspirations from
Bidirectional Encoder Transformer (BERT) models. The likelihood of matching
the entire sequence is proportional to the score of each of the two time-step
matching in the sequence. We performed self-attention on both input sequences
and tuple $\\{obs[t],prob(state[t],state[t+1],obs[t+1]\\}$ as encoder input to
the attention block.

Input: Transitions $transitions$, State Rollouts $rollout$

Output: Heuristic Score

1’ initialize $transition\\_product,rollout\\_product=1,1$ ;

2 $episodes=length(rollout)$ ;

3 for  _t in range(episodes)_ do

4 $activated=$ Transition Activation $(transition[t]*transition[t+1])$ ;

5 $transition\\_product=\sqrt[3]{activated}$ ;

6 $cur\\_rollout=rollout[t]*rollout[t+1]$ ;

7 $overall*=transition\\_product*cur\\_rollout*e^{\frac{1}{episode}}$ ;

8

9 end for

return _overall_

Algorithm 6 2-Sequence Attention

The above simulation of attention has produced an excellent decoder score.
However, the architecture can be confused when there are too many states and a
few positions.

Figure 3: 2-Sequence Architecture Diagram

This leads us to try to improve the architecture with a denoising filter in
the next section.

###  VII-A Simulated Dropout Layer

Like in knowledge graph problems, we would like to capture how central is a
particular state given an observation trail. We distribute the shard of the
probability of each of the observations to the states related and propose that
transitions are likely to take place between two states with a more
significant portion of the probability shard sum. We define centralness as the
sum of the probability shard of all transitions connected to it, expressed by
the matrix $state_{c}apacity,C$.

We have customized a dropout function for each input observation trail, and
the objective is to include the most central state for each observation
sequence. The simulated dropout improves prediction accuracy, effectively
speeds up computation, and allows for effective parallelization of the
architecture. To achieve this aim, we again borrow ideas from the best
practices of machine learning.

Input: Model $model$, State Transition $T$, Observations $obs$

Output: Dropout Function $dropout$

1 initilialize $must\\_traverse=[]$;

2 $state\\_capacity,C=$ Transition Capacity $(model,obs)$ ;

3 $M=[C,CTCT-CT]$ ;

4 foreach  _capacity in M_ do

5 $edge\\_set=$ Maximum Spanning Edges (C) ;

6 $state\\_weights=$ Weight Sum $(edge\\_set)$;

7 $ranking=sort(state\\_weights,decreasing=True)$ ;

8 $must\\_traverse.append(ranking.pop())$ ;

9

10 end foreach

11$dropout=$ lambda s: return $all(must\\_traverse)$ in s;

return _$dropout$_

Algorithm 7 Dropout Function

Instead of a random dropout rate, we have an inclusion function which works
very well empirically.

##  VIII Architecture as Template

Combining features stated in previous sections, here is a compact way of
defining the overall algorithm with custom-made activation functions for each
section of the algorithm. As a result, users can modify the behaviour of the
architecture when performing approximation with varied models and datasets.

Input: Model $model$, Rollout Activation Function $rollout$, Transition
Activation Function $transition$, Dropout Activation Function $dropout$,
Attention Type $attention$

Output: Architecture

1 $dropout\\_fn=$Dropout Function $(dropout)$ ;

2 $pseudo\\_rollout=$Pseudo State Rollout $(rollout)$ ;

3 $attention\\_block=$Attention$(attention,transition)$ ;

4 $architecture=$ Permutation ML-Approx
$(dropout\\_fn,pseudo\\_rollout,attention\\_block)$ ;

return _architecture_

Algorithm 8 ML Architecture for Approximation(MLAA)

###  VIII-A Real-Valued Observations

In this subsection, we consider real-valued input data in the observation
trail. The model records the observation under state as a normal distribution
or other continuous value distribution functions. The machine learning
architecture in Permutation ML-Approx serves as a template for solving the
problem and can be adapted accordingly for approximating observation trails
which include real-valued observations with customized activation functions
which work with real-valued observations.

1

Output: function

2 $function=MLAA($Real-Valued Rollout Activation, Transition Activation
Function, Real-Valued Dropout Activation Function, First Order Attention) ;

return _$function$_

Algorithm 9 Real-Valued Permutation ML-Approx

We input sampled values $L1$ values from the distribution into our
architecture to get a proportional score of these $L1$ values, reflecting the
likelihood that each state has led to these observations.

##  IX Ablation Studies

We have performed experiments on a synthetic dataset, and the results are
satisfying. Furthermore, we have found that this architecture is relatively
robust for sequence-to-sequence prediction by performing ablation studies on
several hyperparameters of the neural approximation structure.

###  IX-A Experimental Settings

Random tours on the state graph generate observation trails on the transition
matrix for some episode time steps. The experiment configuration consists of 9
states, and there are at most six observations associated with each state.

The procedure and architecture above has been able to blend well numerically
and consistently produce reasonably good predictions. We can see from the plot
below that both first and second order compilers improvements improve as the
number of episodes increase. $3$$4$$5$$6$$7$$8$$9$$0$$5\cdot
10^{-2}$$0.1$$0.15$$0.2$$0.2$Episodes of Observation$\%$ Error in Predicted
StringAblation on Clique Size1xF-2 (1st Order)1xF-3 (1st Order)1xF-2 (2nd
Order)1xF-3 (2nd Order)

The algorithm’s performance is slightly less satisfactory for cases with fewer
episodes because there are more possibilities for each state. In contrast,
when the agent visited most states, every state must be bound to a specific
position in the output permutation. Therefore will return a sequence for which
each state is being matched to a position of the maximum score. We also see
that the performance is consistent with the clique size and the number of
transitions in each granularity.

$3$$4$$5$$6$$7$$8$$9$$0$$5\cdot 10^{-2}$$0.1$$0.15$Episodes of Observation$\%$
Error in Predicted StringAblation on Repetitions1xF (1st Order)3xF(1st
Order)1xF(2nd Order)

With repetition and a majority vote on state, the architecture can find unique
state permutations for each observation trail, solving the problem with a low
error rate. The increase in accuracy from repetition comes from the Law of
Large Number.

##  X Use Case

Given an actual dataset for which a TPP model is not readily available, we
need first to construct a model from the dataset. On Kaggle’s Africa Economic
Crisis Data, we have defined states with functions on subsets of columns and
found state permutations for economic history. Algorithms described in this
paper have been able to capture contexts in each case’s dataset. For a series
of economic observations, ML-Approx has been able to interpret economic time-
series data with user-defined states using the case’s historical dataset as
background context. Results from the architecture have provided a flexible and
diversified way of interpreting time-series data.

Different conditions have been returned for different cases given the same
series of figures. The decoding is generally slightly more optimistic for
cases with solid economic history and more pessimistic for cases with weak
economic history.

##  XI Travelling Salesman Problem Heuristic Scorer

In this section, we will appropriate MLAA for the Travelling Salesman Problem.
We will present a heuristic which attempts to greedily balance between edge
cost minimization and every node’s potential as a local minimizer. While TSP
Approx cannot beat a traditional heuristic solver like the LKH solver, we
think the scoring heuristic presented below is relatively novel and can
potentially speed up a TSP heuristic solver because it helps with filtering.
Given two permutations, we score both permutations, the one with the larger
score is more likely to be part of an optimal solution. Our heuristic can be
helpful if the permutation space is too large and only a subset of states is
visible to the algorithm.

Input: Cost Matrix $cost$, Permutation $perm$

Output: node_activation

1 initialize $node\\_activation,prod,clique\\_act=1,[],1$;

2 foreach  _i in $perm$_ do

3 $edge\\_cost=\sum_{edge(i,j)\in perm}wij$ ;

4 $denom=\sum_{j=\Omega}wij-edge\\_cost$ ;

5 $prod.append(\frac{edge\\_cost}{denom})$

6 end foreach

7$combis=permutations(prod,3)$ ;

8 $k=len(combis)^{2}$ ;

9 foreach  _combi in $combis$_ do

10 $p1,p2,p3=combi$ ;

11 $clique\\_act*=p1*p2*p3*e^{\frac{1}{k}}$;

12

13 end foreach

return _$\sqrt[3]{clique\\_act}$_

Algorithm 10 Node Activation

We simulate the attention mechanism with multiplication operations between
node activation and edge activation outputs. Node activation values indicate
how much of the local minimizer potential of the node in permutation has been
exploited. We select all cliques (triangles) and get a product of those to get
a score proportional to the degree to which each node’s minimizing potential
in the permutation is exploited.

Input: Cost Matrix $cost$, States $states$, Length $length$

Output: local minima permutation, score

1 $perms=permutations(states,length)$ ;

2 foreach  _perm in $perms$_ do

3 $edge\\_activation=$Edge Activation $(cost,perm)$ ;

4 $node\\_activation=$Node Activation $(cost,perm)$ ;

5 $score=edge\\_activation*node\\_activation$ ;

6 if  _$score\geq best\\_score$_ then

7 $best=perm$ ;

8 $best\\_score=score$ ;

9

10 end if

11

12 end foreach

return _$best$ , $best\\_score$_

Algorithm 11 Local Minima

Local Minima is simply a wrapper for simulated attention mechanisms between
edge and node activations. For a set of states, it picks a permutation of
states which is the local minima permutation of a particular length and
returns that. We approximate the global optima by concatenating 3-4 local
optima in smaller node subsets, which can drastically speed up computation.
The randomness comes from the random distribution of states into a smaller set
of nodes. By repeating the experiments, we evaluate new partitions of states
and determine whether the best solution it produces is a good fit for the TSP
solution. Of course, the greater the score, the more recommendable is the
current portion of the solution.

$1$$2$$3$$4$$5$$1$$1.1$$1.2$$1.3$lg (Number of Repetitions)Approximation
FactorAblation of TSP Approx9 cities x 3

We appropriated MLAA to solve TSP, and we obtained performance as follows. We
can see that our heuristic can figure out an optimal solution very early on.

###  XI-A Simulated Attention as Improvement Heuristic

Our simulated Attention for TSP can also be appropriated as an improvement
heuristic to give satisfactory results for TSP20, and TSP100 problems, often
with an error rate of less than 10%, returning the optimal solution at times.
TSP solutions from our methods can compare to modern heuristic solvers for TSP
for TSP20 and TSP100. The LKH heuristic is a 3-local heuristic, whereas ours
is a 2-local heuristic. Our heuristic might possess a massive potential for
parallelization on a large scale. We leave the empirical evaluation of this
method on larger cases of TSPs as future work.

##  XII Limitations and Future Work

The method discussed in this paper requires careful balancing of weighing
factors with normalization constants; the outcome is sensitive to changes in
normalization constants. Future works should address methods to robustify the
simulated attention approach to develop an architecture that allows for both
transparencies and does not rely heavily on normalization for correct results.
It will be a massive improvement if the weights of different components in the
input can be explored in an unsupervised manner with randomized optimizers
like Stochastic Gradient Descent instead of relying on hard-coded architecture
parameters. Future work can also choose to simulate skip connections and
prompting.

On top of that, we would also like to expand our algorithms for more extensive
cases of Travelling Photographer Problems and Travelling Salesman Problems
with practical parallel implementations.

##  XIII Conclusion

In conclusion, MLORA can provide the explainability feature that neural
networks are still unable to provide. Thus, we proposed an architecture for
randomized algorithms inspired by machine learning to perform approximations
to provide more human interpretable structures. Furthermore, we can
effectively utilize a simulated attention mechanism to solve other permutation
problems. By combining attention to both edge and nodes independently yet
simultaneously, we can further push the boundaries of randomized algorithms to
solve probabilistic permutation (NP-Hard problems) with heuristics that take
advantage of both repetition and randomness. Such implementations help us
understand the numerical landscape of the attention mechanism better; it is
also a heuristic for paralleling and randomizing the search for permutations.

For both Travelling Photographer Problem and Travelling Salesman Problem, our
architecture can produce satisfactory results for small test cases, which
serves as a proof of concept for the feasibility of our machine learning
inspired architecture for approximation to solve NP-Hard problems.

In the appendix, we list other components of the Machine Learning
Architecture. Transition Capacity reflects the centralness of each state in an
observation sequence.

Input: Model $model$, Observation $obs$

Output: Transition Capacity Matrix $cap$

1 initilialize $cap=[]$;

2 foreach  _ob in $obs$_ do

3 foreach  _pair of states which can produce ob $,(s_{a},s_{b})$_ do

4 $activated=$ Dropout Activation $(s_{a},s_{b},ob)$ ;

5 $cap[s_{a}][s_{b}]+=activated$ ;

6 $cap[s_{b}][s_{a}]+=activated$ ;

7

8 end foreach

9

10 end foreach

return _$Upper\\_Triangle(cap)$_

Algorithm 12 Transition Capacity

We construct an adjacency graph of states, which edge weights is proportional
to likelihood of a transition taking place between the nodes given the
observation sequence.

###  -A Activation Functions

In this subsection, we list the activation functions for transition and
dropout function. We want to capture the best case for two-step transition,
this is reflected by the max function in Transition Activation Function.

Input: Transition Probability $prob$

Output: Activation Value

1 initilialize $draws$ and $thresh$ values ;

2 $distribution=Gaussian(\frac{bp}{2},\sqrt{\frac{bp}{5}})$ ;

3 $samples=sample(distribution,draws)$ ;

4 $activation=max(thresh,min(samples))$ ;

return _activation_

Algorithm 13 Transition Activation Function

For dropout activation function, we want to capture the likelihood that on
average how likely is the node going to be involved. This is reflected by the
median function choice.

Input: Model $model$, State A $state_{a}$, State B, $state_{b}$, Observation
$obs$

Output: Activation Value

1 initilialize $draws$ and $thresh$ values $=5,0$;

2 $p_{a}=model[state_{a}][obs]$ ;

3 $p_{b}=model[state_{b}][obs]$ ;

4 $distribution_{a}=Gaussian(\frac{p_{a}}{2},\sqrt{\frac{p_{a}}{5}})$ ;

5 $distribution_{b}=Gaussian(\frac{p_{b}}{2},\sqrt{\frac{p_{b}}{5}})$ ;

6 $samples_{a}=sample(distribution_{a},draws)$ ;

7 $samples_{b}=sample(distribution_{b},draws)$ ;

8 $value=median(samples_{a})*median(samples_{b})$ ;

9 $activation=max(thresh,value)$ ;

return _activation_

Algorithm 14 Dropout Activation Function

We think these choices of functions reflects our intention to capture the
best, average or worse case for probabilistic events. It might help us
understand why activation functions are crucial for neural networks.

###  -B Real-Valued TPP

For real-valued case, we again have customized activation functions, which
help us find an intermediate mapping between raw numerical values and distance
between average cases of draws to get a better overall heuristic score.

Input: Model $model$, State A $s_{a}$, State B, $s_{b}$, Observation $obs$

Output: Activation Value

1 $dist={energy\\_distance(model[s_{a}][obs],model[s_{b}][obs])}$ ;

2 $activation=\frac{1}{dist}$ ;

return _activation_

Algorithm 15 Real-Valued Dropout Activation Function

For roll-out activation function, we have separate functions for Gaussian
distribution case and non-Gaussian distribution case.

Input: Model $model$, Observation at episode t $obs$, State $state$

Output: Heuristic Score

1 initilialize $thresh,score$ values ;

2 $indices\\_combi=combinations(len(obs),3)$ ;

3 foreach  _$obs\\_indices$ in $indice\\_combi$_ do

4 foreach  _ind in $obs\\_indices$_ do

5 $val=obs[ind]$ ;

6 Observation@State Model, $m=model[state][ind]$ ;

7 $activated=$ Model Heuristic (m, value thresh, gauss) ;

8 $score*=activated$

9 end foreach

10

11 end foreach

return _score_

Algorithm 16 Real-Valued Rollout Activation

Input: Model $model$, Observation $val$, Minimum Threshold $thresh$, Is Model
Gaussian $gaussian$

Output: Observation Activation

1 if  _$gausian$ is True_ then

2 $activation=min(z\\_val(obs,m),thresh)$ ;

3

4else

5 $l1\\_dist=|max(m)-pdf(obs,m)|$ ;

6 $activation=\frac{1}{l1\\_dist}$ ;

7

return _$activation$_

Algorithm 17 Model Heuristic

## References

  * [1] Akari Asai, Kazuma Hashimoto, Hannaneh Hajishirzi, Richard Socher, and Caiming Xiong.  Learning to retrieve reasoning paths over wikipedia graph for question answering, 2019.
  * [2] Xavier Bresson and Thomas Laurent.  The transformer network for the traveling salesman problem, 2021.
  * [3] Jane Chandlee, Rémi Eyraud, Jeffrey Heinz, Adam Jardine, and Jonathan Rawski.  Learning with partially ordered representations.  CoRR, abs/1906.07886, 2019.
  * [4] Uthsav Chitra and Benjamin J. Raphael.  Random walks on hypergraphs with edge-dependent vertex weights.  CoRR, abs/1905.08287, 2019.
  * [5] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  BERT: pre-training of deep bidirectional transformers for language understanding.  CoRR, abs/1810.04805, 2018.
  * [6] Zhang-Hua Fu, Kai-Bin Qiu, and Hongyuan Zha.  Generalize a small pre-trained model to arbitrarily large tsp instances.  Proceedings of the AAAI Conference on Artificial Intelligence, 35(8):7474–7482, May 2021.
  * [7] Carla P. Gomes, Willem-Jan van Hoeve, and Lucian Leahu.  The power of semidefinite programming relaxations for max-sat.  integration of ai and or techniques in constraint programming, pages 104–118, 2006.
  * [8] M. Griffith, N. Spies, K. Krysiak, and et al.  Civic is a community knowledgebase for expert crowdsourcing the clinical interpretation of variants in cancer., 2017.
  * [9] Daniel Hsu, Sham M. Kakade, and Tong Zhang.  A spectral algorithm for learning hidden markov models, 2012.
  * [10] Kejun Huang, Xiao Fu, and Nicholas D. Sidiropoulos.  Learning hidden markov models from pairwise co-occurrences with application to topic modeling, 2018.
  * [11] Nathan Kallus.  Recursive partitioning for personalization using observational data.  volume 70 of Proceedings of Machine Learning Research, pages 1789–1798, International Convention Centre, Sydney, Australia, 06–11 Aug 2017\. PMLR.
  * [12] Jin-Hwa Kim, Jaehyun Jun, and Byoung-Tak Zhang.  Bilinear attention networks, 2018.
  * [13] Wouter Kool, Herke van Hoof, and Max Welling.  Attention, learn to solve routing problems!, 2018.
  * [14] Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut.  Albert: A lite bert for self-supervised learning of language representations, 2019.
  * [15] Juho Lee, Yoonho Lee, Jungtaek Kim, Adam R. Kosiorek, Seungjin Choi, and Yee Whye Teh.  Set transformer.  CoRR, abs/1810.00825, 2018.
  * [16] Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein.  Visualizing the loss landscape of neural nets.  In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems 31, pages 6389–6399. Curran Associates, Inc., 2018.
  * [17] Zhuwen Li, Qifeng Chen, and Vladlen Koltun.  Combinatorial optimization with graph convolutional networks and guided tree search, 2018.
  * [18] S. Lin and B. W. Kernighan.  An effective heuristic algorithm for the traveling-salesman problem, operations research.  Operations Research, 1973.
  * [19] Chenxi Liu, Liang-Chieh Chen, Florian Schroff, Hartwig Adam, Wei Hua, Alan Yuille, and Li Fei-Fei.  Auto-deeplab: Hierarchical neural architecture search for semantic image segmentation, 2019.
  * [20] Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee.  Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks, 2019.
  * [21] Brian McFee and Gert Lanckriet.  Partial order embedding with multiple kernels.  In Proceedings of the 26th Annual International Conference on Machine Learning, ICML ’09, page 721–728, New York, NY, USA, 2009. Association for Computing Machinery.
  * [22] Joseph Mellor, Jack Turner, Amos Storkey, and Elliot J. Crowley.  Neural architecture search without training, 2020.
  * [23] Niv Nayman, Asaf Noy, Tal Ridnik, Itamar Friedman, Rong Jin, and Lihi Zelnik-Manor.  Xnas: Neural architecture search with expert advice, 2019.
  * [24] Byambasuren Odmaa, Yang Yunfei, Sui Zhifang, Dai Damai, Chang Baobao, Li Sujian, and Zan Hongying.  Preliminary study on the construction of chinese medical knowledge graph.  Journal of Chinese Information Processing, 33(10):1, 2019.
  * [25] Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove.  Deepsdf: Learning continuous signed distance functions for shape representation, 2019.
  * [26] Giorgio Roffo.  Ranking to learn and learning to rank: On the role of ranking in pattern recognition applications.  CoRR, abs/1706.05933, 2017.
  * [27] Daniel Selsam, Matthew Lamm, Benedikt Bünz, Percy Liang, Leonardo de Moura, and David L. Dill.  Learning a SAT solver from single-bit supervision.  In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019.
  * [28] Mitchell Stern, William Chan, Jamie Kiros, and Jakob Uszkoreit.  Insertion transformer: Flexible sequence generation via insertion operations.  CoRR, abs/1902.03249, 2019.
  * [29] Yao-Hung Hubert Tsai, Shaojie Bai, Makoto Yamada, Louis-Philippe Morency, and Ruslan Salakhutdinov.  Transformer dissection: A unified understanding of transformer’s attention via the lens of kernel, 2019.
  * [30] Yaoxin Wu, Wen Song, Zhiguang Cao, Jie Zhang, and Andrew Lim.  Learning improvement heuristics for solving routing problems, 2020.
  * [31] Emre Yolcu and Barnabas Poczos.  Learning local search heuristics for boolean satisfiability.  In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32, pages 7992–8003. Curran Associates, Inc., 2019.
  * [32] Jing Zhang and Xindong Wu.  Multi-label inference for crowdsourcing.  In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’18, page 2738–2747, New York, NY, USA, 2018. Association for Computing Machinery.
  * [33] Jiongzhi Zheng, Kun He, Jianrong Zhou, Yan Jin, and Chu-Min Li.  Combining reinforcement learning with lin-kernighan-helsgaun algorithm for the traveling salesman problem.  Proceedings of the AAAI Conference on Artificial Intelligence, 35(14):12445–12452, May 2021.


# Relativistic coupled-cluster calculation of hyperfine-structure constants of
229Th3+ and evaluation of the electromagnetic nuclear moments of 229Th

Fei-Chen Li,1,3 Hao-Xue Qiao,1,∗ Yong-Bo Tang,2,∗ and Ting-Yun Shi3 School of
Physics and Technology, Wuhan University, Wuhan, 430072, China  Physics
Teaching and Experiment Center, Shenzhen Technology University, Shenzhen,
518118, China  State Key Laboratory of Magnetic Resonance and Atomic and
Molecular Physics, Wuhan Institute of Physics and Mathematics, Innovation
Academy for Precision Measurement Science and Technology, Chinese Academy of
Sciences, Wuhan, 430071, China  qhx<EMAIL_ADDRESS>
###### Abstract

229Th is a promising candidate for developing nuclear optical clocks and
searching for new physics beyond the standard model. For this purpose, it is
important to have accurate knowledge of the nuclear properties of 229Th. In
this work, we calculate hyperfine-structure constants for the lowest four
states of 229Th3+ using the relativistic coupled-cluster method based on the
Gaussian basis set. The no-pair Dirac-Coulomb-Breit Hamiltonian with the
lowest-order quantum electrodynamics (QED) correction is the starting point,
and all linear and non-linear terms of single and double excitations are
included in the coupled-cluster calculation. Combining the measured HFS
constants [Phys. Rev. Lett. 106. 223001(2011)] and present atomic
calculations, we extract the magnetic dipole moment, $\mu=0.359(9)$, and the
electric quadrupole moment, $Q=2.95(7)$, of the 229Th nucleus. Our magnetic
dipole moment is perfectly consistent with the recommended value from the all-
order calculation by Safronova et. al.[Phys.Rev.A 88, 060501(R) (2013)], but
our electric quadrupole moment is smaller than their recommended value, by
about $5\%$. A detailed analysis indicates that the non-linear terms of single
and double excitations, not included in the all-order calculation, are crucial
to produce a precise $Q$ value of 229Th. In addition, we also report the
magnetic octupole hyperfine-structure constants and some important non-
diagonal hyperfine transition matrix elements, which are required for further
extraction of the magnetic octupole moment $\Omega$ of 229Th nucleus.

###### pacs:

##  I INTRODUCTION

229Th possesses an extremely low first excited isomeric state of only several
eV Kroger and Reich (1976); Beck  _et al._ (2007), which opens the possibility
to construct a high-precision nuclear optical clock Peik and Tamm (2003);
Campbell  _et al._ (2012); Peik  _et al._ (2021) and provides a strongly
enhanced sensitivity for searching for new physics beyond the standard model
Safronova  _et al._ (2018); Fadeev  _et al._ (2020). To achieve these
attractive objects, accurate knowledge of the nuclear and electronic
properties of 229Th, such as the nuclear moments, plays a fundamental role.
The accurate nuclear moment values not only can help to grasp the hyperfine
structures (HFS) of 229Th atoms and related ions, but also provide a benchmark
tool to improve the nuclear model theory thereby helping to reliably predict
the properties of its isomers such as the isomer transition rate and the
nuclear moments Minkov and Pálffy (2019). The nuclear quadrupole moment of
229Th also can be used to assess the sensitivity of the isomeric transition to
the possible variation of the fine-structure constant $\alpha$ and other
fundamental constants Flambaum (2006); Berengut  _et al._ (2009); Fadeev  _et
al._ (2020).

There have been some reported works on the magnetic dipole and electric
quadrupole moment, $\mu$ and $Q$, of 229Th nucleus Chasman  _et al._ (1977);
Minkov and Pálffy (2019); Gerstenkorn  _et al._ (1974); Bemis  _et al._
(1988); Berengut  _et al._ (2009); Campbell  _et al._ (2011); Safronova  _et
al._ (2013). These results are summarized in Table 1, and one can find obvious
discrepancies between the predicted values from the nuclear theory Minkov and
Pálffy (2019); Chasman  _et al._ (1977) and the deduced values from atomic
structure calculation combined with experimental measurements Gerstenkorn  _et
al._ (1974); Bemis  _et al._ (1988); Berengut  _et al._ (2009); Safronova  _et
al._ (2013). For $\mu$, the previous nuclear theory predictions Minkov and
Pálffy (2019); Chasman  _et al._ (1977) are significantly above the
experimentally deduced values Gerstenkorn  _et al._ (1974); Safronova  _et
al._ (2013), but for $Q$, the nuclear theoretical prediction Minkov and Pálffy
(2019) is lower than the experimentally deduced values Gerstenkorn  _et al._
(1974); Bemis  _et al._ (1988); Berengut  _et al._ (2009); Safronova  _et al._
(2013). Among these results, the values extracted by Safronova et al.
Safronova  _et al._ (2013) using all-order calculations based on 229Th3+
combined with precise hyperfine spectral measurement Campbell  _et al._ (2011)
are considered the most reliable Pyykkö (2018). In their calculations, all
possible single and double excitations are iterated to all orders of
perturbation theory, and a part of triple excitations are included
perturbatively, thus it is termed as SDpT method. However, only the linear
coupled-cluster terms of single- and double-excitations are included in the
SDpT method, and we notice that the correlation effects represented by the
nonlinear terms play a crucial role in the precise prediction for the HFS of
Fr or Fr-like monovalent systems Sahoo  _et al._ (2015); Li  _et al._ (2021a).
Thus it is necessary and important to accurately assess the contribution of
nonlinear terms in the HFS calculations of Fr-like 229Th3+.

Table 1: The magnetic dipole moment $\mu$ ($\mu_{N}$) and electric quadrupole moment $Q$ ($e$b) of 229Th reported in previous studies. “Theoretical prediction” results are obtained by calculations based on various nuclear theoretical models. “Experimentally deduced” values are found from laser spectroscopy measurements in combination with atomic calculations. The $Q$ of the penultimate column was derived by Campbell et al. Campbell  _et al._ (2011), combining their measurements and the calculations by Berengut et al. Berengut  _et al._ (2009). The $\mu$ and $Q$ of the last column were derived by Safronova et al. Safronova  _et al._ (2013), combining their calculations and the measurements by Campbell et al. Campbell  _et al._ (2011). | Theoretical prediction |  | Experimentally deduced
---|---|---|---
Ref. Minkov and Pálffy (2019) |  Ref. Chasman  _et al._ (1977) |  |  Ref. Gerstenkorn  _et al._ (1974) |  Ref. Bemis  _et al._ (1988) |  Ref. Campbell  _et al._ (2011) |  Ref. Safronova  _et al._ (2013)
$\mu$ |  0.530$-$0.655 | 0.54 |  | 0.46(4) |  |  | 0.360(7)
$Q$ | 2.80 |  |  | 4.3(9) | 3.15(3) | 3.11(16) | 3.11(6)

Theoretical investigation for the HFS of 229Th3+ is also of much interest in
some other aspects. It has been reported as the most promising system for the
realization of a single-ion nuclear clock based on a virtual clock transition
composed by a pair of stretched hyperfine states of both nuclear ground and
isomeric manifolds Campbell  _et al._ (2012). Knowledge of the relevant
hyperfine interaction matrix elements is helpful to extract the nuclear
excitation energy Beloy (2014). Moreover, 229Th3+ is also a possible system
for atomic parity-nonconservation experiments Roberts  _et al._ (2013), and
its magnetic dipole HFS constant would be a reliable probe for the estimate of
theoretical accuracy of the related weak matrix elements.

In the present work, we calculate the HFS constants of the low-lying states of
229Th3+ using the relativistic coupled-cluster method based on the Gaussian
basis set. The correlation effects are investigated by the ab initio methods
at different levels including the Dirac-Fock approximation, low-order many-
body perturbation theory, linearized and fully single- and double-excitation
relativistic coupled-cluster method. The magnetic dipole and electric
quadrupole moment, $\mu$ and $Q$, of 229Th nucleus are extracted by combining
present recommended theoretical results with the available experimental
values. The hyperfine interaction matrix elements required to extract the
magnetic octupole moment are also presented. In section II, we provide a brief
overview of the coupled-cluster method and the hyperfine structure theory.
Numerical results and discussions are presented in section III. Finally, a
summary is given in section IV.

##  II Theoretical method

###  II.1 A brief description of the relativistic coupled-cluster approach

For Th3+, being a monovalent atomic system, the exact wavefunction with a
valence orbital $\upsilon$ can be formally expressed in the form

$|\Psi_{\upsilon}\rangle=e^{S}|\Phi_{\upsilon}\rangle,$ (1)

where $|\Phi_{\upsilon}\rangle$ is the zeroth-order wavefunction obtained by
the Dirac-Fock calculation, and the exponential cluster operator, $e^{S}$ =
1+$S$+$\frac{1}{2!}S^{2}$+$\dots$, represents the expansion of wave operators
in the framework of the coupled-cluster theory. According to the number of
particles, $n$, to be excited from the reference configuration, the cluster
operator $S$ can be partitioned into

$S=S_{1}+S_{2}+\dots+S_{n}.$ (2)

In practice, the contributions from the triple and higher excitations are
always expected to be relatively small and the computations are extremely
time-consuming. Thus, only the singles ($S_{1}$) and doubles ($S_{2}$)
excitations are considered in the present coupled-cluster calculation (CCSD),
and Eq.(1) can be written as

$\displaystyle\left|\Psi_{v}\right\rangle_{\rm CCSD}=$
$\displaystyle\left\\{1+S_{1}+S_{2}+\frac{1}{2!}\left(S_{1}^{2}+S_{2}^{2}+S_{1}S_{2}\right)\right.$
(3)
$\displaystyle\left.+\frac{1}{3!}\left(S_{1}^{3}+3S_{1}^{2}S_{2}\right)+\frac{1}{24}S_{1}^{4}\right\\}\left|\Phi_{v}\right\rangle.$

Eq.(3) is a nearly complete expression of the wavefunction within the single
and double excitation approximation (SD), where the first three terms are
linear terms and provide the majority of the contribution. A more convenient
and simplified treatment is to retain only the linear terms, which are
referred to as the linearized coupled-cluster (LCCSD) wavefunction,

$|\Psi_{\upsilon}\rangle_{\rm LCCSD}=(1+S_{1}+S_{2})|\Phi_{\upsilon}\rangle.$
(4)

However, for some strongly correlated systems, the contribution from nonlinear
terms may be important and need to be evaluated carefully, especially in the
high precision calculations of some special properties.

The transition matrix elements of an operator $\hat{O}$ from state
$\left|\Psi_{w}\right\rangle$ to $\left|\Psi_{v}\right\rangle$ can be
evaluated according to

$\displaystyle\bar{O}$
$\displaystyle=\frac{\left\langle\Psi_{w}|\hat{O}|\Psi_{v}\right\rangle}{\left\langle\Psi_{w}\mid\Psi_{v}\right\rangle}$
(5)
$\displaystyle=\frac{\left\langle\Phi_{w}\left|e^{S\dagger}\hat{O}e^{S}\right|\Phi_{v}\right\rangle}{\left\langle\Phi_{w}\left|e^{S\dagger}e^{S}\right|\Phi_{v}\right\rangle},$

where ${e^{S\dagger}}$ stands for the complex conjugate of $e^{S}$. Expanding
the wavefunction $\Psi$ using Eq.(3) or Eq.(4), one would obtain the
transition matrix element within the CCSD or LCCSD approximation,
respectively. The specific calculation steps are described in detail in the
previous work Li  _et al._ (2021b, a).

###  II.2 The hyperfine structure theory

The hyperfine interaction Hamiltonian for a relativistic electron can be
expressed as Schwartz (1955)

$H_{\mathrm{HFI}}=\sum_{k}T_{k}^{e}\cdot T_{k}^{n},$ (6)

where $T_{k}^{e}$ and $T_{k}^{n}$ are the spherical tensor operators with rank
$k$ ($k>0$) in the electronic and nuclear coordinates, respectively. A matrix
element of the HFI between the basis of the hyperfine states $|\alpha I,\gamma
J;FM_{F}\rangle$, which coupled a nuclear eigenstate $|\alpha I,M_{I}\rangle$
and an atomic eigenstate $|\gamma J,M_{J}\rangle$ is

$\displaystyle\left\langle\alpha^{\prime}I^{\prime},\gamma^{\prime}J^{\prime};F^{\prime}M_{F}^{\prime}\left|H_{\mathrm{HFI}}\right|\alpha
I,\gamma
J;FM_{F}\right\rangle=\delta_{F^{\prime}F}\delta_{M_{F}^{\prime}M_{F}}$ (7)
$\displaystyle\times(-1)^{I+J+F}\sum_{k}\left\\{\begin{array}[]{lll}F&J&I\\\
k&I&J^{\prime}\end{array}\right\\}\left\langle\gamma^{\prime}J^{\prime}\left\|T_{k}^{e}\right\|\gamma
J\right\rangle\left\langle\alpha^{\prime}I^{\prime}\left\|T_{k}^{n}\right\|\alpha
I\right\rangle,$

where $F$ is the total angular momentum with F=I+J, $I$ is the nuclear spin,
$J$ is the total electronic angular momentum, $\alpha$ and $\gamma$
encapsulate the remaining nuclear and electronic quantum numbers,
respectively. Then the first-order correction $E_{F,J}^{(1)}$ of hyperfine
interaction to the energy is defined as

$\displaystyle
E_{F}^{(1)}=(-1)^{I+J+F}\sum_{k}\left\\{\begin{array}[]{lll}F&J&I\\\
k&I&J\end{array}\right\\}\left\langle\gamma J\left\|T_{k}^{e}\right\|\gamma
J\right\rangle\left\langle\alpha I\left\|T_{k}^{n}\right\|\alpha
I\right\rangle,$ (8)

Restricted to $k\leq 3$, $E_{F}^{(1)}$ can be parameterized in terms of the
HFS constants $A$, $B$, and $C$. These HFS constants are expressed as follows,
respectively

$\displaystyle A=\mu_{N}\frac{\mu_{I}}{I}\frac{\langle{\gamma
J|}\big{|}{T_{1}^{e}}\big{|}{|\gamma J}\rangle}{\sqrt{J(J+1)(2J+1)}},$ (9)
$\displaystyle
B=2Q\left[\frac{2J(2J-1)}{(2J+1)(2J+2)(2J+3)}\right]^{1/2}{\langle{\gamma
J|}\big{|}{T_{2}^{e}}\big{|}{|\gamma J}\rangle},$ (10)

and

$\displaystyle
C=\Omega_{I}\left[\frac{J(2J-1)(J-1)}{(J+1)(J+2)(2J+1)(2J+3)}\right]^{1/2}{\langle{\gamma
J|}\big{|}{T_{3}^{e}}\big{|}{|\gamma J}\rangle},$ (11)

where $A$, $B$, and $C$ are magnetic dipole (M1), electric quadrupole (E2),
and magnetic octupole (M3) hyperfine structure constants respectively.
$\mu_{N}$ is the nuclear Bohr magneton, and the diagonal nuclear reduced
matrix elements $\left\langle\alpha I\left\|T_{k}^{n}\right\|\alpha
I\right\rangle$ in Eq.(8) are contained in the nuclear moments, $\mu_{I}$
($k$=1), $Q$($k$=2), $\Omega_{I}$ ($k$=3), respectively.

The contributions from second-order hyperfine interactions are generally on
the same order as the magnetic octupole contribution, thus we also take into
account these terms here. In addition, due to the first excited nuclear state
of 229Th has an anomalously small excitation energy, there should also be a
small but observable contribution in second order correction that corresponds
to the hyperfine mixing between the electronic states of the ground nuclear
states and the electronic states of isomer nuclear states Beloy (2014). Then
the second-order correction $E_{F}^{(2)}$ of hyperfine interaction to the
energy in 229Th is defined as

$\displaystyle E_{F}^{(2)}$ $\displaystyle=\sum{\prime}\frac{1}{E_{\alpha
I,\gamma
J}-E_{\alpha^{\prime}I^{\prime},\gamma^{\prime}J^{\prime}}}\sum_{k_{1},k_{2}}\left\\{\begin{array}[]{lll}I&J&F\\\
J^{\prime}&I&k_{1}\end{array}\right\\}\left\\{\begin{array}[]{lll}I&J&F\\\
J^{\prime}&I&k_{2}\end{array}\right\\}$ (12)
$\displaystyle\times\left\langle\alpha
I\left\|T_{k_{1}}^{n}\right\|\alpha^{\prime}I^{\prime}\right\rangle\left\langle\alpha^{\prime}I^{\prime}\left\|T_{k_{2}}^{n}\right\|\alpha
I\right\rangle\left\langle\gamma
J\left\|T_{k_{1}}^{e}\right\|\gamma^{\prime}J^{\prime}\right\rangle\left\langle\gamma^{\prime}J^{\prime}\left\|T_{k_{2}}^{e}\right\|\gamma
J\right\rangle.$

The summation involves all possible excited nuclear states and electronic
states, and $E_{\alpha I,\gamma J}$ includes both nuclear and electronic
energies. $\left\langle\alpha
I\left\|T_{k}^{n}\right\|\alpha^{\prime}I^{\prime}\right\rangle$ and
$\left\langle\gamma
J\left\|T_{k}^{e}\right\|\gamma^{\prime}J^{\prime}\right\rangle$ correspond to
the reduced matrix elements of nuclear part and electronic parts,
respectively. The off-diagonal reduced matrix elements of nuclear part can be
found in Ref. Beloy (2014). In the present work, we focus on M1 and E2 off-
diagonal reduced matrix elements from the fine-structure splitting (electronic
part), because their contributions dominate owing to small energy
denominators.

The single particle reduced matrix elements of the operators $T_{1}^{e}$,
$T_{2}^{e}$, and $T_{3}^{e}$ are given by

$\displaystyle\langle{\kappa_{i}}$ $\displaystyle\|T_{1}^{e}\|$
$\displaystyle\kappa_{j}\rangle=-\langle-\kappa_{i}\|C^{(1)}\|\kappa_{j}\rangle(\kappa_{i}+\kappa_{j})$
(13) $\displaystyle\times$
$\displaystyle\int_{0}^{\infty}{dr\frac{P_{i}(r)Q_{j}(r)+P_{j}(r)Q_{i}(r)}{r^{2}}\times{F(r)}},$
$\displaystyle\langle{\kappa_{i}}\|T_{2}^{e}\|\kappa_{j}\rangle$
$\displaystyle=$ $\displaystyle-\langle\kappa_{i}\|C^{(2)}\|\kappa_{j}\rangle$
(14) $\displaystyle\times$
$\displaystyle\int_{0}^{\infty}{dr\frac{P_{i}(r)P_{j}(r)+Q_{j}(r)Q_{i}(r)}{r^{3}}},$

and

$\displaystyle\langle{\kappa_{i}}\|T_{3}^{e}\|\kappa_{j}\rangle$
$\displaystyle=$
$\displaystyle-\frac{1}{3}\langle-\kappa_{i}\|C^{(3)}\|\kappa_{j}\rangle(\kappa_{i}+\kappa_{j})$
(15) $\displaystyle\times$
$\displaystyle\int_{0}^{\infty}{dr\frac{P_{i}(r)Q_{j}(r)+P_{j}(r)Q_{i}(r)}{r^{4}}},$

where the relativistic angular-momentum quantum number
$\kappa=\ell(\ell+1)-j(j+1)-1/4$, $P$ and $Q$ are the large and small radial
components of Dirac wavefunction, respectively.

In the present work, we employed a finite basis set composed of even-tempered
Gaussian-type functions expressed as
$G_{i}=\mathcal{N}_{i}r^{\ell+1}e^{-\alpha_{i}r^{2}}$, to expand the Dirac
radial wavefunction $P$ and $Q$ as in Refs. Chaudhuri  _et al._ (1999); Li
_et al._ (2021b, a), where $\mathcal{N}_{i}$ is the normalization factor, and
$\alpha_{i}=\alpha\beta^{i-1}$, where the two independent parameters $\alpha$
and $\beta$ are optimized separately for each orbital symmetries. Table 2
lists the Gauss basis parameters. $N$ is the number of basis set for each
symmetry. $N_{c}$ and $N_{v}$ represent the number of core and virtual
orbitals, respectively.

Table 2: The parameters of the Gauss basis set. $N$ is the number of basis set for each symmetry. $N_{c}$ and $N_{v}$ represent the number of core orbitals and virtual orbitals, respectively.  | $s$ | $p$ | $d$ | $f$ | $g$ | $h$ | $i$ | $k$
---|---|---|---|---|---|---|---|---
$\alpha\times 10^{3}$ | 3.3 | 3.0 | 4.6 | 13.5 | 12 | 22 | 23 | 24
$\beta$ | 1.95 | 1.95 | 1.695 | 1.681 | 1.95 | 2.05 | 2.15 | 2.25
$N$ | 40 | 38 | 35 | 32 | 25 | 20 | 15 | 10
$N_{c}$ | 6 | 5 | 3 | 1 | 0 | 0 | 0 | 0
$N_{v}$ | 22 | 22 | 26 | 24 | 19 | 16 | 15 | 10

In our calculations, the no-pair Dirac Hamiltonian is set as the starting
point. Breit interaction and the lower-order quantum electrodynamics (QED)
radiative potential proposed by Flambaum and Ginges Flambaum and Ginges (2005)
are considered at the same foot as coulomb interaction. The Fermi nuclear
distribution was employed to describe the Coulomb potential between electrons
and the nucleus. All the core orbitals and virtual orbitals with energies of
smaller than 10000 a.u. are included in the correlation calculations.

##  III RESULTS AND DISCUSSION

###  III.1 Energies

Table 3: Zeroth-order DF Coulomb correction ${E^{(0)}}$, second-order MBPT correction ${E^{(2)}}$, linearized part of single ($S_{1}$) and double ($S_{2}$) excitations correction ${E^{(S_{1}+S_{2})}}$, the non-linear coupled-cluster terms corrections ${E^{\rm(NL)}}$, the corrections of Breit and QED effects to the eneriges for Th3+ in cm-1. ${E_{\rm MBPT(2)}}$=${E^{(0)}}$+Breit+${E^{(2)}}$ , ${E_{\rm LCCSD}}$=${E^{(0)}}$\+ Breit+${E^{(S_{1}+S_{2})}}$ , ${E_{\rm CCSD}}$=${E_{\rm LCCSD}}$\+ ${E^{\rm(NL)}}$, ${E_{\rm Final}}$=${E_{\rm CCSD}}$+${\rm QED}$, represent the results obtained within second-order MBPT, LCCSD, CCSD, and CCSD results with QED correction approximations, respectively. The experiment energies ${E_{\rm Expt.}}$ with the uncertainty in parentheses are also listed for comparison. Level | ${E^{(0)}}$ | Breit | ${E^{(2)}}$ | ${E^{(S_{1}+S_{2})}}$ | ${E^{\rm(NL)}}$ | QED | ${E_{\rm MBPT(2)}}$ | ${E_{\rm LCCSD}}$ | ${E_{\rm CCSD}}$ | ${E_{\rm Final}}$ |  ${E_{\rm Expt.}}$ Kramida  _et al._ (2020); Blaise and Wyart
---|---|---|---|---|---|---|---|---|---|---|---
$5f_{5/2}$ |  $-$206612 |  $-$737 |  $-$31348 |  $-$25459 | 754 |  $-$204 |  $-$238697 |  $-$232808 |  $-$232054 |  $-$232258 |  $-$231065(200)
$5f_{7/2}$ |  $-$203185 |  $-$869 |  $-$29828 |  $-$24413 | 754 |  $-$198 |  $-$233882 |  $-$228467 |  $-$227714 |  $-$227911 |  $-$226740
$6d_{3/2}$ |  $-$211800 |  $-$48 |  $-$12980 |  $-$11154 | 441 |  $-$89 |  $-$224828 |  $-$223002 |  $-$222562 |  $-$222650 |  $-$221872
$6d_{5/2}$ |  $-$207687 |  $-$129 |  $-$12314 |  $-$10923 | 422 |  $-$68 |  $-$219055 |  $-$217663 |  $-$217241 |  $-$217308 |  $-$216579
$7s_{1/2}$ |  $-$200273 |  $-$90 |  $-$10931 |  $-$9173 | 378 |  $-$182 |  $-$211294 |  $-$209536 |  $-$209157 |  $-$208975 |  $-$207934
$7p_{1/2}$ |  $-$165094 |  $-$167 |  $-$7587 |  $-$6746 | 365 |  $-$3 |  $-$172848 |  $-$172007 |  $-$171643 |  $-$171645 |  $-$170826
$7p_{3/2}$ |  $-$153571 |  $-$57 |  $-$6042 |  $-$5450 | 319 |  $-$7 |  $-$159670 |  $-$159079 |  $-$158760 |  $-$158753 |  $-$158009
Table 4: Comparison of our calculated ${E_{\rm LCCSD}}$, ${E_{\rm CCSD}}$, ${E_{\rm Final}}$ transition energies with other available theoretical and experimental data in ${\rm cm^{-1}}$, the difference of theoretical and experimental data labeled by $\triangle$. The first row lists the absolute energy of the ground state, and the other rows list the excitation energies of other states with respect to the ground state.  Level | ${E_{\rm LCCSD}}$ | $\triangle$ | ${E_{\rm CCSD}}$ | $\triangle$ | ${E_{\rm Final}}$ | $\triangle$ |  ${E_{\rm FSCCSD}}$ Eliav and Kaldor (2012) | $\triangle$ |  ${E_{\rm all-order}}$ Safronova  _et al._ (2006) | $\triangle$ |  ${{E_{\rm Expt.}}}$ Kramida  _et al._ (2020); Blaise and Wyart
---|---|---|---|---|---|---|---|---|---|---|---
$5f_{5/2}$ |  $-$232808 |  $-$1743 |  $-$232054 |  $-$989 |  $-$232258 |  $-$1193 |  $-$231957 |  $-$892 |  $-$230304 | 761 |  $-$231065
$5f_{7/2}$ | 4341 | 16 | 4340 | 15 | 4346 | 21 | 4320 |  $-$5 | 4136 |  $-$189 | 4325
$6d_{3/2}$ | 9806 | 613 | 9492 | 299 | 9608 | 415 | 9416 | 223 | 8304 |  $-$889 | 9193
$6d_{5/2}$ | 15145 | 659 | 14812 | 326 | 14949 | 463 | 14738 | 252 | 13377 |  $-$1109 | 14486
$7s_{1/2}$ | 23272 | 141 | 22896 |  $-$235 | 23282 | 151 | 22833 |  $-$308 | 22229 |  $-$902 | 23131
$7p_{1/2}$ | 60800 | 561 | 60411 | 172 | 60612 | 373 | 60346 | 109 | 59213 |  $-$1026 | 60239
$7p_{3/2}$ | 73729 | 673 | 73293 | 237 | 73504 | 448 | 73206 | 150 | 71932 |  $-$1124 | 73056

We carried out a series of calculations for the energies of some important
low-lying states of 229Th3+ at different correlation levels including Dirac-
Fock(DF), second-order many-body perturbation theory MBPT(2), LCCSD, and full
CCSD calculations. We also classified the contributions from different terms,
including the Zeroth-order DF Coulomb correction ${E^{(0)}}$, the Breit
corrections, second-order MBPT correction $E^{(2)}$, linearized part of single
($S_{1}$) and double ($S_{2}$) excitations correction ${E^{(S_{1}+S_{2})}}$,
the non-linear coupled-cluster terms corrections ${E^{\rm(NL)}}$, and the QED
correction. All these results listed in Table 3 comparing with experimental
values Kramida  _et al._ (2020); Blaise and Wyart  labeled as ${E_{\rm
Expt.}}$. The ${E_{\rm MBPT(2)}}$=${E^{(0)}}$+Breit+${E^{(2)}}$ , ${E_{\rm
LCCSD}}$=${E^{(0)}}$\+ Breit+${E^{(S_{1}+S_{2})}}$ , ${E_{\rm CCSD}}$=${E_{\rm
LCCSD}}$\+ ${E^{\rm(NL)}}$, ${E_{\rm Final}}$=${E_{\rm CCSD}}$+${\rm QED}$,
represent the results obtained within second-order MBPT, LCCSD, CCSD, and CCSD
results with QED correction approximations, respectively.

From the Table 3, the importance of inclusion of correlation effect can be
found. Take the incorrect determination of the ground state in the DF
calculation as an example: the spectral measurements confirm that the ground
state of Th3+ is $5f_{5/2}$ state, while the ground state generated from DF
calculation is the $6d_{3/2}$ state. The MBPT(2) results obviously
overestimated the correlation effect and predicted much lower energies for all
the states tabulated. The contribution from QED correction is about $0.1\%$
for $5f$ and $7s$ states, while it is negligible for other states. The CCSD
results show a better agreement with experiments since the contribution from
the nonlinear single- and double-excitation terms, which is about $0.3\%$ for
$5f$ and $0.2\%$ for other states, has been included. Finally, for the two
$5f$ states, the differences between ${\rm E_{Final}}$ and the experimental
values are about $0.5\%$. The main reason for this discrepancy is the omission
of the higher-order correlation effects, such as the triple- and higher
excitation terms.

Table 4 presents a detailed comparison of our LCCSD, CCSD, and CCSD+QED
transition energies with other $ab$ $initio$ results. It can be observed that
in terms of transition energy, our CCSD results are consistent with the
results obtained by relativistic intermediate Hamiltonian Fock-space coupled-
cluster methods (FSCCSD) Eliav and Kaldor (2012), and both are much closer to
the experimental value than the results obtained by the all-order calculations
including partial extra third-order terms Safronova  _et al._ (2006). The
contributions from different corrections are given in the all-order
calculations Safronova  _et al._ (2006), allowing us to explore the origin of
the difference with our results. It can be found that the DF Coulomb energies
${E^{(0)}}$ obtained from Ref. Safronova  _et al._ (2006) and our calculations
agree well with each other, while the absolute values of Breit correction to
energies in Ref. Safronova  _et al._ (2006) are significantly above our
estimation. In addition, we also calculated the energy under the condition of
$\ell_{max}=6$, and we found that our MBPT(2) and LCCSD results are completely
consistent with their corresponding results. Comparing with the current
calculation results of $\ell_{max}=7$, we observe that for the energies $5f$
states, the contributions of the partial wave $\ell=7$ are about $-$520 ${\rm
cm^{-1}}$. This suggests that the contribution of higher-order fractional
waves to energy is important, and the same conclusion also can be drawn in the
work of Ref. Safronova  _et al._ (2006).

###  III.2 Magnetic dipole moment

Table 5: Determination of the 229Th nuclear magnetic dipole moment using measured $A$ (MHz) from Ref. Campbell  _et al._ (2011) and the calculated $A/\mu$ from the present work. The Present $A/\mu$ column contains ab initio results at different correlation levels in MHz/$\mu_{N}$. The results deduced from SDpT calculation and other methods are also listed here for comparison.  Level | ${{A_{\rm Expt.}}}$ | Present $A/\mu$ | Other $A/\mu$ | $\mu$
---|---|---|---|---
DF | MBPT(3) | LCCSD | CCSD | SDpT Safronova  _et al._ (2013) | LCCSD | CCSD | SDpT Safronova  _et al._ (2013) | Others
$5f_{5/2}$ | 82.2(6) | 203.73 | 250.52 | 232.28 | 230.53 | 229.2 | 0.354(3) | 0.357(3) | 0.359 |
$5f_{7/2}$ | 31.4(7) | 105.79 | 81.49 | 87.14 | 87.12 | 86.1 | 0.360(8) | 0.360(8) | 0.365 |
$6d_{3/2}$ | 155.3(12) | 331.75 | 455.21 | 442.71 | 431.85 | 431.5 | 0.351(3) | 0.360(3) | 0.360 |
$6d_{5/2}$ |  $-$12.6(7) | 121.73 |  $-$34.43 |  $-$35.39 |  $-$23.31 |  $-$36.7 | 0.356(20) | 0.541(30) | 0.343 |
Final |  |  |  |  |  |  | 0.355(9) | 0.359(9) | 0.360(7) | 0.530$-$0.655 Minkov and Pálffy (2019)
|  |  |  |  |  |  |  |  |  | 0.54 Chasman  _et al._ (1977)
|  |  |  |  |  |  |  |  |  | 0.46(4) Gerstenkorn  _et al._ (1974)
Table 6: Determination of the 229Th nuclear electric quadrupole moment using measured $B$ (MHz) from Ref. Campbell  _et al._ (2011) and the calculated $B/Q$ from the present work. The Present $B/Q$ column contains ab initio results at different correlation levels in MHz/eb.. The results deduced from SDpT calculation are also listed. Level | ${{B_{\rm Expt.}}}$ | Present $B/Q$ | Other $B/Q$ | $Q$
---|---|---|---|---
DF | MBPT(3) | LCCSD | CCSD | SDpT Safronova  _et al._ (2013) | LCCSD | CCSD | SDpT Safronova  _et al._ (2013)
$5f_{5/2}$ | 2269(6) | 537 | 436 | 743 | 783 | 725 | 3.06(1) | 2.90(1) | 3.13
$5f_{7/2}$ | 2550(12) | 575 | 388 | 829 | 879 | 809 | 3.07(2) | 2.90(2) | 3.15
$6d_{3/2}$ | 2265(9) | 609 | 717 | 746 | 758 | 738 | 3.04(1) | 3.00(1) | 3.07
$6d_{5/2}$ | 2694(7) | 648 | 474 | 883 | 897 | 873 | 3.05(1) | 3.00(1) | 3.09
Final |  |  |  |  |  |  | 3.05(3) | 2.95(7) | 3.11(6)

The $A/\mu$ for the first four states of 229Th3+ are calculated at different
correlation levels including DF, third-order many-body perturbation theory
MBPT(3), LCCSD and CCSD. The magnetic dipole moment $\mu$ are then extracted
by combining our theoretical results with the experimental HFS constants
Campbell  _et al._ (2011). The results are listed in Table 5, and compared
with other available calculated values Gerstenkorn  _et al._ (1974); Chasman
_et al._ (1977); Minkov and Pálffy (2019); Safronova  _et al._ (2013).
Uncertainties are given in parentheses.

As seen from Table 5, the MBPT(3) calculations significantly overestimated the
correlation effect compared with our CCSD results, which is consistent with
the case of energies. For the $5f$ states, the LCCSD results are very close to
the CCSD results, where the contribution of the nonlinear terms to $A/\mu$ is
about $1\%$, while it is about $2.5\%$ for the $6d_{3/2}$ state. The main
source of theoretical uncertainty is the electron correlation. States with
smaller correlation effects tend to be more conducive to accurate calculation.
The correlation effects ${\rm(CCSD-DF)/CCSD\times 100\%}$ and the non-linear
effects ${\rm(CCSD-LCCSD)/CCSD\times 100\%}$ accounted to the $A/\mu$ of above
states are no more than $25\%$ and $3\%$, respectively, with the only
exception of $6d_{5/2}$ being $565\%$ and $-35\%$, respectively. Therefore,
even if these calculations are carried out under the same theoretical
framework, it is not guaranteed that the results of different states can
achieve the same precision, because different states are different in
sensitivity to various correlation effects. In this case, averaging the
theoretical calculation results of several states can effectively reduce the
uncertainty caused by different sensitivities of different states to the
correlation effect. Here, we abandon the $6d_{5/2}$ state which is strongly
dependent on the correlation effect, and take the average value of the other
three states to obtain the final $\mu$ value, $i.e.$ 0.359(9), where the
uncertainty in parentheses is entirely from the measurement.

Our CCSD value of $0.359(9)$ matches well with the all-order SDpT results
$0.360(7)$, but is much lower than all other reported results Minkov and
Pálffy (2019); Chasman  _et al._ (1977); Gerstenkorn  _et al._ (1974). For
example, our results are about $50\%-80\%$ lower than the previous nuclear
theory predictions, and about $30\%$ lower than the experimentally deduced
values from Fourier spectroscopy in 1974 Gerstenkorn  _et al._ (1974). It must
be pointed out that the results obtained by combining spectroscopy
measurements with atomic structure calculations are more reliable than the
predictions directly using nuclear model theory. Furthermore, current laser
spectroscopy measurement techniques and theoretical calculation method of
atomic structure have improved lot over time, which is also one of the
important reasons why we think the current results as well as Safronova et
al.’s results are more reliable than the 1974 results Gerstenkorn  _et al._
(1974). Remind that the all-order SDpT calculation corresponds to a full
consideration of linear single- and double-excitation terms and further
inclusion of some triple-excitations perturbatively, while our CCSD
calculation includes all the linear and non-linear single- and double-
excitation terms. Thus it is reasonable to extract the contribution of the
triple-excitation terms from the comparison between SDpT results and our LCCSD
results. The comparison of our CCSD result and SDpT result indicates that the
contribution of the partial triple excitations and the non-linear term on
$A/\mu$ to the first three states are really small. Another interesting
feature we observed is that the $A/\mu$ of $6d_{5/2}$ state obtained by the
SDpT method is $-36.7$, which is almost the same as our LCCSD value of
$-35.4$. This is an independent validation for both the SDpT results and our
LCCSD results, and this also suggests a strong dependence for the $A/\mu$ of
$6d_{5/2}$ on the nonlinear single- and double-excitation terms.

###  III.3 Electric quadrupole moments

Similar with the case of magnetic dipole moment, we also determine the 229Th
nuclear electric quadrupole moment $Q$ using measured $B$ (MHz) from Ref.
Campbell  _et al._ (2011) and our calculated $B/Q$. The results are listed in
Table 6, and compared with SDpT results Safronova  _et al._ (2013). From Table
6, one can see that the correlation effects in the MBPT(3) calculation are
significantly different from the LCCSD and full CCSD results. Except for the
$6d_{3/2}$ state, the contributions from the third-order MBPT correction to
the zeroth-order DF values are negative, while the corrections of linear terms
(${\rm LCCSD-DF}$) and nonlinear coupled-cluster terms (${\rm CCSD-LCCSD}$)
are positive for these four states, respectively. The total correlation
effects of the $5f_{5/2,7/2}$ states are very strong, about 30% of the total
CCSD results, while the correlation effects of $6d_{3/2,5/2}$ states are
relatively small. In addition, it can be found that the higher the angular
momentum, the greater the total correlation effect. Furthermore, the
contribution of non-linear terms accounts for approximately a quarter of the
total correlation effect in $5f$ states, but less than a tenth in $6d$ states.
Both the total correlation effects and the non-linear terms corrections are
significantly larger than those in $A/\mu$ for each state, which is opposite
of Ra+ Li  _et al._ (2021a). Such strong and unusual correlation effects
indicate that it is more difficult to accurately calculate the $B/Q$ of ground
state of 229Th3+. Here we take the average value of all four states as final
$Q$ value, $i.e.$ 2.95(7), where the uncertainty in parentheses is also
entirely from the measurement.

It is found that the $Q$ value from the SDpT calculation is significantly
above the present final CCSD value. As we have stated, the contribution of the
triple-excitation terms can be extracted from the comparison between the SDpT
results and our LCCSD results, and the difference between LCCSD and CCSD
results owes to the non-linear terms. Taking the present LCCSD result
$3.05(3)$ as a reference, the SDpT result $3.11(6)$ and the CCSD result
$2.95(7)$ are on both sides of it. It indicates that the contribution of
partial triple-excitation terms is positive, about $2\%$, while the
contribution of SD non-linear terms is negative, about $3\%$. This trend will
lead to a cancellation. Therefore, in order to extract $Q$ accurately, it is
important not only to evaluate the contribution of nonlinear terms accurately,
but also to consider the contribution of triple-excitation terms completely.
Furthermore, comprehensive observations show that the contribution of the
nonlinear term to the $A/\mu$ is positive, while to the $B/Q$ it is negative.
The relative ratio of the correlation represented by nonlinear terms
${\rm(CCSD-LCCSD)/(CCSD-DF)}$ in the $B/Q$ is much large than in the $A/\mu$,
which is different from that of Fr and Ra+.

###  III.4 Magnetic octupole hyperfine interaction

Recently, the rapid development of spectroscopy technology makes the
extraction of some high-order nuclear moments possible. For example, the
nuclear magnetic octupole moments $\Omega$ of 133Cs Gerginov  _et al._ (2003),
137Ba+ Lewty  _et al._ (2013), and 171Yb Singh  _et al._ (2013); de Groote
_et al._ (2021) have been successfully determined. The 229Th3+ is also a good
candidate for the studies of nuclear structure beyond the first two
electromagnetic moments, since its ground state $5f_{5/2}$ has a large angular
momentum and thus a large coupling to $\Omega$. Safronova et al. have
suggested that this coupling effect may induce a hyperfine interval at a level
of a few Hz Safronova  _et al._ (2013). To extract $\Omega$, an accurate
calculation of relevant octupole hyperfine interaction matrix elements,
including diagonal and important non-diagonal matrix elements of low-lying
states in 229Th3+, is necessary. Once the accurate HFS constant $C$ is
measured, these matrix elements can be used to determine the magnetic octupole
moment immediately. In addition, the present octupole matrix elements are
expected to possess a higher level of accuracy than those from the previous
calculations using the third-order perturbation method mentioned in Refs.
Beloy (2014); Johnson  _et al._ (1987). Then the excitation energy of the
229Th nucleus may be determined more accurately to a certain extent using the
method reported by Beloy Beloy (2014).

Table 7 listed the electronic octupole hyperfine interaction matrix elements
from DF, LCCSD, CCSD calculations, including diagonal $C/\Omega$ in
KHz/($\mu_{N}$$\times$b) and non-diagonal matrix elements in MHz of low-lying
states. The uncertainty of these octupole parameters mainly comes from the
unconsidered high-order correlation effects beyond CCSD, which are generally
not greater than the contribution of the SD nonlinear terms ($i.e.$CCSD-
LCCSD). Therefore, we take CCSD results as the recommended values listed in
the “Final” column and the corresponding absolute value of the difference
between CCSD and LCCSD results as the uncertainty enclosed in parentheses. It
can be observed from the Table 7 that for $5f_{5/2,7/2}$ and $6d_{5/2}$
states, the correlation effects are significant compared with the relatively
small matrix elements, which would lead to large uncertainties in the
calculations. For the $6d_{3/2}$ state, the conclusion is opposite. It is
worth noting that the contributions of the non-linear term on $C/\Omega$ to
the $5f$ and $6d$ states are very large which are somewhat different from
$A/\mu$. This may be owing to fact that the value of $C/\Omega$ is too small
to be accurate enough. The larger HFS constant $C$ is, the more conducive to
accurate calculation and measurement. Therefore, the results of $6d_{3/2}$
state are more suitable for roughly predicting the magnitude of HFS constant
$C$. These matrix elements are of great significance to extract the $\Omega$
of 229Th nucleus.

Table 7:  $C/\Omega$ in KHz/($\mu_{N}$$\times$b) and off-diagonal matrix elements in MHz from DF, LCCSD, CCSD calculations. The CCSD results are taken as the recommended values listed in the “Final” column and the corresponding absolute value of the difference between CCSD and LCCSD results as the uncertainty enclosed in parentheses. Level | ${\rm DF}$ | ${\rm LCCSD}$ | ${\rm CCSD}$ | Final
---|---|---|---|---
$5f_{5/2}$ | 0.68 |  $-$0.60 |  $-$0.38 |  $-$0.38(22)
$5f_{7/2}$ | 0.31 | 1.83 | 1.12 | 1.12(71)
$6d_{3/2}$ | 6.77 | 7.91 | 7.88 | 7.88(3)
$6d_{5/2}$ | 1.84 | 0.21 | 0.05 | 0.05(16)
Off-diagonal matrix elements
$\langle 5f_{5/2}||{T}^{e}_{1}||5f_{7/2}\rangle$ | 329 | 1121 | 1189 | 1189(68)
$\langle 5f_{5/2}||{T}^{e}_{2}||5f_{7/2}\rangle$ | 423 | 605 | 637 | 637(32)
$\langle 6d_{3/2}||{T}^{e}_{1}||6d_{5/2}\rangle$ | 355 | 4180 | 3962 | 3962(218)
$\langle 6d_{3/2}||{T}^{e}_{2}||6d_{5/2}\rangle$ | 695 | 815 | 830 | 830(15)

##  IV Conclusion

In summary, we carried out a comprehensive study of the energies and HFS
constants for several low-lying states in the 229Th3+, using a relativistic
CCSD method based on a large Gaussian basis set. To clarify the role of
different correlation corrections, we also provide some intermediate results
including the results from DF, MBPT, and LCCSD calculations. The Breit and QED
corrections to the energies are also investigated. Our calculations indicate
that more higher-order corrections not included in our calculations are
important to further reduce the difference between the theoretical prediction
and experimental measurement. The magnetic dipole moment $\mu$ and the
electric quadrupole moment $Q$ of 229Th are obtained by combining our atomic
calculation with the measured HFS $A$ and $B$ values. Our recommended $\mu$
value of 0.359(9) is in excellent agreement with the SDpT result of
$0.360(7)$. The present electric quadrupole moment, $Q=2.95(7)$, is smaller
than their recommended value, $Q=3.11(6)$, by about $5\%$, possibly owing to
the fact that the electron correlations represented by the non-linear terms,
omitted in SDpT calculations, contribute significantly to the electric
quadrupole moment and reduce the values. Further analysis shows that the
third-order many-body perturbation theory is not a very effective tool to
generate sufficiently accurate nuclear moments $\mu$ and $Q$ of 229Th. It
always strongly overestimates the magnitude and sometimes gives an incorrect
sign of the correlation effect. Additionally, we also present the magnetic
octupole HFS constants $C/\Omega$ and some important non-diagonal hyperfine
transition matrix elements, which are required for further extracting the
magnetic octupole moment $\Omega$ of 229Th nucleus. Our calculations also show
that the $6d_{3/2}$ is a suitable state to carry out precise measurements of
the hyperfine splittings from which $\Omega$ of 229Th can be inferred.

###### Acknowledgements.

We are grateful to Yong-jun Cheng of Shanxi Normal University and Ting-xian
Zhang of Lanzhou University of Technology for many useful discussions. The
work was supported by the National Natural Science Foundation of China (No.
115074094, No. 12074295, No. 11774386), the Strategic Priority Research
Program of the Chinese Academy of Sciences Grant (No. XDB21030300), the Post-
doctoral research project of SZTU (No. 202028555301011), and the project of
Educational Commission of Guangdong Province of China (No.2020KTSCX124).

## References

  * Kroger and Reich (1976) L. A. Kroger and C. W. Reich, “Features of the low-energy level scheme of 229Th as observed in the $\alpha$-decay of 233U,” Nucl. Phys. A 259, 29–60 (1976).
  * Beck  _et al._ (2007) B. R. Beck, J. A. Becker, P. Beiersdorfer, G. V. Brown, K. J. Moody, J. B. Wilhelmy, F. S. Porter, C. A. Kilbourne, and R. L. Kelley, “Energy Splitting of the Ground-State Doublet in the Nucleus Th229,” Phys. Rev. Lett. 98, 142501 (2007).
  * Peik and Tamm (2003) E. Peik and Chr. Tamm, “Nuclear laser spectroscopy of the 3.5 eV transition in Th-229,” Europhys. Lett. 61, 181–186 (2003).
  * Campbell  _et al._ (2012) C. J. Campbell, A. G. Radnaev, A. Kuzmich, V. A. Dzuba, V. V. Flambaum, and A. Derevianko, “Single-ion nuclear clock for metrology at the 19th decimal place,” Phys. Rev. Lett. 108, 120802 (2012).
  * Peik  _et al._ (2021) E. Peik, T. Schumm, M. S. Safronova, A. Pálffy, J. Weitenberg, and P. G. Thirolf, “Nuclear clocks for testing fundamental physics,” Quantum Sci. Technol. 6, 034002 (2021), arXiv:2012.09304 [quant-ph] .
  * Safronova  _et al._ (2018) M. S. Safronova, D. Budker, D. DeMille, Jackson DerekF Kimball, A. Derevianko, and C. W. Clark, “Search for new physics with atoms and molecules,” Rev. Mod. Phys. 90, 025008 (2018).
  * Fadeev  _et al._ (2020) Pavel Fadeev, Julian C. Berengut, and Victor V. Flambaum, “Sensitivity of ${}^{229}\mathrm{Th}$ nuclear clock transition to variation of the fine-structure constant,” Phys. Rev. A 102, 052833 (2020).
  * Minkov and Pálffy (2019) Nikolay Minkov and Adriana Pálffy, “Theoretical predictions for the magnetic dipole moment of ${}^{229m}\mathrm{Th}$,” Phys. Rev. Lett. 122, 162502 (2019).
  * Flambaum (2006) V. V. Flambaum, “Enhanced effect of temporal variation of the fine structure constant and the strong interaction in ${}^{229}\mathrm{Th}$,” Phys. Rev. Lett. 97, 092502 (2006).
  * Berengut  _et al._ (2009) J. C. Berengut, V. A. Dzuba, V. V. Flambaum, and S. G. Porsev, “Proposed experimental method to determine $\alpha$ sensitivity of splitting between ground and 7.6 ev isomeric states in ${}^{229}\mathrm{Th}$,” Phys. Rev. Lett. 102, 210801 (2009).
  * Chasman  _et al._ (1977) R. R. Chasman, I. Ahmad, A. M. Friedman, and J. R. Erskine, “Survey of single-particle states in the mass region $A>228$,” Rev. Mod. Phys. 49, 833–891 (1977).
  * Gerstenkorn  _et al._ (1974) S. Gerstenkorn, P. Luc, J. Verges, D. W. Englekemeir, J. E. Gindler, and F. S. Tomkins, “Structures hyperfines du spectre d′$\acute{e}$tincelle,moment magn$\acute{e}$tique et quadrupolaire de l′isotope 229 du thorium,” J. Phys. France 35, 483–495 (1974).
  * Bemis  _et al._ (1988) C. E. Bemis, F. K. McGowan, Jr. Ford, J. L. C., W. T. Milner, R. L. Robinson, P. H. Stelson, G. A. Leander, and C. W. Reich, “Coulomb excitation of states in 229Th,” phys. scr. 38, 657–663 (1988).
  * Campbell  _et al._ (2011) C. J. Campbell, A. G. Radnaev, and A. Kuzmich, “Wigner crystals of ${}^{229}\mathrm{Th}$ for optical excitation of the nuclear isomer,” Phys. Rev. Lett. 106, 223001 (2011).
  * Safronova  _et al._ (2013) M. S. Safronova, U. I. Safronova, A. G. Radnaev, C. J. Campbell, and A. Kuzmich, “Magnetic dipole and electric quadrupole moments of the 229th nucleus,” Phys. Rev. A 88, 060501(R) (2013).
  * Pyykkö (2018) Pekka Pyykkö, “Year-2017 nuclear quadrupole moments,” Mol. Phys. 116, 1328–1338 (2018).
  * Sahoo  _et al._ (2015) B. K. Sahoo, D. K. Nandy, B. P. Das, and Y. Sakemi, “Correlation trends in the hyperfine structures of ${}^{210,212}\mathrm{Fr}$,” Phys. Rev. A 91, 042507 (2015).
  * Li  _et al._ (2021a) Feichen Li, Yong-Bo Tang, Hao-xue Qiao, and Ting-Yun Shi, “Ab initio calculations of the hyperfine structure of Ra+ and evaluations of the electric quadrupole moment Q of the 209,211,221,223Ra nuclei,” J. Phys. B: At. Mol. Opt. Phys. 54, 145004 (2021a).
  * Beloy (2014) K. Beloy, “Hyperfine structure in ${{}^{229g}\mathrm{Th}}^{3+}$ as a probe of the ${}^{229g}\mathrm{Th}\rightarrow^{229m}\mathrm{Th}$ nuclear excitation energy,” Phys. Rev. Lett. 112, 062503 (2014).
  * Roberts  _et al._ (2013) B. M. Roberts, V. A. Dzuba, and V. V. Flambaum, “Parity nonconservation in fr-like actinide and cs-like rare-earth-metal ions,” Phys. Rev. A 88, 012510 (2013).
  * Li  _et al._ (2021b) Fang Li, Hong Ma, and Yong-Bo Tang, “Relativistic coupled-cluster calculation of hyperfine-structure constants of La2+,” J. Phys. B: At. Mol. Opt. Phys. 54, 065003 (2021b).
  * Schwartz (1955) C. Schwartz, Phys. Rev. 97, 380 (1955).
  * Chaudhuri  _et al._ (1999) Rajat K. Chaudhuri, Prafulla K. Panda, and B. P. Das, “Hybrid approach to relativistic gaussian basis functions: Theory and applications,” Phys. Rev. A 59, 1187–1196 (1999).
  * Flambaum and Ginges (2005) V. V. Flambaum and J. S. M. Ginges, “Radiative potential and calculations of qed radiative corrections to energy levels and electromagnetic amplitudes in many-electron atoms,” Phys. Rev. A 72, 052115 (2005).
  * Kramida  _et al._ (2020) A. Kramida, Yu. Ralchenko, J. Reader, and and NIST ASD Team, NIST Atomic Spectra Database (ver. 5.8), [Online]. Available: https://physics.nist.gov/asd [2021, August 15]. National Institute of Standards and Technology, Gaithersburg, MD. (2020).
  * (26) J. Blaise and J. Wyart,  http://www.lac.universite-paris-saclay.fr/Data/Database/.
  * Eliav and Kaldor (2012) Ephraim Eliav and Uzi Kaldor, “Transition energies of Rn- and Fr-like actinide ions by relativistic intermediate Hamiltonian Fock-space coupled-cluster methods,” Chem. Phys. 392, 78–82 (2012).
  * Safronova  _et al._ (2006) U. I. Safronova, W. R. Johnson, and M. S. Safronova, “Excitation energies, polarizabilities, multipole transition rates, and lifetimes in th iv,” Phys. Rev. A 74, 042511 (2006).
  * Gerginov  _et al._ (2003) Vladislav Gerginov, Andrei Derevianko, and Carol E. Tanner, “Observation of the nuclear magnetic octupole moment of ${}^{133}\mathrm{C}\mathrm{s}$,” Phys. Rev. Lett. 91, 072501 (2003).
  * Lewty  _et al._ (2013) Nicholas C. Lewty, Boon Leng Chuah, Radu Cazan, Murray D. Barrett, and B. K. Sahoo, “Experimental determination of the nuclear magnetic octupole moment of 137ba+ ion,” Phys. Rev. A 88, 012518 (2013).
  * Singh  _et al._ (2013) Alok K. Singh, D. Angom, and Vasant Natarajan, “Observation of the nuclear magnetic octupole moment of 173yb from precise measurements of the hyperfine structure in the ${{}^{3}P}_{2}$ state,” Phys. Rev. A 87, 012512 (2013).
  * de Groote  _et al._ (2021) R. P. de Groote, S. Kujanpää, Á. Koszorús, J. G. Li, and I. D. Moore, “Magnetic octupole moment of ${}^{173}\mathrm{Yb}$ using collinear laser spectroscopy,” Phys. Rev. A 103, 032826 (2021).
  * Johnson  _et al._ (1987) W. R. Johnson, M. Idrees, and J. Sapirstein, “Second-order energies and third-order matrix elements of alkali-metal atoms,” Phys. Rev. A 35, 3218–3226 (1987).