#  USTC-ICTS-19-33 Modular symmetry origin of texture zeros and quark lepton
unification

Jun-Nan Lua,b111E-mail<EMAIL_ADDRESS>Xiang-Gan Liua222E-mail:
<EMAIL_ADDRESS>Gui-Jun Dinga333E-mail<EMAIL_ADDRESS>
aInterdisciplinary Center for Theoretical Study and Department of Modern
Physics,
University of Science and Technology of China, Hefei, Anhui 230026, China
bAHEP Group, Institut de Física Corpuscular – CSIC/Universitat de València,
Parc Científic de Paterna.
C/ Catedrático José Beltrán, 2 E-46980 Paterna (Valencia) - SPAIN

###### Abstract

The even weight modular forms of level $N$ can be arranged into the common
irreducible representations of the inhomogeneous finite modular group
$\Gamma_{N}$ and the homogeneous finite modular group $\Gamma^{\prime}_{N}$
which is the double covering of $\Gamma_{N}$, and the odd weight modular forms
of level $N$ transform in the new representations of $\Gamma^{\prime}_{N}$. We
find that the above structure of modular forms can naturally generate texture
zeros of the fermion mass matrices if we properly assign the representations
and weights of the matter fields under the modular group. We perform a
comprehensive analysis for the $\Gamma^{\prime}_{3}\cong T^{\prime}$ modular
symmetry. The three generations of left-handed quarks are assumed to transform
as a doublet and a singlet of $T^{\prime}$, we find six possible texture zeros
structures of quark mass matrix up to row and column permutations. We present
five benchmark quark models which can produce very good fit to the
experimental data. These quark models are further extended to include lepton
sector, the resulting models can give a unified description of both quark and
lepton masses and flavor mixing simultaneously although they contain less
number of free parameters than the observables.

##  1 Introduction

The standard model (SM) of particle physics has been precisely tested so far.
However, neither significant evidence for the departures from the SM nor
convincing hints for the presence of new physics has been found. The masses of
quarks and charged leptons are parameterized by the Yukawa coupling constants
in SM. The Yukawa sector is still poorly understood, and the SM itself can not
predict the exact values of quark masses and CKM mixing matrix. The
fundamental principle which rules the hierarchical charged fermion mass
spectra, tiny neutrino masses, flavor mixing and CP violation, is still
elusive. Neutrino oscillation experiments have made enormous progress in past
years. The three lepton mixing angles and neutrino mass squared differences
have been precisely measured. The latest global fit of neutrino oscillation
data gives the best fit values and $1\sigma$ errors of the lepton mixing
angels as $\theta_{12}/^{\circ}=33.82^{+0.78}_{-0.76}$,
$\theta_{23}/^{\circ}=49.6^{+1.0}_{-1.2}$ and $\theta_{13}/^{\circ}=8.61\pm
0.13$ [1]. The present neutrino oscillation data favor normal ordering (NO)
neutrino mass spectrum over inverted ordering (IO), CP conservation in
neutrino oscillation is disfavored at $2\sigma$ confidence level, and the
leptonic Dirac CP phase $\delta_{CP}$ around $3\pi/2$ is preferred [2, 3].

Neutrino oscillation provides us new insight to understanding the flavor
puzzle. It is found that the neutrino mixing angles can be reproduced by
extending the SM with a finite discrete non-abelian flavor symmetry [4, 5, 6,
7, 8, 9, 10]. The observed neutrino mixing pattern arises as the result of
particular vacuum alignment of scalar fields called flavons which
spontaneously break certain discrete flavor symmetry. The flavons are SM
singlet and they transform nontrivially under the flavor symmetry group.
Usually a number of flavons are necessary, and the scalar potential as well as
additional shaping symmetries has to be cleverly designed to obtain the
desired vacuum alignment. Another drawback of this approach is that the
predictability of a discrete flavor symmetry model could be degraded by the
possible higher dimensional operators. Moreover, flavor symmetry is usually
used to constrain the neutrino mixing angles while the neutrino masses are
undetermined except in some specific models.

In order to overcome the above mentioned drawbacks of the conventional finite
discrete flavor symmetry, a new approach of modular invariance playing the
role of flavor symmetry was recently proposed in [11]. Modular invariance has
a long history both in string and in field theories. The idea of modular
invariance as flavor symmetry to constrain the Yukawa couplings has been
naturally realized in string theory [12, 13, 14], D-brane compactification
[15, 16], magnetized extra dimensions [17, 18] and in orbifold
compactification [19, 20]. Modular invariance has been used to address several
aspects of the flavour problem in model building [21, 22, 23]. In the present
paper, we shall follow Ref. [11] to take a bottom-up approach based on
supersymmetric modular invariant theories. In the most economical version of
the modular invariant models, the flavon fields other than the modulus are not
needed and the flavor symmetry is uniquely broken by the vacuum expectation
value (VEV) of the complex modulus $\tau$. The complicated vacuum alignment is
not required although one needs some mechanism to fix the value of $\tau$. The
Yukawa couplings are functions of modular forms which are holomorphic
functions of $\tau$, and all higher dimensional operators in the
superpotential are completely determined by modular invariance in the limit of
supersymmetry [11]. It is notable that the neutrino mass models based on
modular invariance could involve only few coupling constants such that
neutrino masses and mixing parameters are correlated. This formalism has been
extended to consistently combine with the generalized CP symmetry, and the
consistency conditions requires that the modulus transforms as
$\tau\rightarrow-\tau^{*}$ up to modular transformations under the action of
CP symemtry [24, 25, 26, 27, 28, 29]. In a symmetric basis where the
representation matrices of both $S$ and $T$ are symmetric, the multiplets of
modular forms become complex conjugated under CP transformation if they are
properly normalized. As a consequence, the generalized CP symmetry would
constrain all the couplings in a modular invariant model to be real [24], and
thus the predictive power of such models is enhanced. It was noted that the
K$\ddot{\mathrm{a}}$hler potential is not at all fixed by the symmetries and
transformation properties of the models, although the superpotential is
completely fixed by the modular transformations. The corrections from the most
general K$\ddot{\mathrm{a}}$hler potential consistent with the symmetries of
the model could potentially reduce the predictive power of this formalism
[30].

The crucial element of this new approach is the modular forms of even weights
and level $N$ which can be arranged into irreducible representations of
inhomogeneous finite modular group $\Gamma_{N}$. Several models of lepton
masses and mixing have been constructed based on the finite modular groups
$\Gamma_{2}\cong S_{3}$ [31, 32, 33, 34], $\Gamma_{3}\cong A_{4}$ [11, 35, 31,
36, 37, 38, 32, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51],
$\Gamma_{4}\cong S_{4}$ [52, 53, 54, 55, 56, 57, 58, 48] and $\Gamma_{5}\cong
A_{5}$ [59, 60, 57]. This new approach has been extended to modular forms of
general integer weights which can be arranged into irreducible representations
of the homogeneous finite modular group $\Gamma^{\prime}_{N}$ [61]. Notice
that $\Gamma^{\prime}_{N}$ is the double covering of $\Gamma_{N}$. The modular
forms of weight 1 and level 3 have been explicitly constructed and shown to
furnish a two dimensional irreducible representation of
$\Gamma^{\prime}_{3}\cong T^{\prime}$ [61]. The phenomenological predictions
of modular symmetry models for leptogenesis have been discussed in [47, 58].
The SU(5) grand unified models with modular symmetry have been constructed
[37, 33]. The modular symmetry has also been applied to dark matter and
radiative neutrino mass models [40, 44, 45].

An interesting attempt to understand the dynamics of fermion mass generation
and flavor mixing is to assume some entries of the fermion mass matrices are
vanishing [62, 63, 64], and such scenarios are more popularly known as
texture-zero models. Usually abelian flavor symmetry is used to realize
texture zero structure exactly or approximately [65, 66]. Systematical and
complete studies of all possibilities have been performed for both the lepton
sector [67] and quark sector [68]. For recent review on texture zeros we refer
the reader to [69, 70]. The homogeneous finite modular group
$\Gamma^{\prime}_{N}$ is the double covering of the inhomogeneous finite
modular group $\Gamma_{N}$, and $\Gamma^{\prime}_{N}$ has twice as many
elements as $\Gamma_{N}$. Besides the irreducible representations of
$\Gamma_{N}$, $\Gamma^{\prime}_{N}$ has other new representations [61]. The
even weight modular forms of level $N$ can be arranged into the irreducible
representations of $\Gamma_{N}$ up to the automorphy factor [11], while the
odd weight modular forms of level $N$ are arranged into the new
representations of $\Gamma^{\prime}_{N}$ [61]. In this work, we shall show
that the above structure of modular forms can naturally produce texture zeros
of the fermion mass matrices if we properly assign the representations and
weights of the matter fields under the modular group. In this sense, the
modular invariance approach has the merits of both abelian flavor symmetry and
discrete non-abelian flavor symmetry.

In the present paper, we shall use the modular forms of level 3 to show
concrete examples. In addition to the representations $\mathbf{1}$,
$\mathbf{1}^{\prime}$, $\mathbf{1}^{\prime\prime}$ and $\mathbf{3}$ of
$\Gamma_{3}\cong A_{4}$, $\Gamma^{\prime}_{3}\cong T^{\prime}$ has three
doublet representations $\mathbf{2}$, $\mathbf{2}^{\prime}$ and
$\mathbf{2}^{\prime\prime}$. The even weight modular forms of level 3
transform as singlets and triplets of $\Gamma^{\prime}_{3}$ while the odd
weight modular forms of level 3 arrange into $\Gamma^{\prime}_{3}$ doublets.
We will perform a comprehensive analysis of possible texture zeros of the
quark mass matrices with the introduction of the $T^{\prime}$ modular
symmetry. Since the mass of the third generation quark is much heavier than
the first two generations, we shall assume that the three generations of the
left-handed quark fields transform as a doublet and a singlet under
$T^{\prime}$ modular symmetry, while the right-handed quark fields can be
assigned to a direct sum of $T^{\prime}$ doublet and singlet or they could be
three singlets of $T^{\prime}$.

This paper is structured as follows: in section 2 we briefly review the basic
aspects of modular symmetry and give the expressions of the modular forms of
level 3. In section 3, we present the possible structures and texture zeros of
the quark mass matrices if both left-handed and right-handed quark fields are
assigned to direct sum of singlet and double representations of $T^{\prime}$.
The texture zero structures of the quark mass matrices for singlet assignments
of right-handed quark fields are given in section 4. Furthermore, in section 5
we give five benchmark quark models which leads to up and down quark mass
matrices with texture zeros. These models contain only ten or eleven
independent real parameters, and they produce excellent fit to the data of
quark masses and CKM mixing matrix. In section 6 we include the lepton sector
to give a unified description of both quark and lepton masses and flavor
mixing simultaneously. The non-minimal Kähler potential and its contribution
in the quark sector are discussed in section 7, and we give most general form
of Kähler potential consistent with modular symmetry. Finally we conclude and
draw our conclusions in section 8. We give the representation matrices of the
generators of $T^{\prime}$ group and the Clebsch-Gordan (CG) coefficients in
our basis in the Appendix A. The modular forms of higher weights $k=5,6,7,8$
and level 3 are collected in the Appendix B. The non-minimal Kähler potential
in lepton sector is discussed in the Appendix C.

##  2 Modular symmetry and modular forms of level $N=3$

The modular group $\overline{\Gamma}$ can be regarded as the group of linear
fraction transformations acting on the complex modulus $\tau$ with
$\mathrm{Im}\tau>0$,

$\tau\rightarrow\gamma\tau=\frac{a\tau+b}{c\tau+d},~{}~{}a,b,c,d\in\mathbb{Z},~{}~{}ad-
bc=1\,.$ (1)

Obviously the linear fraction transformation $\frac{a\tau+b}{c\tau+d}$ is
identical to $\frac{-a\tau-b}{-c\tau-d}$. Therefore the modular group
$\overline{\Gamma}$ is isomorphic to $PSL(2,\mathbb{Z})=SL(2,Z)/\\{I,-I\\}$,
where $SL(2,\mathbb{Z})$ is the group of $2\times 2$ matrices with integer
entries and determinant 1, and $I$ refers to the two-dimensional unit matrix.
The modular group has infinite elements, and it can be generated by two
transformations $S$ and $T$,

$S:\tau\rightarrow-\frac{1}{\tau},~{}~{}~{}~{}~{}~{}T:\tau\rightarrow\tau+1\,,$
(2)

which fulfill the following relations

$S^{2}=(ST)^{3}=1$ (3)

and also $(TS)^{3}=1$ which is equivalent to $(ST)^{3}=1$ if $S^{2}=1$. Let
$N$ be a positive integer. The principal congruence subgroup of level $N$ is

$\Gamma(N)=\left\\{\left(\begin{array}[]{cc}a&b\\\ c&d\end{array}\right)\in
SL(2,\mathbb{Z}),~{}~{}\left(\begin{array}[]{cc}a&b\\\
c&d\end{array}\right)=\left(\begin{array}[]{cc}1{}&~{}0\\\
0{}&~{}1\end{array}\right)\,(\mathrm{mod~{}N})\right\\}\,,$ (4)

which is infinite normal subgroup of $SL(2,\mathbb{Z})$. It is obvious that
$\Gamma(1)\cong SL(2,\mathbb{Z})$. We define
$\overline{\Gamma}(N)=\Gamma(N)/\\{I,-I\\}$ for $N=1,2$ and
$\overline{\Gamma}(N)=\Gamma(N)$ for $N>2$ since the element $-I$ doesn’t
belong to $\Gamma(N)$ for $N>2$. Note $\overline{\Gamma}(1)\cong
PSL(2,\mathbb{Z})\equiv\overline{\Gamma}$. The inhomogeneous finite modular
groups are defined as the quotient groups
$\Gamma_{N}\equiv\overline{\Gamma}/\overline{\Gamma}(N)$. Because the element
$T^{N}$ belongs to $\Gamma(N)$, the finite modular group $\Gamma_{N}$ can be
generated by $S$ and $T$ obeying the relations [71]

$S^{2}=(ST)^{3}=T^{N}=1\,.$ (5)

The groups $\Gamma_{N}$ for $N=2,3,4,5$ are isomorphic to $S_{3}$, $A_{4}$,
$S_{4}$ and $A_{5}$ respectively. Additional relations besides these in Eq.
(5) are necessary in order to render the group finite for $N>5$. The
homogeneous finite modular groups are the quotient groups
$\Gamma^{\prime}_{N}\equiv SL(2,\mathbb{Z})/\Gamma(N)$, and they can generated
by three generators $S$, $T$ and $\mathbb{R}$ which satisfy [61]

$S^{2}=\mathbb{R},~{}~{}(ST)^{3}=T^{N}=\mathbb{R}^{2}=1,~{}~{}\mathbb{R}T=T\mathbb{R}$
(6)

and additional constraints should be imposed for $N>5$. The group
$\Gamma^{\prime}_{2}$ is isomorphic to $S_{3}$, and $\Gamma^{\prime}_{N}$ is
the double covering of $\Gamma_{N}$, and it has twice as many elements as
$\Gamma_{N}$. In the present work, we shall focus on $\Gamma^{\prime}_{3}$
which is the double covering of $A_{4}$ and is isomorphic to the binary
tetrahedral group $T^{\prime}$.

Modular forms of weight $k$ and level $N$ are holomorphic functions $f(\tau)$
which transforms under the action of $\Gamma(N)$ in the following way,

$f\left(\gamma\tau\right)=(c\tau+d)^{k}f(\tau),~{}~{}~{}~{}\gamma=\left(\begin{array}[]{cc}a&b\\\
c&d\end{array}\right)\in\Gamma(N)\,,$ (7)

where $k$ is a generic non-negative integer. The modular forms of weight $k$
and level $N$ span a linear space of finite dimension. As shown in [61], the
modular forms can be organized into some modular multiplets
$f_{\mathbf{r}}\equiv\left(f_{1}(\tau),f_{2}(\tau),...\right)^{T}$ which
transform as certain irreducible representation $\mathbf{r}$ of the finite
modular group $\Gamma^{\prime}_{N}$ [11, 61], i.e.

$f_{\mathbf{r}}(\gamma\tau)=(c\tau+d)^{k}\rho_{\mathbf{r}}(\gamma)f_{\mathbf{r}}(\tau)~{}~{}~{}\mathrm{for}~{}~{}\forall~{}\gamma\in
SL(2,\mathbb{Z})\,,$ (8)

where $\gamma$ is the representative element of the coset $\gamma\Gamma(N)$ in
$\Gamma^{\prime}_{N}$, and $\rho_{\mathbf{r}}(\gamma)$ is the representation
matrix of the element $\gamma$ in the irreducible representation $\mathbf{r}$.

###  2.1  Modular forms of level 3

The linear space of modular forms of weight $k$ and level 3 has dimension
$k+1$. The modular forms of level 3 has been constructed in terms of the
Dedekind eta function $\eta(\tau)$ [61]. There are two linearly independent
modular forms of the lowest weight 1 with $k=1$, and they can arrange into a
doublet $\mathbf{2}$ of $T^{\prime}$,

$Y^{(1)}_{\mathbf{2}}(\tau)\equiv\left(\begin{array}[]{c}Y^{(1)}_{\mathbf{2},1}(\tau)\\\
Y^{(1)}_{\mathbf{2},2}(\tau)\end{array}\right)=\Big{(}\sqrt{2}e^{i7\pi/12}\dfrac{\eta^{3}(3\tau)}{\eta(\tau)}\,,\quad\dfrac{\eta^{3}(3\tau)}{\eta(\tau)}-\dfrac{1}{3}\dfrac{\eta^{3}(\tau/3)}{\eta(\tau)}\Big{)}^{T}\,.$
(9)

Note the overall coefficient of $Y^{(1)}_{\mathbf{2}}$ can not be uniquely
determined. We shall also denote $Y^{(1)}_{\mathbf{2},1}(\tau)=Y_{1}(\tau)$
and $Y^{(1)}_{\mathbf{2},2}(\tau)=Y_{2}(\tau)$ for notation simplicity in the
following. The doublet modular forms $Y^{(1)}_{\mathbf{2},1}$ and
$Y^{(1)}_{\mathbf{2},2}$ have the following $q-$expansions,

$\displaystyle Y^{(1)}_{\mathbf{2},1}(\tau)$ $\displaystyle=$
$\displaystyle\sqrt{2}e^{7\pi
i/12}q^{1/3}(1+q+2q^{2}+2q^{4}+q^{5}+2q^{6}+q^{8}+2q^{9}+2q^{10}+\ldots),$
$\displaystyle Y^{(1)}_{\mathbf{2},2}(\tau)$ $\displaystyle=$ $\displaystyle
1/3+2q+2q^{3}+2q^{4}+4q^{7}+2q^{9}+2q^{12}+\ldots\,,$ (10)

with $q=e^{2\pi i\tau}$. The modular forms of higher weights can be
constructed from the tensor products of $Y^{(1)}_{\mathbf{2}}$. There are
three linearly independent weight 2 modular forms which can be arranged into a
$T^{\prime}$ triplet,

$Y^{(2)}_{\mathbf{3}}\equiv\begin{pmatrix}Y^{(2)}_{\mathbf{3},1}(\tau)\\\
Y^{(2)}_{\mathbf{3},2}(\tau)\\\
Y^{(2)}_{3,3}(\tau)\end{pmatrix}=\begin{pmatrix}e^{i\pi/6}Y^{2}_{2}\\\
\sqrt{2}e^{i7\pi/12}Y_{1}Y_{2}\\\ Y^{2}_{1}\end{pmatrix}$ (11)

At weight 3, we have four independent modular forms which can be decomposed
into two doublets transforming in the representations $\mathbf{2}$ and
$\mathbf{2}^{\prime\prime}$ of $T^{\prime}$,

$\displaystyle
Y^{(3)}_{\mathbf{2}}\equiv\begin{pmatrix}Y^{(3)}_{\mathbf{2},1}\\\
Y^{(3)}_{\mathbf{2},2}\end{pmatrix}=\begin{pmatrix}3e^{i\pi/6}Y_{1}Y^{2}_{2}\\\
\sqrt{2}e^{i5\pi/12}Y^{3}_{1}-e^{i\pi/6}Y^{3}_{2}\end{pmatrix}\,,$
$\displaystyle
Y^{(3)}_{\mathbf{2}^{\prime\prime}}\equiv\begin{pmatrix}Y^{(3)}_{\mathbf{2}^{\prime\prime},1}\\\
Y^{(3)}_{\mathbf{2}^{\prime\prime},2}\end{pmatrix}=\begin{pmatrix}Y^{3}_{1}+(1-i)Y^{3}_{2}\\\
-3Y_{2}Y^{2}_{1}\end{pmatrix}\,.$ (12)

The modular forms of weight 4 can be obtained from the contractions of
$Y^{(1)}_{\mathbf{2}}$ and $Y^{(3)}_{\mathbf{2}}$,
$Y^{(3)}_{\mathbf{2}^{\prime\prime}}$, and they decompose as
$\mathbf{3}\oplus\mathbf{1}\oplus\mathbf{1}^{\prime}$ under $T^{\prime}$,

$\displaystyle Y^{(4)}_{\mathbf{1}}=-4Y^{3}_{1}Y_{2}-(1-i)Y^{4}_{2}\,,$
$\displaystyle
Y^{(4)}_{\mathbf{1}^{\prime}}=\sqrt{2}e^{i5\pi/12}Y^{4}_{1}-4e^{i\pi/6}Y_{1}Y^{3}_{2}\,,$
$\displaystyle
Y^{(4)}_{\mathbf{3}}\equiv\begin{pmatrix}Y^{(4)}_{\mathbf{3},1}\\\
Y^{(4)}_{\mathbf{3},2}\\\
Y^{(4)}_{\mathbf{3},3}\end{pmatrix}=\begin{pmatrix}\sqrt{2}e^{i7\pi/12}Y^{3}_{1}Y_{2}-e^{i\pi/3}Y^{4}_{2}\\\
-Y^{4}_{1}-(1-i)Y_{1}Y^{3}_{2}\\\
3e^{i\pi/6}Y^{2}_{1}Y^{2}_{2}\end{pmatrix}\,.$ (13)

The analytical expressions of modular forms of weights 5, weight 6, weight 7
and weight 8 are reported in the Appendix B. The modular multiplets of level 3
at different weights are summarized in table 1. It is remarkable that the odd
weight modular forms always transform in the doublet irreducible
representations $\mathbf{2}$, $\mathbf{2}^{\prime}$ and
$\mathbf{2}^{\prime\prime}$ of $T^{\prime}$, while the even weight modular
forms arrange themselves into $T^{\prime}$ triplet $\mathbf{3}$ and singlets
$\mathbf{1}$, $\mathbf{1}^{\prime}$ and $\mathbf{1}^{\prime\prime}$ which are
identical with the representation matrices of $A_{4}$ in our working basis.
This interesting structure of modular forms can help to produce texture zeros
of the quark mass matrix, as we shall show in the following.

We shall formulate our models in the framework of $\mathcal{N}=1$ global
supersymmetry. In modular invariant theory, it is generally assumed that the
chiral supermultiplet $\Phi_{I}$ carries a modular weight $-k_{I}$ and
transforms according to certain representation $\rho_{I}$ of
$\Gamma^{\prime}_{N}$ [72, 73, 11],

$\tau\rightarrow\gamma\tau=\frac{a\tau+b}{c\tau+d},~{}~{}~{}\Phi_{I}\rightarrow(c\tau+d)^{-k_{I}}\rho_{I}(\gamma)\Phi_{I}$
(14)

In the present paper, we choose a minimal form of the Kähler potential,

$\mathcal{K}(\tau,\overline{\tau},\Phi_{I},\overline{\Phi}_{I})=-h\Lambda^{2}\log(-i\tau+i\overline{\tau})+\sum_{I}\frac{|\Phi_{I}|^{2}}{(-i\tau+i\overline{\tau})^{k_{I}}}$
(15)

which is invariant up to a Kähler transformation under the modular
transformations given in Eq. (14), and $h$ is a positive constant. It is the
leading order Kähler potential of general matter field $\Phi_{I}$ with modular
weight $-k_{I}$ in the context of string compactifications on abelian
orbifolds [74]. The Kähler potential is less constrained by modular symmetry,
at least the operators
$(-i\tau+i\overline{\tau})^{k-k_{I}}\left(\Phi^{\dagger}_{I}Y^{(k)\dagger}_{\mathbf{r}}Y^{(k)}_{\mathbf{r}}\Phi_{I}\right)_{\mathbf{1}}$
for any integer $k$ and irreducible representation $\mathbf{r}$ can not be
avoided, they can lead to off-diagonal elements of the Kähler metric and the
predictions for neutrino masses and mixing parameters can be changed
considerably [30]. As a consequence, in the bottom-up model construction based
on modular symmetry models [11], it is necessary to constrain the Kähler
potential better to enhance the predictive power. Motivated by top-down model
building in string theory, the modular flavor symmetry is extended to combine
with traditional flavor symmetry [25, 29, 75, 76]. In this new scheme, both
superpotential and Kähler potentials are strongly constrained, the non-
diagonal contributions to the Kähler metric are forbidden by all symmetries of
the theory. Therefore the extra terms depending on modular forms in the Kähler
potential don’t considerably alter the phenomenological predictions which have
been obtained by using just the standard Kähler potential in Eq. (15) [76].
Hence we shall mainly focus on the minimal canonical Kähler potential of Eq.
(15) in the present work, and the non-minimal Kähler potential would be
discussed in section 7 and Appendix C. The superpotential
$\mathcal{W}(\tau,\Phi_{I})$ can be expanded in power series of the
supermultiplets $\Phi_{I}$ as follows,

$\mathcal{W}(\tau,\Phi_{I})=\sum_{n}Y_{I_{1}\ldots
I_{n}}(\tau)\Phi_{I_{1}}\ldots\Phi_{I_{n}}\,,$ (16)

where the function $Y_{I_{1}\ldots I_{n}}(\tau)$ should be modular forms of
weight $k_{Y}$ and level $N$, and should transform in the representation
$\rho_{Y}$ of $\Gamma^{\prime}_{N}$,

$\tau\rightarrow\gamma\tau=\frac{a\tau+b}{c\tau+d},~{}~{}~{}~{}~{}Y_{I_{1}\ldots
I_{n}}(\tau)\rightarrow Y_{I_{1}\ldots
I_{n}}(\gamma\tau)=(c\tau+d)^{k_{Y}}\rho_{Y}(\gamma)Y_{I_{1}\ldots
I_{n}}(\tau)\,.$ (17)

The requirement that $\mathcal{W}$ is invariant under the modular
transformation entails $k_{Y}$ and $\rho_{Y}$ should fulfill the following
constraints

$k_{Y}=k_{I_{1}}+k_{I_{1}}+\ldots+k_{I_{n}},~{}~{}~{}\rho_{Y}\otimes\rho_{I_{1}}\otimes\ldots\rho_{I_{n}}\supset\mathbf{1}\,,$
(18)

where $\mathbf{1}$ denotes the invariant singlet representation of
$\Gamma^{\prime}_{N}$. In the following, we shall investigate the possible
texture zero structures of the quark mass matrix for the assignments that the
three generations of the right-handed quark fields transform as a doublet and
a singlet under $T^{\prime}$ modular symmetry or they are three singlets of
$T^{\prime}$.

Modular weight $k$ | Modular form $Y^{(k)}_{\mathbf{r}}$
---|---
$k=1$ | $Y^{(1)}_{\mathbf{2}}$
$k=2$ | $Y^{(2)}_{\mathbf{3}}$
$k=3$ | $Y^{(3)}_{\mathbf{2}},Y^{(3)}_{\mathbf{2}^{\prime\prime}}$
$k=4$ | $Y^{(4)}_{\mathbf{1}},Y^{(4)}_{\mathbf{1}^{\prime}},Y^{(4)}_{\mathbf{3}}$
$k=5$ | $Y^{(5)}_{\mathbf{2}},Y^{(5)}_{\mathbf{2}^{\prime}},Y^{(5)}_{\mathbf{2}^{\prime\prime}}$
$k=6$ | $Y^{(6)}_{\mathbf{1}},Y^{(6)}_{\mathbf{3}I},Y^{(6)}_{\mathbf{3}II}$
$k=7$ | $Y^{(7)}_{\mathbf{2}I},Y^{(7)}_{\mathbf{2}II},Y^{(7)}_{\mathbf{2}^{\prime}},Y^{(7)}_{\mathbf{2}^{\prime\prime}}$
$k=8$ | $Y^{(8)}_{\mathbf{1}},Y^{(8)}_{\mathbf{1}^{\prime}},Y^{(8)}_{\mathbf{1}^{\prime\prime}},Y^{(8)}_{\mathbf{3}I},Y^{(8)}_{\mathbf{3}II}$
Table 1: Summary of modular forms of level 3 up to weight 8, the subscript
$\mathbf{r}$ denote the transformation property under $T^{\prime}$ modular
symmetry. Here $Y^{(6)}_{\mathbf{3}I}$ and $Y^{(6)}_{\mathbf{3}II}$ stand for
two weight 6 modular forms transforming in the representation $\mathbf{3}$ of
$T^{\prime}$. Similar conventions are adopted for $Y^{(7)}_{\mathbf{2}I}$,
$Y^{(7)}_{\mathbf{2}II}$ and $Y^{(8)}_{\mathbf{3}I}$,
$Y^{(8)}_{\mathbf{3}II}$.

##  3 Quark mass matrices for doublet plus singlet assignments of the right-
handed quarks

In this case, both left-handed and right-handed quark fields transform as a
direct sum of one-dimensional and two-dimensional representations of the
$T^{\prime}$ modular group, i.e.

$Q_{D}\equiv\begin{pmatrix}Q_{1}\\\ Q_{2}\end{pmatrix}\sim\mathbf{2}^{i},\quad
Q_{3}\sim\mathbf{1}^{j}\,,~{}~{}~{}q^{c}_{D}\equiv\begin{pmatrix}q^{c}_{1}\\\
q^{c}_{2}\end{pmatrix}\sim\mathbf{2}^{k},\quad q^{c}_{3}\sim\mathbf{1}^{l}\,,$
(19)

where $i,j,k,l=0,1,2$ and we have denoted $\mathbf{1}\equiv\mathbf{1}^{0}$,
$\mathbf{1}^{{}^{\prime}}\equiv\mathbf{1}^{1}$,
$\mathbf{1}^{{}^{\prime\prime}}\equiv\mathbf{1}^{2}$ for singlet
representations and $\mathbf{2}\equiv\mathbf{2}^{0}$,
$\mathbf{2}^{{}^{\prime}}\equiv\mathbf{2}^{1}$,
$\mathbf{2}^{{}^{\prime\prime}}\equiv\mathbf{2}^{2}$ for the doublet
representations. The notations $Q_{i}$ and $q^{c}_{i}$ $(i=1,2,3)$ stand for
the left-handed and right-handed quark fields respectively, and $q^{c}_{i}$
can be either up type quark fields $u^{c}_{i}$ or down type quark fields
$d^{c}_{i}$. We denote the modular weights of $Q_{D}$, $Q_{3}$, $q^{c}_{D}$
and $q^{c}_{3}$ as $k_{Q_{D}}$, $k_{Q_{3}}$, $k_{q^{c}_{D}}$ and
$k_{q^{c}_{3}}$ respectively in the following, and the modular weights of the
Higgs doublets $H_{u,d}$ are assumed to be vanishing. Thus the most general
form of the superpotential for the quark masses is given by

$\mathcal{W}_{q}=q^{c}_{D}Q_{D}H_{u/d}f_{DD}(Y)+q^{c}_{D}Q_{3}H_{u/d}f_{D3}(Y)+q^{c}_{3}Q_{D}H_{u/d}f_{3D}(Y)+q^{c}_{3}Q_{3}H_{u/d}f_{33}(Y)\,,$
(20)

where $f_{DD}(Y)$, $f_{D3}(Y)$, $f_{3D}(Y)$ and $f_{33}(Y)$ are general
functions of modular forms, and their explicit forms depend on the group
indices $i$, $j$, $k$, $l$ and the modular weights of the quark fields. The
coupling constants in front of each term are neglected in Eq. (20). The Higgs
field in Eq. (20) is $H_{u}$ for up type quark fields $u^{c}$ and it is
$H_{d}$ for down type quark fields $d^{c}$. Each term of Eq. (20) should be a
singlet under $T^{\prime}$, and its modular weight should be zero. Moreover,
from Eq. (20) we see that the quark mass matrix can be divided into four
blocks as follow,

$M_{q}=\left(\begin{array}[]{ccc}~{}S{}&\vdots&~{}C{}\\\
\cdots&\cdots&\cdots\\\ ~{}R{}&\vdots&~{}T{}\\\ \end{array}\right)v_{u/d}\,,$
(21)

where $v_{u/d}$ is the vacuum expectation value (VEV) of the Higgs field
$H_{u/d}$, and the quark mass matrix $M_{q}$ is given in the right-left basis,
$q^{c}_{i}(M_{q})_{ij}Q_{j}$, $S$, $C$, $R$ and $T$ are $2\times 2$, $2\times
1$, $1\times 2$ and $1\times 1$ sub-matrices respectively. Notice that we can
assign the first generation or the second generation quark field instead of
the third generation to be a singlet under $T^{\prime}$ while the remaining
two generations transform as a $T^{\prime}$ doublet, the corresponding quark
mass matrix can be obtained by multiplying certain permutation matrices from
both sides. As a consequence, the results for the quark masses and mixing
matrix are not changed. Using the Kronecker products and the CG coefficients
of $T^{\prime}$ given in Appendix A, we can find out the explicit forms of
$S$, $C$, $R$ and $T$, and each matrix element can be expressed in terms of
modular form. We shall be concerned with modular forms up to weight 6 in the
present work, higher weight modular forms can be discussed in a similar way,
nevertheless more modular invariant interactions as well as more free
parameters would be involved.

###  3.1 General structure of $S$

This $2\times 2$ submatrix is determined by the representations
$\mathbf{2}^{i}$ and $\mathbf{2}^{k}$ as well as the modular weights
$k_{Q_{D}}$ and $k_{q^{c}_{D}}$. Modular invariance requires that $f_{DD}(Y)$
should be a modular form in the triplet representation $\mathbf{3}$ or singlet
representations $\mathbf{1}$, $\mathbf{1}^{\prime}$,
$\mathbf{1}^{\prime\prime}$ of $T^{\prime}$. Hence $S$ would be vanishing if
$k_{q^{c}_{D}}+k_{Q_{D}}$ is odd. As a consequence, the rank of the quark mass
matrix would be less than three and at least one quark is massless which is
disfavored by experimental data. For all possible assignments of
$\mathbf{2}^{i}$ and $\mathbf{2}^{k}$, we can find out the following possible
structures for the $2\times 2$ submatrix $S$.

  * •

$\mathbf{2}^{i}\otimes\mathbf{2}^{k}=\mathbf{2}\otimes\mathbf{2}=\mathbf{2}^{\prime}\otimes\mathbf{2}^{\prime\prime}=\mathbf{3}\oplus\mathbf{1}^{\prime}$

In this case, if the summation of the modular weights $k_{Q_{D}}$ and
$k_{q^{c}_{D}}$ is equal to 2, i.e. $k_{q^{c}_{D}}+k_{Q_{D}}=2$, $f_{DD}(Y)$
would be uniquely proportional to the weight two modular form
$Y^{(2)}_{\mathbf{3}}=(Y^{(2)}_{\mathbf{3},1},Y^{(2)}_{\mathbf{3},2},Y^{(2)}_{\mathbf{3},3})^{T}$
which couples to $q^{c}_{D}$ and $Q_{D}$ to form a modular invariant singlet.
From the CG coefficients of $T^{\prime}$ given in Appendix A, we can
straightforwardly read out the general form of $S$ as follows,

$S_{1}=\left(\begin{array}[]{cc}\kappa\sqrt{2}e^{\frac{i5\pi}{12}}Y^{(2)}_{\mathbf{3},2}&-\kappa
Y^{(2)}_{\mathbf{3},3}\\\ -\kappa
Y^{(2)}_{\mathbf{3},3}&\kappa\sqrt{2}e^{\frac{i7\pi}{12}}Y^{(2)}_{\mathbf{3},1}\\\
\end{array}\right)\,,$ (22)

which is a symmetric matrix, and $\kappa$ is in general a complex Yukawa
coupling parameter. In the case of $k_{q^{c}_{D}}+k_{Q_{D}}=4$, $f_{DD}(Y)$
would be the weight four modular form
$Y^{(4)}_{\mathbf{3}}=(Y^{(4)}_{\mathbf{3},1},Y^{(4)}_{\mathbf{3},2},Y^{(4)}_{\mathbf{3},3})^{T}$,
and the submatrix $S$ reads as

$S_{2}=\left(\begin{array}[]{cc}\kappa\sqrt{2}e^{\frac{i5\pi}{12}}Y^{(4)}_{\mathbf{3},2}&-\kappa
Y^{(4)}_{\mathbf{3},3}\\\ -\kappa
Y^{(4)}_{\mathbf{3},3}&\kappa\sqrt{2}e^{\frac{i7\pi}{12}}Y^{(4)}_{\mathbf{3},1}\\\
\end{array}\right)\,.$ (23)

We have two independent modular forms $Y^{(6)}_{\mathbf{3}I}$ and
$Y^{(6)}_{\mathbf{3}II}$ at weight six, as shown in table 1. Thus $f_{DD}(Y)$
is a linear combination of $Y^{(6)}_{\mathbf{3}I}$ and
$Y^{(6)}_{\mathbf{3}II}$ for $k_{q^{c}_{D}}+k_{Q_{D}}=6$, and we have

$S_{3}=\left(\begin{array}[]{cc}\sqrt{2}e^{\frac{i5\pi}{12}}(\kappa_{1}Y^{(6)}_{\mathbf{3}I,2}+\kappa_{2}Y^{(6)}_{\mathbf{3}II,2})&-(\kappa_{1}Y^{(6)}_{\mathbf{3}I,3}+\kappa_{2}Y^{(6)}_{\mathbf{3}II,3})\\\
-(\kappa_{1}Y^{(6)}_{\mathbf{3}I,3}+\kappa_{2}Y^{(6)}_{\mathbf{3}II,3})&\sqrt{2}e^{\frac{i7\pi}{12}}(\kappa_{1}Y^{(6)}_{\mathbf{3}I,1}+\kappa_{2}Y^{(6)}_{\mathbf{3}II,1})\\\
\end{array}\right)\,,$ (24)

where $\kappa_{1}$ and $\kappa_{2}$ are complex free parameters.

  * •

$\mathbf{2}^{i}\otimes\mathbf{2}^{k}=\mathbf{2}\otimes\mathbf{2}^{\prime}=\mathbf{2}^{\prime\prime}\otimes\mathbf{2}^{\prime\prime}=\mathbf{3}\oplus\mathbf{1}^{\prime\prime}$

In this case, $f_{DD}(Y)$ has to be modular form transforming as $\mathbf{3}$
or $\mathbf{1}^{\prime}$ in order to fulfill modular invariance. Hence
$f_{DD}(Y)$ is proportional to $Y^{(2)}_{\mathbf{3}}$ for the value
$k_{q^{c}_{D}}+k_{Q_{D}}=2$, and $S$ is of the following form

$S_{4}=\left(\begin{array}[]{cc}\kappa\sqrt{2}e^{\frac{i5\pi}{12}}Y^{(2)}_{\mathbf{3},1}&-\kappa
Y^{(2)}_{\mathbf{3},2}\\\ -\kappa
Y^{(2)}_{\mathbf{3},2}&\kappa\sqrt{2}e^{\frac{i7\pi}{12}}Y^{(2)}_{\mathbf{3},3}\\\
\end{array}\right)\,.$ (25)

For the value of $k_{q^{c}_{D}}+k_{Q_{D}}=4$, both $Y^{(4)}_{\mathbf{3}}$ and
$Y^{(4)}_{\mathbf{1}^{\prime}}$ can contribute to the quark mass terms, and
the submatrix $S$ is given by

$S_{5}=\left(\begin{array}[]{cc}\kappa_{1}\sqrt{2}e^{\frac{i5\pi}{12}}Y^{(4)}_{\mathbf{3},1}&\kappa_{2}Y^{(4)}_{\mathbf{1}^{\prime}}-\kappa_{1}Y^{(4)}_{\mathbf{3},2}\\\
-\kappa_{2}Y^{(4)}_{\mathbf{1}^{\prime}}-\kappa_{1}Y^{(4)}_{\mathbf{3},2}&\kappa_{1}\sqrt{2}e^{\frac{i7\pi}{12}}Y^{(4)}_{\mathbf{3},3}\\\
\end{array}\right)\,,$ (26)

which is not a symmetric matrix because of the term
$\kappa_{2}(q^{c}_{D}Q_{D})_{\mathbf{1}^{\prime\prime}}Y^{(4)}_{\mathbf{1}^{\prime}}H_{u/d}$.
Similarly for $k_{q^{c}_{D}}+k_{Q_{D}}=6$, both $Y^{(6)}_{\mathbf{3}I}$ and
$Y^{(6)}_{\mathbf{3}II}$ are relevant, and we find

$S_{6}=\left(\begin{array}[]{cc}\sqrt{2}e^{\frac{i5\pi}{12}}(\kappa_{1}Y^{(6)}_{\mathbf{3}I,1}+\kappa_{2}Y^{(6)}_{\mathbf{3}II,1})&-(\kappa_{1}Y^{(6)}_{\mathbf{3}I,2}+\kappa_{2}Y^{(6)}_{\mathbf{3}II,2})\\\
-(\kappa_{1}Y^{(6)}_{\mathbf{3}I,2}+\kappa_{2}Y^{(6)}_{\mathbf{3}II,2})&\sqrt{2}e^{\frac{i7\pi}{12}}(\kappa_{1}Y^{(6)}_{\mathbf{3}I,3}+\kappa_{2}Y^{(6)}_{\mathbf{3}II,3})\\\
\end{array}\right)\,.$ (27)

  * •

$\mathbf{2}^{i}\otimes\mathbf{2}^{k}=\mathbf{2}\otimes\mathbf{2}^{\prime\prime}=\mathbf{2}^{\prime}\otimes\mathbf{2}^{\prime}=\mathbf{3}\oplus\mathbf{1}$

If the modular weights satisfy $k_{q^{c}_{D}}+k_{Q_{D}}=2$, the mass term of
the first two generation quarks is given by
$\kappa(q^{c}_{D}Q_{D}Y^{(2)}_{\mathbf{3}})_{\mathbf{1}}H_{u/d}$ which gives
rise to

$S_{7}=\left(\begin{array}[]{cc}\kappa\sqrt{2}e^{\frac{i5\pi}{12}}Y^{(2)}_{\mathbf{3},3}&-\kappa
Y^{(2)}_{\mathbf{3},1}\\\ -\kappa
Y^{(2)}_{\mathbf{3},1}&\kappa\sqrt{2}e^{\frac{i7\pi}{12}}Y^{(2)}_{\mathbf{3},2}\\\
\end{array}\right)\,.$ (28)

For $k_{q^{c}_{D}}+k_{Q_{D}}=4$, $f_{DD}(Y)$ can be the weight 4 modular forms
$Y^{(4)}_{\mathbf{3}}$ and $Y^{(4)}_{\mathbf{1}}$. We can read out the general
form of the submatrix $S$ as follow,

$S_{8}=\left(\begin{array}[]{cc}\kappa_{1}\sqrt{2}e^{\frac{i5\pi}{12}}Y^{(4)}_{\mathbf{3},3}&\kappa_{2}Y^{(4)}_{\mathbf{1}}-\kappa_{1}Y^{(4)}_{\mathbf{3},1}\\\
-\kappa_{2}Y^{(4)}_{\mathbf{1}}-\kappa_{1}Y^{(4)}_{\mathbf{3},1}&\kappa_{1}\sqrt{2}e^{\frac{i7\pi}{12}}Y^{(4)}_{\mathbf{3},2}\\\
\end{array}\right)\,.$ (29)

For the value of $k_{q^{c}_{D}}+k_{Q_{D}}=6$, all the three weight 6 modular
forms $Y^{(6)}_{\mathbf{3}I}$, $Y^{(6)}_{\mathbf{3}II}$ and
$Y^{(6)}_{\mathbf{1}}$ are relevant, and $S$ is given by

$S_{9}=\left(\begin{array}[]{cc}\sqrt{2}e^{\frac{i5\pi}{12}}(\kappa_{1}Y^{(6)}_{\mathbf{3}I,3}+\kappa_{2}Y^{(6)}_{\mathbf{3}II,3})&-(\kappa_{1}Y^{(6)}_{\mathbf{3}I,1}+\kappa_{2}Y^{(6)}_{\mathbf{3}II,1})+\kappa_{3}Y^{(6)}_{\mathbf{1}}\\\
-(\kappa_{1}Y^{(6)}_{\mathbf{3}I,1}+\kappa_{2}Y^{(6)}_{\mathbf{3}II,1})-\kappa_{3}Y^{(6)}_{\mathbf{1}}&\sqrt{2}e^{\frac{i7\pi}{12}}(\kappa_{1}Y^{(6)}_{\mathbf{3}I,2}+\kappa_{2}Y^{(6)}_{\mathbf{3}II,2})\\\
\end{array}\right)\,.$ (30)

We summarized all the above possible forms of the submatrix $S$ for different
assignments of $\mathbf{2}^{i}$, $\mathbf{2}^{k}$ and the modular weights
$k_{Q_{D}}$, $k_{q^{c}_{D}}$ in table 2.

|  Expressions of $S$ | Constraints
---|---|---
$S_{0}$ | $\left(\begin{array}[]{cc}0&0\\\ 0&0\\\ \end{array}\right)$ | $k_{q^{c}_{D}}+k_{Q_{D}}<0$
$k_{q^{c}_{D}}+k_{Q_{D}}=0,1,3,5,\ldots$
$S_{1}$ | $\left(\begin{array}[]{cc}\kappa\sqrt{2}e^{\frac{i5\pi}{12}}Y^{(2)}_{\mathbf{3},2}&-\kappa Y^{(2)}_{\mathbf{3},3}\\\ -\kappa Y^{(2)}_{\mathbf{3},3}&\kappa\sqrt{2}e^{\frac{i7\pi}{12}}Y^{(2)}_{\mathbf{3},1}\\\ \end{array}\right)$ | $k_{q^{c}_{D}}+k_{Q_{D}}=2,~{}k+i=0\,(\text{mod}~{}3)$
$S_{2}$ | $\left(\begin{array}[]{cc}\kappa\sqrt{2}e^{\frac{i5\pi}{12}}Y^{(4)}_{\mathbf{3},2}&-\kappa Y^{(4)}_{\mathbf{3},3}\\\ -\kappa Y^{(4)}_{\mathbf{3},3}&\kappa\sqrt{2}e^{\frac{i7\pi}{12}}Y^{(4)}_{\mathbf{3},1}\\\ \end{array}\right)$ | $k_{q^{c}_{D}}+k_{Q_{D}}=4,~{}k+i=0\,(\text{mod}~{}3)$
$S_{3}$ | $\left(\begin{array}[]{cc}\sqrt{2}e^{\frac{i5\pi}{12}}(\kappa_{1}Y^{(6)}_{\mathbf{3}I,2}+\kappa_{2}Y^{(6)}_{\mathbf{3}II,2})&-(\kappa_{1}Y^{(6)}_{\mathbf{3}I,3}+\kappa_{2}Y^{(6)}_{\mathbf{3}II,3})\\\ -(\kappa_{1}Y^{(6)}_{\mathbf{3}I,3}+\kappa_{2}Y^{(6)}_{\mathbf{3}II,3})&\sqrt{2}e^{\frac{i7\pi}{12}}(\kappa_{1}Y^{(6)}_{\mathbf{3}I,1}+\kappa_{2}Y^{(6)}_{\mathbf{3}II,1})\\\ \end{array}\right)$ | $k_{q^{c}_{D}}+k_{Q_{D}}=6,~{}k+i=0\,(\text{mod}~{}3)$
$S_{4}$ | $\left(\begin{array}[]{cc}\kappa\sqrt{2}e^{\frac{i5\pi}{12}}Y^{(2)}_{\mathbf{3},1}&-\kappa Y^{(2)}_{\mathbf{3},2}\\\ -\kappa Y^{(2)}_{\mathbf{3},2}&\kappa\sqrt{2}e^{\frac{i7\pi}{12}}Y^{(2)}_{\mathbf{3},3}\\\ \end{array}\right)$ | $k_{q^{c}_{D}}+k_{Q_{D}}=2,~{}k+i=1\,(\text{mod}~{}3)$
$S_{5}$ | $\left(\begin{array}[]{cc}\kappa_{1}\sqrt{2}e^{\frac{i5\pi}{12}}Y^{(4)}_{\mathbf{3},1}&\kappa_{2}Y^{(4)}_{\mathbf{1}^{\prime}}-\kappa_{1}Y^{(4)}_{\mathbf{3},2}\\\ -\kappa_{2}Y^{(4)}_{\mathbf{1}^{\prime}}-\kappa_{1}Y^{(4)}_{\mathbf{3},2}&\kappa_{1}\sqrt{2}e^{\frac{i7\pi}{12}}Y^{(4)}_{\mathbf{3},3}\\\ \end{array}\right)$ | $k_{q^{c}_{D}}+k_{Q_{D}}=4,~{}k+i=1\,(\text{mod}~{}3)$
$S_{6}$ | $\left(\begin{array}[]{cc}\sqrt{2}e^{\frac{i5\pi}{12}}(\kappa_{1}Y^{(6)}_{\mathbf{3}I,1}+\kappa_{2}Y^{(6)}_{\mathbf{3}II,1})&-(\kappa_{1}Y^{(6)}_{\mathbf{3}I,2}+\kappa_{2}Y^{(6)}_{\mathbf{3}II,2})\\\ -(\kappa_{1}Y^{(6)}_{\mathbf{3}I,2}+\kappa_{2}Y^{(6)}_{\mathbf{3}II,2})&\sqrt{2}e^{\frac{i7\pi}{12}}(\kappa_{1}Y^{(6)}_{\mathbf{3}I,3}+\kappa_{2}Y^{(6)}_{\mathbf{3}II,3})\\\ \end{array}\right)$ | $k_{q^{c}_{D}}+k_{Q_{D}}=6,~{}k+i=1\,(\text{mod}~{}3)$
$S_{7}$ | $\left(\begin{array}[]{cc}\kappa\sqrt{2}e^{\frac{i5\pi}{12}}Y^{(2)}_{\mathbf{3},3}&-\kappa Y^{(2)}_{\mathbf{3},1}\\\ -\kappa Y^{(2)}_{\mathbf{3},1}&\kappa\sqrt{2}e^{\frac{i7\pi}{12}}Y^{(2)}_{\mathbf{3},2}\\\ \end{array}\right)$ | $k_{q^{c}_{D}}+k_{Q_{D}}=2,~{}k+i=2\,(\text{mod}~{}3)$
$S_{8}$ | $\left(\begin{array}[]{cc}\kappa_{1}\sqrt{2}e^{\frac{i5\pi}{12}}Y^{(4)}_{\mathbf{3},3}&\kappa_{2}Y^{(4)}_{\mathbf{1}}-\kappa_{1}Y^{(4)}_{\mathbf{3},1}\\\ -\kappa_{2}Y^{(4)}_{\mathbf{1}}-\kappa_{1}Y^{(4)}_{\mathbf{3},1}&\kappa_{1}\sqrt{2}e^{\frac{i7\pi}{12}}Y^{(4)}_{\mathbf{3},2}\\\ \end{array}\right)$ | $k_{q^{c}_{D}}+k_{Q_{D}}=4,~{}k+i=2\,(\text{mod}~{}3)$
$S_{9}$ | $\left(\begin{array}[]{cc}\sqrt{2}e^{\frac{i5\pi}{12}}(\kappa_{1}Y^{(6)}_{\mathbf{3}I,3}+\kappa_{2}Y^{(6)}_{\mathbf{3}II,3})&-(\kappa_{1}Y^{(6)}_{\mathbf{3}I,1}+\kappa_{2}Y^{(6)}_{\mathbf{3}II,1})+\kappa_{3}Y^{(6)}_{\mathbf{1}}\\\ -(\kappa_{1}Y^{(6)}_{\mathbf{3}I,1}+\kappa_{2}Y^{(6)}_{\mathbf{3}II,1})-\kappa_{3}Y^{(6)}_{\mathbf{1}}&\sqrt{2}e^{\frac{i7\pi}{12}}(\kappa_{1}Y^{(6)}_{\mathbf{3}I,2}+\kappa_{2}Y^{(6)}_{\mathbf{3}II,2})\\\ \end{array}\right)$ | $k_{q^{c}_{D}}+k_{Q_{D}}=6,~{}k+i=2\,(\text{mod}~{}3)$
Table 2: The possible structures of the submatrix $S$, where the left-handed
quark doublet $Q_{D}$ and the right-handed quark field $q^{c}_{D}$ are
assigned to transformed as $\mathbf{2}^{i}$ and $\mathbf{2}^{k}$ respectively
under $T^{\prime}$ modular symmetry, and their modular weights are denoted as
$k_{Q_{D}}$ and $k_{q^{c}_{D}}$ respectively.

###  3.2 General structures of $C$ and $R$

The submatrix $C$ comprises the (13) and (23) entries of the quark mass matrix
$M_{q}$, and it is determined by the modular form $f_{D3}(Y)$. Once the
assignments for $\mathbf{2}^{k}$, $\mathbf{1}^{j}$ and the modular weights
$k_{q^{c}_{D}}$ and $k_{Q_{3}}$ are specified, we can easily read out
$f_{D3}(Y)$ and the explicit form of $C$. The modular invariance requires that
$q^{c}_{D}$, $Q_{3}$ and $f_{D3}(Y)$ should contract into a $T^{\prime}$
singlet. From the Kronecker products
$\mathbf{1}^{a}\otimes\mathbf{2}^{b}=\mathbf{2}^{a+b~{}(\text{mod}~{}3)}$ in
Eq. (A.7), we know that $f_{D3}(Y)$ has to be a modular form transforming as
$\mathbf{2}^{2-a-b~{}(\text{mod}~{}3)}$. As shown in table 1, the modular
weights of the doublet modular forms must be odd. Therefore the sum
$k_{q^{c}_{D}}+k_{Q_{3}}$ should be an odd integer otherwise the submatrix $C$
would be zero. We find that $C$ is vanishing exactly if any of the following
conditions are fulfilled,

$\displaystyle k_{q^{c}_{D}}+k_{Q_{3}}=0,2,4,6,\ldots\,,$ or $\displaystyle
k_{q^{c}_{D}}+k_{Q_{3}}=1~{}\text{with}~{}\mathbf{2}^{k}\otimes\mathbf{1}^{j}=\mathbf{2}~{}\text{or}~{}\mathbf{2}^{\prime}\,,$
or $\displaystyle
k_{q^{c}_{D}}+k_{Q_{3}}=3~{}\text{with}~{}\mathbf{2}^{k}\otimes\mathbf{1}^{j}=\mathbf{2}^{\prime}\,.$
(31)

In the case that $C$ is non-vanishing, it can take the following nontrivial
forms.

  * •

$\mathbf{2}^{k}\otimes\mathbf{1}^{j}=\mathbf{2}$ for $k+j=0\,(\text{mod}~{}3)$

In this case, $f_{D3}(Y)$ should be modular form transforming as
$\mathbf{2}^{\prime\prime}$ under $T^{\prime}$. From table 1, we see that
modular forms in the representation $\mathbf{2}^{\prime\prime}$ appear at
weight 3 and weight 5. For $k_{q^{c}_{D}}+k_{Q_{3}}=3$, $f_{D3}(Y)$ is
proportional to
$Y^{(3)}_{\mathbf{2}^{\prime\prime}}\equiv(Y^{(3)}_{\mathbf{2}^{\prime\prime},1},Y^{(3)}_{\mathbf{2}^{\prime\prime},2})^{T}$,
and the corresponding structure of $C$ is given by

$C_{1}=(\kappa Y^{(3)}_{\mathbf{2}^{\prime\prime},2},~{}-\kappa
Y^{(3)}_{\mathbf{2}^{\prime\prime},1})^{T}\,.$ (32)

For $k_{q^{c}_{D}}+k_{Q_{3}}=5$, we can easily read out the submatrix $C$ as
follow,

$C_{2}=(\kappa Y^{(5)}_{\mathbf{2}^{\prime\prime},2},-\kappa
Y^{(5)}_{\mathbf{2}^{\prime\prime},1})^{T}\,.$ (33)

  * •

$\mathbf{2}^{k}\otimes\mathbf{1}^{j}=\mathbf{2}^{\prime}$ for
$k+j=1\,(\text{mod}~{}3)$

The modular form $f_{D3}(Y)$ should transform as $\mathbf{2}^{\prime}$ for
this assignment. As shown in table 1, the lowest weight modular form in the
doublet representation $\mathbf{2}^{\prime}$ is
$Y^{(5)}_{\mathbf{2}^{\prime}}=(Y^{(5)}_{\mathbf{2}^{\prime},1},Y^{(5)}_{\mathbf{2}^{\prime},2})^{T}$.
Accordingly the relevant quark mass term is
$(q^{c}_{D}Q_{3}Y^{(5)}_{\mathbf{2}^{\prime}})_{\mathbf{1}}H_{u/d}$ for
$k_{q^{c}_{D}}+k_{Q_{3}}=5$, and the block $C$ is of the following form,

$C_{3}=(\kappa Y^{(5)}_{\mathbf{2}^{\prime},2},-\kappa
Y^{(5)}_{\mathbf{2}^{\prime},1})^{T}\,.$ (34)

  * •

$\mathbf{2}^{k}\otimes\mathbf{1}^{j}=\mathbf{2}^{\prime\prime}$ for
$k+j=2\,(\text{mod}~{}3)$

A modular form transforming in the doublet representation $\mathbf{2}$ of
$T^{\prime}$ is necessary in order to form an invariant singlet in this case.
There are three modular forms
$Y^{(1)}_{\mathbf{2}}=(Y^{(1)}_{\mathbf{2},1},Y^{(1)}_{\mathbf{2},2})^{T}$,
$Y^{(3)}_{\mathbf{2}}=(Y^{(3)}_{\mathbf{2},1},Y^{(3)}_{\mathbf{2},2})^{T}$ and
$Y^{(5)}_{\mathbf{2}}=(Y^{(5)}_{\mathbf{2},1},Y^{(5)}_{\mathbf{2},2})^{T}$
transforming as $\mathbf{2}$. For the value of $k_{q^{c}_{D}}+k_{Q_{3}}=1$,
$Y^{(1)}_{\mathbf{2}}$ is involved, and we find the submatrix $C$ is

$C_{4}=(\kappa Y^{(1)}_{\mathbf{2},2},-\kappa Y^{(1)}_{\mathbf{2},1})^{T}\,.$
(35)

For $k_{q^{c}_{D}}+k_{Q_{3}}=3$, the modular form $f_{D3}(Y)$ is
$Y^{(3)}_{\mathbf{2}}$, and $C$ take the form

$C_{5}=(\kappa Y^{(3)}_{\mathbf{2},2},-\kappa Y^{(3)}_{\mathbf{2},1})^{T}\,.$
(36)

Similarly for $k_{q^{c}_{D}}+k_{Q_{3}}=5$, the submatrix $C$ reads as

$C_{6}=(\kappa Y^{(5)}_{\mathbf{2},2},-\kappa Y^{(5)}_{\mathbf{2},1})^{T}\,.$
(37)

In the exactly same fashion, we can fix the possible structures of the
submatrix $R$, and the results are summarized in table 3.

| Expressions of $C$ | Constraints |  | Expressions of $R$ | Constraints
---|---|---|---|---|---
$C_{0}$ | $(0,0)^{T}$ | $\textbf{i):}~{}k_{q^{c}_{D}}+k_{Q_{3}}<0,$ | $R_{0}$ | $(0,0)$ | $\textbf{i):}~{}k_{q^{c}_{3}}+k_{Q_{D}}<0,$
$\textbf{ii):}~{}k_{q^{c}_{D}}+k_{Q_{3}}=0,2,4,6,\ldots,$ | $\textbf{ii):}~{}k_{q^{c}_{3}}+k_{Q_{D}}=0,2,4,6,\ldots,$
$\textbf{iii):}~{}k_{q^{c}_{D}}+k_{Q_{3}}=1,~{}k+j=0,1\,(\text{mod}~{}3)\,,$ | $\textbf{iii):}~{}k_{q^{c}_{3}}+k_{Q_{D}}=1,~{}l+i=0,1\,(\text{mod}~{}3)\,,$
$\textbf{iv):}~{}k_{q^{c}_{D}}+k_{Q_{3}}=3,~{}k+j=1\,(\text{mod}~{}3)\,.$ | $\textbf{iv):}~{}k_{q^{c}_{3}}+k_{Q_{D}}=3,~{}l+i=1\,(\text{mod}~{}3)\,.$
$C_{1}$ | $(\kappa Y^{(3)}_{\mathbf{2}^{\prime\prime},2},-\kappa Y^{(3)}_{\mathbf{2}^{\prime\prime},1})^{T}$ | $k_{q^{c}_{D}}+k_{Q_{3}}=3,~{}~{}k+j=0\,(\text{mod}~{}3)$ | $R_{1}$ | $(\kappa Y^{(3)}_{\mathbf{2}^{\prime\prime},2},-\kappa Y^{(3)}_{\mathbf{2}^{\prime\prime},1})$ | $k_{q^{c}_{3}}+k_{Q_{D}}=3,~{}~{}l+i=0\,(\text{mod}~{}3)$
$C_{2}$ | $(\kappa Y^{(5)}_{\mathbf{2}^{\prime\prime},2},-\kappa Y^{(5)}_{\mathbf{2}^{\prime\prime},1})^{T}$ | $k_{q^{c}_{D}}+k_{Q_{3}}=5,~{}~{}k+j=0\,(\text{mod}~{}3)$ | $R_{2}$ | $(\kappa Y^{(5)}_{\mathbf{2}^{\prime\prime},2},-\kappa Y^{(5)}_{\mathbf{2}^{\prime\prime},1})$ | $k_{q^{c}_{3}}+k_{Q_{D}}=5,~{}~{}l+i=0\,(\text{mod}~{}3)$
$C_{3}$ | $(\kappa Y^{(5)}_{\mathbf{2}^{\prime},2},-\kappa Y^{(5)}_{\mathbf{2}^{\prime},1})^{T}$ | $k_{q^{c}_{D}}+k_{Q_{3}}=5,~{}~{}k+j=1\,(\text{mod}~{}3)$ | $R_{3}$ | $(\kappa Y^{(5)}_{\mathbf{2}^{\prime},2},-\kappa Y^{(5)}_{\mathbf{2}^{\prime},1})$ | $k_{q^{c}_{3}}+k_{Q_{D}}=5,~{}~{}l+i=1\,(\text{mod}~{}3)$
$C_{4}$ | $(\kappa Y^{(1)}_{\mathbf{2},2},-\kappa Y^{(1)}_{\mathbf{2},1})^{T}$ | $k_{q^{c}_{D}}+k_{Q_{3}}=1,~{}~{}k+j=2\,(\text{mod}~{}3)$ | $R_{4}$ | $(\kappa Y^{(1)}_{\mathbf{2},2},-\kappa Y^{(1)}_{\mathbf{2},1})$ | $k_{q^{c}_{3}}+k_{Q_{D}}=1,~{}~{}l+i=2\,(\text{mod}~{}3)$
$C_{5}$ | $(\kappa Y^{(3)}_{\mathbf{2},2},-\kappa Y^{(3)}_{\mathbf{2},1})^{T}$ | $k_{q^{c}_{D}}+k_{Q_{3}}=3,~{}~{}k+j=2\,(\text{mod}~{}3)$ | $R_{5}$ | $(\kappa Y^{(3)}_{\mathbf{2},2},-\kappa Y^{(3)}_{\mathbf{2},1})$ | $k_{q^{c}_{3}}+k_{Q_{D}}=3,~{}~{}l+i=2\,(\text{mod}~{}3)$
$C_{6}$ | $(\kappa Y^{(5)}_{\mathbf{2},2},-\kappa Y^{(5)}_{\mathbf{2},1})^{T}$ | $k_{q^{c}_{D}}+k_{Q_{3}}=5,~{}~{}k+j=2\,(\text{mod}~{}3)$ | $R_{6}$ | $(\kappa Y^{(5)}_{\mathbf{2},2},-\kappa Y^{(5)}_{\mathbf{2},1})$ | $k_{q^{c}_{3}}+k_{Q_{D}}=5,~{}~{}l+i=2\,(\text{mod}~{}3)$

Table 3: The structures of the submatrix $C$ and $R$ for different possible
values of modular weights and the assignments of the quark fields $q^{c}_{D}$,
$q^{c}_{3}$, $Q_{D}$, $Q_{3}$ under the finite modular group $T^{\prime}$.

###  3.3 General structure of $T$

$T$ is the (33) entry of the quark mass matrix, it would be non-zero if any of
the following three conditions are satisfied,

$\displaystyle\quad~{}~{}k_{q^{c}_{3}}+k_{Q_{3}}=0\,,~{}\text{with}~{}\mathbf{1}^{l}\otimes\mathbf{1}^{j}=\mathbf{1}\,,$
(38)
$\displaystyle\text{or}~{}~{}k_{q^{c}_{3}}+k_{Q_{3}}=4\,,~{}\text{with}~{}\mathbf{1}^{l}\otimes\mathbf{1}^{j}=\mathbf{1}~{}\text{or}~{}\mathbf{1}^{\prime\prime}\,,$
$\displaystyle\text{or}~{}~{}k_{q^{c}_{3}}+k_{Q_{3}}=6\,,~{}\text{with}~{}\mathbf{1}^{l}\otimes\mathbf{1}^{j}=\mathbf{1}\,.$

Considering the free coupling constant associated with the term
$q^{c}_{3}Q_{3}H_{u/d}f_{33}(Y)$, the above three cases essentially give the
same prediction for $T$. Moreover, the element $T$ would be exactly vanishing
for

$\displaystyle\quad~{}~{}k_{q^{c}_{3}}+k_{Q_{3}}<0\,,$ (39)
$\displaystyle\text{or}~{}~{}k_{q^{c}_{3}}+k_{Q_{3}}=1,2,3,5\,,$
$\displaystyle\text{or}~{}~{}k_{q^{c}_{3}}+k_{Q_{3}}=4\,,~{}\text{with}~{}\mathbf{1}^{l}\otimes\mathbf{1}^{j}=\mathbf{1}^{\prime}\,,$
$\displaystyle\text{or}~{}~{}k_{q^{c}_{3}}+k_{Q_{3}}=0,6\,,~{}\text{with}~{}\mathbf{1}^{l}\otimes\mathbf{1}^{j}=\mathbf{1}^{\prime}~{}\text{or}~{}\mathbf{1}^{\prime\prime}\,,$

up to weight 6 modular forms.

###  3.4 Possible structures of quark mass matrix

Combining the possible forms of the submatrices $S$, $C$, $R$ and $T$
summarized in table 2 and table 3 , we can straightforwardly obtain the quark
mass matrix. It is well established that no quark is massless, therefore we
only consider the quark mass matrices with non-vanishing determinant in this
paper. As a result, we find that the quark mass matrix can take the following
four possible structures with texture zeros,

$\displaystyle\text{Case
}\mathcal{A}:~{}~{}\left(\begin{array}[]{ccc}\times&\times&0\\\
\times&\times&0\\\ 0&0&\times\\\ \end{array}\right)\,,\quad\text{Case
}\mathcal{B}:~{}~{}\left(\begin{array}[]{ccc}\times&\times&\times\\\
\times&\times&\times\\\ 0&0&\times\\\ \end{array}\right)\,,$ (40)
$\displaystyle\text{Case
}\mathcal{C}:~{}~{}\left(\begin{array}[]{ccc}\times&\times&0\\\
\times&\times&0\\\ \times&\times&\times\\\
\end{array}\right)\,,\quad\text{Case
}\mathcal{D}:~{}~{}\left(\begin{array}[]{ccc}\times&\times&\times\\\
\times&\times&\times\\\ \times&\times&0\\\ \end{array}\right)\,,$

where a cross denotes a non-vanishing entry. It is remarkable that the non-
vanishing elements are correlated with each other in the present approach.

##  4 Quark mass matrices for singlet assignments of the right-handed quarks

In this section, we shall consider another case in which the three generations
of left-handed quarks are assigned to a direct sum of doublet and singlet of
$T^{\prime}$ while the three generations of right-handed quarks are assumed to
transform as one-dimensional representations of the $T^{\prime}$ modular
group, i.e.

$Q_{D}\equiv\begin{pmatrix}Q_{1}\\\ Q_{2}\end{pmatrix}\sim\mathbf{2}^{i},\quad
Q_{3}\sim\mathbf{1}^{j}\,,\quad
q^{c}_{a}\sim\mathbf{1}^{l_{a}}~{}~{}\textrm{with}~{}~{}a=1,2,3\,,$ (41)

where $i,j,l_{1,2,3}=0,1,2$ with $\mathbf{1}\equiv\mathbf{1}^{0}$,
$\mathbf{1}^{{}^{\prime}}\equiv\mathbf{1}^{1}$,
$\mathbf{1}^{{}^{\prime\prime}}\equiv\mathbf{1}^{2}$ for singlet
representations and $\mathbf{2}\equiv\mathbf{2}^{0}$,
$\mathbf{2}^{{}^{\prime}}\equiv\mathbf{2}^{1}$,
$\mathbf{2}^{{}^{\prime\prime}}\equiv\mathbf{2}^{2}$ for the doublet
representations. Thus the most general superpotential for the quark masses is
given by

$\mathcal{W}_{q}=\sum^{3}_{a=1}q^{c}_{a}Q_{D}H_{u/d}f_{aD}(Y)+q^{c}_{a}Q_{3}H_{u/d}f_{a3}(Y)\,,$
(42)

where we have suppressed all coupling constants. As a consequence, we can
divide the quark mass matrix $M_{q}$ into six parts as follow,

$M_{q}=\left(\begin{array}[]{cc}R^{\prime}_{1}{}&~{}C^{\prime}_{1}\\\
R^{\prime}_{2}{}&~{}C^{\prime}_{2}\\\
R^{\prime}_{3}{}&~{}C^{\prime}_{3}\end{array}\right)\,,$ (43)

where $R^{\prime}_{1,2,3}$ and $C^{\prime}_{1,2,3}$ are $1\times 2$ and
$1\times 1$ sub-matrices respectively, and they are determined by the modular
forms $f_{aD}(Y)$ and $f_{a3}(Y)$. We shall not discuss explicitly the case
where the left-handed quarks transform as three one-dimensional
representations of $T^{\prime}$ with the right-handed quark fields assigned to
a singlet and a doublet under $T^{\prime}$, since we only need to transpose
the mass matrix in Eq. (43) to switch the transformation properties of left-
handed and right-handed quarks.

###  4.1 General structures of $R^{\prime}_{a}$ and $C^{\prime}_{a}$

It is easy to say that modular invariance requires the the modular form
$f_{aD}(Y)$ should in the doublet representations
$\mathbf{2}^{2-l_{a}-i~{}(\text{mod}~{}3)}$ of $T^{\prime}$, and its modular
weight should be $k_{q^{c}_{a}}+k_{Q_{D}}$. The submatrix $R^{\prime}_{a}$
would be vanishing exactly, if any of the following relations is satisfied

$\displaystyle\quad~{}~{}k_{q^{c}_{a}}+k_{Q_{D}}=0,2,4,\ldots\,,$ (44)
$\displaystyle\text{or}~{}~{}k_{q^{c}_{a}}+k_{Q_{D}}=1,~{}~{}i+l_{a}=1,2\,(\text{mod}~{}3)\,,$
$\displaystyle\text{or}~{}~{}k_{q^{c}_{a}}+k_{Q_{D}}=3,~{}~{}i+l_{a}=1\,(\text{mod}~{}3)\,.$

Depending on the assignments $\mathbf{2}^{i}$, $\mathbf{1}^{l_{a}}$ and the
modular weights $k_{q^{c}_{a}}$, $k_{Q_{D}}$, $R^{\prime}_{a}$ can take the
following nontrivial forms.

  * •

$\mathbf{2}^{i}\otimes\mathbf{1}^{l_{a}}=\mathbf{2}$ with
$i+l_{a}=0\,(\text{mod}~{}3)$

In this case, $f_{aD}(Y)$ should be in the representation
$\mathbf{2}^{\prime\prime}$, and it can be either
$Y^{(3)}_{\mathbf{2}^{\prime\prime}}$ or $Y^{(5)}_{\mathbf{2}^{\prime\prime}}$
up to weight 6. For $k_{q^{c}_{a}}+k_{Q_{D}}=3$, we can read out the submatrix
$R^{\prime}_{a}$ as

$R^{\prime}_{a,1}=(\kappa Y^{(3)}_{\mathbf{2}^{\prime\prime},2},-\kappa
Y^{(3)}_{\mathbf{2}^{\prime\prime},1})\,.$ (45)

For $k_{q^{c}_{a}}+k_{Q_{D}}=5$, $R^{\prime}_{a}$ is of the form,

$R^{\prime}_{a,2}=(\kappa Y^{(5)}_{\mathbf{2}^{\prime\prime},2},-\kappa
Y^{(5)}_{\mathbf{2}^{\prime\prime},1})\,.$ (46)

  * •

$\mathbf{2}^{i}\otimes\mathbf{1}^{l_{a}}=\mathbf{2}^{\prime}$ with
$i+l_{a}=1~{}(\text{mod}~{}3)$

If we only consider the modular forms with modular weight less than seven,
then $f_{aD}(Y)$ would be proportional to $Y^{(5)}_{\mathbf{2}^{\prime}}$, and
$R^{\prime}_{a}$ takes the following form,

$R^{\prime}_{a,3}=(\kappa Y^{(5)}_{\mathbf{2}^{\prime},2},-\kappa
Y^{(5)}_{\mathbf{2}^{\prime},1})\,.$ (47)

  * •

$\mathbf{2}^{i}\otimes\mathbf{1}^{l_{a}}=\mathbf{2}^{\prime\prime}$ with
$i+l_{a}=2~{}(\text{mod}~{}3)$

The modular invariance requires that $f_{aD}(Y)$ should transform as the
doublet representation $\mathbf{2}$ of $T^{\prime}$. For
$k_{q^{c}_{a}}+k_{Q_{D}}=1$, the quark mass Yukawa term is $\kappa
q^{c}_{a}Q_{D}Y^{(1)}_{\mathbf{2}}H_{u/d}$ such that the submatrix
$R^{\prime}_{a}$ is given by

$R^{\prime}_{a,4}=(\kappa Y^{(1)}_{\mathbf{2},2},-\kappa
Y^{(1)}_{\mathbf{2},1})\,.$ (48)

For $k_{q^{c}_{a}}+k_{Q_{D}}=3$, we can read out $R^{\prime}_{a}$ as

$R^{\prime}_{a,5}=(\kappa Y^{(3)}_{\mathbf{2},2},-\kappa
Y^{(3)}_{\mathbf{2},1})\,.$ (49)

For $k_{q^{c}_{a}}+k_{Q_{D}}=5$, the weight 5 modular form
$Y^{(5)}_{\mathbf{2}}=(Y^{(5)}_{\mathbf{2},1},Y^{(5)}_{\mathbf{2},2})^{T}$ is
involved, and the submatrix $R^{\prime}_{a}$ is

$R^{\prime}_{a,6}=(\kappa Y^{(5)}_{\mathbf{2},2},-\kappa
Y^{(5)}_{\mathbf{2},1})\,.$ (50)

All the above possible forms of $R^{\prime}_{a}$ are summarized in table 4.

Since both $q^{c}_{a}$ and $Q_{3}$ are assigned to singlet representations
$\mathbf{1}^{l_{a}}$ and $\mathbf{1}^{j}$ respectively, the modular form
$f_{a3}(Y)$ in the last term of Eq. (42) should transform as singlet
$\mathbf{1}^{3-l_{a}-j~{}(\text{mod}~{}3)}$ under the $T^{\prime}$ modular
group. Analogous to previous cases, we find the element $C^{\prime}_{a}$ would
be non-zero for

$\displaystyle\quad~{}~{}k_{q^{c}_{a}}+k_{Q_{3}}=0,~{}~{}j+l_{a}=0~{}(\text{mod}~{}3)\,,$
(51)
$\displaystyle\text{or}~{}~{}k_{q^{c}_{a}}+k_{Q_{3}}=4,~{}~{}j+l_{a}=0,2~{}(\text{mod}~{}3)\,,$
$\displaystyle\text{or}~{}~{}k_{q^{c}_{a}}+k_{Q_{3}}=6,~{}~{}j+l_{a}=0~{}(\text{mod}~{}3)\,,$

otherwise $C^{\prime}_{a}$ would be vanishing exactly in particular when the
summation of modular weights $k_{q^{c}_{a}}+k_{Q_{3}}$ is an odd integer or
$k_{q^{c}_{a}}+k_{Q_{3}}<0$.

| Expressions of $R^{\prime}_{a}$ | Constraints
---|---|---
$R^{\prime}_{a,0}$ | $(0,0)$ | $\textbf{i):}~{}k_{q^{c}_{a}}+k_{Q_{D}}<0,$
$\textbf{ii):}~{}k_{q^{c}_{a}}+k_{Q_{D}}=0,2,4,6,\ldots,$
$\textbf{iii):}~{}k_{q^{c}_{a}}+k_{Q_{D}}=1,~{}i+l_{a}=0,1\,(\text{mod}~{}3)\,,$
$\textbf{iv):}~{}k_{q^{c}_{a}}+k_{Q_{D}}=3,~{}i+l_{a}=1\,(\text{mod}~{}3)\,.$
$R^{\prime}_{a,1}$ | $(\kappa Y^{(3)}_{\mathbf{2}^{\prime\prime},2},-\kappa Y^{(3)}_{\mathbf{2}^{\prime\prime},1})$ | $k_{q^{c}_{a}}+k_{Q_{D}}=3,~{}i+l_{a}=0\,(\text{mod}~{}3)$
$R^{\prime}_{a,2}$ | $(\kappa Y^{(5)}_{\mathbf{2}^{\prime\prime},2},-\kappa Y^{(5)}_{\mathbf{2}^{\prime\prime},1})$ | $k_{q^{c}_{a}}+k_{Q_{D}}=5,~{}i+l_{a}=0\,(\text{mod}~{}3)$
$R^{\prime}_{a,3}$ | $(\kappa Y^{(5)}_{\mathbf{2}^{\prime},2},-\kappa Y^{(5)}_{\mathbf{2}^{\prime},1})$ | $k_{q^{c}_{a}}+k_{Q_{D}}=5,~{}i+l_{a}=1\,(\text{mod}~{}3)$
$R^{\prime}_{a,4}$ | $(\kappa Y^{(1)}_{\mathbf{2},2},-\kappa Y^{(1)}_{\mathbf{2},1})$ | $k_{q^{c}_{a}}+k_{Q_{D}}=1,~{}i+l_{a}=2\,(\text{mod}~{}3)$
$R^{\prime}_{a,5}$ | $(\kappa Y^{(3)}_{\mathbf{2},2},-\kappa Y^{(3)}_{\mathbf{2},1})$ | $k_{q^{c}_{a}}+k_{Q_{D}}=3,~{}i+l_{a}=2\,(\text{mod}~{}3)$
$R^{\prime}_{a,6}$ | $(\kappa Y^{(5)}_{\mathbf{2},2},-\kappa Y^{(5)}_{\mathbf{2},1})$ | $k_{q^{c}_{a}}+k_{Q_{D}}=5,~{}i+l_{a}=2\,(\text{mod}~{}3)$
Table 4:  The structures of the submatrix $R^{\prime}_{a}$ for different
possible values of modular weights and the assignments of the quark fields
$q^{c}_{a}$, $Q_{D}$ under the finite modular group $T^{\prime}$.

###  4.2 Possible structures of quark mass matrix

Given the possible forms of $R^{\prime}_{a}$ listed in table 4, we find that
the quark mass matrix can take the following five possible structures with
texture zeros,

$\displaystyle\text{Case}~{}\mathcal{A}:~{}~{}\left(\begin{array}[]{ccc}\times&\times&0\\\
\times&\times&0\\\ 0&0&\times\\\
\end{array}\right)\,,~{}~{}\quad~{}~{}\text{Case}~{}\mathcal{B}:~{}~{}\left(\begin{array}[]{ccc}\times&\times&\times\\\
\times&\times&\times\\\ 0&0&\times\\\ \end{array}\right)\,,$ (58)
$\displaystyle\text{Case}~{}\mathcal{C}:~{}~{}\left(\begin{array}[]{ccc}\times&\times&0\\\
\times&\times&0\\\ \times&\times&\times\\\
\end{array}\right)\,,~{}~{}\quad~{}~{}\text{Case~{}}\mathcal{D}:~{}~{}\left(\begin{array}[]{ccc}\times&\times&\times\\\
\times&\times&\times\\\ \times&\times&0\\\ \end{array}\right)\,,$ (65)
$\displaystyle\text{Case}~{}\mathcal{E}:~{}~{}\left(\begin{array}[]{ccc}\times&\times&0\\\
\times&\times&\times\\\ 0&0&\times\\\ \end{array}\right)\,,$ (69)

up to row and column permutations. The symbol ”$\times$” denote nonzero matrix
element, and we have neglected the quark mass matrices with zero determinant
in Eq. (69). If the we interchange the assignments for the left-handed and
right-handed quark fields in Eq. (41), another new texture of quark mass
matrix can be obtained

$\text{Case}~{}\mathcal{F}:~{}~{}\left(\begin{array}[]{ccc}\times&\times&0\\\
\times&\times&0\\\ 0&\times&\times\\\ \end{array}\right)\,.$ (70)

##  5  Phenomenologically viable models for quark masses and CKM mixing

As shown in section 3 and section 4, both up quark and down quark mass
matrices can take six possible textures with zero elements:
$\text{Case}~{}\mathcal{A}$, $\text{Case}~{}\mathcal{B}$,
$\text{Case}~{}\mathcal{C}$, $\text{Case}~{}\mathcal{D}$,
$\text{Case}~{}\mathcal{E}$ and $\text{Case}~{}\mathcal{F}$ given in Eqs. (40,
69, 70), if we properly assign the modular weights of the quark fields and
their transformation properties under $T^{\prime}$. Combining the up quark
sector with the down quark sector, we can obtain the possible up quark and
down quark mass matrices predicted by $T^{\prime}$ modular symmetry. We find
that many cases can accommodate the experimental data on quark masses and CKM
mixing matrix, and the resulting predictions for quark mass matrices can be
classified according to the number of zero elements and the number of involved
free parameters. In order to show concrete examples, we shall present five
interesting models for quarks in the following.

| $Q_{D}$ | $Q_{3}$ | $u^{c}$ | $c^{c}$ | $t^{c}$ | $d^{c}_{D}\equiv(d^{c},s^{c})$ | $b^{c}$
---|---|---|---|---|---|---|---
$SU(2)_{L}\times U(1)_{Y}$ | $(2,1/6)$ | $(2,1/6)$ | $(1,-2/3)$ | $(1,-2/3)$ | $(1,-2/3)$ | $(1,1/3)$ | $(1,1/3)$
Model I | $T^{\prime}$ | $\mathbf{2}$ | $\mathbf{1}$ | $\mathbf{1}^{\prime\prime}$ | $\mathbf{1^{\prime}}$ | $\mathbf{1}^{\prime\prime}$ | $\mathbf{2}^{\prime}$ | $\mathbf{1}$
$k_{I}$ | $0$ | $-1$ | $3$ | $5$ | $5$ | $4$ | $1$
Model II | $T^{\prime}$ | $\mathbf{2}$ | $\mathbf{1}$ | $\mathbf{1}$ | $\mathbf{1}^{\prime\prime}$ | $\mathbf{1}^{\prime\prime}$ | $\mathbf{2}^{\prime}$ | $\mathbf{1}$
$k_{I}$ | $0$ | $-1$ | $3$ | $3$ | $5$ | $4$ | $1$
Model III | $T^{\prime}$ | $\mathbf{2}$ | $\mathbf{1}$ | $\mathbf{1}^{\prime\prime}$ | $\mathbf{1}^{\prime}$ | $\mathbf{1}$ | $\mathbf{2}^{\prime}$ | $\mathbf{1}^{\prime\prime}$
$k_{I}$ | $0$ | $-1$ | $3$ | $5$ | $1$ | $4$ | $5$
Model IV | $T^{\prime}$ | $\mathbf{2}$ | $\mathbf{1}$ | $\mathbf{1}$ | $\mathbf{1}^{\prime\prime}$ | $\mathbf{1}^{\prime\prime}$ |  $\mathbf{1}$ | $\mathbf{1}^{\prime\prime}$ | $\mathbf{1}$
$k_{I}$ | $0$ | $-1$ | $5$ | $5$ | $1$ |  $3$ | $3$ | $1$
Model V | $T^{\prime}$ | $\mathbf{2}$ | $\mathbf{1}$ | $\mathbf{1}$ | $\mathbf{1}^{\prime\prime}$ | $\mathbf{1}$ |  $\mathbf{1}$ | $\mathbf{1}^{\prime\prime}$ | $\mathbf{1}$
$k_{I}$ | $0$ | $-1$ | $5$ | $5$ | $1$ |  $3$ | $3$ | $1$

Table 5:  The transformation properties of the quark fields under the Standard
Model gauge group $SU(2)_{L}\times U(1)_{Y}$ and under $T^{\prime}$ modular
symmetry for different models, where $-k_{I}$ refers to the modular weights.
The two Higgs doublets $H_{u,d}$ are invariant under $T^{\prime}$ and their
modular weights are assumed to be vanishing.

Model I:

6 zero elements and 10 free parameters

The classification of the quark fields under the standard model gauge symmetry
and $T^{\prime}$ modular symmetry are listed in table 5. The quark $SU(2)$
doublets are assigned to doublet and singlet $Q_{D}\sim\mathbf{2}$,
$Q_{3}\sim\mathbf{1}$, the up type quark $SU(2)$ singlets $u^{c}$, $c^{c}$ and
$t^{c}$ are assigned to $\mathbf{1}^{\prime\prime}$, $\mathbf{1^{\prime}}$ and
$\mathbf{1}^{\prime\prime}$ respectively, and the right-handed down type
quarks are assumed to transform as doublet and singlet
$d^{c}_{D}\equiv(d^{c},s^{c})\sim\mathbf{2}^{\prime}$,
$b^{c}\sim\mathbf{1}^{\prime}$. Then we can read out the modular invariant
superpotentials for up and down sectors as follow,

$\displaystyle\mathcal{W}_{u}=y_{1}^{u}u^{c}Q_{D}Y^{(3)}_{\mathbf{2}}H_{u}+y_{2}^{u}c^{c}Q_{D}Y^{(5)}_{\mathbf{2}^{\prime}}H_{u}+y_{3}^{u}t^{c}Q_{D}Y^{(5)}_{\mathbf{2}}H_{u}+y_{4}^{u}t^{c}Q_{3}Y^{(4)}_{\mathbf{1}}H_{u}\,,$
(71)
$\displaystyle\mathcal{W}_{d}=y_{1}^{d}d^{c}_{D}Q_{D}Y^{(4)}_{\mathbf{3}}H_{d}+y_{2}^{d}d^{c}_{D}Q_{D}Y^{(4)}_{\mathbf{1}^{\prime}}H_{d}+y_{3}^{d}b^{c}Q_{3}H_{d}\,,$

where the coupling constant $y^{u}_{1,2,3,4}$ and $y^{d}_{1,3}$ can be taken
to be real positive by rephasing the quark fields without loss of generality,
while the phase of $y^{d}_{2}$ can not be removed. Applying the decomposition
rules of the $T^{\prime}$ tensor products in Appendix A, we find the quark
mass matrices are given by

$\displaystyle
M_{u}=\left(\begin{array}[]{ccc}y_{1}^{u}Y^{(3)}_{\mathbf{2},2}{}&-y_{1}^{u}Y^{(3)}_{\mathbf{2},1}{}&0\\\
y_{2}^{u}Y^{(5)}_{\mathbf{2}^{\prime},2}{}&-y_{2}^{u}Y^{(5)}_{\mathbf{2}^{\prime},1}{}&0\\\
y_{3}^{u}Y^{(5)}_{\mathbf{2},2}{}&-y_{3}^{u}Y^{(5)}_{\mathbf{2},1}{}&y_{4}^{u}Y^{(4)}_{\mathbf{1}}\\\
\end{array}\right)v_{u}\,,$ (75) $\displaystyle
M_{d}=\left(\begin{array}[]{ccc}\sqrt{2}e^{\frac{5i\pi}{12}}y^{d}_{1}Y^{(4)}_{\mathbf{3},1}{}&~{}-y^{d}_{1}Y^{(4)}_{\mathbf{3},2}+y^{d}_{2}Y^{(4)}_{\mathbf{1}^{\prime}}{}&~{}0\\\
-y^{d}_{1}Y^{(4)}_{\mathbf{3},2}-y^{d}_{2}Y^{(4)}_{\mathbf{1}^{\prime}}{}&~{}\sqrt{2}e^{\frac{7i\pi}{12}}y^{d}_{1}Y^{(4)}_{\mathbf{3},3}{}&~{}0\\\
0{}&~{}0{}&~{}y^{d}_{3}\\\ \end{array}\right)v_{d}\,.$ (79)

We see that there are totally six zero entries in $M_{u}$ and $M_{d}$, and the
down quark mass matrix $M_{d}$ is block diagonal. Apart form the dependence of
$M_{u}$ and $M_{d}$ on the VEV of the complex modulus $\tau$, we have six real
input parameters $y^{u}_{1,2,3,4}$, $y^{d}_{1,3}$ and one complex parameter
$y^{d}_{2}$ to describe the quark masses, mixing angles and phases.

Model II:

6 zero elements and 10 free parameters

The transformation rules of the left-handed quarks and the right-handed down
quarks under the $T^{\prime}$ modular symmetry are identical with those of
Model I while the assignments of the right-handed up quarks are different, as
shown in table 5. The superpotential for the quark Yukawa interactions is
given by

$\displaystyle\mathcal{W}_{u}=y_{1}^{u}u^{c}Q_{D}Y^{(3)}_{\mathbf{2}^{\prime\prime}}H_{u}+y_{2}^{u}c^{c}Q_{D}Y^{(3)}_{\mathbf{2}}H_{u}+y_{3}^{u}t^{c}Q_{D}Y^{(5)}_{\mathbf{2}}H_{u}+y_{4}^{u}t^{c}Q_{3}Y^{(4)}_{\mathbf{1^{\prime}}}H_{u}\,,$
(80)
$\displaystyle\mathcal{W}_{d}=y_{1}^{d}d^{c}_{D}Q_{D}Y^{(4)}_{\mathbf{3}}H_{d}+y_{2}^{d}d^{c}_{D}Q_{D}Y^{(4)}_{\mathbf{1}^{\prime}}H_{d}+y_{3}^{d}b^{c}Q_{3}H_{d}\,,$

where the phases of the couplings $y_{1,2,3,4}^{u}$ and $y^{d}_{1,3}$ can be
absorbed into the quark fields, while the phase of $y_{2}^{d}$ can not be
eliminated by field redefinition. The superpotential in Eq. (80) leads to the
following up and down quark mass matrices,

$\displaystyle
M_{u}=\left(\begin{array}[]{ccc}y_{1}^{u}Y^{(3)}_{\mathbf{2}^{\prime\prime},2}{}&-y_{1}^{u}Y^{(3)}_{\mathbf{2}^{\prime\prime},1}{}&0\\\
y_{2}^{u}Y^{(3)}_{\mathbf{2},2}{}&-y_{2}^{u}Y^{(3)}_{\mathbf{2},1}{}&0\\\
y_{3}^{u}Y^{(5)}_{\mathbf{2},2}{}&-y_{3}^{u}Y^{(5)}_{\mathbf{2},1}{}&y_{4}^{u}Y^{(4)}_{\mathbf{1^{\prime}}}\\\
\end{array}\right)v_{u}\,,$ (84) $\displaystyle
M_{d}=\left(\begin{array}[]{ccc}\sqrt{2}e^{\frac{5i\pi}{12}}y^{d}_{1}Y^{(4)}_{\mathbf{3},1}{}&~{}-y^{d}_{1}Y^{(4)}_{\mathbf{3},2}+y^{d}_{2}Y^{(4)}_{\mathbf{1}^{\prime}}{}&~{}0\\\
-y^{d}_{1}Y^{(4)}_{\mathbf{3},2}-y^{d}_{2}Y^{(4)}_{\mathbf{1}^{\prime}}{}&~{}\sqrt{2}e^{\frac{7i\pi}{12}}y^{d}_{1}Y^{(4)}_{\mathbf{3},3}{}&~{}0\\\
0{}&~{}0{}&~{}y^{d}_{3}\\\ \end{array}\right)v_{d}\,.$ (88)

which depends on ten real input parameters including the real and imaginary
part of the modulus $\tau$.

Model III:

6 zero elements and 10 free parameters

In this model, we assign the three generations of left-handed quark fields to
doublet $\mathbf{2}$ and singlet $\mathbf{1}$, the right-handed up type quarks
$u^{c}$, $c^{c}$ and $t^{c}$ transform as $\mathbf{1}^{\prime\prime}$,
$\mathbf{1}^{\prime}$ and $\mathbf{1}$ respectively while the three right-
handed down type quarks transform as
$\mathbf{2}^{\prime}\oplus\mathbf{1}^{\prime\prime}$. The quark masses are
described by

$\displaystyle\mathcal{W}_{u}=y_{1}^{u}u^{c}Q_{D}Y^{(3)}_{\mathbf{2}}H_{u}+y_{2}^{u}c^{c}Q_{D}Y^{(5)}_{\mathbf{2}^{\prime}}H_{u}+y_{3}^{u}t^{c}Q_{3}H_{u}\,,$
(89)
$\displaystyle\mathcal{W}_{d}=y_{1}^{d}d^{c}_{D}Q_{D}Y^{(4)}_{\mathbf{3}}H_{d}+y_{2}^{d}d^{c}_{D}Q_{D}Y^{(4)}_{\mathbf{1}^{\prime}}H_{d}+y_{3}^{d}b^{c}Q_{D}Y^{(5)}_{\mathbf{2}}H_{d}+y_{4}^{d}b^{c}Q_{3}Y^{(4)}_{\mathbf{1}^{\prime}}H_{d}\,,$

where all coupling constants except $y^{d}_{2}$ can be taken to be real by
using the freedom of field redefinition. We can read out the up and down quark
mass matrices as follow,

$M_{u}=\left(\begin{array}[]{ccc}y_{1}^{u}Y^{(3)}_{\mathbf{2},2}{}&~{}-y_{1}^{u}Y^{(3)}_{\mathbf{2},1}{}&~{}0\\\
y_{2}^{u}Y^{(5)}_{\mathbf{2}^{\prime},2}{}&~{}-y_{2}^{u}Y^{(5)}_{\mathbf{2}^{\prime},1}{}&~{}0\\\
0{}&~{}0{}&~{}y_{3}^{u}\end{array}\right)v_{u}\,,~{}~{}M_{d}=\left(\begin{array}[]{ccc}\sqrt{2}e^{\frac{5i\pi}{12}}y^{d}_{1}Y^{(4)}_{\mathbf{3},1}{}&~{}-y^{d}_{1}Y^{(4)}_{\mathbf{3},2}+y^{d}_{2}Y^{(4)}_{\mathbf{1}^{\prime}}{}&~{}0\\\
-y^{d}_{1}Y^{(4)}_{\mathbf{3},2}-y^{d}_{2}Y^{(4)}_{\mathbf{1}^{\prime}}{}&~{}\sqrt{2}e^{\frac{7i\pi}{12}}y^{d}_{1}Y^{(4)}_{\mathbf{3},3}{}&~{}0\\\
y^{d}_{3}Y^{(5)}_{\mathbf{2},2}{}&~{}-y^{d}_{3}Y^{(5)}_{\mathbf{2},1}{}&~{}y^{d}_{4}Y^{(4)}_{\mathbf{1}^{\prime}}\\\
\end{array}\right)v_{d}\,.$ (90)

We see that $M_{u}$ and $M_{d}$ are expressed in terms of eleven free input
parameters: $y^{u}_{1,2,3}$, $y^{d}_{1,3,4}$, $|y^{d}_{2}|$,
$\text{arg}(y^{d}_{2})$, $\text{Re}\tau$, $\text{Im}\tau$. The up quark mass
matrix is block diagonal in this model.

Model IV:

5 zero elements and 11 free parameters

The classification of the left-handed quark fields under the $T^{\prime}$
modular symmetry is the same as that of Model III, while both up type and down
type right-handed quark fields are assigned to transform as singlets of
$T^{\prime}$ , i.e., $u^{c}\sim\mathbf{1}$,
$c^{c}\sim\mathbf{1}^{\prime\prime}$, $t^{c}\sim\mathbf{1}^{\prime\prime}$,
$d^{c}\sim\mathbf{1}$, $s^{c}\sim\mathbf{1}^{\prime\prime}$ and
$b^{c}\sim\mathbf{1}$ . The superpotential for quark masses is of the
following form

$\displaystyle\mathcal{W}_{u}$ $\displaystyle=$ $\displaystyle
y_{1}^{u}u^{c}Q_{D}Y^{(5)}_{\mathbf{2}^{\prime\prime}}H_{u}+y_{2}^{u}c^{c}Q_{D}Y^{(5)}_{\mathbf{2}}H_{u}+y_{3}^{u}t^{c}Q_{D}Y^{(1)}_{\mathbf{2}}H_{u}+y_{4}^{u}u^{c}Q_{3}Y^{(4)}_{\mathbf{1}}H_{u}+y_{5}^{u}c^{c}Q_{3}Y^{(4)}_{\mathbf{1}^{\prime}}H_{u}\,,$
$\displaystyle\mathcal{W}_{d}$ $\displaystyle=$ $\displaystyle
y_{1}^{d}d^{c}Q_{D}Y^{(3)}_{\mathbf{2}^{\prime\prime}}H_{d}+y_{2}^{d}s^{c}Q_{D}Y^{(3)}_{\mathbf{2}}H_{d}+y_{3}^{d}b^{c}Q_{3}H_{d}\,,$
(91)

where all coefficients except $y^{u}_{5}$ can be taken to be real positive.
Using the decomposition rules of $T^{\prime}$ group in Appendix A, we obtain

$M_{u}=\left(\begin{array}[]{ccc}y_{1}^{u}Y^{(5)}_{\mathbf{2}^{\prime\prime},2}{}&~{}-y_{1}^{u}Y^{(5)}_{\mathbf{2}^{\prime\prime},1}{}&~{}y_{4}^{u}Y^{(4)}_{\mathbf{1}}\\\
y_{2}^{u}Y^{(5)}_{\mathbf{2},2}{}&~{}-y_{2}^{u}Y^{(5)}_{\mathbf{2},1}{}&~{}y_{5}^{u}Y^{(4)}_{\mathbf{1}^{\prime}}\\\
y_{3}^{u}Y^{(1)}_{\mathbf{2},2}{}&~{}-y_{3}^{u}Y^{(1)}_{\mathbf{2},1}{}&~{}0\\\
\end{array}\right)v_{u}\,,~{}~{}~{}~{}M_{d}=\left(\begin{array}[]{ccc}y^{d}_{1}Y^{(3)}_{\mathbf{2}^{\prime\prime},2}{}&~{}-y^{d}_{1}Y^{(3)}_{\mathbf{2}^{\prime\prime},1}{}&~{}0\\\
y^{d}_{2}Y^{(3)}_{\mathbf{2},2}{}&~{}-y^{d}_{2}Y^{(3)}_{\mathbf{2},1}{}&~{}0\\\
0{}&~{}0{}&~{}y^{d}_{3}\\\ \end{array}\right)v_{d}\,,$ (92)

which involve eleven free real parameters including the real and imaginary
parts of the modulus $\tau$.

Model V:

4 zero elements and 11 free parameters

In this model, the assignments of quark fields are almost same as those of
Model IV except for the transformation rule of $t^{c}$, as shown in table 5.
The modular invariant superpotential in the quark sector is

$\displaystyle\mathcal{W}_{u}$ $\displaystyle=$ $\displaystyle
y_{1}^{u}u^{c}Q_{D}Y^{(5)}_{\mathbf{2}^{\prime\prime}}H_{u}+y_{2}^{u}c^{c}Q_{D}Y^{(5)}_{\mathbf{2}}H_{u}+y_{3}^{u}u^{c}Q_{3}Y^{(4)}_{\mathbf{1}}H_{u}+y_{4}^{u}c^{c}Q_{3}Y^{(4)}_{\mathbf{1}^{\prime}}H_{u}+y_{5}^{u}t^{c}Q_{3}H_{u}\,,$
$\displaystyle\mathcal{W}_{d}$ $\displaystyle=$ $\displaystyle
y_{1}^{d}d^{c}Q_{D}Y^{(3)}_{\mathbf{2}^{\prime\prime}}H_{d}+y_{2}^{d}s^{c}Q_{D}Y^{(3)}_{\mathbf{2}}H_{d}+y_{3}^{d}b^{c}Q_{3}H_{d}\,,$
(93)

where the coupling constants $y^{u}_{1,2,3,5}$ and $y^{d}_{1,2,3}$ can be
taken to be real and positive without loss of generality while the phase of
$y^{u}_{4}$ can not be absorbed into quark fields. The up and down quark mass
matrices are given as

$M_{u}=\left(\begin{array}[]{ccc}y_{1}^{u}Y^{(5)}_{\mathbf{2}^{\prime\prime},2}{}&~{}-y_{1}^{u}Y^{(5)}_{\mathbf{2}^{\prime\prime},1}{}&~{}y_{3}^{u}Y^{(4)}_{\mathbf{1}}\\\
y_{2}^{u}Y^{(5)}_{\mathbf{2},2}{}&~{}-y_{2}^{u}Y^{(5)}_{\mathbf{2},1}{}&~{}y_{4}^{u}Y^{(4)}_{\mathbf{1}^{\prime}}\\\
0{}&~{}0{}&~{}y_{5}^{u}\\\
\end{array}\right)v_{u}\,,~{}~{}~{}~{}M_{d}=\left(\begin{array}[]{ccc}y^{d}_{1}Y^{(3)}_{\mathbf{2}^{\prime\prime},2}{}&~{}-y^{d}_{1}Y^{(3)}_{\mathbf{2}^{\prime\prime},1}{}&~{}0\\\
y^{d}_{2}Y^{(3)}_{\mathbf{2},2}{}&~{}-y^{d}_{2}Y^{(3)}_{\mathbf{2},1}{}&~{}0\\\
0{}&~{}0{}&~{}y^{d}_{3}\\\ \end{array}\right)v_{d}\,.$ (94)

###  5.1 Numerical results

Parameters | $\mu_{i}\pm 1\sigma$
---|---
$y_{u}/10^{-6}$ | $2.73325\pm 0.84731$
$y_{c}/10^{-3}$ | $1.41719\pm 0.04960$
$y_{t}$ | $0.50232\pm 0.01200$
$y_{d}/10^{-6}$ | $5.12495\pm 0.56374$
$y_{s}/10^{-4}$ | $1.01438\pm 0.05478$
$y_{b}/10^{-3}$ | $5.56096\pm 0.06103$
$y_{e}/10^{-6}$ | $2.07526\pm 0.01245$
$y_{\mu}/10^{-4}$ | $4.38107\pm 0.02629$
$y_{\tau}/10^{-3}$ | $7.48026\pm 0.03898$
$\theta_{12}^{q}$ | $0.22736\pm 0.00073$
$\theta_{13}^{q}/10^{-2}$ | $0.34938\pm 0.01258$
$\theta_{23}^{q}$ | $0.04015\pm 0.00064$
$\delta_{CP}^{q}/^{\circ}$ | $69.21330\pm 3.11460$
$\sin^{2}\theta_{12}^{l}$ | $0.310_{-0.012}^{+0.013}$
$\sin^{2}\theta_{23}^{l}$ | $0.563_{-0.024}^{+0.018}$
$\sin^{2}\theta_{13}^{l}$ | $0.02237_{-0.00065}^{+0.00066}$
$\delta_{CP}^{l}/^{\circ}$ | $221_{-28}^{+39}$
$\frac{\Delta m_{21}^{2}}{10^{-5}\text{eV}^{2}}$ | $7.39_{-0.20}^{+0.21}$
$\frac{\Delta m_{31}^{2}}{10^{-3}\text{eV}^{2}}$ | $2.528_{-0.031}^{+0.029}$
Table 6:  The best fit values $\mu_{i}$ and $1\sigma$ uncertainties of the
quark and lepton parameters when evolve to the GUT scale as calculated in
[77], with the SUSY breaking scale $M_{\text{SUSY}}=10$ TeV and
$\tan\beta=10$, where the error widths represent $1\sigma$ intervals. The
quark masses are given as $m_{u,c,t}=y_{u,c,t}v_{u}$ and
$m_{d,s,b}=y_{d,s,b}v_{d}$. The values of lepton mixing angles, leptonic Dirac
CP violation phases $\delta^{l}_{CP}$ and the neutrino mass squared difference
are taken from NuFIT 4.1 [1].

In previous work, the texture zero structure of the quark mass matrices is an
assumption or the zero entry is highly suppressed when imposing certain flavor
symmetry, and the mass matrix is assumed to be symmetric or hermitian. In the
present formalism, the texture zero is exactly dictated by the modular
symmetry, and the nonvaninshing entries are correlated. Therefore modular
symmetry provides a natural framework to realize texture zero. Moreover, it is
remarkable that the down quark mass matrix $M_{d}$ is block diagonal, the
(13), (23), (31) and (32) elements of $M_{d}$ are vanishing exactly for all
the five models summarized in table 5. As a consequence, the small off-
diagonal entries $V_{ub}$, $V_{cb}$, $V_{td}$ and $V_{ts}$ of the CKM matrix
completely arise from the up type quark sector.

The above predictions for up and down quark mass matrices $M_{u}$ and $M_{d}$
in Eqs. (79, 88, 90, 92, 94) are given at the scale where the modulus $\tau$
obtains the vacuum expectation value. Here we assume that the flavor symmetry
breaking scale is very large around the grand unified theory (GUT) scale
$2\times 10^{16}$ GeV. For each set of given values of the coupling constants
and the modulus $\tau$, we numerically diagonalize $M_{u}$ and $M_{d}$,
subsequently we extract the predictions for quark masses and CKM mixing matrix
which may be compared with their GUT scale values. It is well-known that the
quark masses and mixing parameters at GUT scale can be obtained from the
measured values at low energy experiments by performing renormalization group
evolution (RGE). The running of best-fit and error values to the GUT scale are
generally dependent on SUSY parameters, primarily depends on the SUSY breaking
scale $M_{\mathrm{SUSY}}$ and $\tan\beta$. In our numerical analysis, we shall
extract the GUT scale values of all Yukawa couplings and the CKM parameters
from [77], assuming a SUSY breaking scale $M_{\mathrm{SUSY}}=10$ TeV and
$\tan\beta=10$. The central values as well as the $1\sigma$ uncertainty ranges
of different observables used in our analysis are listed in table 6.

Following Ref. [11] and other papers in the literature, we shall treat the VEV
of the complex modulus $\tau$ as a free parameter. The modulus value is
expected to be dynamically fixed as a minimum of scalar potential in
supergravity theory. The modulus stabilization could be achieved in $SL(2,Z)$
modular invariant theories after the non-perturbative superpotential in
supergravity theory is taken into account [78, 79]. It has been shown that
modulus stabilization can also be realized in the bottom-up modular symmetry
models in a similar manner [46].

In order to determine the optimum values of the input parameters for which the
experimental data can be accommodated best, we define a $\chi^{2}$ function

$\chi^{2}=\sum^{n}_{i=1}\left(\frac{P_{i}(x_{1},x_{2},\ldots,x_{m})-\mu_{i}}{\sigma_{i}}\right)^{2}\,,$
(95)

where $P_{i}$ are predictions for the physical observables derived from the up
type and down type quark mass matrices $M_{u}$ and $M_{d}$ as complex
nonlinear functions of the free parameters of the model, $\mu_{i}$ and
$\sigma_{i}$ are the GUT scale central values and $1\sigma$ deviations
respectively of the corresponding quantities listed in table 6. The measured
values of the top and bottom quark masses can be reproduced exactly by
properly choosing the values of the overall parameters $y^{u}_{1}v_{u}$ and
$y^{d}_{1}v_{d}$. Hence we include the mass ratios $m_{u}/m_{c}$,
$m_{c}/m_{t}$, $m_{d}/m_{s}$ and $m_{s}/m_{b}$ instead of quark masses
$m_{u,c,t}$ and $m_{d,s,b}$ individually. We take the complex modulus $\tau$
as random complex number in the fundamental domain
$\\{\tau:|\text{Re}\tau|\leq 1/2,\text{Im}\tau>0,|\tau|\geq 1\\}$. The
absolute values of all coupling constants are scanned in the region
$[0,10^{6}]$ while the phases are freely varied in the range $[0,2\pi]$. The
function $\chi^{2}$ is numerically minimized by using the minimization
algorithms incorporated in the package MINUIT developed by CERN to determine
the best fit values of the input parameters. From the best fit values of the
input parameters, one can obtain the corresponding values of quark masses and
CKM mixing matrix.

Model I  | Input |  | $\text{Re}\tau$ | $\text{Im}\tau$ | $y^{u}_{2}/y^{u}_{1}$ | $y^{u}_{3}/y^{u}_{1}$ | $y^{u}_{4}/y^{u}_{1}$
---|---|---|---|---
0.03428 | 2.49711 | 3.05239 | 125.82400 | 759.06706
| $|y^{d}_{2}/y^{d}_{1}|$ | $\text{arg}(y^{d}_{2}/y^{d}_{1})/\pi$ | $y^{d}_{3}/y^{d}_{1}$ | $y_{1}^{u}v_{u}/\text{GeV}$ | $y_{1}^{d}v_{d}/\text{GeV}$
---|---|---|---|---
3.59552 | 1.13410 | 1.00599 | 6.59371 | 0.96242
Obs |  | $m_{u}/m_{c}$ | $m_{c}/m_{t}$ | $m_{d}/m_{s}$ | $m_{s}/m_{b}$ | $\theta_{12}^{q}$ | $\theta_{13}^{q}$ | $\theta_{23}^{q}$ | $\delta_{CP}^{q}/^{\circ}$
---|---|---|---|---|---|---|---
0.00197 | 0.00280 | 0.05007 | 0.01824 | 0.22732 | 0.00354 | 0.03890 | 70.20896
Model II  | Input |  | $\text{Re}\tau$ | $\text{Im}\tau$ | $y^{u}_{2}/y^{u}_{1}$ | $y^{u}_{3}/y^{u}_{1}$ | $y^{u}_{4}/y^{u}_{1}$
---|---|---|---|---
0.03504 | 2.45660 | 712.70240 | 82120.92932 | $7.67910\times 10^{6}$
| $|y^{d}_{2}/y^{d}_{1}|$ | $\text{arg}(y^{d}_{2}/y^{d}_{1})/\pi$ | $y^{d}_{3}/y^{d}_{1}$ | $y_{1}^{u}v_{u}/\text{GeV}$ | $y_{1}^{d}v_{d}/\text{GeV}$
---|---|---|---|---
3.27938 | 1.13321 | 1.00207 | 0.00932 | 0.96618
Obs |  | $m_{u}/m_{c}$ | $m_{c}/m_{t}$ | $m_{d}/m_{s}$ | $m_{s}/m_{b}$ | $\theta_{12}^{q}$ | $\theta_{13}^{q}$ | $\theta_{23}^{q}$ | $\delta_{CP}^{q}/^{\circ}$
---|---|---|---|---|---|---|---
0.00197 | 0.00282 | 0.04977 | 0.01830 | 0.22718 | 0.00356 | 0.03586 | 69.37967
Model III  | Input |  | $\text{Re}\tau$ | $\text{Im}\tau$ | $y^{u}_{2}/y^{u}_{1}$ | $y^{u}_{3}/y^{u}_{1}$ | $|y^{d}_{2}/y^{d}_{1}|$
---|---|---|---|---
0.03527 | 2.45148 | 2.85554 | 13.51464 | 3.23841
| $\text{arg}(y^{d}_{2}/y^{d}_{1})/\pi$ | $y^{d}_{3}/y^{d}_{1}$ | $y^{d}_{4}/y^{d}_{1}$ | $y_{1}^{u}v_{u}/\text{GeV}$ | $y_{1}^{d}v_{d}/\text{GeV}$
---|---|---|---|---
1.13453 | 8.91712 | 831.75015 | 6.47117 | 0.94256
Obs |  | $m_{u}/m_{c}$ | $m_{c}/m_{t}$ | $m_{d}/m_{s}$ | $m_{s}/m_{b}$ | $\theta_{12}^{q}$ | $\theta_{13}^{q}$ | $\theta_{23}^{q}$ | $\delta_{CP}^{q}/^{\circ}$
---|---|---|---|---|---|---|---
0.00223 | 0.00275 | 0.04968 | 0.01784 | 0.22729 | 0.00356 | 0.03558 | 69.02269
Model IV | Input |  | $\text{Re}\tau$ | $\text{Im}\tau$ | $y^{u}_{2}/y^{u}_{1}$ | $y^{u}_{3}/y^{u}_{1}$ | $y^{u}_{4}/y^{u}_{1}$ | $|y^{u}_{5}/y^{u}_{1}|$
---|---|---|---|---|---
0.04374 | 2.08683 | 235.73307 | 0.00453 | 94.95425 | 9006.45768
| $\text{arg}(y^{u}_{5}/y^{u}_{1})/\pi$ | $y^{d}_{2}/y^{d}_{1}$ | $y^{d}_{3}/y^{d}_{1}$ | $y_{1}^{u}v_{u}/\text{GeV}$ | $y_{1}^{d}v_{d}/\text{GeV}$
---|---|---|---|---
1.75886 | 27.41769 | 56.40063 | 3.65402 | 0.01717
Obs |  | $m_{u}/m_{c}$ | $m_{c}/m_{t}$ | $m_{d}/m_{s}$ | $m_{s}/m_{b}$ | $\theta_{12}^{q}$ | $\theta_{13}^{q}$ | $\theta_{23}^{q}$ | $\delta_{CP}^{q}/^{\circ}$
---|---|---|---|---|---|---|---
0.00191 | 0.00282 | 0.05024 | 0.01823 | 0.22740 | 0.00349 | 0.04037 | 70.55234
Model V  | Input |  | $\text{Re}\tau$ | $\text{Im}\tau$ | $y^{u}_{2}/y^{u}_{1}$ | $y^{u}_{3}/y^{u}_{1}$ | $|y^{u}_{4}/y^{u}_{1}|$ | $\text{arg}(y^{u}_{4}/y^{u}_{1})/\pi$
---|---|---|---|---|---
0.04369 | 2.08551 | 233.40867 | 94.45134 | 8951.18611 | 1.76267
| $y^{u}_{5}/y^{u}_{1}$ | $y^{d}_{2}/y^{d}_{1}$ | $y^{d}_{3}/y^{d}_{1}$ | $y_{1}^{u}v_{u}/\text{GeV}$ | $y_{1}^{d}v_{d}/\text{GeV}$
---|---|---|---|---
0.03654 | 27.38334 | 56.40088 | 3.66649 | 0.01717
Obs |  | $m_{u}/m_{c}$ | $m_{c}/m_{t}$ | $m_{d}/m_{s}$ | $m_{s}/m_{b}$ | $\theta_{12}^{q}$ | $\theta_{13}^{q}$ | $\theta_{23}^{q}$ | $\delta_{CP}^{q}/^{\circ}$
---|---|---|---|---|---|---|---
0.00192 | 0.00280 | 0.05030 | 0.01821 | 0.22731 | 0.00351 | 0.04011 | 70.97865

Table 7:  The best fit values of the input parameters, quark mass ratios and
the CKM parameters for the five models summarized in table 5, where Obs is the
abbreviation of observable. The fit is performed for the GUT scale
extrapolated data given in table 6. Notice that the measured values of the top
quark mass $m_{t}$ and bottom quark mass $m_{b}$ are reproduced exactly here.

As shown in section 5, Model I, Model II and Model III have ten free
parameters and the remaining Model IV and Model V have eleven free parameters.
We shall fit all the ten or eleven free parameters of each model using the ten
observables including six quark masses $m_{u,c,t}$, $m_{d,s,b}$, three quark
mixing angles $\theta^{q}_{12}$, $\theta^{q}_{13}$, $\theta^{q}_{23}$ and one
quark CP violation phase $\delta^{q}_{CP}$. The results of the fitting are
shown in table 7. We see that all the five models can give very good fit to
the data, almost all observables in quark sector lie within their $1\sigma$
ranges. When extending these models to include the lepton sector in the
following section, we fix the complex modulus $\tau$ to be the best fit values
in table 7 which are determined from the precisely measured quark masses and
CKM mixing matrix.

##  6 Quark lepton unification

Inspired by the success of $T^{\prime}$ modular symmetry in explaining the
quark masses and CKM mixing matrix, we shall extend this framework to the
lepton sector. Since the solar mixing angle $\theta^{l}_{12}$ and atmospheric
mixing angle $\theta^{l}_{23}$ are large, and the reactor mixing angle
$\theta^{l}_{13}$ is of the same order as the Cabibbo angle, the lepton mixing
matrix doesn’t have a hierarchical structure. Therefore we shall not use the
doublet representations of $T^{\prime}$ to distinguish the first two
generation leptons from the third generation. The three generations of left-
handed lepton doublets $L=(L_{1},L_{2},L_{3})^{T}$ are assigned to be in a
triplet $\mathbf{3}$ under $T^{\prime}$, while the right-handed charged
leptons $e^{c}$, $\mu^{c}$ and $\tau^{c}$ are assumed to transform as singlet
representations $\mathbf{1}$, $\mathbf{1}^{\prime}$ or
$\mathbf{1}^{\prime\prime}$ of $T^{\prime}$ modular group. For the irreducible
representations $\mathbf{3}$, $\mathbf{1}$, $\mathbf{1}^{\prime}$ and
$\mathbf{1}^{\prime\prime}$, two distinct elements of $T^{\prime}$ group are
described by the same matrices that represent the elements in $A_{4}$.
Therefore the group $T^{\prime}$ can not be distinguished from $A_{4}$ when
working with these representations. As a consequence, the flavor symmetry is
essentially the $A_{4}$ modular group in the lepton sector. A systematical
classification of lepton models with $A_{4}$ modular symmetry has been
performed in [43]. In the present work, we shall focus on two economical
models which are named as $\mathcal{L}_{1}$ and $\mathcal{L}_{2}$, the model
$\mathcal{L}_{2}$ is identical to the lepton model $\mathcal{D}_{6}$ in [43].
The transformation properties of the lepton chiral superfields and the right-
handed neutrinos $N^{c}=(N^{c}_{1},N^{c}_{2},N^{c}_{3})^{T}$ under
$T^{\prime}$ and their modular weights are summarized in table 8. Note that
the Higgs doublets $H_{u,d}$ transform trivially under $T^{\prime}$ with zero
modular weight.

| $L$ | $e^{c}$ | $\mu^{c}$ | $\tau^{c}$ | $N^{c}$
---|---|---|---|---|---
$SU(2)_{L}\times U(1)_{Y}$ | $(2,-1/2)$ | $(1,1)$ | $(1,1)$ | $(1,1)$ | $(1,0)$
$\mathcal{L}_{1}$ | $T^{\prime}$ | $\mathbf{3}$ | $\mathbf{1}$ | $\mathbf{1}^{\prime}$ | $\mathbf{1}^{\prime\prime}$ | $\mathbf{3}$
$k_{I}$ | $1$ | $1$ | $1$ | $3$ | $1$
$\mathcal{L}_{2}$ | $T^{\prime}$ | $\mathbf{3}$ | $\mathbf{1}^{\prime}$ | $\mathbf{1}^{\prime}$ | $\mathbf{1}$ | $\mathbf{3}$
$k_{I}$ | $1$ | $1$ | $3$ | $1$ | $1$
Table 8:  The transformation properties of the lepton chiral superfields under
the Standard Model gauge group $SU(2)_{L}\times U(1)_{Y}$ and under
$T^{\prime}$ modular symmetry for the models $\mathcal{D}_{6}$ and
$\mathcal{D}_{9}$ of [43], where $-k_{I}$ refers to the modular weights.

###  6.1 Lepton sector

In the following, we shall present the neutrino and charged lepton mass terms
and the corresponding lepton mass matrices for the models $\mathcal{L}_{1}$
and $\mathcal{L}_{2}$. The neutrino masses are assumed to be generated from
the type I seesaw mechanism. As listed in table 8, the right-handed neutrinos
$N^{c}$ compose a $T^{\prime}$ triplet $\mathbf{3}$.

  * •

$\mathcal{L}_{1}$ lepton model

In this case, the modular invariant superpotential in the lepton sector reads

$\displaystyle\mathcal{W}_{e}$ $\displaystyle=\alpha
e^{c}(LY^{(2)}_{\mathbf{3}})_{\mathbf{1}}H_{d}+\beta\mu^{c}(LY^{(2)}_{\mathbf{3}})_{\mathbf{1}^{\prime\prime}}H_{d}+\gamma\tau^{c}(LY^{(4)}_{\mathbf{3}})_{\mathbf{1}^{\prime}}H_{d}\,,$
(96a) $\displaystyle\mathcal{W}_{\nu}$
$\displaystyle=g_{1}((N^{c}L)_{\mathbf{3}_{S}}Y^{(2)}_{\mathbf{3}})_{\mathbf{1}}H_{u}+g_{2}((N^{c}L)_{\mathbf{3}_{A}}Y^{(2)}_{\mathbf{3}})_{\mathbf{1}}H_{u}+\Lambda((N^{c}N^{c})_{\mathbf{3}_{S}}Y^{(2)}_{\mathbf{3}})_{\mathbf{1}}\,.$
(96b)

The resulting charged lepton mass matrix $M_{e}$, the Dirac neutrino mass
matrix $M_{D}$ as well as the Majorana mass matrix $M_{N}$ for heavy neutrinos
are given by

$\displaystyle M_{e}=\left(\begin{array}[]{ccc}\alpha
Y^{(2)}_{\mathbf{3},1}{}&~{}\alpha Y^{(2)}_{\mathbf{3},3}{}&~{}\alpha
Y^{(2)}_{\mathbf{3},2}\\\ \beta Y^{(2)}_{\mathbf{3},3}&\beta
Y^{(2)}_{\mathbf{3},2}{}&~{}\beta Y^{(2)}_{\mathbf{3},1}\\\ \gamma
Y^{(4)}_{\mathbf{3},2}{}&~{}\gamma Y^{(4)}_{\mathbf{3},1}{}&~{}\gamma
Y^{(4)}_{\mathbf{3},3}\\\
\end{array}\right)v_{d}\,,~{}~{}~{}M_{N}=\left(\begin{array}[]{ccc}2Y^{(2)}_{\mathbf{3},1}{}&~{}-Y^{(2)}_{\mathbf{3},3}{}&~{}-Y^{(2)}_{\mathbf{3},2}\\\
-Y^{(2)}_{\mathbf{3},3}{}&~{}2Y^{(2)}_{\mathbf{3},2}{}&~{}-Y^{(2)}_{\mathbf{3},1}\\\
-Y^{(2)}_{\mathbf{3},2}{}&~{}-Y^{(2)}_{\mathbf{3},1}{}&~{}2Y^{(2)}_{\mathbf{3},3}\\\
\end{array}\right)\Lambda\,,$ (103) $\displaystyle
M_{D}=\left(\begin{array}[]{ccc}2g_{1}Y^{(2)}_{\mathbf{3},1}{}&~{}(-g_{1}+g_{2})Y^{(2)}_{\mathbf{3},3}{}&~{}(-g_{1}-g_{2})Y^{(2)}_{\mathbf{3},2}\\\
(-g_{1}-g_{2})Y^{(2)}_{\mathbf{3},3}{}&~{}2g_{1}Y^{(2)}_{\mathbf{3},2}{}&~{}(-g_{1}+g_{2})Y^{(2)}_{\mathbf{3},1}\\\
(-g_{1}+g_{2})Y^{(2)}_{\mathbf{3},2}{}&~{}(-g_{1}-g_{2})Y^{(2)}_{\mathbf{3},1}{}&~{}2g_{1}Y^{(2)}_{\mathbf{3},3}\end{array}\right)v_{u}\,.$
(107)

The light neutrino mass matrix is given by the seesaw formula
$m_{\nu}=-M^{T}_{D}M^{-1}_{N}M_{D}$. The phases of the parameters $\alpha$,
$\beta$ and $\gamma$ can be absorbed into the right-handed charged lepton
fields, and the measured charged lepton masses can be reproduced by adjusting
their values. The light neutrino mass matrix $m_{\nu}$ only depends on the
complex parameter $g_{2}/g_{1}$ and modulus $\tau$ besides the overall scale
$g^{2}_{1}v^{2}_{u}/\Lambda$. It is found that the modulus $\tau$ lies in
narrow regions in order to be compatible with the experimental data on lepton
mixing angles and neutrino masses [43].

  * •

$\mathcal{L}_{2}$ lepton model

From table 8, we see that the models $\mathcal{L}_{1}$ and $\mathcal{L}_{2}$
differ in the representation assignments for the right-handed charged leptons,
while the transformation properties of $L$ and $N^{c}$ are exactly the same in
the two models. The superpotential for the charged lepton masses is given by

$\mathcal{W}_{e}=\alpha
e^{c}(LY^{(2)}_{\mathbf{3}})_{\mathbf{1}^{\prime\prime}}H_{d}+\beta\mu^{c}(LY^{(4)}_{\mathbf{3}})_{\mathbf{1}^{\prime\prime}}H_{d}+\gamma\tau^{c}(LY^{(2)}_{\mathbf{3}})_{\mathbf{1}}H_{d}\,,$
(108)

which leads to the following charged lepton mass matrix

$M_{e}=\left(\begin{array}[]{ccc}\alpha Y^{(2)}_{\mathbf{3},3}{}&~{}\alpha
Y^{(2)}_{\mathbf{3},2}{}&~{}\alpha Y^{(2)}_{\mathbf{3},1}\\\ \beta
Y^{(4)}_{\mathbf{3},3}{}&~{}\beta Y^{(4)}_{\mathbf{3},2}{}&~{}\beta
Y^{(4)}_{\mathbf{3},1}\\\ \gamma Y^{(2)}_{\mathbf{3},1}{}&~{}\gamma
Y^{(2)}_{\mathbf{3},3}{}&~{}\gamma
Y^{(2)}_{\mathbf{3},2}\end{array}\right)\,.$ (109)

The superpotential $\mathcal{W}_{\nu}$ for neutrino sector coincides with that
of Eq. (96b), and the relevant neutrino mass matrices $M_{D}$ and $M_{N}$ are
given by Eq. (107).

###  6.2 Numerical results

We can combine the benchmark quark models Model I, Model II, Model III, Model
IV and Model V presented in section 5 with the $\mathcal{L}_{1}$ and
$\mathcal{L}_{2}$ lepton models to give a unified description of both quark
and lepton sectors. Notice that the modulus $\tau$ in the quark and lepton
mass matrices should be identical, and we transmit the best fit value of
$\tau$ in table 7 obtained from quark sector to the lepton sector. We perform
a global fit to the complete models including both quark and lepton sectors,
the models would be viable if the observed quark and lepton masses and mixing
parameters can be accommodated for certain values of input parameters. It is
remarkable that the resulting models have less free input parameters than the
number of observable quantities including quark and lepton masses and mixing
parameters. Hence it is highly nontrivial that the model can successfully fit
the data. In particular, the light neutrino mass matrix would only depend on a
complex parameter $g_{2}/g_{1}$ and a overall factor
$g^{2}_{1}v^{2}_{u}/\Lambda$. Consequently three light neutrino masses, three
lepton mixing angles and three CP violation phases are pinned down by three
real input parameters $|g_{2}/g_{1}|$, $\text{arg}(g_{2}/g_{1})$ and
$g^{2}_{1}v^{2}_{u}/\Lambda$. Thus the models could make six non-trivial
predictions which agree well with the present data. The extrapolated values of
the charged lepton masses at the GUT scale are listed in table 6 for
$\tan\beta=10$ and $M_{\text{SUSY}}=10$ TeV. For the neutrino masses and
mixing angles, we use the latest results of the global data analysis from
NuFIT 4.1 [1], the RGE effects from the low scale to the GUT scale are
neglected, since the running of the neutrino masses and mixing angles in the
MSSM is known to be negligible in the case of $\tan\beta\leq 30$. It is found
that the RGE dependence can really be safely neglected in a sizable region of
the $\tan\beta-M_{SUSY}$ plane [35].

Since the global analysis of neutrino oscillation data favors normal mass
ordering over inverted ordering [1], we shall assume normal ordering neutrino
masses in the $\chi^{2}$ analysis. Similar to section 5.1, the absolute value
of each coupling constant freely varies between $0$ and $10^{6}$, all phases
are treated as random numbers in the range of $0$ and $2\pi$, and the complex
modulus $\tau$ is scanned in the fundamental domain of the modular group. The
results of the fit are shown in table 9. We see that very good fit to the data
is obtained, and all observables from the model fall well into the
experimentally allowed regions. From the fitted values of the input
parameters, we can further obtain predictions for the unmeasured observables
such as the Dirac CP phase $\delta^{l}_{CP}$ and Majorana CP phase
$\alpha_{21}$ and $\alpha_{31}$ in the lepton sector, the lightest neutrino
mass $m_{1}$, and the effective Majorana mass $|m_{ee}|$ in neutrinoless
double beta decay, as shown in table 9.

As an example, we consider the scenario that the Model I in quark sector is
combined with the lepton model $\mathcal{D}_{6}$. We have 16 real input
parameters $|y^{u}_{1,2,3,4}|$, $|y^{d}_{1,3}|$, $|y^{d}_{2}|$,
$\text{arg}(y^{d}_{2})$, $|\alpha|$, $|\beta|$, $|\gamma|$, $|g_{1}|$,
$|g_{2}|$, $\mathrm{arg}(g_{2})$, $\mathrm{Re}\tau$ and $\mathrm{Im}\tau$ to
describe the 18 measured quantities including the quark masses $m_{u,c,t}$ and
$m_{d,s,b}$, the CKM mixing parameters $\theta^{q}_{12}$, $\theta^{q}_{13}$,
$\theta^{q}_{23}$ and $\delta^{q}_{CP}$, the charged lepton masses
$m_{e,\mu,\tau}$, the neutrino mass-squared splittings $\Delta m^{2}_{21}$ and
$\Delta m^{2}_{31}$, and the lepton mixing angles $\theta^{l}_{12}$,
$\theta^{l}_{13}$ and $\theta^{l}_{23}$. From table 6, we see that the charged
fermion masses and the CKM mixing matrix has been measured very precisely and
their $1\sigma$ ranges are quite narrow. Hence the input parameters are also
constrained to be in very narrow regions to accommodate the experimental data.
We find that the allowed regions of $\tau$ by the experimental data of quark
and lepton sectors have overlapping area and excellent agreement with the
measured values of the observables in table 6 can be achieved, as shown in
figure 1. In order to make the common region of $\tau$ visible, we require
that the quark mass ratios $y_{c}/y_{t}$, $y_{d}/y_{s}$, $y_{s}/y_{b}$ and the
CP violating phase $\delta^{q}_{CP}$ lie in the $2\sigma$ intervals, the quark
mixing angles $\theta^{q}_{12}$, $\theta^{q}_{13}$, $\theta^{q}_{23}$ and the
charged lepton mass ratio $y_{e}/y_{\mu}$, $y_{\mu}/y_{\tau}$ in the $5\sigma$
intervals, and the three neutrino mixing angles and neutrino masses squared
differences in their $3\sigma$ ranges. It is notable that all the quark and
lepton mixing parameters are predicted to lie in narrow regions. In
particular, the atmospheric mixing angle $\theta_{23}$ is in the second
octant, and the lepton Dirac CP phase $\delta^{l}_{CP}$ is determined to be
around $3\pi/2$.

Model I+$\mathcal{L}_{1}$ | Input |  | $\beta/\alpha$ | $\gamma/\alpha$ | $|g_{2}/g_{1}|$ | $\text{arg}(g_{2}/g_{1})/\pi$ | $\alpha v_{d}/\text{GeV}$ | $(g_{1}^{2}v_{u}^{2}/\Lambda)/\text{eV}$
---|---|---|---|---|---
3600.66055 | 1898.95134 | 1.11486 | 0.37185 | 0.00325 | 0.58722
Obs  |  | $m_{e}/m_{\mu}$ | $m_{\mu}/m_{\tau}$ | $m_{1}/\text{eV}$ | $m_{2}/\text{eV}$ | $m_{3}/\text{eV}$ | $|m_{ee}|/\text{eV}$
---|---|---|---|---|---
0.00474 | 0.05857 | 0.13019 | 0.13048 | 0.13955 | 0.13035
| $\sin^{2}\theta_{12}^{l}$ | $\sin^{2}\theta_{13}^{l}$ | $\sin^{2}\theta_{23}^{l}$ | $\delta_{CP}^{l}/^{\circ}$ | $\alpha_{21}/^{\circ}$ | $\alpha_{31}/^{\circ}$
---|---|---|---|---|---
0.30390 | 0.02228 | 0.54711 | 261.66339 | 0.54808 | 180.62051
Model II+$\mathcal{L}_{1}$ | Input |  | $\beta/\alpha$ | $\gamma/\alpha$ | $|g_{2}/g_{1}|$ | $\text{arg}(g_{2}/g_{1})/\pi$ | $\alpha v_{d}/\text{GeV}$ | $(g_{1}^{2}v_{u}^{2}/\Lambda)/\text{eV}$
---|---|---|---|---|---
3585.24816 | 1893.19896 | 1.11643 | 0.37294 | 0.00327 | 0.55907
Obs |  | $m_{e}/m_{\mu}$ | $m_{\mu}/m_{\tau}$ | $m_{1}/\text{eV}$ | $m_{2}/\text{eV}$ | $m_{3}/\text{eV}$ | $|m_{ee}|/\text{eV}$
---|---|---|---|---|---
0.00475 | 0.05864 | 0.12393 | 0.12423 | 0.13368 | 0.12407
| $\sin^{2}\theta_{12}^{l}$ | $\sin^{2}\theta_{13}^{l}$ | $\sin^{2}\theta_{23}^{l}$ | $\delta_{CP}^{l}/^{\circ}$ | $\alpha_{21}/^{\circ}$ | $\alpha_{31}/^{\circ}$
---|---|---|---|---|---
0.29874 | 0.02190 | 0.55119 | 260.84074 | 0.60930 | 180.67191
Model III+$\mathcal{L}_{1}$ | Input |  | $\beta/\alpha$ | $\gamma/\alpha$ | $|g_{2}/g_{1}|$ | $\text{arg}(g_{2}/g_{1})/\pi$ | $\alpha v_{d}/\text{GeV}$ | $(g_{1}^{2}v_{u}^{2}/\Lambda)/\text{eV}$
---|---|---|---|---|---
3592.54592 | 1898.54478 | 1.11717 | 0.37262 | 0.00326 | 0.55611
Obs  |  | $m_{e}/m_{\mu}$ | $m_{\mu}/m_{\tau}$ | $m_{1}/\text{eV}$ | $m_{2}/\text{eV}$ | $m_{3}/\text{eV}$ | $|m_{ee}|/\text{eV}$
---|---|---|---|---|---
0.00474 | 0.05868 | 0.12326 | 0.12356 | 0.13307 | 0.12341
| $\sin^{2}\theta_{12}^{l}$ | $\sin^{2}\theta_{13}^{l}$ | $\sin^{2}\theta_{23}^{l}$ | $\delta_{CP}^{l}/^{\circ}$ | $\alpha_{21}/^{\circ}$ | $\alpha_{31}/^{\circ}$
---|---|---|---|---|---
0.30802 | 0.02205 | 0.55173 | 260.76865 | 0.61384 | 180.67073
Model IV+$\mathcal{L}_{2}$ | Input |  | $\beta/\alpha$ | $\gamma/\alpha$ | $|g_{2}/g_{1}|$ | $\text{arg}(g_{2}/g_{1})/\pi$ | $\alpha v_{d}/\text{GeV}$ | $(g_{1}^{2}v_{u}^{2}/\Lambda)/\text{eV}$
---|---|---|---|---|---
4.24588 | 0.00031 | 1.15637 | 0.37586 | 0.00326 | 0.35546
Obs  |  | $m_{e}/m_{\mu}$ | $m_{\mu}/m_{\tau}$ | $m_{1}/\text{eV}$ | $m_{2}/\text{eV}$ | $m_{3}/\text{eV}$ | $|m_{ee}|/\text{eV}$
---|---|---|---|---|---
0.00474 | 0.05858 | 0.07848 | 0.07896 | 0.09315 | 0.07846
| $\sin^{2}\theta_{12}^{l}$ | $\sin^{2}\theta_{13}^{l}$ | $\sin^{2}\theta_{23}^{l}$ | $\delta_{CP}^{l}/^{\circ}$ | $\alpha_{21}/^{\circ}$ | $\alpha_{31}/^{\circ}$
---|---|---|---|---|---
0.30248 | 0.02219 | 0.58084 | 250.10633 | 1.49641 | 181.28459
Model V+$\mathcal{L}_{2}$ | Input |  | $\beta/\alpha$ | $\gamma/\alpha$ | $|g_{2}/g_{1}|$ | $\text{arg}(g_{2}/g_{1})/\pi$ | $\alpha v_{d}/\text{GeV}$ | $(g_{1}^{2}v_{u}^{2}/\Lambda)/\text{eV}$
---|---|---|---|---|---
4.22492 | 0.00031 | 1.15711 | 0.37543 | 0.00326 | 0.35468
Obs |  | $m_{e}/m_{\mu}$ | $m_{\mu}/m_{\tau}$ | $m_{1}/\text{eV}$ | $m_{2}/\text{eV}$ | $m_{3}/\text{eV}$ | $|m_{ee}|/\text{eV}$
---|---|---|---|---|---
0.00474 | 0.05855 | 0.07831 | 0.07878 | 0.09299 | 0.07828
| $\sin^{2}\theta_{12}^{l}$ | $\sin^{2}\theta_{13}^{l}$ | $\sin^{2}\theta_{23}^{l}$ | $\delta_{CP}^{l}/^{\circ}$ | $\alpha_{21}/^{\circ}$ | $\alpha_{31}/^{\circ}$
---|---|---|---|---|---
0.30324 | 0.02235 | 0.58126 | 250.12705 | 1.49671 | 181.28533

Table 9:  The best fit values of the input parameters, the charged lepton mass
ratios, the light neutrino masses and the lepton mixing parameters for the
quark-lepton unified models, where Obs is the abbreviation of observable. The
modulus $\tau$ is taken to be the best fit values in table 7 which give rise
to realistic values of quark masses and CKM mixing matrix. The fit is
performed for the GUT scale extrapolated data given in table 6. Notice that
the measured value tau lepton mass $m_{\tau}$ are reproduced exactly here.
Figure 1: The allowed region of $\tau$ by the experimental data in the
$\text{Re}\tau-\text{Im}\tau$ plane and the correlation between different
observables. The quark masses and mixing parameters can be obtained in the
blue region while the lepton masses and parameters can be obtained in the
orange region. The red dashed lines denote the experimentally preferred
$3\sigma$ ranges adapted from [1].

##  7  Non-minimal Kähler potential

In the bottom-up models with modular symmetry, the finite modular symmetry is
not very restrictive for the Kähler potential. It is shown that the Kähler
potential is not completely fixed by the modular symmetry [30], and the same
holds true in discrete flavor symmetry models [80, 81]. Besides the minimal
Kähler potential chosen in Eq. (15), the Kähler potential can receive
unsuppressed contributions from modular forms, and many other terms are also
compatible with the modular symmetry. In the quark sector, the most general
Kähler potential can be written as

$\mathcal{K}=\mathcal{K}_{\text{min}}+\Delta\mathcal{K}\,,$ (110)

with

$\mathcal{K}_{\text{min}}=(-i\tau+i\bar{\tau})^{-k_{Q_{i}}}Q^{\dagger}_{i}Q_{i}+(-i\tau+i\bar{\tau})^{-k_{q^{c}_{i}}}{q^{c}_{i}}^{\dagger}q^{c}_{i}\,,$
(111)

and

$\displaystyle\Delta\mathcal{K}$
$\displaystyle=\sum_{k_{1},k_{2},\mathbf{r}_{1},\mathbf{r}_{2}}\lambda^{ij}_{Q}(-i\tau+i\bar{\tau})^{-k_{Q_{i}}+k_{1}}(Q^{\dagger}_{i}Y^{(k_{1})\dagger}_{\mathbf{r}_{1}}Y^{(k_{2})}_{\mathbf{r}_{2}}Q_{j})_{\mathbf{1}}$
$\displaystyle+\sum_{k_{3},k_{4},\mathbf{r}_{1},\mathbf{r}_{2}}\lambda^{ij}_{q^{c}}(-i\tau+i\bar{\tau})^{-k_{q^{c}_{i}}+k_{3}}(q^{c\dagger}_{i}Y^{(k_{3})\dagger}_{\mathbf{r}_{1}}Y^{(k_{4})}_{\mathbf{r}_{2}}q^{c}_{j})_{\mathbf{1}}+\mathrm{h.c.}~{}~{}\,.$
(112)

Note that there are generally a few independent contractions into $T^{\prime}$
singlets for each operator, and the modular weights should satisfy the
following constraints

$-k_{Q_{i}}+k_{1}=-k_{Q_{j}}+k_{2},\quad-
k_{q^{c}_{i}}+k_{3}=-k_{q^{c}_{j}}+k_{4}\,.$ (113)

The weight $k$ of the modular forms $Y^{(k)}_{\mathbf{r}}$ can run from $1$ to
$+\infty$, consequently infinity possible terms could be present in the most
general Kähler potential. The presence of free coefficients $\lambda^{ij}_{Q}$
and $\lambda^{ij}_{q^{c}}$ accompanying each operator in Eq. (112) could
reduce the predictivity of modular invariance models, and the impacts of these
additional terms on the predictions for both masses and mixing parameters are
sizable, as shown in [30]. After the complex modulus $\tau$ acquires vacuum
expectation value $\langle\tau\rangle$, this additional terms
$\Delta\mathcal{K}$ lead to departures from the canonical Kähler metric, i.e.

$\displaystyle\mathscr{K}^{ij}_{Q}=\frac{\partial^{2}\mathcal{K}}{\partial
Q^{\dagger}_{i}\partial Q_{j}}=-\langle
i\tau+i\bar{\tau}\rangle^{-k_{Q_{i}}}\delta^{ij}+\Delta\mathcal{K}^{ij}_{Q}(\langle\tau\rangle)\,,$
(114)
$\displaystyle\mathscr{K}^{ij}_{q^{c}}=\frac{\partial^{2}\mathcal{K}}{\partial
q^{c\dagger}_{i}\partial q^{c}_{j}}=-\langle
i\tau+i\bar{\tau}\rangle^{-k_{q^{c}_{i}}}\delta^{ij}+\Delta\mathcal{K}^{ij}_{q^{c}}(\langle\tau\rangle)\,.$
(115)

Here both Kählier metric matrices $\mathscr{K}_{Q}$ and $\mathscr{K}_{q^{c}}$
are hermitian and positive, and they can be rewritten as

$\displaystyle\mathscr{K}_{Q}=\Sigma_{Q}D^{2}_{Q}\Sigma^{\dagger}_{Q},\quad\mathscr{K}_{u^{c}}=\Sigma_{u^{c}}D^{2}_{u^{c}}\Sigma^{\dagger}_{u^{c}},\quad\mathscr{K}_{d^{c}}=\Sigma_{d^{c}}D^{2}_{d^{c}}\Sigma^{\dagger}_{d^{c}}\,,$
(116)

where $\Sigma_{Q,u^{c},d^{c}}$ are unitary matrices and $D_{Q,u^{c},d^{c}}$
are diagonal matrices with positive non-vanishing entries. In order to obtain
a canonical Kähler metric
$\mathscr{K}^{ij}\rightarrow\widehat{\mathscr{K}}^{ij}=\delta^{ij}$ such that
the quark fields are canonically normalized, we should perform field
redefinitions

$\displaystyle
Q=H_{Q}\hat{Q},\quad~{}~{}~{}~{}H_{Q}=\Sigma_{Q}D^{-1}_{Q}\Sigma^{\dagger}_{Q}\,,$
$\displaystyle
u^{c}=H_{u^{c}}\hat{u}^{c},\quad~{}~{}~{}H_{u^{c}}=\Sigma_{u^{c}}D^{-1}_{u^{c}}\Sigma^{\dagger}_{u^{c}}\,,$
$\displaystyle
d^{c}=H_{d^{c}}\hat{d}^{c},\quad~{}~{}~{}H_{d^{c}}=\Sigma_{d^{c}}D^{-1}_{d^{c}}\Sigma^{\dagger}_{d^{c}}\,,$
(117)

where $H_{Q,u^{c},d^{c}}$ are hermitian. If $D_{Q,u^{c},d^{c}}$ or
$\Sigma_{Q,u^{c},d^{c}}$ are proportional to the unit matrix, we see that one
should rescale the quark fields to get canonical kinetic terms, and the effect
of such rescaling can be absorbed into the arbitrary coefficients of the
model. The field redefinition in Eq. (117) would affect the superpotential or
equivalently the Yukawa coupling matrices,

$\mathcal{W}=u^{c\,T}Y_{u}QH_{u}+d^{c\,T}Y_{d}QH_{d}\rightarrow\widehat{\mathcal{W}}=\hat{u}^{c\,T}\hat{Y}_{u}\hat{Q}H_{u}+\hat{d}^{c\,T}\hat{Y}_{d}\hat{Q}H_{d}\,,$
(118)

with

$\hat{Y}_{u}=H_{u^{c}}^{T}Y_{u}H_{Q},~{}~{}\quad~{}~{}\hat{Y}_{d}=H_{d^{c}}^{T}Y_{d}H_{Q}\,.$
(119)

Diagonalizing the quark mass matrices in the basis of canonically normalized
fields, we can obtain quark mass eigenvalues and CKM matrix,

$\hat{U}_{u}^{\dagger}\hat{Y}_{u}^{\dagger}\hat{Y}_{u}\hat{U}_{u}=\text{diag}(y^{2}_{u},~{}y^{2}_{c},~{}y^{2}_{t}),~{}~{}~{}~{}~{}~{}\hat{U}_{d}^{\dagger}\hat{Y}_{d}^{\dagger}\hat{Y}_{d}\hat{U}_{d}=\text{diag}(y^{2}_{d},~{}y^{2}_{s},~{}y^{2}_{b}),~{}~{}~{}~{}~{}~{}V_{\text{CKM}}=\hat{U}^{\dagger}_{u}\hat{U}_{d}\,.$
(120)

Since $D_{Q,u,d}$ is generally not proportional to unit matrix because of the
non-diagonal terms $\Delta\mathcal{K}^{ij}_{Q}(\langle\tau\rangle)$ and
$\Delta\mathcal{K}^{ij}_{q^{c}}(\langle\tau\rangle)$ in the Kähler metrics,
the additional term $\Delta\mathcal{K}$ in the Kähler potential can lead to
nontrival effects on the observables, and it might significantly modify the
predictions of a model [30, 80, 81].

In the present work, we have focused on doublet plus singlet assignment
$Q_{D}\sim\mathbf{2}^{i},Q_{3}\sim\mathbf{1}^{j}$ for the left-handed quarks.
From Eq. (112), we know the most general form of the extra Kähler potential is
given by

$\displaystyle\Delta\mathcal{K}_{Q}$
$\displaystyle=\sum_{k_{1},k_{2},k_{3},\mathbf{r}_{1},\mathbf{r}_{2}}(-i\tau+i\bar{\tau})^{-k_{Q_{D}}+k_{1}}(Q_{D}^{\dagger}Y^{(k_{1})\dagger}_{\mathbf{r}_{1}}Q_{D}Y^{(k_{1})}_{\mathbf{r}_{2}})_{\mathbf{1}}$
$\displaystyle\qquad\qquad+(-i\tau+i\bar{\tau})^{-k_{Q_{3}}+k_{1}}Q_{3}^{\dagger}Q_{3}(Y^{(k_{1})\dagger}_{\mathbf{r}_{1}}Y^{(k_{1})}_{\mathbf{r}_{2}})_{\mathbf{1}}$
$\displaystyle\qquad\qquad+(-i\tau+i\bar{\tau})^{-k_{Q_{D}}+k_{2}}(Q_{D}^{\dagger}Y^{(k_{2})\dagger}_{\mathbf{r}_{1}}Q_{3}Y^{(k_{3})}_{\mathbf{r}_{2}})_{\mathbf{1}}+\mathrm{h.c.}\,,$
(121)

where we suppress all couplings in front of each operator, and the modular
weights fulfill $k_{2}-k_{3}=k_{Q_{D}}-k_{Q_{3}}$. In a similar fashion, if
the right-handed quarks transform as a doublet and a singlet
$q^{c}_{D}\sim\mathbf{2}^{k},q^{c}_{3}\sim\mathbf{1}^{l}$ under $T^{\prime}$,
the Kähler potential $\Delta\mathcal{K}_{q^{c}}$ is of the following form

$\displaystyle\Delta\mathcal{K}_{q^{c}}$
$\displaystyle=\sum_{k_{1},k_{2},k_{3},\mathbf{r}_{1},\mathbf{r}_{2}}(-i\tau+i\bar{\tau})^{-k_{q^{c}_{D}}+k_{1}}(q^{c\dagger}_{D}Y^{(k_{1})\dagger}_{\mathbf{r}_{1}}q^{c}_{D}Y^{(k_{1})}_{\mathbf{r}_{2}})_{\mathbf{1}}$
$\displaystyle\qquad\qquad+(-i\tau+i\bar{\tau})^{-k_{q^{c}_{3}}+k_{1}}q^{c\dagger}_{3}q^{c}_{3}(Y^{(k_{1})\dagger}_{\mathbf{r}_{1}}Y^{(k_{1})}_{\mathbf{r}_{2}})_{\mathbf{1}}$
$\displaystyle\qquad\qquad+(-i\tau+i\bar{\tau})^{-k_{q^{c}_{D}}+k_{2}}(q^{c\dagger}_{D}Y^{(k_{2})\dagger}_{\mathbf{r}_{1}}q^{c}_{3}Y^{(k_{3})}_{\mathbf{r}_{2}})_{\mathbf{1}}+\mathrm{h.c.}\,,$
(122)

with $k_{2}-k_{3}=k_{q^{c}_{D}}-k_{q^{c}_{3}}$. For the singlet assignment
$q^{c}_{a}\sim\mathbf{1}^{l_{a}}$ ($a=1,2,3$), $\Delta\mathcal{K}_{q^{c}}$
reads as

$\displaystyle\Delta\mathcal{K}_{q^{c}}$
$\displaystyle=\sum_{a,b,k_{1},k_{2},\mathbf{r}_{1},\mathbf{r}_{2}}(-i\tau+i\bar{\tau})^{-k_{q^{c}_{a}}+k_{1}}(q^{c\dagger}_{a}Y^{(k_{1})\dagger}_{\mathbf{r}_{1}}q^{c}_{b}Y^{(k_{2})}_{\mathbf{r}_{2}})_{\mathbf{1}}+\mathrm{h.c.}\,,$
(123)

with $k_{1}-k_{2}=k_{q^{c}_{a}}-k_{q^{c}_{b}}$. Since the new parameters
appearing in the Kähler potential considerably reduce the predictability of
the approach [30], a better understanding of the Kähler potential is required.
However, the problem of better controlling the Kähler potential remains an
open question. In some more complete setting such as string theory, the Kähler
potential is constrained to a better extent [74], nevertheless the complete
expressions for higher-order terms are still unknown. Recently a new approach
based on top-down model building in string theory has been proposed [25, 29,
75, 76], it is a hybrid of the top-down and bottom-up approaches to modular
flavor symmetry [75]. This new approach combines the traditional flavor
symmetry and the modular flavor symmetry, and put forward the concept of
eclectic flavor groups. This new scheme severely restricts the representations
and modular weights of matter fields, such that both superpotential potential
and Kähler potential are strongly constrained [76]. In order to show the
impact of the additional terms in the Kähler potential, we give a concrete
example in the following. The transformation properties of the quark fields
under the $SU(2)_{L}\times U(1)_{Y}$ gauge group and $T^{\prime}$ modular
symmetry are summarized in table 10. The modular invariant superpotentials for
up and down quark Yukawa couplings read as,

$\displaystyle\mathcal{W}_{u}=y_{1}^{u}u^{c}Q_{D}Y^{(3)}_{\mathbf{2}}H_{u}+y_{2}^{u}c^{c}Q_{D}Y^{(1)}_{\mathbf{2}}H_{u}+y_{3}^{u}t^{c}Q_{3}H_{u}\,,$
(124)
$\displaystyle\mathcal{W}_{d}=y_{1}^{d}d^{c}_{D}Q_{D}Y^{(4)}_{\mathbf{3}}H_{d}+y_{2}^{d}d^{c}_{D}Q_{D}Y^{(4)}_{\mathbf{1}^{\prime}}H_{d}+y_{3}^{d}b^{c}Q_{3}H_{d}\,,$

| $Q_{D}$ | $Q_{3}$ | $u^{c}$ | $c^{c}$ | $t^{c}$ | $d^{c}_{D}\equiv(d^{c},s^{c})$ | $b^{c}$
---|---|---|---|---|---|---|---
$SU(2)_{L}\times U(1)_{Y}$ | $(2,1/6)$ | $(2,1/6)$ | $(1,-2/3)$ | $(1,-2/3)$ | $(1,-2/3)$ | $(1,1/3)$ | $(1,1/3)$
Model VI | $T^{\prime}$ | $\mathbf{2}$ | $\mathbf{1}$ | $\mathbf{1}^{\prime\prime}$ | $\mathbf{1}^{\prime\prime}$ | $\mathbf{1}$ | $\mathbf{2}^{\prime}$ | $\mathbf{1}$
$k_{I}$ | $0$ | $-1$ | $3$ | $1$ | $1$ | $4$ | $1$

Table 10:  The transformation properties of the quark fields under the
Standard Model gauge group $SU(2)_{L}\times U(1)_{Y}$ and under $T^{\prime}$
modular symmetry for the example model in which the contribution of non-
minimal Kähler potential is considered, where $-k_{I}$ refers to the modular
weights. The two Higgs doublets $H_{u,d}$ are invariant under $T^{\prime}$ and
their modular weights are vanishing.

where the coupling constants $y^{u}_{1,2,3}$ and $y^{d}_{1,3}$ can be taken to
be real and positive by rephasing the quark fields, while $y^{d}_{2}$ is
generically a complex number. Applying the decomposition rules of the
$T^{\prime}$ tensor products in Appendix A, we find the quark mass matrices
are given by

$\displaystyle
M_{u}=\left(\begin{array}[]{ccc}y_{1}^{u}Y^{(3)}_{\mathbf{2},2}{}&-y_{1}^{u}Y^{(3)}_{\mathbf{2},1}{}&0\\\
y_{2}^{u}Y^{(1)}_{\mathbf{2},2}{}&-y_{2}^{u}Y^{(1)}_{\mathbf{2},1}{}&0\\\
0{}&0{}&y_{3}^{u}\\\ \end{array}\right)v_{u}\,,$ (128) $\displaystyle
M_{d}=\left(\begin{array}[]{ccc}\sqrt{2}e^{\frac{5i\pi}{12}}y^{d}_{1}Y^{(4)}_{\mathbf{3},1}{}&~{}-y^{d}_{1}Y^{(4)}_{\mathbf{3},2}+y^{d}_{2}Y^{(4)}_{\mathbf{1}^{\prime}}{}&~{}0\\\
-y^{d}_{1}Y^{(4)}_{\mathbf{3},2}-y^{d}_{2}Y^{(4)}_{\mathbf{1}^{\prime}}{}&~{}\sqrt{2}e^{\frac{7i\pi}{12}}y^{d}_{1}Y^{(4)}_{\mathbf{3},3}{}&~{}0\\\
0{}&~{}0{}&~{}y^{d}_{3}\\\ \end{array}\right)v_{d}\,.$ (132)

We see that both the up quark mass matrix $M_{u}$ and the down quark mass
matrix $M_{d}$ are block diagonal. As a result, only the Cabibbo angle could
be accommodated, while the other two quark mixing angles $\theta_{13}^{q}$,
$\theta_{23}^{q}$ and the CP-violation phase $\delta_{CP}^{q}$ are vanishing
if only the superpotential is considered. Now we proceed to consider the
contribution of the Kähler potential. With the transformations properties of
quark fields in table 10, we can use Eqs. (121,122,123) and straightforwardly
write down the most general Kähler potential $\mathcal{K}$. As an illustration
example, we only consider one extra term of Kähler potential in right-hand up
quark sector.

$\displaystyle\Delta\mathcal{K}_{u^{c}}$
$\displaystyle=\lambda_{u^{c}}(-i\tau+i\bar{\tau})^{1}u^{c\dagger}(Y^{(4)\dagger}_{\mathbf{3}}Y^{(2)}_{\mathbf{3}})_{\mathbf{1}^{\prime\prime}}t^{c}+\mathrm{h.c.}\,,$
(133)

where $\lambda_{u^{c}}$ is an arbitrary complex number. The minimal Kähler
potential $\mathcal{K}^{u^{c}}_{\text{min}}$ is

$\displaystyle\mathcal{K}^{u^{c}}_{\text{min}}$
$\displaystyle=(-i\tau+i\bar{\tau})^{-3}(u^{c\dagger}u^{c})_{\mathbf{1}}+(-i\tau+i\bar{\tau})^{-1}(c^{c\dagger}c^{c})_{\mathbf{1}}+(-i\tau+i\bar{\tau})^{-1}(t^{c\dagger}t^{c})_{\mathbf{1}}\,.$
(134)

Then we can find the Kähler metric $\mathscr{K}_{u^{c}}$ as follow,

$\mathscr{K}_{u^{c}}=\begin{pmatrix}\langle-i\tau+i\bar{\tau}\rangle^{-3}&0&0\\\
0&\langle-i\tau+i\bar{\tau}\rangle^{-1}&0\\\
0&0&\langle-i\tau+i\bar{\tau}\rangle^{-1}\end{pmatrix}+\langle-i\tau+i\bar{\tau}\rangle^{1}\begin{pmatrix}0&0&\lambda_{u^{c}}Y\\\
0&0&0\\\ \lambda_{u^{c}}^{*}Y^{*}&0&0\end{pmatrix}+\dots\,,$ (135)

where
$Y=Y^{(4)*}_{\mathbf{3},3}Y^{(2)}_{\mathbf{3},2}+Y^{(4)*}_{\mathbf{3},1}Y^{(2)}_{\mathbf{3},3}+Y^{(4)*}_{\mathbf{3},2}Y^{(2)}_{\mathbf{3},1}$.
We can rescale the up type quark superfields
$u^{c}\rightarrow\braket{-i\tau+i\bar{\tau}}^{3/2}u^{c}$,
$c^{c}\rightarrow\braket{-i\tau+i\bar{\tau}}^{1/2}c^{c}$ and
$t^{c}\rightarrow\braket{-i\tau+i\bar{\tau}}^{1/2}t^{c}$ which can be absorbed
into the arbitrary coefficients in this model. Then the Kähler metrix
$\mathcal{K}_{u^{c}}$ simplifies into

$\mathscr{K}_{u^{c}}=\begin{pmatrix}1&0&0\\\ 0&1&0\\\
0&0&1\end{pmatrix}+\begin{pmatrix}0&~{}0~{}&\lambda^{\prime}_{u^{c}}\\\
0&~{}0~{}&0\\\ (\lambda^{\prime}_{u^{c}})^{*}&~{}0~{}&0\end{pmatrix}+\dots\,.$
(136)

where
$\lambda_{u^{c}}^{\prime}\equiv\langle-i\tau+i\bar{\tau}\rangle^{3}\lambda_{u^{c}}Y$.
The Kähler metric is not diagonal anymore because of the additional term in
Eq. (133) so that mixture between the third and the first two generations can
be induced. A numerical analysis is performed, we find that the correct values
of quark masses and quark mixing angles and CP violation phase can be
obtained. The numerical fitting results are shown in table 12. The non-minimal
Kähler potential in the lepton sector is discussed in Appendix C.

Model VI  | Input |  | $\text{Re}\tau$ | $\text{Im}\tau$ | $|\lambda^{\prime}_{u^{c}}|$ | $\text{arg}(\lambda^{\prime}_{u^{c}})/\pi$ | $y^{u}_{2}/y^{u}_{1}$ | $y^{u}_{3}/y^{u}_{1}$
---|---|---|---|---|---
$-0.08817$ | 2.51405 | 1.00000 | 0.19108 | 5.01136 | 0.92896
| $|y^{d}_{2}/y^{d}_{1}|$ | $\text{arg}(y^{d}_{2}/y^{d}_{1})/\pi$ | $y^{d}_{3}/y^{d}_{1}$ | $y_{1}^{u}v_{u}/\text{GeV}$ | $y_{1}^{d}v_{d}/\text{GeV}$
---|---|---|---|---
3.71694 | 0.14153 | 1.00590 | 0.14779 | 0.96250
Obs |  | $m_{u}/m_{c}$ | $m_{c}/m_{t}$ | $m_{d}/m_{s}$ | $m_{s}/m_{b}$ | $\theta_{12}^{q}$ | $\theta_{13}^{q}$ | $\theta_{23}^{q}$ | $\delta_{CP}^{q}/^{\circ}$
---|---|---|---|---|---|---|---
0.00194 | 0.00282 | 0.04973 | 0.01824 | 0.22541 | 0.00349 | 0.03977 | 69.28416

Table 11:  The best fit values of the input parameters as well as quark mass
ratios and flavor mixing parameters for the Model VI.

##  8  Conclusion and summary

Modular invariance as the flavor symmetry is a promising approach to
understand the puzzle of quark and lepton masses and flavor mixing. The Yuakwa
couplings are modular forms of a certain level $N$ in this approach. We have
generalized this formalism to modular forms of general integer weights which
can be arranged into irreducible representations of the homogeneous finite
modular group $\Gamma^{\prime}_{N}$ [61]. Notice that $\Gamma^{\prime}_{N}$ is
the double covering of $\Gamma_{N}$. It is well known that $SU(2)$ is the
double covering group of $SO(3)$, and two $SU(2)$ elements correspond to one
$SO(3)$ element with the same Euler angles. $\Gamma^{\prime}_{N}$ can be
regarded as the inverse image of the inhomogeneous finite modular group
$\Gamma_{N}$ under this map for smaller $N$. In addition to the irreducible
representations of $\Gamma_{N}$, $\Gamma^{\prime}_{N}$ has other new
representations [61]. The even weight modular forms of level $N$ transform in
the irreducible representations of $\Gamma_{N}$ [11], while the odd weight
modular forms of level $N$ are arranged into the new representations of
$\Gamma^{\prime}_{N}$ [61]. The odd weight modular forms provide us
interesting opportunity for fermion mass model building.

With the idea of reduce the number of free parameters, the ansatz of texture
zero has been widely studied. It is shown that the texture zeros of the
fermion mass matrices can be enforced by means of Abelian symmetries. In the
present work, we show that the texture zeros of the fermion mass matrices can
be naturally produced if we properly assign the representations and weights of
the matter fields under the modular symmetry. As a concrete example, we impose
the $\Gamma^{\prime}_{3}\cong T^{\prime}$ modular symmetry on the quark
sector. We assign the first two generations of the left-handed quark doublets
to a $T^{\prime}$ doublet, and the third generation of the left-handed quark
is a singlet of $T^{\prime}$. As regards the right-handed quark fields, we
have considered two scenarios that the three generations of right-handed
quarks transform as a doublet and a singlet under $T^{\prime}$ or they are
three singlets of $T^{\prime}$. We find that the quark mass matrix can take
six possible forms with zero entries up to row and column permutations:
$\text{Case}~{}\mathcal{A}$, $\text{Case}~{}\mathcal{B}$,
$\text{Case}~{}\mathcal{C}$, $\text{Case}~{}\mathcal{D}$,
$\text{Case}~{}\mathcal{E}$ and $\text{Case}~{}\mathcal{F}$ given in Eqs. (40,
69, 70). Combing the up and down quark sectors together, we can obtain the
possible up quark and down quark mass matrices predicted by $T^{\prime}$
modular symmetry. It is usually assumed the mass matrix is symmetric or
hermitian in texture zero models. In our approach, the explicit form of mass
matrix is completely fixed by modular symmetry and it is generally neither
symmetric nor hermitian. We present five interesting quark models summarized
in table 5. These models contain only ten or eleven independent real
parameters including the complex modulus $\tau$, and the experimental data of
quark masses and CKM mixing matrix can be accommodated very well.

Furthermore, we investigate the lepton sector with the $T^{\prime}$ modular
symmetry. The lepton fields are assumed to transform as triplet $\mathbf{3}$
or singlets $\mathbf{1}$, $\mathbf{1}^{\prime}$, $\mathbf{1}^{\prime\prime}$
under $T^{\prime}$ such that the $T^{\prime}$ modular symmetry can not be
distinguished from the $A_{4}$ modular symmetry in the lepton sector since
doublet representations of $T^{\prime}$ are not involved. A systematical
classification of lepton models with $A_{4}$ modular symmetry has been
performed in [43]. We combine the five quark models in table 5 with the
$\mathcal{D}_{6}$ and $\mathcal{D}_{9}$ lepton models of [43] to give a
unified description of both quark and lepton masses and flavor mixing. It is
highly nontrivial that the resulting models can produce very good fit to the
experimental data although they contain less number of free parameters than
the observables.

In summary, the modular symmetry can naturally produce texture zeros in
fermion mass matrix if odd weight modular forms are considered. The modular
invariance approach has the merits of both abelian flavor symmetry and
discrete non-abelian flavor symmetry. It is interesting to investigate the
possible texture zero structures of the lepton sector which can be obtained
from modular symmetry.

In the present paper, we have followed the bottom-up model construction
approach of modular symmetry to describe the flavor structure of the SM. It is
interesting to investigate whether the benchmark quark-lepton unification
models in section 6 can be embedded in a string theory framework. It has been
shown that the finite modular group $T^{\prime}$ can naturally appear in
simple string constructions such as $Z_{3}$ orbifold [25, 29]. Therefore we
expect that certain (not all) models of the present paper or at least texture
zero structure of quark mass matrix could be realized in the low-energy
effective theory, although explicit model building in string theory is needed.
The phenomenological bottom-up models based on modular invariance can explain
many different aspects of the flavor puzzle in SM. However, the possible
connections between this approach and the more fundamental theory such as
string theory are still not clear, see [25, 29, 75] for recent progress toward
this direction. Discrete flavor symmetries and modular invariance could appear
simultaneously in top-down model building in string theory. Recently it is
suggested to extend the traditional discrete flavor group by finite modular
symmetries [76]. The Kähler potential and superpotential are severely
constrained in this new scheme, in particularly the off-diagonal contributions
to the Kähler metric are forbidden. As a result, the modular form dependent
terms in the Kähler potential does not considerably change the
phenomenological predictions obtained by assuming a minimal Kähler potential.
Hence it is a reasonable and good approximation to take the standard minimal
Kähler potential in bottom-up model construction.

## Acknowledgements

This work is supported by the National Natural Science Foundation of China
under Grant Nos 11975224 and 11835013.

## Appendix

##  Appendix A Group Theory of $T^{\prime}$

The $T^{\prime}$ group is the double covering of the tetrahedral group
$A_{4}$. All the elements of $T^{\prime}$ can be generated by three generators
$S$, $T$ and $\mathbb{R}$ which obey the following relations,

$S^{2}=\mathbb{R},~{}~{}(ST)^{3}=T^{3}=\mathbb{R}^{2}=1,~{}~{}\mathbb{R}T=T\mathbb{R}\,.$
(A.1)

Hence the generator $\mathbb{R}$ commutes with all elements of the group.
Besides the $A_{4}$ representations: one triplet $\mathbf{4}$ and three
singlets $\mathbf{1}$, $\mathbf{1}^{\prime}$ and $\mathbf{1}^{\prime\prime}$,
the $T^{\prime}$ group has three two-dimensional irreducible representations
$\mathbf{2}$, $\mathbf{2}^{\prime}$ and $\mathbf{2}^{\prime\prime}$. In the
present work, we shall adopt the same basis of [61], in particular using the
following explicit representation matrices for the generators $S$, $T$ and
$\mathbb{R}$ in different irreps,

$\displaystyle\begin{array}[]{cccc}\mathbf{1:}&S=1,&T=1,&\mathbb{R}=1\,,\\\
\mathbf{1}^{\prime}:&S=1,&T=\omega,&\mathbb{R}=1\,,\\\
\mathbf{1}^{\prime\prime}:&S=1,&T=\omega^{2},&\mathbb{R}=1\,,\\\
\mathbf{2:}{}&~{}S=-\dfrac{1}{\sqrt{3}}\left(\begin{array}[]{cc}i&\sqrt{2}e^{i\pi/12}\\\
-\sqrt{2}e^{-i\pi/12}&-i\\\
\end{array}\right),&T=\left(\begin{array}[]{cc}\omega&0\\\ 0&1\\\
\end{array}\right),&\mathbb{R}=-\left(\begin{array}[]{cc}1&0\\\ 0&1\\\
\end{array}\right)\,,\\\
\mathbf{2}^{\prime}:{}&~{}S=-\dfrac{1}{\sqrt{3}}\left(\begin{array}[]{cc}i&\sqrt{2}e^{i\pi/12}\\\
-\sqrt{2}e^{-i\pi/12}&-i\\\
\end{array}\right),&T=\left(\begin{array}[]{cc}\omega^{2}&0\\\ 0&\omega\\\
\end{array}\right),&\mathbb{R}=-\left(\begin{array}[]{cc}1&0\\\ 0&1\\\
\end{array}\right)\,,\\\
\mathbf{2}^{\prime\prime}:{}&~{}S=-\dfrac{1}{\sqrt{3}}\left(\begin{array}[]{cc}i&\sqrt{2}e^{i\pi/12}\\\
-\sqrt{2}e^{-i\pi/12}&-i\\\
\end{array}\right),&T=\left(\begin{array}[]{cc}1&0\\\ 0&\omega^{2}\\\
\end{array}\right),&\mathbb{R}=-\left(\begin{array}[]{cc}1&0\\\ 0&1\\\
\end{array}\right)\,,\\\
\mathbf{3:}&S=\frac{1}{3}\left(\begin{array}[]{ccc}-1&2&2\\\ 2&-1&2\\\
2&2&-1\end{array}\right),&T=\left(\begin{array}[]{ccc}1{}&0{}&0\\\
0{}&\omega{}&0\\\
0{}&0{}&\omega^{2}\end{array}\right),&~{}~{}\mathbb{R}=\left(\begin{array}[]{ccc}1{}&0{}&0\\\
0{}&1{}&0\\\ 0{}&0{}&1\end{array}\right)\,,\end{array}$

with $\omega=e^{i2\pi/3}$. The generator $\mathbb{R}$ is represented by an
identity matrix for the odd-dimensional representations $\mathbf{1}$,
$\mathbf{1}^{\prime}$, $\mathbf{1}^{\prime\prime}$ and $\mathbf{3}$. Therefore
the elements of $T^{\prime}$ coincide two by two and can be described by the
same matrices that represent the elements in $A_{4}$ for these
representations. The Kronecker products between different irreducible
representations of $T^{\prime}$ are given by

$\displaystyle\mathbf{1}^{a}\otimes\mathbf{r}^{b}=\mathbf{r}^{b}\otimes\mathbf{1}^{a}=\mathbf{r}^{a+b~{}(\text{mod}~{}3)},~{}~{}~{}~{}~{}{\rm
for}~{}~{}\mathbf{r}=\mathbf{1},\mathbf{2}\,,$
$\displaystyle\mathbf{1}^{a}\otimes\mathbf{3}=\mathbf{3}\otimes\mathbf{1}^{a}=\mathbf{3}\,,$
$\displaystyle\mathbf{2}^{a}\otimes\mathbf{2}^{b}=\mathbf{3}\oplus\mathbf{1}^{a+b+1~{}(\text{mod}~{}3)}\,,$
$\displaystyle\mathbf{2}^{a}\otimes\mathbf{3}=\mathbf{3}\otimes\mathbf{2}^{a}=\mathbf{2}\oplus\mathbf{2}^{\prime}\oplus\mathbf{2}^{\prime\prime}\,,$
$\displaystyle\mathbf{3}\otimes\mathbf{3}=\mathbf{3}_{S}\oplus\mathbf{3}_{A}\oplus\mathbf{1}\oplus\mathbf{1}^{\prime}\oplus\mathbf{1}^{\prime\prime}\,,$
(A.7)

where $a,b=0,1,2$ and we have denoted $\mathbf{1}\equiv\mathbf{1}^{0}$,
$\mathbf{1}^{\prime}\equiv\mathbf{1}^{1}$,
$\mathbf{1}^{\prime\prime}\equiv\mathbf{1}^{2}$ for singlet representations
and $\mathbf{2}\equiv\mathbf{2}^{0}$,
$\mathbf{2}^{\prime}\equiv\mathbf{2}^{1}$,
$2^{\prime\prime}\equiv\mathbf{2}^{2}$ for the doublet representations. The
notations $\mathbf{3}_{S}$ and $\mathbf{3}_{A}$ stand for the symmetric and
antisymmetric triplet combinations respectively. The CG coefficients in our
working basis can be also be found in [61], we would like to list them in the
following for completeness. We shall use $\alpha_{i}$ to denote the elements
of the first representation, $\beta_{i}$ to indicate these of the second
representation of the product.

$\displaystyle\mathbf{1}^{a}\otimes\mathbf{1}^{b}$ $\displaystyle=$
$\displaystyle\mathbf{1}^{a+b~{}(\text{mod}~{}3)}\sim\alpha\beta\,,$ (A.8)
$\displaystyle\mathbf{1}^{a}\otimes\mathbf{2}^{b}$ $\displaystyle=$
$\displaystyle\mathbf{2}^{a+b~{}(\text{mod}~{}3)}\sim\left(\begin{array}[]{c}\alpha\beta_{1}\\\
\alpha\beta_{2}\\\ \end{array}\right)\,,$ (A.11)
$\displaystyle\mathbf{1}^{\prime}\otimes\mathbf{3}$ $\displaystyle=$
$\displaystyle\mathbf{3}\sim\left(\begin{array}[]{c}\alpha\beta_{3}\\\
\alpha\beta_{1}\\\ \alpha\beta_{2}\end{array}\right)\,,$ (A.15)
$\displaystyle\mathbf{1}^{\prime\prime}\otimes\mathbf{3}$ $\displaystyle=$
$\displaystyle\mathbf{3}\sim\left(\begin{array}[]{c}\alpha\beta_{2}\\\
\alpha\beta_{3}\\\ \alpha\beta_{1}\end{array}\right)\,.$ (A.19)

$\displaystyle\mathbf{2}\otimes\mathbf{2}=\mathbf{2}^{\prime}\otimes\mathbf{2}^{\prime\prime}=\mathbf{3}\oplus\mathbf{1}^{\prime}~{}$
with
$\displaystyle~{}\left\\{\begin{array}[]{l}\mathbf{1}^{\prime}\sim\alpha_{1}\beta_{2}-\alpha_{2}\beta_{1}\\\\[7.22743pt]
\mathbf{3}\sim\left(\begin{array}[]{c}e^{i\pi/6}\alpha_{2}\beta_{2}\\\
\frac{1}{\sqrt{2}}e^{i7\pi/12}(\alpha_{1}\beta_{2}+\alpha_{2}\beta_{1})\\\
\alpha_{1}\beta_{1}\end{array}\right)\end{array}\right.$ (A.25)
$\displaystyle\mathbf{2}\otimes\mathbf{2}^{\prime}=\mathbf{2}^{\prime\prime}\otimes\mathbf{2}^{\prime\prime}=\mathbf{3}\oplus\mathbf{1}^{\prime\prime}~{}$
with
$\displaystyle~{}\left\\{\begin{array}[]{l}\mathbf{1}^{\prime\prime}\sim\alpha_{1}\beta_{2}-\alpha_{2}\beta_{1}\\\\[7.22743pt]
\mathbf{3}\sim\left(\begin{array}[]{c}\alpha_{1}\beta_{1}\\\
e^{i\pi/6}\alpha_{2}\beta_{2}\\\
\frac{1}{\sqrt{2}}e^{i7\pi/12}(\alpha_{1}\beta_{2}+\alpha_{2}\beta_{1})\end{array}\right)\end{array}\right.$
(A.31)
$\displaystyle\mathbf{2}\otimes\mathbf{2}^{\prime\prime}=\mathbf{2}^{\prime}\otimes\mathbf{2}^{\prime}=\mathbf{3}\oplus\mathbf{1}~{}$
with
$\displaystyle~{}\left\\{\begin{array}[]{l}\mathbf{1}\sim\alpha_{1}\beta_{2}-\alpha_{2}\beta_{1}\\\\[7.22743pt]
\mathbf{3}\sim\left(\begin{array}[]{c}\frac{1}{\sqrt{2}}e^{i7\pi/12}(\alpha_{1}\beta_{2}+\alpha_{2}\beta_{1})\\\
\alpha_{1}\beta_{1}\\\
e^{i\pi/6}\alpha_{2}\beta_{2}\end{array}\right)\end{array}\right.$ (A.37)
$\displaystyle\mathbf{2}\otimes\mathbf{3}=\mathbf{2}\oplus\mathbf{2}^{\prime}\oplus\mathbf{2}^{\prime\prime}~{}$
with
$\displaystyle~{}\left\\{\begin{array}[]{l}\mathbf{2}\sim\left(\begin{array}[]{c}\alpha_{1}\beta_{1}-\sqrt{2}e^{i7\pi/12}\alpha_{2}\beta_{2}\\\
-\alpha_{2}\beta_{1}+\sqrt{2}e^{i5\pi/12}\alpha_{1}\beta_{3}\end{array}\right)\\\\[7.22743pt]
\mathbf{2}^{\prime}\sim\left(\begin{array}[]{c}\alpha_{1}\beta_{2}-\sqrt{2}e^{i7\pi/12}\alpha_{2}\beta_{3}\\\
-\alpha_{2}\beta_{2}+\sqrt{2}e^{i5\pi/12}\alpha_{1}\beta_{1}\end{array}\right)\\\\[7.22743pt]
\mathbf{2}^{\prime\prime}\sim\left(\begin{array}[]{c}\alpha_{1}\beta_{3}-\sqrt{2}e^{i7\pi/12}\alpha_{2}\beta_{1}\\\
-\alpha_{2}\beta_{3}+\sqrt{2}e^{i5\pi/12}\alpha_{1}\beta_{2}\end{array}\right)\\\\[7.22743pt]
\end{array}\right.$ (A.47)
$\displaystyle\mathbf{2}^{\prime}\otimes\mathbf{3}=\mathbf{2}\oplus\mathbf{2}^{\prime}\oplus\mathbf{2}^{\prime\prime}~{}$
with
$\displaystyle~{}\left\\{\begin{array}[]{l}\mathbf{2}\sim\left(\begin{array}[]{c}\alpha_{1}\beta_{3}-\sqrt{2}e^{i7\pi/12}\alpha_{2}\beta_{1}\\\
-\alpha_{2}\beta_{3}+\sqrt{2}e^{i5\pi/12}\alpha_{1}\beta_{2}\end{array}\right)\\\\[7.22743pt]
\mathbf{2}^{\prime}\sim\left(\begin{array}[]{c}\alpha_{1}\beta_{1}-\sqrt{2}e^{i7\pi/12}\alpha_{2}\beta_{2}\\\
-\alpha_{2}\beta_{1}+\sqrt{2}e^{i5\pi/12}\alpha_{1}\beta_{3}\end{array}\right)\\\\[7.22743pt]
\mathbf{2}^{\prime\prime}\sim\left(\begin{array}[]{c}\alpha_{1}\beta_{2}-\sqrt{2}e^{i7\pi/12}\alpha_{2}\beta_{3}\\\