# Fault-Tolerant Four-Dimensional Constellation for Coherent Optical
Transmission Systems

Jingtian Liu$\ast$ Élie Awwad  Yves Jaouën  Communication and Electronics
department, LTCI, Télécom Paris, Institut Polytechnique de Paris, 19 place
Marguerite Perey, 91120 Palaiseau, France
<EMAIL_ADDRESS>
††journal: opticajournal††articletype: Research Article

We propose a 4-dimensional 2-ary amplitude ring-switched modulation format
with 64 symbols, which is denoted as 4D-2A-RS64 encoded over two polarization
tributaries to improve the transmission performance over long-haul optical
fibers in the presence of the non-linear Kerr effect. At a spectral efficiency
of 6 bits per 4D, simulation results show that this format outperforms the
polarization division multiplexed (PDM) 8QAM-star modulation as well as the
4D-2A-8PSK over links without inline dispersion management. We evaluate the
performance for a WDM transmission of $11\times 90~{}\mathrm{Gbaud}$ channels
over a multi-span SSMF link. For an achievable information rate of
$4.8\mathrm{bit/s/Hz}$, the maximum transmission distance is improved by
$10.6\%$ (400 km) and $4\%$ (160 km) compared to PDM-8QAM-star and 4D-2A-8PSK
respectively. The achieved gains are composed of a linear part and a non-
linear part, respectively from the improved Euclidean-distance distribution
and the constant power property of the 4D modulation. The geometric shaping of
the proposed scheme is easy to implement and is robust to Mach-Zehnder
modulator (MZM) imbalances and quantization errors stemming from the finite
digital-to-analog converter (DAC) resolution. This robustness is compared to
the one of other geometric-shaped non-linearity tolerant 4D schemes such as
the 4D-2A-8PSK and the 4D-64PRS that can be both outperformed by our scheme in
severe conditions.

##  1 Introduction

In recent years, several studies have shown that multi-dimensional modulation
formats can improve the performance of coherent transmission systems as they
offer higher Euclidean distances, better spectral efficiency (SE) granularity,
and a potential to mitigate nonlinear fiber effects compared to conventional
formats such as polarization division multiplexed 2D formats like PDM
quadrature phase shift keying (PDM-QPSK) or PDM $M$-QAM (quadrature and
amplitude modulation with $M$ 2D symbols) [1, 2]. For instance, in [3], the
mitigation of cross-polarization modulation (XpolM) was realized by an 8D-QPSK
using the polarization balanced concept with an SE of 2 bits over two
polarization states and one time slot (4D). The experimental demonstration
showed a $1$ dB improvement in net system margin relative to PDM Binary-Phase-
Shift-Keying (PDM-BPSK) over dispersion-managed (DM) systems. Later on, the
same concept was used in [4] to design modulation formats with a higher SE of
2.5 and 3.5 bits per 4D symbol respectively. However, their non-linear gains
in DM systems vanish over dispersion-unmanaged (DUM) systems, because a large
accumulated dispersion destroys their 8D structure. In this work, our interest
is in achieving performance gains over DUM systems using low-complexity multi-
dimensional formats and aiming at higher SE values.

Over the DUM system, for a higher SE of 6 bits per 4D symbol, several 4D
modulation formats such as 4D-64 Set Partitioned (SP)-12QAM [5], Honey Comb
(HC) grid [6] and others [7, 8, 9] were designed by increasing the Euclidean
distances and enhancing the non-linearity tolerance through the minimization
of the peak-to-average-ratio (PAPR) of the constellation which leads to lower
non-linear interference (NLI) generated by both intra-channel and inter-
channel effects [10].

Later on, Kojima et al. in [11, 12] proposed the 4D-2A-8PSK achieving higher
gains, in both linear and nonlinear regimes, than PDM-8QAM-star, PDM-8PSK, and
circular PDM-8QAM over DM systems. The gains stem from a larger Euclidean
distance, a 4D constant modulus constraint, and Gray labeling. The Euclidean
distance gain comes from the nesting of two 8PSK constellations of different
amplitudes followed by set partitioning. The complementary amplitudes of the
two polarization tributaries ensure a constant energy property at each time
slot, thus eliminating energy variations and reducing nonlinear effects. The
Gray labeling also brings an improved achievable information rate (AIR)
compared to PDM-8QAM-star which is not Gray labeled. The authors were able to
achieve flexible SE values from 5 to 7 bit/4D through tailored mapping using
Boolean equations. The ring ratios of the 4D-2A-8PSK are computed based on an
optimization of the AIR at different signal-to-noise ratios (SNR) [13].
However, its optimized ring ratio varies between $0.6$ and $1$ for SNR values
between $5$ and $12$ dB, which shows that it is very sensitive to changes in
the ring ratio. The performance of 4D-2A-8PSK was not explored over DUM
systems.

Then, based on the 4D-2A-8PSK, a 4D-64PRS constellation (PRS standing for
polarization ring switching) was proposed in [14] by a new mapping technique
named ‘Orthant-symmetric labeled constellation’, a method that maps the
possible combinations in quadrants separately to binary labels to facilitate
the upcoming set partitioning. The 4D-64PRS has a slightly lower minimum
squared Euclidean distance (MSED) than 4D-2A-8PSK but also has a lower kissing
number, i.e. number of symbol pairs at MSED, which yields gains of up to 0.7
dB in SNR in the AWGN channel compared with PDM-8QAM-star. It also has a
better performance in the presence of non-linear effects because of its 4D
constant modulus constraint. AIR-based optimization combined with geometric
shaping is used to find the optimum ring ratio and angle for SNR levels
between $3$ and $13$ dB. The optimized ring ratio varies in the range of
$0.53-0.61$ and the relative rotation angle between the two rings is fine-
tuned between $23.4$ to $27.2$ degrees. Both methods combine geometric shaping
[15, 16] and AIR-based optimization [17, 18] to find the best ring ratio and
constellation point locations that maximize the AIR at a given SNR value. More
recently, new geometric shaping schemes achieving higher spectral efficiency
were also investigated in [19, 20]. The fine geometric optimizations increase
the constraints on the generation of the designed constellation (DAC
resolution, stability, and accuracy of optical modulators,…). Moreover, their
performance fluctuates considerably with the design parameters.

Apart from geometric-shaped multi-dimensional designs, probabilistic
constellation shaping (PCS) based schemes [21, 22, 23] were also proposed to
offer both linear and non-linear gains; however, they are not considered in
this work in which we focus on transmissions that do not use distribution
matching to shape and un-shape the probabilities of the transmitted symbols at
the transmitter and receiver side respectively. Leveraging PCS and multi-
dimensional modulations for mitigation of Kerr-induced non-linear interference
is left for future investigation. In this paper, we propose a new
4-dimensional structure based on an 8-PSK modulation surrounding a QPSK on
each polarization, with the outer ring having twice the value of the inner
ring. The Peak-to-Average Power Ratio (PAPR) of the transmitted constellation
is minimized to one by ring switching over the two polarization tributaries.
The new modulation has an SE of 6 bits per 4D. Through numerical simulations,
we demonstrate the advantage in non-linearity tolerance compared to PDM-8QAM
format and other state-of-the-art 4D formats especially when severe
transmitter imperfections are considered.

Achievable information rates (AIRs) are information-theoretic metrics related
to the amount of reliable information that can be transmitted over a given
channel. AIRs usually include symbol-wise mutual information (MI) and
generalized mutual information (GMI), also called bit-wise MI [18, 24]. While
MI is a measure of the maximum amount of transmittable information between the
received and transmitted symbols, GMI takes into account the effect of the bit
mapping and de-mapping. Both metrics have been extensively used to design
modulation formats and to predict the ultimate performance of discrete
modulation formats with forward error correction (FEC) codes [25, 26]. In this
paper, we choose GMI as the performance metric and compute it through
numerical simulations as the achievable rate of a bit-wise decoder defined as
Eq. (16) in [27].

The paper is organized as follows. In section 2, we introduce the structure of
the designed 4D-2A-RS64 and then analyze in section 3 its robustness to
changes induced by I-Q gain and quadrature imbalances and to DAC quantization
at the transmitter side. In section 4, numerical results for our proposed 4D
modulation and three other modulation formats are shown, over an AWGN channel
and then over a non-linear WDM multi-span transmission. Finally, we conclude
by highlighting the benefits of the novel modulation 4D scheme.

##  2 Principle

In multi-span transmission systems, non-linear interference (NLI) arising from
the Kerr effect, and accumulated amplified spontaneous emission noise from
EDFAs dominate the AIR performance of a given modulation format. Fewer
temporal power variations help in reducing the former and a larger minimum
squared Euclidean distance (MSED) of the constellation increases the
performance in the presence of the latter. Hence, we conceive our
constellation in this spirit. Fig. 1 shows the construction and binary
labeling of the proposed scheme, named four-dimensional two-amplitude ring-
switched constellation with 64 symbols: 4D-2A-RS64. Two PSK modulations are
considered: a QPSK with a radius $R_{1}$ and an 8-PSK with a radius
$R_{2}=2R_{1}$. The 8-PSK on Y polarization (Pol Y) has a phase rotation of
$\frac{\pi}{8}$ with respect to the QPSK on X polarization (Pol X) to optimize
the distribution of the Euclidean distances between 4D symbol pairs as will be
shown in Section 4.A. Then, a 4D symbol, defined as two I-Q symbols
multiplexed on two orthogonal polarization tributaries over one time slot,
exists only when the two 2D symbols are colored the same in Fig. 1, which
means that two 2D symbols from different rings $R_{1}$ and $R_{2}$ are chosen,
leading to $64$ combinations. If a QPSK symbol with radius $R_{1}$ is chosen
for Pol X, an 8-PSK symbol with radius $R_{2}$ is selected for Pol Y. This
configuration contains 32 distinct 4D symbols. Inversely, the other
configuration also contains 32 symbols.

Figure 1: 2D-projections of the designed 4D-2A-RS64 modulation and associated
bit-to-symbol mapping with $R_{2}=2R_{1}$. Red (resp. blue) circle markers and
star markers on $R_{2}$ have the same labeled $\left[b_{5},b_{6}\right]$ bits.

Now, we define the bit mapping rules to enhance the performance over an AWGN
channel by maximizing the MSED and minimizing the bit error probability. The
bit mapping is inspired by the one used in [28]. We decompose the 2D
constellation in the I-Q plane into four areas defined by the four quadrants
as shown in Fig. 1 and we associate the symbols on the main axes in Pol X to
the clockwise neighboring quadrant. Each 4D symbol is labeled by a six-bit
vector $\left[b_{1},\dots,b_{6}\right]$ as follows:

  * •

The quadrants in each polarization are defined by 4 out of the 6 bits, namely,
$\left[b_{1},b_{2}\right]$ and $\left[b_{3},b_{4}\right]$ for the first and
second polarization tributary respectively as we show in Fig. 1. These 4 bits
determine which out of the 16 quadrant combinations is to be chosen.

  * •

In each quadrant combination, there are 4 possible combinations of 2D symbols.
$\left[b_{5},b_{6}\right]$ are assigned to those as shown with the labels in
Fig. 1.

A last optimization step of the constellation can typically be a geometric
shaping consisting of changing the ring ratio between $R_{1}$ and $R_{2}$,
$r=R_{1}/R_{2}$, or the relative orientation of the QPSK and 8-PSK
constellations. Based on an optimization of the achievable information rates
taking into account the bit mapping, the authors in [28] and [29] improved the
linear performance of 4D-64PRS and 4D-2A-8PSK over a given signal-to-noise
ratio (SNR) value for a target pre-FEC BER. However, a trade-off exists
between the complexity of the implementation of the geometrically shaped
constellation and the target performance. Indeed, overly precise optimization
is difficult to achieve in practice because it increases the cost and the
footprint of the transmitter (high effective number of bits (ENOB)
requirements at high symbol rates). In Fig. 2(a), we report the optimal ring
ratio (dashed-line curve and right y-axis) for each SNR and find it to be
between $0.47$ and $0.51$. SNR is defined as the symbol energy over the power
spectral density of the AWGN noise. In what follows, the ring ratio of our
scheme is set to $0.5$, which is close to optimal over the studied SNR range.

Figure 2: Solid line on (a), (b), (c): GMI (with optimized parameters for
SNR=$8$ dB) versus SNR over AWGN channel; Dashed line on (a): Optimum ring
ratio $r=R_{1}/R_{2}$ vs SNR. Asterisks: minimum GMI obtained by adding MZM
imbalances of $\pm 15^{\circ}$ for I-Q quadrature and $1.7$ dB for I-Q gain
imbalances. Empty circles: maximum GMI when ring ratio is optimized for each
SNR level. (d) GMI vs SNR for various modulation formats with 6 bit/4D-symbol.
The insets within the figures show the 2D-projections of the three
constellations.

In Fig. 2(d), we report the GMI of the three 4D schemes with optimized
parameters at SNR$=8$ dB and no implementation penalties, and compare them to
the performance of PDM-8QAM-star. At an FEC overhead of $\mathrm{25\%}$,
corresponding to a net rate of $4.8$ bit/4D-symbol, we see that 4D-2A-RS64,
4D-64PRS and 4D-2A-8PSK have $0.4$ dB, $0.7$ dB and $0.3$ dB gain,
respectively, over PDM-8QAM-star. The 4D-2A-RS64 performs better than
4D-2A-8PSK by $0.1$ dB. Conversely, at the SNR required by PDM-8QAM-star to
achieve a net rate of $4.8$ bit/4D-symbols, 4D-2A-RS64 achieves a GMI gain of
$0.15$ bit/4D-symbol compared to PDM-8QAM-star.

##  3 Tolerance to transmitter impairments

Before further investigating the performance of our proposed modulation, we
test its robustness to ring ratio variations and relative rotation angle
variations between the two rings and compare it to the robustness of 4D-64PRS
and 4D-2A-8PSK. These ring and phase variations or deviations from their
optimal values can be introduced by imbalances at the Mach-Zehnder modulator
(MZM) such as gain imbalances, quadrature imbalances or even skew between I
and Q components of each polarization tributary [30, 31]. First, we fix the
structure of the two-dimensional per-polarization projections of all three
constellations using the optimized ring ratio for an SNR of $8$ dB. In this
case, the performance of each 4D modulation is given by the full-line curves
in Fig. 2(a-c) for SNR values between $4$ and $12$ dB. The optimal ring ratio
value is also given in the legend of each sub-figure. The curves plotted with
empty-circle markers show the maximum GMI values of each scheme when using the
optimal ring ratio for each SNR value. Our proposed scheme shows a near-
optimal performance with a fixed ring ratio in the studied SNR range. Later,
we emulate gain and quadrature imbalances at the MZM using the following model
[30]:

$\begin{split}E_{x}&=I_{x}+\alpha_{x}e^{j\theta_{x}}Q_{x}\\\
E_{y}&=I_{y}+\alpha_{y}e^{j\theta_{y}}Q_{y}\end{split}$ (1)

where $I_{x}$ and $Q_{x}$ (resp. $I_{y}$ and $Q_{y}$) are the ideal in-phase
and quadrature components of each polarization tributary, $E_{x}$ and $E_{y}$
are the generated complex amplitude at the outputs of each I-Q modulator,
$\theta_{x}$ (resp. $\theta_{y}$) are the I-Q phase angles for the $x$ (resp.
$y$) polarization tributary and $\alpha_{x}\leq 1$ (resp. $\alpha_{y}\leq 1$)
is a gain discrepancy between the $I$ and the $Q$ components of each
polarization tributary. When $\theta_{x}$ (resp. $\theta_{y}$) are equal to
$90^{\circ}$, the quadrature is perfect between the I and Q components. When
$\alpha_{x}$ and $\alpha_{y}$ are equal to 1, no gain disparity is introduced
between the two components. According to an Optical Internetworking Forum
(OIF) agreement on 400G transceivers [32], that is used as a basis for
interoperability, the maximum allowable fluctuations for the MZM should be in
the range of $\pm 5^{\circ}$ for the I-Q quadrature imbalance and $1$ dB for
the I-Q gain imbalance. To align with these requirements, these imbalances
should be estimated and corrected through dedicated DSP algorithms and
feedback loops. In our study, we chose the ranges for the considered
imbalances similarly to previous works [30, 33, 31], with worst-case values
going beyond the typical values. In Fig. 2(a-c), the worst-case GMI values are
shown through the curves with asterisk markers. These are obtained for maximum
deviations of $\theta_{x}$ (resp. $\theta_{y}$) of $90^{\circ}\pm 15^{\circ}$
and a maximum gain imbalance of $1.7$ dB between $I_{x}$ and $Q_{x}$ (resp.
$I_{y}$ and $Q_{y}$) components. With these simulated imbalances, at SNR$=8$
dB, our 4D-2A-RS64 scheme has a minimum GMI floor of $4.75$ bit/4D-symbol and
GMI fluctuation of $0.17$ bit/4D-sym. It shows better fault tolerance than
4D-64PRS and 4D-2A-8PSK, which have minimum GMI floors of $4.62$ bits/4D-sym
and $4.64$ bits/4D-sym, and GMI fluctuations of $0.36$ bits/4D-sym and $0.21$
bits/4D-sym respectively.

To get a full view of the performance degradation of the three 4D schemes, we
study the GMI variations for an I-Q gain imbalance within the range of $0$ to
$1.8$ dB and an I-Q quadrature imbalance within the range of $0$ to $15$
degrees for each polarization tributary. In Fig. 3(a), we can see that, as the
I-Q gain imbalance increases, the performance of 4D-64PRS deteriorates faster
than the one of the proposed 4D-2A-RS64. The former becomes worse than the
latter for I-Q angle imbalances higher than $12$ degrees in the absence of an
I-Q gain imbalance, and higher than $8$ degrees for an I-Q gain imbalance of
$1.8$ dB. It is important to note that all three 4D modulation schemes possess
symmetrical characteristics along the I and Q axes; consequently, we obtain
similar performance degradation for I-Q gain imbalances between $0$ to $-1.8$
dB and I-Q quadrature imbalances between $0$ and $-15$ degrees. 4D-64PRS
requires the generation of precise amplitude and phase values to achieve the
geometric shaping gain as shown in Fig. 2(b). As the MZM imbalance increases,
its performance gradually deteriorates until it falls below that of
4D-2A-RS64. 4D-2A-8PSK comprises two 8PSK constellations, hence the inner ring
of 8-PSK necessitates a fine amplitude and phase mapping, leading to
continuous performance degradation as the imbalances increase; moreover, its
initial imbalance-free performance is lower than that of 4D-2A-RS64.

Figure 3: (a) GMI versus I-Q gain and quadrature imbalances at SNR=$8$ dB. (b)
Required SNR versus DAC resolutions for different formats at a GMI of 5
bits/4D symbol.

An ideal digital-to-analog converter (DAC) is used in [11] and [14]. However,
it is interesting to take into account the quantization error stemming from
the finite resolution of DACs when evaluating the performance of geometric-
shaped (GS) constellations. Moreover, high-resolution DACs not only add higher
hardware costs but also a higher energy consumption [34, 35]. To test the
impact of DAC quantization, each transmitted sequence is oversampled to 2
samples per symbol, shaped using a root-raised cosine (RRC) filter with a
roll-off factor equal to $0.1$, and uniformly quantized by a DAC with a
vertical resolution between $3$ and $8$ bits. Even though the widely used DACs
and ADCs in high-speed optical transceivers use $6$ to $8$ physical bits, the
effective number of bits (ENoB) that take into account the signal distortions
and noise added by the converters may drop below $4$ bits [36, 37]. In our
study, we consider a frequency independent quantization DAC. The back-to-back
transmission performance is measured over an AWGN channel at a symbol rate of
$90$ Gbaud. In Fig. 3(b), we show the minimum required SNR to achieve a GMI of
$5$ bit/4D-sym. For DAC resolutions ranging from 5 to 8 bits, there is a fine
mapping for each constellation symbol and the SNR penalties are practically
negligible. In such cases, as illustrated in Fig. 2(d), 4D-2A-RS64 incurs a
slight SNR penalty compared to 4D-64PRS. At $4$ bits, the GMI penalties are
$0.1$ dB, $0.2$ dB, and $0.4$ dB for 4D-2A-RS64, 4D-64PRS, and 4D-2A-8PSK
respectively. In the meanwhile, 4D-64PRS requires the lowest SNR value. At $3$
bits, our 4D-2A-RS64 scheme shows a gain of $0.6$ dB and $~{}1.1$dB relative
to 4D-64PRS and 4D-2A-8PSK respectively. The penalty of the 4D-64PRS comes
from its sensitivity to phase changes; indeed, the phases of the outer ring
constellation points cannot be accurately achieved with 3-bit resolution DACs.
The 4D-2A-8PSK penalty comes from the use of an 8-PSK on its inner ring which
requires a higher resolution to generate the constellation accurately.
Although our proposed scheme outperforms the 4D-64PRS only when the studied
transceiver impairments are severe (I-Q imbalance of more than $10$ degrees or
a DAC resolution lower than $4$ bits), it still shows an advantage with
respect to 4D-2A-8PSK which was implemented in commercially available
transceivers as mentioned in [38]. We believe that our proposed scheme merits
to be considered as a relevant candidate among geometric-shaped (GS) 4D
modulations as recent works are still looking into the design optimization of
GS 4D schemes using machine learning tools [39] or simplified computation of
AIR [40].

##  4 Performance evaluation

###  4.1 Analysis of the performance over an AWGN channel

Table 1:  Metrics for different 4D constellations | PDM-8QAM-star | 4D-2A-8PSK | 4D-64PRS | 4D-2A-RS64
---|---|---|---|---
MSED | 0.84 | 0.88 | 0.69 | 0.8
$n_{d}$ | 192 | 128 | 32 | 64
|  $(d_{D_{H}=1}^{2}$,
---
$n_{D_{H}=1})$
| (0.84, 128)
---
(3.15, 64)
| (0.88, 128)
---
(3.46, 64)
| (0.69, 32)
---
(0.90, 64)
(0.98, 64)
(5.50, 32)
| (0.8, 64)
---
(0.94, 64)
(1.39, 16)
(3.2, 32)
(5.46, 16)
Gray Labeled | No | Yes | Yes | Yes
Constant Modulus | No | Yes | Yes | Yes
PAPR | 1.58 | 1 | 1 | 1
Figure 4:  Histograms of SEDs in three 4D formats: (a) 4D-2A8PSK, (b)
4D-64PRS, and (c) Our 4D-2A-RS64. The red bars show the number of pairs with
Hamming distance of 1 at the SED $d^{2}$. The blue bars show the number of
pairs with Hamming distance greater than 1 at the SED $d^{2}$.

To easily compare the performance of the four formats, we list the following
metrics and features for each format in Table 1:

  * •

$\mathrm{MSED}$: the Minimum Squared Euclidean Distance between 4D-symbols.
Each polarization tributary is normalized to a unit energy.

  * •

$n_{d}$: the number of pairs of 4D symbols at MSED.

  * •

The pairs $(d_{D_{H}=1}^{2},n_{D_{H}=1})$ contain the values of Squared
Euclidean Distances (SEDs) for which the Hamming distance ($D_{H}$) between
the mapped bit patterns is equal to $1$ and the number of symbols at those
SEDs.

  * •

Gray Labeled indicates whether all the constellation pairs at MSED are at a
Hamming distance of 1.

  * •

Constant Modulus indicates whether the 4D-energy is constant and PAPR is the
peak-to-average power ratio.

From Table 1, we see that PDM-8QAM has the worst linear performance because it
has a moderate MSED and an excessive $n_{d}$, in addition to the absence of
Gray labeling. Among the three 4D formats, 4D-2A-8PSK has the largest MSED
thus obtaining the best performance for high SNRs, followed by 4D-2A-RS64,
then 4D-64PRS. In addition, among the four schemes, 4D-64PRS has the smallest
number of symbol pairs at MSED, $n_{d}$, thus obtaining the best performance
at low SNR values, followed by 4D-2A-RS64, then 4D-2A-8PSK with an increasing
$n_{d}$. All three 4D formats are Gray-labeled and have a constant 4D modulus.
Finally, to complement Table 1, we show in Fig. 4 the SED distribution for the
three 4D formats. From these histograms and the third row in Tab. 1, we see
that, although the 4D-64PRS has the smallest MSED, only $32$ pairs are located
at this distance. Also, the pairs at the three smallest SEDs have a Hamming
distance equal to $1$, which provides gains at low SNR values. The 4D-2A-RS64
constellation has a smaller MSED than the 4D-2A-8PSK, but only $64$ pairs are
located at MSED compared to $128$ for the latter. In addition, for its $128$
pairs with the smallest SED, 4D-2A-RS64 has $64$ pairs at an SED of $0.94$
that contribute to a small performance improvement compared to 4D-2A-8PSK at
low SNR values. The loss of performance compared to 4D-64PRS is mainly due to
the $32$ symbol pairs at a SED of $0.94$ and with a Hamming distance larger
than $1$ as can be seen in Fig. 4(c).

###  4.2 Performance over the non-linear channel

Over long-haul multi-span fiber transmissions, NLI induced by the Kerr effect
will add distortions to the propagating signal in addition to the additive ASE
noise from the optical amplifiers. This NLI can be reduced by forcing a
constant energy constraint on the transmitted signal, hence minimizing the
peak-to-average power ratio (PAPR) at the transmitter side which leads to
lower NLI generated by both intra-channel and inter-channel propagation
effects [10]. We test the performance gain over fiber propagation through
numerical simulations. We compare the BERs, system margins, and GMI values of
the three 4D formats to the conventional format PDM-8QAM-star, all having an
entropy of 6 bit/4D. The transmitted signal consists of $11$ WDM channels
$\times~{}90$ Gbaud, each modulated over dual-polarization. The channel
spacing is 100 GHz and the central channel is located at $1550$ nm. All
channels are pulse-shaped with a root-raised-cosine filter (RRC) with a roll-
off factor equal to $0.1$. In order to better compare linear gain, nonlinear
gain, and gain changes under MZM imbalance, we used an ideal laser with no
phase noise. The tested fibers are standard single mode fibers (SSMF) with a
chromatic dispersion coefficient $D=17~{}\mathrm{ps/nm/km}$, a non-linearity
coefficient $\gamma=1.32~{}\mathrm{(W\cdot km)^{-1}}$, a fiber attenuation
parameter $\alpha=0.2~{}\mathrm{dB/km}$. Polarization mode dispersion (PMD) is
added with a PMD coefficient equal to $0.04~{}\mathrm{ps/\sqrt{km}}$. The link
configuration was a homogeneous multi-span link with $N\times
80~{}\mathrm{km}$ spans. The propagation was modeled through a split-step
Fourier method (SSMF) implementing the non-linear Manakov equation with 50
wave plates per span to simulate the polarization effects [41]. Each span was
followed by an erbium-doped fiber amplifier (EDFA) with a noise figure of 5
dB. After propagation, the central channel is filtered and detected through a
coherent receiver. We then apply on the detected signal a matched RRC filter
with the same roll-off factor, and digitally compensate for the chromatic
dispersion. Genie-aided linear channel equalization is then applied to
compensate for the polarization changes prior to decoding (linear channel
response is saved and fed to the frequency-domain MMSE channel equalizer).
Finally, a genie-aided XPM-induced phase estimation filter with an averaging
window of $\mathrm{\omega=64}$ symbols is used to compensate for the XPM-
dominated phase rotation as in [42]. With these two perfect estimations, the
measured performance is only related to the used modulation format (no penalty
caused by imperfect channel estimation). No additional non-linear compensation
algorithms such as digital back-propagation were used at the receiver side.

Figure 5: GMIs versus transmitted power after transmission over 20 spans of 80
km SSMF in the non-linear regime for 11 channels with 100 GHz spacing with a
symbol rate of 90 Gbaud. Left: Overall trends. Right: Zoomed trends around a
GMI of $4.8$ bit/4D.

Fig. 5 shows GMI curves versus single-channel launch power values after
transmission over 20 spans of 80km SSMF. All three 4D formats outperform
PDM-8QAM-star in the linear regime as well as in the non-linear regime. At an
FEC overhead of $\mathrm{25\%}$ corresponding to a net rate of $4.8$
bit/4D-symbol, we zoomed in on the corresponding area to better show the
achieved gains. 4D-2A-RS64, 4D-64PRS and 4D-2A-8PSK show $0.5$ dB, $0.73$ dB
and $0.25$ dB gains, respectively, over PDM-8QAM-star. 4D-2A-RS64 performs
better than 4D-2A-8PSK by $0.25$ dB. While it does not beat the performance of
4D-64PRS, 4D-2A-RS64 does not require precise geometric shaping to maintain
these gains and is less sensitive to implementation errors. At the SNR
required for PDM-8QAM-star to reach 4.8 bit/4D-symbol, the GMI of 4D-2A-RS64
is 0.2bit/4D higher than PDM-8QAM-star. We obtain a higher gain in the fiber
channel than in the back-to-back case because the 4D modulations are
mitigating NLI in the presence of Kerr effect better than PDM-8QAM. It is also
worth noting that in the single-channel test, we obtain nearly the same gains
(not presented in Fig. 5), which is attributed to the constant modulus
characteristics of the three 4D formats that effectively reduce the generation
of self-phase modulation (SPM).

Figure 6: (a) Pre-FEC and post-FEC BERs versus transmitted power after
transmission over 20 spans of 80km SSMF in the non-linear regime for 11
channels with 100 GHz spacing and a symbol rate of 90 Gbaud. (b) System margin
(SNR gain compared to SNR with BER threshold = $4\times 10^{-2}$) versus
transmitted power after transmission over 20 spans of 80 km SSMF in the non-
linear regime for 11 channels with 100 GHz spacing with a symbol rate of 90
Gbaud.

In Fig. 6(a), we show post-FEC and pre-FEC BER curves versus launch power
after transmission over 20 spans. We used low-density parity-check (LDPC)
codewords of length $64800$ bits from the DVB-S2 standard [43] with a rate
$R_{C}=0.8$ and belief propagation (BP) decoding at the receiver side with 50
decoding iterations for a pre-FEC BER of $\mathrm{4\times 10^{-2}}$. The
4D-2A-RS64 gain in post-FEC BER compared to PDM-8QAM-star is in good agreement
with the predictions of GMI gain of $0.5$ dB in Fig. 5. At the optimal power
operation point, all three 4D formats have a lower pre-FEC BER than PDM-8QAM-
star. 4D-2A-RS64 and 4D-64PRS achieve the same minimum pre-FEC BER while
4D-2A-8PSK has a slightly lower pre-FEC BER value. While the three 4D schemes
have the same constant modulus property at the transmitter side, the lower
minimum pre-FEC BER value of 4D-2A-8PSK is mainly due to its highest MSED
value among the three schemes.

As the GMI of the three 4D formats saturates around the optimal power in Fig.
5, we also show in Fig. 6(b) the system margin versus launch power for a
target pre-FEC BER of $4\times 10^{-2}$. The system margin is defined as the
difference between the required $\mathrm{SNR_{elec}}$ to achieve a given
target pre-FEC BER and the $\mathrm{SNR_{elec}}$ at any launch power of the
transmission system over the studied link configuration. $\mathrm{SNR_{elec}}$
is the SNR measured from the constellation at the end of the DSP chain at the
receiver side (i.e., at the input of the decision circuit). The system margin
captures both the linear gain and the non-linear gain brought by the reduced
energy variations of the considered 4D schemes. 4D-2A-RS64, 4D-64PRS, and
4D-2A-8PSK show $0.5$ dB, $0.8$ dB, and $0.3$ dB gain in system margin,
respectively, over PDM-8QAM-star at the optimal launch power. Although
chromatic dispersion will gradually increase the PAPR of the transmitted
signal, we still measure gains when the signal has a constant-energy property
at the transmitter side. We observe similar performance trends for the four
schemes when propagating a single $90~{}\mathrm{Gbaud}$ channel over the same
link.

Figure 7: (a) GMIs versus the number of spans [80 km/span] for different 4D
modulation formats with 6 bit/4D-symbol at 90 Gbaud. Solid lines with empty
circles: GMI without MZM imbalances. Dashed line with filled circles: minimum
GMI obtained by adding MZM imbalances of $\pm 15^{\circ}$ for the I-Q
quadrature and $1.7$ dB for the I-Q gain imbalances. (b) GMIs versus the
number of spans [80 km/span] for different 4D modulation formats with 6
bit/4D-symbol at 45 Gbaud.

Finally, the solid lines in Fig. 7(a) show GMI curves of the central channel
versus the transmission distance given as a number of spans at the optimal
launch power for each distance when transmitting $90~{}\mathrm{Gbaud}\times
11$ channels. For a net rate of $4.8$ bit/4D-symbol, 4D-2A-RS64 increases the
covered distance by $400$ km relative to PDM-8QAM-star and more than $160$ km
relative to 4D-2A-8PSK, an increase of $\mathrm{10.6\%}$ and $\mathrm{4\%}$,
respectively. 4D-64PRS obtains the best performance, allowing transmission
over two extra spans compared to 4D-2A-RS64. At the maximum transmission
distance of PDM-8QAM-star, which corresponds to 47 spans, 4D-2A-RS64 offers a
GMI gain of $0.2$ bit/4D-symbol which is higher than the linear gain observed
in Fig. 2(d). The relative additional non-linear gain in terms of GMI for the
nonlinear channel with respect to the gain in the linear channel is
$(0.2-0.15)/0.15=33\%$, which is higher than the relative gains of 4D-2A-8PSK
and 4D-64PRS, $(0.13-0.1)/0.1=30\%$ and $(0.29-0.26)/0.26=11.5\%$
respectively. Next, we add the same worst-case simulated MZM imbalances as the
ones used for the linear AWGN channel. The dashed lines in Fig. 7(a) show that
our 4D-2A-RS64 scheme has the best performance, allowing the transmission over
almost 2 extra spans compared to 4D-64PRS and 5 extra spans compared to
PDM-8QAM-star.

We also test the performance at a lower channel rate. In Fig. 7(b), we show
the results for $45~{}\mathrm{Gbaud}\times 11$ channels. First, for a perfect
transmitter (solid lines), at a GMI of $4.8$ bit/4D-symbol, 4D-2A-RS64
increases the covered distance by 800km relative to PDM-8QAM-star and more
than 160 km relative to 4D-2A-8PSK, an increase of $\mathrm{12.5\%}$ and
$\mathrm{2.3\%}$ respectively. Since the dispersion-induced inter-symbol-
interference grows more slowly over the propagation distance with symbols at
$45$ Gbaud than at $90$ Gbaud, the PAPR has a slower increase over distance.
Moreover, the optimal power will also be higher for higher baud rates, hence
the non-linear effects will be magnified and will deteriorate the signal even
more. If we compare gains in transmission distance for the three 4D constant
modulus formats when moving from $45$ Gbaud to $90$ Gbaud, we notice that
4D-64PRS and 4D-2A-RS64 offer a gain of 8 spans (respectively 5) instead of 16
spans (respectively 10) with respect to PDM-8QAM, showing a gain reduction of
$50\%$ while the gain of 4D-2A-8PSK reduces from 8 to 3 spans, showing a gain
reduction of $62.5\%$. Finally, with the simulated worst-case MZM imbalances,
our 4D-2A-RS64 also shows the best performance, allowing transmission over 5
extra spans compared to 4D-64PRS and 7 extra spans compared to PDM-8QAM-star.

##  5 Conclusion

We designed a new 4D constellation, named 4D-2A-RS64, with an entropy of $6$
bit/4D, which showed better linear performance and higher fiber non-linearity
tolerance than conventional modulation formats PDM-8QAM-star or a recent 4D
format 4D-2A-8PSK optimized for the non-linear channel. We demonstrated
through numerical simulations that the maximum transmission distances are
increased with respect to both formats for different symbol rates. While the
proposed modulation did not beat the performance of another 4D competitor, the
4D-64PRS format, we demonstrated that 4D-2A-RS64 imposes less stringent
constraints on the transmitter compared to the other 4D formats. Hence, its
generation using non-ideal components (extreme imbalances in MZMs and limited-
resolution DACs) shows smaller performance degradation than state-of-the-art
4D formats such as 4D-2A-8PSK and 4D-64PRS in both linear and non-linear
transmission regimes. In the design of future high-order modulation schemes
with higher spectral efficiency, we can still optimize linear performance by
increasing MSED and considering the distribution of all squared Euclidean
distances. For the nonlinear part, it is crucial to maintain temporal energy
fluctuations as much as possible. Furthermore, higher spectral efficiency
often requires more symbols and more precise mapping, which means that
sensitivity to MZM imbalance and DAC variations will increase. Extending the
proposed solution to achieve even higher spectral efficiency by using more
rings remains an open problem for future exploration. In addition, the design
of signaling schemes that are optimized for the non-linear fiber channel using
both multi-dimensional modulations and probabilistic shaping could help in
achieving additional linear and non-linear gains as it has been suggested in
[44, 45].

## Acknowledgments.

This work was supported by Huawei Technologies France. We thank Djalal
Bendimerad and Hartmut Hafermann for fruitful discussions on multi-dimensional
formats.

## Disclosures.

The authors declare no conflict of interest.

## Data availability.

Data underlying the results presented in this paper are not publicly available
at this time but may be obtained from the authors upon reasonable request.

## References

  * [1] M. Karlsson and E. Agrell, “Which is the most power-efficient modulation format in optical links?” Optics express 17, 10814–10819 (2009).
  * [2] G. Welti and J. Lee, “Digital transmission with coherent four-dimensional modulation,” IEEE Transactions on Information Theory 20, 497–502 (1974).
  * [3] A. Shiner, M. Reimer, A. Borowiec, S. O. Gharan, J. Gaudette, P. Mehta, D. Charlton, K. Roberts, and M. O’Sullivan, “Demonstration of an 8-dimensional modulation format with reduced inter-channel nonlinearities in a polarization multiplexed coherent system,” Optics express 22, 20366–20374 (2014).
  * [4] D. F. Bendimerad, H. Hafermann, and H. Zhang, “Nonlinearity-tolerant 8D modulation formats by set-partitioning PDM-QPSK,” in _2018 Optical Fiber Communications Conference and Exposition (OFC),_ (IEEE, 2018), pp. 1–3.
  * [5] T. Nakamura, E. L. T. de Gabory, H. Noguchi, W. Maeda, J. Abe, and K. Fukuchi, “Long haul transmission of four-dimensional 64SP-12QAM signal based on 16QAM constellation for longer distance at same spectral efficiency as PM-8QAM,” in _2015 European Conference on Optical Communication (ECOC),_ (IEEE, 2015), pp. 1–3.
  * [6] H. Buelow, X. Lu, L. Schmalen, A. Klekamp, and F. Buchali, “Experimental performance of 4D optimized constellation alternatives for PM-8QAM and PM-16QAM,” in _Optical Fiber Communication Conference,_ (Optica Publishing Group, 2014), pp. M2A–6.
  * [7] M. Sjodin, E. Agrell, and M. Karlsson, “Subset-optimized polarization-multiplexed PSK for fiber-optic communications,” IEEE communications letters 17, 838–840 (2013).
  * [8] R. Rios-Müller, J. Renaudier, L. Schmalen, and G. Charlet, “Joint coding rate and modulation format optimization for 8QAM constellations using BICM mutual information,” in _Optical Fiber Communication Conference,_ (Optica Publishing Group, 2015), pp. W3K–4.
  * [9] Q. Zhuge, M. Reimer, A. D. Shiner, A. Borowiec, D. Charlton, F. Zhang, Q. Meng, W. Wei, D. V. Plant, and M. O’Sullivan, “Flexible modulation and nonlinear tolerance for coherent transceivers,” in _Opto-Electronics and Communications Conference, OECC 2015,_ (Institute of Electrical and Electronics Engineers Inc., 2015), p. 7340127.
  * [10] R. Dar, M. Feder, A. Mecozzi, and M. Shtaif, “Properties of nonlinear noise in long, dispersion-uncompensated fiber links,” Optics Express 21, 25685–25699 (2013).
  * [11] K. Kojima, T. Yoshida, T. Koike-Akino, D. S. Millar, K. Parsons, M. Pajovic, and V. Arlunno, “Nonlinearity-tolerant four-dimensional 2A8PSK family for 5–7 bits/symbol spectral efficiency,” Journal of Lightwave Technology 35, 1383–1391 (2017).
  * [12] K. Kojima, T. Koike-Akino, D. S. Millar, and K. Parsons, “BICM capacity analysis of 8QAM-alternative modulation formats in nonlinear fiber transmission,” in _2015 Tyrrhenian International Workshop on Digital Communications (TIWDC),_ (IEEE, 2015), pp. 57–59.
  * [13] A. Alvarado, E. Agrell, D. Lavery, and P. Bayvel, “LDPC codes for optical channels: Is the “FEC limit” a good predictor of post-FEC BER?” in _Optical Fiber Communication Conference,_ (Optica Publishing Group, 2015), pp. Th3E–5.
  * [14] B. Chen, C. Okonkwo, H. Hafermann, and A. Alvarado, “Polarization-ring-switching for nonlinearity-tolerant geometrically shaped four-dimensional formats maximizing generalized mutual information,” Journal of Lightwave Technology 37, 3579–3591 (2019).
  * [15] S. Zhang, F. Yaman, E. Mateo, T. Inoue, K. Nakamura, and Y. Inada, “Design and performance evaluation of a GMI-optimized 32QAM,” in _2017 European Conference on Optical Communication (ECOC),_ (IEEE, 2017), pp. 1–3.
  * [16] B. Chen, C. Okonkwo, H. Hafermann, and A. Alvarado, “Increasing achievable information rates via geometric shaping,” in _2018 European Conference on Optical Communication (ECOC),_ (IEEE, 2018), pp. 1–3.
  * [17] B. Chen, C. Okonkwo, D. Lavery, and A. Alvarado, “Geometrically-shaped 64-point constellations via achievable information rates,” in _2018 20th International Conference on Transparent Optical Networks (ICTON),_ (IEEE, 2018), pp. 1–4.
  * [18] A. Alvarado and E. Agrell, “Four-dimensional coded modulation with bit-wise decoders for future optical communications,” Journal of Lightwave Technology 33, 1993–2003 (2015).
  * [19] E. Sillekens, G. Liga, D. Lavery, P. Bayvel, and R. I. Killey, “High-cardinality geometrical constellation shaping for the nonlinear fibre channel,” Journal of Lightwave Technology 40, 6374–6387 (2022).
  * [20] S. Goossens, Y. C. Gültekin, O. Vassilieva, I. Kim, P. Palacharla, C. Okonkwo, and A. Alvarado, “Introducing 4D Geometric Shell Shaping for Mitigating Nonlinear Interference Noise,” Journal of Lightwave Technology 41, 599–609 (2022).
  * [21] K. Wu, G. Liga, A. Sheikh, F. M. Willems, and A. Alvarado, “Temporal energy analysis of symbol sequences for fiber nonlinear interference modelling via energy dispersion index,” Journal of Lightwave Technology 39, 5766–5782 (2021).
  * [22] K. Wu, G. Liga, A. Sheikh, Y. C. Gültekin, F. M. Willems, and A. Alvarado, “List-encoding CCDM: A nonlinearity-tolerant shaper aided by energy dispersion index,” Journal of Lightwave Technology 40, 1064–1071 (2022).
  * [23] V. Neskorniuk, A. Carnio, D. Marsella, S. K. Turitsyn, J. E. Prilepsky, and V. Aref, “Memory-aware end-to-end learning of channel distortions in optical coherent communications,” Optics Express 31, 1–20 (2023).
  * [24] L. Schmalen, A. Alvarado, and R. Rios-Müller, “Performance prediction of nonbinary forward error correction in optical transmission experiments,” Journal of Lightwave Technology 35, 1015–1027 (2017).
  * [25] A. Alvarado, E. Agrell, D. Lavery, R. Maher, and P. Bayvel, “Replacing the soft-decision FEC limit paradigm in the design of optical communication systems,” Journal of Lightwave Technology 33, 4338–4352 (2015).
  * [26] A. Alvarado, T. Fehenberger, B. Chen, and F. M. Willems, “Achievable information rates for fiber optics: Applications and computations,” Journal of Lightwave Technology 36, 424–439 (2017).
  * [27] A. Alvarado and E. Agrell, “Four-dimensional coded modulation with bit-wise decoders for future optical communications,” Journal of Lightwave Technology 33, 1993–2003 (2015).
  * [28] B. Chen, C. Okonkwo, H. Hafermann, and A. Alvarado, “Eight-dimensional polarization-ring-switching modulation formats,” IEEE Photonics Technology Letters 31, 1717–1720 (2019).
  * [29] K. Kojima, D. S. Millar, T. Koike-Akino, and K. Parsons, “Constant modulus 4D optimized constellation alternative for DP-8QAM,” in _2014 The European Conference on Optical Communication (ECOC),_ (IEEE, 2014), pp. 1–3.
  * [30] S. A. Jacobs and R. A. Marsland, “Constellation recovery and impairment evaluation through minimization of the blind evm,” Opt. Express 24, 21708–21721 (2016).
  * [31] J. Liang, Y. Fan, Z. Tao, X. Su, and H. Nakashima, “Transceiver Imbalances Compensation and Monitoring by Receiver DSP,” Journal of Lightwave Technology 39, 5397–5404 (2021).
  * [32] F. OFI Internetworking Forum, “Implementation agreement 400zr,” (2022).
  * [33] C. R. S. Fludger and T. Kupfer, “Transmitter Impairment Mitigation and Monitoring for High Baud-Rate, High Order Modulation Systems,” in _ECOC 2016; 42nd European Conference on Optical Communication,_ (2016), pp. 1–3.
  * [34] T. Pfau, S. Hoffmann, and R. Noé, “Hardware-efficient coherent digital receiver concept with feedforward carrier recovery for $M$-QAM constellations,” Journal of Lightwave Technology 27, 989–999 (2009).
  * [35] S. Almonacil, F. Boitier, and P. Layec, “Performance model and design rules for optical systems employing low-resolution DAC/ADC,” Journal of Lightwave Technology 38, 3007–3014 (2020).
  * [36] C. Laperle, “Advances in high-speed adcs, dacs, and dsp for optical transceivers,” in _Optical Fiber Communication Conference/National Fiber Optic Engineers Conference 2013,_ (Optica Publishing Group, 2013), p. OTh1F.5.
  * [37] X. Chen, S. Chandrasekhar, S. Randel, W. Gu, and P. Winzer, “Experimental Quantification of Implementation Penalties from Limited ADC Resolution for Nyquist Shaped Higher-Order QAM,” in _Optical Fiber Communication Conference,_ (Optica Publishing Group, 2016), p. W4A.3.
  * [38] O. Vassilieva, I. Kim, and P. Palacharla, “Practical Considerations for Geometric and Probabilistic Shaping in Optical Communication Systems,” in _2023 European Conference on Optical Communication (ECOC),_ (IEEE, 2023), p. M.B.3.1.
  * [39] M. Yankov, O. Jovanovic, D. Zibar, and F. Da Ros, “Rate Adaptive Geometric Constellation Shaping Using Autoencoders and Many-To-One Mapping,” in _2023 European Conference on Optical Communication (ECOC),_ (IEEE, 2023), p. M.B.3.4.
  * [40] K. Gumus, B. Chen, T. Bradley, and C. Oknonkwo, “A Simplified Method for Optimising Geometrically Shaped Constellations of Higher Dimensionality,” in _2023 European Conference on Optical Communication (ECOC),_ (IEEE, 2023), p. We.A.6.3.
  * [41] S. Fazel, D.-F. Bendimerad, N. Rossi, P. Ramantanis, and Y. Frignac, “Influence of Modeling Methods on the Estimation of the Nonlinear Noise Statistics Considering Joint PMD and Kerr Effects in Fiber Transmission Systems1,” arXiv preprint arXiv:2002.06268 (2020).
  * [42] T. Fehenberger, M. P. Yankov, L. Barletta, and N. Hanik, “Compensation of XPM interference by blind tracking of the nonlinear phase in WDM systems with QAM input,” in _2015 European Conference on Optical Communication (ECOC),_ (IEEE, 2015), pp. 1–3.
  * [43] E. ETSI, “Digital video broadcasting (dvb); second generation framing structure, channel coding and modulation systems for broadcasting, interactive services, news gathering and other broadband satellite applications,” Part II: S2-Extensions (DVB-S2X) pp. 22–27 (2005).
  * [44] T. Fehenberger, A. Alvarado, G. Böcherer, and N. Hanik, “On probabilistic shaping of quadrature amplitude modulation for the nonlinear fiber channel,” J. Lightwave Technol. 34, 5063–5073 (2016).
  * [45] P. Skvortcov, I. Phillips, W. Forysiak, T. Koike-Akino, K. Kojima, K. Parsons, and D. S. Millar, “Huffman-coded sphere shaping for extended-reach single-span links,” IEEE Journal of Selected Topics in Quantum Electronics 27, 1–15 (2021).