# Pulse Jitter and Single-pulse Variability in Millisecond Pulsars

S.Q. Wang  Xinjiang Astronomical Observatory, Chinese Academy of Sciences,
Urumqi, Xinjiang 830011, China  Key Laboratory of Radio Astronomy, Chinese
Academy of Sciences, Urumqi, Xinjiang, 830011, China  Xinjiang Key Laboratory
of Radio Astrophysics, Urumqi, Xinjiang, 830011, People’s Republic of China
N. Wang  Xinjiang Astronomical Observatory, Chinese Academy of Sciences,
Urumqi, Xinjiang 830011, China  Key Laboratory of Radio Astronomy, Chinese
Academy of Sciences, Urumqi, Xinjiang, 830011, China  Xinjiang Key Laboratory
of Radio Astrophysics, Urumqi, Xinjiang, 830011, China  J.B. Wang  Institute
of Optoelectronic Technology, Lishui University, Lishui, Zhejiang, 323000,
China  G. Hobbs  CSIRO Astronomy and Space Science, PO Box 76, Epping, NSW
1710, Australia  H. Xu  National Astronomical Observatories, Chinese Academy
of Sciences, Beijing 100101, China  B.J. Wang  National Astronomical
Observatories, Chinese Academy of Sciences, Beijing 100101, China  S. Dai
School of Science, Western Sydney University, Locked Bag 1797, Penrith South
DC, NSW 2751, Australia  S.J. Dang  School of Physics and Electronic Science,
Guizhou Normal University, Guiyang 550001, China  D. Li  National Astronomical
Observatories, Chinese Academy of Sciences, Beijing 100101, China  Y. Feng
Zhejiang Lab, Hangzhou, Zhejiang 311121, China  C.M. Zhang  National
Astronomical Observatories, Chinese Academy of Sciences, Beijing 100101, China

###### Abstract

Understanding the jitter noise resulting from single-pulse phase and shape
variations is important for the detection of gravitational waves using pulsar
timing array. We presented measurements of jitter noise and single-pulse
variability of 12 millisecond pulsars that are part of the International
Pulsar Timing Array sample using the Five-hundred-meter Aperture Spherical
radio Telescope (FAST). We found that the levels of jitter noise can vary
dramatically among pulsars. A moderate correlation with a correlation
coefficient of 0.57 between jitter noise and pulse width is detected. To
mitigate jitter noise, we performed matrix template matching using all four
Stokes parameters. Our results revealed a reduction in jitter noise ranging
from 6.7% to 39.6%. By performing longitude-resolved fluctuation spectrum
analysis, we identified periodic intensity modulations in 10 pulsars. In PSR
J0030+0451, we detected single-pulses with energies more than 10 times the
average pulse energy, suggesting the presence of giant pulses. We also
observed a periodic mode-changing phenomenon in PSR J0030+0451. We examined
the achievable timing precision by selecting a sub-set of pulses with a
specific range of peak intensity, but no significant improvement in timing
precision is achievable.

Radio pulsars (1353); Millisecond pulsars (1062)

††software: DSPSR (van Straten & Bailes, 2011), PSRCHIVE (Hotan et al., 2004),
TEMPO2 (Hobbs et al., 2006), and PSRSALSA (Weltevrede, 2016)

##  1 INTRODUCTION

Pulsar timing enable the most stringent tests of fundamental physics,
including the constrain of nuclear physics at extreme densities (Demorest et
al., 2010), testing general relativity (Kramer et al., 2006), detecting and
characterizing low-frequency gravitational waves (GWs, Hellings & Downs 1983;
Jenet et al. 2005), improving the solar system planetary ephemeris (Guo et
al., 2019) and developing a pulsar-based timescale (Hobbs et al., 2020). By
monitoring the pulse times of arrival (ToAs) of an ensemble of the most stable
millisecond pulsars (MSPs), known as the pulsar timing array (PTA), it is
possible to detect nanohertz GWs (Detweiler, 1979; Hellings & Downs, 1983;
Foster & Backer, 1990; Jenet et al., 2005). The nanohertz GW sources include
inspiraling supermassive black hole binaries, decaying cosmic string networks,
relic post-inflation GWs, and even non-GW imprints of axionic dark matter
(Burke-Spolaor et al., 2019).

There are currently six individual PTAs, namely the European Pulsar Timing
Array (EPTA; Kramer & Champion 2013), the North American Nanohertz Observatory
for Gravitational Waves (NANOGrav; McLaughlin 2013), the Parkes Pulsar Timing
Array (PPTA; Manchester & IPTA 2013), the MeerKAT Pulsar Timing Array (MPTA;
Miles et al. 2023, the Indian Pulsar Timing Array (InPTA; Tarafdar et al.
2022), and the Chinese Pulsar Timing Array (CPTA; Xu et al. 2023). The timing
data of thses individual PTAs have been used to search for GWs, e.g. Agazie et
al. (2023); Antoniadis et al. (2023); Reardon et al. (2023); Xu et al. (2023).
The individual PTAs also collaborate under the International Pulsar Timing
Array (IPTA), pooling their data sets to improve the detection sensitivity of
GWs (Hobbs et al., 2010; Manchester & IPTA, 2013; Verbiest et al., 2016;
Perera et al., 2019).

The success of GW detection with PTA requires the highest timing precision
possible. The precision of ToAs is limited by many noise contributions,
including those introduced by the pulsar itself, the interstellar medium along
the line of sight, and the measurement process (Cordes & Shannon, 2010; Lam et
al., 2016; Verbiest & Shaifullah, 2018). On short timescales, the excess noise
in the timing residuals is typical dominated by white noise, which includes
radiometer noise, jitter noise and scintillation noise (Liu et al., 2012;
Shannon et al., 2014; Lam et al., 2019). Radiometer noise describes the noise
temperature/power in the profile, and its magnitude depends on the signal-to-
noise ratio (S/N) of the average profile. Jitter noise is a stochastic process
that affects all pulsars and is induced by intrinsic variations in single-
pulse amplitude and phase (Cordes & Downs, 1985). Scintillation noise is
caused by changes in the interstellar impulse response from multipath
scattering (Cordes & Shannon, 2010). The pulse broadening function results
from diffractive interstellar scattering/scintillation (DISS), which depends
on observational frequency strongly.

Jitter noise is expected to dominate the white noise budget in high S/N
observations (Shannon et al., 2014). Several studies have been conducted to
investigate the effects of jitter noise on pulsar timing. Using Parkes
telescope, Osłowski et al. (2011) determined that the timing precision of PSR
J0437$-$4715 is limited to approximately 30 ns in an hour duration due to the
presence of jitter noise. By measuring the jitter noise of 22 MSPs as a part
of PPTA, Shannon et al. (2014) found that PSR J1909-3744 shows the lowest
levels of jitter noise of about 10 ns in an hour duration. Lam et al. (2019)
detected jitter noises in 43 MSPs as part of NANOGrav and found that the level
of jitter noise is frequency-dependent. Recently, Parthasarathy et al. (2021)
conducted jitter measurements in 29 pulsars using the MeerKAT radio telescope
and identified that PSR J2241$-$5236 shows the lowest jitter level of about 4
ns in an hour duration.

The Five-hundred-meter Aperture Spherical radio Telescope (FAST) is the most
sensitive single dish telescope, providing an opportunity to detect and
constrain jitter noises in MSPs, e.g. Feng et al. (2021); Wang et al. (2020,
2021). In the paper, we present measurements of jitter noise and single-pulse
variability in 12 MSPs using FAST. In Section 2, we describe our observations.
In Section 3, we present the measurements of jitter noises, the single-pulse
properties, and examine the achievable timing precision by using a subset of
pulses with a specific range of peak intensity. we discuss and summarize our
results in Section 4.

##  2 OBSERVATIONS AND DATA PROCESSING

For our observations of 12 MSPs, we used the central beam of the 19-beam
receiver of FAST with a frequency range of 1000 to 1500 MHz (Jiang et al.,
2019). Each pulsar was observed once, the Modified Julian Date (MJD) and
duration of each pulsar are shown in Table 1. Data for the pulsars were
recorded in search mode PSRFITS format with four polarizations, 8-bit samples
at an interval of 8.192 $\mu s$, and 1024 frequency channels with a channel
bandwidth of 0.488 MHz. Note that FAST does not possess the signal processing
capability to provide coherently dedispersed search mode data streams.

For each pulsar, individual pulses were extracted with ephemerides from the
Australia Telescope National Facility (ATNF) pulsar catalogue (Manchester et
al. 2005), using the dspsr software package (van Straten & Bailes, 2011). To
remove radio frequency interference (RFI), we used paz in the psrchive
software package (Hotan et al., 2004) to zap channels using median smoothed
difference, and a degraded bandpass of about 50 MHz was removed from both
edges of the data. Then, we used pazi to check RFI for each pulsar by eye.
Polarization calibration was achieved by correcting for the differential gain
and phase between the receptors through separate measurements using a noise
diode signal.

Rotation Measure (RM) is determined using rmfit, and the results are presented
in Table 1. Subsequently, the pulses for each pulsar are RM-corrected. To
generate ToAs, noise-free standard templates were created using the psrchive
program paas, and ToAs were subsequently calculated by cross-correlating the
pulse profile with the template using pat with the FDM algorithm. Timing
residuals were obtained using the tempo2 software package (Hobbs et al.,
2006). Fluctuation analysis was conducted using the PSRSALSA package
(Weltevrede, 2016).

##  3 Results

###  3.1 Jitter noise measurement

Table 1: Jitter measurements for 12 MSPs. Columns 2, 3, 4 and 5 present the period, DM, observed MJD, and integration time, respectively, columns 6 and 7 present the measured RM and maximum $S/N_{\rm peak}$ of a single-pulse, respectively, columns 8 and 9 present the implied jitter value which scales to single-pulse and one-hour-long duration, respectively, column 10 and 11 present the implied $\sigma_{\rm S/N}$ and $\sigma_{\rm DISS}$ in one-hour-long duration, respectively. The last column reports a reference to a previously published jitter measurement at L-band, in which P21, L19, L16 and S14 refer to Parthasarathy et al. (2021) at 1284 MHz, Lam et al. (2019) at 1500 MHz, Lam et al. (2016) at 1400 MHz and Shannon et al. (2014) at 1400 MHz, respectively.  NAME | Period | DM | MJD | Duration | RM | ${\rm S/N_{\rm peak,max}}$ |  $\sigma_{\rm J}$(1) |  $\sigma_{\rm J}$(h) |  $\sigma_{\rm S/N}$(h) |  $\sigma_{\rm DISS}$(h) |  $\sigma_{\rm J}$(h)
---|---|---|---|---|---|---|---|---|---|---|---
| ms | ${\rm cm^{-3}\,pc}$ |  | s | $\rm rad\,m^{-2}$ |  |  $\mu$s | ns | ns | ns | ns
J0030+0451 | 4.86 | 4.34 | 59454.80 | 570 |  2.44$\pm$0.21 | 69.93 |  40.1$\pm$0.8 |  46$\pm$1 |  43.4$\pm$0.1 | 0.27 | ¡60 (P21)
|  |  |  |  |  |  |  |  |  |  |  $61.1_{-3.5}^{+3.5}$ (L19)
|  |  |  |  |  |  |  |  |  |  | 153 (L16)
J0613$-$0200 | 3.06 | 38.78 | 59478.03 | 571 |  21.79$\pm$0.23 | 10.47 |  37.3$\pm$0.2 |  34.4$\pm$0.2 |  33.0$\pm$0.1 | 6.62 |  $133_{-8}^{+8}$ (L19)
|  |  |  |  |  |  |  |  |  |  | ¡44 (L16)
|  |  |  |  |  |  |  |  |  |  | ¡400 (S14)
J0636+5128 | 2.86 | 11.11 | 59132.92 | 558 |  -2.14$\pm$0.05 | 16.89 |  33.2$\pm$0.4 |  29.6$\pm$0.4 |  12.33$\pm$0.07 | 2.48 |  $62_{-22}^{+16}$ (L19)
J0751+1807 | 3.48 | 30.25 | 59478.04 | 421 |  42.53$\pm$0.19 | 12.71 |  33.1$\pm$0.5 |  32.5$\pm$0.5 |  29.26$\pm$0.08 | 6.00 | -
J1012+5307 | 5.26 | 9.02 | 59446.18 | 535 |  3.84$\pm$0.05 | 16.99 |  22.3$\pm$0.1 |  27.0$\pm$0.2 |  29.20$\pm$ 0.06 | 1.98 |  $67\pm 6$ (L19)
|  |  |  |  |  |  |  |  |  |  | ¡103 (L16)
J1643$-$1224 | 4.62 | 62.41 | 59479.37 | 421 |  -296.71$\pm$0.23 | 9.25 |  36.0$\pm$0.2 |  40.8$\pm$0.3 |  39.89$\pm$0.03 | 12.06 | ¡60 (P21)
|  |  |  |  |  |  |  |  |  |  |  $120_{-10}^{+8}$ (L19)
|  |  |  |  |  |  |  |  |  |  | 155 (L16)
|  |  |  |  |  |  |  |  |  |  | ¡500 (S14)
J1713+0747 | 4.57 | 15.92 | 59446.46 | 575 |  13.16$\pm$0.08 | 62.18 |  22.1$\pm$0.2 |  24.9$\pm$0.3 |  6.43$\pm$0.01 | 3.65 |  $29.3_{-0.9}^{+3.2}$ (L19)
|  |  |  |  |  |  |  |  |  |  | 28/36 (L16)
|  |  |  |  |  |  |  |  |  |  |  $35.0\pm 0.8$ (S14)
J1744$-$1134 | 4.08 | 3.14 | 59486.39 | 397 |  5.77$\pm$0.09 | 11.53 |  27.6 $\pm$ 0.2 |  29.4 $\pm$ 0.2 |  28.4$\pm$0.1 | 0.11 |  30$\pm$6 (P21)
|  |  |  |  |  |  |  |  |  |  |  46.5$\pm$1.3 (L19)
|  |  |  |  |  |  |  |  |  |  | 31 (L16)
|  |  |  |  |  |  |  |  |  |  |  37.8$\pm$ 0.8 (S14)
J1911+1347 | 4.63 | 30.99 | 59477.61 | 571 |  -7.85$\pm$0.31 | 18.58 |  24.5$\pm$0.2 |  27.8$\pm$0.2 |  27.7$\pm$0.1 | 4.47 |  $43.5_{-2.3}^{+2.3}$ (L19)
J1918$-$0642 | 7.65 | 26.46 | 59449.54 | 1169 |  -56.24$\pm$0.19 | 33.73 |  38.2$\pm$0.3 |  55.7$\pm$0.5 |  41.2$\pm$0.2 | 5.35 | ¡55 (P21)
|  |  |  |  |  |  |  |  |  |  |  $59_{-13}^{+9}$ (L19)
|  |  |  |  |  |  |  |  |  |  | ¡101 (L16)
J1944+0907 | 5.19 | 24.36 | 59449.51 | 1169 |  -34.61$\pm$0.17 | 29.37 |  145$\pm$1 |  174$\pm$1 |  79$\pm$1 | 3.54 |  $311_{-7}^{+6}$ (L19)
|  |  |  |  |  |  |  |  |  |  | 196 (L16)
J2145$-$0750 | 16.05 | 9.00 | 59463.65 | 1171 |  0.42$\pm$0.05 | 132.28 |  79.5$\pm$0.4 |  168$\pm$1 |  22.6$\pm$0.1 | 1.37 |  200 $\pm$ 20 (P21)
|  |  |  |  |  |  |  |  |  |  |  $173^{+3}_{-4}$ (L19)
|  |  |  |  |  |  |  |  |  |  | 85 (L16)
|  |  |  |  |  |  |  |  |  |  |  192 $\pm$ 6 (S14)
Figure 1: Summary of white noise components for 12 pulsars. The right panel
shows the three contributions, radiometer noise as black squares, jitter noise
as blue circles, and scintillation noise as red triangles. All the noise
contributions are scaled to one hour integration.

On short timescales, the excess noise in the timing residuals is typical
dominated by radiometer noise, jitter noise, and scintillation noise (Shannon
et al., 2014; Lam et al., 2016). The measurement error of ToA can be
summarized as (Shannon et al., 2014; Lam et al., 2016):

$\sigma^{2}_{\rm total}=\sigma^{2}_{\rm S/N}+\sigma^{2}_{\rm
J}+\sigma^{2}_{\rm DISS},$ (1)

where $\sigma_{\rm S/N}$, $\sigma_{\rm J}$ and $\sigma_{\rm DISS}$ are the
uncertainties induced by radiometer noise, jitter noise and scintillation
noise, respectively.

We assume that all of the excess error in the ToA measurements is attributed
to jitter noise. Jitter noise can then be obtained by calculating the
quadrature difference between the observed root mean square (rms) timing
residual and the radiometer noise (Shannon et al., 2014):

$\sigma_{\rm J}^{2}{(N_{\rm p})}=\sigma_{\rm obs}^{2}{(N_{\rm p})}-\sigma_{\rm
sim}^{2}{(N_{\rm p})},$ (2)

where ${N_{\rm p}}$ is the number of averaged pulses, $\sigma_{\rm obs}^{2}$
and $\sigma_{\rm sim}^{2}$ are the variances of the observed ToAs and
simulated ToAs, respectively. Using the method that was presented in
Parthasarathy et al. (2021), we simulated a set of idealized TOAs which match
the model, then Gaussian noise is added to the idealized TOAs with magnitudes
equal to the uncertainties of the observed TOAs. Using the TEMPO2 FAKE tool
(Hobbs et al., 2006), we created 1000 realizations of these simulated data
sets for each pulsar, and the mean $\sigma_{\rm sim}^{2}$ is taken as the
variance of the simulated ToAs. Following Parthasarathy et al. (2021), we
think that the jitter noise is detected if $\sigma_{\rm obs}^{2}{(N_{\rm
p})}>0.95\sigma_{\rm sim}^{2}{(N_{\rm p})}$.

For each pulsar, the profiles are frequency-averaged as well as polarization-
averaged, and then we folded the profile with ${N_{\rm p}}$ of 16, 32, 64,
128, 256, 512, 1024, 2048, and 4096 single pulses. Subsequent, the jitter
noises of these 12 pulsars were measured using frequency-averaged ToAs. Note
that we used only the total intensity (Stokes I) of pulses to generate the
frequency-averaged ToAs. Considering the jitter noises generally can be scaled
as $1/\sqrt{N_{\rm p}}$ (e.g. Shannon et al. 2014), the extrapolated jitter
noise for a one-hour-long duration is calculated as: $\sigma_{\rm J}({\rm
h})=\sigma_{\rm J}(1)/\sqrt{3600/P}$, where $\sigma_{\rm J}({1})$ is the
implied jitter noise for a single-pulse, and $P$ is the pulsar spin period in
seconds. The extrapolated jitter noise levels for single-pulse and one-hour-
long duration are presented in Table 1.

In our sample, we have detected jitter noises in all 12 pulsars, with
different levels of jitter noise for each pulsar. The fraction of jitter noise
($F_{\rm J}=\sigma_{\rm J}$/$\sigma_{\rm total}$) ranges from 68% to 99%.
Jitter noise is particularly dominant in bright pulsars, such as PSR
J2145$-$0750 with an $F_{\rm J}$ of 99%. A jitter parameter is defined as
$k_{\rm J}=\sigma_{\rm J}(1)/P$ (Lam et al., 2019), where $P$ is the pulsar
spin period. We analyzed the relationship between $k_{\rm J}$ and the duty
cycle for each pulsar, where the duty cycle is defined as the full width at
half maximum of the pulse profile (W50) divided by the pulse period ($P$). We
found a Spearman correlation coefficient of $R=0.82$ between jitter noise and
pulse W50 width for the pulsars in our sample. However, we note that PSR
J1944+0907 exhibits both high jitter noise and a wide pulse profile, which may
introduce biases in the estimation of the correlation coefficient. When PSR
J1944+0907 is excluded, the correlation coefficient decreases to 0.56.
Therefore, we suggested that there is no significant relationship between the
level of pulse jitter and the pulse width. Our result is consistent with
previous studies conducted by NANOGrav (Lam et al., 2019) and MPTA
(Parthasarathy et al., 2021), which reported correlation coefficients of 0.62
and 0.64, respectively.

In Figure 2, we showed our results and the previously reported values of
jitter noise at different frequencies for each pulsar. Jitter noise for many
pulsars exhibits frequency dependence, with different levels at different
observing frequencies (Lam et al., 2019; Parthasarathy et al., 2021). We
compared our measurements of jitter noise with previously reported values.
Details of the comparison for each pulsar are provided in Section 3.3.

Scintillation noise is associated with the propagation of pulsar signals
through the interstellar medium. The scintillation noise which is induced by
stochastic broadening can be described as (Cordes & Shannon, 2010)

$\sigma_{\rm DISS}^{2}=\tau^{2}/N_{\rm scint},$ (3)

where $\tau$ and $N_{\rm scint}$ are the pulse-broadening time-scale and the
number of scintles, respectively. The number of scintles $N_{\rm
scint}=(1+\eta\Delta\nu/\nu_{\rm d})(1+\eta\Delta T/t_{\rm d})$, where
$\Delta\nu$ and $\Delta T$ are the observing bandwidth and duration, $\nu_{\rm
d}$ and $t_{\rm d}$ are the diffractive scintillation bandwidth and time,
respectively. The scintillation filling factor is approximately $\eta\approx
0.3$ (Cordes & Shannon, 2010). Using the NE2001 model (Cordes & Lazio, 2002),
we estimated $\nu_{\rm d}$, $t_{\rm d}$, and $\tau$ for each pulsar. Taking
$\Delta\nu=500$ MHz and $\Delta T=3600\,$s, the $\sigma_{\rm DISS}$ in one-
hour-long duration for each pulsar is estimated.

In Figure 1, we present a summary of radiometer noise (black squares), jitter
noise (blue circles), and scintillation noise (red triangles) for the 12
pulsars. It is evident that $\sigma_{\rm scint}$ is typically much smaller
than $\sigma_{\rm J}$ or $\sigma_{\rm S/N}$ in our sample. The mean levels of
jitter noise, radiometer noise, and scintillation noise in one-hour-long
duration are 58 ns, 33 ns, and 4 ns, respectively.

Figure 2: The jitter noises at different frequencies for 12 pulsars. The black
circles denote the measured jitter noise at 430 MHz, 820 MHz, 1400 MHz and
2300 MHz of Lam et al. (2016), the blue squares denote the measured jitter
noise at 327/430 MHz, 820 MHz, 1500 MHz and 2300 MHz of Lam et al. (2019), the
red triangles denote the measured jitter noise at 730 MHz, 1400 MHz and 3100
MHz of Shannon et al. (2014), the green diamond denote the measured jitter
noise at 1284 MHz of Parthasarathy et al. (2021), and the magenta stars denote
the measured jitter noise at 1250 MHz of our work. Note the levels of jitter
noise are scaled to one-hour-long duration.

###  3.2 Reduce jitter noise using polarization

Our measurements of white noise components in pulsars indicate the substantial
impact of jitter noise on PTA with FAST. For the generation of ToAs, narrow
features in the pulse profile can provide strong constraints during template
matching (van Straten, 2006). Considering that the polarized components of
pulsar profile may exhibit sharp features (e.g. Dai et al. 2015; Wahl et al.
2022), we tried to use polarization information to mitigate jitter noise. We
performed matrix template matching using all four Stokes parameters (van
Straten 2006; Osłowski et al. 2013) to generate frequency-averaged ToAs for
each pulsar. Then, using the same jitter measurement method as shown in
Section 3.1, we measured the radiometer noise ($\sigma_{\rm S/N,P}$) and
jitter noise ($\sigma_{\rm J,P}$) for each pulsar. However, we only detected
jitter noises in 8 pulsars. The extrapolated noise levels for a one-hour-long
duration for these 8 pulsars are shown in Table 2.

In Figure 3, we compared the jitter and radiometer noises which were measured
using all four Stokes parameters with that measured using only the total
intensity. The level of radiometer noise increases, which is expected because
the Stokes Q, U, and V are generally much weaker than the Stokes I.
Simultaneously, the level of jitter noise decreases. For these 8 pulsars, the
radiometer noise level increases by percentages ranging from 6.1% to 38.6%,
while the jitter noise level decreases by percentages ranging from 6.7% to
39.6%. Therefore, matrix template matching using all four Stokes parameters
proves to be a valuable method for reducing jitter noise in pulsars. In the
case of 7 pulsars, the total white noise level decreases by percentages
ranging from 0.19% to 13.4%. However, for J1918$-$0642, it increases by 0.08%.
The consideration of all four Stokes parameters could lead to improved timing
precision on short timescales for most pulsars (7 out of 8 pulsars in our
sample).

Table 2: The measured jitter, radiometer and total white noises in one-hour-long duration for 8 pulsars, the subscripts P and I are for the results using all four Stokes parameters and only I, respectively. NAME |  $\sigma_{\rm J,P}$(h) |  $\sigma_{\rm S/N,P}$(h) |  $\sigma_{\rm total,P}$(h) |  $\sigma_{\rm total,I}$(h)
---|---|---|---|---
| ns | ns | ns | ns
J0613$-$0200 |  20.9$\pm$0.5 |  41.63$\pm$0.05 | 46.8 | 47.7
J0636+5128 |  21.9$\pm$0.4 |  17.09$\pm$0.06 | 27.8 | 32.1
J0751+1807 |  23.1$\pm$0.5 |  33.09$\pm$0.09 | 40.4 | 43.7
J1012+5307 |  16.3$\pm$0.2 |  30.98$\pm$0.03 | 35.0 | 39.8
J1713+0747 |  22.4$\pm$0.3 |  8.37$\pm$0.02 | 23.9 | 25.7
J1918$-$0642 |  42.8$\pm$0.9 |  55.2$\pm$0.2 | 69.8 | 69.3
J1944+0907 |  155$\pm$1 |  105.4$\pm$0.2 | 187.4 | 191.1
J2145$-$0750 |  156.7$\pm$0.7 |  26.92$\pm$0.08 | 159.0 | 169.5
Figure 3: Comparison of white noise components for 8 pulsars by using all four
Stokes parameters (open symbols) and only I (filled symbols), respectively.
The circle, square and star are for the jitter, radiometer and total white
noises, respectively.

###  3.3 Single-pulse phenomenology

Figure 4: Single-pulse stacks for each pulsar. The $S/N_{\rm peak}$ of single-
pulse versus pulse number is shown in the left panel and the average profile
formed from all of single-pulses is shown in the lower panel, in which
different pulse components are labled.

Figure 5: The LRFS for 10 pulsars. The slide panels show the horizontally integrated power of LRFS. The averaged profile is overlayed over LRFS.  Table 3: Pulse intensity modulation periods in 10 pulsars. PSR | Component | Modulation period (P) | Type
---|---|---|---
J0030+0451 | MP |  5.5$\pm$0.1 | coherent
J0613$-$0200 | C1 |  6$\pm$1. | diffuse
| C2 |  72.9$\pm$ 0.4 | diffuse
J0636+5128 | P |  84.3$\pm$ 0.9 | coherent
| LP |  2.20$\pm$ 0.01 | coherent
| TP |  2.20 $\pm$ 0.01 | coherent
J0751+1807 | MP |  128.0 $\pm$ 0.1/9.4 $\pm$ 0.1 | diffuse
| MP |  6.3 $\pm$ 0.1 | diffuse
J1012+5307 | C1 |  7.1$\pm$ 0.1 | diffuse
| C2 |  7.1$\pm$ 0.1 | diffuse
| C3/C4 |  11.1$\pm$ 0.1 | diffuse
J1643$-$1224 | P |  46$\pm$1 | diffuse
J1713+0747 | C1 |  6.2$\pm$0.1 | diffuse
J1918$-$0642 | C1 |  3.3 $\pm$ 0.1 | diffuse
J1944+0907 | C1/C2/C3 |  128 $\pm$ 2 | coherent
J2145$-$0750 | C1 |  3.9$\pm$ 0.1/ 2.0$\pm$ 0.1 | diffuse

We present the single-pulse stacks for these 12 pulsars in Figure 4. It is
worth noting that there are some bright pulses with narrow pulse widths.
Therefore, we define the $S/N_{\rm peak}$ of a single-pulse as $I_{\rm
peak}/\sigma_{\rm off}$, where $I_{\rm peak}$ represents the peak intensity of
a single-pulse in the on-pulse region and $\sigma_{\rm off}$ is the root-mean-
square (rms) of the off-pulse region. The $S/N_{\rm peak}$ of single-pulses
versus pulse number for each pulsar is shown in the right panel of Figure 4,
and the average profile of the entire observation is presented in the lower
panel Figure 4. We found that the single-pulses in MSPs are unstable and
exhibit significant variations in $S/N_{\rm peak}$.

To investigate pulse intensity modulation behavior, we conducted a longitude-
resolved fluctuation spectrum (LRFS, Backer 1970; Weltevrede 2016) analysis.
Using the PSRSALSA package (Weltevrede, 2016), we obtained the LRFS for each
pulsar, and we observed that 10 pulsars exhibit periodic intensity
modulations, as shown in Figure 5. In the side panel of Figure 5, the spectra
of horizontally integrated LRFS are presented, where the vertical axis is
measured in cycles per period (${P/P_{3}}$) with the vertical band separation
${P_{3}}$. The peak of the side panel corresponds to the modulation period of
pulse intensity. Following the method of Weltevrede (2016), if the vertical
extension of ${P_{3}}$ is smaller than 0.05 cycles per period, it is defined
as coherent modulation; otherwise, it belongs to diffuse modulation.

The modulation periods of the pulse intensities for pulsars are shown in Table
3. There are 10 pulsars exhibit periodic intensity modulations, namely PSR
J0030+0451, PSR J0613$-$0200, PSR J0636+5128, PSR J0751+1807, PSR J1012+5307,
PSR J1643$-$1224, PSR J1713+0747, PSR J1918$-$0642, PSR J1944+0907, and PSR
J2145$-$0750, with 6 pulsars being reported for the first time. The periodic
intensity modulations of PSR J1918$-$0642, PSR J1012+5307, PSR J2145$-$0750,
and PSR J1713+0747 were previously reported by Edwards & Stappers (2003) and
Liu et al. (2016), respectively, which are consistent with our results. The
comparisons for these four pulsars to the previously published results are
shown in the follow subsections. PSR J0030+0451, PSR J0636+5128, and PSR
J1944+0907 exhibit coherent modulations, while the remaining 7 pulsars show
diffuse modulations. The pulse intensity modulation periods of these pulsars
range from $\sim 6-665$ ms ($\sim 2-128\,P$ ).

####  3.3.1 PSR J0030+0451

Figure 6: Energy distribution for the on-pulse region of PSR J0030+0451. The
energies are normalized by the mean on-pulse energy. The blue solid line
represents the fitting results of the combination of a Gaussian (blue dashed
line, represents the weak mode) and a log-normal (blue dot-dashed line,
represents the bright mode) components. The vertical red line represents the
2$\langle E\rangle$ where the Gaussian and log-normal functions intersect.
Figure 7: Averaged profile of the bright (black line) and weak (red line)
modes for PSR J0030+0451.  Figure 8: Profile of the brightest single-pulse
with the pulse energy of 11$\langle E\rangle$ for PSR J0030+0451.

PSR J0030+0451 exhibits a main pulse (MP) and a relatively weak interpulse
(IP) (Figure 4). The energy distribution of the MP is shown in Figure 6, where
pulse energy is calculated across the on-pulse region labeled as MP in Figure
4. This distribution exhibits two peaks, which can be fitted by combination of
a Gaussian and a log-normal functions, corresponding to the weak and bright
modes, respectively. The Gaussian and log-normal functions intersect at
approximately 2$\langle E\rangle$ (marked by the vertical red line in Figure
6). To identify the two modes, we employ a criterion where a single pulse with
energy greater than 2$\langle E\rangle$ is classified as belonging to the
bright mode, otherwise, it is assigned to the weak mode.

In Figure 7, the averaged profiles of the weak and bright modes are presented.
In the bright mode, the profile exhibits a significantly stronger main pulse
compared to the weak mode, while the interpulses have the same shape and flux
density in both modes. LRFS analysis reveals that the MP shows periodic
intensity modulations with a period of 5.5$\pm 0.1\,P$, whereas no such
periodic modulations are observed in the interpulse (Figure 5). The periodical
modulation spans the entire profile of the MP. We concluded that PSR
J0030+0451 exhibits a periodic mode-changing phenomenon, which has never
reported in MSPs before.

We also note that the maximum energy of MP exceeds 10$\langle E\rangle$, which
suggests the existence of giant pulses. The profile of the brightest pulse is
shown in Figure 8 with a W50 width of 58 $\mu$s, much narrower than the
average pulse of 577 $\mu$s. The energy and peak intensity of this pulse are
about 11 and 70 times greater than the average pulse, respectively. PSR
J0030+0451 might be a potential giant pulse emitter. However, due to the
limitations of our observation with an 8.192 $\mu$s interval, it is
insufficient to identify typical giant pulses with nanosecond durations.
Further observations with a higher sampling interval are required.

The measured $\sigma_{\rm J}({\rm h})$ at different frequencies for previously
reported results are shown in Figure 2, where there is a tendency that the
jitter noise increases with observing frequency. Our measured $\sigma_{\rm
J}({\rm h})$ is $43.4\pm 0.1\,$ns at 1250 MHz, which is smaller than the
results of 60 ns at 1284 MHz (Parthasarathy et al., 2021), 153 ns at 1400 MHz
(Lam et al., 2016), and $61.1^{+3.5}_{-3.5}$ ns at 1500 MHz (Lam et al.,
2019). Therefore, it is reasonable that our measured jitter noise at 1250 MHz
is smaller than the reported values at higher observing frequencies.

####  3.3.2 PSR J0613$-$0200

PSR J0613$-$0200 exhibits three components, namely C1,C2 and C3 (Figure 4).
Among these components, C2 exhibits greater flux density compared to C1 and
C3. We performed LRFS analysis for this pulsar, and the results are presented
in Figure 5. We detected periodic modulations across the C1 and C2 components,
with modulation periods of $6\pm 1\,P$ and $72.9\pm 0.4\,P$, respectively.

The measured $\sigma_{\rm J}({\rm h})$ at different frequencies for previously
reported results are shown in Figure 2. Similar to the case of PSR J0030+0451,
there is also a tendency for the jitter noise to increase with observing
frequency for PSR J0613$-$0200\. Our measured $\sigma_{\rm J}({\rm h})$ is
$34.4\pm 0.2$ ns at 1250 MHz, which roughly agrees with the results of $<400$
ns at 1400 MHz (Shannon et al., 2014), of $<44$ ns at 1400 MHz (Lam et al.,
2016), but is much smaller than the result of $133^{+8}_{-8}$ ns at 1500 MHz
(Lam et al., 2019).

####  3.3.3 PSR J0636+5128

PSR J0636+5128 exhibits a single component in its pulse profile (Figure 4).
The pulsar exhibits complicated periodical modulations (Figure 5). A
modulation period of $84.3\pm 0.9\,P$ occurs across the entire profile, while
modulation periods of $2.20\pm 0.01\,P$ and $2.26\pm 0.01\,P$ occur only in
the leading and trailing parts of the profile, respectively.

The measured $\sigma_{\rm J}({\rm h})$ at different frequencies for previously
reported results are shown in Figure 2. Our measured $\sigma_{\rm J}({\rm h})$
is $29.6\pm 0.4\,$ns at 1250 MHz, which is smaller than the results of
$62^{+16}_{-22}$ ns at 1500 MHz and $47^{+13}_{-11}$ ns at 820 MHz (Lam et
al., 2019), but with much smaller uncertainties.

####  3.3.4 PSR J0751+1807

PSR J0751+1807 exhibits two components, C1 and C2, with C2 being stronger
(Figure 4). The bright single-pulses occur across both the C1 and C2
components (Figure 4). We performed LRFS analysis and found periodical
modulations in both components (Figure 5). The modulation period of C1 is
$128.0\pm 0.1\,P$, while the modulation period of the C2 is $6.3\pm 0.1\,P$.
The correlation coefficient between the energy of the two components is 0.35,
indicating no significant correlation between them. We report new jitter
measurement of this pulsar with $\sigma_{\rm J}({\rm h})=32.5\pm 0.5$ ns.

####  3.3.5 PSR J1012+5307

PSR J1012+5307 exhibits a main pulse and an interpulse in its pulse profile
(Figure 4). The pulsar presents complex modulations in both the main pulse and
interpulse (Edwards & Stappers, 2003). Our highly sensitive observation allows
for a more detailed study of the modulations (Figure 4). By calculating the
LRFS, we found that the two components of the mean pulse (C1 and C2) exhibit
different modulation periods, with modulation periods of 7.1$\pm$0.1$P$ and
7.6$\pm$0.1$P$, respectively. For the interpulse, periodical modulation is
observed only in the first and second components (C3 and C4), with a similar
modulation period of $11.1\pm 0.1\,P$.

The measured $\sigma_{\rm J}({\rm h})$ at different frequencies for previously
reported results are shown in Figure 2, showing a tendency for the jitter
noise to increase with observing frequency. Our measured $\sigma_{\rm J}({\rm
h})$ is $27.0\pm 0.2\,$ns at 1250 MHz, which roughly agrees with the result of
$<103$ ns at 1400 MHz (Lam et al., 2016), but is smaller than the result of
$67\pm 6$ ns at 1500 MHz (Lam et al., 2019).

####  3.3.6 PSR J1643$-$1224

PSR J1643$-$1224 exhibits a single component in its pulse profile (Figure 4).
The pulsar exhibits periodical modulation with a period of $46\pm 1\,P$
occurring across the leading part of the profile, as determined by LRFS
analysis (Figure 5).

The measured $\sigma_{\rm J}({\rm h})$ at different frequencies from
previously reported results are presented in Figure 2, showing a tendency for
the jitter noise to increase with observing frequency. Our measured
$\sigma_{\rm J}({\rm h})$ is $40.8\pm 0.3$ ns at 1250 MHz, which roughly
agrees with the result of $<60$ ns at 1284 MHz (Parthasarathy et al., 2021)
and $<500$ ns at 1400 MHz (Shannon et al., 2014), but is smaller than the
results of $155$ ns at 1400 MHz (Lam et al., 2016) and $120^{+8}_{-10}$ ns at
1500 MHz (Lam et al., 2019).

####  3.3.7 PSR J1713+0747

PSR J1713+0747 is a bright MSP. The profile of the pulsar changed between
April 15, 2021, and April 17, 2021 (Xu et al., 2021), and shows a slow
recovery continuing for the last several years (Jennings et al., 2022). Our
observation was on August 20, 2021, during which the profile had not
completely recovered yet. PSR J1713+0747 exhibits multiple components, but it
is dominated by a central component (C1) in its pulse profile (Figure 4). We
identified a periodical modulation in our observation with a period of $6.2\pm
0.1\,P$ (Figure 5). Considering the diffuse modulation, we think that our
result is roughly consistent with the previous result of $6.9\pm 0.1\,P$
reported by Liu et al. (2016).

The measured $\sigma_{\rm J}({\rm h})$ at different frequencies for previously
reported results are shown in Figure 2, where there is a tendency that the
jitter noise decreases with observing frequency. Our measured $\sigma_{\rm
J}({\rm h})$ is $24.9\pm 0.3$ ns at 1250 MHz, which is smaller than the
results of $35.0\pm 0.8$ ns at 1400 MHz (Shannon et al., 2014), $28/36$ ns at
1400 MHz (Lam et al., 2016), and $29.3^{+3.2}_{-0.9}$ ns at 1500 MHz (Lam et
al., 2019). Note that the profile in our observation has not recovered yet,
which may cause the change in the level of jitter noise. For pulsars
exhibiting mode changing phenomena, the levels of jitter noise in different
modes can vary, as seen in PSR J1909$-$3744 (Miles et al., 2022).

####  3.3.8 PSR J1744$-$1134

PSR J1744$-$1134 exhibits a main pulse and a weak interpulse in its pulse
profile (Figure 4). The measured $\sigma_{\rm J}({\rm h})$ at different
frequencies for previously reported results are shown in Figure 2, indicating
a pattern where the jitter noise decreases and then increases with observing
frequency. Our measured $\sigma_{\rm J}({\rm h})$ is $29.4\pm 0.2$ ns at 1250
MHz, which agrees with the results of $31$ ns at 1400 MHz (Lam et al., 2016)
and $30\pm 6$ ns at 1284 MHz (Parthasarathy et al., 2021), but is smaller than
the results of $37.8\pm 0.8$ ns at 1400 MHz (Shannon et al., 2014) and
$46.5\pm 1.3$ ns at 1500 MHz (Lam et al., 2019).

####  3.3.9 PSR J1911+1347

PSR J1911+1347 exhibits a relatively bright pulse component (C1) surrounded by
broader emission in its pulse profile (Figure 4). The narrow bright pulses
primarily occur in the C1 component. The measured $\sigma_{\rm J}({\rm h})$ at
different frequencies for previously reported results are shown in Figure 2,
showing an increase in jitter noise with observing frequencies. Our measured
$\sigma_{\rm J}({\rm h})$ is $27.8\pm 0.2$ ns at 1250 MHz, which is smaller
than the result of $43.5^{+2.3}_{-2.3}$ ns at 1500 MHz (Lam et al., 2019).

####  3.3.10 PSR J1918$-$0642

PSR J1918$-$0642 exhibits a main pulse and an interpulse in its pulse profile.
The main pulse exhibits a relatively bright pulse component (C1) with broader
emission flanking it, and the narrow bright pulse only occurs in the C1
(Figure 4). Previous work by Edwards & Stappers (2003) suggested that PSR
J1918$-$0642 may exhibit quasi-periodic modulations with periods ranging from
2 to 4 $P$, but the limited sensitivity of their study made their results
uncertain. In our analysis, we detected periodical modulation in this pulsar
with a period of $3.3\pm 0.1\,P$ through the calculation of the LRFS (Figure
5). However, the modulation only occurs across the C1 of the main pulse.

The measured $\sigma_{\rm J}({\rm h})$ at different frequencies for previously
reported results are shown in Figure 2, with the jitter noise increasing as
the observing frequency increases. Our measured $\sigma_{\rm J}({\rm h})$ is
$55.7\pm 0.5$ ns at 1250 MHz, which agrees with the results of $<101$ ns at
1400 MHz (Lam et al., 2016), $56^{+9}_{-13}$ ns at 1500 MHz (Lam et al.,
2019), and $<$55 ns at 1284 MHz (Parthasarathy et al., 2021).

####  3.3.11 PSR J1944+0907

PSR J1944+0907 exhibits a wide profile with about four components (C1, C2, C3,
and C4) in its pulse profile (Figure 4). The bright narrow single-pulse is
primarily observed in the C1. We identified periodical modulation that occurs
across the C1, C2, and C3 components with the same modulation period of
$128\pm 2\,P$ by calculating the LRFS (Figure 5).

The measured $\sigma_{\rm J}({\rm h})$ at different frequencies for previously
reported results are shown in Figure 2, with the jitter noise increasing as
the observing frequency increases. Our measured $\sigma_{\rm J}({\rm h})$ is
$174\pm 1$ ns at 1250 MHz, which is smaller than the results of $196$ ns at
1400 MHz (Lam et al., 2016) and $311^{+6}_{-7}$ ns at 1500 MHz (Lam et al.,
2019).

####  3.3.12 PSR J2145$-$0750

PSR J2145$-$0750 is a bright millisecond pulsar with multiple components in
its pulse profile (Figure 4). We observed periodical modulation occurring
across the C1 component with a modulation period of $3.9\pm 0.1\,P$ (Figure
5), which is consistent with the previous result of $\sim 2-4\,P$ (Edwards &
Stappers, 2003).

The measured $\sigma_{\rm J}({\rm h})$ at different frequencies for previously
reported results are shown in Figure 2, where no obvious tendency of
increase/decrease with observing frequency is detected. Our measured
$\sigma_{\rm J}({\rm h})$ is $168\pm 1$ ns at 1250 MHz. This value agrees with
the result of $173^{+3}_{-4}$ ns at 1500 MHz (Lam et al., 2019), but is larger
than the result of $85$ ns at 1400 MHz (Lam et al., 2016), and smaller than
the result of $192\pm 6$ ns at 1400 MHz (Shannon et al., 2014) and $200\pm 20$
ns at 1284 MHz (Parthasarathy et al., 2021).

###  3.4 Timing a subselection of single-pulses

Figure 9: Wrms of the bright (red circles) and weak (blue squares) pulses at
different $S/N_{\rm crit}$ for each pulsar. The black line is for the Wrms in
60 s subintegrations using all the single-pulses. Note that the y-axes is
plotted on the same scale.

The variability of single-pulses introduces jitter noise, which can have a
significant impact on highly sensitive observations. It may be possible to
improve the timing precision by selecting a sub-set of single pulses. Miles et
al. (2022) conducted timing analysis on PSR J1909$-$3744 which shows mode
changing phenomenon, and found that timing precision improved by approximately
10% by only timing the strong mode. Osłowski et al. (2014) studied the rms of
the timing residual of PSR J0437$-$4715 for average profiles formed by
integrating for one minute but rejecting the brightest single pulses with
different S/N. They found that the rms of the timing residual decreases with
the increasing S/N, but no significantly improvement is achievable. Following
the method of Osłowski et al. (2014), we divided the observation into several
segments, each lasting 60 s. Within each segment, we separately summed all the
pulses, as well as the bright and weak pulses. We then formed three pulse
templates based on the average profiles of the bright, weak, and all of the
pulses from the entire observation using the PAAS. Pulse ToAs were obtained by
cross-correlating the summed profiles of the segments with the corresponding
standard template, and timing residuals were calculated using the TEMPO2
software (a similar method was demonstrated in Wang et al. 2020). We
considered timing precision improvement achievable if the weighted rms (Wrms)
of the timing residuals obtained using a sub-selection of pulses was lower
than that obtained using all of the pulses.

We note that there are some bright pulses with narrow pulse widths pulsars
(Figure 4). Therefore, we used $S/N_{\rm peak}$ to define bright pulses. We
defined a threshold value ($S/N_{\rm crit}$) to classify pulses as either
bright or weak. If the $S/N_{\rm peak}$ of a single-pulse is greater than
$S/N_{\rm crit}$, it is classified as a bright pulse; otherwise, it is
classified as a weak pulse. For each pulsar, we divided the single-pulses into
five groups based on $S/N_{\rm crit}$ (Table 4). Here, $S/N_{\rm mean}$
represents the mean $S/N_{\rm peak}$, while $S/N_{\rm max}$ and $S/N_{\rm
min}$ are the maximum and minimum $S/N_{\rm peak}$ values, respectively. We
then calculated the Wrms of the timing residuals for the bright and weak
pulses at these five $S/N_{\rm crit}$ values for each pulsar (Figure 9). As
the number of bright pulses decreases with increasing $S/N_{\rm crit}$, it is
expected that the Wrms of the bright pulses increases (the red lines in Figure
9), while that of the weak pulses decreases (the blue lines in Figure 9).
However, when comparing the timing precision achieved using a sub-set of
pulses with a specific range of $S/N_{\rm peak}$ to the timing precision
achieved using all of the single-pulses (the black lines in Figure 9), we did
not observe a improvement in timing precision for the pulsars in our sample,
just like the case of PSR J0437$-$4715 (Osłowski et al., 2014).

Table 4: The threshold value ($S/N_{\rm crit}$) that was used to classify the bright/weak pulses. Groups | $S/N_{\rm crit}$
---|---
1 | $S/N_{\rm mean}-2/3(S/N_{\rm mean}-S/N_{\rm min})$
2 | $S/N_{\rm mean}-1/3(S/N_{\rm mean}-S/N_{\rm min})$
3 | $S/N_{\rm mean}$
4 |  $S/N_{\rm mean}+1/3(S/N_{\rm max}-S/N_{\rm main})$,
5 | $S/N_{\rm mean}+2/3(S/N_{\rm max}-S/N_{\rm main})$

##  4 Discussion And Conclusions

We presented measurements of jitter noise, radiometer noise, and scintillation
noise for 12 millisecond pulsars that are part of the IPTA sample, using FAST.
In our analysis, we determined that the mean levels of jitter noise,
radiometer noise, and scintillation noise in a one-hour-long duration are
approximately 58 ns, 33 ns, and 4 ns, respectively. This indicates that the
contribution of scintillation noise is likely negligible, being roughly one-
tenth the level of the jitter/radiometer noise, consistent with previous
published results (e.g., Lam et al. (2016)). Our results suggest that jitter
noise significantly contributes to white noise, emphasizing its notable impact
on PTA observations with FAST.

Jitter noise is known to exhibit frequency dependence (Lam et al., 2019). When
considering previously published results on jitter noise, we observe a diverse
trend: while the jitter noise for some pulsars increases with observing
frequency, it decreases for others. Lam et al. (2019) conducted a
comprehensive analysis of the frequency dependence of jitter noises in a large
sample of MSPs, revealing a wide range of indexes in a power-law model,
spanning from $-3.2$ to 9. MSPs typically demonstrate simple power-law spectra
with a mean index of approximately $-1.67$ (Dai et al., 2015), resulting in
radiometer noise increasing with observing frequency. For pulsars, the
interplay between jitter noise and radiometer noise at varying observing
frequencies becomes crucial. If the jitter noise increases with frequency,
exhibiting a spectrum steeper than the radiometer noise, it dominates at
higher frequencies; otherwise, radiometer noise prevails. Optimizing
observations can be achieved through the use of multiple telescopes with
distinct observing frequencies. For instance, higher frequency observations
for specific pulsars might significantly imporve arrival-time precision (Lam
et al., 2018). The Qitai Radio Telescope (QTT), currently under construction,
will enable high-sensitivity observations spanning from 150 MHz up to 115 GHz
(Wang et al., 2023). The synergy between FAST and QTT will promise a
substantial contribution to the capabilities of IPTA.

Our results indicate the substantial impact of jitter noise on PTA with FAST.
The investigation of single pulses represents a potential avenue for improving
timing precision in MSPs. For instance, Miles et al. (2022) conducted timing
analysis on PSR J1909$-$3744, which exhibits mode-changing phenomena. They
observed an approximately 10% improvement in timing precision by exclusively
considering the strong mode. In our sample, where some pulsars exhibit bright
and narrow single pulses or periodic intensity modulations, we explored the
potential for improving timing precision by selecting specific subsets of
single pulses. However, our analysis revealed that selecting a subset based on
a specific range of $S/N_{\rm peak}$ did not improve achievable timing
precision.

To improve pulsar timing precision, considering information about polarized
radiation has been proposed (van Straten, 2006; Osłowski et al., 2013).
Accordingly, we conducted matrix template matching using all four Stokes
parameters (van Straten 2006) for each pulsar to generate pulse ToAs, and
measured radiometer and jitter noise. We successfully detected jitter noise in
8 pulsars. Notably, we observed a decrease in jitter noise ranging from 6.7%
to 39.6%, while radiometer noise increased by 6.1% to 38.6%. The total white
noise also decreased for 7 out of the 8 pulsars in our sample, ranging from
0.19% to 13.4%. Therefore, matrix template matching using all four Stokes
parameters is a valuable method for reducing jitter noise in pulsars. In
future work, we plan to generate pulse ToAs by considering all four Stokes
parameters on a large dataset of MSPs using FAST, and present the noise
analysis of them with the expectation of further improving pulsar timing
precision.

The highly sensitive and rapidly sampled observations conducted with FAST have
provided us with the opportunity to study single pulses in a larger sample of
MSPs. Utilizing LRFS analysis, we identified periodical intensity modulations
in 10 pulsars: PSR J0030+0451, PSR J0613$-$0200, PSR J0636+5128, PSR
J0751+1807, PSR J1012+5307, PSR J1643$-$1224, PSR J1713+0747, PSR
J1918$-$0642, PSR J1944+0907, and PSR J2145$-$0750\. Notably, 6 of these
pulsars were reported for the first time. Our findings for PSR J1918$-$0642,
PSR J1012+5307, PSR J2145$-$0750, and PSR J1713+0747 are consistent with
previous results from Edwards & Stappers (2003) and Liu et al. (2016). PSR
J0030+0451, PSR J0636+5128, and PSR J1944+0907 exhibit coherent periodical
modulations, whereas the remaining 7 pulsars show diffuse periodical
modulations. The modulation periods for these MSPs range from approximately 6
to 665 ms (2 to 128 P), which is notably shorter than those observed in normal
pulsars, typically ranging from a few seconds to several minutes (Basu et al.,
2020). The origin of periodic modulations in normal pulsars and MSPs remains
unknown. In normal pulsars, these modulations are attributed to global changes
in currents within the pulsar magnetosphere (Latham et al., 2012) and are
generally observed across the entire pulse profile (Basu et al., 2020). Some
normal pulsars exhibit phase-locked modulations of the main pulse and
interpulse (Weltevrede et al., 2007). However, in MSPs, periodic modulations
may occur only across a portion of the profile. We did not observe phase-
locked phenomena in MSPs with interpulse emission. For example, in PSR
J1012+5307, the modulation periods of the mean pulse and interpulse are
different. In PSR J0030+0451, periodical modulation is only detected in the
mean pulse. The distinct modulation characteristics between MSPs and normal
pulsars may be attributed to the presence of multiple emission regions in the
magnetospheres of MSPs, with different pulse components originating from
distinct locations (e.g., Dyks et al. 2010).

In particular, we identified brighter, narrower single pulses in PSR
J0030+0451, with energies exceeding 10 times the mean pulse energy. This
strongly suggests the presence of giant pulses. Additionally, we observed a
periodic mode changing phenomenon in PSR J0030+0451, a behavior not previously
reported in MSPs. Until now, mode changing phenomena have only been detected
in two MSPs, namely PSR B1957+20 (Mahajan et al., 2018) and PSR J1909$-$3744
(Miles et al., 2022). However, the mode cycles observed in PSR B1957+20 and
PSR J1909$-$3744 are not periodic. Further observations with FAST,
encompassing a larger sample of MSPs, may unveil more instances of diverse
emission behaviors, such as mode changing, nulling, or giant pulses. This
expanded dataset could provide valuable constraints on the emission properties
of MSPs.

## Acknowledgments

This is work is supported by the National Natural Science Foundation of China
(No. 12288102, No. 12203092, No. 12041304), the Major Science and Technology
Program of Xinjiang Uygur Autonomous Region (No. 2022A03013-3), the National
SKA Program of China (No. 2020SKA0120100), the National Key Research and
Development Program of China (No. 2022YFC2205202, No. 2021YFC2203502), the
Natural Science Foundation of Xinjiang Uygur Autonomous Region (No.
2022D01B71, No. 2022D01D85), the Tianshan talent Program and the Special
Research Assistant Program of CAS, the CAS Project for Young Scientists in
Basic Research (No. YSBR-063), the Key Research Project of Zhejiang Laboratory
(No. 2021PE0AC03). the Zhejiang Provincial Natural Science Foundation of China
under Grant (No. LY23A030001). This work made use of the data from the Five-
hundred-meter Aperture Spherical radio Telescope, which is a Chinese national
megascience facility, operated by National Astronomical Observatories, Chinese
Academy of Sciences. The research is partly supported by the Operation,
Maintenance and Upgrading Fund for Astronomical Telescopes and Facility
Instruments, budgeted from the Ministry of Finance of China (MOF) and
administrated by the Chinese Academy of Sciences (CAS).

## References

  * Agazie et al. (2023) Agazie, G., Anumarlapudi, A., Archibald, A. M., et al. 2023, ApJ, 951, L8, doi: 10.3847/2041-8213/acdac6
  * Antoniadis et al. (2023) Antoniadis, J., Arumugam, P., Arumugam, S., et al. 2023, arXiv e-prints, arXiv:2306.16214, doi: 10.48550/arXiv.2306.16214
  * Backer (1970) Backer, D. C. 1970, Nature, 227, 692, doi: 10.1038/227692a0
  * Basu et al. (2020) Basu, R., Mitra, D., & Melikidze, G. I. 2020, ApJ, 889, 133, doi: 10.3847/1538-4357/ab63c9
  * Burke-Spolaor et al. (2019) Burke-Spolaor, S., Taylor, S. R., Charisi, M., et al. 2019, A&A Rev., 27, 5, doi: 10.1007/s00159-019-0115-7
  * Cordes & Downs (1985) Cordes, J. M., & Downs, G. S. 1985, ApJS, 59, 343, doi: 10.1086/191076
  * Cordes & Lazio (2002) Cordes, J. M., & Lazio, T. J. W. 2002, arXiv e-prints, astro.  https://arxiv.org/abs/astro-ph/0207156
  * Cordes & Shannon (2010) Cordes, J. M., & Shannon, R. M. 2010, arXiv e-prints, arXiv:1010.3785.  https://arxiv.org/abs/1010.3785
  * Dai et al. (2015) Dai, S., Hobbs, G., Manchester, R. N., et al. 2015, MNRAS, 449, 3223, doi: 10.1093/mnras/stv508
  * Demorest et al. (2010) Demorest, P. B., Pennucci, T., Ransom, S. M., Roberts, M. S. E., & Hessels, J. W. T. 2010, Nature, 467, 1081, doi: 10.1038/nature09466
  * Detweiler (1979) Detweiler, S. 1979, ApJ, 234, 1100, doi: 10.1086/157593
  * Dyks et al. (2010) Dyks, J., Rudak, B., & Demorest, P. 2010, MNRAS, 401, 1781, doi: 10.1111/j.1365-2966.2009.15679.x
  * Edwards & Stappers (2003) Edwards, R. T., & Stappers, B. W. 2003, A&A, 407, 273, doi: 10.1051/0004-6361:20030716
  * Feng et al. (2021) Feng, Y., Hobbs, G., Li, D., et al. 2021, ApJ, 908, 105, doi: 10.3847/1538-4357/abd326
  * Foster & Backer (1990) Foster, R. S., & Backer, D. C. 1990, ApJ, 361, 300, doi: 10.1086/169195
  * Guo et al. (2019) Guo, Y. J., Li, G. Y., Lee, K. J., & Caballero, R. N. 2019, MNRAS, 489, 5573, doi: 10.1093/mnras/stz2515
  * Hellings & Downs (1983) Hellings, R. W., & Downs, G. S. 1983, ApJ, 265, L39, doi: 10.1086/183954
  * Hobbs et al. (2010) Hobbs, G., Archibald, A., Arzoumanian, Z., et al. 2010, Classical and Quantum Gravity, 27, 084013, doi: 10.1088/0264-9381/27/8/084013
  * Hobbs et al. (2020) Hobbs, G., Guo, L., Caballero, R. N., et al. 2020, MNRAS, 491, 5951, doi: 10.1093/mnras/stz3071
  * Hobbs et al. (2006) Hobbs, G. B., Edwards, R. T., & Manchester, R. N. 2006, MNRAS, 369, 655, doi: 10.1111/j.1365-2966.2006.10302.x
  * Hotan et al. (2004) Hotan, A. W., van Straten, W., & Manchester, R. N. 2004, PASA, 21, 302, doi: 10.1071/AS04022
  * Jenet et al. (2005) Jenet, F. A., Hobbs, G. B., Lee, K. J., & Manchester, R. N. 2005, ApJ, 625, L123, doi: 10.1086/431220
  * Jennings et al. (2022) Jennings, R. J., Cordes, J. M., Chatterjee, S., et al. 2022, arXiv e-prints, arXiv:2210.12266, doi: 10.48550/arXiv.2210.12266
  * Jiang et al. (2019) Jiang, P., Yue, Y., Gan, H., et al. 2019, Science China Physics, Mechanics, and Astronomy, 62, 959502, doi: 10.1007/s11433-018-9376-1
  * Kramer & Champion (2013) Kramer, M., & Champion, D. J. 2013, Classical and Quantum Gravity, 30, 224009, doi: 10.1088/0264-9381/30/22/224009
  * Kramer et al. (2006) Kramer, M., Stairs, I. H., Manchester, R. N., et al. 2006, Science, 314, 97, doi: 10.1126/science.1132305
  * Lam et al. (2018) Lam, M. T., McLaughlin, M. A., Cordes, J. M., Chatterjee, S., & Lazio, T. J. W. 2018, ApJ, 861, 12, doi: 10.3847/1538-4357/aac48d
  * Lam et al. (2016) Lam, M. T., Cordes, J. M., Chatterjee, S., et al. 2016, ApJ, 819, 155, doi: 10.3847/0004-637X/819/2/155
  * Lam et al. (2019) Lam, M. T., McLaughlin, M. A., Arzoumanian, Z., et al. 2019, ApJ, 872, 193, doi: 10.3847/1538-4357/ab01cd
  * Latham et al. (2012) Latham, C., Mitra, D., & Rankin, J. 2012, MNRAS, 427, 180, doi: 10.1111/j.1365-2966.2012.21985.x
  * Liu et al. (2012) Liu, K., Keane, E. F., Lee, K. J., et al. 2012, MNRAS, 420, 361, doi: 10.1111/j.1365-2966.2011.20041.x
  * Liu et al. (2016) Liu, K., Bassa, C. G., Janssen, G. H., et al. 2016, MNRAS, 463, 3239, doi: 10.1093/mnras/stw2223
  * Mahajan et al. (2018) Mahajan, N., van Kerkwijk, M. H., Main, R., & Pen, U.-L. 2018, ApJ, 867, L2, doi: 10.3847/2041-8213/aae713
  * Manchester et al. (2005) Manchester, R. N., Hobbs, G. B., Teoh, A., & Hobbs, M. 2005, AJ, 129, 1993, doi: 10.1086/428488
  * Manchester & IPTA (2013) Manchester, R. N., & IPTA. 2013, Classical and Quantum Gravity, 30, 224010, doi: 10.1088/0264-9381/30/22/224010
  * McLaughlin (2013) McLaughlin, M. A. 2013, Classical and Quantum Gravity, 30, 224008, doi: 10.1088/0264-9381/30/22/224008
  * Miles et al. (2022) Miles, M. T., Shannon, R. M., Bailes, M., et al. 2022, MNRAS, 510, 5908, doi: 10.1093/mnras/stab3549
  * Miles et al. (2023) —. 2023, MNRAS, 519, 3976, doi: 10.1093/mnras/stac3644
  * Osłowski et al. (2014) Osłowski, S., van Straten, W., Bailes, M., Jameson, A., & Hobbs, G. 2014, MNRAS, 441, 3148, doi: 10.1093/mnras/stu804
  * Osłowski et al. (2013) Osłowski, S., van Straten, W., Demorest, P., & Bailes, M. 2013, MNRAS, 430, 416, doi: 10.1093/mnras/sts662
  * Osłowski et al. (2011) Osłowski, S., van Straten, W., Hobbs, G. B., Bailes, M., & Demorest, P. 2011, MNRAS, 418, 1258, doi: 10.1111/j.1365-2966.2011.19578.x
  * Parthasarathy et al. (2021) Parthasarathy, A., Bailes, M., Shannon, R. M., et al. 2021, MNRAS, 502, 407, doi: 10.1093/mnras/stab037
  * Perera et al. (2019) Perera, B. B. P., DeCesar, M. E., Demorest, P. B., et al. 2019, MNRAS, 490, 4666, doi: 10.1093/mnras/stz2857
  * Reardon et al. (2023) Reardon, D. J., Zic, A., Shannon, R. M., et al. 2023, ApJ, 951, L6, doi: 10.3847/2041-8213/acdd02
  * Shannon et al. (2014) Shannon, R. M., Osłowski, S., Dai, S., et al. 2014, MNRAS, 443, 1463, doi: 10.1093/mnras/stu1213
  * Tarafdar et al. (2022) Tarafdar, P., Nobleson, K., Rana, P., et al. 2022, PASA, 39, e053, doi: 10.1017/pasa.2022.46
  * van Straten (2006) van Straten, W. 2006, ApJ, 642, 1004, doi: 10.1086/501001
  * van Straten & Bailes (2011) van Straten, W., & Bailes, M. 2011, PASA, 28, 1, doi: 10.1071/AS10021
  * Verbiest & Shaifullah (2018) Verbiest, J. P. W., & Shaifullah, G. M. 2018, Classical and Quantum Gravity, 35, 133001, doi: 10.1088/1361-6382/aac412
  * Verbiest et al. (2016) Verbiest, J. P. W., Lentati, L., Hobbs, G., et al. 2016, MNRAS, 458, 1267, doi: 10.1093/mnras/stw347
  * Wahl et al. (2022) Wahl, H. M., McLaughlin, M. A., Gentile, P. A., et al. 2022, ApJ, 926, 168, doi: 10.3847/1538-4357/ac4045
  * Wang et al. (2023) Wang, N., Xu, Q., Ma, J., et al. 2023, Science China Physics, Mechanics, and Astronomy, 66, 289512, doi: 10.1007/s11433-023-2131-1
  * Wang et al. (2020) Wang, S. Q., Hobbs, G., Wang, J. B., et al. 2020, ApJ, 902, L13, doi: 10.3847/2041-8213/abbb89
  * Wang et al. (2021) Wang, S. Q., Wang, J. B., Wang, N., et al. 2021, ApJ, 913, 67, doi: 10.3847/1538-4357/abf937
  * Weltevrede (2016) Weltevrede, P. 2016, A&A, 590, A109, doi: 10.1051/0004-6361/201527950
  * Weltevrede et al. (2007) Weltevrede, P., Stappers, B. W., & Edwards, R. T. 2007, A&A, 469, 607, doi: 10.1051/0004-6361:20066855
  * Xu et al. (2021) Xu, H., Huang, Y. X., Burgay, M., et al. 2021, The Astronomer’s Telegram, 14642, 1
  * Xu et al. (2023) Xu, H., Chen, S., Guo, Y., et al. 2023, Research in Astronomy and Astrophysics, 23, 075024, doi: 10.1088/1674-4527/acdfa5


# POS tagging, lemmatization and dependency parsing of West Frisian

###### Abstract

We present a lemmatizer/POS-tagger/dependency parser for West Frisian using a
corpus of 44,714 words in 3,126 sentences that were annotated according to the
guidelines of Universal Dependency version 2. POS tags were assigned to words
by using a Dutch POS tagger that was applied to a literal word-by-word
translation, or to sentences of a Dutch parallel text. Best results were
obtained when using literal translations that were created by using the
Frisian translation program Oersetter. Morphologic and syntactic annotations
were generated on the basis of a literal Dutch translation as well. The
performance of the lemmatizer/tagger/annotator when it was trained using
default parameters was compared to the performance that was obtained when
using the parameter values that were used for training the LassySmall UD 2.5
corpus. A significant improvement was found for ‘lemma’. The Frisian
lemmatizer/PoS tagger/dependency parser is released as a web app and as a web
service.

Keywords: keyword A, keyword B, keyword C

languageresourceLanguage Resources

POS tagging, lemmatization and dependency parsing of West Frisian

Wilbert Heeringa1, Gosse Bouma2, Martha Hofman1, Eduard Drenth1,
---
Jan Wijffels3, Hans Van de Velde1,4
1Fryske Akademy, 2University of Groningen, 3BNOSAC, 4Utrecht University
1Leeuwarden, 2Groningen, 3Brussels, 4Utrecht
1{wheeringa, mhofman, edrenth<EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS>

Abstract content

##  1\. Introduction

Since small languages are struggling with restricted possibilities for
developing language tools, there is a need to develop an efficient strategy
that makes the development of tools feasible, for example by taking a route
via a larger and related language for which the desired tools have already
been developed. There are several papers that focus on tagging (historical)
texts via a closely-related language.

In the simplest case texts of a small language are tagged by taggers which are
trained on the basis of tagged corpora of closely-related languages. This
approach is mentioned by ?) who tagged Catalan data using a Spanish PoS-
tagger. He obtained an accuracy of 58.42%. ?) tagged 17th-century Dutch texts
with a modern Dutch POS-tagger and got accuracies of 62.8% and 63.7%. The low
accuracies show that this approach is not enough, but it can serve a baseline
to which other approaches can be compared. In the case of West Frisian, we
consider using a tagger that is trained on the basis of Dutch texts.

?) obtained higher accuracies of up to 89.08% when the Spanish tagger was
adapted to Catalan. Words in the tagger parameter files were translated into
the low-resource language by using translation dictionaries that were created
with unsupervised lexicon induction techniques that rely only on raw textual
data.

Results can also be improved by adapting the spelling of corpora prior to
tagging them with a tagger. For example, ?) investigated PoS-tagging of 17th-
century Dutch texts. They found that “that modernizing the spelling of corpora
prior to tagging them with a tagger trained on contemporary Dutch results in a
large increase in accuracy, but that spelling normalization alone is not
sufficient to obtain state-of-the-art results.”

?) initiated an approach that draws on parallel corpora. The source side of a
parallel corpus is tagged with an existing tagger and the tags along the word
alignment links are projected on the target side. Then a new tagger is trained
on the target side.

?) PoS-tagged 17th-century Dutch texts. He translated the texts into modern
Dutch word by word. On the basis of the modern Dutch translation the texts
were PoS tagged with Frog. Compared the the previously mentioned method the
advantage is that no parallel text is required. ?) evaluated four word-by-word
translation methods. In the first method, the machine translation system Moses
was used with two versions of the Dutch Statenvertaling Bible, one from the
year 1637 and one from 1888. The second approach used the Integrated Language
Bank (GTB), an online collection of historical dictionaries, with links of
historical words to their modern counterparts. The lexicon service makes it
possible to retrieve modern lemmas for historical words. The third used a
lexicon that was learned from two versions of a Dutch Bible, one from the
seventeenth century and one from the nineteenth century. The fourth method
employed orthographic rules learned from the learned lexicon. The rules
converted historical character sequences to their modern equivalent. The
lexicon-based methods outperformed the method that used orthographic rules.

In this paper we present a corpus of lemmatized, tagged and annotated text in
the West Frisian language together with a web application and web service
which can be used for lemmatization, part-of-speech (PoS) tagging and
dependency parsing of Frisian text. West Frisian is an autochthonous minority
language spoken in the Dutch province of Fryslân where it is recognized as an
official language, in addition to Dutch. In 2018 about 400,000 native speakers
were reported [Klinkenberg et al., 2018].

The corpus was built by taking a route via Standard Dutch. We investigated
some of the techniques mentioned above. In Section 2. the building of the
corpus is described, and different procedures are compared to each other. In
Section 3. the training of the lemmatizer/tagger/annotator is described and
performance measurements are provided. In Section 4. a web app and a web
service are presented and some results are shown. Conclusions and future
prospects are given in Section 5..

##  2\. Building the corpus

###  2.1. Source texts

We aimed to develop a Frisian lemmatizer/PoS tagger/dependency parser that can
be used for a wide range of texts. Therefore, texts of different genres should
be included in the corpus. We considered several (well-known) corpora that
were PoS tagged: the Brown Corpus, the Lancaster-Oslo-Bergen Corpus, the
Stockholm-Umeå Corpus, the Balanced Corpus of Contemporary Written Japanese,
the NOAH’s Corpus of Swiss German Dialects, the STEVIN reference corpus of 500
million words, the SoNaR corpus. In all of these corpora we found news texts.
Novels, legal texts/laws and Wikipedia texts were found in three corpora, and
scientific texts were found in two corpora. We decided to include texts of the
same genres in our newly built corpus.

Texts were selected from the Frisan-Dutch parallel corpus that is used for the
online Frisan/Dutch - Dutch/Frisian translation program Oersetter
‘translator’. The corpus consists of 120 texts that are available both in
Frisian and Dutch. The Frisian texts consists of 3,002,327 words, and the
Dutch texts consists of 5,057,248 words.

We added texts from the Frisian broadcast station Omrop Fryslân as well as the
Wikipedia text ‘Frysk’. The distributution of the genres in the newly compiled
corpus is shown in Table 1. Legal texts are not included yet.

source | #tokens | %tokens | #words | %words | #sentences | %sentences
---|---|---|---|---|---|---
news | 8737 | 17 | 7998 | 17 | 582 | 19
science | 2293 | 4 | 2069 | 5 | 107 | 3
novels | 17176 | 34 | 14272 | 32 | 1446 | 46
museum | 9275 | 18 | 8335 | 19 | 486 | 16
Wikipedia | 13780 | 27 | 12040 | 27 | 505 | 16
total | 51261 |  | 44714 |  | 3126 |
Table 1: Distribution of genres across the newly compiled corpus.

###  2.2. Universal dependencies

We decided to annotate the corpus according to the guidelines of Universal
Dependencies (UD) version 2111See
https://universaldependencies.org/guidelines.html.. UD is a project that aims
to develop a cross-linguistically consistent treebank annotation for many
languages. “The general philosophy is to provide a universal inventory of
categories and guidelines to facilitate consistent annotation of similar
constructions across languages, while allowing language-specific extensions
when necessary.”222See https://universaldependencies.org/introduction.html.

###  2.3. Adding lemmas

In the corpus lemmas of the tokens were added manually. When the corpus
consisted of about 30.000 tokens, a lemmatizer was trained on the basis of the
lemmatized Frisian texts using the function udpipe_train from the R package
udpipe [Wijffels, 2020]. Using this tagger, newly added corpus texts were
lemmatized, and the lemmas were manually corrected. The techniques that are
made available in the R package udpipe are described by ?) and ?).

###  2.4. Adding PoS tags

For adding PoS tags three procedures were investigated. In the first procedure
the Frisian text was tagged with a Dutch PoS-tagger that was trained with the
UD Dutch LassySmall treebank. This corpus contains sentences from the
Wikipedia section of the Lassy Small Treebank. The Lassy Small Treebank is a
manually verified treebank for Dutch. Since not all material in the Lassy
Small Treebank could be made freely available, only the material from the
Wikipedia (wiki) section was included in the UD Dutch LassySmall treebank [Van
Noord et al., 2013].

In the second procedure we followed ?). A Dutch literal word-for-word
translation is made from the the Frisian text. This literal translation is
tagged by the Dutch PoS tagger that was trained with the Lassysmall UD 2.8
corpus.333The corpus is found at
https://github.com/UniversalDependencies/UD_Dutch-LassySmall. The tags of the
Dutch words are projected on the corresponding Frisian words. In order to
obtain a literal translation, we investigated three translation programs:

  * •

Google Translate. Google Translate uses a neural machine translation engine
called Google Neural Machine Translation (GNMT). With GNMT the quality of the
tranlation is improved by applying an example-based machine translation method
in which the system “learns from millions of examples” [Schuster et al.,
2016]. We called the API by using the function gl_translate from the R package
googleLanguageR [Edmondson, 2020]. As ?) wrote, GNMT translates “whole
sentences at a time, rather than just piece by piece…” This makes Google
Translate less suitable for generating a word-for-word translation. The best
way to translate an individual word is submitting the word being enclosed in
single quotes.

  * •

The online Frisan/Dutch - Dutch/Frisian translation program Oersetter
‘translator’444Available at https://taalweb.frl/oersetter.. This program is a
statistical machine translation (SMT) system. A parallel training corpus has
been compiled and been used to automatically learn a phrase-based SMT
model[Van Gompel et al., 2014]. The translation system is built around the
open-source SMT software Moses. We used the web service of the Oersetter.

  * •

A newer version of the Oersetter555Available at
https://frisian.eu/TEST/oersetter/.. This program is a neural machine
translation (NMT) system. More specifically a transformer-based sequence-to-
sequence model is employed. Marian-NMT is used as the sofware that powers the
program. With the new Oersetter higher scores on automatic evaluation measures
were achieved.666See experiments 5 and 6 at https://bitbucket.org/fryske-
akademy/oersetter2-pipeline/src/master/data/evaluation/README.md. Like Google
Translate this program is intended to translate word phrases or sentences. In
order to get an individual word translated by an individual word, the word to
be translated should be put between double quotes. We used the web service of
this program777We used the version that was online on June 3rd, 2021.

In the third procedure the approach that was initiated by ?) was used. The
Dutch parallel text was tagged with the tagger that was trained with the
LassySmall corpus. Then the sentences of the Dutch texts were aligned with the
corresponding sentences in the Frisian texts using fast_align [Dyer et al.,
2013]888See https://github.com/clab/fast_align. and the tags of the words in
the Dutch sentences were projected on the corresponding Frisian words in the
Frisian sentences.

We tested the different procedures to the scientific texts, the museum texts,
a part of the news texts and a part of the novel texts. The results are shown
in Table 2. The highest percentage was found when tags were obtained via a
literal Dutch translation that was obtained with the (old) Oersetter.

| % correct
---|---
Baseline | 51.5%
Alignment with Dutch text | 74.2%
Google Translate | 76.0%
Oersetter | 89.8%
Oersetter new | 81.2%
Table 2: Percentages of correct PoS tags per translation procedure. |  |  |  $p$ value
---|---|---|---
Align. Dutch | $>$ | Baseline | $<.0001$
Google Transl. | $>$ | Align. Dutch | $<.001$
Oersetter new | $>$ | Google Transl. | $<.0001$
Oersetter | $>$ | Oersetter new | $<.0001$
Table 3: Comparison of percentages of correct PoS tags among translation
procedures.

Using Fishers exact test the percentages of correct PoS tags among the
procedures were compared. The results are presented in Table 3. From this
table it can be concluded that the Oersetter performs significantly better
than any of of the other translation procedures. The lower percentage for the
technique where Frisian and Dutch sentences are aligned to each other can be
explained by that fact that the aligner was not able to process swaps
correctly.

By the time that we built the corpus, we started tagging the corpus via the
Dutch text that was aligned to the Frisian text. Later on we tagged the corpus
via literal translations that were generated by the Oersetter program. The
tags were corrected manually. When the corpus consisted of about 30.000
tokens, we were able to train a Frisian tagger that had a moderate accuracy.
We used the function udpipe_train from the R package udpipe. Newly added
Frisian texts were tagged with this tagger and manually corrected afterwards.

###  2.5. Adding morphologic and syntactic annotations

Initially, 575 Dutch sentences from Oersetter parallel corpus were
morphologically/syntactically annotated with a Dutch UD annotator, and the
annotations were projected on the corresponding Frisian sentences. Corrections
of the annotations were made where alignments between Dutch and Frisian
sentences went wrong. Later, literal Dutch word-by-word translations were
generated by the Oersetter and subsequently annotated. Before annotating, the
translations were manually corrected. A Dutch annotator was used that was
trained on the LassySmall corpus using the function udpipe_train from the R
package udpipe.

Since correction work is involved in both procedures, it doesn’t really matter
which one is used. But the latter method also works if there is no Dutch
translation of the Frisian text available.

##  3\. Training and performance

We randomized the order of the sentences in the lemmatized/tagged/annotated
data set and split the sentences in training data (80% of the sentences),
validation data (10%) and test data (10%).

A lemmatizer, PoS tagger and dependency parser was trained by using the
function udpipe_train in the R package udpipe [Wijffels, 2020] 999See also
https://cran.r-project.org/web/packages/udpipe/vignettes/udpipe-
annotation.html. on the basis of the training data. As options for the
tokenizer, PoS tagger, lemmatizer and dependency parser the default values
were used.101010See Sections 3.3, 3.4 and 3.5 at
https://ufal.mff.cuni.cz/udpipe/1/users-manual.

We present the tagger, lemmatizer and parser performance, measured on the
testing portion of the data, evaluated in three different settings: using raw
text only, using gold tokenization only, and using gold tokenization plus gold
morphology (UPOS, XPOS, FEATS and Lemma). A $k$-fold cross-validation was
carried out with $k$=10. First, the data set was split into $k$ sets. Then $k$
times (or folds) a set is selected as test set and another set is selected as
validation set. When the sets are numbered from 1 to $k$ in the $i$th fold set
$i$ is selected as test set, and set $i+1$ is selected as validation set. If
$i$ is $k$ the first set is selected as validation set. The joint remaining
$k-2$ sets are used as training sets. The results are shown in Table 4 and
show the mean and standard deviation across the 10 folds per metric and for
each of the three settings.

| Raw text | Gold tok | Gold tok + mor
---|---|---|---
| mean | sd | mean | sd | mean | sd
f1 words | 100.0 | 0.01 |  |  |  |
f1 sents | 91.4 | 1.88 |  |  |  |
UPOS | 94.4 | 0.48 | 94.4 | 0.47 |  |
XPOS | 87.8 | 0.56 | 87.8 | 0.56 |  |
UFeats | 89.6 | 0.43 | 89.6 | 0.42 |  |
AllTags | 85.3 | 0.73 | 85.3 | 0.73 |  |
Lemma | 94.9 | 0.48 | 94.9 | 0.48 |  |
UAS | 72.3 | 1.32 | 72.7 | 1.26 | 78.4 | 1.18
LAS | 66.3 | 1.35 | 66.6 | 1.32 | 73.9 | 1.36
Table 4: Lemmatizer, tagger and parser performance, measured on the testing
portion of the data, evaluated in three different settings. The figures
represent percentages. When training the default UDPipe settings were used.

Additionally, a lemmatizer, PoS tagger and dependency parser was trained using
the options that were used for training on the basis of the LassySmall UD 2.5
corpus.111111For the settings see
https://github.com/bnosac/udpipe/tree/master/inst/models-
ud-2.5orhttps://lindat.mff.cuni.cz/repository/xmlui/handle/11234/1-3131 where
the parameters can be found in ‘udpipe-
ud-2.5-191206-reproducible_training.zip’. The results are shown in Table 5.
Compared to the results in Table 4, we found a significant improvement for
‘lemma’ ($p<0.0001$)121212Rand Wilcox’s function medmcp from the Rallfun-v38
package was used. This function compares medians using a percentile bootstrap
method. This function is very robust, even for small samples with a non-normal
distribution [Wilcox, in press]. both for ‘Raw text’ and ‘Gold tok’.

| Raw text | Gold tok | Gold tok + mor
---|---|---|---
| mean | sd | mean | sd | mean | sd
f1 words | 100.0 | 0.01 |  |  |  |
f1 sents | 89.7 | 2.41 |  |  |  |
UPOS | 94.6 | 0.28 | 94.6 | 0.30 |  |
XPOS | 88.1 | 0.50 | 88.1 | 0.52 |  |
UFeats | 89.8 | 0.47 | 89.8 | 0.50 |  |
AllTags | 85.5 | 0.43 | 85.5 | 0.45 |  |
Lemma | 96.0 | 0.25 | 96.0 | 0.25 |  |
UAS | 72.5 | 1.15 | 73.1 | 1.12 | 78.5 | 0.87
LAS | 66.4 | 1.08 | 67.0 | 1.13 | 73.9 | 1.21
Table 5: Lemmatizer, tagger and parser performance, measured on the testing
portion of the data, evaluated in three different settings. The figures
represent percentages. The settings for training on the basis of UD 2.5
corpora were used.

UAS (Unlabelled Attachment Score) and LAS (Labelled Attachment Score) are
standard metrics to evaluate dependency parsing. UAS is the proportion of
tokens whose head has been correctly assigned, and LAS is the proportion of
tokens whose head has been correctly assigned with the right dependency label
(subject, object, etc).

A lemmatizer, PoS tagger and dependency parser was trained using the
LassySmall UD 2.8 corpus with the options that were used for training on the
basis of the LassySmall UD 2.5 corpus. The LassySmall UD 2.8 corpus as it is
available at GitHub is much larger than the current Frisian corpus and
consists of 7341 sentences, 83571 words and 98242 tokens when we put the
train, development, and test sections together. The performance results are
shown in Table 6.

We compared the results with the performance results in Table 5. For all
metrics we found significant differences ($p<0.0001$), usually the LassySmall
figures are significantly better than the ones for the Frisian corpus.
Exceptions are ‘f1 word’, ‘f1 sents’ and ‘lemma’ (both for ‘Raw text’ and
‘Gold tok’).

| Raw text | Gold tok | Gold tok + mor
---|---|---|---
| mean | sd | mean | sd | mean | sd
f1 words | 99.9 | 0.03 |  |  |  |
f1 sents | 81.3 | 1.31 |  |  |  |
UPOS | 95.6 | 0.30 | 95.9 | 0.30 |  |
XPOS | 93.7 | 0.38 | 94.1 | 0.36 |  |
UFeats | 95.1 | 0.26 | 95.6 | 0.26 |  |
AllTags | 93.1 | 0.38 | 93.5 | 0.36 |  |
Lemma | 94.2 | 0.25 | 94.4 | 0.25 |  |
UAS | 81.3 | 0.74 | 83.4 | 0.71 | 87.1 | 0.59
LAS | 77.5 | 0.73 | 79.4 | 0.65 | 84.0 | 0.66
Table 6: Lemmatizer, tagger and parser performance using the LassySmall UD 2.8
corpus, measured on the testing portion of the data, evaluated in three
different settings. The figures represent percentages. The settings for
training on the basis of UD 2.5 corpora were used.

##  4\. Tools and results

The Frisian lemmatizer/PoS tagger/dependency parser is released as a web app
for human end users131313See https://frisian.eu/udpipeapp/., as well as a web
service for software to interact with141414See
https://frisian.eu/udpipeservice/.. The source code is available at
Bitbucket.151515Online at https://bitbucket.org/fryske-
akademy/udpipe/src/master/. The user can choose from different output formats:
a tab-separated file, an Excel file or a file in CoNLL-U format.

The online app has also a tab called ‘graphs’. The graphs provided in this tab
visualize the frequencies of POS tags in the submitted text, the most
frequently occurring tokens per PoS tag, frequencies of keyword word
sequences, co-occurrence networks and word clouds. The graphs are made by
using several R packages.161616The following packages are used: lattice,
udpipe, textrank, igraph, ggraph, ggplot2, ggwordcloud. An example is shown in
Figure 1. The graph is obtained on the basis of the Wikipedia text ’Ingelsk’
(English)171717The text was retrieved on June 3rd, 2021. and shows the
frequencies of the PoS tags in the text. On the basis of the same text Figure
2 was made. This figure visualizes frequencies of co-occurrences of
prepositions within a sentence. The co-occurrence of the prepositions fan ‘of’
and yn within a sentence is found to be most frequent.

Figure 1: Frequencies of PoS tags in the Wikipedia text ‘Ingelsk’.

Figure 2: Co-occurrences of prepositions within Frisian sentences in the
Wikipedia text ‘Ingelsk’.

##  5\. Conclusions and future prospects

In this paper we described how we built a West Frisian corpus as training
material for a UD lemmatizer, PoS tagger and dependency parser. Much time is
saved by not building the corpus from scratch, but rather by taking a route
via a larger and related language, in this case Standard Dutch.

A Dutch tagger/annotator is applied to a (literal) Dutch translation of the
Frisian text. As for obtaining PoS tags, we investigated several alternatives
such as aligning the Frisian text to a Dutch parallel text, or translating the
Frisian text into Dutch by using Google Translate. However, the better PoS
tags were obtained on the basis of a literal Dutch translation that was
created by using the Frisian/Dutch - Dutch/Frisian translation program
Oersetter. Manual correction of the PoS tags is still necessary.

Morphological and syntactical annotations are also preferably generated on the
basis of an Oersetter translation, but the translations need to be carefully
checked and corrected before submitting them to a Dutch annotator.

The current corpus consists of 44,714 words. When training a lemmatizer/PoS
tagger/dependency parser on the basis of this corpus, reasonable accuracies
were obtained.

The performance of the lemmatizer/tagger/annotator when it was trained using
default parameters was compared to the performance that was obtained when
using the parameter values that were used for training the LassySmall UD 2.5
corpus. A significant improvement was found for ‘lemma’. Once the corpus is
complete, the parameters will be tuned to achieve optimum performance.

We compared the performance obtained on the basis of the latter parameters
with the performance of a lemmatizer/tagger/annotator that was trained on the
basis of the LassySmall UD 2.8 corpus using the same parameters. The
LassySmall UD 2.8 corpus is much larger than the current Frisian corpus and
the performance when using this corpus was significantly better for most
metrics compared to the Frisian corpus. Therefore, we aim to extend the
Frisian corpus up to 100,000 words. Not only texts of the genres that are
currently in the corpus will be added, but also legal texts, minutes and texts
from manuals. We noticed that many mistakes are made when processing text that
contains hollandisms and non-standard Frisian variants. This will also be
taken into account when extending the corpus.

The tools that we developed will be of great help for studying the Frisian
language on the basis of (large) corpora. One can think of named entity
resolution, coreference resolution, sentiment analysis, question answering,
textometry, authorship discrimination, language detection, and measuring the
influence of Dutch on Frisian.

In addition to the output formats currently provided, the XML/TEI format will
be added. This will make the output also searchable for search engines like
BlackLab [De Does et al., 2017] which is used in the publicly available search
system of Frisian corpora.181818See https://frisian.eu/frisian-corpora, where
currently mainly Middle Frisian and minimally annotated modern Frisian texts
can be searched.

##  6\. Acknowledgements

This research was made possible by a CLARIAH-Plus project financed by the
Dutch Research Council (Grant 184.034.023).

##  7\. References

## References

  * De Does et al., 2017 De Does, J., Niestadt, J., and Depuydt, K.  (2017).  Creating research environments with BlackLab.  In Odijk J. et al., editors, CLARIN in the Low Countries, pages 245–257. Ubiquity Press, London.
  * Dyer et al., 2013 Dyer, C., Chahuneau, V., and Smith, N. A.  (2013).  A simple, fast, and effective reparameterization of ibm model 2.  In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 644–648.
  * Edmondson, 2020 Edmondson, M., (2020).  googleLanguageR: Call Google’s ’Natural Language’ API, ’Cloud Translation’ API, ’Cloud Speech’ API and ’Cloud Text-to-Speech’ API.  R package version 0.3.0.
  * Hupkes and Bod, 2016 Hupkes, D. and Bod, R.  (2016).  Pos-tagging of historical dutch.  In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 77–82.
  * Klinkenberg et al., 2018 Klinkenberg, E., Jonkman, R., and Stefan, N.  (2018).  Taal yn Fryslân. De folgjende generaasje [Language in Fryslân. The next generation].  Morgan Kaufman Publishers, Ljouwert.
  * Scherrer, 2014 Scherrer, Y.  (2014).  Unsupervised adaptation of supervised part-of-speech taggers for closely related languages.  In Proceedings of the First Workshop on Applying NLP Tools to Similar Languages, Varieties and Dialects (VarDial), pages 30–38. Association for Computational Linguistics and Dublin City University.
  * Schuster et al., 2016 Schuster, M., Johnson, M., and Thorat, N.  (2016).  Zero-shot translation with google’s multilingual neural machine translation system.  Google AI Blog, 22.
  * Straka and Straková, 2017 Straka, M. and Straková, J.  (2017).  Tokenizing, POS tagging, lemmatizing and parsing UD 2.0 with UDPipe.  In Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 88–99, Vancouver, Canada. Association for Computational Linguistics.
  * Straka et al., 2016 Straka, M., Hajic, J., and Straková, J.  (2016).  Udpipe: trainable pipeline for processing conll-u files performing tokenization, morphological analysis, pos tagging and parsing.  In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 4290–4297.
  * Tjong Kim Sang, 2016 Tjong Kim Sang, E.  (2016).  Improving part-of-speech tagging of historical text by first translating to modern text.  In International Workshop on Computational History and Data-Driven Humanities, pages 54–64. Springer.
  * Turovsky, 2016 Turovsky, B.  (2016).  Found in translation: More accurate, fluent sentences in google translate.  Blog. Google. November, 15.
  * Van Gompel et al., 2014 Van Gompel, M., Van den Bosch, A., and Dijkstra, A.  (2014).  Oersetter: Frisian-Dutch statistical machine translation.  In P. Boersma, et al., editors, Philologia Frisica anno 2012, pages 287–296. Fryske Akademy, Ljouwert.
  * Van Noord et al., 2013 Van Noord, G., Bouma, G., Van Eynde, F., de Kok, D., van der Linde, J., Schuurman, I., Sang, E. T. K., and Vandeghinste, V.  (2013).  Large scale syntactic annotation of written Dutch: Lassy.  In Peter Spyns et al., editors, Essential Speech and Language Technology for Dutch: Results by the STEVIN programme, pages 147–164. Springer Berlin Heidelberg, Berlin, Heidelberg.
  * Wijffels, 2020 Wijffels, J., (2020).  udpipe: Tokenization, Parts of Speech Tagging, Lemmatization and Dependency Parsing with the ’UDPipe’ ’NLP’ Toolkit.  R package version 0.8.4-1.
  * Wilcox, in press Wilcox, R. R.  (in press).  Introduction to Robust Estimation and Hypothesis Testing.  Academic press, San Diego, CA, 5th edition.
  * Yarowsky et al., 2001 Yarowsky, D., Ngai, G., , and Wicentowski, R.  (2001).  Inducing multilingual text analysis tools via robust projection across aligned corpora.  In Proceedings of the 1st International Conference on Human Language Technology Research (HLT), pages 161–168.