# Multiple scattering suppression for in vivo optical coherence tomography
measurement using B-scan-wise multi-focus averaging method

Yiqiang Zhu  1 Lida Zhu  1 Yiheng Lim  1 Shuichi Makita  1 Yu Guo  1and
Yoshiaki Yasuno1  1Computational Optics Group, University of Tsukuba, Tsukuba,
Ibaraki, Japan
<EMAIL_ADDRESS>https://optics.bk.tsukuba.ac.jp/COG/

††journal: opticajournal††articletype: Research Article

We demonstrate a method that reduces the noise caused by multi-scattering (MS)
photons in an in vivo optical coherence tomography image. This method combines
a specially designed image acquisition (i.e., optical coherence tomography
scan) scheme and subsequent complex signal processing. For the acquisition,
multiple cross-sectional images (frames) are sequentially acquired while the
depth position of the focus is altered for each frame by an electrically
tunable lens. In the signal processing, the frames are numerically defocus-
corrected, and complex averaged. Because of the inconsistency in the MS-photon
trajectories among the different electrically tunable lens-induced defocus,
this averaging reduces the MS signal. This method was validated using a
scattering phantom and in vivo unanesthetized small fish samples, and was
found to reduce MS noise even for unanesthetized in vivo measurement.

##  1 Introduction

Optical coherence tomography (OCT) is a noninvasive imaging technique with a
resolution of a few to tens of micrometers, and has been used for clinical
imaging in fields such as ophthalmology[1, 2, 3, 4] and cardiology[5, 6].
Because of its long imaging depth, OCT recently has been adopted for
noninvasive and nondestructive microscopy [7]. OCT can visualize the deep
tissue region at around several hundred of micrometers to a few millimeters
from the sample surface, and it is significantly larger than that of
conventional microscopy, which is only around a few tens to hundreds of
micrometers.

Conventionally, the imaging depth of OCT was believed to be dominated by two
factors, the depth of focus of the probe optics and the scattering of the
sample. To overcome the former limit, several methods have been successful.
For instance, the fusion of multiple images with several depth focus positions
[8], computational refocusing methods [9, 10, 11, 12], and the combination of
complex signal processing and focus fusion such as Gabor-domain OCT [13, 14].
To overcome the tissue-scattering limit of the imaging depth, longer
wavelength probe has been applied. In general, a 1.3-$\mu$m wavelength probe
has a better image penetration than a 830-nm or visible OCT. For retinal
imaging, a 1.0-$\mu$m probe was shown to have a higher penetration [15, 16,
17, 18], and it has become a common probe wavelength band for clinical retinal
imaging. More recently, even longer wavelength, such as 1.7 $\mu$m, has been
used to investigate samples with high scattering, such as cardiovascular
tissues [19] and brain tissues [20], and have demonstrated higher penetration
than OCT using shorter wavelengths.

Because these two limiting factors of imaging depth have been overcome, the
multiple scattering (MS) has gradually been recognized as an additional
limiting factor. In general, OCT imaging theory is based on the assumption
that most MS photons are rejected by a confocal pinhole (i.e., a single-mode
fiber tip) [21] and the single-scattering (SS) photons govern the imaging. In
practice, however, some MS photons are captured through the confocal pinhole
and appear in the image. Because an MS photon has a longer optical path than
that of the SS photons, a photon that undergoes multiple scattering at a
specific depth appears in the image to be located at a deeper location. In
cotrast, the contribution of SS photons becomes less at the deeper depth in
the image. Hence, deeper regions in an OCT image are more dominated by MS
photons. This dominance of MS photon degrades the resolution and contrast of
the OCT image in the deep regions [22]. In addition, it degrades the
quantitative measurement capability of functional OCT, such as polarization
sensitive OCT [23].

Several methods have been used to mitigate the MS effect. For example, Badon
et al.proposed the smart OCT method, which modulates the pupil using a spatial
light modulator (SLM) and pre-defined reflection matrix[24]. Borycki and
associates used SLM-based pupil modulation and spatial correlation theory to
reduce the MS effect (or equally, the coherent cross-talk) of full-field
swept-source OCT[25, 26, 27]. Liu et al.proposed the aberration-diverse method
for MS suppression[28]. In this method, intentional astigmatism was introduced
using a deformable mirror, and multiple, typically twelve, OCT volumes were
acquired with different astigmatism axes. After correcting the astigmatism
using computational adaptive optics [29], the volumes were coherently
averaged. The paths of the MS photons are not consistent across the different
astigmatism axes, whereas those of the SS photons are consistent. Hence, the
coherent average can reduce the MS-photon contributions.

Although the aforementioned modalities have successfully mitigated the MS
effect, they require expensive wavefront manipulation devices, such as an SLM
or deformable mirror. We previously proposed the multi-focus-averaging (MFA)
method, which is based on an approach similar to that of the aberration
diverse method, but we used a cost-effective electrical tunable lens (ETL)
[30]. Multiple, typically seven, OCT volumes are acquired with different
defocus positions, and the volumes are coherently averaged after correcting
the defocus using computational defocus correction. This method was also
applied to Jones-matrix based PS-OCT (JM-OCT), and mitigation of polarization
artifacts was demonstrated [23].

Although the aberration diverse method and MFA perform well on static samples,
such as static phantom and postmortem samples, its application to in vivo
measurement still poses a great challenge. Because these methods rely on
coherent (i.e., complex) averaging of multiple volumes, the phases of these
volumes should be consistent. In other words, the sample should be highly
stable during the multiple volumetric acquisition, which usually takes a few
tens of seconds.

In this work, we propose a new version of the MFA method, referred to as the
B-scan-wise-MFA (B-MFA) method for MS suppression in in vivo measurement. This
new method sequentially acquires multiple cross-sectional OCT frames, instead
of volumes, with different defocus. The defocus of each frame is corrected by
applying a one-dimensional (1-D) version of computational refocusing, and all
defocus-corrected frames are coherently averaged to reduce the MS signals.
Note that throughout the manuscript, a single cross-sectional scan is denoted
as a frame, while a B-scan refers to a set of frames acquired at the same
location. This method requires phase stability only during the acquisition
time of a few frames (i.e., a B-scan), not of volumes, so the required stable
time is typically less than 100 ms. Hence, this method is applicable to in
vivo measurement. The performance of the B-MFA method was validated by
measuring a scattering phantom and in vivo small fish.

##  2 Principle and implementation of B-MFA method

###  2.1 Data acquisition and signal-processing flow

Figure 1:  Example trajectories of single-scattering (SS) and multi-scattering
(MS) photons with two focus configurations of an electrically tunable lens
(ETL). The SS photon was scattered only once, and hence its trajectory is
consistent regardless of the ETL configuration. In contrast, the trajectories
of MS photons become inconsistent when the ETL configuration altered.

Before describing the details of the B-MFA method, we first present an
overview of this method. The B-MFA method is based on the assumption that the
trajectories of MS photons are not consistent if the depth positions of the
focus are different[30], as depicted in Fig. 1. Hence, the first step of B-MFA
is to acquire multiple frames with different focus positions using an ETL. The
details of the data acquisition protocol are described in Section 2.2. In the
second step, we correct the defocus using a 1-D phase-only spatial frequency
filter (Section 2.3). Finally, the frames are complex averaged after the axial
shifts and phase offsets of the frames have been corrected. Because we used a
JM-OCT in our particular implementation, we additionally correct the bulk-
phase offset in the four polarization channels of the JM-OCT. The details of
the shift and phase corrections as well as the complex-averaging process are
described in Section 2.4

Because MS-photons trajectories are different in frames with different focus
depths, the randomized MS signal is reduced by the complex averaging, whereas
the defocus corrected SS-signal is not. In this manuscript, we refer to this
complex-averaged image as a “B-MFA image.” For volumetric measurement, we
sequentially acquire cross-sectional B-MFA images at several slow-scan
positions.

###  2.2 Data acquisition protocol

Figure 2:  Scanning protocol for the B-MFA method(a) and previously proposed
MFA method [30] (b). N refers to the number of repeated acquisitions (i.e.,
frames) which are complex averaged after defocus correction to reduce the MS
signals. For both methods, the defocus is altered by an ETL. For B-MFA, the
defocus is altered for each frame, whereas for MFA, the defocus is altered for
each volume.

The first step of the B-MFA imaging is to acquire multiple frames with
different depth positions of focus. Here the depth positions of the focus are
actively controlled by an ETL that is equipped in the sample arm of the OCT
[30]. The implementation details of the OCT system are described later in
Section 3.1.

The data acquisition protocol of B-MFA is summarized in a schematic time chart
in Fig. 2(a). As shown in this diagram, multiple frames are sequentially
acquired at each slow-scan (i.e., B-scan) position. For every frame
acquisition, the ETL updates the depth position of the focus so that all
frames are acquired using different focus positions. This measurement process
is repeated for each slow-scan location to obtain a volumetric dataset.

In our typical implementation, a single frame acquisition time including the
focus transition time of ETL is 13.4 ms and the number of frames at a single
slow-scan location (i.e., the number of frames per B-scan) is five (see
Section 3.2 for details). Hence, the typical acquisition time for a single
slow-scan location is 67 ms.

###  2.3 Computational refocusing

####  2.3.1 Computational refocusing using a 1-D phase filter

After data acquisition, each frame is then processed for 1-D computational
refocusing. Here, the 1-D lateral complex signal at each depth of the frame is
processed by a 1-D phase-only spatial frequency filter designed based on the
Fresnel-diffraction model [10]. For a defocus distance (i.e., the distance
from the focus to the imaging depth) of $z_{d}$, the phase only filter is

$H^{-1}\left(f_{x};z_{d}\right)=\exp\left(\frac{-i\pi\lambda_{c}z_{d}f_{x}^{2}}{2}\right),$
(1)

where $f_{x}$ denotes the spatial frequencies corresponding to the fast-scan
lateral position $x$, and $\lambda_{c}$ is the center wavelength of the probe
beam.

The refocused frame $S^{\prime}(x;z)$ is obtained using this filter and two
sequential 1-D Fourier transform operations as

$S^{\prime}(x;z)=\mathscr{F}_{x}^{-1}\left[{\mathscr{F}_{x}\left[{S(x;z)}\right]H^{-1}\left(f_{x};z_{d}(z)\right)}\right],$
(2)

where $S(x;z)$ is the original complex frame and
$\mathscr{F}_{x}\left[{\quad}\right]$ and
$\mathscr{F}_{x}^{-1}\left[{\quad}\right]$ are the 1-D Fourier transform and
its inverse Fourier transform, respectively, along the fast-scan ($x$)
direction. Here, $z_{d}$ is considered to be a function of the depth in image
$z$. In our implementation, $z_{d}$ is estimated from the measured data, as
described in detail in the next section (Section 2.3.2).

Note that this method corrects the defocus only along the fast-scan direction.
The impact of this limitation is discussed in Section 5.3.

####  2.3.2 Estimation of the defocus distance

In our method, the defocus distance $z_{d}$ is estimated from the measured OCT
images, where the information entropy of the OCT images is used as a sharpness
metric. Note that here, we use the information entropy of an 2D en-face OCT
image at each depth, although the refocusing was performed for each lateral
($x$-) line individually. This is because a 1-D lateral signal is not
informative enough to compute a reliable sharpness metric.

The defocus-distance estimation is performed at each depth, and hence these
initial estimates are obtained as a function of depth. We then extract the
estimates from the depth region in which the estimated focus distances are
linear to the depth, and use them to estimate the defocus distances over the
range of all depths in the image. Specifically, the estimated defocus distance
is linear-fit to the depth using an intensity-weighted linear regression. This
linearly fitted line gives the final estimates of the defocus distance
throughout the whole depth range.

###  2.4 Shift and phase-offset corrections and complex averaging in JM-OCT

In our implementation, we used JM-OCT, which acquires four OCT cross-sectional
images at each slow-scan position[31] that corresponds to the four
polarization entries of the Jones matrix. Here, we describe the methods to
correct the mutual phase offsets in the four images in addition to the shift
and phase-offset corrections in the multiple frame acquisitions because they
are crucial to complex averaging the refocused OCT signals.

Figure 3:  Flow of phase-offset corrections. The phase offsets in the images
obtained with different defocus are corrected for each polarization channel.
We first compute the product of the target (tar) image and the complex
conjugate of the reference (ref) image (step 1, green boxes). Each product
image is complex averaged along the depth for each polarization channel (step
2, light-blue arrow), and the resultant 1-D arrays are further complex
averaged over the polarization channels (purple arrows). The phase of the
resulting 1-D complex array represents the phase offset of each A-line. The
phase offset of each A-line in the target image is then corrected (step 3,
yellow arrow and box). These processes are repeated for all JM cross-sections
(step 4).

A Jones matrix cross-sectional image (JM cross-section) consists of four
complex OCT images corresponding to the four polarization channels. For
computational refocusing, we estimate the defocus distance using only one
polarization channel with the method described in Section 2.3.2, and we apply
it to all polarization channels. Namely, the defocuses of all the polarization
channels are corrected with the same estimated defocus distance.

Because of the deformation of the ETL, there are non-negligible depth shifts
among the images taken with different defocus. We compute the shift using the
cross-correlation function of linear-intensity OCT images, where the linear
intensity images are obtained from a single polarization channel and the
intensity image of the first defocus value is used as the reference. Before
computing the cross-correlation function, the images are up-sampled four times
along the depth using Fourier-domain zero-padding to achieve sub-pixel
accuracy. The cross-correlation function is computed along the depth by a
direct method (i.e., not the Fourier-domain method). The amount of shift is
determined from the peak of the cross correlation function. After correcting
the shift in the up-sampled images, the images are down-sampled to the
original pixel resolution using Fourier-domain de-padding. The sub-pixel
shifts of the other three polarization channels are corrected using this
method but we use the shift estimate obtained using the first polarization
channel.

After correcting the axial shifts, the mutual phase offsets in the images with
different defocus are estimated and corrected. Here, we consider the phase
offsets in the JM cross-sections, where a JM cross-section is a set of four
complex OCT images one for each of the four polarization channels (PCs). We
first estimate the phase offsets of the images at each PC independently. For
this estimation, we use only the depth region of 30 pixels from the sample
surface (see Appendix A for details of the sample surface detection). The
first JM cross-section is used as a reference, and the phase offsets between
the complex images of the reference JM cross-section and the images of the
target JM cross-section are estimated. The complex image of each PC in the
target JM cross-section is multiplied by the complex conjugate of the
corresponding PC’s complex image in the reference JM cross-section (Step 1 in
Fig. 3), and then complex averaged along the depth (Step 2). Because this
operation is performed for each of four PCs, four 1-D complex arrays along the
fast scan direction are given (the phase offset of each PC in the figure). By
averaging these four 1-D complex arrays corresponding to the PCs and taking
the phase, we obtain a 1-D array of the phase offsets where each entry of the
array represents the phase offset of each A-line. Finally, the mutual phase
shift of the reference JM cross-section and the target JM cross-section is
corrected for all depths by subtracting the estimated phase offsets from the
complex images in the target JM cross-section, i.e., multiplying the conjugate
phase in complex (Step 3). This operation is repeated for all frames (i.e., JM
cross-sections) so that the phase offsets of all the frames are corrected
(Step 4).

After the phase offset corrections, the OCT images of each PC are complex
averaged over the frames to generate four MS-reduced complex OCT images, which
forms a MS-reduced JM cross-section. The final B-MFA image is obtained by
averaging the squared intensities of four complex images of the MS-reduced JM
cross-section.

Note that the aforementioned operations are for one slow-scan position, i.e.,
B-scan position. We repeat these operations for all other slow-scan positions
to obtain a B-MFA volume.

##  3 Validation design

###  3.1 JM-OCT setup

A custom-built passive-polarization-delay (PPD) based JM-OCT was used to
evaluate the B-MFA method. A PPD module splits the probe beam into two
orthogonal polarizations and applies different delays to them. In addition to
this probe-beam polarization multiplexing, polarization-diversity detection is
used. Hence, four complex OCT images corresponding to the four polarization
channels (i.e., two multiplexed polarizations of the probe beam times two
detection polarizations) are obtained.

The center wavelength and scanning bandwidth of the light source (AXP50124-8,
AXUSN, MA, USA) are 1,310 nm and 106 nm, respectively. The effective focal
length of the objective lens (LMS03, Thorlabs, NJ, USA) used in the system is
36 mm. The lateral and axial resolutions are 17 $\mu$m and 14 $\mu$m in
tissue, respectively. The depth-of-focus (DOF) without the ETL is 0.36 mm. The
A-line rate of the system is 50,000 A-lines/s. More details of the JM-OCT
principle [31] and the implementation of the particular JM-OCT system used in
this study [32, 33] can be found in elsewhere.

An ETL (EL-10-30, CI-NIR-LD-MV, Optotune, Switzerland) is used in the sample
arm to axially shift the focus position. The details of the sample arm
equipped with the ETL are described in [30].

Note that, although we used a JM-OCT in this study, the B-MFA method can be
applied to standard (i.e., non-polarization-sensitive) OCT.

###  3.2 Measurement protocol

At each slow-scan (i.e., B-scan) position, five continuous cross-sectional
frames were acquired as the defocus was incremented 0.18 mm at each
acquisition. The defocus increment was equivalent to half the DOF, where the
DOF is that without the ETL. The total focus shift of five frames was 0.72 mm.
These parameters were determined by an optimization experiment that is
described in details in Section 5.1. Each cross-sectional frame consists of
256 A-lines, and the acquisition time of a single frame was 5.12 ms. By
accounting for the defocus transition time of ETL (7.5 ms) and the pullback
time of the galvanometer scanner (0.8 ms), the five continuous frames were
acquired in 67.1 ms. The phase should be stable during this acquisition time.
The acquisition was repeated for 256 slow-scan positions, and the total time
to acquire a volume was 17.18 s. In summary, the volumetric acquisition time
and required phase-stable duration were 17.18 s and 67.1 ms, respectively.

For reference, we acquired or generated three additional volumetric images.
The first image is a single acquisition image, which was made by extracting
only the third frame of the five sequential frames acquired for B-MFA.
Although no averaging was performed, the defocus was corrected.

The second reference is the single acquisition image (SFA) image. Here, we
acquired a volume following the B-MFA protocol but without shifting the focus.
This volume was processed in a manner identical to that of the B-MFA, i.e.,
the defocus correction, shift and phase corrections, and complex averaging
were all the same.

The third reference is a standard MFA image [30]. Here, five OCT volumes were
sequentially acquired with different defocus, as shown in Fig. 2(b). The
increments in defocus between two consecutive volume acquisitions were the
same as that of the B-MFA. Each volume was acquired with a standard raster
scan with 256 $\times$ 256 A-lines, and the total acquisition time of the
sequential five volumes was 9.92 s. For MFA, the phase should be stable during
the five volume acquisitions, and hence it was 9.92 s, which is 148 times
longer than the required phase-stable time of B-MFA.

For all scan protocols, the lateral scanning range was 1.5 mm $\times$ 1.5 mm.
The lateral field was covered with 256 $\times$ 256 lateral sampling points,
which yielded a lateral pixel separation of 5.86 $\mu$m that is around 1/3 of
the lateral resolution.

###  3.3 Samples

Figure 4: Schematic (a) and photograph (b) of the scattering phantom. The
phantom consists of a scattering layer, which is a mixture of polystyrene
micro-particles and ultrasound gel, and a glass plate embedded in the
scattering layer. The glass plate is used to provide a scattering-free area.
(c) An in vivo medaka sample held in a 3-D printed container. The container
was filled with water and fish was not anesthetized. A thin cover glass was
put on the container to prevent the fish from accidentally jumping out. The
red box roughly indicates the measured area of the OCT.

A scattering phantom and ten in vivo medaka fish were measured to evaluate the
proposed method. The scattering phantom consists of four parts, two cover
glasses, a scattering layer, and a glass plate under the scattering layer that
creates a space without a back scattering signal. The scattering layer is a
mixture of 0.04-mL polystyrene microparticles (diameter of 10 $\mu$m,
72968-10ML-F, Sigma-Aldrich, MS, USA) and 0.5-mL ultrasound gel (proJelly,
Jex, Japan). The schematic and photograph of the phantom are shown in Fig.
4(a) and (b).

The medaka, also known as the Japanese rice fish, is a small fish similar in
size to a zebrafish and is widely used as a model animal in biological
research. We measured ten in vivo medakas without anesthesia. A fish was
placed in a 3-D printed container with a water-filled groove that was 5 mm
$\times$ 8 mm $\times$ 31 mm (height $\times$ width $\times$ length) in size.
We placed a thin glass slip above the container to prevent the fish from
accidentally jumping out during the measurement. Figure 4(c) shows a
photograph of the container and a sample. In this figure, the red box roughly
indicates the scan area.

The protocol of the fish experiment follows the animal experiment guidelines
of the University of Tsukuba and is approved by the Institutional Animal Care
and Use Committee of the University of Tsukuba.

##  4 Result

###  4.1 Scattering phantom

Figure 5: Cross-sectional images obtained by the single acquisition (a), SFA
(b), MFA (c), and B-MFA methods (d). The scale bar indicates 300 $\mu$m. The
regions indicated by the green dashed boxes are enlarged in the second row
(e–h). In the superficial regions in the glass plates (blue arrows), the MFA
and B-MFA images have lower signal intensities than the other images because
of the reduction of the MS signal. (i) Averaged intensity depth profiles of
each image, which are averaged using A-lines in the orange brace in (a), where
zero dB refers to the noise floor (the signal in the air region) in the single
acquisition image. The black arrow indicates the surface of the scattering
region. In the deep region of the scattering sample (purple shading) and the
superficial area of the glass plate (light-blue shading), the MFA and B-MFA
show lower signal intensity than the other methods because of the reduction in
MS.

Figure 5 shows the intensity cross-sectional images of the scattering phantom
for the single acquisition, SFA, MFA, and B-MFA methods from left to right.
The images in the second row are magnified views of the deep regions,
indicated in the images in the first row. In the shallow regions of the
sample, the standard MFA image has fewer particles than the other images [Fig.
5(c)]. This difference can be attributed to the fact that only the standard
MFA uses 2-D computational refocusing, whereas the others use 1-D refocusing.
This is discussed in detail in Section 5.3. In the deep regions (indicated by
the green boxes), the B-MFA and MFA images [Fig. 5(h) and (g), respectively]
show particles with higher contrast than the single acquisition and SFA images
[Fig. 5(e) and (f)], as indicated by the yellow arrows. Inside the glass plate
region, the B-MFA and MFA images [Fig. 5(h) and (g), respectively] have the
lowest noise intensity (indicated by the blue arrows), whereas the single
acquisition and SFA images [Fig. 5(e) and (f)] have the highest and
intermediate noise intensities, respectively. This may indicate that standard
OCT measurement noise has been mitigated in the SFA image because of the
complex averaging, while both the measurement noise and MS signal are
mitigated in the MFA and B-MFA images.

This noise and MS suppression are more clearly and quantitatively shown in the
averaged intensity depth profiles in Fig. 5(i). Here, the central 216 A-lines
[indicated by the orange brace in Fig. 5(a)] were averaged. The black arrow
indicates the surface of the scattering layer. In the superficial region of
the scattering layer, the single acquisition (blue), SFA (grey) and B-MFA
(orange) curves are similar to each other. In the deeper regions (the purple-
background region in the plot), the B-MFA and MFA curves show intensities that
are lower than those of the single acquisition and SFA curves by around 2 to 3
dB. The glass plate region is indicated by light blue background in the plot.
Near the superficial depth in the glass (indicated by the blue brace), the
single acquisition image (blue line) yields the highest intensity. The SFA
(grey line) shows lower intensity than the single acquisition (2.02-dB
reduction in average). This could be attributed to the reduction in
measurement noise caused by the complex averaging. The B-MFA (orange line)
shows a further reduction (2.07 dB with respect to SFA and 4.09 dB with
respect to the single acquisition in average). This may indicate additional
suppression of the MS signal. The MFA (green line) shows the lowest signal
values, i.e., the best reductions in measurement noise and MS-signal
suppression (0.58 dB with respect to B-MFA and 4.66 dB with respect to single
acquisition in average).

###  4.2 In vivo small fish sample

Figure 6: The intensity cross-sections and en-face images of a small fish
sample including single acquisition (a, m), SFA (b, n), MFA (c, o), and B-MFA
(d, p) images. The yellow dashed line indicates the depth of the en-face
images. (e–h) and (i–l) are the enlarged images of the orange and blue boxed
regions, respectively. The dark-blue and red boxes indicate the regions used
to compute the signal-to-signal ratio (SSR). The braces, labeled (1), (2), and
(3), represent the depth regions used to compute the sharpness metrics. The
scale bar indicates 300 $\mu$m. Cross-sectional images of other fish are shown
in the Supplementary Material (Fig. S1-S3).

Figure 6 shows the intensity cross-sections of one of the ten fish samples.
The images are, from top to bottom, single acquisition, SFA, MFA, and B-MFA
images, respectively. Enlarged images of the blue and orange regions are shown
to the right of the cross-sectional images. Several anatomic features, such as
a layered hyperscattering structure in the muscle region (enlarged in the blue
dashed boxes) and a dark region surrounded by hyperscattering, which may
indicate a notochord (enlarged in orange dashed boxes) are visible in all
images, but these features are most clearly visualized in the B-MFA image.

Figure 6(m–p) shows the en-face images at a deep depth indicated by the yellow
dashed line in Fig. 6(a). The B-MFA image [Fig. 6(p)] exhibits a lower signal
intensity but higher contrast than the single acquisition and SFA images [Fig.
6(m) and (n), respectively]. These findings may suggest that the B-MFA method
reduces the MS signal and improves the image contrast. We also notice that the
en-face MFA image [Fig. 6(o)] shows reduced signal intensity but does not show
improved contrast. We suspect that this reduced intensity in the MFA image is
not fully due to the MS reduction, but is also due to signal washout caused by
the motion of the sample. Details are discussed in Section 5.4.

Table 1: Information entropy of the en-face slab projections of the small fish. The depths of (1)–(3) correspond to the depths indicated in Fig. 6(a) (braces). For all depths, B-MFA shows the smallest entropy, which indicates the sharpest image.  Depth (mm) | Information entropy |
---|---|---
Single acquisition |  SFA |  Standard MFA |  B-MFA
(1) 1.05-1.12 | 4.62 | 4.60 | 4.57 | 4.56
(2) 1.41-1.48 | 4.18 | 4.16 | 4.08 | 4.06
(3) 1.77-1.85 | 4.52 | 4.42 | 4.39 | 4.29

To quantitatively compare the sharpness of the images, we computed the
information entropy of the en-face slab projections. The projections were
computed at three depth regions indicated by the pink braces in Fig. 6(a). The
computed information entropy values are summarized in Table 1. At all three
depth regions, the B-MFA shows the smallest information entropy. Namely, B-MFA
provides the sharpest image of the four methods.

To quantitatively evaluate the image contrast, we computed the signal-to-
signal ratio (SSR). SSR is defined as the mean intensity ratio between two
manually selected ROIs, where one ROI was chosen to include a high-scattering-
intensity structure [dark blue box in Fig. 6(a)] and the other was selected in
a low scattering region [red box in Fig. 6(a)] in a cross-sectional image. The
size of each ROI was 35 pixels $\times$ 16 pixels (205.1 $\mu$m $\times$ 115.8
$\mu$m). Examples of the SSRs are shown in Fig. 6(a–d).

The SSRs of the SFA, B-MFA, and MFA images were compared with that of the
single acquisition image by computing the SSR enhancement (SSRE), which is
defined as the difference in SSR from that of the single acquisition image.
For the images presented in Fig. 6, the SSREs were 0.42 dB (SFA), 1.37 dB
(B-MFA), and 0.36 dB (MFA). Namely, of the four images, the B-MFA image
provides the largest SSRE.

Figure 7: Box plot of the SSR enhancement (SSRE) with respect to the single
acquisition images of ten fish samples . The horizontal labels represent the
methods. The top and bottom of each box indicate the upper and lower
quartiles, respectively. The center lines indicate the medians. Paired t-tests
revealed that B-MFA shows a significantly higher SSER than SFA and MFA (p =
0.0008 and 0.0009, respectively). ** represents the statistical significance
of p < 0.01, and ns stands for “non-significant.”

The SSREs of all ten fish were computed using ROIs that include the same
anatomical structures for all fish and plotted in Fig. 7. The B-MFA achieved
the best mean SSRE of 1.82 dB, which is significantly larger than that of SFA
(SSRE = 0.72 dB, p = 0.0008) and MFA (SSRE = -0.52 dB, p = 0.0009). The
statistical comparison was done using paired t-tests.

We note that the MFA method showed the largest interquartile range. This is
because the MFA is more susceptible to sample motion, and the motions of the
in vivo fish samples highly varied from case to case. This susceptibility of
MFA to the sample motion indicates that the B-MFA might be the best method for
in vivo small fish samples.

The intensity cross-sectional image of the other nine fish are presented in
the Supplementary Material (Fig. S1-S3)

##  5 Discussion

###  5.1 Scan protocol optimization

Measurement protocol |  $\Delta z$ |  N |  $\mathrm{D}=(\mathrm{N}-1)\times\Delta z$
---|---|---|---
1 | 1 DOF (0.36 mm) | 1, 2, 3, 4 | 0, 0.36, 0.72, 1.08
2 | 1/2 DOF (0.18 mm) | 1, 2, 3, $\cdots$, 7 | 0, 0.18, 0.36, $\cdots$, 1.08
3 | 1/3 DOF (0.12 mm) | 1, 2, 3, $\cdots$, 10 | 0, 0.12, 0.24, $\cdots$, 1.08
4 | 1/4 DOF (0.09 mm) | 1, 2, 3, $\cdots$, 13 | 0, 0.09, 0.18, $\cdots$, 1.08
5 | 1/6 DOF (0.06 mm) | 1, 2, 3, $\cdots$, 15 | 0, 0.06, 0.12, $\cdots$, 0.84
Table 2: Summary of B-MFA measurement protocols used for the scan-protocol
optimization. We used five focus shifting steps ($\Delta z$) and several
values for the number of frames to be averaged (N). The total focus shifting
distance (D) was defined from $\Delta z$ and N.

To determine the optimal measurement protocol for B-MFA, five focus shifting
steps ($\Delta z$) and several values for the number of total frames per
B-scan (N) were examined. The results for the focus shifting step, total frame
number, and total focus shifting distance
$\mathrm{D}=(\mathrm{N}-1)\times\Delta z$ are summarized in Table 2. A
scattering phantom (Section 3.3) was measured using all protocols.

Figure 8:  Schematic illustration for the scatterer selection. Five scatterers
(red boxes) and a superficial region in the glass plate (dashed light-blue
boxes) were selected to compute the signal-to-background ratio (SBR). The
former (red boxes) is used to compute the signal level, whereas the latter
(light-blue boxes) is used to compute the background level.

To evaluate the image contrast, the signal-to-background ratio (SBR) was
computed. Here the “signal” was defined as the mean signal intensity of five
manually selected particles in a scattering region of the phantom. The
particles were selected at a depth 10 pixels below the top surface depth of
the glass plate as schematically indicated by the pink dashed line in Fig.
8(a), and each of the five particles were selected from different cross-
sectional image in a volume. Each scatterer was cropped by a 3 $\times$ 3
pixels (17.6 $\times$ 21.7 $\mu$m) window in a cross-sectional image, as
indicated by the red boxes in Fig. 8(b), and the mean intensity of this window
was used as the intensity of that scatterer. The background was defined as the
mean signal intensity of an ROI located at the same depth as the particles but
in the glass plate, i.e., a region without scattering. The ROI extends over 60
pixel $\times$ 3 pixel (351.5 $\mu$m $\times$ 21.7 $\mu$m, lateral times
depth), and is indicated by the dashed light blue box in Fig. 8(b). Because
there should be no scattering in the glass, the background intensity is used
as a measure of the MS signal.

Figure 9: SBRs obtained from several B-MFA measurement protocols. Each symbol
in the plot represents a focus shifting step ($\Delta z$), and the horizontal
axis represents the total focus shifting distance (D). The highest SBR was
obtained with D = 0.72 mm ($2\times\mathrm{DOF}$).

The SBR of each protocol is shown in Fig. 9 where each symbol corresponds to
each focus shifting step $\Delta z$. The plot demonstrates that the SBR is
highest at $\mathrm{D}=0.72$ mm, which is twice the DOF. At this $\mathrm{D}$,
the SBRs are 28.74, 28.76, 28.63, 28.52, and 27.54 dB for $\Delta z$ = 1/6,
1/4, 1/3, 1/2, and 1 $\times$ DOF, respectively. Here, the numbers of complex
averaged frames (N) of each protocol are 13, 9, 7, 5, and 3, respectively.
Among these five protocols, the first four give similarly high SBR, and they
can be the candidates for the optimal protocol.

Because B-MFA is used for in vivo measurements, a shorter acquisition time is
preferable. Therefore, we have used the protocol of ($\Delta z$ = 1/2 DOF, N =
5, $\mathrm{D}=2$ DOF) for the measurements shown in the Section 4.

Note that this protocol was selected to best suit small fish samples. For
other types of samples, it could be worth optimizing the protocol again to
adapt it for the new samples.

###  5.2 Phase stability requirements of MFA and B-MFA

The previously proposed MFA method used a 2-D computational refocusing [34],
in which the complex en-face OCT signal at a depth is two-dimensionally
Fourier-transformed and 2-D quadratic phase is applied in the spatial-
frequency domain. Hence, the phase of the OCT signal should be stable over the
volume or at least over several frames that cover an area larger than the
lateral resolution.

In contrast, B-MFA uses 1-D computational refocusing as described in Section
2.3.1. Hence, the phase stability is required only within a frame.

Because the sample motion can cause significant phase error, the lower
requirement for the phase stability of the B-MFA is an important advantage for
in vivo measurement.

###  5.3 Refocus artifact of B-MFA and conventional MFA in the phantom result

Figure 10: Refocus artifacts in en-face images of the phantom (first two rows)
and a small fish sample (the bottom). The left images were obtained by the
B-MFA method, whereas the right images were obtained by the MFA method.
Because the 1-D computational refocusing used by B-MFA refocuses the imaging
only along the fast-scan direction (horizontal direction of these images), the
structures in the B-MFA images are elongated along the slow-scan direction
(vertical direction in the image). In contrast, MFA image does not show this
effect because it uses 2-D refocusing. In the B-MFA images, the elongation is
less pronounced in the deep depths of the phantom (c), which is because the
physical focus was close to this depth, and hence, the structure was sharp
even without refocusing.

Comparing the B-MFA and MFA cross-sectional images [Fig. 5(d) and (c),
respectively] at the superficial region scattering layer, we notice that the
B-MFA image exhibits more scattering particles than the MFA image. This
difference in the numbers of particle can be attributed to the difference in
the 1-D and 2-D computational refocusing. Because 1-D refocusing refocuses the
image only along the fast-scan direction, the refocused particle signal must
be elongated along the slow-scan (vertical) direction, as shown in the en-face
slice of the same data [Fig. 10(a)]. Hence, the images of the scatterers that
are not really in a particular B-scan smear into the B-scan, and this causes
an artifactual increase in the number of scatterers in that B-scan. On the
other hand, the 2-D refocusing used in MFA isotropically refocuses the image
as shown in the corresponding en-face slice [Fig. 10(b)]. Hence, the
artifactual increase of the scatterer does not occur.

A similar artifact also can be seen in the small fish image shown in Fig.
10(e) and (f), although it is less evident than in the phantom case, because
the tissue microstructure is aligned roughly along the slow-scan direction.

In the other depth of the phantom, which is around 1.5 mm from the surface,
the elongation artifact is negligible [Fig. 10(c) and (d)] because the
physical focus is located near this depth.

To solve this problem in the future, we may be able to adopt a computational
refocusing method[35, 36] for the MFA or B-MFA method that is less susceptible
to phase instability.

###  5.4 Signal reduction of the MFA in the in vivo result

In the in vivo fish measurements, the MFA image [the third row of Fig. 6]
showed lower signal intensity than the B-MFA image [the fourth row of Fig. 6].
In addition, the SSRE of MFA is smaller than that of B-MFA for the in vivo
measurement (fifth paragraph of Section 4.2). This can be attributed to the
signal washout caused by the sample motion, which reduces not only the MS
signal but also the SS signal. These findings also emphasize the advantage of
the B-MFA method for in vivo imaging over the MFA method.

##  6 Conclusion

We proposed the B-MFA method, which suppresses MS signals in in vivo imaging.
The method was validated using phantom and in vivo small fish measurements.
The subjective observation of the images and objective evaluation of SSRE
showed that the B-MFA method improves the image contrast by reducing the MS
signals. In addition, the B-MFA showed superior performance than MFA for in
vivo measurements.

Here, we conclude that the B-MFA method can reduce noise caused by the MS
signal in OCT images and is a better option for in vivo measurement than our
previously proposed MFA method.

## Appendix

##  Appendix A Sample-surface detection

The details of the sample surface detection used in Section 2.4 are as
follows. We first manually select the rough depth region in which the sample
surface is searched, which is typically around 90 pixels (around 652 $\mu$m in
air). Then, we compute the first order derivative of linear OCT intensity
along the depth, where the derivative is defined as the difference between two
neighboring pixels. The surface is defined at the top-most depth where the
derivative is larger than a predefined threshold. The threshold is 50 in our
particular case, but this value is in arbitrary unit and may differ in various
OCT systems.

## Disclosures

Y. Zhu, L. Zhu, Lim, Makita, Guo, Yasuno: Sky Technology (F), Nikon (F), Kao
Corp. (F), Topcon (F), Panasonic (F), Santec (F). L. Zhu is currently employed
by Santec.

## Funding

Core Research for Evolutional Science and Technology (JPMJCR2105); Japan
Society for the Promotion of Science (21H01836, 22K04962, 22KF0058); China
Scholarship Council (201908130130).

## Data, Materials, and Code Availability

Data underlying the results presented in this paper are not publicly available
at this time but may be obtained from the authors upon reasonable request.

## Supplemental document

See Supplement 1 for supporting content.

## References

  * [1] W. Geitzenauer, C. K. Hitzenberger, and U. M. Schmidt-Erfurth, “Retinal optical coherence tomography: past, present and future perspectives,” Br. J. Ophthalmol. 95, 171–177 (2011).
  * [2] A. Geevarghese, G. Wollstein, H. Ishikawa, and J. S. Schuman, “Optical coherence tomography and glaucoma,” Annu. Rev. Vis. Sci. 7, 693–726 (2021).
  * [3] M. Ang, M. Baskaran, R. M. Werkmeister, J. Chua, D. Schmidl, V. Aranha dos Santos, G. Garhöfer, J. S. Mehta, and L. Schmetterer, “Anterior segment optical coherence tomography,” Prog. Retin. Eye Res. 66, 132–156 (2018).
  * [4] N. Venkateswaran, A. Galor, J. Wang, and C. L. Karp, “Optical coherence tomography for ocular surface and corneal diseases: a review,” Eye Vis. 5 (2018).
  * [5] T. Yonetsu, B. E. Bouma, K. Kato, J. G. Fujimoto, and I.-K. Jang, “Optical coherence tomography–15 years in cardiology–,” Circulation 77, 1933–1940 (2013).
  * [6] L. Vignali, E. Solinas, and E. Emanuele, “Research and clinical applications of optical coherence tomography in invasive cardiology: a review,” Curr. Cardiol. Rev. 10, 369–376 (2014).
  * [7] Y. Chen, C.-P. Liang, Y. Liu, A. H. Fischer, A. V. Parwani, and L. Pantanowitz, “Review of advanced imaging techniques,” J. Pathol. Inform. 3, 22 (2012).
  * [8] W. Drexler, U. Morgner, F. X. Kärtner, C. Pitris, S. A. Boppart, X. D. Li, E. P. Ippen, and J. G. Fujimoto, “In vivo ultrahigh-resolution optical coherence tomography,” Opt. Lett. 24, 1221–1223 (1999).
  * [9] S. Coquoz, A. Bouwens, P. J. Marchand, J. Extermann, and T. Lasser, “Interferometric synthetic aperture microscopy for extended focus optical coherence microscopy,” Opt. Express 25, 30807–30819 (2017).
  * [10] Y. Yasuno, J.-i. Sugisaka, Y. Sando, Y. Nakamura, S. Makita, M. Itoh, and T. Yatagai, “Non-iterative numerical method for laterally superresolving fourier domain optical coherence tomography,” Opt. Express 14, 1006–1020 (2006).
  * [11] T. S. Ralston, D. L. Marks, P. Scott Carney, and S. A. Boppart, “Interferometric synthetic aperture microscopy,” Nat. Phys. 3, 129–134 (2007).
  * [12] A. Kumar, W. Drexler, and R. A. Leitgeb, “Numerical focusing methods for full field OCT: a comparison based on a common signal model,” Opt. Express 22, 16061–16078 (2014).
  * [13] J. P. Rolland, P. Meemon, S. Murali, K. P. Thompson, and K. sung Lee, “Gabor-based fusion technique for optical coherence microscopy,” Opt. Express 18, 3632–3642 (2010).
  * [14] C. Yoon, Y. Qi, H. Mestre, C. Canavesi, O. J. Marola, A. Cogliati, M. Nedergaard, R. T. Libby, and J. P. Rolland, “Gabor domain optical coherence microscopy combined with laser scanning confocal fluorescence microscopy,” Biomed. Opt. Express 10, 6242–6257 (2019).
  * [15] A. Unterhuber, B. Považay, B. Hermann, H. Sattmann, A. Chavez-Pirson, and W. Drexler, “In vivo retinal optical coherence tomography at 1040 nm-enhanced penetration into the choroid,” Opt. Express 13, 3252–3258 (2005).
  * [16] B. Považay, K. Bizheva, B. Hermann, A. Unterhuber, H. Sattmann, A. Fercher, W. Drexler, C. Schubert, P. Ahnelt, M. Mei, R. Holzwarth, W. J. Wadsworth, J. Knight, and P. S. J. Russel, “Enhanced visualization of choroidal vessels using ultrahigh resolution ophthalmic OCT at 1050 nm,” Opt. Express 11, 1980–1986 (2003).
  * [17] E. C. W. Lee, J. F. de Boer, M. Mujat, H. Lim, and S. H. Yun, “In vivo optical frequency domain imaging of human retina and choroid,” Opt. Express 14, 4403–4411 (2006).
  * [18] Y. Yasuno, Y. Hong, S. Makita, M. Yamanari, M. Akiba, M. Miura, and T. Yatagai, “In vivo high-contrast imaging of deep posterior eye by 1-$\mu$m swept source optical coherence tomography and scattering optical coherence angiography,” Opt. Express 15, 6121–6139 (2007).
  * [19] Y. Li, J. Jing, E. Heidari, J. Zhu, Y. Qu, and Z. Chen, “Intravascular optical coherence tomography for characterization of atherosclerosis with a 1.7 micron swept-source laser,” Sci. Rep. 7, 14525 (2017).
  * [20] J. Zhu, H. R. Freitas, I. Maezawa, L.-w. Jin, and V. J. Srinivasan, “1700 nm optical coherence microscopy enables minimally invasive, label-free, in vivo optical biopsy deep in the mouse brain,” Light Sci. Appl 10, 145 (2021).
  * [21] G. Yao and L. V. Wang, “Monte carlo simulation of an optical coherence tomography signal in homogeneous turbid media,” Phys Med Biol 44, 2307 (1999).
  * [22] R. K. Wang, “Signal degradation by multiple scattering in optical coherence tomography of dense tissue: a monte carlo study towards optical clearing of biotissues,” Phys. Med. Biol. 47, 2281 (2002).
  * [23] L. Zhu, S. Makita, J. Tamaoki, Y. Zhu, P. Mukherjee, Y. Lim, M. Kobayashi, and Y. Yasuno, “Polarization-artifact reduction and accuracy improvement of jones-matrix polarization-sensitive optical coherence tomography by multi-focus-averaging based multiple scattering reduction,” Biomed. Opt. Express 15, 256–276 (2024).
  * [24] A. Badon, D. Li, G. Lerosey, A. C. Boccara, M. Fink, and A. Aubry, “Smart optical coherence tomography for ultra-deep imaging through highly scattering media,” Sci. Adv. 2, e1600370 (2016).
  * [25] D. Borycki, M. Hamkało, M. Nowakowski, M. Szkulmowski, and M. Wojtkowski, “Spatiotemporal optical coherence (STOC) manipulation suppresses coherent cross-talk in full-field swept-source optical coherence tomography,” Biomed. Opt. Express 10, 2032–2054 (2019).
  * [26] M. Wojtkowski, P. Stremplewski, E. Auksorius, and D. Borycki, “Spatio-temporal optical coherence imaging – a new tool for in vivo microscopy,” Photonics Lett. Pol. 11, 44–49 (2019).
  * [27] E. Auksorius, D. Borycki, P. Wegrzyn, B. L. Sikorski, K. Lizewski, I. Zickiene, M. Rapolu, K. Adomavicius, S. Tomczewski, and M. Wojtkowski, “Spatio-temporal optical coherence tomography provides full thickness imaging of the chorioretinal complex,” iScience 25, 105513 (2022).
  * [28] S. Liu, M. R. E. Lamont, J. A. Mulligan, and S. G. Adie, “Aberration-diverse optical coherence tomography for suppression of multiple scattering and speckle,” Biomed. Opt. Express 9, 4919–4935 (2018).
  * [29] S. G. Adie, B. W. Graf, A. Ahmad, P. S. Carney, and S. A. Boppart, “Computational adaptive optics for broadband optical interferometric tomography of biological tissue,” Proc. Natl. Acad. Sci. 109, 7175–7180 (2012).
  * [30] L. Zhu, S. Makita, J. Tamaoki, A. Lichtenegger, Y. Lim, Y. Zhu, M. Kobayashi, and Y. Yasuno, “Multi-focus averaging for multiple scattering suppression in optical coherence tomography,” Biomed. Opt. Express 14, 4828–4844 (2023).
  * [31] Y. Yasuno, “Multi-contrast jones-matrix optical coherence tomography—the concept, principle, implementation, and applications,” IEEE J. Sel. Top. Quantum Electron. 29, 1–18 (2023).
  * [32] A. Miyazawa, S. Makita, E. Li, K. Yamazaki, M. Kobayashi, S. Sakai, and Y. Yasuno, “Polarization-sensitive optical coherence elastography,” Biomed. Opt. Express 10, 5162–5181 (2019).
  * [33] E. Li, S. Makita, Y.-J. Hong, D. Kasaragod, and Y. Yasuno, “Three-dimensional multi-contrast imaging of in vivo human skin by jones matrix optical coherence tomography,” Biomed. Opt. Express 8, 1290–1305 (2017).
  * [34] L. Zhu, S. Makita, D. Oida, A. Miyazawa, K. Oikawa, P. Mukherjee, A. Lichtenegger, M. Distel, and Y. Yasuno, “Computational refocusing of jones matrix polarization-sensitive optical coherence tomography and investigation of defocus-induced polarization artifacts,” Biomed. Opt. Express 13, 2975–2994 (2022).
  * [35] S. Ruiz-Lopera, R. Restrepo, C. Cuartas-Vélez, B. E. Bouma, and N. Uribe-Patarroyo, “Computational adaptive optics in phase-unstable optical coherence tomography,” Opt. Lett. 45, 5982–5985 (2020).
  * [36] S. Ruiz-Lopera, R. Restrepo, T. M. Cannon, M. Villiger, B. E. Bouma, and N. Uribe-Patarroyo, “Computational refocusing in phase-unstable polarization-sensitive optical coherence tomography,” Opt. Lett. 48, 4765–4768 (2023).

## Supplementary

This file supplements Section 4.2 by showing the intensity cross-sectional
images of the other medaka fish samples (samples 2 to 10) measured for
validation of MS reduction by the B-scan-wise-multi-focus averaging (B-MFA)
method. All of the results show that several anatomic features, such as a
hyper-scattering layer structure and the boundary of hollow structure are
better visible in the B-MFA image than single acquisition, SFA, and
conventional MFA.

Figure 11:  The intensity cross-sectional mages of the medaka (samples 2–4).
(a1-a3) single acquisition, (b1-b3) SFA, (c1-c3) conventional MFA, (d1-d3)
B-MFA image. The blue and red boxes indicate the region used to compute the
signal-to-signal ratio (SSR).  Figure 12: The intensity cross-sectional image
of medaka (samples 5–7). The image types and their order are identical to
those of Fig. 11.  Figure 13: The intensity cross-sectional image of medaka
(samples 8–10). The image types and their order are identical to those of Fig.
11.


#  LoFT: Enhancing Faithfulness and Diversity for Table-to-Text
Generation via Logic Form Control

Yilun Zhao 1 Zhenting Qi∗2 Linyong Nan1
Lorenzo Jaime Yu Flores1 Dragomir Radev1
1Yale University 2 Zhejiang University
<EMAIL_ADDRESS><EMAIL_ADDRESS>Equal Contributions.

###### Abstract

Logical Table-to-Text (LT2T) generation is tasked with generating logically
faithful sentences from tables. There currently exists two challenges in the
field: 1) _Faithfulness_ : how to generate sentences that are factually
correct given the table content; 2) _Diversity_ : how to generate multiple
sentences that offer different perspectives on the table. This work proposes
LoFT, which utilizes logic forms as fact verifiers and content planners to
control LT2T generation. Experimental results on the LogicNLG dataset
demonstrate that LoFT is the first model that addresses unfaithfulness and
lack of diversity issues simultaneously. Our code is publicly available at
https://github.com/Yale-LILY/LoFT.

##  1 Introduction

Figure 1: An example of logical table-to-text generation. (a) Statements
generated by previous models Nan et al. (2022): the generation suffers from 1)
_Lack of diversity_ , as three of the generated statements are focused on the
same table regions (i.e., “Hale Irwin” and “Gil Morgan”), and three of them
use the similar reasoning operations (i.e., comparative); 2) _Unfaithfulness_
, as one of the generated statements is factually incorrect given the table
content. (b) Statements generated by LoFT: By utilizing logic forms to
_control_ the generation, our method can generate multiple factually correct
sentences that each use a different reasoning operation to offer various
perspectives on the tabular data.

Table-to-Text (T2T) generation aims to produce natural language descriptions
from structured tables. A statement generated from tabular data can be
inferred based on different levels of information (e.g., value of a specific
cell, logical operation result across multiple cells). Although current T2T
models Lebret et al. (2016); Wiseman et al. (2017); Puduppully et al. (2019);
Parikh et al. (2020) have shown remarkable progress in fluency and coherence,
they mainly focus on surface-level realizations without much logical
inference.

Recently, Chen et al. (2020a) proposed LogicNLG, which is tasked with
generating textual descriptions that require logical reasoning over tabular
data (i.e., LT2T generation). LT2T generation is challenging as it requires a
model to learn the logical inference knowledge from table-text pairs and
generate multiple _factually correct_ sentences. Another challenge for LT2T
generation is the _diversity_ of generated text. Natural Language Generation
(NLG) encourages the diverse output of statements over a single input, as it
provides various perspectives on the data and offers users more choices. In
LT2T generation, requirements for diversity naturally emerge from the need to
apply different logical operations to extract different levels of table
information. However, current methods Chen et al. (2021); Nan et al. (2022);
Liu et al. (2022a); Zhao et al. (2022b) that address issues of unfaithfulness
have overlooked the importance of diversity. As shown in Figure 1, multiple
statements generated using current methods Nan et al. (2022) might only cover
information from the same table region or logical operation. Such issues
related to lack of diversity could limit the deployment of LT2T models in the
real world.

In this work, we attribute _unfaithfulness_ and lack of _diversity_ to the
absence of _controllability_ over generation. Specifically, due to the large
number of combinations of different logical operations and table regions, the
space of factually correct statements is exponentially large. However,
LogicNLG uses the whole table as the input, without providing annotations
related to any other explicit control attribute. As a result, it is hard and
uncontrollable for neural models to decide a favorable choice of logical
selections solely based on the table input. We believe such
_uncontrollability_ leads to unfaithfulness and lack of diversity issues.

This work proposes LoFT, a framework that utilizes logic forms as mediators to
enable _controllable_ LT2T generation. Logic forms Chen et al. (2020d, b) are
widely used to retrieve evidence and explain the reasons behind table fact
verification Yang et al. (2020); Yang and Zhu (2021); Ou and Liu (2022). In
this work, logic forms are used as: 1) fact verifiers to ensure the factual
correctness of each generated sentence; and 2) content planners to control
which logical operation and table region to use during the generation.
Experimental results show that LoFT surpasses previous methods in faithfulness
and diversity simultaneously.

##  2 Related Work

#### Logical Table-to-Text (LT2T) Generation

LogicNLG Chen et al. (2020a) is tasked with generating logically faithful
sentences from tables. To improve the faithfulness of generated statements,
Nan et al. (2022) trained a system both as a generator and a faithfulness
discriminator with additional replacement detection and unlikelihood learning
tasks. Liu et al. (2022a) pre-trained a model on a synthetic corpus of table-
to-logic-form generation. Zhao et al. (2022b) demonstrated that faithfulness
of LT2T can be improved by pre-training a generative language model over
synthetic Table QA examples. However, these methods overlook the importance of
diversity in T2T generation, and might generate multiple statements that cover
the same table regions or reasoning operations. Previous methods in NLG
proposed to improve diversity by modifying the decoding techniques Li et al.
(2016). However, these approaches degrade faithfulness as measured against
baselines Perlitz et al. (2022). To enable controllable generation and improve
diversity, Perlitz et al. (2022) used logical types of statements as a
control. However, such methods still suffer from problems related to
unfaithfulness, and may generate statements covering limited table regions.
This work proposes to leverage the logic form as a fact checker and content
planner to control LT2T generation, which tackles the challenges about
faithfulness and diversity at the same time.

#### Table Fact Verification via Logic Form

Logic forms are widely used in Table Fact Verification Chen et al. (2020b).
Specifically, given an input statement, the model Yang et al. (2020); Yang and
Zhu (2021); Ou and Liu (2022) will first translate it into logic form. Then
the logic form will be executed over the table, and return true/false as the
entailment label for a given statement. While several works Chen et al.
(2020d); Shu et al. (2021); Liu et al. (2021) focused on generating fluent
statements from logic forms, the utilization of logic forms to benefit LT2T
generation is still unexplored.

(a) LoFT training stage.

(b) LoFT inference stage.

Figure 2: The illustration of LoFT. (a) During the training stage, the SASP
model is first applied to translate each statement in the LogicNLG training
set into the logic form. Then LoFT is trained to generate the reference
statement given the translated logic form and serialized table data. (b)
During the inference stage, given each table, the logic form synthesis
pipeline was first applied to synthesize candidate logic forms that cover
different table regions and logical operations. LoFT is applied to generate
statements for each candidate logic form. Then a statement verifier is used to
filter out those potentially unfaithful statements. As a result, LoFT can
generate a diverse set of faithful statements covering different table regions
and reasoning operations. For each table in the LogicNLG test set, we randomly
sampled five candidate statements for evaluation.

##  3 LoFT

This section first introduces the logic form utilized, and then delves into
the training and inference process of LoFT. We also explain how the use of
logic forms can enhance both faithfulness and text-diversity in LT2T
generation.

###  3.1 Logic Form Implementation

Logic forms are widely used to retrieve evidence and explain the reasons
behind table fact verification. We use the same implementation as Chen et al.
(2020d), which covers 8 types of the most common logical operations (e.g.,
count, aggregation) to describe a structured table. Each logical operation
corresponds to several Python-based functions. For example, the definition of
function all_greater(view, header, value) under “majority” category is:
checking whether all the values under header column are greater than value,
with the scope (i.e., view) of all or a subset of table rows. The complete
list of logical operation types and corresponding function definitions are
shown in Table 4 in Appendix.

###  3.2 LoFT Training

#### Training Task Formulation

Given the serialized tabular data with selected columns as $T$, the training
objective of LoFT is to generate a sentence $\bm{y}=(y_{1},y_{2},\dots,y_{n})$
that is both fluent and faithful, with the translated logic form $l$ as
control.

$\bm{y}=\mathrm{argmax}\prod_{i=1}^{n}P(y_{i}|y_{<i},T,\,l;\,\theta)$ (1)

where $\theta$ denotes the parameters of a seq2seq LM.

#### Training Dataset Collection

Since the LogicNLG dataset does not contain logic form annotations, we had to
augment each statement in the training set with its corresponding logic forms.
To construct {statement, logic form} parallel data for the LogicNLG training
set, we adapted SASP Ou and Liu (2022), the state-of-the-art model for TabFact
dataset, which leverages structure-aware semantic parsing over tables to
translate the given statement into logic form. In this work, given an example
in the LogicNLG training set, SASP was applied to generate its logic form,
resulting in a total of 15,637 examples for LoFT training.

###  3.3 LoFT Inference

During the inference stage, for each given table, we first applied the logic
form synthesis pipeline to synthesize multiple candidate logic forms Liu et
al. (2022a). For each of these logic forms, the system generates its
corresponding statement. The faithfulness of these statements were further
checked by a verifier.

#### Logic Form Synthesis Pipeline

To synthesize a candidate set of logic forms paired with each supporting
table, we applied a similar logic form synthesis pipeline as Liu et al.
(2022a).

We extracted templates of logic forms from the collected LoFT training
dataset. Specifically, we categorized functions with similar definitions
(e.g., max/min, greater/less) into smaller groups to obtain a more abstract
template. Each function category corresponded to one unique table reasoning
skill. For each template, we masked specific entities in the logic forms as
typed placeholders (i.e., col to denote a column header, obj to denote an
object). Finally, we obtained 45 different templates, covering 8 table logical
operations. Table 4 shows the complete list of reasoning operations and
corresponding function definitions.

Given the table and each set of selected columns, the pipeline would
synthesize a total of 20 candidate logic forms whose execution result over the
table is True. To generate a candidate logic form, the pipeline first sampled
a logic form using a weighted-sampling technique with the weight equal to the
template distribution in the LoFT training dataset (Section 3.2). The weighted
sampling is to ensure that the generated candidate logic forms follow a
similar distribution as LogicNLG. To instantiate the sampled template, a
bottom-up sampling strategy is adopted to fill in each placeholder of the
template and finally generate the logic form.

#### Statement Generation & Verification

Through the logic form synthesis pipeline, we obtained a large number of
candidate logic forms. For each logic form, we used LoFT to generate the
corresponding statement. The candidate statements might still contain some
factually incorrectness, thus we applied an NLI-based verifier to filter out
those potentially unfaithful generations. Specifically, we used the TabFact
Chen et al. (2020b) dataset to train a classifier, which adopts RoBERTa-base
as the backbone. We fed each generated statement and its corresponding table
into the classifier, and only kept those statements that were predicted as
entailed. Then we randomly sampled five statements as the output for each
table in LogicNLG.

###  3.4 Enhancing LT2T via Logic Form Control

This subsection provides two perspectives to explain why logic forms can help
improve both faithfulness and diversity of LT2T generation.

#### Logic Form as Content Planner

Logic forms pass column or cell values as arguments, guiding the model to
focus on relevant table regions. The function category of the logic form, such
as count, helps the model better organize logical-level content planning.

#### Logic Form as Fact Verifier

Logic forms are defined with unambiguous semantics, hence are reliable
mediators to achieve faithful and controllable logical generations. During the
inference stage, we synthesize candidate logic forms with 100% execution
correctness. The sampled logic form serves as a fact verifier and conveys
accurate logical-level facts for controllable LT2T generation.

Model | Surface-level | Diversity-level | Faithfulness-level
---|---|---|---
BLEU-1/2/3$\uparrow$ | Distinct-2$\uparrow$ | s-BLUE-4$\downarrow$ | SP-Acc$\uparrow$ | NLI-Acc$\uparrow$ | TAPEX-Acc$\uparrow$
GPT2-TabGen Chen et al. (2020a) | 48.8/27.1/12.6 | 59.0 | 55.3 | 42.1 | 68.7 | 45.0
GPT2-C2F Chen et al. (2020a) | 46.6/26.8/13.3 | 60.3 | 52.8 | 42.7 | 72.2 | 44.1
DCVED∗ Chen et al. (2021) | 49.5/28.6/15.3 | – | – | 43.9 | 76.9 | –
DEVTC‡ Perlitz et al. (2022) | 51.3/30.6/16.3 | 73.7 | 21.3 | 44.3 | 77.9 | 55.6
R2D2 Nan et al. (2022) | 51.8/32.4/18.6 | 60.1 | 51.5 | 50.8 | 85.6 | 60.2
LoFT | 48.1/27.7/14.9 | 79.5 | 17.7 | 57.7 | 86.9 | 61.8

Table 1: Performance on the LogicNLG test set. ${\ddagger}$: results from our
own implementation; $*$: code not released and we used the results reported in
original papers. LoFT achieves great improvement on faithfulness and
diversity.

Diversity | DEVTC | R2D2 | LoFT
---|---|---|---
Criteria | Best$\uparrow$ | Worst$\downarrow$ | Best$\uparrow$ | Worst$\downarrow$ | Best$\uparrow$ | Worst$\downarrow$
Table Coverage | 8 | 16 | 5 | 20 | 29 | 5
Reasoning Op | 19 | 1 | 2 | 37 | 24 | 2

Table 2: Number of times the system was selected as best or worst by majority vote (including ties). LoFT outperforms other baselines in terms of diversity for both table coverage and reasoning operations. Model | Faithfulness $\uparrow$ | Fluency $\uparrow$
---|---|---
Agreement / $\kappa$ | Agreement / $\kappa$
DEVTC | 63.5 / 0.69 | 86.5 / 0.80
R2D2 | 71.5 / 0.73 |  90.0 / 0.84
LoFT |  75.0 / 0.76 | 88.0 / 0.81
Table 3: Human evaluation results on the criteria of faithfulness and fluency,
with the total agreement by Fleiss’ Kappa ($\kappa$) Fleiss (1971). LoFT has
the best performance in terms of faithfulness, while achieving comparable
performance in fluency.

##  4 Experimental Setup

We next discuss the evaluation metrics, baselines, and implementation details
for the experiments.

###  4.1 Evaluation Metrics

We applied various automated evaluation metrics at different levels to
evaluate the model performance from multiple perspectives.

#### Surface-level

Following Chen et al. (2020a), we used BLEU-1/2/3 to measure the consistency
of generated statements with the reference.

#### Diversity-level

We used Distinct-$n$ Li et al. (2016) and self-BLEU-$n$ Zhu et al. (2018) to
measure the diversity of five generated statements for each table.
Distinct-$n$ is defined as the total number of distinct $n$-grams divided by
the total number of tokens in the five generated statements; Self-BLEU-$n$
measures the average $n$-gram BLEU score between generated statements. We
measured Distinct-$2$ and Self-BLEU-$4$ in our experiment.

#### Faithfulness-level

Similar as the previous works Chen et al. (2020a); Nan et al. (2022); Liu et
al. (2022a), we used a parsing-based evaluation metric (i.e., SP-Acc) and two
NLI-based evaluation metrics (i.e., NLI-Acc and TAPEX-Acc) to measure the
faithfulness of generation. SP-Acc directly extracts the meaning
representation from the generated sentence and executes it against the table
to verify the correctness. NLI-Acc and TAPEX-Acc use TableBERT Chen et al.
(2020b) and TAPEX Liu et al. (2022b) respectively as their backbones, and were
finetuned on the TabFact dataset Chen et al. (2020b). Liu et al. (2022a) found
that NLI-Acc is overly positive about the predictions, while TAPEX-Acc is more
reliable to evaluate the faithfulness of generated sentences.

###  4.2 Baseline Systems

We implemented following baseline systems for the performance comparison:
GPT2-TabGen Chen et al. (2020a) directly fine-tunes GPT-2 over the LogicNLG
dataset; GPT2-C2F Chen et al. (2020a) first produces a template which
determines the global logical structure, and then generates the statement
conditioned on the template; DCVED Chen et al. (2021) applies a de-confounded
variational encoder-decoder to reduce the spurious correlations during LT2T
generation training; DEVTC Perlitz et al. (2022) utilized reasoning operation
types as an explicit control to increase the diversity of LT2T generation; and
R2D2 Nan et al. (2022) trains a generative language model both as a generator
and a faithfulness discriminator with additional replacement detection and
unlikelihood learning tasks, to enhance the faithfulness of LT2T generation.

###  4.3 Implementation Details

Following Shu et al. (2021), we converted each logic form into a more human-
readable form for both LoFT training and inference data. LoFT was implemented
using fairseq library Ott et al. (2019), with BART-Large Lewis et al. (2020)
as the backbones. All experiments were conducted on an 8 NVIDIA RTX-A5000 24GB
cluster. Both LoFT and the statement verifier was trained for 5,000 steps with
a batch size of 128. The best checkpoints were selected by the validation
loss.

##  5 Experimental Results

This section discusses automated and human evaluation results of different
systems.

###  5.1 Main Results

Table 1 presents the results on LogicNLG. LoFT outperforms all the baselines
on the criteria of diversity and faithfulness, and is the first model that
achieves state-of-the-art results on both faithfulness- and diversity-level.
It is worth noting that in the LogicNLG setting, a generated statement is
allowed to cover a different table region or reasoning operations from the
references, as long as it is fluent and factually correct. However, in such
cases, the reference-based metrics will be low, explaining why the BLEU-1/2/3
scores of LoFT are lower than other models.

###  5.2 Human Evaluation

We conducted the human evaluation with four expert annotators using the
following three criteria: (1) _Faithfulness_ (scoring 0 or 1): if all facts
contained in the generated statement are entailed by the table content; (2)
_Diversity_ (voting the best & worst): if the five generated statements cover
information from different table regions, and use different reasoning
operations; (3) _Fluency_ (scoring 0 or 1): if the five generated statements
are fluent and without any grammar mistakes.

We chose R2D2 Nan et al. (2022) and DEVTC Perlitz et al. (2022) for
comparison, as they achieved best-performance results in faithfulness and
diversity, respectively. We sampled 50 tables from the LogicNLG test set. For
each table, we selected all five generated statements from each model’s
output. To ensure fairness, the model names were hidden to the annotators, and
the display order between three models was randomly shuffled. Human evaluation
results show that LoFT delivers improvements in both faithfulness (Table 3)
and diversity (Table 2), while achieving comparable performance in fluency
(Table 3).

##  6 Conclusions

This work proposes LoFT, which utilizes logic forms as fact verifiers and
content planners to enable controllable LT2T generation. Experimental results
on LogicNLG demonstrate that LoFT delivers a great improvement in both
diversity and faithfulness of LT2T generation.

## Limitations

The first limitation of our approach is that LoFT does not explore long text
generation Moosavi et al. (2021). LoFT only supports the generation of
multiple single sentences. To enable long text generation (i.e., generate a
long paragraph that delivers various perspectives on the table data), a global
content planner Su et al. (2021) needs to be designed to highlight which
candidate sentences should be mentioned and in which order. Additionally, we
believe that LoFT can also be applied to text generation over hybrid context
with both textual and tabular data Chen et al. (2020c); Zhao et al. (2022a);
Nakamura et al. (2022).

The second limitation of our work is that the statement verifier discussed in
Section 3.3 was trained using the same data as NLI-Acc and TAPEX-Acc. This
might bring some bias for NLI-based metrics on faithulness-level evaluation.
In the future, we will exploit a more robust automated evaluation system
Fabbri et al. (2021); Liu et al. (2022c) to comprehensively evaluate the LT2T
model performances from different perspectives.

Moreover, we applied the SASP model Ou and Liu (2022) to convert statements
into logic forms (Section 3.2). Some converted logic forms may be inconsistent
with the original statement. We believe that future work could incorporate the
Logic2Text Chen et al. (2020d) dataset into training data to further improve
the LoFT performance.

## Ethical Consideration

We used the LogicNLG Chen et al. (2020a) dataset for training and inference.
LogicNLG is publicly available under MIT
license111https://opensource.org/licenses/MIT and widely used in NLP research
and industry.

## References

  * Chen et al. (2020a) Wenhu Chen, Jianshu Chen, Yu Su, Zhiyu Chen, and William Yang Wang. 2020a.  Logical natural language generation from open-domain tables.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7929–7942, Online. Association for Computational Linguistics.
  * Chen et al. (2020b) Wenhu Chen, Hongmin Wang, Jianshu Chen, Yunkai Zhang, Hong Wang, Shiyang Li, Xiyou Zhou, and William Yang Wang. 2020b.  Tabfact: A large-scale dataset for table-based fact verification.  In _International Conference on Learning Representations_.
  * Chen et al. (2020c) Wenhu Chen, Hanwen Zha, Zhiyu Chen, Wenhan Xiong, Hong Wang, and William Wang. 2020c.  Hybridqa: A dataset of multi-hop question answering over tabular and textual data.  _Findings of EMNLP 2020_.
  * Chen et al. (2021) Wenqing Chen, Jidong Tian, Yitian Li, Hao He, and Yaohui Jin. 2021.  De-confounded variational encoder-decoder for logical table-to-text generation.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 5532–5542, Online. Association for Computational Linguistics.
  * Chen et al. (2020d) Zhiyu Chen, Wenhu Chen, Hanwen Zha, Xiyou Zhou, Yunkai Zhang, Sairam Sundaresan, and William Yang Wang. 2020d.  Logic2text: High-fidelity natural language generation from logical forms.
  * Fabbri et al. (2021) Alexander R. Fabbri, Wojciech Kryściński, Bryan McCann, Caiming Xiong, Richard Socher, and Dragomir Radev. 2021.  SummEval: Re-evaluating summarization evaluation.  _Transactions of the Association for Computational Linguistics_ , 9:391–409.
  * Fleiss (1971) Joseph L Fleiss. 1971.  Measuring nominal scale agreement among many raters. _Psychological bulletin_ , 76(5):378.
  * Lebret et al. (2016) Rémi Lebret, David Grangier, and Michael Auli. 2016.  Neural text generation from structured data with application to the biography domain.  In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_ , pages 1203–1213, Austin, Texas. Association for Computational Linguistics.
  * Lewis et al. (2020) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020.  BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7871–7880, Online. Association for Computational Linguistics.
  * Li et al. (2016) Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016.  A diversity-promoting objective function for neural conversation models.  In _NAACL 2016_.
  * Liu et al. (2022a) Ao Liu, Haoyu Dong, Naoaki Okazaki, Shi Han, and Dongmei Zhang. 2022a.  PLOG: Table-to-logic pretraining for logical table-to-text generation.  In _EMNLP 2022_.
  * Liu et al. (2021) Ao Liu, Congjian Luo, and Naoaki Okazaki. 2021.  Improving logical-level natural language generation with topic-conditioned data augmentation and logical form generation.  _arXiv preprint arXiv:2112.06240_.
  * Liu et al. (2022b) Qian Liu, Bei Chen, Jiaqi Guo, Morteza Ziyadi, Zeqi Lin, Weizhu Chen, and Jian-Guang Lou. 2022b.  TAPEX: Table pre-training via learning a neural SQL executor.  In _International Conference on Learning Representations_.
  * Liu et al. (2022c) Yixin Liu, Alexander R. Fabbri, Pengfei Liu, Yilun Zhao, Linyong Nan, Ruilin Han, Simeng Han, Shafiq Joty, Chien-Sheng Wu, Caiming Xiong, and Dragomir Radev. 2022c.  Revisiting the gold standard: Grounding summarization evaluation with robust human evaluation.
  * Moosavi et al. (2021) Nafise Sadat Moosavi, Andreas Rücklé, Dan Roth, and Iryna Gurevych. 2021\.  Scigen: a dataset for reasoning-aware text generation from scientific tables.  In _Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2)_.
  * Nakamura et al. (2022) Kai Nakamura, Sharon Levy, Yi-Lin Tuan, Wenhu Chen, and William Yang Wang. 2022\.  HybriDialogue: An information-seeking dialogue dataset grounded on tabular and textual data.  In _Findings of the Association for Computational Linguistics: ACL 2022_ , pages 481–492, Dublin, Ireland. Association for Computational Linguistics.
  * Nan et al. (2022) Linyong Nan, Lorenzo Jaime Flores, Yilun Zhao, Yixin Liu, Luke Benson, Weijin Zou, and Dragomir Radev. 2022.  R2D2: Robust data-to-text with replacement detection.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 6903–6917, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Ott et al. (2019) Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019.  fairseq: A fast, extensible toolkit for sequence modeling.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations)_ , pages 48–53, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Ou and Liu (2022) Suixin Ou and Yongmei Liu. 2022.  Learning to generate programs for table fact verification via structure-aware semantic parsing.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 7624–7638, Dublin, Ireland. Association for Computational Linguistics.
  * Parikh et al. (2020) Ankur Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui, Bhuwan Dhingra, Diyi Yang, and Dipanjan Das. 2020.  ToTTo: A controlled table-to-text generation dataset.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1173–1186, Online. Association for Computational Linguistics.
  * Perlitz et al. (2022) Yotam Perlitz, Liat Ein-Dot, Dafna Sheinwald, Noam Slonim, and Michal Shmueli-Scheuer. 2022.  Diversity enhanced table-to-text generation via type control.  _arXiv preprint arXiv:2205.10938_.
  * Puduppully et al. (2019) Ratish Puduppully, Li Dong, and Mirella Lapata. 2019.  Data-to-text generation with entity modeling.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 2023–2035, Florence, Italy. Association for Computational Linguistics.
  * Shu et al. (2021) Chang Shu, Yusen Zhang, Xiangyu Dong, Peng Shi, Tao Yu, and Rui Zhang. 2021.  Logic-consistency text generation from semantic parses.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 4414–4426, Online. Association for Computational Linguistics.
  * Su et al. (2021) Yixuan Su, David Vandyke, Sihui Wang, Yimai Fang, and Nigel Collier. 2021.  Plan-then-generate: Controlled data-to-text generation via planning.  In _Findings of the Association for Computational Linguistics: EMNLP 2021_ , pages 895–909, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Wiseman et al. (2017) Sam Wiseman, Stuart Shieber, and Alexander Rush. 2017.  Challenges in data-to-document generation.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pages 2253–2263, Copenhagen, Denmark. Association for Computational Linguistics.
  * Yang et al. (2020) Xiaoyu Yang, Feng Nie, Yufei Feng, Quan Liu, Zhigang Chen, and Xiaodan Zhu. 2020\.  Program enhanced fact verification with verbalization and graph attention network.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 7810–7825, Online. Association for Computational Linguistics.
  * Yang and Zhu (2021) Xiaoyu Yang and Xiaodan Zhu. 2021.  Exploring decomposition for table-based fact verification.  In _Findings of the Association for Computational Linguistics: EMNLP 2021_ , pages 1045–1052, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Zhao et al. (2022a) Yilun Zhao, Yunxiang Li, Chenying Li, and Rui Zhang. 2022a.  MultiHiertt: Numerical reasoning over multi hierarchical tabular and textual data.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 6588–6600, Dublin, Ireland. Association for Computational Linguistics.
  * Zhao et al. (2022b) Yilun Zhao, Linyong Nan, Zhenting Qi, Rui Zhang, and Dragomir Radev. 2022b.  ReasTAP: Injecting table reasoning skills during pre-training via synthetic reasoning examples.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 9006–9018, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Zhu et al. (2018) Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. 2018.  Texygen: A benchmarking platform for text generation models.  In _The 41st International ACM SIGIR Conference on Research; Development in Information Retrieval_ , SIGIR ’18, page 1097–1100, New York, NY, USA. Association for Computing Machinery.

##  Appendix A Appendix

Reasoning Op | Function Category | Name | Arguments | Output | Description
---|---|---|---|---|---
Unique | UNIQUE | only | view | bool | returns whether there is exactly one row in the view
Aggregation | AGGREGATION | avg/sum | view, header, string | number | returns the average/sum of the values under the header column
Count | COUNT | count | view | number | returns the number of rows in the view
Ordinal | ORD_ARG | nth_argmax/nth_argmin | view, header string | view | returns the row with the n-th max/min value in header column
ORDINAL | nth_max/nth_min | view, header string | number | returns the n-th max/n-th min of the values under the header column
SUPER_ARG | argmax/argmin | view, header string | view | returns the row with the max/min value in header column
Comparative | COMPARE | eq/not_eq | object, object | bool | returns if the two arguments are equal
round_eq | object, object | bool | returns if the two arguments are roughly equal under certain tolerance
greater/less | object, object | bool | returns if 1st argument is greater/less than 2nd argument
diff | object, object | object | returns the difference between two arguments
Majority | MAJORITY | all_eq/not_eq | view, header string, object | bool | returns whether all the values under the header column are equal/not equal to 3rd argument
all_greater/less | view, header string, object | bool | returns whether all the values under the header column are greater/less than 3rd argument
all_greater_eq/less_eq | view, header string, object | bool | returns whether all the values under the header column are greater/less or equal to 3rd argument
most_eq/not_eq | view, header string, object | bool | returns whether most of the values under the header column are equal/not equal to 3rd argument
most_greater/less | view, header string, object | bool | returns whether most of the values under the header column are greater/less than 3rd argument
most_greater_eq/less_eq | view, header string, object | bool | returns whether most of the values under the header column are greater/less or equal to 3rd argument
Conjunction | FILTER | filter_eq/not_eq | view, header string, object | view | returns the subview whose values under the header column is equal/not equal to 3rd argument
filter_greater/less | view, header string, object | view | returns the subview whose values under the header column is greater/less than 3rd argument
filter_greater_eq /less_eq | view, header string, object | view | returns the subview whose values under the header column is greater/less or equal than 3rd argument
OTHER | filter_all | view, header string | view | returns the view itself for the case of describing the whole table
Other | OTHER | hop | view, header string | object | returns the value under the header column of the row
OTHER | and | bool, bool | bool | returns the boolean operation result of two arguments

Table 4: A complete list of function definitions for the logic forms (Similar
as Chen et al. (2020d)).