11institutetext: Dept. de Astrofísica Molecular, Instituto de Física
Fundamental (IFF-CSIC), C/ Serrano 121, 28006 Madrid, Spain.
11email<EMAIL_ADDRESS>&<EMAIL_ADDRESS>22institutetext:
Observatorio Astronómico Nacional (OAN, IGN), C/ Alfonso XII, 3, 28014,
Madrid, Spain. 33institutetext: Centro de Desarrollos Tecnológicos,
Observatorio de Yebes (IGN), 19141 Yebes, Guadalajara, Spain.

#  CN and CCH derivatives of ethylene and ethane: Confirmation of the
detection of CH3CH2CCH in TMC-1 ††thanks: Based on observations with the Yebes
40m radio telescope (projects 19A003, 20A014, 20D023, 21A011, 21D005, 22A007,
22B029, and 23A024) and the IRAM 30m radio telescope. The 40m radio telescope
at Yebes Observatory is operated by the Spanish Geographic Institute (IGN,
Ministerio de Transportes y Movilidad Sostenible). IRAM is supported by
INSU/CNRS (France), MPG (Germany), and IGN (Spain).

J. Cernicharo 11 B. Tercero 2233 M. Agúndez 11 C. Cabezas 11 R. Fuentetaja 11
N. Marcelino 2233 and P. de Vicente 33

(Received: 07/02/2024; Accepted: 08/03/2024)

We present a study of CH3CH2CCH, CH3CH2CN, CH2CHCCH, and CH2CHCN in TMC-1
using the QUIJOTE1 line survey. We confirm the presence of CH3CH2CCH in TMC-1,
which was previously reported as tentative by our group. From a detailed study
of the ethynyl and cyanide derivatives of CH2CH2 and CH3CH3 in TMC-1, we found
that the CH2CHCCH/CH2CHCN and CH3CH2CCH/CH3CH2CN abundance ratios are
1.5$\pm$0.1 and 4.8$\pm$0.5, respectively. The derived CH2CHCCH/CH3CH2CCH
abundance ratio is 15.3$\pm$0.8, and that of CH2CHCN over CH3CH2CN is
48$\pm$5\. All the single substituted isotopologs of vinyl cyanide have been
detected, and we found that the first and second carbon substitutions in
CH2CHCN provide a 12C/13C ratio in line with that found for other three-carbon
bearing species such as HCCNC and HNCCC. However, the third 13C isotopolog,
CH2CH13CN, presents an increase in its abundance similar to that found for
HCCCN. Finally, we observed eight $b$-type transitions of CH2CHCN, and we find
that their intensity cannot be fitted adopting the dipole moment $\mu_{b}$
derived previously. These transitions involve the same rotational levels as
those of the $a$-type transitions. From their intensity, we obtain
$\mu_{b}$=0.80$\pm$0.03 D, which is found to be in between earlier values
derived in the laboratory using intensity measurements or the Stark effect.
Our chemical model indicates that the abundances of CH3CH2CCH, CH3CH2CN,
CH2CHCCH, and CH2CHCN observed in TMC-1 can be explained in terms of gas-phase
reactions.

###### Key Words.:

molecular data — line: identification — ISM: molecules — ISM: individual
(TMC-1) — astrochemistry

##  1 Introduction

The ultra-sensitive line survey QUIJOTE111Q-band Ultrasensitive Inspection
Journey to the Obscure TMC-1 Environment performed with the Yebes 40m radio
telescope towards the starless cold core TMC-1 (Cernicharo et al. 2021a),
together with an ultra-deep line survey of the carbon-rich star IRC+10216
(Pardo et al. 2022), has recently permitted the unambiguous detection of
nearly 70 new molecules in space (Cernicharo et al. 2021a; Pardo et al. 2021;
Cernicharo et al. 2023a; Cabezas et al. 2023; Cernicharo et al. 2023b, and
references therein). Together with the discoveries of the GOTHAM line survey
on TMC-1 (McGuire et al. 2018), and the IRAM 30m and Yebes 40m survey of
G+0693-0.027 toward the Galactic center (see, e.g., Rivilla et al.  2023), the
number of molecules discovered in space has increased from 200 to around 300
species in the past four years, $\sim$70 of which lie in TMC-1. These results
place strong constraints on the chemical networks and models of interstellar
clouds and on our view on the formation of aromatic rings in these cold
objects (McGuire et al. 2018, 2021; Cernicharo et al. 2021b, 2023c; Agúndez et
al. 2023a).

TMC-1 is known to harbor abundant cyanopolyynes and the unsaturated carbon-
chain radicals CnH and CnN. Even vibrationally excited C6H has been found in
TMC-1 with the QUIJOTE line survey (Cernicharo et al. 2023d). Nevertheless,
the chemistry of this cold core also produces large abundances for more
saturated hydrocarbon species such as CH3CHCH2 (Marcelino et al. 2007), CH3CCH
(see, e.g., Cabezas et al. 2021; Agúndez et al.  2021, and references threin),
CH2CCH (Agúndez et al.  2021, 2022), vinyl acetylene (Cernicharo et al.
2021b), benzyne (Cernicharo et al. 2021a), cyclopentadiene (Cernicharo et al.
2021c), and indene (Cernicharo et al. 2021c; Burkhardt et al.  2021). In
addition, the presence of benzene and naphthalene has been inferred from the
detection of their CN and CCH derivatives (McGuire et al. 2018, 2021; Loru et
al.  2023). Recently, the CN functionalized forms of indene were also found in
the same source by Sita et al.  (2022).

In this Letter, we present a systematic study of the ethynyl and cyanide
derivatives of ethylene (CH2CH2) and ethane (CH3CH3) in TMC-1, confirming the
previous tentative identification of CH3CH2CCH (Cernicharo et al. 2021b). We
also report abundances for the singly 13C, 15N, and D substituted isotopologs
of CH2CHCN.

##  2 Observations

The observational data used in this work are part of QUIJOTE (Cernicharo et
al. 2021a), which is a spectral line survey of TMC-1 in the Q band carried out
with the Yebes 40m telescope at the position $\alpha_{J2000}=4^{\rm h}41^{\rm
m}41.9^{\rm s}$ and
$\delta_{J2000}=+25^{\circ}41^{\prime}27.0^{\prime\prime}$, corresponding to
the cyanopolyyne peak (CP) in TMC-1. The receiver was built within the
Nanocosmos project222https://nanocosmos.iff.csic.es/ and consists of two cold
high-electron mobility transistor amplifiers that cover the 31.0-50.3 GHz band
with horizontal and vertical polarizations. The receiver temperatures achieved
in the 2019 and 2020 runs vary from 22 K at 32 GHz to 42 K at 50 GHz. Some
power adaptation in the down-conversion chains have reduced the receiver
temperatures during 2021 to 16 K at 32 GHz and 30 K at 50 GHz. The backends
are $2\times 8\times 2.5$ GHz fast Fourier transform spectrometers with a
spectral resolution of 38 kHz, providing the whole coverage of the Q band in
both polarizations. A more detailed description of the system is given by
Tercero et al. (2021).

The data of the QUIJOTE line survey presented here were gathered in several
observing runs between November 2019 and July 2023. All observations were
performed using the frequency-switching observing mode with a frequency throw
of 8 and 10 MHz. The total observing time on the source for data taken with
frequency throws of 8 MHz and 10 MHz was 737 and 465 hours, respectively.
Hence, the total observing time on source was 1202 hours. The measured
sensitivity varied between 0.07 mK (70 $\mu$K) at 32 GHz and 0.2 mK at 49.5
GHz. The sensitivity of QUIJOTE is about 50 times better than that of previous
line surveys in the Q band of TMC-1 (Kaifu et al. 2004). For each frequency
throw, different local oscillator frequencies were used in order to remove
possible side-band effects in the down-conversion chain. A detailed
description of the QUIJOTE line survey is provided in Cernicharo et al.
(2021a). The data analysis procedure has been described by Cernicharo et al.
(2022).

The averaged main-beam efficiency measured during our observations varied from
0.66 at 32.4 GHz to 0.50 at 48.4 GHz (Tercero et al. 2021) and can be given
across the Q band by $B_{\rm eff}$=0.797 exp[$-$($\nu$(GHz)/71.1)2]. The
averaged forward telescope efficiency is 0.97. The telescope beam size at
half-power intensity is 54.4′′ at 32.4 GHz and 36.4′′ at 48.4 GHz.

We also included in this work data from the 3 mm line survey performed with
the IRAM 30m telescope. These data cover the full available band at the
telescope, between 71.6 GHz and 117.6 GHz. The EMIR E0 receiver was connected
to the Fourier Transform Spectrometers (FTS) in its narrow mode. The FTS
provide a spectral resolution of 49 kHz and a total bandwidth of 7.2 GHz per
spectral setup. Observations were performed in several runs. Between January
and May 2012, we completed the scan 82.5–117.6 GHz (Cernicharo et al.  2012).
In August 2018, after the upgrade of the E090 receiver, we extended the survey
down to 71.6 GHz. More recent high-sensitivity observations in 2021 were used
to improve the signal-to-noise ratio (S/N) in several frequency windows
(Agúndez et al.  2022; Cabezas et al. 2022). The final 3mm line survey has a
sensitivity of 2-10 mK. However, at some selected frequencies, the sensitivity
is as low as 0.6 mK. All the observations were performed using the frequency-
switching method with a frequency throw of 7.14 MHz. The IRAM 30m beam varied
between 34′′ and 21′′ at 72 GHz and 117 GHz, respectively, while the beam
efficiency took values of 0.83 and 0.78 at the same frequencies, following the
relation Beff= 0.871 exp[$-$($\nu$(GHz)/359)2]. The forward efficiency at 3 mm
is 0.95.

The intensity scale used in this study is the antenna temperature
($T_{A}^{*}$). Calibration was performed using two absorbers at different
temperatures and the atmospheric transmission model ATM (Cernicharo 1985;
Pardo et al. 2001). The absolute calibration uncertainty was 10 $\%$. However,
the relative calibration between lines within the QUIJOTE survey is certainly
better as all lines were observed simultaneously. The data were analyzed with
the GILDAS package333http://www.iram.fr/IRAMFR/GILDAS.

Figure 1: Observed lines of CH3CH2CCH in TMC-1. The line parameters are given
in Table LABEL:line_parameters. The abscissa corresponds to the rest frequency
assuming a velocity for the source of 5.83 km s-1. The ordinate is the antenna
temperature corrected for atmospheric and telescope losses in mK. Quantum
numbers are indicated in the top right corner of each panel. The red line
corresponds to the synthetic spectrum derived from the LTE model described in
Sect. 3.1. Blanked channels correspond to negative features produced in the
folding of the frequency-switching data. For some lines, we found a frequency
shift of up to $\pm$40 kHz with respect to the predictions (see Sect. 3.1).
This is indicated in magenta in the corresponding panels.

##  3 Results

The line identification was performed using the MADEX code (Cernicharo 2012)
and the CDMS and JPL catalogs (Müller et al. 2005; Pickett et al. 1998). The
intensity scale used in this study is the antenna temperature ($T_{A}^{*}$).
Consequently, the telescope parameters and source properties were used to
model the emission of the different species to produce synthetic spectra in
this temperature scale. The source was assumed to be circular with a uniform
brightness temperature and a radius of 40′′ (Fossé et al. 2001). The procedure
to derive line parameters is described in Appendix A. To model the observed
line intensities, MADEX uses a local thermodynamical equilibrium (LTE)
hypothesis supported by rotational diagrams or adopts a large velocity
gradient approach (LVG). In the later case, MADEX uses the formalism described
by Goldreich & Kwan (1974). Unfortunately, no collisional rates are available
for the species studied in this work. The permanent dipolar moments and
spectroscopic sources for the molecular species observed in this work are
discussed in the next sections.

###  3.1 CH3CH2CCH (ethyl acetylene)

In spite of the large observed abundance of CH3CH2CN in hot cores and hot
corinos (see Sect. 3.2), the homologous acetylenic form, CH3CH2CCH, has not
been reported in this type of environment so far. The detection is less
favorable due to the modest dipole moment ($\mu_{a}$=0.763 D and
$\mu_{b}$=0.17 D; Landsberg & Suenram  1983) compared to that of the cyanide
form (see Sect. 3.2). We recently reported a tentative detection of this
species toward the cold starless core TMC-1. The claim was based on the
observation of a few lines and a spectral stacking of all the transitions in
the QUIJOTE line survey (Cernicharo et al. 2021b). Here, we present the
detection of nine lines of the molecule, including some $K_{a}$=2 transitions.
The data are shown in Fig. 1, and their line parameters are given in Table
LABEL:line_parameters. The laboratory data for the rotational spectroscopy of
this species were summarized by Steber et al.  (2012). We note, however, that
in the microwave domain, Bestmann & Dreizler  (1985) reported a small internal
rotation splitting of about 100 kHz, but the uncertainty of their measurements
was on the same order. Although the laboratory data cover high quantum numbers
(up to $J$=46 and $K$=28) and the frequency predictions should be accurate
enough in the QUIJOTE band, our observations show frequency shifts with
respect to the predictions of up to 40 kHz (our channel width) for four
transitions (indicated in magenta in Fig. 1 in the top left corner of the
panels). This difference is compatible with the uncertainties of the
laboratory measurements in the Q band and with the possible splitting between
the $A$ and $E$ symmetry species at these frequencies. To derive the column
density of this molecule, we assumed a rotational temperature of 9 K. This
assumption was based on the low dipole moment of the molecule and on the
results we obtained for CH2CHCCH (see Sect. 3.3 and Cernicharo et al. 2021b).
The derived value is $N$ = (6.2$\pm$0.2)$\times$1011 cm-2 (see Table 1). Using
the column density derived in Sect. 3.3 for vinyl acetylene, we obtain a
CH2CHCCH/CH3CH2CCH abundance ratio of 15.3$\pm$0.8.

Figure 2: Observed lines of CH3CH2CN in TMC-1. The line parameters are given
in Table LABEL:line_parameters. The abscissa corresponds to the rest frequency
assuming a velocity for the source of 5.83 km s-1. The ordinate is the antenna
temperature corrected for atmospheric and telescope losses in mK. Quantum
numbers are indicated in the top left corner of each panel. The red line
corresponds to the synthetic spectrum derived from the model described in
Sect. 3.2. Blanked channels correspond to negative features produced in the
folding of the frequency-switching data.

###  3.2 CH3CH2CN (ethyl cyanide)

CH3CH2CN and several of its vibrationally excited states exhibit a dense
spectrum in warm molecular clouds such as Orion-KL and SgrB2 (Johnson et al.
1977; Gib et al.  2000; Daly et al.  2013; Margulès et al.  2018). However,
the molecule was not detected in cold dark clouds until recently, when we
reported several of its rotational lines toward TMC-1 (Cernicharo et al.
2021b). In this work, we present the spectra for this species using the latest
QUIJOTE data. The spectroscopic laboratory frequency measurements are from
Fukuyama et al.  (1996) and Brauer et al.  (2009), from which we derived the
rotational and distortion constants and implemented them in MADEX. Nine
$a$-type lines were detected, and they are shown in Fig. 2, while their line
parameters are given in Table LABEL:line_parameters. Some of the $K_{a}$=1 and
all $K_{a}$=2 lines show hyperfine structure. The frequencies including
hyperfine structure were adopted from the CDMS catalog (Müller et al. 2005). A
rotational diagram analysis of the data provided a rotational temperature of
5.5$\pm$0.5 K (see Fig. 3) and a column density of (1.3$\pm$0.1)$\times$1011
cm-2. The abundance ratio of CH3CH2CCH and ethyl cyanide in TMC-1 is
4.8$\pm$0.5. Although the frequencies of the 13C, 15N, and D isotopologs of
ethyl cyanide are well known (Demyk et al.  2007; Richard et al.  2012;
Margulès et al.  2009), the intensity of the lines of the main isotopolog
indicates that the lines from these isotopologs will be well below the current
sensitivity limit of the QUIJOTE line survey.

Figure 3: Rotational diagram of the observed lines of CH3CH2CN from the data
of Table LABEL:line_parameters. A rotational temperature of 5.5$\pm$0.5 K is
derived. Table 1: Derived column densities and abundances

Species |  Trot | $N$ | $X^{a}$ |
---|---|---|---|---
| (K) |  (cm-2) |  |
CH3CH2CCH | 9.0 |  (6.2$\pm$0.2)$\times$1011 |  6.2$\times$10-11 | A
CH3CH2CN |  5.5$\pm$0.5 |  (1.3$\pm$0.1)$\times$1011 |  1.3$\times$10-11 |
CH2CHCCH |  10.4$\pm$0.9 |  (9.5$\pm$0.2)$\times$1012 |  9.5$\times$10-09 | B
CH2CHCN |  4.3$\pm$0.2 |  (6.2$\pm$0.2)$\times$1012 |  6.2$\times$10-09 |
13CH2CHCN | 4.3 |  (6.2$\pm$0.2)$\times$1010 |  6.2$\times$10-12 | C
CH213CHCN | 4.3 |  (5.9$\pm$0.2)$\times$1010 |  5.9$\times$10-12 | C
CH2CH13CN | 4.3 |  (8.1$\pm$0.3)$\times$1010 |  8.1$\times$10-12 | C
CH2CDCN | 4.3 |  (4.7$\pm$0.3)$\times$1010 |  4.7$\times$10-12 | C
$trans$-CHDCHCN | 4.3 |  (4.5$\pm$0.3)$\times$1010 |  4.5$\times$10-12 | C
$cis$-CHDCHCN | 4.3 |  (4.9$\pm$0.3)$\times$1010 |  4.9$\times$10-12 | C
CH2CHC15N | 4.3 |  (2.2$\pm$0.2)$\times$1010 |  2.2$\times$10-12 | C
444 $a$$a$footnotetext: We assumed $N$(H2)=1022 cm-2 to derive the molecular
abundances (Cernicharo & Guélin  1987). $A$$A$footnotetext: We assumed that
the molecule is thermalized at the kinetic temperature of the cloud (9 K; see
text). $B$$B$footnotetext: A rotational diagram provides Trot=10.5$\pm$1.5. We
assumed that the molecule is thermalized at the kinetic temperature of the
cloud (9 K; see text). $C$$C$footnotetext: Rotational temperature assumed to
be identical to that of the main isotopolog.

###  3.3 CH2CHCCH (vinyl acetylene)

In spite of the similarity in structure with CH2CHCN, which has been detected
toward different astrophysical environments (see Sect. 3.4), vinyl acetelyne
was only recently detected through the sensitivity of the QUIJOTE line survey
(Cernicharo et al. 2021b). Due to the small dipole moment of the molecule
($\mu_{a}$=0.43 D, $\mu_{b}\sim$0, Sobolev et al.  1962; Thorwirth & Lichau
2003), the derived abundance was rather large ($\sim$1013 cm-2). The
rotational spectroscopy in the laboratory for this species was summarized by
Thorwirth & Lichau  (2003) and Thorwirth et al.  (2004). The observed lines in
our study are shown in Fig. 4, and their parameters are given in Table
LABEL:line_parameters. A rotational diagram analysis of the data provides
Trot=10.4$\pm$0.9 K (see Fig. 5), which is compatible with a thermalized
molecule at the kinetic temperature of the cloud, as expected for a species
with a low dipole moment ($T_{K}$=9 K; Agúndez et al. 2023b). The derived
column density is (9.5$\pm$0.2)$\times$1012 cm-2, which agrees well with the
value we reported previously. The derived CH2CHCCH/CH2CHCN abundance ratio is
1.5$\pm$0.1.

Figure 4: Observed lines of CH2CHCCH in TMC-1. The line parameters are given
in Table LABEL:line_parameters. The abscissa corresponds to the rest frequency
assuming a velocity for the source of 5.83 km s-1. The ordinate is the antenna
temperature corrected for atmospheric and telescope losses in mK. Quantum
numbers are indicated in the top left corner of each panel. The red line
corresponds to the synthetic spectrum derived from the model described in
Sect. 3.3. Blanked channels correspond to negative features produced in the
folding of the frequency-switching data.  Figure 5: Rotational diagram of the
observed lines of CH2CHCCH from the data of Table LABEL:line_parameters,
including the lines observed at 3mm. A rotational temperature of 10.4$\pm$0.9
K is derived.

###  3.4 CH2CHCN (vinyl cyanide; acrylonitrile)

CH2CHCN was detected in the early years of astrochemistry toward SgrB2
(Gardner & Winnewisser  1975). The molecule was detected toward TMC-1 by
Matthews & Seers  (1981) and more recently by Cernicharo et al. (2021b). Vinyl
cyanide was also detected toward the carbon-rich star IRC+10216 (Agúndez et
al. 2008) and in high-mass star-forming regions (Belloche et al.  2013; López
et al.  2014). There is a large set of laboratory data on the rotational
spectroscopy of this species that covers frequencies up to 1.67 THz, $J$=129,
and $K_{a}$=28 (Müller et al. 2008; Kisiel et al.  2009, and references
therein). The frequency predictions have uncertainties lower than 1 kHz in the
frequency domains covered in this study.

Figure 6: Observed lines of CH2CHCN in TMC-1. The line parameters are given in
Table LABEL:line_parameters. The abscissa corresponds to the rest frequency
assuming a velocity for the source of 5.83 km s-1. The ordinate is the antenna
temperature corrected for atmospheric and telescope losses in mK. Quantum
numbers are indicated in the top left corner of each panel. The red line
corresponds to the synthetic spectrum derived from the model described in
Sect. 3.4. Blanked channels correspond to negative features produced in the
folding of the frequency switching data. The $J_{u}$=4,5 ($K_{a}$=0,1)
transitions of the isotopologs of CH2CHCN are detected and are shown in Fig.
8.  Figure 7: Rotational diagram of the observed lines of CH2CHCN. The black
and red points correspond to $a$-type and $b$-type transitions, respectively.
A rotational temperature of 4.3$\pm$0.2 K is derived. The estimated
$b$-component of the dipolar moment is 0.80$\pm$0.03 D. The blue and green
points correspond to the $b$-type transitions when using $\mu_{b}$=0.687 D
(Kraśnicki & Kisiel  2011) and 0.894 D (Stolze & Sutter  1985), respectively.
Their error bars are identical to those of the red points.

We have observed 23 $a$-type and 8 $b$-type transitions of CH2CHCN in the 7 mm
and the 3 mm domains. Several of these lines exhibit well-resolved hyperfine
components. The lines observed in the Q band are shown in Fig. 6. The line
parameters for all observed lines are given in Table LABEL:line_parameters. A
rotational diagram is shown in Fig. 7. The initial dipole moments used in the
analysis of the data were those measured by Stolze & Sutter  (1985) ($\mu_{a}$
= 3.815 D and $\mu_{b}$ = 0.894 D). The $a$\- and $b$-type transitions are
well fit with a common rotational temperature of 4.3$\pm$0.2 K. However, the
column density derived from the $b$-type transitions is a factor 1.3 times
lower than the corresponding density from the $a$-type transitions. The effect
is systematic as all $b$-type transitions are well reproduced with a common
column density. This discrepancy cannot be attributed to a calibration problem
of the data. All $a$\- and $b$-type lines were observed with the same set of
observational data, gathered at the same time, and with the same systematics.
Moreover, the same upper levels are involved in both transition types. It
cannot be due to an opacity problem because for the strongest $a$-type
transitions, we estimate opacities $\sim$0.15. Hence, it seems that the
relative value of the dipole moments is not well determined.

The dipole moment of acrylonitrile has been the subject of several laboratory
studies. The first determination was made by Wilcox & Goldstein  (1954), who
derived $\mu_{a}$ = 3.68 D and $\mu_{b}$ = 1.25 D. Subsequent measurements by
Stolze & Sutter  (1985) provided different values, $\mu_{a}$ = 3.815 $\pm$
0.012 D and $\mu_{b}$ = 0.894 $\pm$ 0.068 D. The more recent determination of
$\mu_{b}$ = 0.687 $\pm$ 0.008 D (Kraśnicki & Kisiel  2011) produced the
reverse, that is, the column density from $b$-type transitions is 1.3 times
higher than the value derived from the $a$-type transitions. To coherently
interpret our data, we fit the value of $\mu_{b}$ necessary to produce similar
column densities for both types of transitions keeping $\mu_{a}$ to the value
derived by Stolze & Sutter  (1985). We obtain a value of 0.80$\pm$0.03 D,
which is between the two experimental determinations. The rotational diagram
shown in Fig. 7 permits us to simultaneously fit all transitions with
Trot=4.3$\pm$0.2 K and a column density for acrylonitrile of
(6.2$\pm$0.2)$\times$1012 cm-2 (see also Cernicharo et al. 2021b). The
computed synthetic spectra are presented in Fig. 6 and agree excellently with
all the observed transitions, $b$ and $a$ type.

Rotational spectroscopy for the 13C, 15N, and deuterated isotopologs of
acrylonitrile was performed by Colmont et al.  (1997); Müller et al. (2008);
Kisiel et al.  (2009), and Kraśnicki et al.  (2011). The 13C isotopologs were
detected toward Sgr B2 (Müller et al. 2008) and Orion-KL (López et al.  2014).
In our study of TMC-1, we detected six $a$-type transitions for the three 13C
substituted isotopologs, some of which exhibit the corresponding hyperfine
splitting. We also solidly detected the isotopologs CH2CHC15N, CH2CDCN,
$trans$-CHDCHCN, and $cis$-CHDCHCN (all these lines are $a$ type) for the
first time in the ISM. A tentative detection in Orion-KL of the deuterated
species was previously reported (López et al.  2014), but the detection was
based on a small number of weak lines with severe overlap with lines from
other molecular species. The lines observed toward TMC-1 are shown in Fig. 8.
In their analysis, we adopted the same rotational temperature and dipole
moments as for the main isotopolog. The derived column densities for all the
isotopologs are given in Table 1. The three singly deuterated species show
similar column densities and a deuterium enhancement $\sim$130, which is
significantly lower than was found in other species (Cernicharo et al. 2024;
Tercero et al. 2024). The first and second 13C isotopologs show a 12C/13C
abundance ratio of 100$\pm$7, while for the substitution in the third
position, the ratio is 76$\pm$5\. The CH2CHCN/CH2CHC15N abundance ratio is
282$\pm$35\. These values are in line with those derived for HCCCN by Tercero
et al. (2024) and HNCCC and HCCNC by Cernicharo et al. (2024). These two
papers provide a detailed discussion of the 13C enhancement in these species.

Figure 8: Observed lines of the isotopologs of CH2CHCN. The line parameters
are given in Table LABEL:line_parameters. The abscissa corresponds to the rest
frequency assuming a velocity for the source of 5.83 km s-1. The ordinate is
the antenna temperature corrected for atmospheric and telescope losses in mK.
The species are indicated at the end of each row, and the quantum numbers are
indicated at the top of each column. The red line corresponds to the synthetic
spectrum derived from the model described in Sect. 3.4. Blanked channels
correspond to negative features produced in the folding of the frequency-
switching data.

##  4 Discussion

The observational data presented here make it worthwhile to revisit the
chemistry of the CCH and CN derivatives of CH2CH2 and CH3CH3. We carried out
chemical modeling calculations using a gas-phase chemical model. The model was
essentially the same as presented previously in Cernicharo et al. (2021b).
Briefly, we adopted typical parameters of cold dark clouds, that is, a gas
kinetic temperature of 10 K, a volume density of H2 of 2 $\times$ 104 cm-3, a
cosmic-ray ionization rate of H2 of 1.3 $\times$ 10-17 s-1, a visual
extinction of 30 mag, and the so-called low metal elemental abundances
(Agúndez & Wakelam 2013). The core of the chemical network is based on the
RATE12 network from the UMIST database (McElroy et al. 2013), with some
updates from the more recent literature (e.g., Loison et al. 2015). The
relevant reactions that describe the formation of the CCH and CN derivatives
of CH2CH2 and CH3CH3 are discussed below. The calculated fractional abundances
of CH2CHCCH, CH2CHCN, CH3CH2CCH, and CH3CH2CN are shown as a function of time
in Fig. 9. The chemical model produces abundances at times of a few 105 yr
that are in reasonable good agreement with the observed values, with
discrepancies one order of magnitude or smaller. Taking into account that the
chemistry of these four molecules is not particularly well known, we can
consider that the agreement between observations and model is satisfactory,
although a better knowledge of the reactions of formation and destruction of
these molecules is highly desirable.

The CCH and CN derivatives of C2H4 are formed in the chemical model through
the reactions of C2H and CN with C2H4, which are rapid and proceed through H
atom elimination, according to an extensive number of studies (Opansky & Leone
1996; Vakhtin et al. 2001; Bouwman et al. 2012; Krishtal et al. 2009; Dash &
Rajakumar 2015; Sims et al. 1993; Choi et al. 2004; Gannon et al. 2007;
Balucani et al. 2015). In addition, CH2CHCCH is also formed by the reaction of
CH with CH3CCH and CH2CCH2 (Daugey et al. 2005; Goulay et al. 2009; Ribeiro &
Mebel 2017) and by the reaction C2 \+ C2H6 (Páramo et al. 2008), although this
latter reaction probably occurs through H abstraction, in which case, it would
not produce CH2CHCCH. On the other hand, CH2CHCN is also produced in the
reaction CN + CH2CHCH3 (Sims et al. 1993; Morales et al. 2010; Gannon et al.
2007; Huang et al. 2009) and in the dissociative recombination of the cation
C3H4N+, although for this latter process, the branching ratios for the
different fragmentation products are unknown.

In the case of the CCH and CN derivatives of C2H6, the formation pathways are
more uncertain. The most obvious chemical route would be CCH + C2H6 and CN +
C2H6. These reactions are fast (Opansky & Leone 1996; Sims et al. 1993), but
occur through H abstraction rather than H elimination (Dash & Rajakumar 2015;
Georgievskii & Klippenstein 2007; Espinosa-García & Rangel 2023). The main
routes to CH3CH2CCH in the model are the dissociative recombination of
C4H${}_{7}^{+}$ and CH + CH2CHCH3. This latter reaction was measured to be
rapid at low temperatures (Daugey et al. 2005), and the reaction was found to
occur mostly through H elimination (Loison & Bergeat 2009). Trevitt et al.
(2013) reported that CH3CH2CCH is produced with a branching ratio of 0.12,
although theoretical calculations (Ribeiro & Mebel 2016; He et al. 2019)
pointed to 1,2-butadiene as the only C4H6 isomer formed. For CH3CH2CN, the
only formation channel in the model is the dissociative recombination of
C3H6N+, although it is unknown how large cations, such as C4H${}_{7}^{+}$ and
C3H6N+, fragment upon reaction with electrons.

Figure 9: Calculated fractional abundances of the CCH and CN derivatives of
C2H4 and C2H6 as a function of time. The horizontal dashed lines correspond to
the abundances observed in TMC-1.

In summary, the formation of CH2CHCCH, CH2CHCN, CH3CH2CCH, and CH3CH2CN in
TMC-1 can be explained in terms of standard gas-phase routes involving
neutral-neutral and ion-neutral reactions. However, there are still important
uncertainties regarding the different products resulting from the various
possible reactions of formation of these molecules.

###### Acknowledgements.

We thank Ministerio de Ciencia e Innovación of Spain (MICIU) for funding
support through projects PID2019-106110GB-I00, and PID2019-106235GB-I00,
PID2022-137980NB-I00, MCIN/AEI/ 10.13039/501100011033. We thank the Consejo
Superior de Investigaciones Científicas (CSIC) for funding through project PIE
202250I097. We also thank ERC for funding through grant
ERC-2013-Syg-610256-NANOCOSMOS.

## References

  * Agúndez et al. (2008) Agúndez, M., Fonfría, J.P., Cernicharo, J. et al. 2008, A&A, 479, 493
  * Agúndez & Wakelam (2013) Agúndez, M. & Wakelam, V. 2013, Chem. Rev., 113, 8710
  * Agúndez et al. (2021) Agúndez, M., Cabezas, C., Tercero, B. et al. 2021, A&A, 647, L10
  * Agúndez et al. (2022) Agúndez, M., Marcelino, N., Cabezas, C. et al. 2022, A&A, 657, A96
  * Agúndez et al. (2023a) Agúndez, M., Marcelino, N. Tercero, B. et al. 2023a, A&A, 677, L13
  * Agúndez et al. (2023b) Agúndez, M., Marcelino, N. Tercero, B. et al. 2023b, A&A, 677, A106
  * Balucani et al. (2015) Balucani, N., Leonori, F., Petrucci, R., et al. 2015, Chem. Phys., 449, 34
  * Belloche et al. (2013) Belloche, A., Müller, H.S.P., Menten, K.M. et al. 2013, A&A, 559, A47
  * Bestmann & Dreizler (1985) Bestmann, G. & Dreizler, H. 1985, Z. Naturforsch, 40a, 263
  * Bouwman et al. (2012) Bouwman, J., Goulay, F., Leone, S. R., & Wilson, K. R. 2012, J. Phys. Chem. A, 116, 3907
  * Brauer et al. (2009) Brauer, C.S., Pearson, J.C., Drouin, B.J. & Yu, S. 2009, ApJS, 184, 133
  * Burkhardt et al. (2021) Burkhardt, A.M., Long Kelvin Lee, K., Changala, P.B. et al. 2021, ApJ, 913, L18
  * Cabezas et al. (2021) Cabezas, C., Endo, Y., Roueff, E., et al. 2021, A&A, 646, L1
  * Cabezas et al. (2022) Cabezas, C., Agúndez, M., Marcelino, N., et al. 2022, A&A, 657, L4
  * Cabezas et al. (2023) Cabezas, C., Agúndez, M., Marcelino, N., et al. 2023, A&A, 676, L5
  * Cernicharo (1985) Cernicharo, J. 1985, Internal IRAM report (Granada: IRAM)
  * Cernicharo & Guélin (1987) Cernicharo, J. & Guélin, M. 1987, A&A, 176, 299
  * Cernicharo (2012) Cernicharo, J., 2012, in ECLA 2011: Proc. of the European Conference on Laboratory Astrophysics, EAS Publications Series, 2012, Ed.: C. Stehl, C. Joblin, & L. d’Hendecourt (Cambridge: Cambridge Univ. Press), 251; https://nanocosmos.iff.csic.es/?page$\\_$id=1619
  * Cernicharo et al. (2012) Cernicharo, J., Marcelino, N., Rouef, E., et al. 2012, ApJ, 759, L43
  * Cernicharo et al. (2020) Cernicharo, J., Marcelino, N., Agúndez, M. et al. 2020, A&A, 642, L8
  * Cernicharo et al. (2021a) Cernicharo, J., Agúndez, M., Kaiser, R., et al. 2021a, A&A, 652, L9
  * Cernicharo et al. (2021b) Cernicharo, J., Agúndez, M., Cabezas, C. et al. 2021b, A&A, 647, L2
  * Cernicharo et al. (2021c) Cernicharo, J., Agúndez, M., Cabezas, C. et al. 2021c, A&A, 649, L15
  * Cernicharo et al. (2022) Cernicharo, J., Fuentetaja, R., Agúndez, M. et al. 2022, A&A, 663, L9
  * Cernicharo et al. (2023a) Cernicharo, J., Pardo, J. R., Cabezas, C. et al. 2023a, A&A, 670, L19
  * Cernicharo et al. (2023b) Cernicharo, J., Cabezas, C., Pardo, J.R. et al. 2023b, A&A, 672, L13
  * Cernicharo et al. (2023c) Cernicharo, Tercero, B., Marcelino, N. et al. 2023c, A&A, 674, L4
  * Cernicharo et al. (2023d) Cernicharo, J., Fuentetaja, R., Agúndez, M. et al. 2023d, A&A, 680, L4
  * Cernicharo et al. (2024) Cernicharo, J., et al. 2024, A&A, in press, 10.48550/arXiv.2401.15997
  * Choi et al. (2004) Choi, N., Blitz, M. A., McKee, K., et al. 2004, Chem. Phys. Lett., 384, 68
  * Colmont et al. (1997) Colmont, J.M., Wlodarczak, G., Priem, D. et al. 1997, J. Mol. Spectrosc., 181, 330
  * Daly et al. (2013) Daly, A.M., Bermúdez, C., López, A. et al. 2013, A&A, 768, A81
  * Dash & Rajakumar (2015) Dash, M. R. & Rajakumar, B. 2015, PCCP, 17, 3142
  * Daugey et al. (2005) Daugey, N., Caubet, P., Retail, B., et al. 2005, PCCP, 7, 2921
  * Demyk et al. (2007) Demyk, K., Mäder, H., Tercero, B. et al. 2007, A&A, 466, 255
  * Espinosa-García & Rangel (2023) Espinosa-García, J. & Rangel, C. 2023, J. Chem. Phys., 159, 124307
  * Fossé et al. (2001) Fossé, D., Cernicharo, J., Gerin, M., Cox, P. 2001, ApJ, 552, 168
  * Fukuyama et al. (1996) Fukuyama, Y., Odashima, H., Takagi, K. & Tsunekawa, S. 1996, ApJS, 104, 329
  * Gannon et al. (2007) Gannon, K. L., Glowacki, D. R., Blitz, M. A., et al. 2007, J. Phys. Chem. A, 111, 6679
  * Gardner & Winnewisser (1975) Gardner, F.F. & Winnewisser, G. 1975, ApJ, 195, L127
  * Georgievskii & Klippenstein (2007) Georgievskii, Y. & Klippenstein, S. J. 2007, J. Phys. Chem., 111, 3802
  * Goulay et al. (2009) Goulay, F., Trevitt, A. J., Meloni, G., et al. 2009, J. Am. Chem. soc., 131, 993
  * Gib et al. (2000) Gib, E., Nummelin, A., Irvine, W.M. et al. 2000, ApJ, 545, 309
  * Goldreich & Kwan (1974) Goldreich, P. & Kwan, J. 1974, ApJ, 189, 441
  * He et al. (2019) He, C., Thomas, A. M., Galimova, G. R., et al. 2019, J. Phys. Chem. A, 123, 10543
  * Huang et al. (2009) Huang, C. H., Kaiser, R. I., & Chang, A. H. H. 2009, J. Phys. Chem. A, 113, 12675
  * Johnson et al. (1977) Johnson, D. R., Lovas, F. J., Gottlieb, C. A. et al. 1977, ApJ, 218, L370
  * Kaifu et al. (2004) Kaifu, N., Ohishi, M., Kawaguchi, K., et al. 2004, PASJ, 56, 69
  * Kisiel et al. (2009) Kisiel, Z., Pszczółkowski. L- Drouin, B.J. et al. 2009, J. Mol. Spectrosc., 258, 26
  * Kraśnicki & Kisiel (2011) Kraśnicki, A. & Kisiel, Z. 2011, J. Mol. Spectrosc., 270, 83
  * Kraśnicki et al. (2011) Kraśnicki, A. Kisiel, Z. Drouin, B.J. et al., 2011, J. Mol. Struct., 1006, 20
  * Krishtal et al. (2009) Krishtal, S. P., Mebel, A. M., & Kaiser, R. I. 2009, J. Phys. Chem. A, 113, 11112
  * Landsberg & Suenram (1983) Landsberg, B.M. & Suenram, R.D. 1983, J. Mol. Spectrosc., 98, 210
  * Loison & Bergeat (2009) Loison, J.-C. & Bergeat, A. 2009, PCCP, 11, 655
  * Loison et al. (2015) Loison, J.-C., Hébrard, E., Dobrijevic, M., et al. 2015, Icarus, 247, 218
  * López et al. (2014) López, A., Tercero, B., Kisiel, Z. et al. 2014, A&A, 572, A44
  * Loru et al. (2023) Loru, D., Cabezas. C., Cernicharo, J. et al., 2023, A&A, 677, A166
  * Marcelino et al. (2007) Marcelino, N., Cernicharo, J., Agúndez, M. et al. 2007, ApJ, 665, L127
  * Margulès et al. (2009) Margulès, L., Motiyenko, R., Demyk, K. et al. 2009, A&A, 493, 565
  * Margulès et al. (2018) Margulès, L., Tercero, B., Guillemin, J.C. et al. 2018, A&A, 610, A44
  * Matthews & Seers (1981) Matthews, H.E & Sears, T.J. 1981, ApJ, 272, 149
  * McElroy et al. (2013) McElroy, D., Walsh, C., Markwick, A. J., et al. 2013, A&A, 550, A36
  * McGuire et al. (2018) McGuire, B.A., Burkhardt, A.M., Kalenskii, S., et al. 2018, Science, 359, 202
  * McGuire et al. (2021) McGuire, B.A., Loomis, R.A., Burkhardt, A.M., et al. 2021, Science, 371, 1265
  * Morales et al. (2010) Morales, S. B., Le Picard, S. D., Canosa, A., & Sims, I. R. 2010, Farady Discuss., 147, 155
  * Müller et al. (2005) Müller, H. S. P., Schlöder, F., Stutzki, J., Winnewisser, G. 2005, J. Mol. Struct., 742, 215
  * Müller et al. (2008) Müller, H. S. P., Belloche, A., Menten, K. et al. 2008, J. Mol. Spectrosc., 251, 319
  * Opansky & Leone (1996) Opansky, B. J. & Leone, S. R. 1996, J. Phys. Chem., 100, 19904
  * Páramo et al. (2008) Páramo, A., Canosa, A., Le Picard, S. D., & Sims, I. R. 2008, J. Phys. Chem. A, 112, 9591
  * Pardo et al. (2001) Pardo, J. R., Cernicharo, J., Serabyn, E. 2001, IEEE Trans. Antennas and Propagation, 49, 12
  * Pardo et al. (2021) Pardo, J. R., Cabezas, C., Fonfría, J.P. et al. 2021, A&A, 652, L13
  * Pardo et al. (2022) Pardo, J. R., Cernicharo, J., Tercero, B. et al. 2022, A&A, 658, A39
  * Pickett et al. (1998) Pickett, H. M., Poynter, R. L., Cohen, E. A., et al. 1998, J. Quant. Spectrosc. Radiat. Transfer, 60, 883
  * Ribeiro & Mebel (2016) Ribeiro, J. M. & Mebel, A. M. 2016, J. Phys. Chem. A, 120, 1800
  * Ribeiro & Mebel (2017) Ribeiro, J. M. & Mebel, A. M. 2017, PCCP, 19, 14543
  * Richard et al. (2012) Richard, C., Margulès, L., Motiyenco, R. & Guillemin, J.-C. 2012, A&A, 543, A135
  * Rivilla et al. (2023) Rivilla, V.M., Sanz-Novo, M., Jiménez-Serra, I. et al. 2023, ApJ, 953, L20
  * Sims et al. (1993) Sims, I. R., Queffelec, J.-L., Travers, D., et al. 1993, Chem. Phys. Lett., 211, 461
  * Sita et al. (2022) Sita, M.L., Changala, P.B., Xue, C. et al. 2022, ApJ, 938, L12
  * Sobolev et al. (1962) Sobolev, G. A., Shcherbakov, A. M., & Akishin, P. A. 1962, Opt. Spectrosc., 12, 78
  * Steber et al. (2012) Steber, A. L., Harris, B. J., Neill, J. L. et al. 2012, J. Mol. Spectrosc., 280, 3
  * Stolze & Sutter (1985) Stolze, M. & Sutter, D.H. 1985, Z. Naturforsch, 40a, 998
  * Tercero et al. (2021) Tercero, F., López-Pérez, J. A., Gallego, et al. 2021, A&A, 645, A37
  * Tercero et al. (2024) Tercero, B. et al. 2024, A&A, in press, 10.48550/arXiv.2402.01318
  * Thorwirth & Lichau (2003) Thorwirth, S., & Lichau, H. et al. 2003, A&A, 398, L11
  * Thorwirth et al. (2004) Thorwirth, S., Müller, H. S. P., Lichau, H. et al. 2004, J. Mol. Struct., 695, 263
  * Trevitt et al. (2013) Trevitt, A. J., Prendergast, M. B., Goulay, F., et al. 2013, J. Phys. Chem. A, 117, 6450
  * Vakhtin et al. (2001) Vakhtin, A. B., Heard, D. E., Smith, I. W. M., & Leone, S. R. 2001, Chem. Phys. Lett., 348, 21
  * Wilcox & Goldstein (1954) Wilcox, W. S. & Goldstein, J. H. 1954, J. Chem. Phys., 22, 516

##  Appendix A Line parameters

The line parameters for all observed transitions with the Yebes 40m and IRAM
30m radio telescopes were derived by fitting a Gaussian line profile to them
using the GILDAS package. A velocity range of $\pm$20 km s-1 around each
feature was considered for the fit after a polynomial baseline was removed.
Negative features produced in the folding of the frequency switching data were
blanked before baseline removal.

Table 2: Observed line parameters for the species.  Molecule |  $J$a |  $F$b |  $\nu_{rest}$ c |  $\int T_{A}^{*}dv$ d |  vLSR e |  $\Delta$v f |  $T_{A}^{*}$ g | Notes
---|---|---|---|---|---|---|---|---
|  |  | (MHz) |  (mK km s-1) |  (km s-1) |  (km s-1) | (mK) |
CH3CH2CCH | $4_{1,4}-3_{1,3}$ |  |  33600.696$\pm$0.020 |  0.27$\pm$0.05 |  5.83$\pm$0.00 |  0.83$\pm$0.18 |  0.31$\pm$0.06 |
| $4_{0,4}-3_{0,3}$ |  |  34463.749$\pm$0.020 |  0.61$\pm$0.07 |  5.83$\pm$0.00 |  1.15$\pm$0.15 |  0.50$\pm$0.06 |
| $4_{2,3}-3_{2,2}$ |  |  34529.087$\pm$0.030 |  0.27$\pm$0.09 |  5.83$\pm$0.00 |  0.92$\pm$0.25 |  0.28$\pm$0.09 |
| $4_{1,3}-3_{1,2}$ |  |  35438.395$\pm$0.020 |  0.31$\pm$0.07 |  5.83$\pm$0.00 |  0.80$\pm$0.00 |  0.36$\pm$0.08 | A
| $5_{1,5}-4_{1,4}$ |  |  41987.921$\pm$0.030 |  0.50$\pm$0.14 |  5.83$\pm$0.00 |  1.03$\pm$0.22 |  0.46$\pm$0.19 | B
| $5_{0,5}-4_{0,4}$ |  |  43027.572$\pm$0.030 |  0.62$\pm$0.06 |  5.83$\pm$0.00 |  1.00$\pm$0.00 |  0.54$\pm$0.10 | A
| $5_{2,4}-4_{2,3}$ |  |  43152.200$\pm$0.030 |  0.35$\pm$0.09 |  5.83$\pm$0.00 |  0.80$\pm$0.00 |  0.43$\pm$0.11 | A
| $5_{2,3}-4_{2,2}$ |  |  43290.425$\pm$0.030 |  0.39$\pm$0.08 |  5.83$\pm$0.00 |  0.90$\pm$0.18 |  0.40$\pm$0.08 |
| $5_{1,4}-4_{1,3}$ |  |  44283.987$\pm$0.030 |  0.38$\pm$0.09 |  5.83$\pm$0.00 |  0.71$\pm$0.15 |  0.50$\pm$0.09 |
CH3CH2CN | $4_{1,4}-3_{1,3}$ | 4-3 |  34823.952$\pm$0.001 |  0.90$\pm$0.09 |  5.57$\pm$0.11 |  0.80$\pm$0.00 |  1.06$\pm$0.06 | C
|  | 5-4 & 3-2 |  34824.090$\pm$0.001 |  1.02$\pm$0.09 |  5.84$\pm$0.10 |  0.80$\pm$0.00 |  1.19$\pm$0.06 | C,D
| $4_{0,4}-3_{0,3}$ | 3-2 |  35722.142$\pm$0.001 |  0.95$\pm$0.06 |  5.65$\pm$0.25 |  0.80$\pm$0.00 |  1.11$\pm$0.08 | C
|  | 5-4 & 4-3 |  35722.239$\pm$0.001 |  1.72$\pm$0.06 |  5.81$\pm$0.09 |  0.70$\pm$0.10 |  2.38$\pm$0.08 | C,D
| $4_{2,3}-3_{2,2}$ | 4-3 |  35791.730$\pm$0.002 |  0.20$\pm$0.06 |  5.83$\pm$0.11 |  0.69$\pm$0.19 |  0.27$\pm$0.08 |
|  | 5-4 |  35792.188$\pm$0.001 |  0.33$\pm$0.07 |  5.81$\pm$0.08 |  0.70$\pm$0.16 |  0.44$\pm$0.08 |
|  | 3-2 |  35792.305$\pm$0.001 |  0.22$\pm$0.06 |  5.85$\pm$0.08 |  0.52$\pm$0.22 |  0.39$\pm$0.08 |
| $4_{2,2}-3_{2,1}$ | 4-3 |  35865.827$\pm$0.002 |  0.32$\pm$0.06 |  6.06$\pm$0.11 |  0.57$\pm$0.14 |  0.52$\pm$0.10 | E
|  | 5-4 |  35866.279$\pm$0.001 |  0.25$\pm$0.08 |  5.72$\pm$0.08 |  0.56$\pm$0.17 |  0.42$\pm$0.10 | E
|  | 3-2 |  35866.396$\pm$0.001 |  0.33$\pm$0.09 |  5.82$\pm$0.08 |  0.73$\pm$0.25 |  0.42$\pm$0.10 | E
| $4_{1,3}-3_{1,2}$ | 3-2 |  36379.581$\pm$0.001 |  0.69$\pm$0.09 |  5.71$\pm$0.08 |  0.78$\pm$0.12 |  0.84$\pm$0.08 | C
|  | 5-4 & 4-3 |  36379.712$\pm$0.001 |  1.44$\pm$0.09 |  5.78$\pm$0.00 |  0.74$\pm$0.05 |  1.82$\pm$0.08 | C,D
| $5_{1,5}-4_{1,3}$ | Main |  43516.229$\pm$0.001 |  1.76$\pm$0.12 |  5.77$\pm$0.03 |  0.89$\pm$0.08 |  1.86$\pm$0.12 | D
| $5_{0,5}-4_{0,4}$ | Main |  44597.010$\pm$0.001 |  2.86$\pm$0.14 |  5.93$\pm$0.02 |  0.88$\pm$0.05 |  3.05$\pm$0.11 | D
| $5_{2,3}-4_{2,2}$ | 5-4 |  44877.939$\pm$0.001 |  0.71$\pm$0.06 |  6.13$\pm$0.08 |  1.00$\pm$0.22 |  0.67$\pm$0.13 |
|  | 6-5 & 4-3 |  44878.172$\pm$0.001 |  0.73$\pm$0.08 |  5.97$\pm$0.08 |  1.11$\pm$0.21 |  0.62$\pm$0.13 | D
| $5_{1,4}-4_{1,3}$ | Main |  45909.544$\pm$0.001 |  1.89$\pm$0.12 |  5.93$\pm$0.02 |  0.78$\pm$0.06 |  2.28$\pm$0.10 |
CH2CHCCH | $4_{1,4}-3_{1,3}$ |  |  35463.207$\pm$0.001 |  1.76$\pm$0.06 |  5.85$\pm$0.01 |  0.79$\pm$0.03 |  2.10$\pm$0.07 |
| $4_{0,4}-3_{1,0}$ |  |  36270.155$\pm$0.001 |  2.53$\pm$0.06 |  5.80$\pm$0.01 |  0.77$\pm$0.02 |  3.10$\pm$0.06 |
| $4_{2,3}-3_{2,2}$ |  |  36298.867$\pm$0.001 |  0.92$\pm$0.07 |  5.77$\pm$0.03 |  0.83$\pm$0.07 |  1.04$\pm$0.08 |
| $4_{2,2}-3_{2,1}$ |  |  36327.096$\pm$0.001 |  0.60$\pm$0.05 |  5.80$\pm$0.03 |  0.69$\pm$0.07 |  0.83$\pm$0.08 |
| $4_{1,3}-3_{1,2}$ |  |  37123.615$\pm$0.001 |  1.72$\pm$0.07 |  5.79$\pm$0.03 |  0.69$\pm$0.03 |  2.36$\pm$0.06 |
| $5_{1,5}-4_{1,4}$ |  |  44323.533$\pm$0.001 |  2.45$\pm$0.10 |  5.82$\pm$0.01 |  0.71$\pm$0.04 |  3.25$\pm$0.11 |
| $5_{0,5}-4_{0,4}$ |  |  45316.216$\pm$0.001 |  2.54$\pm$0.07 |  5.78$\pm$0.01 |  0.57$\pm$0.03 |  4.19$\pm$0.12 |
| $5_{2,4}-4_{2,3}$ |  |  45369.726$\pm$0.001 |  1.05$\pm$0.09 |  5.86$\pm$0.03 |  0.63$\pm$0.06 |  1.58$\pm$0.12 |
| $5_{2,3}-4_{2,2}$ |  |  45426.153$\pm$0.001 |  0.89$\pm$0.14 |  5.85$\pm$0.05 |  0.61$\pm$0.10 |  1.37$\pm$0.10 |
| $5_{1,4}-4_{1,3}$ |  |  46398.742$\pm$0.001 |  2.49$\pm$0.09 |  5.77$\pm$0.01 |  0.63$\pm$0.03 |  3.70$\pm$0.13 |
| $8_{0,8}-7_{0,7}$ |  |  72357.946$\pm$0.001 |  8.39$\pm$2.23 |  5.90$\pm$0.08 |  0.58$\pm$0.16 |  13.69$\pm$4.21 |
| $8_{1,7}-7_{1,6}$ |  |  74197.205$\pm$0.001 |  4.17$\pm$1.48 |  5.54$\pm$0.07 |  0.34$\pm$0.12 |  11.41$\pm$3.69 |
| $9_{0,9}-8_{0,8}$ |  |  81330.920$\pm$0.001 |  4.56$\pm$0.44 |  5.78$\pm$0.03 |  0.66$\pm$0.08 |  6.52$\pm$0.81 |
| $9_{1,8}-8_{1,7}$ |  |  83451.437$\pm$0.001 |  3.63$\pm$0.33 |  5.89$\pm$0.03 |  0.61$\pm$0.07 |  5.56$\pm$0.67 |
| $10_{1,10}-9_{1,9}$ |  |  88558.997$\pm$0.001 |  |  |  |  $\leq$7.53 |
| $10_{0,10}-9_{0,9}$ |  |  90279.458$\pm$0.001 |  |  |  |  $\leq$7.59 |
| $10_{1,9}-9_{1,8}$ |  |  92698.125$\pm$0.001 |  1.46$\pm$0.33 |  6.19$\pm$0.10 |  0.61$\pm$0.13 |  2.23$\pm$0.84 | F
CH2CHCN | $4_{1,4}-3_{1,3}$ | 4-4 |  37017.811$\pm$0.001 |  2.42$\pm$0.11 |  5.80$\pm$0.02 |  0.78$\pm$0.05 |  2.90$\pm$0.10 |
|  | 4-3 |  37018.828$\pm$0.001 |  30.87$\pm$0.15 |  5.74$\pm$0.01 |  0.73$\pm$0.01 |  39.86$\pm$0.10 |
|  | 5-4 &3-2 |  37018.981$\pm$0.001 |  55.77$\pm$0.15 |  5.91$\pm$0.01 |  0.84$\pm$0.01 |  62.76$\pm$0.10 | D
|  | 3-3 |  37020.299$\pm$0.001 |  1.73$\pm$0.09 |  5.80$\pm$0.02 |  0.64$\pm$0.04 |  2.56$\pm$0.10 |
| $4_{0,4}-3_{0,3}$ | 4-4 |  37903.585$\pm$0.001 |  3.21$\pm$0.30 |  5.80$\pm$0.02 |  0.68$\pm$0.07 |  4.42$\pm$0.11 |
|  | 3-2 |  37904.770$\pm$0.001 |  39.15$\pm$0.37 |  5.73$\pm$0.03 |  0.75$\pm$0.03 |  48.98$\pm$0.11 |
|  | 5-4 & 4-3 |  37904.880$\pm$0.001 |  95.35$\pm$0.38 |  5.85$\pm$0.03 |  0.72$\pm$0.03 |  124.52$\pm$0.11 | D
|  | 3-3 |  37906.477$\pm$0.001 |  1.93$\pm$0.06 |  5.78$\pm$0.02 |  0.58$\pm$0.02 |  3.15$\pm$0.11 | G
| $4_{2,3}-3_{2,2}$ | 4-4 & 4-3 |  37939.247$\pm$0.001 |  4.87$\pm$0.06 |  5.78$\pm$0.01 |  0.67$\pm$0.01 |  6.80$\pm$0.08 |
|  | 5-4 |  37939.764$\pm$0.001 |  6.25$\pm$0.07 |  5.79$\pm$0.01 |  0.71$\pm$0.01 |  8.23$\pm$0.08 | D
|  | 3-2 & 3-3 |  37939.897$\pm$0.001 |  3.60$\pm$0.07 |  5.79$\pm$0.01 |  0.66$\pm$0.02 |  5.17$\pm$0.08 | D
| $4_{2,2}-3_{2,1}$ | 4-3 & 4-4 |  37973.988$\pm$0.001 |  5.73$\pm$0.08 |  5.82$\pm$0.01 |  0.81$\pm$0.01 |  6.67$\pm$0.10 | D
|  | 5-4 |  37974.504$\pm$0.001 |  5.97$\pm$0.09 |  5.79$\pm$0.01 |  0.68$\pm$0.01 |  8.30$\pm$0.10 |
|  | 3-2 & 3-3 |  37974.634$\pm$0.001 |  3.92$\pm$0.09 |  5.76$\pm$0.01 |  0.71$\pm$0.02 |  5.18$\pm$0.10 | D
| $4_{1,3}-3_{1,2}$ | 4-4 |  38846.763$\pm$0.001 |  2.03$\pm$0.08 |  5.76$\pm$0.01 |  0.71$\pm$0.03 |  2.70$\pm$0.10 |
|  | 4-3 |  38847.641$\pm$0.001 |  31.24$\pm$0.12 |  5.73$\pm$0.02 |  0.70$\pm$0.01 |  41.80$\pm$0.10 |
|  | 5-4 & 3-2 |  38847.790$\pm$0.001 |  59.73$\pm$0.12 |  5.87$\pm$0.02 |  0.78$\pm$0.01 |  72.44$\pm$0.10 | D
|  | 3-3 |  38848.933$\pm$0.001 |  2.07$\pm$0.07 |  5.78$\pm$0.01 |  0.64$\pm$0.03 |  3.06$\pm$0.10 |
| $1_{1,0}-1_{0,1}$ | 1-0 |  45332.905$\pm$0.001 |  0.75$\pm$0.12 |  5.74$\pm$0.06 |  0.82$\pm$0.18 |  0.81$\pm$0.11 |
|  | 2-2 |  45333.979$\pm$0.001 |  1.53$\pm$0.10 |  5.75$\pm$0.02 |  0.56$\pm$0.04 |  2.54$\pm$0.11 |
|  | 0-1 |  45334.170$\pm$0.001 |  0.48$\pm$0.12 |  5.73$\pm$0.14 |  0.72$\pm$0.08 |  0.65$\pm$0.11 | H
|  | 1-2 |  45334.610$\pm$0.001 |  0.79$\pm$0.15 |  5.81$\pm$0.11 |  1.09$\pm$0.20 |  0.68$\pm$0.11 |
|  | 2-1 |  45335.116$\pm$0.001 |  0.60$\pm$0.13 |  5.64$\pm$0.07 |  0.65$\pm$0.16 |  0.87$\pm$0.11 |
|  | 1-1 |  45335.747$\pm$0.001 |  0.41$\pm$0.09 |  5.61$\pm$0.08 |  0.58$\pm$0.09 |  0.67$\pm$0.11 |
| $2_{1,1}-2_{0,2}$ | 2-1 |  45793.994$\pm$0.001 |  0.50$\pm$0.06 |  5.81$\pm$0.04 |  0.65$\pm$0.09 |  0.72$\pm$0.10 |
|  | 2-3 |  45794.671$\pm$0.001 |  0.35$\pm$0.07 |  5.60$\pm$0.10 |  0.84$\pm$0.20 |  0.39$\pm$0.10 |
|  | 1-1 |  45794.837$\pm$0.001 |  0.92$\pm$0.07 |  5.74$\pm$0.02 |  0.53$\pm$0.04 |  1.63$\pm$0.10 |
|  | 3-3 |  45795.213$\pm$0.001 |  2.20$\pm$0.06 |  5.78$\pm$0.02 |  0.63$\pm$0.02 |  3.27$\pm$0.10 |
|  | 2-2 |  45795.889$\pm$0.001 |  1.53$\pm$0.06 |  5.76$\pm$0.02 |  0.68$\pm$0.03 |  2.12$\pm$0.10 |
|  | 3-2 |  45796.431$\pm$0.001 |  0.49$\pm$0.08 |  5.70$\pm$0.09 |  1.11$\pm$0.21 |  0.42$\pm$0.10 |
|  | 1-2 |  45796.732$\pm$0.001 |  0.18$\pm$0.05 |  5.99$\pm$0.08 |  0.53$\pm$0.14 |  0.32$\pm$0.10 |
| $5_{1,5}-4_{1,4}$ | 5-5 |  46265.718$\pm$0.001 |  0.89$\pm$0.09 |  5.87$\pm$0.03 |  0.56$\pm$0.05 |  1.50$\pm$0.14 |
|  | Main |  46266.971$\pm$0.001 |  77.60$\pm$0.11 |  6.05$\pm$0.01 |  0.87$\pm$0.01 |  83.80$\pm$0.14 | D
|  | 4-4 |  46268.396$\pm$0.001 |  0.87$\pm$0.08 |  5.78$\pm$0.03 |  0.55$\pm$0.06 |  1.50$\pm$0.14 |
| $3_{1,2}-3_{0,3}$ | 2-2 |  46493.246$\pm$0.001 |  0.81$\pm$0.11 |  5.77$\pm$0.03 |  0.55$\pm$0.07 |  1.37$\pm$0.15 |
|  | 4-4 |  46493.381$\pm$0.001 |  2.43$\pm$0.06 |  5.73$\pm$0.01 |  0.61$\pm$0.02 |  3.78$\pm$0.15 |
|  | 3-3 |  46493.768$\pm$0.001 |  1.53$\pm$0.04 |  5.75$\pm$0.01 |  0.59$\pm$0.03 |  2.47$\pm$0.15 |
|  | 4-3 |  46494.646$\pm$0.001 |  0.52$\pm$0.12 |  6.10$\pm$0.13 |  1.12$\pm$0.22 |  0.44$\pm$0.15 | H
|  | 2-3 |  46494.953$\pm$0.001 |  0.87$\pm$0.20 |  5.79$\pm$0.20 |  1.00$\pm$0.25 |  0.82$\pm$0.15 | H
| $5_{0,5}-4_{0,4}$ | 5-5 |  47353.354$\pm$0.001 |  1.67$\pm$0.09 |  5.84$\pm$0.02 |  0.60$\pm$0.04 |  2.64$\pm$0.15 |
|  | Main |  47354.670$\pm$0.001 |  112.87$\pm$0.10 |  5.94$\pm$0.01 |  0.75$\pm$0.01 |  142.33$\pm$0.15 | D
|  | 4-4 |  47356.233$\pm$0.001 |  1.52$\pm$0.09 |  5.87$\pm$0.02 |  0.61$\pm$0.04 |  2.34$\pm$0.15 |
| $5_{2,4}-4_{2,3}$ | 5-4 |  47419.606$\pm$0.001 |  5.03$\pm$0.13 |  5.79$\pm$0.01 |  0.62$\pm$0.02 |  7.58$\pm$0.18 |
|  | 6-5 & 4-3 |  47419.877$\pm$0.001 |  9.22$\pm$0.13 |  5.74$\pm$0.01 |  0.62$\pm$0.02 |  14.05$\pm$0.18 | D
| $4_{1,3}-4_{0,4}$ | 5-5 & 3-3 |  47436.292$\pm$0.001 |  3.06$\pm$0.11 |  5.89$\pm$0.02 |  0.74$\pm$0.04 |  3.86$\pm$0.10 | D
|  | 4-4 |  47436.559$\pm$0.001 |  1.43$\pm$0.08 |  5.72$\pm$0.02 |  0.58$\pm$0.04 |  2.32$\pm$0.10 |
| $5_{2,3}-4_{2,2}$ | 5-4 |  47489.042$\pm$0.001 |  5.01$\pm$0.04 |  5.80$\pm$0.01 |  0.62$\pm$0.01 |  7.59$\pm$0.18 |
|  | 6-5 & 4-3 |  47489.311$\pm$0.001 |  9.44$\pm$0.04 |  5.73$\pm$0.01 |  0.63$\pm$0.01 |  14.13$\pm$0.18 | D
| $5_{1,4}-4_{1,3}$ | 4-5 |  48551.488$\pm$0.001 |  1.26$\pm$0.16 |  5.77$\pm$0.04 |  0.60$\pm$0.08 |  1.99$\pm$0.22 |
|  | Main |  48552.598$\pm$0.001 |  79.78$\pm$0.18 |  6.01$\pm$0.01 |  0.87$\pm$0.01 |  86.41$\pm$0.22 | D
|  | 4-4 |  48553.851$\pm$0.001 |  1.24$\pm$0.17 |  5.84$\pm$0.04 |  0.58$\pm$0.09 |  2.03$\pm$0.22 |
| $5_{1,4}-5_{0,5}$ | 4-4 & 6-6 |  48634.220$\pm$0.001 |  2.39$\pm$0.17 |  5.84$\pm$0.03 |  0.74$\pm$0.07 |  3.05$\pm$0.19 | D
|  | 5-5 |  48634.427$\pm$0.001 |  1.70$\pm$0.17 |  5.79$\pm$0.04 |  0.73$\pm$0.09 |  2.19$\pm$0.19 |
| $8_{1,8}-7_{1,7}$ | Main |  73981.555$\pm$0.001 |  39.79$\pm$2.58 |  5.80$\pm$0.02 |  0.53$\pm$0.04 |  70.52$\pm$5.36 |
| $8_{0,8}-7_{0,7}$ | Main |  75585.693$\pm$0.001 |  60.11$\pm$1.90 |  5.78$\pm$0.01 |  0.55$\pm$0.02 |  103.47$\pm$3.93 |
| $8_{2,7}-7_{2,6}$ | Main |  75838.862$\pm$0.001 |  13.07$\pm$2.11 |  5.79$\pm$0.04 |  0.62$\pm$0.14 |  19.67$\pm$2.89 | H
| $8_{2,6}-7_{2,5}$ | Main |  76128.883$\pm$0.001 |  5.25$\pm$0.85 |  5.82$\pm$0.03 |  0.38$\pm$0.06 |  13.14$\pm$2.07 |
| $4_{1,4}-3_{0,3}$ | 3-2 |  80768.685$\pm$0.001 |  2.30$\pm$0.42 |  5.83$\pm$0.01 |  0.56$\pm$0.09 |  3.86$\pm$0.51 |
|  | 5-4 & 4-3 |  80768.827$\pm$0.001 |  5.04$\pm$0.44 |  6.36$\pm$0.05 |  0.65$\pm$0.06 |  7.28$\pm$0.51 |
| $9_{1,9}-8_{1,8}$ | Main |  83207.507$\pm$0.001 |  21.83$\pm$0.34 |  5.80$\pm$0.01 |  0.57$\pm$0.01 |  35.96$\pm$0.70 |
| $9_{0,9}-8_{0,8}$ | Main |  84946.003$\pm$0.001 |  26.90$\pm$0.85 |  5.81$\pm$0.01 |  0.50$\pm$0.02 |  60.18$\pm$1.26 |
| $9_{2,8}-8_{2,7}$ | Main |  85302.649$\pm$0.001 |  5.23$\pm$0.95 |  5.73$\pm$0.03 |  0.35$\pm$0.07 |  13.94$\pm$2.51 |
| $9_{2,7}-8_{2,6}$ | Main |  85715.426$\pm$0.001 |  15.79$\pm$3.37 |  5.75$\pm$0.12 |  1.22$\pm$0.22 |  12.12$\pm$2.92 | H
| $9_{1,8}-8_{1,7}$ | Main |  83712.818$\pm$0.001 |  18.47$\pm$1.40 |  5.82$\pm$0.02 |  0.50$\pm$0.04 |  34.86$\pm$3.26 |
| $5_{1,4}-4_{0,4}$ | 4-3 |  89130.841$\pm$0.001 |  7.12$\pm$1.40 |  5.70$\pm$0.12 |  1.42$\pm$0.04 |  4.72$\pm$3.26 | H
| $5_{1,4}-4_{0,4}$ | 6-5 & 5-4 |  89130.919$\pm$0.001 |  2.99$\pm$0.99 |  5.76$\pm$0.03 |  0.37$\pm$0.08 |  7.69$\pm$1.37 | D,H
| $10_{1,10}-9_{1,9}$ | Main |  92426.251$\pm$0.001 |  11.50$\pm$0.73 |  5.80$\pm$0.02 |  0.49$\pm$0.03 |  21.94$\pm$1.72 |
| $10_{0,10}-9_{0,9}$ | Main |  94276.637$\pm$0.001 |  15.45$\pm$1.06 |  5.81$\pm$0.02 |  0.61$\pm$0.05 |  23.89$\pm$1.98 |
| $10_{2,9}-9_{2,8}$ | Main |  94760.783$\pm$0.001 |  3.59$\pm$0.59 |  5.82$\pm$0.04 |  0.49$\pm$0.10 |  6.96$\pm$1.23 |
| $10_{2,8}-9_{2,7}$ | Main |  95325.476$\pm$0.001 |  2.72$\pm$0.44 |  5.91$\pm$0.06 |  0.74$\pm$0.16 |  3.45$\pm$0.68 |
| $10_{1,9}-9_{1,8}$ | Main |  96982.440$\pm$0.001 |  8.04$\pm$0.28 |  5.80$\pm$0.01 |  0.54$\pm$0.02 |  13.96$\pm$0.56 |
| $6_{1,6}-5_{0,5}$ | Main |  97286.826$\pm$0.001 |  4.73$\pm$0.35 |  5.80$\pm$0.02 |  0.60$\pm$0.05 |  7.44$\pm$0.66 |
| $11_{1,11}-10_{1,10}$ | Main |  101637.233$\pm$0.001 |  3.66$\pm$0.94 |  6.01$\pm$0.06 |  0.39$\pm$0.14 |  8.84$\pm$2.03 | F
| $11_{0,11}-10_{0,10}$ | Main |  103575.398$\pm$0.001 |  4.31$\pm$1.54 |  6.03$\pm$0.11 |  0.54$\pm$0.18 |  7.45$\pm$3.97 | F
| $11_{1,10}-10_{1,9}$ | Main |  106641.391$\pm$0.001 |  8.18$\pm$1.71 |  5.67$\pm$0.08 |  0.73$\pm$0.18 |  10.39$\pm$3.53 | F
13CH2CHCN | $4_{1,4}-3_{1,3}$ | 4-3 |  36058.327$\pm$0.001 |  0.20$\pm$0.09 |  5.47$\pm$0.11 |  0.47$\pm$0.42 |  0.40$\pm$0.20 | F
| $4_{1,4}-3_{1,3}$ | 3-2 & 5-4 |  36058.479$\pm$0.001 |  0.23$\pm$0.32 |  5.97$\pm$0.43 |  0.58$\pm$0.66 |  0.37$\pm$0.20 | D
| $4_{0,4}-3_{0,3}$ | 3-2 |  36910.202$\pm$0.001 |  0.33$\pm$0.28 |  5.72$\pm$0.27 |  0.64$\pm$0.20 |  0.48$\pm$0.12 |
| $4_{0,4}-3_{0,3}$ | 4-3 & 5-4 |  36910.311$\pm$0.001 |  1.06$\pm$0.30 |  5.82$\pm$0.10 |  0.72$\pm$0.20 |  1.39$\pm$0.12 | D
| $4_{1,3}-3_{1,2}$ | 4-3 |  37815.361$\pm$0.001 |  0.30$\pm$0.09 |  5.76$\pm$0.10 |  0.70$\pm$0.21 |  0.40$\pm$0.10 |
| $4_{1,3}-3_{1,2}$ | 3-2 & 5-4 |  37815.510$\pm$0.001 |  0.65$\pm$0.10 |  5.86$\pm$0.05 |  0.76$\pm$0.08 |  0.81$\pm$0.10 | D
| $5_{1,5}-4_{1,4}$ | Main |  45066.772$\pm$0.001 |  0.70$\pm$0.24 |  6.21$\pm$0.15 |  0.82$\pm$0.33 |  0.80$\pm$0.34 |
| $5_{0,5}-4_{0,4}$ | Main |  46113.185$\pm$0.001 |  1.21$\pm$0.11 |  5.97$\pm$0.03 |  0.73$\pm$0.08 |  1.57$\pm$0.18 |
| $5_{1,4}-4_{1,3}$ | Main |  47262.702$\pm$0.001 |  0.81$\pm$0.13 |  6.10$\pm$0.06 |  0.76$\pm$0.14 |  1.00$\pm$0.21 |
CH213CHCN | $4_{1,4}-3_{1,3}$ | 4-3 |  36803.089$\pm$0.001 |  0.56$\pm$0.53 |  5.58$\pm$0.52 |  1.17$\pm$1.12 |  0.45$\pm$0.15 | F, H
| $4_{1,4}-3_{1,3}$ | 3-2 & 5-4 |  36803.242$\pm$0.001 |  0.36$\pm$0.28 |  6.03$\pm$0.14 |  0.62$\pm$0.31 |  0.55$\pm$0.15 | D, F
| $4_{0,4}-3_{0,3}$ | 3-2 |  37700.144$\pm$0.001 |  0.24$\pm$0.23 |  5.71$\pm$0.40 |  0.76$\pm$0.59 |  0.29$\pm$0.12 |
| $4_{0,4}-3_{0,3}$ | 4-3 & 5-4 |  37700.255$\pm$0.001 |  0.86$\pm$0.24 |  5.80$\pm$0.08 |  0.65$\pm$0.13 |  1.24$\pm$0.12 | D
| $4_{1,3}-3_{1,2}$ | 4-3 |  38657.133$\pm$0.001 |  0.41$\pm$0.06 |  6.23$\pm$0.07 |  0.88$\pm$0.18 |  0.43$\pm$0.08 | H
| $4_{1,3}-3_{1,2}$ | 3-2 & 5-4 |  38657.283$\pm$0.001 |  0.66$\pm$0.05 |  5.88$\pm$0.02 |  0.71$\pm$0.03 |  0.87$\pm$0.08 | D
| $5_{1,5}-4_{1,4}$ | Main |  45996.957$\pm$0.001 |  0.93$\pm$0.18 |  6.14$\pm$0.16 |  1.66$\pm$0.40 |  0.53$\pm$0.18 | F
| $5_{0,5}-4_{0,4}$ | Main |  47097.459$\pm$0.001 |  0.94$\pm$0.10 |  5.97$\pm$0.03 |  0.65$\pm$0.07 |  1.37$\pm$0.16 |
| $5_{1,4}-4_{1,3}$ | Main |  48314.102$\pm$0.001 |  1.17$\pm$0.28 |  6.13$\pm$0.16 |  1.27$\pm$0.35 |  0.87$\pm$0.25 | H
CH2CH13CN | $4_{1,4}-3_{1,3}$ | 4-3 |  36858.005$\pm$0.001 |  0.46$\pm$0.09 |  5.70$\pm$0.07 |  0.65$\pm$0.13 |  0.67$\pm$0.08 |
| $4_{1,4}-3_{1,3}$ | 3-2 & 5-4 |  36858.158$\pm$0.001 |  0.70$\pm$0.10 |  5.95$\pm$0.06 |  0.82$\pm$0.13 |  0.81$\pm$0.08 | D
| $4_{0,4}-3_{0,3}$ | 3-2 |  37737.209$\pm$0.001 |  1.08$\pm$0.30 |  5.68$\pm$0.14 |  0.97$\pm$0.23 |  1.04$\pm$0.12 |
| $4_{0,4}-3_{0,3}$ | 4-3 & 5-4 |  37737.318$\pm$0.001 |  0.95$\pm$0.25 |  5.82$\pm$0.05 |  0.54$\pm$0.08 |  1.64$\pm$0.12 | D
| $4_{1,3}-3_{1,2}$ | 4-3 |  38672.519$\pm$0.001 |  0.53$\pm$0.11 |  5.76$\pm$0.08 |  0.80$\pm$0.20 |  0.62$\pm$0.12 |
| $4_{1,3}-3_{1,2}$ | 3-2 & 5-4 |  38672.669$\pm$0.001 |  0.82$\pm$0.12 |  5.85$\pm$0.05 |  0.70$\pm$0.10 |  1.11$\pm$0.12 | D
| $5_{1,5}-4_{1,4}$ | Main |  46066.040$\pm$0.002 |  1.38$\pm$0.09 |  6.01$\pm$0.03 |  0.97$\pm$0.07 |  1.33$\pm$0.13 |
| $5_{0,5}-4_{0,4}$ | Main |  47145.610$\pm$0.003 |  1.95$\pm$0.14 |  5.92$\pm$0.03 |  0.91$\pm$0.08 |  2.02$\pm$0.20 |
| $5_{1,4}-4_{1,3}$ | Main |  48333.801$\pm$0.002 |  1.19$\pm$0.14 |  6.13$\pm$0.06 |  0.89$\pm$0.11 |  1.25$\pm$0.22 |
CH2CDCN | $4_{1,4}-3_{1,3}$ | 4-3 |  36186.447$\pm$0.002 |  0.19$\pm$0.07 |  5.76$\pm$0.07 |  0.44$\pm$0.27 |  0.41$\pm$0.12 | F
| $4_{1,4}-3_{1,3}$ | 3-2 & 5-4 |  36186.600$\pm$0.002 | … | … | … | … | D, I
| $4_{0,4}-3_{0,3}$ | Main |  37227.628$\pm$0.002 |  0.97$\pm$0.08 |  5.87$\pm$0.05 |  0.90$\pm$0.13 |  0.73$\pm$0.09 |
| $4_{1,3}-3_{1,2}$ | 4-3 |  38369.798$\pm$0.002 |  0.31$\pm$0.17 |  5.73$\pm$0.12 |  0.70$\pm$0.28 |  0.42$\pm$0.09 |
| $4_{1,3}-3_{1,2}$ | 3-2 & 5-4 |  38369.948$\pm$0.002 |  0.66$\pm$0.07 |  5.82$\pm$0.05 |  1.03$\pm$0.08 |  0.60$\pm$0.09 | D
| $5_{1,5}-4_{1,4}$ | Main |  45221.454$\pm$0.002 |  0.94$\pm$0.08 |  5.90$\pm$0.04 |  1.06$\pm$0.11 |  0.84$\pm$0.11 |
| $5_{0,5}-4_{0,4}$ | Main |  46487.192$\pm$0.002 |  0.59$\pm$0.09 |  5.93$\pm$0.06 |  0.80$\pm$0.16 |  0.70$\pm$0.13 |
| $5_{1,4}-4_{1,3}$ | Main |  47949.865$\pm$0.002 |  0.16$\pm$0.07 |  6.43$\pm$0.03 |  0.24$\pm$0.57 |  0.61$\pm$0.16 | H
HCDCHCN | $4_{1,4}-3_{1,3}$ | 4-3 |  35817.812$\pm$0.021 |  0.23$\pm$0.05 |  6.09$\pm$0.06 |  0.52$\pm$0.17 |  0.41$\pm$0.09 | F
| $4_{1,4}-3_{1,3}$ | 3-2 & 5-4 |  35817.965$\pm$0.021 |  0.28$\pm$0.06 |  5.96$\pm$0.07 |  0.62$\pm$0.15 |  0.43$\pm$0.09 | D
| $4_{0,4}-3_{0,3}$ | Main |  36806.410$\pm$0.041 |  0.91$\pm$0.12 |  5.83$\pm$0.05 |  0.84$\pm$0.13 |  1.02$\pm$0.15 |
| $4_{1,3}-3_{1,2}$ | 4-3 |  37882.137$\pm$0.021 |  0.22$\pm$0.15 |  5.44$\pm$0.14 |  0.60$\pm$0.28 |  0.34$\pm$0.12 |
| $4_{1,3}-3_{1,2}$ | 3-2 & 5-4 |  37882.286$\pm$0.021 | … | … | … | … | D, I
| $5_{1,5}-4_{1,4}$ | Main |  44762.173$\pm$0.036 |  0.56$\pm$0.19 |  5.98$\pm$0.12 |  0.72$\pm$0.18 |  0.73$\pm$0.12 | H
| $5_{0,5}-4_{0,4}$ | Main |  45967.195$\pm$0.070 |  1.07$\pm$0.21 |  5.98$\pm$0.06 |  0.61$\pm$0.12 |  1.64$\pm$0.16 | H
| $5_{1,4}-4_{1,3}$ | Main |  47341.909$\pm$0.036 |  0.27$\pm$0.19 |  6.10$\pm$0.32 |  0.82$\pm$0.58 |  0.31$\pm$0.17 | F, H
DCHCHCN | $4_{1,4}-3_{1,3}$ | 4-3 |  34701.375$\pm$0.014 |  0.07$\pm$0.03 |  5.80$\pm$0.07 |  0.33$\pm$1.37 |  0.21$\pm$0.09 | F
| $4_{1,4}-3_{1,3}$ | 3-2 & 5-4 |  34701.527$\pm$0.014 | … | … | … | … | D, I
| $4_{0,4}-3_{0,3}$ | Main |  35480.289$\pm$0.019 | … | … | … |  $\leq$0.60 |
| $4_{1,3}-3_{1,2}$ | 4-3 |  36302.640$\pm$0.014 |  0.19$\pm$0.05 |  6.40$\pm$0.07 |  0.50$\pm$0.13 |  0.35$\pm$0.08 | F
| $4_{1,3}-3_{1,2}$ | 3-2 & 5-4 |  36302.789$\pm$0.014 |  0.33$\pm$0.06 |  6.49$\pm$0.07 |  0.78$\pm$0.16 |  0.40$\pm$0.08 | F, D
| $5_{1,5}-4_{1,4}$ | Main |  43371.731$\pm$0.029 |  0.59$\pm$0.09 |  6.32$\pm$0.08 |  0.98$\pm$0.14 |  0.57$\pm$0.13 |
| $5_{0,5}-4_{0,4}$ | Main |  44330.192$\pm$0.037 |  1.12$\pm$0.13 |  6.01$\pm$0.05 |  0.89$\pm$0.13 |  1.19$\pm$0.20 |
| $5_{1,4}-4_{1,3}$ | Main |  45373.023$\pm$0.029 |  0.63$\pm$0.18 |  6.42$\pm$0.19 |  1.26$\pm$0.44 |  0.47$\pm$0.22 | F
CH2CHC15N | $4_{1,4}-3_{1,3}$ |  |  35955.662$\pm$0.003 |  0.46$\pm$0.07 |  5.62$\pm$0.06 |  0.74$\pm$0.12 |  0.59$\pm$0.09 |
| $4_{0,4}-3_{0,3}$ |  |  36795.189$\pm$0.003 | … | … | … | … | J
| $4_{1,3}-3_{1,3}$ |  |  37685.857$\pm$0.003 |  0.46$\pm$0.10 |  5.82$\pm$0.09 |  0.83$\pm$0.21 |  0.52$\pm$0.13 |
| $5_{1,5}-4_{1,4}$ |  |  44938.534$\pm$0.005 |  0.40$\pm$0.16 |  5.64$\pm$0.15 |  0.81$\pm$0.21 |  0.46$\pm$0.20 | F, H
| $5_{0,5}-4_{0,4}$ |  |  45970.298$\pm$0.007 |  0.28$\pm$0.09 |  5.87$\pm$0.07 |  0.44$\pm$0.17 |  0.59$\pm$0.17 |
| $5_{1,4}-4_{1,3}$ |  |  45970.298$\pm$0.007 | … | … | … | … | I
Table 2: continued. 555 $a$$a$footnotetext: Rotational quantum numbers.
$b$$b$footnotetext: Total angular momentum including the spin of N. When
”Main” is indicated, it refers to the collapsed hyperfine components with
$F_{u}-F_{l}$=$J+1$$\rightarrow$$J$, $J$$\rightarrow$$J-1$, and
$J-1$$\rightarrow$$J-2$. In the latter case, the rest frequency corresponds to
that of the strongest component.
$c$$c$footnotetext: Rest frequencies used to derive the vLSR. If the
uncertainty on vLSR is zero, then the frequencies correspond to those derived
assuming vLSR=5.83 km s-1 (Cernicharo et al. 2020).
$d$$d$footnotetext: Integrated line intensity in mK km s-1.
$e$$e$footnotetext: Velocity of the line with respect to the local standard of
rest in km s-1.
$f$$f$footnotetext: Line width at half-intensity using a Gaussian fit in the
line profile (in km s-1).
$g$$g$footnotetext: Antenna temperature (in mK). Upper limits correspond to
3$\sigma$ values.
$A$$A$footnotetext: The line is partially blended with an unknown feature (see
Fig. 1). The line width has been fixed.
$B$$B$footnotetext: Data only from the set with frequency-switching throw of
10 MHz.
$C$$C$footnotetext: Blend of the hyperfine components of the same rotational
transition. If the uncertainty on the velocity is zero, then the frequencies
were fixed to the predicted ones and the line width was also fixed. The line
parameters are correlated. The value for the total integrated intensity of the
line, however, is well derived.
$D$$D$footnotetext: Unresolved hyperfine structure. The frequency of the
strongest component was taken as reference. Hence, the derived velocity
correspond to an average of the blended hyperfine components.
$E$$E$footnotetext: Data only from the set with frequency switching throw of 8
MHz.
$F$$F$footnotetext: Line detected at 3$\sigma$ level. The line parameters are
uncertain.
$G$$G$footnotetext: Blended with a negative feature produced in the folding of
the frequency switching data.
$H$$H$footnotetext: Blended with another feature. A fit is still possible, but
the derived line parameters are uncertain.
$I$$I$footnotetext: Blended with an unknown feature.
$J$$J$footnotetext: Blended with CH3CN.


11institutetext: Instituto Superior Técnico
Lisboa, Portugal

# Learning to compute inner consensus
A novel approach to modeling agreement between Capsules††thanks: Supported by
Calouste Gulbenkian Foundation

Gonçalo Faria
<EMAIL_ADDRESS>
###### Abstract

This project considers Capsule Networks, a recently introduced machine
learning model that has shown promising results regarding generalization and
preservation of spatial information with few parameters. The Capsule Network’s
inner routing procedures thus far proposed, a priori, establish how the
routing relations are modeled, which limits the expressiveness of the
underlying model. In this project, I propose two distinct ways in which the
routing procedure can be learned like any other network parameter.

##  1 Introduction

Starting with the developments made by Frank Rosenblatt surrounding the
Perceptron algorithm [1], innovative techniques have marked the beginning of a
new, biologically inspired, approach in Artificial Intelligence that is,
surprisingly, better suited to deal with naturally unstructured data.

The now called field of Deep Learning has expanded these ideas by creating
models that stack multiple layers of Perceptrons. These Multilayer
Perceptrons, commonly known as Neural Networks, achieve greater representation
capacity, due to the layered manner the computational complexity is added,
especially when compared with its precursor. Attributable to this
compositional approach they are especially hard-wired to learn a nested
hierarchy of concepts.

Aided by the increase in computational power as well as efforts to collect
high quality labeled data, in the last decade, a particular subset of Neural
Networks, called Convolutional Neural Networks(CNN) [2], have accomplished
remarkable results [3]. As their common trait, having to deal with high
dimension unstructured data, computer vision [3], speech recognition [4],
natural language processing [5], machine translation [6] and medical image
analysis [7, 8] are the fields in which these models have shown greater
applicability.

Colloquially, a CNN as presented by Yann LeCun and others, is a model that
uses multiple layers of feature detectors that have local receptive fields and
shared parameters interleaved with sub-sampling layers [3, 9, 10]. For
attaining translation invariance, by design, these sub-sampling layers discard
spatial information [11], which, when applied to the classification task,
assist in amplifying the aspects of its input that are useful for
discriminating and suppress irrelevant variations that are not.

Translation invariance, however helpful in attaining a model that has the same
classification when applied to entities in different viewpoints, inevitably
requires training on lots of redundant data. This redundancy is artificially
introduced to force the optimization process to find solutions that can not
distinguish between different viewpoints of the same entity. Additionally,
disregarding spatial information produces models incapable of dealing with
recognition tasks, such as facial identity recognition, that require knowledge
of the precise spatial relationships between high-level parts, like a nose or
a mouth.

To address these drawbacks, adaptations on CNNs have been proposed. This
project focuses on improving an existing equivariant approach, introduced by
Sara Sabour, Geoffrey E. Hinton and Nicholas Frosst, called Capsule Networks
[12, 13].

The Capsule Network model proposed by Sabour et al. [12] uses, layer wise,
dynamic routing. Since dynamic routing is a sequential extremely
computationally expensive procedure, if the network were to be scaled, in
order to be suited to solve more challenging datasets, would incur in overly
expensive costs in training and inference time. Furthermore, when applied to
many capsules, the gradient flow through the dynamic routing computations is
dampened. This inhibits learning, regardless of the computational resources
used. Additionally, both dynamic routing as well as other routing procedures
proposed, a priori, establish the way in which the routing relations are
modeled, which limits the expressiveness of the underlying model.

In this work, I propose two distinct ways in which the routing procedure can
be discriminatively learned. In parallel with [13], I employ routing to local
receptive fields with parameter sharing, in order to reduce vanishing
gradients, take advantage of the fact that correlated capsules tend to
concentrate in local regions and reduce the number of model parameters.

##  2 Related Work

Capsules were first introduced in [11], whereas the logic of encoding
instantiation parameters was established in a transforming autoencoder.

More recently, further work on capsules [12] garnered some attention achieving
state-of-the-art performance on MNIST, with a shallow Capsule Network using an
algorithm named Dynamic routing.

Shortly thereafter, a new routing algorithm, based on an Expectation-
Maximization extension applied to a Gaussian Mixing Model, was proposed in
[13]. Additionally, capsule vectors were replaced by matrices to reduce the
number of parameters and also convolutional capsules were introduced. State-
of-the-art performance was achieved on the smallNORB dataset using a
relatively small Capsule Network.

An analysis of Dynamic routing algorithm as an optimization problem was
presented in [14] as well as a discussion of possible ways to improve Capsule
Networks. Furthermore, researchers have proposed several extentions [15, 16,
17, 18, 19, 20, 21, 22] showing that there is still room for improvement.

##  3  Capsule Networks

In parallel with a neural network, a Capsule Network [12, 13] is, in essence,
multiple levels of capsule layers which, in part, are composed of various
capsules. A capsule is a group of artificial neurons which learns to recognise
an implicitly defined entity, in the network’s input, and outputs a tensor
representation, the pose, which captures the properties of that entity
relative to an implicitly defined canonical version and an activation
probability. The activation probability was designed to express the presence
of the entity the capsule represents in the network’s input. In the high-level
capsules this activation probability corresponds to the inter-class invariant
discriminator used for classification. Moreover, every capsule’s pose is
equivariant, meaning that as the entity moves over the appearance manifold,
the pose moves by a corresponding amount.

Every capsule layer is either calculated from multiple feature maps, when they
correspond to the primary capsule layer, or by a series of transformations,
followed by a mechanism called routing, applied to the outputs of the previous
capsule layer.

Disentangling the internal representations in viewpoint invariant, presence
probability and viewpoint equivariant, instantiation parameters may prove to
be a more generalizable approach for representing knowledge than the
conventional CNN view, that only strives for representational invariance.
Early evidence of this has been revealed by experimenting on the effects of
varying individual components of the individual capsules. The MNIST trained
network presented in [12], without being explicitly designed to, learned to
capture properties such as stroke, width and skew. In addition to this, as
mentioned by Hinton and others [12], the same network, despite never being
trained with digits, that were subject to affine transformations, was able to
accurately classify them during test time, which seems to suggest greater
generalization capacity when in comparison with conventional CNNs.

###  3.1 Routing Procedure

Routing consists of a dot-product self-attention [23] procedure that assigns,
for each output capsule, a distribution of compatibility probabilities to the
transformed previous layer’s capsules, the capsule votes. These compatibility
probabilities, after multiplied to corresponding capsule votes are combined
and result in the output capsule’s pose. Furthermore, the routing procedure
also assigns the activation probability to the respective output capsule,
usually based on the amount of agreement between the votes with higher
compatibility probabilities.

This procedure provides the framework for a consensus mechanism in the multi-
layer capsule hierarchy, which, for the higher levels of the input’s domain,
will solve the problem of assigning parts to wholes. Additionally, it can be
applied to local receptive fields with shared transformation matrices. Figure
1 contains a diagram representing how routing is applied convolutionally in
one dimension. The 2D and 3D convolutional routing is extrapolated in the same
manner as the usual 2D and 3D convolution would.

Figure 1: 1D Convolutional Routing(adapted from [24]). Respectively,
$P_{i}^{l}$ and $a_{i}^{l}$, denote the pose and activation corresponding to
the $ith$ capsule in the $l$ capsule layer.

The vote transformations are traditionally linear transformations. However,
due to some stability issues that occur during training, equation 1, proposed
by [14], will be used instead. The capsule vote from the $ith$ input capsule
to the $jth$ output capsule is obtained by multiplying the matrix $W_{ij}$ to
the pose of the $ith$ input capsule $P_{i}$ divided by the frobenius norm of
$W_{ij}$.

$v_{ij}=\frac{W_{ij}}{\|W_{ij}\|_{\mathcal{F}}}P_{i}$ (1)

In algorithm 1, I present a generalization of the routing procedure that
encompasses most of the routing algorithms proposed. The definition of the
$activation$ and $compatibility$ functions is what caracterizes the particular
routing procedure.

The routing algorithm returns the output capsule’s pose $\mu$ and activation
$p$ given a subset of capsules $\gamma$. For some capsule $i\in\gamma$ the
capsule activation is represented by $a_{i}$, the capsule vote is represented
by $v_{i}$ and the compatibility probability is represented by $c_{i}$. The
set of all $v_{i}$, $a_{i}$ and $c_{i}$ is represented, respectively, by $V$,
$A$, $C$. The symbol $S^{t}$ represents some state values at timestep $t$ that
might be shared either by the compatibility function across iterations or by
the compatibility and activation functions.

Algorithm 1 Generic Routing Mechanism

1:procedure Routing($V,A,\gamma$)

2: $\forall i\in\gamma:c_{i}\leftarrow 1/\|\gamma\|$

3: $s^{0}\leftarrow s_{0}$

4: $\mu\leftarrow\frac{\sum_{i\in\gamma}c_{i}v_{i}}{\sum{i\in\gamma}c_{i}}$

5: $p\leftarrow activation_{\beta}(\mu,s,V,c,\gamma)$

6: for $t$ iterations do

7: $\forall i\in\gamma:c_{i},s_{i}^{t}\leftarrow
compatibility_{\theta}(c_{i},s_{i}^{t-1},v_{i},a_{i},\mu,p)$

8: $\mu\leftarrow\frac{\sum_{i\in\gamma}c_{i}v_{i}}{\sum{i\in\gamma}c_{i}}$

9: $p\leftarrow activation_{\beta}(\mu,S^{t},V,C,\gamma,A)$

10: end for

11: return $\mu,p$

12:end procedure

##  4 Learning the Routing Procedure

In hopes of improving the performance of Capsule Networks and of incorporating
routing into the whole training process, instead of designing an alternative
routing procedure, that necessarily constrains the parts to whole
relationships that can be modeled, I present methods for parameterizing it,
such that, either for each layer or each network, the routing procedure itself
can be discriminatively learned like any other model parameter.

In the following subsections I present two distinct alternatives. The first
one exposes routing as a classic clustering algorithm based on the application
of parametric kernel functions. The second takes a less structured approach
that defines the activation and compatibility functions simply as neural
networks.

###  4.1 Similarity Learning Approach

Since for each input, in every output capsule, the routing computations are
reminiscent of an agglomerative fuzzy clustering algorithm, following the
avenue taken in [14], in this subsection, routing it’s analyzed as the
optimization of a clustering-like objective function. The resulting cluster is
interpreted as the agreement over the capsule votes and is used as the routing
procedure’s output capsule’s pose. Similarly with [14] I propose 2, inspired
by the algorithm presented in [25].

$\min\mathcal{L}(C,\mu)=-\sum_{i=1}^{n}c_{i}\langle\mu,v_{i}\rangle_{\theta}+\lambda_{1}D_{KL}(C\parallel
U)+\lambda_{2}D_{KL}(C\parallel A)$ (2)

subject to

$\sum_{i=1}^{n}c_{i}=1,c_{i}\in[0,1],1\leq i\leq n$ (3)

where $\langle\cdot,\cdot\rangle_{\theta}$ is a kernel function defined by
$\theta$, $\mu$ is the output capsule’s pose, $v_{i}$, $a_{i}$ and $c_{i}$
are, respectively, the $ith$ input capsule vote, its activation and
compatibility probability. $A=[a_{i}]$ and $C=[c_{i}]$ are vectors of n
components. The compatibility probability corresponds to the weight of the
$ith$ input capsule vote has in the output capsule’s pose. The parameters
$\lambda_{1}$ and $\lambda_{2}$, which are both positive or equal to zero, are
the weights of the penalty terms applied to $C$. The symbol $U$ denotes the
uniform distribution.

The first term in the objective function is the weighted average of the
similarity between the output pose and the $ith$ input votes. The weights are
the compatibility probabilities. In this way, the more compatible an input
capsule is the more significant its vote similarity is in the optimization
process. The remaining terms are penalty terms weighted by $\lambda_{1}$ and
$\lambda_{2}$. The second term, as in [25], is proportional to the
Kullback–Leibler(KL) divergence[26] of the compatibility probabilities and the
uniform distribution. Additionally, the third term is proportional to the KL
diverge between the compatibility probabilities and the activations. In other
words, the expectation of the logarithmic difference between the $A$ and $C$,
where the expectation is taken with respect to $C$. The penalty terms were
introduced in order to pay, independently, both for the nonalignment between
the activations and compatibility probabilities aswell as the nonuniformity of
the compatibility probabilities.

The minimization problem is solved by partially optimizing
$\mathcal{L}(C,\mu)$ for $C$ and $\mu$.

For a fixed $C$, $\mu$ is updated as

$\mu=\sum_{i=1}^{n}c_{i}v_{i}$ (4)

For a fixed $\mu$, $C$ is updated as follows: Using the Lagrangian multiplier
technique, We obtain the unconstrained minimization problem 5.

$\tilde{\mathcal{L}}(C,\alpha)=-\sum_{i=1}^{n}c_{i}\langle\mu,v_{i}\rangle_{\theta}+\lambda_{1}D_{KL}(C\parallel
U)+\lambda_{2}D_{KL}(C\parallel
A)-\alpha(\sum_{i=1}^{n}c_{i}-1),\lambda_{2}>=0,\lambda_{1}>=0$ (5)

where $\alpha$ is the Lagrangian multiplier. If $(\hat{C},\hat{\alpha})$ is a
minimizer of $\tilde{\mathcal{L}}(C,\alpha)$ then the gradient must be zero.
Thus,

$\frac{\partial\tilde{\mathcal{L}}(\hat{C},\hat{\alpha})}{\partial
c_{i}}=-\langle\mu,v_{i}\rangle_{\theta}+(\lambda_{1}+\lambda_{2})(1+log\hat{c_{i}})-\lambda_{2}loga_{i}-\hat{\alpha}=0,1\leq
i\leq n\\\ $ (6)

and

$\frac{\partial\tilde{\mathcal{L}}(\hat{C},\hat{\alpha})}{\partial\alpha}=\sum_{i=1}^{n}\hat{c_{i}}-1=0$
(7)

From 6, we obtain

$\hat{c_{i}}=exp(\frac{\langle\mu,v_{i}\rangle_{\theta}}{\lambda_{1}+\lambda_{2}})exp(\frac{\hat{\alpha}}{\lambda_{1}+\lambda_{2}})a_{i}^{\frac{\lambda_{2}}{\lambda_{1}+\lambda_{2}}}exp(-1),1\leq
i\leq n$ (8)

By substituting 8 into 7, we have

$\sum_{i=1}^{n}\hat{c_{i}}=exp(-1)exp(\frac{\hat{\alpha}}{\lambda_{1}+\lambda_{2}})\sum_{i=1}^{n}exp(\frac{\langle\mu,v_{i}\rangle_{\theta}}{\lambda_{1}+\lambda_{2}})a_{i}^{\frac{\lambda_{2}}{\lambda_{1}+\lambda_{2}}}=1$
(9)

which, when solved for $\hat{\alpha}$, results in

$\hat{\alpha}=(\lambda_{1}+\lambda_{2})log\frac{1}{exp(-1)\sum_{j=1}exp(\frac{\langle\mu,v_{j}\rangle_{\theta}}{\lambda_{1}+\lambda_{2}})a_{j}^{\frac{\lambda_{2}}{\lambda_{1}+\lambda_{2}}}}$
(10)

It follows that by substituting 10 into 8, leading to

$\hat{c_{i}}=\frac{a_{i}^{\frac{\lambda_{2}}{\lambda_{1}+\lambda_{2}}}exp(\frac{\langle\mu,v_{i}\rangle_{\theta}}{\lambda_{1}+\lambda_{2}})}{\sum_{j=1}^{n}a_{j}^{\frac{\lambda_{2}}{\lambda_{1}+\lambda_{2}}}exp(\frac{\langle\mu,v_{j}\rangle_{\theta}}{\lambda_{1}+\lambda_{2}})},1\leq
i\leq n$ (11)

and $C$ can be updated by 11.

Figure 2, contains an illustrative example, applied to a toy dataset, of the
derived clustering algorithm for different pairs of $\lambda_{1}$ and
$\lambda_{2}$.

(a) Optimal Values of C for different values of $\lambda_{1}$ and $\lambda_{2}$. | (b) Distribution of activations.
---|---
|
Figure 2: An illustrative example of the routing Algorithm obtained by
minimizing 2 applied to a toy dataset, using cosine similarity. The position
of the bubbles corresponds to the lower level capsule votes in the higher
level capsule space. The votes were obtained by sampling two distinct
multivariate Gaussian distributions with different means and covariances. The
bubble chart in (b), presents an overview of the toy dataset. The area of the
bubbles, encodes the activation probabilities of the corresponding capsule
votes. The activation probabilities were obtained by assigning the votes to
one of the two distributions obtained after scaling each of the covariance
matrices of the vote distributions. For clarity, I normalized the activation
probabilities, however, they need not be. The bubble charts in (a), show the
the optimal values of compatibility probabilities, encoded by the area of the
blue bubbles, and output capsule’s pose,encoded by the yellow bubble, for
different values of $\lambda_{1}$ and $\lambda_{2}$.

Applied to the general routing procedure framework, presented in algorithm 1,
we can take the compatibility function to be equation 12.

$compatibility_{\theta,\lambda_{1},\lambda_{2}}(V,a_{i},\mu,\gamma,s_{i})\doteq(\frac{a_{i}^{\frac{\lambda_{2}}{\lambda_{1}+\lambda_{2}}}exp(\frac{\langle\mu,v_{i}\rangle_{\theta}}{\lambda_{1}+\lambda_{2}})}{\sum_{j\in\gamma}a_{j}^{\frac{\lambda_{2}}{\lambda_{1}+\lambda_{2}}}exp(\frac{\langle\mu,v_{j}\rangle_{\theta}}{\lambda_{1}+\lambda_{2}})},s_{i})$
(12)

In this way, we obtain a compatibility function that is parameterized by
$\theta$, $\lambda_{1}$ and $\lambda_{2}$ which are learned discriminatively
using back-propagation during the training process of the entire capsule
network. The remaining function for the definition of the routing procedure is
the activation which is present in 13.

$activation_{\beta}(\mu,V,C,\gamma)\doteq
sigmoid(\beta_{1}\sum_{i\in\gamma}c_{i}\langle\mu,v_{i}\rangle_{\theta}-\beta_{2}D_{KL}(C\parallel
A)+\beta_{3}),\beta_{2}\geq 0,\beta_{1}\geq 0$ (13)

In the context of convolutional routing, $\lambda_{1}$, $\lambda_{2}$,
$\beta_{1}$, $\beta_{2}$, and $\beta_{3}$ are distinct for each channel in
every convolutional routing layer. The parameters of the kernel function could
either be shared across convolutional channel or layer, in the following
sections, we assume they only shared across layers.

###  4.2 Connectionist Approach

An alternative approach to the more formal presented in the previous
subsection, to modeling the routing procedure, is to allow the $activation$
and $compatibility$ functions in Algorithm 1 to be Neural Networks. More
precisely I employed a LSTM cell [27], which is designed to keep track of
arbitrary long-term dependencies, in conjunction with two distinct neural
networks to obtain a learnable routing mechanism that plays an active role
throughout the iterations.

Figure 3 presents a diagram of the LSTM cell employed. The input of the cell
is a concatenation of the previous iteration intermediate output capsule pose
$\mu$, the capsule vote from the $ith$ input capsule $v_{i}$ and the
corresponding compatibility probabilities $c_{i}$ and the activation $a_{i}$.
After initialized with zeros($s_{0}=\mathbf{0}$), the cell state $s_{i}^{t}$
and hidden state $h_{i}^{t}$ are updated throughout the iterations to obtain
representations that are subsequently fed to two distinct neural networks.

The neural network $f_{\theta}$ is applied to the outputs of the hidden state
to produce the compatibility probabilities at the end of every iteration.
Eventually, at the end of the iterative process, the cell states,
correspondent to every input capsule, after combined, are fed to the neural
network $g_{\beta}$ and result in the output capsule’s final activation as
indicated by 14. The definition of the compatibility function is presented in
algorithm 2.

$activation_{\beta}(C,\gamma,S^{t})\doteq
g_{\beta}(\sum_{i\in\gamma}c_{i}s^{t}_{i})$ (14)

Algorithm 2 Connectionist Approach

1:procedure $compatibility$($c_{i},v_{i},a_{i},\mu,s_{i}^{t-1}$)

2: $x_{i}^{t}\leftarrow concat(\mu,c_{i},v_{i},a_{i})$

3: $h^{t}_{i},s_{i}^{t}\leftarrow LSTM(x_{i}^{t},s_{i}^{t-1})$

4: $c_{i}\leftarrow f_{\theta}(h^{t}_{i})$

5: return $c_{i},s^{t}$

6:end procedure

The parameters from the LSTM and both neural networks are shared across every
single input capsule. Additionally, the dimension of the LSTM’s cell state and
hidden state, as well as the architectures of the neural networks used, become
hyperparameters.

Figure 3: Diagram of of the LSTM cell used in the routing mechanism(adapted
from [28]).

##  5 Experiments

In order to evaluate the effectiveness of the proposed routing algorithms,
when compared with the extension of Expectation-Maximization(EM) applied to a
Gaussian Mixture Model, introduced in [13], experiments were conducted in both
MNIST dataset [29] and smallNORB [30]. They consist in training the same
Capsule Network architecture, for 100 epochs, with every routing procedure
running for three iterations, for both of the proposed algorithms and the one
present in [13] and comparing the respective test set results.

The MNIST dataset was selected, in the early stages, as proof of concept. It
is a large database of handwritten digits that has become the classic
benchmark for machine learning models. The training set is composed of 60000
examples and the test set 10000.

The smallNORB dataset was chosen due to it being much closer to natural images
and yet devoid of context and color. It is composed of 50 toys belonging to 5
generic categories(four-legged animals, human figures, airplanes, trucks, and
cars). The objects were imaged by two cameras under 6 lighting conditions, 9
elevations (30 to 70 degrees every 5 degrees), and 18 azimuths (0 to 340 every
20 degrees). Figure 4 contain some sampled examples of the dataset. The
training set is composed of 5 instances of each category(24300 examples), and
the test set of the remaining 5 instances(24300 examples).

For the experiments with smallNORB, this dataset was preprocessed precisely in
the same manner as in [13]. The images were normalized and downsampled to
$48\times 48$. During training, they were randomly cropped into $32\times 32$
patches and random brightness and contrast were added. During test and
validation, however, each crop was correspondent to the center of the image.

Figure 4: Examples of the classes animals, human figures and airplanes.

All the experiments where made using a Tensorflow [31] implementation of the
underlying models 111https://github.com/goncalorafaria/learning-inner-
consensus. The optimizer used was Adam [32] with a learning rate of 3-e3
scheduled by an exponential decay. Additionally, the models were trained using
kaggle notebooks, approximately 40 hours each, a free research tool for
machine learning that provides a Tesla P100 Nvidia GPU as part of an
accelerated computing environment.

Apart from the implementation of the routing procedure, all of the models
contained in the experiments instantiate the Capsule Network architecture
present in Table 1 and use 4x4 matrices as poses. The model is a slight
modification of the smaller Capsule Network used in [13]. More precisely,
after the initial convolutions batch normalization [33] is applied, and the
transformation used is the one described in equation 1. Applied to smallNORB,
the model has 86K parameters, excluding the ones pertaining to the different
routing procedures.

Layer | Details | Output shape
---|---|---
Input |  | $B\times 32\times 32\times 1$
Convolutional layer + relu + BatchNorm | K=5, S=2, Ch=64 | $B\times 16\times 16\times 64$
Primary Capsules | K=1, S=1, Ch=8 | $B\times 16\times 16\times 8\times(4\times 4+1)$
Convolutional Capsule Layer 1 | K=3, S=2, Ch=16 | $B\times 7\times 7\times 16\times(4\times 4+1)$
Convolutional Capsule Layer 2 | K=3, S=1, Ch=16 | $B\times 5\times 5\times 16\times(4\times 4+1)$
Capule Class Layer | flatten, O=5 | $B\times 1\times 1\times 5\times(4\times 4+1)$
Table 1:  Specification of the Capsule Network model used in the experiment with SmallNORB dataset. K denotes convolutional kernel size, S stride, Ch number of output chanels, O number of classes and B the batch size.  Layer | hidden layers in $f_{\theta}$ | hidden layers in $g_{\beta}$ |  number of units in the lstm’s hidden and cell states
---|---|---|---
Convolutional Capsule Layer 1 | [] | [] | 16
Convolutional Capsule Layer 2 | [32,32] | [64,64] | 16
Capule Class Layer | [64,64] | [124,124] | 16
Table 2:  The detailed specification of the hyperparameters used in the
experiments with the Connectionist approach’s routing mechanism. The
representation of the hidden layers is a list where the $ith$ list element
corresponds to the number of neurons in the $ith$ hidden layer.

The model present in table 1 contains three layers to which routing is
applied. When used in the described experiments, the routing algorithm
presented in section 4.2, used the hyperparameters described in table 2.
Moreover, the routing algorithm presented in section 4.1 used the kernel
defined in equation 15, a combination of gaussian kernels[34]. The first two
layers in the Capsule Network have Q = 4 and the last one has Q = 10.

$\langle
x,x^{\prime}\rangle_{\theta}\doteq\sum^{Q}_{q=1}\theta^{q}_{1}exp(-\frac{\|x-x^{\prime}\|^{2}}{2(\theta^{q}_{2})^{2}})$
(15)

The choice of routing hyperparameters was based on making the routing
procedure in the deeper layers have more parameters. What motivated this
design choice was the intuition that, the more complex the features, further
complex needed to be the routing. Ideally, if there were not computational
limitations, these parameters would have been chosen using a randomized grid
search with cross-validation or other more suitable hyperparameter search
method.

Routing | MNIST | smallNORB
---|---|---
Connectionist | 0.53% | 6.7%
EM [24] | (not available) | 6.3%
EM (www0wwwjs1)222https://github.com/www0wwwjs1/Matrix-Capsules-EM-Tensorflow | 0.9% | 8.2%
EM (original authors)[13] | (not available) | 2.2%
Similarity Learning | 1.0% | 8.19%
Table 3: Test set percentual error rate obtained in the experiments. During
the optimization process, starting from the tenth epoch, the parameters for
each model, were recorded(5000 instances). The reported results correspond to
the test set percentual error of the parameters that achieved higher accuracy
on the validation set(10% original training set). Given that the original
authors of EM routing did not make their implementations publicly available, I
also compare my work with the two best open-source versions that I have found.

The results pertaining to all of the experiments are contained in table 3. The
table contains the percentual error rate achieved in the evaluation of the
models.

##  6 Discussion

The results we obtained on smallNORB are far from the ones reported by [13].
However, when in comparison with the best open-source versions, the same does
not hold. I speculate that the original authors did not present all of the
needed implementation details, which would indeed explain the observed
discrepancy. When in comparision with the best open-source versions, the
Connectionist approach is on par or surpasses them while the Similarity
Learning approach does not.

The main challenge I encountered while fitting the models was the
computational time required. Although the convolutional Capsule networks
require less floating-point operations and fewer training parameters, than any
comparable CNN, they are much slower and ran out of memory with relatively
small models. This is due to Tensorflow beeing optimized for CNNs, and not
having natively the operations required to compute the Capsule’s routing
convolutions. To compute the routing Convolutions, for each native TensorFlow
operation employed, TensorFlow’s functional API makes a copy of the underlying
tensors, which greatly increases the memory requirements.

##  7 Conclusion

In this paper, I have presented two distinct ways to learn Capsule Network’s
inner routing mechanism using backpropagation. The Similarity Learning and the
Connectionist approach. The proposed methods build on the work of [12] and
[13] by augmenting the part whole relationships that can be modeled.

The experimental results show that the Connectionist approach is on par or
surpasses with the best open-source implementations of Capsule Networks using
EM while the Similarity Learning does not. When in comparison with the
original paper, with EM routing, the results of the introduced approaches fall
short. I suspect this is due to missing implementation details that are not
mentioned in the original paper. When these are made available, I expect the
results from the routing mechanisms proposed to considerably improve.

Future work will focus on additional applications, of the proposed extension
of capsule networks, in different domains and computer vision tasks,
experiment with distinct kernel parameters for distinct convolutional
channels, and study the interpretability of the proposed models. Particularly,
for applications in object detection tasks, I intend to use the capsule poses
of the last layer to more effectively identify the location of bounding boxes.
The study of the interpretability of the proposed models will be aimed at
facilitating model building and present mechanisms for verifying the
usefulness of the model predictions. Aditionally, I intend to augment the
Similarity learning approach to more general divergence functions such as the
Jensen–Tsallis divergence.

ACKNOWLEDGMENTS A special thank you to the Gulbenkian Foundation, the
scientific committee of the Artificial Intelligence Program and this project’s
tutor Professor Cesar Analide. Addittionally, I would also like to thank the
reviewers for their constructive comments which helped to improve the proposed
methodology.

## References

  * [1] Rosenblatt, F.:  The perceptron: A probabilistic model for information storage and organization in the brain.  Psychological Review (1958)
  * [2] LeCun, Y.:  Handwritten Digit Recognition with a Back-Propagation Network.  Advances in Neural Information Processing Systems (1989)
  * [3] Krizhevsky, A., Sutskever, I., Hinton, G.E.:  ImageNet Classification with Deep Convolutional Neural Networks.  In: ImageNet Classification with Deep Convolutional Neural Networks. (2012)
  * [4] Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A., Jaitly, N., … Kingsbury, B.:  Deep Neural Networks for Acoustic Modeling in Speech Recognition.  IEEE Signal Processing Magazine (2012)
  * [5] Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C.:  A Neural Probabilistic Language Model.  In: Journal of Machine Learning Research. (2003)
  * [6] Yonghui Wu, Mike Schuster, Z.C.Q.V.L.M.N.W.M.M.K.Y.C.Q.G.K.M.  (2016)
  * [7] Kleesiek, J., Urban, G., Hubert, A., Schwarz, D., Maier-Hein, K., Bendszus, M., Biller, A.:  Deep MRI brain extraction: A 3D convolutional neural network for skull stripping.  NeuroImage (2016)
  * [8] Dou, Q., Chen, H., Yu, L., Zhao, L., Qin, J., Wang, D., Mok, V.C., Shi, L., Heng, P.A.:  Automatic Detection of Cerebral Microbleeds from MR Images via 3D Convolutional Neural Networks.  IEEE Transactions on Medical Imaging (2016)
  * [9] Sermanet, P., Lecun, Y.:  Traffic sign recognition with multi-scale convolutional networks.  In: Proceedings of the International Joint Conference on Neural Networks. (2011)
  * [10] Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.:  Going deeper with convolutions.  In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. (2015)
  * [11] Hinton, G.E., Krizhevsky, A., Wang, S.D.:  Transforming auto-encoders.  In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). (2011)
  * [12] Sara Sabour, N.F., Hinton, G.E.:  Dynamic routing between capsules.  In: Advances in Neural Information Processing Systems. (2017)
  * [13] Hinton, G., Sabour, S., Frosst, N.:  Matrix capsules with EM routing.  In: International Conference on Learning Representations. (2018)
  * [14] Wang, D., Lui, Q.:  An Optimization View on Dynamic Routing Between Capsules.  Journal of Geotechnical and Geoenvironmental Engineering (2018)
  * [15] Deliège, A., Cioppa, A., Droogenbroeck, M.V.:  Hitnet: a neural network with capsules embedded in a hit-or-miss layer, extended with hybrid data augmentation and ghost capsules.  CoRR abs/1806.06519 (2018)
  * [16] Xiang, C., Zhang, L., Tang, Y., Zou, W., Xu, C.:  Ms-capsnet: A novel multi-scale capsule network.  IEEE Signal Processing Letters 25 (2018) 1850–1854
  * [17] Amer, M., Maul, T.:  Path capsule networks.  CoRR abs/1902.03760 (2019)
  * [18] O’ Neill, J.:  Siamese Capsule Networks.  arXiv e-prints (2018) arXiv:1805.07242
  * [19] Rawlinson, D., Ahmed, A., Kowadlo, G.:  Sparse unsupervised capsules generalize better.  CoRR abs/1804.06094 (2018)
  * [20] Ma, D., Wu, X.:  Tcdcaps: Visual tracking via cascaded dense capsules.  CoRR abs/1902.10054 (2019)
  * [21] do Rosario, V.M., Borin, E., Jr., M.B.:  The multi-lane capsule network (MLCN).  CoRR abs/1902.08431 (2019)
  * [22] Zhao, Y., Birdal, T., Deng, H., Tombari, F.:  3d point-capsule networks.  CoRR abs/1812.10775 (2018)
  * [23] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.:  Attention is all you need.  In: NIPS. (2017)
  * [24] Gritzman, A.D.  (2019)
  * [25] Li, M.J., Ng, M.K., Cheung, Y.M., Huang, J.Z.:  Agglomerative fuzzy K-Means clustering algorithm with selection of number of clusters.  IEEE Transactions on Knowledge and Data Engineering (2008)
  * [26] Kullback, S., Leibler, R.A.:  On information and sufficiency.  The Annals of Mathematical Statistics 22 (1951) 79–86
  * [27] Hochreiter, S., Schmidhuber, J.:  Long short-term memory.  Neural computation 9 (1997) 1735–80
  * [28] :  Understanding lstm networks.  (http://colah.github.io/posts/2015-08-Understanding-LSTMs/) Accessed: 2019-08-15.
  * [29] LeCun, Y., Cortes, C.:  The mnist database of handwritten digits.  (2005)
  * [30] LeCun, Y., Huang, F.J., Bottou, L.:  Learning methods for generic object recognition with invariance to pose and lighting.  Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. 2 (2004) II–104 Vol.2
  * [31] Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Mané, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viégas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., Zheng, X.:  TensorFlow: Large-scale machine learning on heterogeneous systems (2015) Software available from tensorflow.org.
  * [32] Kingma, D.P., Ba, J.:  Adam: A method for stochastic optimization.  CoRR abs/1412.6980 (2014)
  * [33] Ioffe, S., Szegedy, C.:  Batch normalization: Accelerating deep network training by reducing internal covariate shift.  CoRR abs/1502.03167 (2015)
  * [34] Risser, L., Vialard, F.X., Wolz, R., Murgasova, M., Holm, D.D., Rueckert, D.:  Simultaneous multi-scale registration using large deformation diffeomorphic metric mapping.  IEEE Transactions on Medical Imaging 30 (2011) 1746–1759