11institutetext: Department of Computer Science, University of Oxford, UK
11email<EMAIL_ADDRESS>
# Triple-View Feature Learning
for Medical Image Segmentation

Ziyang Wang  Irina Voiculescu

###### Abstract

Deep learning models, e.g. supervised Encoder-Decoder style networks, exhibit
promising performance in medical image segmentation, but come with a high
labelling cost. We propose TriSegNet, a semi-supervised semantic segmentation
framework. It uses triple-view feature learning on a limited amount of
labelled data and a large amount of unlabeled data. The triple-view
architecture consists of three pixel-level classifiers and a low-level shared-
weight learning module. The model is first initialized with labelled data.
Label processing, including data perturbation, confidence label voting and
unconfident label detection for annotation, enables the model to train on
labelled and unlabeled data simultaneously. The confidence of each model gets
improved through the other two views of the feature learning. This process is
repeated until each model reaches the same confidence level as its
counterparts. This strategy enables triple-view learning of generic medical
image datasets. Bespoke overlap-based and boundary-based loss functions are
tailored to the different stages of the training. The segmentation results are
evaluated on four publicly available benchmark datasets including Ultrasound,
CT, MRI, and Histology images. Repeated experiments demonstrate the
effectiveness of the proposed network compared against other semi-supervised
algorithms, across a large set of evaluation measures.

##  1 Introduction

The promising performance of deep learning for medical imaging relies not only
on network architecture engineering, but also on the availability of
sufficient high-quality manually annotated data, which is hard to come by. Co-
training, and self-training are two widely studied approaches in semi-
supervised learning.

Self-training first initializes a model with labelled data. Then the model
generates pseudo masks for unlabelled data. A condition is set for the
selection of pseudo masks, and the model is retrained by expanding its
training data [29]. Co-training is normally used to train two separate models
with two views which benefit each other [3] by expanding the size of the
training data. Deep co-training was first proposed by [18] pointing out the
challenge of ‘collapsed neural networks’: training two models on the same
dataset cannot enable multi-view feature learning because the two models will
necessarily end up similar. Output smearing, diversity augmentation, and
dropout for pseudo label editing were designed to mitigate that effect [5].

##  2 Methodology

The central idea in our method is that several (three in our case) different
views of the data and their associated learned parameters are developed in
separate models simultaneously. This gives each model a chance to complement
the others in what they contribute to the learning process. Moreover, they do
not each process the same dataset: rather, they take carefully crafted
disjoint parts of the data and learn what they can from each of those subsets.

A multi-view co-training semi-supervised learning method for classification is
also proposed in [5], but for a classification task, not for semantic
segmentation. Detailed mask prediction requires an entirely novel technique
[7, 24]. Our pseudo-labels are generated afresh at intermediate steps
selecting from amongst the output of two individual CNN components in order to
train the third one. Our confidence estimation is different from the
uncertainty-aware scheme [26, 25]: the triple framework decides which pseudo-
labels to propagate further through selecting those in which the individual
component models have had high confidence and have ‘voted’ for. This also has
the effect of increasing the overall confidence of the framework in a way that
we specifically show how to measure. Pseudo-labels with too low a confidence
level are not used in the following step. How low that level of confidence
depends on the stage of the training: the further into the training process,
the more confident the framework needs to become. Thus the number of images
available for training gets increased gradually.

The architecture of our Triple-view feature learning for medical image
semantic Segmentation Network (TriSegNet) is illustrated in Figure 1. The
label editing including unconfident label detection and confidence label
voting based on a confidence estimate improves the feature expression of un-
annotated data. Each of the three views of the feature learning makes use of
random data perturbation for regularization.

Figure 1: The Architecture of TriSegNet Framework

A pre-trained ResNet is utilized as a low-level feature learning module,
shared between the three high-level feature learning pixel-level classifiers
A, B and C. For the purpose of triple-view learning [7], the architecture and
parameters initialization are developed separately. A, B, C are Encoder-
Decoder-based. In order to transfer sufficient semantic information, they
fully model long-range dependencies, variation size of feature expression,
model skip connection [19], they bypass spatial information [4], and process
multi-scale feature maps [14]. Inspired by [9], the low-level feature learning
module is shared, and the three views of classifiers enable the ResNet to
extract low-level features in a generic manner. A, B and C not only extract
features, but also vote, and generate pseudo labels, generally benefiting each
other in the semi-supervised process.

###  2.1 Training Setup

Figure 2: Data Pipeline and Label Processing stages

We denote by $\mathcal{L}$, $\mathcal{U}$, $\mathcal{T}$ a small labelled
dataset, a large size of unlabelled dataset, and a dataset for testing. Each
batch of labeled data is $(x,y_{gt}){\in}\mathcal{L}$,
$(x,y_{gt}){\in}\mathcal{T}$; batches of only raw data are
$(x){\in}\mathcal{U}$; $y_{\mathit{pred}},y_{\mathit{pseudo}}$ is the dense
map generated by different views of TriSegNet for prediction. We split the
training process into three stages with an inference stage. Stage 1
initializes the three views of classifiers $f_{n},n={1,2,3}$ with labeled
dataset $\mathcal{L}$, repeatedly, as shown in Fig. 2(d), with data
perturbation as per Fig. 2(a). To properly initialize the three classifiers
with unbalanced data, overlap loss is used. The confidence weight during the
training process is estimated and recorded as $\alpha_{n},n={1,2,3}$ until
inference is complete. Each model estimates its confidence of the labelled
data by on randomly selected pseudo labels, also compared against the other
two models as sketched in Figure 2(c). Stage 2 learns from the generated
pseudo labels in an iterative manner. Unlabeled data $(x)\in\mathcal{U}$ are
used by $f_{n}$ as $(x,y_{n\\_pred})$ pairs. Any data corresponding to low
confidence labels gets removed. A fresh vote from another model $f_{n}$
generates pseudo label
$(x,y_{n\\_pseudo})=\alpha_{n-1}{\times}f_{n-1}(x)+\alpha_{n+1}{\times}f_{n+1}(x)$
to train the rest of the model. The loss function at this stage is mixed
overlap- and boundary-based loss. Stage 3 is to train additionally the lowest
confident classifier with $\mathcal{L}$, $\mathcal{U}$. The learning rate for
all training stages is set to $2{\times}10^{-4}$, with an Adam optimizer. The
training epoch is set to 150 in Stage 1, 100 in Stage 2, and 150 or early
stopped until the least confident classifier reaches the same confidence as
other classifiers in Stage 3. The batch size is set to 16. Inference is to
predict jointly on an unseen image $(x)\in\mathcal{T}$ to
$y_{pred}=\sum_{n=1}^{3}\alpha_{n}{\times}f_{n}(x)$.

###  2.2 Label Processing

To encourage meaningful differences among views for co-training, it is
essential to pin down each classifier’s architecture, as well as its label
editing. In TriSegNet we propose three label editing approaches: Data
perturbation, low confidence label detection, and Confidence label voting.

Data Perturbation. Mean Teacher [20] uses perturbation to assess the
consistency of the same image under different disturbances. This is extended
in Dual Student to pixel-wise tasks in guided collaborative training [13].
Naive-student [6] uses perturbation for pseudo labels in an iterative manner
to train the student network. To effect differences between the three views,
TriSegNet also uses perturbation in an iterative manner, in each training
epoch and separately for each of the views, as detailed in Fig. 2. From the
input data, 70% is selected, then 80% of this is processed by left-right and
up-down flips, and 25% has 90∘, 180∘, 270∘ rotations applied. Random
brightness and contrast changes are applied to 20% and 30% of the data
respectively. Data perturbation is used both on labelled data and on the
pseudo labels predicted from each classifier.

Low Confidence Label Detection. A badly performing view may negatively
influence the whole framework in that it predicts pseudo labels for the other
two views. Uncertainty estimation might be tackled with agreement-based
entropy-maps or using pseudo labels for training, or computing uncertainty-
weighted with a Bayesian deep network by adding dropout layers [26, 7, 25].
Instead, we mark any unreliable pseudo labels as noisy [11, 23], thus
temporarily removing low confidence labels from one view with the help of the
other two views. Retaining only high confidence pseudo labels boosts the
overall performance. A simple adaptive denoising learning strategy is to
calculate the overlap-based difference between each prediction in different
views and record it for each pseudo label in each training epoch. The higher
the label difference, the higher the probability of it not being accepted.
During each training epoch with prediction label $(x,y_{\mathit{pred}})$, our
strategy detects and removes specific number of the high difference labels
$(x,y_{\mathit{pseudo}})$ raised by the other two views. More low confidence
labels get detected at the beginning of the training iteration, and fewer
towards the end., because the training process evolves from underfitting to
overfitting. The number $N(t)$ of removed labels is

$N(t)=\left\\{\begin{array}[]{lr}0.05(1-\zeta)y,&0<t<0.01(1-\zeta)x\\\
\frac{-y}{x}t+0.06(1-\zeta)y,&~{}~{}~{}~{}~{}~{}0.01(1-\zeta)x\leq t\&t\leq
0.05(1-\zeta)x\\\ 0.01(1-\zeta)y,&0.05(1-\zeta)x<t\leq x\end{array}\right.$
(1)

where $t$ is the current training iteration, $\zeta$ is the
difference/disagreement level, $x$ is the total number of training iterations,
and $y$ is the total number of $y_{\mathit{pred}}$.

Confidence Label Voting. The uncertainty of the pseudo labels can sabotage
expanding the training data, potentially influencing the whole framework
performance. Luo [17] studied pyramid prediction network and uncertainty
rectified pyramid consistency. Yu [30] proposed a Monte Carlo Dropout to
generate an uncertainty map in student-teacher networks. Xia [26] proposed
uncertainty weighted label fusion. Unlike the student-teacher network or
single encoder-decoder networks, or directly calculating the average maximum
posterior probability of another two views, Tri-net [5] proposes a direct
confidence estimate. In our algorithm, labelled training data and 20% of un-
perturbed pseudo label data is considered as a validation set for each view to
obtain confidence weights; thus two views jointly generate pseudo labels for
the last view using confidence voting together after low confidence label
detection for cross pseudo supervision [7].

###  2.3 Loss Function

The weight of pseudo-label-based semi-supervision loss relative to labeled-
based supervised loss increases gradually during training, as the model
becomes progressively more confident in generating and using pseudo labels [7,
15, 20]. A low to high confidence strategy has been explored in uncertainty-
aware schemes [30]. TriSegNet enables coarse-to-fine model training using
gradually more precise pseudo labels: in Stage $1$ an overlap-based loss is
used (Equation 2), whereas Stages $2$ and $3$ rely on a mixed overlap- and
boundary-based loss (Equation 3). Overlap-based loss is more robust when the
datasets are imbalanced (e.g. fewer True pixels). Once the pseudo labels from
each view become dependable, the dual loss is triggered, enabling the three
views to segment more precisely [8].

$\small\mathit{Loss}_{\mathit{stage1}}=\left(1-\frac{TP+10^{-6}}{TP+\alpha
FN+\beta FP+10^{-6}}\right)^{\gamma}$ (2)

$\small\begin{split}Loss_{stage2\&3}=\underbrace{\sum_{i,j=1}^{256}\sqrt{\|\bigtriangledown
P_{x_{i,j}}+\bigtriangledown P_{y_{i,j}}+10^{-6}\|}}_{Loss_{Boundary}}+\\\
\underbrace{\|\sum_{i,j=1}^{256}P_{i,j}(1-G_{i,j})^{2}\|+\|\sum_{i,j=1}^{256}(1-P_{i,j})G_{i,j}^{2}\|}_{Loss_{Overlap}}\end{split}$
(3)

The ground truth, and pseudo label are denoted as
$G,P{\in}[0,1]^{256{\times}256}$, where 1 and 0 denote the region of interest
or the background on a $256{\times}256$ image. Constants $\alpha$, $\beta$ and
$\gamma$ control the non-linearity of the loss [16].

##  3 Experiments and Results

###  3.1 Datasets and Experimental Setup

Our experiments have been carried out on four public benchmark datasets: (1)
The Ultrasound Nerve Segmentation Kaggle Challenge [12] Identifying nerve
structures in ultrasound images is critical to inserting a patient’s pain
management catheter. We use 5,635 images of size $580{\times}420$. (2) The MRI
Cardiac Segmentation is from the Automated Cardiac Diagnosis MICCAI Challenge
2017 [2]. We use 1,203 images of size $232{\times}256$, from 150 patients. (3)
The Histology Nuclei Segmentation is a Pan-Cancer histology dataset from the
University of Warwick [10]. We use 7,901 images of size $256{\times}256$. (4)
The CT Spine set includes 10 CTs from a CSI 2014 Segmentation Challenge [27],
covering the entire thoracic and lumbar spine. We use 5,602 images of size
$512{\times}512$.

Ultrasound, MRI and Histology images have been resized to $256{\times}256$,
and CTs to $512{\times}512$. All four are pre-processed with data
perturbation. The data has been split into labelled training data, test data
and unlabeled raw training data. The test data is always 10% of the set,
selected randomly for each run. The training data is the remainder of the
dataset; for separate experiments, 2%, 5%, 10% or 20% of this is available
labelled, and the rest is available raw. The validation data is selected as a
separate random 20% sample of the training set (both labelled and not), and is
never seen by the training process.

We used Python 3.8.8, Tensorflow 2.6.0 [1] and CUDA 11.3, on four Nvidia
GeForce RTX3090 GPUs. The runtimes averaged 2–3.5 hours on the MRI data, and
6–8 hours on the Ultrasound, CT and Histology data, including the data
transfer, three training stages, inference and evaluation. Semi-supervised
learning for medical image segmentation,111https://github.com/HiLab-
git/SSL4MIS an online collection of implementations has been used in the
baseline testing models and additional Dropout.

Experiments Under the Assumption of 2% Data as Labeled Data
---
| Ultrasound Nerve | CT Spine | MRI Cardiac | Histology Nuclei
IOU | Sen | IOU | Sen | IOU | Sen | IOU | Sen
UNet | 0.1628 | 0.2020 | 0.8657 | 0.9210 | 0.3888 | 0.8351 | 0.6574 | 0.7814
Linknet | 0.0919 | 0.1280 | 0.8438 | 0.8962 | 0.1498 | 0.9329 | 0.6905 | 0.8219
FPN | 0.1227 | 0.1320 | 0.8653 | 0.8990 | 0.4802 | 0.5143 | 0.6942 | 0.8284
TriSegNet | 0.2800 | 0.3678 | 0.9526 | 0.7024 | 0.9923 | 0.8411 | 0.6946 | 0.8293
Experiments Under the Assumption of 5% Data as Labeled Data
UNet | 0.2762 | 0.3234 | 0.9207 | 0.9576 | 0.6805 | 0.8053 | 0.7075 | 0.5953
Linknet | 0.2505 | 0.2885 | 0.9020 | 0.9519 | 0.6762 | 0.7988 | 0.7552 | 0.8645
FPN | 0.2703 | 0.3093 | 0.9221 | 0.9521 | 0.7721 | 0.8351 | 0.7031 | 0.8333
TriSegNet | 0.3765 | 0.5090 | 0.9385 | 0.9660 | 0.8094 | 0.8880 | 0.7691 | 0.8757
Experiments Under the Assumption of 10% Data as Labeled Data
UNet | 0.3554 | 0.4090 | 0.9320 | 0.9660 | 0.8492 | 0.9253 | 0.8012 | 0.8880
Linknet | 0.3464 | 0.3991 | 0.9247 | 0.9527 | 0.7832 | 0.8641 | 0.7957 | 0.8862
FPN | 0.2416 | 0.4866 | 0.8316 | 0.9721 | 0.8078 | 0.8591 | 0.8034 | 0.8973
TriSegNet | 0.4260 | 0.5789 | 0.9463 | 0.9714 | 0.8545 | 0.9159 | 0.8114 | 0.8981
Experiments Under the Assumption of 20% Data as Labeled Data
UNet | 0.4352 | 0.5254 | 0.9447 | 0.9699 | 0.8984 | 0.9448 | 0.8104 | 0.9019
Linknet | 0.4333 | 0.5237 | 0.9295 | 0.9678 | 0.8712 | 0.9258 | 0.8027 | 0.8971
FPN | 0.3956 | 0.5953 | 0.7605 | 0.8920 | 0.8857 | 0.9307 | 0.8176 | 0.9094
TriSegNet | 0.4981 | 0.6528 | 0.9337 | 0.9494 | 0.9020 | 0.9459 | 0.8530 | 0.9244
Table 1: Evaluation Results on Direct Comparison between TriSegNet and
Existing Fully-Supervised Algorithms Figure 3: Sample qualitative results; Two
images are selected from each test set where the first row illustrates the raw
images. The rest of them illustrate the MS against GT where yellow, green,
red, and black represent TP, FN, FP and TN at the pixel level.

###  3.2 Evaluation and Results

Performance has been assessed using a wide range of evaluation metrics. These
include the frequently reported similarity measures: Dice, IOU, accuracy,
precision, sensitivity, specificity, as well as difference measures: relative
volume difference (RVD), Hausdorff distance (HD), average symmetric surface
distance (ASSD). To penalise mislabelled areas and avoid overly inflated
segmentation scores, we also evaluate the boundary match between the machine
segmentation (MS) and the Ground Truth (GT), using the Directed Boundary Dice
relative to GT (DBDG) and to MS (DBDM) and Symmetric Boundary Dice (SBD) [28].
Experiments first compare TriSegNet with fully-supervised baseline methods
like UNet[19], LinkNet[4] and FPN[14]. Table 1 reports the quantitative
results for each dataset, using 2%–20% of labeled data. The examples in Figure
3 illustrate these comparisons qualitatively.

Model | Dice | Acc | Pre | Rec/Sen | Spe | RVD | HD | ASSD | DBDG | DBDM | SBD
---|---|---|---|---|---|---|---|---|---|---|---
[20] | 0.895 | 0.992 | 0.909 | 0.881 | 0.996 | 0.226 | 21.876 | 4.077 | 0.561 | 0.649 | 0.599
[22] | 0.881 | 0.991 | 0.918 | 0.847 | 0.997 | 0.273 | 18.335 | 3.954 | 0.543 | 0.651 | 0.589
[31] | 0.878 | 0.990 | 0.858 | 0.899 | 0.994 | 0.332 | 24.235 | 5.046 | 0.577 | 0.564 | 0.559
[30] | 0.890 | 0.991 | 0.903 | 0.878 | 0.996 | 0.244 | 22.851 | 4.669 | 0.550 | 0.636 | 0.585
[21] | 0.899 | 0.992 | 0.909 | 0.889 | 0.996 | 0.205 | 28.388 | 4.773 | 0.582 | 0.639 | 0.605
TriSegNet | 0.932 | 0.995 | 0.934 | 0.930 | 0.997 | 0.208 | 7.831 | 2.075 | 0.712 | 0.611 | 0.657
Table 2: Direct comparison of TriSegNet with other algorithms, on the MRI Test
Set

To illustrate a wider range of evaluation measures with semi-supervised
algorithms, there is only room to report on one dataset. The MRI Cardiac data
has been chosen for this purpose. Table 2 documents the performance of
TriSegNet against Tarvainen [20], Vu [22], Zhang [31], Yu [30], Verma [21]
with UNet as backbone, showing that it outperforms previous methods under most
of the considered metrics.

In order to assess the contribution of each of the components, specifically
focused ablation experiments have been designed. They illustrate the essential
role of models A, B and C being different by considering, instead of the
three, two copies of one of them ($A{\times}2$, $A{\times}3$, etc.). Table 3
illustrates that the IOU metric is negatively influenced by such choices,
performing best when each of A, B and C is present in its own right.

Table 3: Ablation Studies on Contributions of Architecture and Modules Label Process | Dual Loss Design | Classifier A | Classifier B | Classifier C | IOU
---|---|---|---|---|---
|  |  ✓$\times$ 2 | ✓ |  | 0.8724
|  |  |  ✓$\times$ 2 | ✓ | 0.8739
|  | ✓ |  |  ✓$\times$ 2 | 0.8641
✓ |  |  ✓$\times$ 3 |  |  | 0.8666
| ✓ |  ✓$\times$ 3 |  |  | 0.8579
|  |  |  ✓$\times$ 3 |  | 0.8598
✓ |  |  |  ✓$\times$ 3 |  | 0.8605
|  |  |  |  ✓$\times$ 3 | 0.8619
✓ |  |  |  |  ✓$\times$ 3 | 0.8739
✓ |  | ✓ | ✓ | ✓ | 0.8787
| ✓ | ✓ | ✓ | ✓ | 0.8841
✓ | ✓ | ✓ | ✓ | ✓ | 0.9020

##  4 Conclusion

Four medical datasets with different labelled/unlabelled assumptions have been
used in the experiments. A series of experiments are designed including the
comparison between TriSegNet and fully-supervised learning algorithms, as well
as its comparison with other semi-supervised learning algorithms. Ablation
studies justify the design decisions. Although a common Dice-based loss
function is used for the initial stages of the training process, a bespoke
boundary-overlap-based loss is used in the more advanced stages. This
increases the confidence of the model in its predictions and hence the
reliability of the pseudo labels it generates. Overall, TriSegNet demonstrates
promising performance in most evaluation metrics, showing great potential in
semi-supervised learning for general medical image segmentation.

## References

  * [1] Abadi, M., et al.: TensorFlow: Large-scale machine learning on heterogeneous systems (2015), https://www.tensorflow.org/, software available from tensorflow.org
  * [2] Bernard, O., et al.: Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: is the problem solved? IEEE Transactions on Medical Imaging 37(11), 2514–2525 (2018)
  * [3] Blum, A., Mitchell, T.: Combining labeled and unlabeled data with co-training. In: Proceedings of the eleventh annual conference on Computational learning theory. pp. 92–100 (1998)
  * [4] Chaurasia, A., Culurciello, E.: Linknet: Exploiting encoder representations for efficient semantic segmentation. In: 2017 IEEE Visual Communications and Image Processing. pp. 1–4. IEEE (2017)
  * [5] Chen, D.D., et al.: Tri-net for semi-supervised deep learning. In: International Joint Conferences on Artificial Intelligence (2018)
  * [6] Chen, L.C., et al.: Naive-student: Leveraging semi-supervised learning in video sequences for urban scene segmentation. In: European Conference on Computer Vision. pp. 695–714. Springer (2020)
  * [7] Chen, X., et al.: Semi-supervised semantic segmentation with cross pseudo supervision. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 2613–2622 (2021)
  * [8] Chen, X., et al.: Learning active contour models for medical image segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11632–11640 (2019)
  * [9] Donahue, J., et al.: DeCAF: A deep convolutional activation feature for generic visual recognition. In: International Conference on Machine Learning. pp. 647–655. PMLR (2014)
  * [10] Gamper, J., et al.: Pannuke: an open pan-cancer histology dataset for nuclei instance segmentation and classification. In: European Congress on Digital Pathology. pp. 11–19. Springer (2019)
  * [11] Huang, J., et al.: O2u-net: A simple noisy label detection approach for deep neural networks. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 3326–3334 (2019)
  * [12] Kaggle: Ultrasound nerve segmentation. https://www.kaggle.com/c/ultrasound-nerve-segmentation
  * [13] Ke, Z., et al.: Guided collaborative training for pixel-wise semi-supervised learning. In: European Conference on Computer Vision. pp. 429–445. Springer (2020)
  * [14] Kim, S.W., et al.: Parallel feature pyramid network for object detection. In: Proceedings of the European Conference on Computer Vision. pp. 234–250 (2018)
  * [15] Laine, S., Aila, T.: Temporal ensembling for semi-supervised learning. arXiv preprint arXiv:1610.02242 (2016)
  * [16] Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision. pp. 2980–2988 (2017)
  * [17] Luo, X., et al.: Efficient semi-supervised gross target volume of nasopharyngeal carcinoma segmentation via uncertainty rectified pyramid consistency. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 318–329. Springer (2021)
  * [18] Qiao, S., et al.: Deep co-training for semi-supervised image recognition. In: Proceedings of the European Conference on Computer Vision. pp. 135–152 (2018)
  * [19] Ronneberger, O., Fischer, P., Brox, T.: U-Net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 234–241. Springer (2015)
  * [20] Tarvainen, A., Valpola, H.: Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. pp. 1195–1204 (2017)
  * [21] Verma, V., et al.: Interpolation consistency training for semi-supervised learning. In: International Joint Conference on Artificial Intelligence. pp. 3635–3641 (2019)
  * [22] Vu, T.H., et al.: Advent: Adversarial entropy minimization for domain adaptation in semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 2517–2526 (2019)
  * [23] Wang, Z., Zhang, Z., Voiculescu, I.: RAR-U-Net: a residual encoder to attention decoder by residual connections framework for spine segmentation under noisy labels. In: Proceedings of the IEEE International Conference on Image Processing. IEEE (2021)
  * [24] Wang, Z., et al.: Computationally-efficient vision transformer for medical image semantic segmentation via dual pseudo-label supervision. In: IEEE International Conference on Image Processing (ICIP) (2022)
  * [25] Wang, Z., et al.: An uncertainty-aware transformer for MRI cardiac semantic segmentation via mean teachers. Annual Conference on Medical Image Understanding and Analysis (2022)
  * [26] Xia, Y., et al.: 3D semi-supervised learning with uncertainty-aware multi-view co-training. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. pp. 3646–3655 (2020)
  * [27] Yao, J., Burns, J.E., etc: Detection of vertebral body fractures based on cortical shell unwrapping. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 509–516. Springer (2012)
  * [28] Yeghiazaryan, V., Voiculescu, I.D.: Family of boundary overlap metrics for the evaluation of medical image segmentation. SPIE Journal of Medical Imaging 5(1), 015006 (2018)
  * [29] You, X., et al.: Segmentation of retinal blood vessels using the radial projection and semi-supervised approach. Pattern recognition 44(10-11), 2314–2324 (2011)
  * [30] Yu, L., Wang, S., Li, X., Fu, C.W., Heng, P.A.: Uncertainty-aware self-ensembling model for semi-supervised 3D left atrium segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 605–613. Springer (2019)
  * [31] Zhang, Y., et al.: Deep adversarial networks for biomedical image segmentation utilizing unannotated images. In: International conference on Medical Image Computing and Computer-Assisted Intervention. pp. 408–416. Springer (2017)

##  Appendix 0.A Algorithm of TriSegNet

The training of TriSegNet consists of four stages which is briefly illustrated
in Algorithm 1. The code of TriSegNet will be publicly
available.222https://github.com/ziyangwang007/CV-SSL-MIS

Input: A batch of $(x,y)$ from labeled dataset $\mathcal{L}$, unlabeled
dataset $\mathcal{U}$, or test dataset $\mathcal{T}$. $\mathit{DA}$,
$\mathit{LCLE}$, and $\mathit{CLV}$ are label processing approaches Data
Augmentation, Low Confidence Label Editing, and Confident Label Voting,
respectively.

Output: Three trained high-level feature learning classifiers.
$f_{n},n={1,2,3}$

Stage one: Initialization

$\theta_{1},\theta_{2},\theta_{3}\leftarrow$ initial parameters of classifiers
$f_{1},f_{2},f_{3}$, and the loss function $L_{Supervision}$.

while  _n $\leftarrow$ [1, 2, 3] (3 classifiers)_ do

$(x,y_{gt})$ sampled from $DA(\mathcal{L})$ with augmentation;

Generate prediction $y_{pred}=f_{n}(x)$ ;

Calculate loss $L_{Supervision}$ with $y_{pred}=y_{gt}$ ;

Update $\theta_{n}\leftarrow\theta_{n}-\Delta L$ ;

end while

Stage two: Classifiers Training with Pseudo Label Processing

$\alpha_{1},\alpha_{2},\alpha_{3}\leftarrow$ initial confidence weight of
classifiers $f_{1},f_{2},f_{3}$ ;

while  _n $\leftarrow$ [1, 2, 3], i $\leftarrow$ [1…5] (5 Iterations)_ do

$x$ sampled from $DA(\mathcal{U})$ with augmentation ;

Generate pseudo label for Classifier $f_{n}$
$y_{n\\_pseudo}=CLV(LCLE(f_{n-1}(x),LCLE(f_{n+1}(x)))$ ;

Calculate loss $L_{SemiSupervision}$ with $y_{pred}=y_{n\\_pseudo}$ ;

Update $\theta_{n}\leftarrow\theta_{n}-\Delta L$ ;

Update $\alpha_{n}\leftarrow$ with $L$ by evaluation dataset ;

end while

Stage three: One low confidence Classifer Training

while  _n $\leftarrow$ [1, 2, 3], i $\leftarrow$ [1…5]_ do

If Network $f_{n}$ is with the lowest $\alpha_{n}$ Then

$(x,y_{gt})$ sampled from $DA(\mathcal{L})$ ;

$(x,y_{n\\_pseudo})$ sampled from $DA(\mathcal{U})$ ;

Generate pseudo label for Classifer $f_{n}$
$y_{n\\_pseudo}=CLV(LCLE(f_{n-1}(x),LCLE(f_{n+1}(x)))$ ;

Calculate loss $L$ with $y_{pred}=y_{n\\_pseudo}$ ;

Update $\theta_{n}\leftarrow\theta_{n}-\Delta L_{SemiSupervision}$ ;

Update $\alpha_{n}\leftarrow$ with $L$ by evaluation dataset ;

end while

Algorithm 1 Training Stage of TriSegNet for Medical Image Segmentation

##  Appendix 0.B The CNN Architecture of Multi-View Learning

To properly encourage the differences of the three views of feature learning
on dense prediction, not only the data feed and initialization of parameters,
but three different advanced CNN are proposed in TriSegNet. We utilize three
different techniques for CNN i.e. skip connection, efficiently passing feature
information through residual learning, and multi-scale feature learning. The
parameters of three classifiers are briefly illustrated in Table 4 and the
source code has been released
online.333https://github.com/qubvel/segmentation_models/tree/master/segmentation_models

Model | Total Params | Trainable params | Non-trainable params
---|---|---|---
Classifier A | 32,561,120 | 32,513,562 | 47,558
Classifier B | 20,323,985 | 20,317,169 | 47,558
Classifier C | 17,594,453 | 17,592,149 | 2,304
Table 4: The Computation Cost Information of Three Classifier

##  Appendix 0.C Evaluation Methods, Qualitative, and Quantitative Results

Table 2 reports the TriSegNet performance direct comparison with other
algorithms with several strict and novel quantitative evaluation metrics to
which the boundaries of the machine segmentation(MS) match those of the ground
truth(GT), using the Directed Boundary Dice relative to GT (DBDG), Directed
Boundary Dice relative to MS (DBDM) and Symmetric Boundary Dice (SBD).

In a von Neumann neighbourhood $N_{x}$ of each pixel $x$ on the boundary
$\partial G$ of the ground truth,

$\small
DBD_{G}=DBD(G,M)=\displaystyle\frac{\rule{0.0pt}{11.62495pt}\sum\limits_{x\in\partial
G}\text{Dice}(N_{x})}{\left|\partial G\right|}$ (4) $\small
DBD_{M}=DBD(M,G)=\displaystyle\frac{\rule{0.0pt}{11.62495pt}\sum\limits_{x\in\partial
M}\text{Dice}(N_{Y})}{\left|\partial M\right|}$ (5) $\small
SBD=\displaystyle\frac{\rule{0.0pt}{11.62495pt}\sum\limits_{x\in\partial
G}DSC(N_{x})+\sum\limits_{y\in\partial M}DSC(N_{y})}{\left|\partial
G\right|+\left|\partial M\right|}$ (6)

where Dice is $Dice(N_{x})=\frac{2|G(N_{x})\cap
M(N_{y})|}{|G(N_{x})|+|M(N_{y})|}$. The symmetric average is being brought
down by DBDG when the latter features isolated areas of false negative labels.
These measures penalise mislabelled areas in the machine segmentation.

Some of example qualitative results on MRI Cardiac test set are briefly
sketched in Figure 4. Eight images are selected from MRI test set where the
first row illustrates raw images. The rest of them illustrate the MS by each
semi-supervised algorithm against GT where yellow, green, red, and black
represent true positive, false negative, false positive and true negative at
pixel level. The proposed method shows fewer false positive and false negative
pixels, and significantly low HD as well, because the TriSegNet is beneficial
with different views of high-level pixel-level classifier and proposed mixed
boundary- and overlap-based loss function.

Figure 4: Sample Qualitative Results on MRI Cardiac Test Set


11institutetext: Dept. of Molecular Astrophysics, Instituto de Física
Fundamental (IFF-CSIC), C/ Serrano 121, 28006 Madrid, Spain
11email<EMAIL_ADDRESS>22institutetext: Centro de Desarrollos
Tecnológicos, Observatorio de Yebes (IGN), 19141 Yebes, Guadalajara, Spain
33institutetext: Observatorio Astronómico Nacional (OAN, IGN), Madrid, Spain

# The spatial distribution of an aromatic molecule, C6H5CN, in the cold dark
cloud TMC-1††thanks: Based on observations carried out with the Yebes 40m
telescope (projects 22A007 and 22B029). The 40m radio telescope at Yebes
Observatory is operated by the Spanish Geographic Institute (IGN, Ministerio
de Fomento).

J. Cernicharo 11 B. Tercero 2233 N. Marcelino 2233 M. Agúndez 11 and P. de
Vicente 22

(Received 21 April 2023; accepted 21 May 2023)

We present a highly sensitive 2D line survey of TMC-1 obtained with the Yebes
40m radio telescope in the Q-band (31.13-49.53 GHz). These maps cover a region
of 320′′$\times$320′′ centred on the position of the QUIJOTE1 line survey with
a spatial sampling of 20′′. The region covering 240′′$\times$240′′, where a
longer integration time was used, shows a homogenous sensitivity of 2-4 mK
across the band. We present in this work the first determination of the
spatial extent of benzonitrile (C6H5CN), which follows that of cyanopolyynes
rather well, but differs significantly from that of the radicals CnH and CnN.
We definitively conclude that aromatic species in TMC-1 are formed from
chemical reactions involving smaller species in the densest zones of the
cloud.

###### Key Words.:

molecular data – line: identification – ISM: molecules – ISM: individual
(TMC-1) – astrochemistry

##  1 Introduction

In the last five years, two line surveys of the starless cold core TMC-1,
QUIJOTE111Q-band Ultrasensitive Inspection Journey to the Obscure TMC-1
Environment (Cernicharo et al. 2020, 2021a, 2023) and GOTHAM222 GBT
Observations of TMC-1: Hunting for Aromatic Molecules (McGuire et al. 2018,
2020), have been carried out and have provided the discovery of a panoply of
new radicals, aromatic and polyaromatic compounds, cations, anions, and
sulphur-bearing species. Roughly one-third of the molecules discovered in
space have resulted from these two line surveys. The GOTHAM observations were
taken with the Green Bank 100m radio telescope in the X-, K-, and Ka-bands.
Most detections in this line survey were performed through a sophisticated
statistical frequency stacking procedure. The QUIJOTE line survey is an
ongoing line survey in the Q-band (31.1-50.3 GHz) obtained with the Yebes 40m
radio telescope. The detection technique in QUIJOTE is the classical, and
reliable, line-by-line detection without any spectral stacking. The present
sensitivity of QUIJOTE is 0.12-0.25 mK (Cernicharo et al. 2023).

The modelling of this emission is often tackled with very limited information
on the spatial extent of the observed emission. In GOTHAM and QUIJOTE the only
available spatial information is provided by the variation of the telescope
half power beam with the frequency across the line survey. While GOTHAM fits
four velocity components with different spatial sizes, QUIJOTE assumes a
source radius of 40′′ based on previous observations of TMC-1 in several
molecular species. None of these methods is satisfactory to obtain the
accurate column densities needed to put constraints on the chemical models of
the source. The spatial size of the observed molecules, together with the
issues related to the line opacities and radiative transfer, can only be
addressed through spatial mapping of the molecular emission.

To overcome these issues, the QUIJOTE line survey has been complemented with
high-sensitivity maps obtained with the Yebes 40m radio telescope and covering
a region of 320′′$\times$320′′ centred on the QUIJOTE position (see Sect. 2).
These maps are a faithful companion to the QUIJOTE line survey and we call
these supplementary spatial data SANCHO.333 Surveying the Area of the
Neighbour TMC-1 Cloud through Heterodyne Observations The goal of these maps
is to permit the study of the spatial distribution of any QUIJOTE line with
intensity $\geq$20 mK with a signal-to-noise ratio $\geq$10, which means the
low-energy transitions of most of the abundant species such as cyanopolyynes
and radicals (CnS, CnH, and CnN) among other molecules. Most of the
isotopologues 13C, 34S, D, and 15N of these species can be also spatially
traced with SANCHO data.

In this letter we present the first determination of the spatial distribution
of an aromatic molecule (benzonitrile, C6H5CN) in the starless cold dark cloud
TMC-1, and compare it with that of other molecules.

Figure 1: Stacked line of C6H5CN obtained from the average in each position of
52 individual lines (see Appendix B). Only the spectra around the region of
maximum emission (see Fig. 2) are shown. The offset positions in arcseconds
are indicated in each panel. The abscissa in each panel corresponds to the
local standard of rest velocity of the stacked data (in km s-1). The intensity
scale corresponds to the antenna temperature corrected for atmospheric
transmission and antenna losses.

##  2 Observations

New receivers, built within the Nanocosmos444ERC grant
ERC-2013-Syg-610256-NANOCOSMOS.
https://nanocosmos.iff.csic.es/ project and installed at the Yebes 40m
radiotelescope, were used for the observations of TMC-1
($\alpha_{J2000}=4^{\rm h}41^{\rm m}41.9^{\rm s}$ and
$\delta_{J2000}=+25^{\circ}41^{\prime}27.0^{\prime\prime}$). The present
observations of TMC-1 complement the QUIJOTE line survey (Cernicharo et al.
2021a). A detailed description of the telescope, receivers, and backends is
given in Tercero et al. (2021). Briefly, the receiver consists of two cold
high electron mobility transistor amplifiers covering the 31.1-50.3 GHz band
with horizontal and vertical polarisations. Receiver temperatures in the first
QUIJOTE observations achieved during 2020 vary from 22 K at 32 GHz to 42 K at
50 GHz. However, some power adaptation in the down-conversion chains in 2021
reduced the receiver temperatures to 16 K at 32 GHz and 25 K at 50 GHz. The
backends are $2\times 8\times 2.5$ GHz fast Fourier transform spectrometers
(FFTs) with a spectral resolution of 38.15 kHz providing the whole coverage of
the Q-band in both polarisations. The final Doppler correction was made during
the pipeline processing of the raw data and the final spectra have the correct
frequency scale in every part of the 31.1-50.3 GHz band.

The telescope beam size varies from 56.7′′ at 31 GHz to 35.6′′ at 49.5 GHz.
The intensity scale used in this work, antenna temperature ($T_{A}^{*}$), was
calibrated using two absorbers at different temperatures and the atmospheric
transmission model (ATM) (Cernicharo 1985; Pardo et al. 2001). Calibration
uncertainties of 10 % were adopted. The beam efficiency of the Yebes 40m
telescope in the Q-band is given as a function of frequency by $B_{\rm
eff}$=0.797 exp[$-$($\nu$(GHz)/71.1)2]. The forward telescope efficiency is
0.95.

The SANCHO maps were performed in the on-the-fly mode using frequency
switching with a throw of 10 MHz. We prefer this observing mode as many
molecular species have extended emission (Cernicharo & Guélin 1987). The speed
of the telescope was 5′′/sec and the data of the whole Q-band in the two
polarisations were recorded every two seconds (16 individual spectra per
position). The maps were done moving the telescope in right ascension from
$-$130′′ to +130′′ with a declination step of 10′′. Once such a map was
achieved, then the telescope was moved in declination from $-$130′′ to 130′′
with a right ascension step of 10′′. This procedure was repeated until we
reached 100 hours of observing time on the source. Some additional maps with a
total observing time on source of 15 hours were added, covering offsets up to
$\pm$160′′ in right ascension and declination. A total of 3468624 spectra,
corresponding to 216789 different positions, two polarisations, and eight
FFTs, were recorded in February and December 2022, and January 2023. The raw
data file size is near 1 TByte. All data were analysed using the GILDAS
package. 555http://www.iram.fr/IRAMFR/GILDAS

##  3 Results

For the final SANCHO data, we produced six different types of maps that were
obtained by resampling the raw data within a square grid with a point
separation of 10′′ and 20′′ ($S_{grid}$), and with three different spatial
tolerances around each position for adding raw data. These tolerance zones
correspond to circles of 10′′, 15′′, and 20′′ in radius ($T_{rad}$). Each
position in the maps contains the whole Q-band spectrum, with both
polarisations averaged. The highest sensitivity is achieved for the maps with
$S_{grid}$ = 20′′ and $T_{rad}$ = 20′′, which are those that we discuss in
this work (see Appendix A). In this process no baseline is applied as the
whole Q-band spectrum is treated simultaneously (31.13-49.53 GHz with 483928
channels at each position of the map). These maps are fully sampled spatially.
However, due to the spatial tolerance used in creating the maps we expect to
have produced some spatial smoothing (i.e. the emission in adjacent positions
has some degree of correlation; see Appendix A).

Figure 2: Colour plot of spatial distribution of the integrated intensity
between 5.3 and 6.5 km s-1 of the stacked lines of C6H5CN. The colour scale is
in mK km s-1. The sampling of the data is 20′′ and the integrating area in
each position corresponds to a circle of 20′′ of radius. The black dot
indicates the centre of the map, which corresponds to the position observed
with the QUIJOTE line survey.

Maps for each individual transition of a given molecule were obtained from the
selected final map (with a given $S_{grid}$ and $T_{rad}$) by extracting the
spectral data over $\pm$14 km s-1 around the frequency of the line. Transition
frequencies were obtained from the spectral information of the MADEX code
(Cernicharo 2012), the JPL catalogue (Pickett et al. 1998), or the CDMS
database (Müller et al. 2005). A baseline is removed from each extracted
spectrum after defining the windows for all lines potentially present in the
selected velocity range using the QUIJOTE line survey as a reference for
detected lines (only features above 5 mK were blanked in this process).
Appendix A provides examples of several molecular lines at the central
position of the map without and with baseline removal. The sensitivity over
the Q-band as a function of frequency, $T_{rad}$, and position in the map is
provided in Appendix A (see Table 1).

It is worth noting that all maps produced in this way were observed
simultaneously, and hence they have the same pointing and the same calibration
uncertainties. Moreover, the relative calibration between different lines of a
given molecule is much better than the global calibration uncertainty of 10 %
(see e.g. Appendix A and Fig. 5). The achieved sensitivity in each position
depends on the adopted $T_{rad}$ in adding data. In Appendix A we show the
effect of adopting different gridding steps and tolerance circles. For
$S_{grid}$ = 20′′ and $T_{rad}$ = 20′′, the sensitivity is 1.4-4 mK across the
Q-band (the sensitivity is the same for $S_{grid}$=10′′). This sensitivity is
degraded by a factor of $\sim$2 outside the square defined by
$\Delta\alpha$=$\pm$120′′ and $\Delta\delta$=$\pm$120′′. Hence, SANCHO
provides a whole spectrum of the Q-band over a map of 169 positions gridded
every 20′′ (240′′$\times$240′′) with a sensitivity that is similar to that
achieved by GOTHAM in a single position. However, the sensitivity towards the
centre of the map is worse by a factor of 10 compared to that of the QUIJOTE
line survey.

Figure 3: Spatial distribution of the integrated intensity of the stacked
lines of C6H5CN at different velocities. The selected velocity range is
$\pm$0.1 km s-1 around the velocity indicated in the upper right corner of
each panel (in km s-1). The sampling of the data is 20′′ and the integrating
area in each position corresponds to a circle of 20′′ radius. The white and
black contours represent the integrated intensity of the $J$=28-27 transition
of HC7N at the same velocity ($\pm$0.1 km s-1). The first white contour
corresponds to 2 mK kms-1 and the step is 4 mK kms-1. The first black contour
corresponds to 20 mK kms-1 and the step is 6 mK kms-1. The white or black dot
indicates the centre of the map.

The sensitivity reached by SANCHO allows the detection of all lines with
intensities greater than 10 mK in the central position of the maps. This is
enough to detect many of the lines of C6H5CN observed with the QUIJOTE line
survey presented in Fig. B.1 of Cernicharo et al. (2021b). However, the
spatial distribution of the integrated intensity of each individual line
appears noisy, which prevents us from reaching a conclusion on the spatial
extent of this aromatic species. We therefore used the maps generated for each
individual line to produce a spectrally stacked map. We added 49 lines of
benzonitrile which are free of resolved hyperfine structure and of blending
with lines from other species (see Table 2). Each line was multiplied by a
factor to take into account the observed intensity with respect to a reference
line, the strongest one of benzonitrile in the QUIJOTE line survey (the
130,13-120,12 transition at 32833.827 MHz). The procedure is described in
detail in Appendix B. The stacking scheme allows one to use the parameters of
the reference line to estimate the column density and increases the S/N of the
stacked spectral map with respect the observed individual line maps of
benzonitrile.

The resulting spectra at different positions in the map are shown in Fig. 1.
The sensitivity of the map of the stacked lines at the ($\Delta\alpha$,
$\Delta\delta$) positions (0′′, 0′′), (120′′, 0′′), and (120′′, $-$120′′) is
0.4 mK, 0.6 mK, and 0.9 mK, respectively (see Fig. 1). The spatial
distribution of the integrated intensity of this aromatic species is shown in
Fig. 2. Finally, the spatial distribution of the emission at different
velocities is shown in Fig. 3.

Figure 4: Integrated intensity between 5.3 and 6.5 kms-1 of different
molecular transitions (colours) compared with that of C6H5CN (black contours;
first contour and step are 0.75 mK kms-1). For each molecular transition the
integrated intensity has been normalised to the maximum value within the area
covered by the map. The colour scale (bottom right) is the same for all
molecular transitions. The molecule, transition, and maximum intensity (shown
in brackets) are indicated in the bottom right corner of each panel. The black
dot corresponds to the centre of the map.

##  4 Discussion

The comparison of the velocity maps of C6H5CN and HC7N shown in Fig. 3 clearly
indicates that the two species coexist spatially. The emission of the two
species covers the velocity range 5.3-6.3 km s-1, with some weak emission
still at 6.5 km s-1. The observed emission follows the well-known filamentary
structure of TMC-1 with a dense condensation around the central position. Two
different velocities seem to be traced spatially by our data, both with a
filamentary aspect: one at 5.5 km s-1 with its maximum over the QUIJOTE
position (known as the cyanopolyynes source) and the other at 5.9 km s-1 that
peaks at $\Delta\alpha$=40′′ and $\Delta\delta$=-20′′, which dominates the
line profile. Given our velocity resolution of $\sim$0.25 km s-1, the observed
velocity structure could also be interpreted as resulting from a velocity
gradient perpendicular to the TMC-1 filament, or as a twisting motion of
filaments, or fibres, similar to those found in other molecular clouds (Kirk
et al.  2013; Hacar et al.  2013, 2017). This behaviour is similar for all the
species we study in this work. The TMC-1 filament was observed in the emission
of molecules such as SO, NH3, and CCS, with higher spatial resolution than we
used (see e.g. Lique et al.  2006; Fehér et al.  2016; Dobashi et al.  2019,
and references therein). These authors found a complex spatial velocity
structure dominated by clumpy filaments at velocities between 5.3 and 6.3 km
s-1.

The spatial coexistence of benzonitrile and other molecules is fully confirmed
in Fig. 4, where we compare the spatial distribution of the integrated
intensity of several species, including radicals, cyanopolyynes, methyl-
bearing species, and isomers of HC3N. This is the first time that many of the
selected molecules have been mapped in TMC-1. Moreover, the signal-to-noise
ratio of all these maps is extremely high as the selected lines have
intensities $\geq$100 mK, even for the HC7N $J$=38-37 for which the upper
level energy is 40.1 K. The integrated intensities are proportional to the
column densities because the lines are mostly optically thin. The map of HC3N
was obtained using the integrated intensity of the weak satellite hyperfine
lines, which also guarantees low line opacities.

Opacity effects, the beam size of the telescope, different excitation
conditions, and the spatial smearing produced by the co-addition of the on-
the-fly data could produce some small differences between the maps. However,
some of these differences are real and are related to the global aspect of the
spatial distribution. For example, cyanopolyynes and benzonitrile are
significantly shifted towards the north-east by more than a beam with respect
to the radicals CnH and CnN. The spatial shift of the radicals C4H, C6H, and
C3N with respect to the cyanopolyynes HC3N and HC5N may be caused by the
structure of the H2 density. The HC3N peak is also a local maximum in H2
density (see Pratap et al.  1997), and radicals are probably more efficiently
destroyed at high densities than closed-shell molecules. A different spatial
distribution was also found in TMC-1 by Fossé et al. (2001) for C6H, $c$-C3H2,
and H2C3. Significant differences are also found for the two transitions of
HC7N shown in Fig. 4. This is probably due to the high energy of the $J$=38
level, which renders this transition very sensitive to the density or to small
changes in the kinetic temperature. The analysis of the spatial distribution
of several transitions of radicals, cyanopolyynes, hydrocarbons, and their
isotopologues will be published in forthcoming papers.

The formation of aromatic cycles in TMC-1 has been a matter of debate in
recent years (McGuire et al. 2018, 2021; Cernicharo et al. 2021b, 2022). It is
not yet clear whether these aromatic rings are formed through a bottom-up
mechanism or a top-down process. The presence of large polycyclic aromatic
hydrocarbons (PAHs) in diffuse interstellar clouds is inferred from the
observation of intense unidentified infrared bands, and the aromatic cycles
observed in TMC-1 could result from the destruction of this reservoir of PAHs
inherited from a previous diffuse stage. A similar process has been invoked to
account for the formation of small hydrocarbons in photodissociation regions
(Pety et al. 2005). Alternatively, aromatic rings could be formed from small
hydrocarbons in a bottom-up process. For example, the reaction of propene, an
abundant hydrocarbon in TMC-1 (Marcelino et al. 2007), with CH can produce
1,3-butadiene (Loison & Bergeat 2009), which can lead to $c$-C5H7 upon
reaction with CH (Cernicharo et al. 2021b) and to benzene by reacting with C2H
(Jones et al. 2011). Even so, chemical models based on these neutral-neutral
routes are not able to account for the observed abundances of cycles such as
$c$-C5H7 (Cernicharo et al. 2022), which indicates that ion-neutral reactions
may also play a role in the synthesis of aromatic cycles. Whatever the true
mechanism of formation of aromatic rings, the fact that benzonitrile has a
spatial distribution very similar to that of cyanopolyynes supports the
argument that aromatic molecules are formed through a bottom-up chemical
process, as is thought to occur for long carbon chains in TMC-1.

##  5 Conclusions

We have presented the SANCHO maps of TMC-1 in the frequency range 31.13-49.53
GHz. The sensitivity of the spectra over an area of 240′′$\times$240′′, for a
gridding step of 20′′ and $T_{rad}$=20′′, is 2-4 mK across the band. We have
determined, for the first time, the spatial distribution of C6H5CN
(benzonitrile). We find that it correlates very well with that of
cyanopolyynes, and that it extends over the TMC-1 filament. Hence, we conclude
that this species is formed through bottom-up chemical processes involving the
same type of reactions as those forming the other species found in the cloud.

###### Acknowledgements.

We thank ERC for funding through grant ERC-2013-Syg-610256-NANOCOSMOS. We also
thank Ministerio de Ciencia e Innovación of Spain (MICIU) for funding support
through projects PID2019-106110GB-I00, PID2019-107115GB-C21 / AEI /
10.13039/501100011033, and PID2019-106235GB-I00.

## References

  * Agúndez et al. (2021) Agúndez, M., Cabezas, C., Tercero, B., et al. 2021, A&A, 647, L10
  * Cernicharo (1985) Cernicharo, J. 1985, Internal IRAM report (Granada: IRAM)
  * Cernicharo & Guélin (1987) Cernicharo, J., Guélin, M., 1987, A&A, 183, L10
  * Cernicharo (2012) Cernicharo, J., 2012, in ECLA 2011: Proc. of the European Conference on Laboratory Astrophysics, EAS Publications Series, 2012, Ed.: C. Stehl, C. Joblin, & L. d’Hendecourt (Cambridge: Cambridge Univ. Press), 251; https://nanocosmos.iff.csic.es/?page$\\_$id=1619
  * Cernicharo et al. (2020) Cernicharo, J., Marcelino, N., Agúndez, M. et al. 2020, A&A, 642, L8
  * Cernicharo et al. (2021a) Cernicharo, J., Agúndez, M., Kaiser, R., et al. 2021a, A&A, 652, L9
  * Cernicharo et al. (2021b) Cernicharo, J., Agúndez, M., Kaiser, R.I. et al. 2021b, A&A, 655, L1
  * Cernicharo et al. (2022) Cernicharo, J., Fuentetaja, R., Agúndez, M., et al. 2022, A&A, 663, L9
  * Cernicharo et al. (2023) Cernicharo, J., Pardo, J.R., Cabezas, C. et al. 2023, A&A, 670, L19
  * Dobashi et al. (2019) Dobashi, K., Shimoikura, T., Ochiai, T. et al. 2019, ApJ, 879:88
  * Fehér et al. (2016) Fehér, O., Tóth, L.V., Ward-Thompson, D. et al. 2016, A&A, 590, A75
  * Fossé et al. (2001) Fossé, D., Cernicharo, J., Gerin, M., Cox, P. 2001, ApJ, 552, 168
  * Hacar et al. (2013) Hacar, A., Tafalla, M., Kauffmann, J. & Kovács, A. 2013, A&A, 554, A55
  * Hacar et al. (2017) Hacar, A., Tafalla, M. & Alves, J. 2017, A&A, 606, A123
  * Jones et al. (2011) Jones, B. M., Zhang, F., & Kaiser, R. I. 2011, PNAS, 108, 452
  * Kirk et al. (2013) Kirk, H., Myers, P.C., Bourke, T.L. et al. 2012, ApJ, 766, 115
  * Lique et al. (2006) Lique, F., Cernicharo, J. & Cox, P. 2006, ApJ, 653, 1342
  * Loison & Bergeat (2009) Loison, J.-C. & Bergeat, A. 2009, PCCP, 11, 655
  * Marcelino et al. (2007) Marcelino, N., Cernicharo, J., Agúndez, M. et al. 2007, ApJ, 665, L127
  * McGuire et al. (2018) McGuire, B. A., Burkhardt, A. M., Kalenskii, S., et al. 2018, Science, 359, 202
  * McGuire et al. (2020) McGuire, B. A., Burkhardt, A. M., Loomis, R. A., et al. 2020, ApJ, 900, L10
  * McGuire et al. (2021) McGuire, B. A., Loomis, R. A., Burkhardt, A. M., et al. 2021, Science, 371, 1265
  * Müller et al. (2005) Müller, H.S.P., Schlöder, F., Stutzki, J., Winnewisser, G. 2005, J. Mol. Struct., 742, 215
  * Pardo et al. (2001) Pardo, J. R., Cernicharo, J., Serabyn, E. 2001, IEEE Trans. Antennas and Propagation, 49, 12
  * Pety et al. (2005) Pety, J., Teyssier, D., Fossé, D., et al. 2005, A&A, 435, 885
  * Pickett et al. (1998) Pickett, H.M., Poynter, R. L., Cohen, E. A., et al. 1998, J. Quant. Spectrosc. Radiat. Transfer, 60, 883
  * Pratap et al. (1997) Pratap, P., Dickens, J.E., Snell, R.L. et al. 1997, ApJ, 486, 862
  * Tercero et al. (2021) Tercero, F., López-Pérez, J. A., Gallego, et al. 2021, A&A, 645, A37

##  Appendix A Baselines, gridding step, tolerance circle, and sensitivity

Our on-the-fly frequency switching observations suffer from frequency ripples.
However, the period of these ripples is always larger than 10 MHz, while the
lines in the Q-band are typically 0.10-0.18 MHz wide. Hence, it is possible to
remove a polynomial baseline to each line of the survey at each position of
the map with total confidence that neither the line intensity nor the line
profile are perturbed by the baseline removal step. These effects are also
present in the QUIJOTE line survey, and we follow a procedure similar to that
previously described for these frequency switching observations (Cernicharo et
al. 2022).

Table 1: Sensitivity of the maps as a function of the frequency and of the
distance to the central position.

Frequency |  $\sigma(10$”$)^{a}$ |  $\sigma(15$”$)^{a}$ |  $\sigma(20$”$)^{a}$
---|---|---|---
(MHz) | (mK) | (mK) | (mK)
| P1 P2 P3 | P1 P2 P3 | P1 P2 P3
31120 | 3.6 4.7 7.4 | 3.2 3.4 6.0 | 2.3 2.6 4.5
31500 | 2.6 3.8 6.1 | 2.3 2.8 4.9 | 1.9 2.0 3.6
32500 | 2.2 3.2 6.2 | 2.0 2.9 4.8 | 1.4 1.6 3.2
33500 | 2.2 2.8 3.9 | 1.9 2.1 3.7 | 1.3 1.6 2.7
34500 | 2.5 3.0 5.0 | 2.2 2.8 4.1 | 1.6 1.9 3.2
35500 | 2.5 2.6 4.8 | 2.3 2.2 4.2 | 1.7 1.8 2.9
36500 | 2.2 3.2 5.9 | 2.2 2.7 5.0 | 1.5 2.0 3.2
37500 | 2.6 3.3 5.8 | 2.4 2.9 4.9 | 1.7 1.9 3.2
38500 | 2.7 3.5 6.0 | 2.3 3.3 5.3 | 1.6 2.2 3.9
39500 | 2.9 3.5 6.1 | 2.5 3.1 5.2 | 2.0 1.9 4.2
40500 | 2.9 4.0 6.3 | 2.6 3.3 4.7 | 1.9 2.1 3.9
41500 | 3.4 4.1 7.3 | 3.1 3.6 5.6 | 2.1 2.5 4.2
42500 | 3.3 3.8 6.8 | 2.9 3.6 5.6 | 2.2 2.5 4.3
43500 | 3.7 5.2 6.1 | 3.2 3.9 5.5 | 2.1 2.8 4.5
44500 | 3.2 5.1 6.9 | 2.9 4.3 6.4 | 2.2 3.2 4.7
45500 | 3.8 5.6 8.4 | 3.5 4.7 7.4 | 2.9 3.4 6.0
46500 | 4.1 6.4 9.1 | 3.9 5.4 7.4 | 2.9 3.9 6.3
47500 | 5.5 7.6 10.0 | 4.9 6.8 9.0 | 3.5 4.8 7.8
48500 | 6.3 7.8 11.6 | 5.8 6.9 9.4 | 3.7 4.7 7.7
49000 | 6.2 7.1 12.2 | 5.5 6.2 10.8 | 3.9 4.6 8.3
49500 | 6.3 7.1 12.1 | 5.7 6.5 10.5 | 4.0 4.7 7.8
666
The three selected positions in the map, P1, P2, and P3, correspond to the
central position $\Delta\alpha$=0′′,$\Delta\delta$=0′′, to offset
$\Delta\alpha$=100′′, $\Delta\delta$=0′′, and to offset $\Delta\alpha$=120′′,
$\Delta\delta$=-120′′, respectively.
$a$$a$footnotetext: Root mean square noise derived from a polynomial baseline
removal to a frequency range of $\pm$3 MHz (157 channels) around each selected
frequency. The value adopted for $T_{rad}$ is indicated in brackets.

An example of the baseline removal procedure in the SANCHO maps is shown in
Fig. 5 where we consider the $N$=4-3 and $N$=5-4 transitions of CCCN. These
lines exhibit two line components separated by $\sim$19 MHz due to the fine
structure of the rotational transitions. The lines are shown in Fig. 5 before
and after baseline subtraction. For the $N$=4-3 transition the derived
sensitivity is 2 mK. The observed intensity ratio of the two fine components
is 1.35, while the theoretical value is 1.33 (for optically thin emission).
For the two componentes of the $N$=5-4 transition the derived sensitivity is
4.2 mK and the line strength ratio is 1.21, which is very close to the
theoretical value of 1.24. The number of channels in Fig. 5 is 630.

The folding of the frequency switching data can produce negative features that
could affect the intensity and line profile of some of the observed lines.
However, at the level of sensitivity of SANCHO (2-4 mK) only a few lines with
intensities greater than 10 mK could be affected by this issue. However, at
better sensitivities, such as those of the QUIJOTE line survey (Cernicharo et
al. 2020, 2021a, 2023), this effect could be a concern and is mitigated by the
use of two sets of data with different frequency switching throws and similar
sensitivity.

The effect of the gridding step and tolerance circle on the final aspect of
the maps was analysed with the $J$=28-27 transition of HC7N at 31583.709 MHz
(beam size 56′′), and the $J$=1-0 line of C34S at 48206.942 MHz (beam size
39′′). Values of $S_{grid}$=10′′ and 20′′, and $T_{rad}$=10′′, 15′′, and 20′′
were used. These maps are shown in Fig. 6.

Figure 5: Observed lines of CCCN before and after baseline subtraction. For
each transition of CCCN the upper panel shows the raw data and the fitted
baseline (red line) at the centre position of the map with $S_{grid}$=20′′ and
$T_{rad}$=20′′. The panel below each transition shows the data after a
polynomial baseline removal.

Figure 6: Integrated intensity of the $J$=28-27 transition of HC7N (upper
panels) and C34S $J$=1-0 (lower panels) for different values of $S_{grid}$ and
$T_{rad}$. The telescope beam size for the two transitions is 56′′ and 39′′,
respectively. The black dot indicates the centre of the maps. The maximum
value of the integrated intensity changes by less than 10% from $T_{rad}$=10”
to 20′′.

For HC7N both values of $S_{grid}$ produce oversampled maps. When $T_{rad}$ is
changed from 10′′ to 20′′ the maps show a better aspect, but as a counterpart
a considerable spatial smoothing (smearing) is introduced, together with a
small dilution of the maximum intensity. The same applies to the maps of C34S,
which are fully sampled in this case. Depending on the physical parameters we
want to study, different values of $S_{grid}$ and $T_{rad}$ can be used
depending on the intensity of the mapped lines and on the desired signal-to-
noise ratio in each position on the maps. For weak lines the spatial shape of
the emission could be analysed with $T_{rad}$=20′′, while the maps for lines
with intensity greater than 100 mK, $T_{rad}$=10′′ or 15′′ could be better
adapted. In this work we focus on the study of the extended emission of the
weak lines of benzonitrile, and hence we adopt $T_{rad}$=20′′ and
$S_{grid}$=20′′.

We estimated the sensitivity of the adopted values of $S_{grid}$ and $T_{rad}$
by selecting 21 frequencies across the Q-band. A range of $\pm$3 MHz (157
channels) was selected. Three different positions are considered at distances
from the centre of the map of 0′′, 100′′, and 170′′. The results are given in
Table 1.

##  Appendix B Maps of C6H5CN

In order to spectrally stack the lines of C6H5CN and to produce a high signal-
to-noise ratio map of its spatial distribution, we defined a selection
criterion of the lines to be stacked at each position of the maps. The
intensity of all lines of benzonitrile in TMC-1 in the Q-band are below 10 mK
(Cernicharo et al. 2021b). Some of these lines have an intensity of 1-2 mK
which is well below the detection sensitivity of the maps with $T_{rad}$=20′′
(see Table 1). In addition, several of these lines show hyperfine structure
(see Fig. B.1 of Cernicharo et al. 2021b). We used the last product of the
QUIJOTE line survey (Cernicharo et al. 2023) to select all lines of
benzonitrile that appear as a single feature and are free of blending from
other lines. The QUIJOTE data were also used to define the windows for
baseline removal for each of the selected lines.

A second selection criterion was the intensity of the lines relative to the
strongest one that was taken as reference (the 130,13-120,12 transition). Only
lines with one-third of the intensity of the reference line in the QUIJOTE
data were selected. The final list of lines used for stacking corresponds to
the 49 transitions given in Table 2. This table also provides the measured
intensity of the lines with QUIJOTE (Cernicharo et al. 2023) and the applied
multiplicative factor for stacking. This multiplicative factor scales all
lines to the intensity of the reference line. The data were weighted by
1/$\sigma^{2}$, where $\sigma$ is the measured sensitivity of each line after
multiplication by the intensity scale factor. Hence, we can add lines scaled
to the same intensity to produce a stacked spectrum at each position of the
map. Most of the selected lines are seen in the central position of the map
without stacking, but with a limited signal-to-noise ratio. Figure 7 shows a
sub-sample of the selected transitions at the centre position. The
multiplicative factor and the measured sensitivity (after multiplication to
scale the signal to that of the reference line), are indicated in each panel
(see also Table 2).

The final stacked spectra are shown in Fig. 1. The improvement of sensitivity
after stacking is a factor of 3.5 with respect to that of the unstacked
strongest lines.

Figure 7: Selected transitions of C6H5CN among the 49 lines (see Table 2) used
to produce the spectrally stacked map of benzonitrile at each position of the
map. The spectra shown here correspond to the central position. The transition
quantum numbers are indicated in blue in the top left corner of each panel.
Each line arises from the individual maps with $S_{grid}$=20′′ and
$T_{rad}$=20′′. Each line spectrum was multiplied by the factor $f$ (shown in
black in each panel) to scale its intensity to that of the strongest one (the
130,13-120,12 transition). The noise of each spectra, after multiplication by
$f$, is shown in violet ($\sigma$; in mK). It is used to weight the data
during stacking of all the lines at each position of the map (see text). The
vertical dashed red lines in each panel indicate the local standard of rest
velocity of 5.86 km s-1. The spectra of the resulting stacked data at
different positions of the map are shown in Fig. 1. Table 2: Lines of C6H5CN
used for spectral stacking at each position.

Transition | Frequency | T${}_{A}^{*}$a |  $f_{mul}$b
---|---|---|---
| (MHz) | (mK) |
112,9 -102,8 | 32023.223 | 9.7 | 1.05
122,11-112,10 | 32314.461 | 8.8 | 1.16
131,13-121,12 | 32763.506 | 8.4 | 1.21
130,13-120,12 | 32833.827 | 10.2 | 1.00
121,11-111,10 | 33249.003 | 6.2 | 1.65
123,10-113,9 | 33336.175 | 4.6 | 2.43
123,9 -113,8 | 34389.613 | 4.3 | 2.37
132,12-122,11 | 34864.915 | 8.9 | 1.15
122,10-112,9 | 34898.780 | 10.1 | 1.01
141,14-131,13 | 35202.608 | 5.9 | 1.73
140,14-130,13 | 35249.171 | 10.1 | 1.01
131,12-121,11 | 35661.310 | 6.1 | 1.67
133,11-123,10 | 36066.955 | 4.3 | 2.37
134,10-124,9 | 36321.233 | 5.1 | 2.00
134,9-124,8 | 36539.047 | 5.0 | 2.04
142,13-132,12 | 37391.925 | 8.3 | 1.23
133,10-123,9 | 37477.470 | 4.4 | 2.32
151,15-141,14 | 37638.179 | 6.0 | 1.70
150,15-140,14 | 37668.578 | 9.3 | 1.10
132,11-122,10 | 37709.061 | 9.8 | 1.04
141,13-131,12 | 38038.711 | 5.7 | 1.79
143,12-133,11 | 38772.958 | 4.4 | 2.32
144,11-134,10 | 39147.318 | 5.5 | 1.85
144,10-134,9 | 39501.269 | 4.6 | 2.22
152,14-142,13 | 39897.901 | 8.9 | 1.15
161,16-151,15 | 40071.306 | 5.2 | 1.96
160,16-150,15 | 40090.922 | 8.3 | 1.23
151,14-141,13 | 40401.006 | 4.7 | 2.17
142,12-132,11 | 40445.137 | 9.9 | 1.03
143,11-133,10 | 40564.446 | 4.5 | 2.27
154,12-144,11 | 41967.092 | 5.0 | 2.04
162,15-152,14 | 42385.689 | 7.8 | 1.31
171,17-161,16 | 42502.779 | 4.3 | 2.37
154,11-144,10 | 42513.064 | 5.9 | 1.73
170,17-160,16 | 42515.315 | 7.1 | 1.44
161,15-151,14 | 42762.895 | 4.1 | 2.49
152,13-142,12 | 43099.531 | 7.8 | 1.31
153,12-143,11 | 43625.762 | 4.4 | 2.32
164,13-154,12 | 44775.824 | 4.7 | 2.17
180,18-170,17 | 44941.103 | 6.9 | 1.48
171,16-161,15 | 45132.664 | 4.7 | 2.17
164,12-154,11 | 45576.805 | 4.8 | 2.13
162,14-152,13 | 45668.515 | 6.9 | 1.48
173,15-163,14 | 46721.270 | 3.5 | 2.91
182,17-172,16 | 47318.725 | 4.5 | 2.27
190,19-180,18 | 47367.825 | 5.6 | 1.82
181,17-171,16 | 47513.403 | 3.5 | 2.91
174,14-164,13 | 47568.715 | 3.5 | 2.91
172,15-162,14 | 48155.082 | 5.3 | 1.92
777
Selected transitions to generate the map of benzonitrile (C6H5CN).
$a$$a$footnotetext: Observed intensity in the QUIJOTE line survey in mK.
$b$$b$footnotetext: Applied multiplicative factor to the individual map of
each selected transition.