# Transfer Learning from Synthetic In-vitro Soybean Pods Dataset
for In-situ Segmentation of On-branch Soybean Pods

Si Yang1,2,3 , Lihua Zheng1,2, Xieyuanli Chen3, Laura Zabawa3, Man Zhang1,2,
Minjuan Wang1,2,*
1 College of Information and Electrical Engineering, China Agricultural
University
2 Key Lab of Smart Agriculture Systems, Ministry of Education, China
Agricultural University
3 Institute of Geodesy and Geoinformation, University of Bonn, Germany
{yangsi, zhenglh, cauzm<EMAIL_ADDRESS>{xieyuanli.chen,
<EMAIL_ADDRESS>
###### Abstract

The mature soybean plants are of complex architecture with pods frequently
touching each other, posing a challenge for in-situ segmentation of on-branch
soybean pods. Deep learning-based methods can achieve accurate training and
strong generalization capabilities, but it demands massive labeled data, which
is often a limitation, especially for agricultural applications. As lacking
the labeled data to train an in-situ segmentation model for on-branch soybean
pods, we propose a transfer learning from synthetic in-vitro soybean pods.
First, we present a novel automated image generation method to rapidly
generate a synthetic in-vitro soybean pods dataset with plenty of annotated
samples. The in-vitro soybean pods samples are overlapped to simulate the
frequently physically touching of on-branch soybean pods. Then, we design a
two-step transfer learning. In the first step, we finetune an instance
segmentation network pretrained by a source domain (MS COCO dataset) with a
synthetic target domain (in-vitro soybean pods dataset). In the second step,
transferring from simulation to reality is performed by finetuning on a few
real-world mature soybean plant samples. The experimental results show the
effectiveness of the proposed two-step transfer learning method, such that
AP50 was 0.80 for the real-world mature soybean plant test dataset, which is
higher than that of direct adaptation and its AP50 was 0.77. Furthermore, the
visualizations of in-situ segmentation results of on-branch soybean pods show
that our method performs better than other methods, especially when soybean
pods overlap densely.

Keywords:Deep learning; Transfer learning; Computer vision; Instance
segmentation; Plant phenotyping

##  1 Introduction

The crop yield of soybean (Glycine max L.) is heavily influenced by three
major factors: the number of pods per plant, the number of seeds per pod and
the seed size [11, 29]. The most crucial parameter is hereby the number of
pods per plant, which is an important agronomic indicator [21]. Identifying
and segmenting on-branch soybean pods is the prerequisite for acquiring
morphological phenotypic traits of mature soybean plants. Traditionally,
soybean pod counting is performed manually. However, the soybean pods are
small with various shapes and the uncertain number of pods per plant as well
as random occlusion by branches and other pods. This results in an error
prone, time-consuming, and labor-intensive manual phenotyping procedure [24].
Therefore, it is infeasible for large-scale mature soybean plant phenotype
investigation.

Figure 1: Overview of the proposed two-step transfer learning for in-situ
segmentation of on-branch soybean pods

Aided by the rapid gains in imaging technology, high throughput phenotyping
became possible when the crops are sparsely or regularly placed with weakly
physical contact [1, 33]. Some open-source image analysis methods have been
used in high throughput crop phenotyping. The main idea of these methods is
segmenting crops based on classic and ordinary image processing techniques,
such as the watershed algorithm [18, 15], morphological opening and closing
operation [26], and tailored algorithm with handcrafted features [13, 31],
etc. Nevertheless, these methods can realize high throughput crop instance
segmentation, but they are sensitive to changing illumination conditions and
texture features of object, and are also inadequate in robustness and
generalization ability. Additionally, the crops need to be sparsely
distributed with small overlap under a consistent light condition to ease
segmentation.

Deep learning has greatly improved the performance on almost every computer
vision task, and can achieve an effective segmentation by learning deep
features from large annotated image datasets to solve the above-mentioned
problems [20]. Some existing methods are used in high throughput crop
phenotyping, including fruit counting, detection, and instance segmentation
[34, 23, 22, 33], where the changes in the shapes of fruits are relatively
small. Deep learning applied in plant phenotyping has grown exponentially in
the past few years [16]. However, training an accurate deep learning algorithm
with strong generalization ability requires a large amount of labeled data
which is one of the disadvantages of deep learning. Compared with relatively
common tasks, such as classification in the ImageNet dataset[8] and object
detection in COCO dataset[19], the demands for large annotated data for
specialized tasks in agricultural applications is even more pronounced [9, 12,
4]. However, manual annotation is expensive, especially for the instance
segmentation tasks in the plant phenotyping realm. Although many techniques
aim to decrease the cost of manual labeling, such as domain adaptation [25] or
active learning [4], without compromising performance, the tedious, painful,
labor-intensive, and time-consuming labeling process is still needed to
evaluate the algorithm. Especially in high-throughput crop phenotyping, the
annotation of crop instance dataset poses a tremendous challenge.

An option to reduce the cost of manual annotation is learning from synthetic
data [10, 28]. Although the synthetic data is not faithful compared with real-
word images, the critical characteristic of synthetic dataset is that ground
truth annotations can be automatically obtained [2]. Furthermore, the
synthesizing data approach is able to create almost unlimited amount of
labeled datasets [7]. Furthermore, synthetic data can represent changes in a
variety of conditions, which is usually difficult to achieve through applying
image augmentation techniques on real world images [30]. Kuznichov et al. [17]
proposed a method to segment and count the leaves of Arabidopsis, avocado, and
banana by using synthetic leaf textures with different sizes and angles to
simulate images obtained in real agricultural scenes. Toda et al. [27] proved
that a synthetic dataset, rendering the combination and direction of seeds,
was sufficient to train an instance segmentation network to segment barley
seeds from real-world images. Collectively, synthetic datasets have great
potential in the computer vision-based plant phenotyping research field [32].

On-branch soybean pods segmentation and counting pose a challenge, since they
feature a complex architecture and a high level of overlap with each other. As
lacking the labeled data for training an in-situ segmentation model of on-
branch soybean pods, we present a two-step transfer learning method based on a
synthetic high throughput in-vitro soybean pods dataset which is prepared by
our novel automated image generation method. Figure 1 illustrates the overview
of our proposed two-step transfer learning for in-situ segmentation of on-
branch soybean pods.

The main contribution of this study are as follows:

(1) A novel synthetic image generation method is proposed for automatically
creating labeled high throughput in-vitro soybean pods image sets.

(2) A new hybrid sim/real and in-vitro/on-branch dataset, including synthetic
in-vitro soybean pods and real-world mature soybean plants, is designed for
transferring from simulation to reality and from in-vitro segmentation to on-
branch segmentation robustly.

(3) The proposed two-step transfer learning method with a tiny Swin
transformer-based instance segmentation network achieves a decent performance
for in-situ segmentation of on-branch soybean pods. To the best of our
knowledge, this is the first work utilizing synthetic high throughput in-vitro
soybean pods for simulating on-branch soybean pods of mature soybean plants.

##  2 Methods

###  2.1 Synthetic in-vitro soybean pods image generation

In our proposed pipeline of computer vision-based high throughput in-vitro
soybean pods phenotype investigation, the soybean pods are picked manually
from a single plant sample. They are then tiled upon a simple background like
black-colored flannel randomly [5]. After that, these pods are scanned by the
camera sensor of an iPhone 8 plus (Apple) mounted on a tripod and saved as an
image with the size of $3024\times 4032$ shown in Figure 2. The working
distance of the camera sensor was fixed at about 30 cm above the background.

Figure 2: Examples of scanned raw in-vitro soybean pods images in which the
soybean pods were picked from one plant sample manually and randomly tiled
upon the black-colored flannel Figure 3: Examples of SPIP. each column
represent a different cultivars. (a) BJ103, (b) BJ125, (c) BJ218, (d) BJ226,
(e) BJ264, (f) BJ335, (g) BJ351, (h)BJ356

At the stage of dataset preparation, we randomly chose 6 samples for each
soybean plant accession (total of 48; 6 samples for 8 cultivars). One sample
of each soybean plant accession (total of 8 soybean plants; one sample for 8
cultivars) is chosen to create the synthetic image dataset. The soybean pods
picked from one sample manually are tiled above the black flannel randomly,
scanned by our camera sensor, and saved as an individual image file. The rest
samples of each accession (total of 40 soybean plants; 5 samples for 8
cultivars) are used as real-world on-branch soybean plants training dataset.

We now introduce our proposed method for generating synthetic images. First,
we prepare a background image pool (BIP) and a soybean pod image pool (SPIP).
The BIP was consist of 10 images, which prepared by capturing the actual black
flannel background 10 times and cropped at the fixed size of $1024\times
1024$. The SPIP was constituted by segmenting each pod from raw in-vitro high
throughput soybean pods images and saved as an individual pod image processed
with Photoshop111Adobe Photoshop CC 2020, Adobe Systems Incorporated, San
Jose, CA, USA; http://www.adobe.com/Photoshop, as shown in Figure 3. The
images of the SPIP are zero-padded, to prevent the pod from moving outside the
image after rotation and shift. Last, generate high-throughput in-vitro
soybean pods images based on the BIP and SPIP prepared in the previous step.
Algorithm 1 depicts the detail of the above described last step, generating
high-throughput in-vitro soybean pods image, using pseudo code.

Algorithm 1 Generation of fine-labeled high throughput in-vitro soybean pods
image

0: BIP and SPIP

0: a high-throughput in-vitro soybean pods image, a fine-labeled mask image
with different color

1: create a raw canvas, a mask canvas.

2: paste a background image chosen from the BIP randomly onto the raw canvas.

3: fill the mask canvas with absolute black color.

4: while generate a satisfying coordinate or less than max iteration do

5: select one pod image from SPIP randomly.

6: rotate, shift and zoom at random.

7: generate new pod position coordinate $\left(x_{i},y_{i}\right)$ randomly.

8: if $\left(0,0\right)<\left(x_{i},y_{i}\right)<\left(w,h\right)$ and overlap
degree of $\left(x_{i},y_{i}\right)$ and $\left(x_{j},y_{j}\right)$ satisfy
$threshold$ then

9: paste the selected pod image onto the generated coordinate of canvas.

10: create the mask image with different color according to the selected
image.

11: paste the created mask image on the mask canvas’s counterpart position.

12: end if

13: end while

Note that, in Algorithm 1, $\left(x_{i},y_{i}\right)$ is coordinate of the
$i^{th}$ pod will be pasted on canvas, while $\left(x_{j},y_{j}\right)$ is
coordinate of $0^{th}\sim j^{th}$ pods has been pasted on canvas. $w$ and $h$
are the width and height of canvas. The parameter $threshold$ is dynamic
calculated related to the size of the soybean pod bounding box, in detail, the
width of $i^{th}$ and $j^{th}$ soybean pod. The new coordinate
$\left(x_{i},y_{i}\right)$ is generated randomly but with two limitations, the
boundary of the canvas as well as the minimum Euclidean distance between the
newly generated coordination $\left(x_{i},y_{i}\right)$ and the coordinates
$\left(x_{j},y_{j}\right)$ of the soybean pods pasted on the canvas before.
These two restriction factors keep the pod within the canvas, and adjust the
degree of overlap respectively.

###  2.2 Real-world mature soybean plant dataset

As described in section 2.1, we generate the synthetic labeled high throughput
in-vitro soybean pods images. Now we are preparing a real-world mature soybean
plant dataset by capturing images of individual mature soybean plants using
the camera sensor of an iPhone 8 plus (Apple) mounted on a tripod, and saving
them as an image with the size of $3024\times 4032$ as shown in Figure 4. The
working distance between the camera sensor and the black-colored flannel
background is adjusted to about 1 m to capture the whole soybean plant image.
We randomly chose 60 specimens of mature soybean plants to constitute the
real-world mature soybean plant dataset. 40 of them are used for training and
validation mentioned in section 2.1, 20 for testing. In addition, the mature
soybean plants are not used in generating the synthetic in-vitro soybean pods
image dataset. After manual annotation, the images are down-scaled since the
size of the image acquired by our came sensor is $3024\times 4032$, which is
too large for the computing of the instance segmentation network.

Figure 4:  Examples of the captured mature soybean plant images which the
background is black-colored flannel

###  2.3 Model training for two-step transfer learning

Swin Transformer, a hierarchical Transformer whose representation is computed
with shifted windows, performed pretty well in dense prediction tasks [20]. To
adapt the tiny Swin Transformer to our in-situ segmentation of on-branch
soybean pods of mature soybean plants with complicated structures and frequent
physical overlap, we reduce the number of classes to two. Each instance mask
is classified as a foreground object or background, while only visualizing the
foreground mask. We use our synthetic in-vitro soybean pods dataset with large
amounts of labeled data and a real-world mature soybean plant dataset with a
few labeled samples for our two-step transfer learning in the target domain.
In the first step, we finetune the pre-trained model weights trained by the
MS-COCO dataset [19] on our synthetic in-vitro soybean pods dataset. Then, in
the second step, transferring from simulation to reality is performed by
finetuning on the few real-world mature soybean plant samples, as shown in
Figure 5. To increase the diversity of dataset, the up-down, rotation,
brightness and Gaussian blur image augmentations are used herein. The mean
pixel value is set as the average pixel value of the simulated dataset. Tiny
Swin Transformer is chosen as the backbone to exact features. Two evaluation
metrics included average precision (AP) and recall, used to evaluate in the
original research [14], are also introduced herein as the evaluation criteria.
The comparative experiments of different instance segmentation network with
our two step transfer learning is supplied in the supplemental material.

Figure 5: Tiny Swin Transformer based on-branch soybean pods instance
segmentation of mature soybean plant with two step transfer learning

##  3 Experiments and results

###  3.1 Software libraries and hardware

The processing unit was a computer with an Intel Core<EMAIL_ADDRESS>CPU,
16GB RAM, and a single GPU (8G, Geforce GTX2060 supper, NVIDIA). The synthetic
image generation-related processes operated on the environment, including
Integration Develop Environment (IDE), integrating Python 3.6, and OpenCV3
(ver. 3.4.2). The manual annotation of real-world mature soybean plant images
was implemented with the Photoshop program.

###  3.2 Preparation of pods instance segmenatation dataset

We generated high throughput in-vitro soybean pods images with the size of
$1024\times 1024$. The pods were randomly and densely located inside the
canvas region by our procedure, as mentioned in section 2.1.

We prepared a number of training datasets of synthetic in-vitro soybean pods
images to do transfer learning from the source domain (MS COCO dataset) to the
target domain (synthetic in-vitro soybean pods). The prepared synthetic
datasets include different amounts of (training/validation/testing) data with
different overlapping degrees as shown in Table 1 and Table 2 to validate the
efficiency of data mount and overlapping degrees of in-vitro soybean pods for
in-situ segmentation of on-branch soybean pods. In other words, we
investigated 40 datasets, 10 datasets for all 4 different overlap degrees. The
visualization of different overlapping degrees of soybean pods is shown in
Figure 6. We also prepared another new 200 sets of image pairs as a synthetic
test dataset, and those synthetic images were not used in the model training
or validation. The overlapping degree is a decisive coefficient for the
purpose of calculating the dynamic overlap threshold as mentioned in section
2.1. When the overlapping degree decreases, the overlap threshold also
decreases, resulting in a dense overlap of soybean pods. The overlapping
degree is the key parameter to simulate the real world frequently physically
touching on-branch soybean pods in our proposed synthetic dataset generation
scheme.

An example of the real-world mature soybean plant dataset is shown in Figure
7. The real-world mature soybean plant dataset is labeled by Photoshop. The
software is used for pod matting, and each matting pod was saved as a binary
image. We found that there is a plethora of soybean pods per image leading to
more time intensive labeling. The time of manual annotation process with
LabelMe is about 90 min per image. The real-world mature soybean plant dataset
is constituted of 60 image pairs of raw soybean images and its mask images, 36
of those images for training, 4 for validation, and 20 for testing.

Overall, the prepared different amounts of datasets for our two-step transfer
learning consist of an in-vitro soybean pods dataset (Dataset_in-vitro) and a
real-world mature soybean plant dataset (Dataset_on-branch) summarized in
Table 3. The Dataset_in-vitro is constituted of synthetic in-vitro soybean
pods images and real-world in-vitro soybean pods images.

Figure 6: The synthetic in-vitro high throughput soybean pods image with different overlapping degrees. (a) the overlapping coefficient is 0.4 (b) the overlapping coefficient is 0.3 (c) the overlapping coefficient is 0.2 (d) the overlapping coefficient is 0.1 Table 1:  Synthetic in-vitro soybean pods datasets of one overlap degree with different amounts of (training/validation/testing) data. Dataset ID | Training | Validation | Testing
---|---|---|---
1 | 200 | 20 | 200
2 | 400 | 40
3 | 600 | 60
4 | 800 | 80
5 | 1000 | 100
6 | 1200 | 120
7 | 1400 | 140
8 | 1600 | 160
9 | 1800 | 180
10 | 2000 | 200
Table 2:  Synthetic in-vitro soybean pods datasets with different overlapping coefficients. Overlapping coefficient |  Image size |  Pod count |  Processing time per image/s
---|---|---|---
0.1 | 1024 * 1024 | 212 | 28
0.2 | 192 | 25
0.3 | 156 | 22
0.4 | 112 | 18
Table 3: The detail of the prepared dataset for our two-step transfer learning |  | Synthetic in-vitro soybean pods dataset
---
Dataset in-vitro
| Real-world mature soybean plant dataset
---
(Dataset on-branch)
| Training | validation | Testing | Training | validation | Testing
Different amount of (training/validation) data as Table 1 illustrated | 200 | 36 | 4 | 20
Figure 7: One labeled image data of real-world soybean pods test dataset
annotated by Adobe Photoshop software. (a) raw image, (b) instance masks,
bounding boxes, class names

###  3.3 Efficiency of different amounts of data with different overlapping
degree

To investigate the impact of different amounts of data with different
overlapping degrees, we explicitly present a set of two-step transfer learning
experiments on the prepared dataset as mentioned in section 3.2. We use our
real world mature soybean plant test dataset to evaluate the tiny Swin
transformer-based two-step transfer learning model trained on the different
amounts of data with different overlapping degree datasets. The variation in
AP50 of different datasets is shown in Figure 8. We found that the best
performance is transfer learning the dataset which includes 1600 synthetic in-
vitro soybean pods images and its overlapping degree is 0.1. And we can
witness that increasing the overlapping degree can increase the performance
dramatically. We also learned that increasing the training data amount is not
significant.

Figure 8: The heat map of evaluation metric (AP50) instance segmentation
results on real world mature tiny Swin transformer based two-step transfer
learning model trained with different amounts of data with different
overlapping coefficient.

###  3.4 Effectiveness of two-step transfer learning

We investigate the effectiveness of our two-step transfer learning from the
synthetic in-vitro high throughput soybean pods dataset for in-situ
segmentation of on-branch soybean pods. As mentioned in section 3.2, the
prepared dataset for our two-step transfer learning consists of an in-vitro
soybean pods dataset (Dataset_in-vitro) and a real-world mature soybean plant
dataset (Dataset_on-branch). The Dataset_in-vitro was constituted of synthetic
in-vitro soybean pods images and real-world in-vitro soybean pods images.

(1) COCO, Dataset_in-vitro. The backbone network parameters were initialized
with the parameters trained by COCO dataset. Then the network is finetuned by
our synthetic in-vitro soybean pods dataset (Dataset_in-vitro) in the target
domain.

(2) COCO, Dataset_on-branch. The backbone network parameters were initialized
with the parameters trained by COCO dataset. Then the network is finetuned by
real-world mature soybean plant dataset with fewer samples (Dataset_on-branch)
in the target domain which is one-step transfer learning.

(3) COCO, Dataset_in-vitro, Dataset_on-branch. This is the proposed approach,
named ”Ours”. The backbone network parameters were initialized with the
parameters trained by COCO dataset. Then the network is finetuned by our
synthetic in-vitro soybean pods dataset (Dataset_in-vitro). We then apply
transfer learning from Dataset_in-vitro to Dataset_on-branch.

The quantitative and qualitative results of in-situ instance segmentation of
on-branch soybean pods with different instance segmentation network and
different transfer learning strategies are shown in Table 4 and Figure 9
respectively. As shown in Table 4, our proposed method outperforms the
baseline methods, which is in line with the results shown in Figure 9, where
the proposed two-step transfer learning method achieves more accurate
segmentation results. Thus, the results demonstrate that the synthetic in-
vitro soybean pods dataset can be the supplement of mature soybean plant
dataset for in-situ segmentation of on-branch soybean pods. More evaluation
results can be seen in the supplemental material. The comparative experiments
reveal that the model which was fine-tuned on the synthetic in-vitro soybean
pod dataset only can learn the pods region roughly. However, if the model
transfer learning from synthetic in-vitro soybean pod dataset in the first
step, and then fine-tuning on the real-world soybean plant dataset, the
performance will be improved compared with directly finetuned on real-world
soybean plant dataset.

Table 4:  Evaluation on mature soybean plant test dataset with different instance segmentation models and transfer learning strategies. Transfer learning strategy | Recall@[.5:.95] | AP50 | AP75 | AP@[.5:.95]
---|---|---|---|---
Yolact+(COCO,Dataset_in-vitro) | 0.042 | 0.043 | 0.019 | 0.021
Yolact+(COCO,Dataset_on-branch) | 0.319 | 0.474 | 0.079 | 0.168
Yolact+ours | 0.419 | 0.579 | 0.219 | 0.269
Mask RCNN+(COCO,Dataset_in-vitro) | 0.052 | 0.043 | 0.021 | 0.022
Mask RCNN+(COCO,Dataset_on-branch) | 0.547 | 0.687 | 0.379 | 0.392
Mask RCNN+ours | 0.555 | 0.694 | 0.423 | 0.398
Blendmask+(COCO,Dataset_in-vitro) | 0.260 | 0.037 | 0.009 | 0.015
Blendmask+(COCO,Dataset_on-branch) | 0.543 | 0.656 | 0.345 | 0.352
Blendmask+ours | 0.556 | 0.686 | 0.345 | 0.352
Swin transformer+(COCO,Dataset_in-vitro) | 0.125 | 0.100 | 0.037 | 0.045
Swin transformer+(COCO,Dataset_on-branch) | 0.535 | 0.773 | 0.519 | 0.477
Swin transformer+ours | 0.575 | 0.80 | 0.579 | 0.522
Figure 9: One example of visualized output results of real-world mature
soybean plant dataset with different transfer learning strategies. The network
is tiny Swin Transformer based instance segmentation network. The first column
is the result of the pre-trained model purely retrained by our synthetic in-
vitro soybean pods dataset. The second column is the result of the pre-trained
model only retrained by our real world mature soybean plant. The third column
is the result of the pre-trained model first retrained by our synthetic in-
vitro soybean pods dataset and finetuned on our real world mature soybean
plant dataset.

##  4 Conclusion

The proposed method can be employed in automatic pods counting after instance
segmentation of on-branch soybean pods. It solves the high cost and low
efficiency problems of traditional artificial pods counting such as poor
accuracy, and provides a richer and more accurate phenotypic parameter
reference for breeding experts to carry out the breeding design. Moreover,
farmers can utilize it to estimate yields of soybean quickly.

The major contribution and advantages of our method are: (1) A novel synthetic
image generation method is proposed for automatically creating labeled high
throughput in-vitro soybean pods image set. (2) A new hybrid sim/real and in-
vitro/on-branch dataset, including synthetic invitro soybean pods dataset and
real-world mature soybean plant dataset, designed for transferring from
simulation to reality and from in-vitro segmentation to on-branch segmentation
robustly. (3) The proposed two-step transfer learning method on a tiny Swin
transformer based instance segmentation network achieves a decent performance
for in-situ segmentation of on-branch soybean pods, which is the first work
utilizing a synthetic high throughput in-vitro soybean pods images dataset for
mimicking on-branch soybean pods of the mature soybean plant which are of
complex architecture with pods frequently touching each other.

However, the 2D image based method does not carry the depth (range)
information about the plant leading a poor segmentation performance for badly
occluded pods.

In the future, we plan to similarly analyze in-situ segmentation and seed-per-
pod estimation of on-branch soybean pod for mature soybean plant phenotype
investigation. Another avenue for future work is incorporating spatial
information into the analysis to improve segmentation accuracy.

##  5 Acknowledgement

This work was supported jointly by National Natural Science Foundation of
China (31971786), the China Agriculture Research System(CARS-04), the National
Key R&D Program of China (2017YFD0101400) and the Chinese Universities
Scientific Fund(2021TC111). All of the mentioned support is gratefully
acknowledged. In addition, thanks for all the help of the teachers and
students of the related universities.

## References

  * [1] JeongHo Baek, Eungyeong Lee, Nyunhee Kim, Song Lim Kim, Inchan Choi, Hyeonso Ji, Yong Suk Chung, Man-Soo Choi, Jung-Kyung Moon, and Kyung-Hwan Kim.  High throughput phenotyping for various traits on soybean seeds using image analysis.  Sensors, 20(1):248, 2020.
  * [2] Ruud Barth, Joris IJsselmuiden, Jochen Hemming, and Eldert J Van Henten.  Data synthesis methods for semantic segmentation in agriculture: A capsicum annuum dataset.  Computers and electronics in agriculture, 144:284–296, 2018.
  * [3] Daniel Bolya, Chong Zhou, Fanyi Xiao, and Yong Jae Lee.  Yolact: Real-time instance segmentation.  In Proceedings of the IEEE/CVF international conference on computer vision, pages 9157–9166, 2019.
  * [4] Akshay L Chandra, Sai Vikas Desai, Vineeth N Balasubramanian, Seishi Ninomiya, and Wei Guo.  Active learning with point supervision for cost-effective panicle detection in cereal crops.  Plant Methods, 16(1):1–16, 2020.
  * [5] Fangguo Chang, Wenhuan Lv, Peiyun Lv, Yuntao Xiao, Wenliang Yan, Shu Chen, Lingyi Zheng, Ping Xie, Ling Wang, Benjamin Karikari, et al.  Exploring genetic architecture for pod-related traits in soybean using image-based phenotyping.  Molecular Breeding, 41(4):1–21, 2021.
  * [6] Hao Chen, Kunyang Sun, Zhi Tian, Chunhua Shen, Yongming Huang, and Youliang Yan.  Blendmask: Top-down meets bottom-up for instance segmentation.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8573–8581, 2020.
  * [7] Michael Danielczuk, Matthew Matl, Saurabh Gupta, Andrew Li, Andrew Lee, Jeffrey Mahler, and Ken Goldberg.  Segmenting unknown 3d objects from real depth images using mask r-cnn trained on synthetic data.  In 2019 International Conference on Robotics and Automation (ICRA), pages 7283–7290. IEEE, 2019.
  * [8] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009.
  * [9] Sai Vikas Desai, Vineeth N Balasubramanian, Tokihiro Fukatsu, Seishi Ninomiya, and Wei Guo.  Automatic estimation of heading date of paddy rice using deep learning.  Plant Methods, 15(1):1–11, 2019.
  * [10] Debidatta Dwibedi, Ishan Misra, and Martial Hebert.  Cut, paste and learn: Surprisingly easy synthesis for instance detection.  In Proceedings of the IEEE international conference on computer vision, pages 1301–1310, 2017.
  * [11] Walter R Fehr and James R Justin.  Principles of cultivar development, vol. 2, crop species.  Soil Science, 145(5):390, 1988.
  * [12] Sambuddha Ghosal, Bangyou Zheng, Scott C Chapman, Andries B Potgieter, David R Jordan, Xuemin Wang, Asheesh K Singh, Arti Singh, Masayuki Hirafuji, Seishi Ninomiya, et al.  A weakly supervised deep learning framework for sorghum head detection and counting.  Plant Phenomics, 2019, 2019.
  * [13] Frank E Groves, Freddie M Bourland, et al.  Estimating seed surface area of cottonseed.  J. Cotton Sci, 14:74–81, 2010.
  * [14] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick.  Mask r-cnn.  In Proceedings of the IEEE international conference on computer vision, pages 2961–2969, 2017.
  * [15] C Igathinathane, LO Pordesimo, EP Columbus, WD Batchelor, and SR Methuku.  Shape identification and particles size distribution from basic shape parameters using imagej.  Computers and electronics in agriculture, 63(2):168–182, 2008.
  * [16] Andreas Kamilaris and Francesc X Prenafeta-Boldú.  Deep learning in agriculture: A survey.  Computers and electronics in agriculture, 147:70–90, 2018.
  * [17] Dmitry Kuznichov, Alon Zvirin, Yaron Honen, and Ron Kimmel.  Data augmentation for leaf segmentation and counting tasks in rosette plants.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 0–0, 2019.
  * [18] Michael R Lamprecht, David M Sabatini, and Anne E Carpenter.  Cellprofiler™: free, versatile software for automated biological image analysis.  Biotechniques, 42(1):71–75, 2007.
  * [19] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick.  Microsoft coco: Common objects in context.  In European conference on computer vision, pages 740–755. Springer, 2014.
  * [20] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo.  Swin transformer: Hierarchical vision transformer using shifted windows.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10012–10022, 2021.
  * [21] Michael McGuire, Chinmay Soman, Brian Diers, and Girish Chowdhary.  High throughput soybean pod-counting with in-field robotic data collection and machine-vision based data analysis.  arXiv preprint arXiv:2105.10568, 2021.
  * [22] Anjana K Nellithimaru and George A Kantor.  Rols: Robust object-level slam for grape counting.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 0–0, 2019.
  * [23] Michael P Pound, Jonathan A Atkinson, Darren M Wells, Tony P Pridmore, and Andrew P French.  Deep learning for multi-task plant phenotyping.  In Proceedings of the IEEE International Conference on Computer Vision Workshops, pages 2055–2063, 2017.
  * [24] Luis G Riera, Matthew E Carroll, Zhisheng Zhang, Johnathon M Shook, Sambuddha Ghosal, Tianshuang Gao, Arti Singh, Sourabh Bhattacharya, Baskar Ganapathysubramanian, Asheesh K Singh, et al.  Deep multi-view image fusion for soybean yield estimation in breeding applications deep multi-view image fusion for soybean yield estimation in breeding applications.  arXiv preprint arXiv:2011.07118, 2020.
  * [25] Shunsuke Sakurai, Hideaki Uchiyama, Atsushi Shimada, Daisaku Arita, and Rin-ichiro Taniguchi.  Two-step transfer learning for semantic plant segmentation.  In ICPRAM, pages 332–339, 2018.
  * [26] Takanari Tanabata, Taeko Shibaya, Kiyosumi Hori, Kaworu Ebana, and Masahiro Yano.  Smartgrain: high-throughput phenotyping software for measuring seed shape through image analysis.  Plant physiology, 160(4):1871–1880, 2012.
  * [27] Yosuke Toda, Fumio Okura, Jun Ito, Satoshi Okada, Toshinori Kinoshita, Hiroyuki Tsuji, and Daisuke Saisho.  Training instance segmentation neural network with synthetic datasets for crop seed phenotyping.  Communications biology, 3(1):1–12, 2020.
  * [28] Jordan Ubbens, Mikolaj Cieslak, Przemyslaw Prusinkiewicz, and Ian Stavness.  The use of plant models in deep learning: an application to leaf counting in rosette plants.  Plant methods, 14(1):1–10, 2018.
  * [29] Lucas C Uzal, Guillermo L Grinblat, Rafael Namías, Mónica G Larese, Julieta Sofia Bianchi, Eligio Natalio Morandi, and Pablo M Granitto.  Seed-per-pod estimation for plant breeding using deep learning.  Computers and electronics in agriculture, 150:196–204, 2018.
  * [30] Wenqiang Xu, Yonglu Li, and Cewu Lu.  Srda: Generating instance segmentation annotation via scanning, reasoning and domain adaptation.  In Proceedings of the European Conference on Computer Vision (ECCV), pages 120–136, 2018.
  * [31] Si Yang, Lihua Zheng, Wanlin Gao, Bingbing Wang, Xia Hao, Jiaqi Mi, and Minjuan Wang.  An efficient processing approach for colored point cloud-based high-throughput seedling phenotyping.  Remote Sensing, 12(10):1540, 2020.
  * [32] Si Yang, Lihua Zheng, Huijun Yang, Man Zhang, Tingting Wu, Shi Sun, Federico Tomasetto, and Minjuan Wang.  A synthetic datasets based instance segmentation network for high-throughput soybean pods phenotype investigation.  Expert Systems with Applications, 192:116403, 2022.
  * [33] Yang Yu, Kailiang Zhang, Li Yang, and Dongxing Zhang.  Fruit detection for strawberry harvesting robot in non-structural environment based on mask-rcnn.  Computers and Electronics in Agriculture, 163:104846, 2019.
  * [34] Laura Zabawa, Anna Kicherer, Lasse Klingbeil, Reinhard Töpfer, Heiner Kuhlmann, and Ribana Roscher.  Counting of grapevine berries in images via semantic segmentation using convolutional neural networks.  ISPRS Journal of Photogrammetry and Remote Sensing, 164:73–83, 2020\.

##  6 Appendix

A. Addition results of our two-step transfer learning method with different
instance segmentation networks

We conduct our two- step transfer learning method with different instance
segmentation networks (Yolact [3], Mask RCNN [14], BlendMask [6], tiny Swin
Transformer-based [20] instance segmentation network ) on our on-branch
soybean pods phenotyping task. We also compare our two-step transfer learning
method with the model purely trained by our synthetic in-vitro soybean pods
dataset and the model only finetuned on a real world mature soybean plants
dataset. The visualized results can be seen in Figure 10. we can find that the
performance of all the instance segmentation models can be improved observably
by our two-step transfer learning method. And we can conclude that the model
purely trained by our synthetic in-vitro soybean pods dataset can realize a
very rough segmentation; the model only finetuned on a few real world mature
soybean plants dataset can realize in-situ segmentation of on-branch soybean
pods with low accuracy; the performance can be increased by our two-step
transfer learning method which firstly trained by our synthetic dataset and
then finetuned by a real world mature soybean plants dataset.

Figure 10: The effectiveness and efficiency of our two-step transfer learning
method with different instance segmentation network

B. Addition results of different real world mature soybean plants

As shown in Figure 11 and Figure 12, we report another on-branch soybean pod
in-situ instance results of different mature soybean plant in the test dataset
with the best model which finetuned the tiny Swin transformer based instance
segmentation network with the 1600 synthetic in-virtu soybean pods images with
0.1 overlapping degree and a few samples of real world mature soybean plant
images. These samples evaluation results of the best model as shown in Table
5.

Table 5:  Some mature soybean plant samples evaluation results of the best model. Sample | (1) | (2) | (3) | (4)
---|---|---|---|---
Recall@[.5;.95] | 0.566 | 0.517 | 0.596 | 0.577
AP50 | 0.924 | 0.779 | 0.876 | 0.777
AP75 | 0.637 | 0.581 | 0.689 | 0.680
AP@[.5;.95] | 0.763 | 0.618 | 0.686 | 0.698
Figure 11: On-branch soybean pod in-suit instance result of different mature
soybean plant sample in the test dataset with the best model Figure 12: On-
branch soybean pod in-suit instance result of different mature soybean plant
sample in the test dataset with the best model


# The Raman gap and collisional absorption

Ido Barth<EMAIL_ADDRESS>Racah Institute of Physics, The Hebrew
University of Jerusalem, Jerusalem, 91904 Israel  Department of Physics,
University of California, Berkeley, California 94720, USA  Pierre Michel
Lawrence Livermore National Laboratory, California 94551, USA

###### Abstract

One of the long-standing puzzles observed in many laser-plasma experiments is
the gap in the Raman backscattering spectrum. This gap is characterized by the
absence of backscattered light between some critical wavelength and twice the
incident laser wavelength. The latter is associated with the absolute Raman
instability from the quarter-critical density surface. Supported by particle-
in-cell (PIC) simulations, it is suggested that the gap can result from the
collisional damping of the backscattered light. A linear analysis of the
competition between the Raman growth rate and the damping rate in a non-
homogenous plasma predicts the gap’s existence and width as a function of the
system’s parameters. The theory is compared with the PIC simulations and past
experiments.

##  I Introduction

The stimulated Raman backscattering (SRS) spectra in laser plasma fusion
experiments e.g., NIF and Omega, pose a long-standing puzzle. The wavelength
of the backscattered light outside the plasma, $\lambda_{1}$, is theoretically
limited to the range $[\lambda_{0},2\lambda_{0}]$, where $\lambda_{0}$ is the
incident laser wavelength. This range results from the Raman resonance
conditions

$\displaystyle\omega_{0}$ $\displaystyle=\omega_{1}+\omega_{2}$ (1)
$\displaystyle k_{2}$ $\displaystyle=k_{0}+k_{1}$ (2)

where $\omega_{0}$, $\omega_{1}$, and $\omega_{2}$ are the incident laser,
backscattered light, and the electron plasma wave (EPW), respectively, and
$k_{0,1,2}$ are the respective wave numbers at the location of the Raman
interaction inside the plasma. $\omega_{2}$ can be approximated as the plasma
frequency, $\omega_{p}$, by neglecting the thermal correction that changes the
SRS spectrum by a few percent for a few KeV electron temperatures.

However, a typical experimental SRS spectrum is characterized by a gap in the
spectrum between some critical wavelength, $\lambda_{\text{gap}}$ and
$2\lambda_{0}$.Simon  _et al._ (1986) This gap, which was not foreseen at the
beginning, was first found in experiments, starting with a small amplitude and
becoming even more prominent for higher laser intensities. Since 1985, many
theoretical explanations have been suggested, including Electron density
steepening near the quarter-critical density,Tanaka  _et al._ (1982); Shepard
_et al._ (1986) Large EPW seed driven by a non-Maxwellian distribution with
fast electrons,Simon  _et al._ (1986) a competition between SRS and stimulated
Brillouin scattering (SBS),Baldis  _et al._ (1989) high sensitivity to
(damping driven) detuning near the quarter-critical densityBarr, Boyd, and
Mackwood (1993), nonlinear saturation of the plasma wave driven by Langmuir
decay instability (LDI),Drake and Batha (1991) density-dependent diffraction
threshold,Rose and Mounaix (2011) and back-scatter being overtaken by absolute
side-scatter at the relevant densities.Rosenberg  _et al._ (2020); Michel  _et
al._ (2019)

However, the variety and complexity of these explanations hint that the Raman
gap effect is not well understood. Additionally, as far as we know, the Raman
gap was never demonstrated within ab initio particle-in-cell (PIC)
simulations.

In this paper, we study the effect of collisional damping of the backscattered
light on the SRS spectrum as observed outside the plasma. The idea is that the
backscattered light is absorbed in the plasma on its way out. The amount of
absorption depends on the optical depths of the plasma. Particularly, the
larger the backscattered wavelength, the higher density it was scattered from,
and the longer the way the backscattered light passes in the plasma and
through higher densities. The latter is because, typically, the incident laser
penetrates along the density gradient towards higher plasma densities.
Therefore, it is anticipated that for some reflecting point, $z_{r}$, the
absorption will be strong enough to cancel the amplification such that the
total gain would be zero. From this point up to the quarter critical density
(where the Raman backscattering becomes absolute and thus much higher), no
backscattered light is anticipated to exit the plasma, resulting in a gap in
the SRS spectrum, as many experiments exhibit.

The paper is organized as follows. In Sec. II, we present a linear analysis of
the problem and show that the gap can result from collisional absorption. A
theoretical prediction of the gap location is derived using a single,
dimensionless parameter. In Sec. III, we study the Raman gap effect via one-
dimension (1D) particle-in-cell (PIC) simulations and discuss the possible
explanations in the relevant parameter regimes. In Sec. IV, we compare our
theoretical prediction with past experiments and PIC simulation results. In
Sec. V, we briefly discuss other theoretical explanations for the Raman gap
effect developed in the literature and test their validity against past
experimental results and our PIC simulations. Sec. VI summarizes the
conclusions.

##  II Theory

Our analysis is based on the competition between the Raman amplification and
the collisional absorption of the backscattered light. If the former is
larger, then backscattered light will be observed, while in the opposite case,
all of the backscattered light will be absorbed in the plasma, and no
reflected light will be measured. Since both amplification and absorption
depend on the reflection point (defined as the origin of the backscattered
light for a given wavelength), one can compare the growth and dumping rates
for a wave backscattered from a given location in the plasma. If the
amplification gain is larger than the total damping, SRS backscattered light
is expected to be seen in the wavelength associated with the given reflecting
point. On the contrary, if the total damping is larger than the total
amplification, no light is expected to be backscattered from the plasma at
that wavelength. The gap effect can be explained by this competition, where
the beginning of the gap is determined by the balance between the SRS
amplification and the collisional damping, as we will show next for a simple,
idealized plasma profile.

###  II.1 Density profile

For simplicity, we consider a slab geometry and a density gradient along the
$z-$axis. The typical ion density, $n_{i}$ in laser-plasma experiments is
generated by isothermal expansion and, therefore, has an exponential
profile,Michel (2023) $n_{i}\sim e^{z/c_{s}t}$, where $c_{s}$ is the sound
speed and the density gradient is toward the positive $z$ direction. Since the
Raman instability occurs on timescales much shorter than the isothermal plasma
expansion, we define a snapshot of the profile

$n_{i}=n_{\text{min}}+\Delta n\;\frac{e^{\alpha z/l}-1}{e^{\alpha}-1}$ (3)

where $l$ is the plasma length, defined as the distance between the two chosen
densities, $n_{\text{min}}$ and $n_{\text{max}}$. The density difference is
$\Delta n=n_{\text{max}}-n_{\text{min}}$. Therefore, $l=\alpha c_{s}t$ for

$\alpha=\ln\left(\frac{n_{\text{max}}}{n_{\text{min}}}\right).$ (4)

It is noted that in order to get the full SRS spectrum, the value of
$n_{\text{max}}$ must be a little above $n_{\text{cr}}/4$, and the value of
$n_{\text{min}}$ should be small enough. Since the simulating of long plasmas
is expensive, $n_{\text{min}}$ in PIC simulations cannot be too small (see
Sec. III), but in the analytical solution below, we will take the limit
$n_{\text{min}}\rightarrow 0$. A realization of the density profile with the
parameters $n_{\text{min}}=0.01\,n_{\text{cr}}$,
$n_{\text{max}}=0.25\,n_{\text{cr}}$, and $l=1$ mm is illustrated in the inset
of Fig. 1b. Surprisingly, as shown below, although the total amplification
depends on $l$, the spectral width of the gap does not.

###  II.2 Growth rate

To evaluate the Raman amplification from plasma with a density profile (3), we
employ the famous Rosenbluth formula for inhomogeneous plasmas providing the
convective amplification factor exp$[\Gamma_{R}]$ for the backscattered light
after integration through a resonance region in an inhomogeneous
plasma,Rosenbluth (1972)

$\Gamma_{R}=\frac{\pi\,a_{0}^{2}\,k_{2}^{2}}{4k_{1}}\,L_{n}$ (5)

where the density profile length scale is

$L_{n}=n_{e}\left(\frac{dn_{e}}{dz}\right)^{-1}.$ (6)

For linear polarization, the dimensionless wave amplitude reads
$a_{0}=\sqrt{2e^{2}\lambda I/(\pi m_{e}c^{5})}\approx
0.02\,\lambda_{\mu}\sqrt{I_{15}}$, where $\lambda_{\mu}=\lambda_{0}/(1\mu m)$
and $I_{15}$ is the incident laser intensity in units of $10^{15}W/cm^{2}$.
Note that it is justified to use this approximation that depends only on the
local gradient because further away from the SRS reflection point, $z_{r}$,
where the resonance condition is exactly fulfilled, the detuning becomes
large. Therefore, the gain is determined in a small neighborhood of $z_{r}$.
For Raman backscattering, the wave number amplitude of the plasma wave $k_{2}$
is given by the resonance condition [Eq. (2)] where

$k_{0,1}=\frac{1}{c}\sqrt{\omega_{0,1}^{2}-\omega_{p}^{2}}$ (7)

are evaluated at the backscattering point, $z_{r}$, via the (spatially
dependent) plasma frequency,

$\omega_{p}=\sqrt{4\pi e^{2}n_{e}(z_{r})/m_{e}}.$ (8)

Similarly, the backscattered light frequency is determined at the reflecting
point, $z_{r}$, via the approximated Eq. (1),

$\omega_{1}=\omega_{0}-\omega_{p}(z_{r})$ (9)

while the incident laser frequency, $\omega_{0}$, does not depend on the
plasma parameters.

Figure 1: (a): SRS gain from the Rosenbluth formula in Eq. (5) (blue),
collisional damping loss from Eq. (10) (red), and the difference between the
two, $\Gamma_{\text{tot}}$, (green) that crosses zero (dashed black) at the
gap location. (b): The total gain as a function of the backscattered light
wavelength outside the plasma. The inset is the electron density profile of
Eq. (3).

###  II.3 Collisional absorption

The amount of absorption that light experiences on its way out of the plasma,
$\exp(-\Gamma_{d})$, can be estimated by integrating the local damping rate
over the distance between the reflecting point, $z_{r}$, and the plasma edge,
$z=0$, (i.e., where $n_{i}=n_{\text{min}}$)

$\Gamma_{d}=\int_{0}^{z_{r}}\kappa\,dz.$ (10)

The simplest model for the local damping rate of light in plasma is given
byKruer (2003)

$\kappa=\frac{\omega_{1}}{c}\Im[\epsilon]\approx\frac{\omega_{p}^{2}\nu_{ei}}{\omega_{1}^{2}v_{g}}.$
(11)

Here, $\Im[\epsilon]$ is the imaginary part of the electric permittivity,
$v_{g}=c^{2}k_{1}/\omega_{1}$ is the group velocity of the backscattered
light, and

$\nu_{ei}=\frac{Z\,\ln\Lambda}{3(2\pi)^{1.5}}\;\frac{\omega_{p}}{n_{e}\,\lambda_{D}^{3}}$
(12)

is the electron-ion collision rateKruer (2003) , which depends on the
temperature through the Debye length, $\lambda_{D}=\sqrt{k_{B}T/4\pi
e^{2}n_{e}}$. The Coulomb logarithm can be estimated byMichel (2023)

$\ln{\Lambda}=\ln{\sqrt{\frac{b_{\perp}^{2}+b_{max}^{2}}{b_{\perp}^{2}+b_{min}^{2}}}},$
(13)

where, $b_{\perp}=\frac{Z^{*}e^{2}}{4\pi\epsilon_{0}m_{e}v_{e}^{2}}$,
$b_{max}=\frac{v_{e}}{\omega}$, and $b_{min}=\frac{\hbar}{2m_{e}v_{e}}$. Here,
$v_{e}$ is the thermal velocity of the electrons (estimated in the low-field
limitMichel (2023)), $Z^{*}=\left<Z^{2}\right>/\left<Z\right>$ is the
effective ionization number, $e$ is the electron charge, and $m_{e}$ is the
electron mass. These quantities can be estimated from the diagnostics of a
given experiment. Therefore, if the experimental parameters are measured or
estimated with sufficient accuracy, one can predict the location of the
beginning of the gap in the SRS spectrum by equating

$\Gamma_{\rm R}=\Gamma_{d},$ (14)

and solving (numerically) for the $\omega_{1}$. Because the diagnostics are
placed outside the plasma, the location of the beginning of the Raman gap in
the wavelength $x-$axis is $\lambda_{1}^{\rm out}=2\pi c/\omega_{1}$. For the
example presented in Fig. 1, the gap location is found to be at $608$ nm,
which, in the left panel, it is associated with the crossing point of
$\Gamma_{\rm tot}=\Gamma_{\rm R}-\Gamma_{d}$ and the $x$-axis. It is also
indicated in the right panel of the figure by a vertical dashed line.

###  II.4 Single dimensionless parameter

It is notable that Eq. (14), which defines the location of the gap, can be
simplified by isolating all the physical parameters except the laser
frequency, $\omega_{0}$, into one side of the equation, leaving on the other
side only an integral expression that, for a given $\omega_{0}$, depends only
on the reflecting point $z_{r}$,

$\frac{I\left(k_{B}T\right)^{1.5}}{Z^{*}\ln\Lambda}=\frac{2k_{1}\,m_{e}^{2.5}\,c^{4}}{3\left(2\pi\right)^{1.5}k_{2}^{2}\,\lambda_{0}^{2}\,e^{2}}\int_{0}^{z_{r}}\frac{dz}{\omega_{1}^{2}v_{g}n_{e}}.$
(15)

It is noted that for a given $\omega_{0}$, the electron density at $z=z_{r}$
determines all other frequencies and wave numbers via the SRS resonance
conditions [Eqs. (1-2)] and Eqs. (7,9),

$\displaystyle\omega_{p}$ $\displaystyle=\omega_{0}\sqrt{n}$ (16)
$\displaystyle\omega_{1}$ $\displaystyle=\omega_{0}\left(1-\sqrt{n}\right)$
(17) $\displaystyle k_{0}$ $\displaystyle=\frac{\omega_{0}}{c}\sqrt{1-n}$ (18)
$\displaystyle k_{1}$
$\displaystyle=\frac{\omega_{0}}{c}\sqrt{\left(1-\sqrt{n}\right)^{2}-n}$ (19)
$\displaystyle k_{2}$
$\displaystyle=\frac{\omega_{0}}{c}\left(\sqrt{1-n}+\sqrt{\left(1-\sqrt{n}\right)^{2}-n}\right)$
(20)

where, $n=n_{e}(z_{r})/n_{\text{cr}}$ is the dimensionless electron density at
$z=z_{r}$. Therefore, by solving Eq. (15) for $z_{r}$, one can find the
spectral gap location, $\lambda_{\text{gap}}=2\pi c/\omega_{1}$, which is the
wavelength outside the plasma of the light that was backscattered from
$z=z_{r}$.

By employing the definitions of all the parameters in the right-hand-side
(RHS) of Eq. (15), the laser frequency can also be isolated such that the RHS
will depend only on $z_{r}$. At the same time, the left-hand-side (LHS)
includes an additional term of $\lambda_{0}^{3}$. Therefore, it is
constructive to define one dimensionless parameter that includes all of the
physical parameters in the problem,

$\xi=\frac{I_{15}T_{\rm eV}^{1.5}\lambda_{\mu}^{3}}{Z\ln\Lambda},$ (21)

where $I_{15}=I/10^{15}Wcm^{-2}$, $T_{\rm eV}=k_{B}T/eV$, and
$\lambda_{\mu}=\lambda_{0}/\mu m$. Now, Eq. (15) can be rewritten in a
dimensionless form,

$\xi=2996\frac{\left(1-\sqrt{n}\right)^{2}\sqrt{(1-\sqrt{n})^{2}-n}}{\left(\sqrt{1-n}+\sqrt{(1-\sqrt{n})^{2}-n}\right)^{2}}\;J,$
(22)

where,

$J=\int_{y_{0}}^{y_{1}}\frac{y}{\sqrt{1-y}}\,dy$ (23)

with $y=\frac{\omega_{0}^{2}}{\omega_{1}^{2}}n$. The integration limits are
$y_{0}=\frac{\omega_{0}^{2}}{4\omega_{1}^{2}}e^{-\alpha/2}$ and
$y_{1}=(1-\frac{\omega_{0}}{\omega_{1}})^{2}$. Note that $y_{0}\rightarrow 0$
when $n_{\text{min}}\rightarrow 0$ because then $\alpha\rightarrow\infty$.
Assuming this limit, the analytical solution of the dimensionless integral
reads

$J=\frac{1}{6}\cos(3\theta_{1})-\frac{3}{2}\cos(\theta_{1})+\frac{4}{3}$ (24)

where

$\theta_{1}=\sin^{-1}\left(1-\frac{1}{1-\sqrt{n}}\right)$ (25)

Notably, we obtained in Eq. (22) a relation between the density at the gap
location, $n=n_{\text{gap}}$, and the dimensionless parameter, $\xi$, which is
determined by the system’s parameters in Eq. (21),

$\xi=f(n_{\text{gap}}).$ (26)

This relation is depicted in a solid blue line in Fig. 5 (left $y-$axis). By
numerical inverting Eq. (26), one can find the theoretical gap location in
terms of the plasma density as a function of the systems’ dimensionless
parameter $n_{\text{gap}}=n_{\text{gap}}(\xi)$. However, this is a multi-
valued function, so we omit the higher branch (near the quarter critical
density, $n_{cr}/4$) because the group velocity of the backscattered light
becomes very small, and the SRS interaction is nearly absolute. Finally, for a
given laser frequency, $\omega_{0}$, the wavelength at the gap,
$\lambda_{\text{gap}}$ can also be calculated from Eq. (26) and Eq. (17). The
right $y-$axis of Fig. 5 is associated with the theoretical prediction for the
spectral gap location in terms of $\lambda_{\text{gap}}/\lambda_{0}$.

##  III PIC simulations

Figure 2: PIC simulation results for negligible collisions ($\ln\Lambda=1$).
Upper panel: A snapshot of the transverse electric field, $E_{y}$ (blue), and
the longitudinal electric field, $E_{x}$ (yellow), at a time when the whole
incident laser pulse has passed the plasma (denoted in dashed black lines)
from left to right. Lower panel: The spectrum of the backscattered light
between $-4.8$ mm and zero of the upper panel. No noteworthy gap is observed
in the Raman spectrum in this case. Figure 3: PIC simulation results for high
collisions ($\ln\Lambda=80$). Panels are the same as in Fig. 2. A gap in the
SRS spectrum is observed between about $640$ nm and the (thermally shifted)
absolute SRS at about $710$ nm. The theoretical location of the lower end of
the gap, $608$ nm, is denoted by a vertical dashed line.

To study the Raman gap effect, we have run 1D PIC simulations using the code
EPOCH Arber  _et al._ (2015) with $250$ cells per $\mu$m and $50$ particles
per cell. It is noted that in addition to the physical noise, PIC simulations
also exhibit numerical noise, which depends on the resolution. Therefore, the
total reflectivity is not well reproduced, but relative effects can be
investigated better within a fixed resolution. In the simulations, we consider
typical physical parameters of inertial confinement fusion (ICF) experiments
as follows: The laser wavelength and intensity were $\lambda_{0}=0.351\,\mu$m
and $I_{0}=2.5\times 10^{15}$ W/cm2; the pulse duration was $\tau=12$ ps; both
electron and ions ($Z=1$) density profiles ($n_{e}=n_{i}$) were exponential as
defined in Eq. (3) with $n_{\rm min}=0.05\,n_{\rm cr}$, $n_{\rm
max}=0.27\,n_{\rm cr}$, and $l=0.6$ mm; the electron temperature was
$T_{e}=1000$ eV while the ions were immobile.

In Fig. 2 (upper panel), we plot a snapshot of the electric field, $E_{y}$ of
the backscattered light (blue) and the longitudinal electric field, $E_{x}$,
of the EPW inside the plasma, for negligible collisions ($\ln\Lambda=1$).
Dashed lines depict the plasma boundaries. The incident laser pulse propagates
from left to right, and the reflected light propagates from right to left.
Thus, at the snapshot time, the left part of the figure represents the
reflected light while the small right part (right to the plasma) is the
transmitted pulse. The spectral analysis of the reflected part is shown in the
lower panel of Fig. 2. Notably, it does not exhibit the gap effect as the SRS
intensity around $600-700$ nm is similar to the intensity at smaller
wavelength, i.e., within the range of $50$ percent from the averaged intensity
at $500-600$ nm.

On the contrary, in Fig. 3, we present the results of an identical PIC
simulation but with $\ln\Lambda=80$, i.e., stronger collisions. First, we note
that the total SRS reflectivity in the case with collision, $4$ percent, is
significantly smaller than that without collisions, $35$ percent, and much
more similar to the experimental values of SRS reflectivity. Second and most
notably, the spectrum exhibits a prominent gap between $630$ nm and the
absolute SRS at $710$ nm (that agrees with the thermal shift), in which the
SRS intensity is about $95$ percent less than the average intensity at the
range of $510-590$ nm. Finally, the theoretical prediction for the gap
location for this simulation’s parameters is $608$ nm (denoted by a dashed
black line), which reasonably agrees with the gap location in the simulation.

The comparison between these two simulations suggests that collisional
absorption is a plausible explanation for the Raman gap effect. Moreover,
other explanations for the effect can be tested by their ability to explain
the results of the 1D PIC simulations presented here. We will remember this
argument when discussing the alternative explanations in Sec. V.

To further illustrate how the gap evolves when increasing the collisionality,
we repeat the same simulation with two intermediate values of $\ln\Lambda$ and
plot the spectra in Fig. 4. Each spectrum in the figure is normalized by its
associated total SRS reflectivity, as denoted in the legend. It can be seen
that as the collisionality (i.e., the value of $\ln\Lambda$) increases, the
ratio between the SRS intensity in the range of $[640-700]$ nm and the
intensity in the range of $[500-600]$ nm decreases.

We conclude that the introduction of collisions in the PIC simulations yields
the appearance of the Raman gap, in agreement with the linear analysis derived
in Sec. II. In the next section, we will test the theoretical prediction for
the location of the gap against a few past experiments.

Figure 4: Illustration of the evolution of the Raman gap with collisionality.
The SRS spectra of four PIC simulations with different values of
$\ln\Lambda,\\{1,10,40,80\\}$, are presented. For better visualization, each
spectrum is normalized by its total SRS reflectivity (see legend).

##  IV Comparison with past experiments

To validate our theory against past experiments, we have analyzed a few
published experimental Raman spectra. To this end, a reasonable estimation of
the physical parameters in each experiment is required. The relevant
parameters for the analysis are the electron temperature, the laser intensity,
and the degree of ionization. Also, we implicitly assumed an exponential
density profile. However, in cases where the density profile is different but
known, the theoretical gap location can be easily calculated by numerically
integrating Eq. (15) over the measured density profile.

Unfortunately, although the Raman gap effect was observed in many LPI
experiments, the diagnostic of the laser or the plasma parameters was poor in
many of them. Particularly in most of the relevant publications, the
temperature of the electron temperature was not measured or, at least, not
reported in the paper. Besides, even when measurements or estimations for
values of the physical parameters are given, the uncertainty in the
parameters’ values is usually large. These uncertainties must be reflected by
error bars in the graph. In the absence of reported estimation for the error
bars, we arbitrarily assumed an error of $10-20$ percent, where we note that
the uncertainty in the degree of ionization, $Z$, is relevant only in
experiments with a gold target.

The location of the gap was extracted from the figures in the papers by a
rough estimation of the location where the SRS spectrum decreased by about
$90$ percent from the averaged value at wavelengths below the gap. We employ
this definition because it is consistent with the definition of the
theoretical location of the gap, $\Gamma_{tot}=0$, meaning no SRS
amplification of the noise.

Because of the quality of the figures in the literature, the estimation was
done by eye and a ruler. For each spectrum, we associate some error estimated
from the spectral curve’s slope. The data extracted from the literature, the
error bar associated with the physical parameters, and our extracted gap
locations are summarized in Table 1.

The results are presented in Fig. 5. Each experiment is denoted by a different
color listed in the legend. The theoretical location of the gap [Eq. (26)] is
denoted by a solid blue line. The range of values of the dimensionless
parameter, $\xi$, where the theory predicts there should not be a gap, is
denoted by a solid horizontal red line associated with the quarter critical
density and a gap "location" of $2\lambda_{0}$. Interestingly, there is one
experiment ("Shiva-red-gold") with parameters in this regime, and indeed, only
a tiny gap was observed in the SRS spectrum in this experiment. A good
agreement between the experimental dots and the theoretical curve can be seen.
In addition, the PIC simulation result is denoted by a black triangle and
agrees well with the theory.

Figure 5: The theoretical gap location as a function of the dimensionless parameter, $\xi$ (solid blue line), in comparison with several past experiments (colored circles) and our PIC simulation example (black triangle). Also denoted the gap’s upper limit at twice the laser wavelength i.e., for quarter critical density (solid red line). The left $y-$scale is the gap density normalized by the critical density, while the right $y-$scale is the associated normalized wavelength of the backscattered light. Table 1:  List of the parameters in past experiments and PIC simulations.  Name | Source | Target |  $\lambda_{0}$ [nm] | $I_{15}$ | $Z^{*}$ | $T_{\rm KeV}$ |  $\lambda_{\text{gap}}$ [nm] |
---|---|---|---|---|---|---|---|---
Shiva-red | Ref. Simon  _et al._ , 1986 (Fig 6) | Gold111For golds target, we assumed partial ionization of $Z=50$ with $10\%$ error. | $1064$ | $2\pm 1$ | $50\pm 5$ | $5\pm 1$ | $2110\pm 20$ |
Argus-green | Ref. Simon  _et al._ , 1986 (Fig 8) | Gold111For golds target, we assumed partial ionization of $Z=50$ with $10\%$ error. | $532$ | $1\pm 0.5$ | $50\pm 5$ | $2\pm 0.5$ | $920\pm 30$ |
UCLA-GDL | Ref. Simon  _et al._ , 1986 (Fig 14) | CH222For CH targets, we assumed fully ionized plasma without uncertainty. | $351$ | $1\pm 0.5$ | $4.75\pm 0$ | $0.7\pm 0.3$ | $580\pm 30$ |
Shepard | Ref. Shepard  _et al._ ,1986 | Gold111For golds target, we assumed partial ionization of $Z=50$ with $10\%$ error. | $526$ | $0.7\pm 0.3$ | $50\pm 5$ | $2\pm 1$ | $850\pm 50$ |
Drake | Ref. Drake and Batha,1991 | Gold111For golds target, we assumed partial ionization of $Z=50$ with $10\%$ error. | $530$ | $0.9\pm 0.3$ | $50\pm 5$ | $3\pm 1$ | $950\pm 20$ |
PIC simulation | This work (Fig. 3) | Immobile ions333Numerical collisions strength was set to $\ln{\Lambda}=80$. The numerical error is estimated as $20\%$. Other numerical parameters can be found in Sec. III. | $351$ | $2.5$ | $1$ | $1$ | $635\pm 20$ |

##  V discussion

The Raman gap that was observed in many experiments poses a long-standing
puzzle. In the literature, we found seven different theories to explain the
effect. While we cannot discern the level of verisimilitude claimed in each
explanation, we point out that it is reasonable to check if they can explain
the gap effect observed in our PIC simulation results. First, as mentioned in
Sec. III, the simulations were one-dimensional and considered immobile ions
but exhibited the Raman gap effect. Therefore, theories that are based on the
dynamics in more than one dimensionRose and Mounaix (2011); Rosenberg  _et
al._ (2020) or involve interaction with ion-acoustic wavesBaldis  _et al._
(1989); Drake and Batha (1991) (i.e., ion motion) can not explain the gap
effect in the simulations. Second, PIC simulations are costly, and thus, the
laser pulse duration in the simulation was only $12$ ps. Although this
timescale is enough for studying the Raman backscattering, it is not
sufficient for developing a population of fast electrons (especially without
the TPD process, which is a 2D effect). Therefore, the simulations do not
support the theory based on enhanced EPW seed driven by fast (non-Maxwellian)
electron distributionSimon  _et al._ (1986). Third, the theory based on high
sensitivity to detuning near the quarter critical densityBarr, Boyd, and
Mackwood (1993) considered extreme density gradients, i.e., very short plasmas
(about $100$ wavelength only). The plasma length in the PIC simulations was
too long ($600\mu$m) to be adequately comprehended by this approach. Finally,
the idea of electron density profiles that steepen near the quarter critical
density such that Raman backscattering near $2\lambda_{0}$ is substantially
reducedTanaka  _et al._ (1982); Shepard  _et al._ (1986) may require a
tailored density profile to explain the observed Raman gap effect. In
contrast, for typical density profiles in our PIC simulations, the gap effect
was not observed in the absence of collisions but was observed when collisions
were introduced.

It is important to emphasize that we do not claim that the aforementioned
theories do not contribute to forming the gap in the SRS spectrum. We argue
that, on the one hand, the effect in both experiments and PIC simulations can
be understood as a result of the competition between SRS amplification and
collisional absorption. On the other hand, other theories, most of which are
much more complicated, might explain the experiments but not the PIC
simulations. Additionally, our theoretical prediction for the gap width is in
reasonable agreement with past experiments without the need for other
supportive effects.

##  VI conclusions

In conclusion, we developed a theory for the Raman gap effect based on the
competition between the SRS growth rate in inhomogeneous plasmas and the
collisional absorption of the backscattered light on its way out of the
plasma. The gap formation was observed in a series of PIC simulations where
only the collisionality changes. The location of the gap is calculated via
linear analysis and exhibits a reasonable agreement with several past
experiments and our PIC simulations. Previous theoretical explanations
suggested in the literature for the Raman gap can not explain the PIC
simulation results. Therefore, we conclude that the Raman gap can often result
from the collisional absorption of the backscattered light without the support
of other explanations.

## Data Availability Statement

The data that support the findings of this study are available from the
corresponding author upon reasonable request.

## Acknowledgments

This work was performed under the auspices of the U.S. Department of Energy by
Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344, and
was supported by the PAZI Foundation, Grant No. 2020-191. The computations in
this paper were run on the ICPL cluster at the Hebrew University of Jerusalem.
One of the authors (IB) would like to thank Jonathan Wurtele for his
hospitality at UC Berkeley, where this work was mainly conducted.

## References

  * Simon  _et al._ (1986) A. Simon, W. Seka, L. M. Goldman, and R. W. Short, Phys. Fluids 29, 1704 (1986).
  * Tanaka  _et al._ (1982) K. Tanaka, L. M. Goldman, W. Seka, M. C. Richardson, J. M. Soures, and E. A. Williams, Phys. Rev. Lett. 48, 1179 (1982).
  * Shepard  _et al._ (1986) C. L. Shepard, J. A. Tarvin, R. L. Berger, G. E. Busch, R. R. Johnson, and R. J. Schroeder, Phys. Fluids 29, 583 (1986).
  * Baldis  _et al._ (1989) H. A. Baldis, P. E. Young, R. P. Drake, W. L. Kruer, K. Estabrook, E. A. Williams, and T. W. Johnston, Phys. Rev. Lett. 62, 2829 (1989).
  * Barr, Boyd, and Mackwood (1993) H. Barr, T. Boyd, and A. Mackwood, Phys. Lett. A 180, 435 (1993).
  * Drake and Batha (1991) R. P. Drake and S. H. Batha, Phys. Fluids B 3, 2936 (1991).
  * Rose and Mounaix (2011) H. A. Rose and P. Mounaix, Phys. Plasmas 18, 042109 (2011).
  * Rosenberg  _et al._ (2020) M. J. Rosenberg, A. A. Solodov, W. Seka, R. K. Follett, J. F. Myatt, A. V. Maximov, C. Ren, S. Cao, P. Michel, M. Hohenberger, J. P. Palastro, C. Goyon, T. Chapman, J. E. Ralph, J. D. Moody, R. H. H. Scott, K. Glize, and S. P. Regan, Physics of Plasmas 27, 042705 (2020).
  * Michel  _et al._ (2019) P. Michel, M. J. Rosenberg, W. Seka, A. A. Solodov, R. W. Short, T. Chapman, C. Goyon, N. Lemos, M. Hohenberger, J. D. Moody, S. P. Regan, and J. F. Myatt, Phys. Rev. E 99, 033203 (2019).
  * Michel (2023) P. Michel,  _Introduction to Laser-Plasma Interactions_, Graduate Texts in Physics (Springer International Publishing, 2023).
  * Rosenbluth (1972) M. N. Rosenbluth, Phys. Rev. Lett. 29, 565 (1972).
  * Kruer (2003) W. Kruer,  _The Physics Of Laser Plasma Interactions_, Frontiers in Physics (CRC Press, 2003).
  * Arber  _et al._ (2015) T. D. Arber, K. Bennett, C. S. Brady, A. Lawrence-Douglas, M. G. Ramsay, N. J. Sircombe, P. Gillies, R. G. Evans, H. Schmitz, A. R. Bell, and C. P. Ridgers, Plasma Physics and Controlled Fusion 57, 113001 (2015).