11institutetext:  Department of Computer Science and Engineering, University
at Buffalo,
The State University of New York, Buffalo, NY, USA
11email:
<EMAIL_ADDRESS>Population Health Sciences, Weill Cornell Medicine, New York, NY, USA
22email<EMAIL_ADDRESS>
# Improving Joint Learning of Chest X-Ray and Radiology Report by Word Region
Alignment

Zhanghexuan Ji Z. Ji and M.A. Shaikh—Equal contributions.11 Mohammad Abuzar
Shaikh††footnotemark:  11 Dana Moukheiber 11
Sargur Srihari 11 Yifan Peng 22 Mingchen Gao 11

###### Abstract

Self-supervised learning provides an opportunity to explore unlabeled chest
X-rays and their associated free-text reports accumulated in clinical routine
without manual supervision. This paper proposes a Joint Image Text
Representation Learning Network (JoImTeRNet) for pre-training on chest X-ray
images and their radiology reports. The model was pre-trained on both the
global image-sentence level and the local image region-word level for visual-
textual matching. Both are bidirectionally constrained on Cross-Entropy based
and ranking-based Triplet Matching Losses. The region-word matching is
calculated using the attention mechanism without direct supervision about
their mapping. The pre-trained multi-modal representation learning paves the
way for downstream tasks concerning image and/or text encoding. We demonstrate
the representation learning quality by cross-modality retrievals and multi-
label classifications on two datasets: OpenI-IU and MIMIC-CXR. Our code is
available at https://github.com/mshaikh2/JoImTeR˙MLMI˙2021.

###### Keywords:

Self-supervised Learning Multi-modality Attention

##  1 Introduction

Chest X-ray is the most common medical imaging study globally for conducting
clinical routines to assess chest regions. Because of its popularity, large,
labeled datasets such as ChestX-ray14 dataset [24], CheXpert [10], OpenI-IU
[5], and MIMIC-CXR [11, 12], were collected as benchmarks for data-driven deep
learning models to archive expert-level performance in analyzing chest
regions. Among these biomedical datasets, OpenI-IU and MIMIC-CXR contain
radiology reports along with corresponding radiographs. Given the large size
of collected images and manual labeling being impractical, the disease labels
are usually derived using natural language processing tools applied to the
corresponding radiology reports.

Recently, self-supervised representation learning has been explored to extract
underlying information from the data by performing proxy tasks that explore
the organization of the data itself. This is a promising direction for
learning from a large amount of unlabeled biomedical data, where manual
labeling is tedious, time-consuming, subjective, and requires domain
knowledge. Self-supervised learning provides a great potential to investigate
the biomedical data, including both medical images and their associated
reports, accumulated during clinical routines. Ideally, both the modalities of
the data encode the same medical condition and should be cross-referable.

Self-Attention mechanism was introduced to find the cross references within
the same data modality [23]. This concept has contributed tremendously to the
recent success of natural language processing models, such as BERT [6]. These
models are pre-trained by predicting masked tokens to learn the underlying
semantic representations from unlabeled textual data. Once the representation
learning models are pre-trained, they can be fine-tuned and used as a backbone
for a wide range of downstream natural language processing tasks.

Motivated by the above discussion, we propose to establish the cross
references of the chest radiology images and reports to jointly learn the
image-text representations. Learning cross-modal visual and textual
representation is an essential task that can combine the semantic information
contained within images and their descriptive reports [17, 16]. These
approaches have also been explored in biomedical image analysis [18]. The
proposed representation learning mechanism will provide the foundation for a
wide range of biomedical vision-and-language tasks, such as clinical inter-
modal and intra-modal image-text retrieval, medical visual question answering
[1], and automatic clinical report generation [19].

Contributions: We propose JoImTeRNet - a self-supervised pre-training network
trained on multimodal inputs. Our network extracts and fuses the
representations of the visual and textual modalities using both global image-
sentence matching and local attention-based region-phrase matching. Phrases
vary from length of one to three words. The proposed local region-phrase
alignment enhances the joint representation learning by performing automatic
fine-grained matching between image region-of-interests with phrases in
reports. The local region-phrase matching is further enhanced using a soft-
attention mechanism in the image encoder, without the need for explicit manual
bounding box annotation or object detection on images. The quality of the
learned representation is tested on the downstream classification and
retrieval tasks.

##  2 Joint Image Text Representation Learning Network

Figure 1: The architecture of the proposed JoImTeRNet.

We propose a Joint Image Text Representation Learning Network (JoImTeRNet)
shown in Fig. 1. The JoImTeRNet architecture consists of an image and a text
encoder. The representations are matched through a list of matching tasks,
Text to Image Matching (TIM), Masked Language Modeling (MLM), Phrase to Region
Alignment (PRA), and Word to Region Alignment (WRA). The learned image and
text representation are mapped to a shared feature space, given the hypothesis
that the radiographs and their corresponding report contain consistent
semantic meaning.

Given an X-ray image I and its corresponding radiology report T, we first
encode them with an image encoder $F_{I}$ and a text encoder $F_{T}$. The
image encoder contains one input convolution layer, 6 residual blocks [8] and
a global average pooling (GAP) layer. In the meantime, we also get the output
from the Soft-Attention (SA) [22] block placed after the ResBlock5 to extract
the region features $r\in\mathbb{R}^{D\times M}$, such that
$v,r=F_{I}(\text{I})$ where $v\in\mathbb{R}^{D}$ is the global image features
from GAP. Sentence and word level features $s$ and $w$ are extracted using a
Transformer [23] based text encoder $F_{T}$, such that $w,s=F_{T}(\text{T})$,
where $w\in\mathbb{R}^{D\times N}$ and $s\in\mathbb{R}^{D}$. Three transformer
layers are deployed in $F_{T}$ to encode the text report with the self-
attention mechanism.

###  2.1 Matching Images and Sentences

To learn the joint representation of image and text pairs, we use the cross-
entropy based matching (CEM) loss [26] and the ranking-based triplet matching
(TM) loss [3]. Given a batch of image-text pairs
${(\text{I}_{i},\text{T}_{i})}_{i=1}^{B}$ (B is the batch size) and their
corresponding visual features $v$ and sentence features $s$ from $F_{I}$ and
$F_{T}$, the image-to-text CEM loss $L_{CEM}^{\text{IT}}$ is defined as the
negative log posterior probability of the images being matched with their
corresponding texts, i.e.,

$L_{CEM}^{\text{IT}}=-\sum_{i=1}^{B}log(P(\text{T}_{i}|\text{I}_{i}))=-\sum_{i=1}^{B}log(\frac{e^{\gamma
S(\text{I}_{i},\text{T}_{i})}}{\sum_{j=1}^{B}e^{\gamma
S(\text{I}_{i},\text{T}_{j})}})$ (1)

where $\gamma$ is the smoothing factor. $P(\text{T}_{i}|\text{I}_{i})$ is the
posterior probability of $\text{T}_{i}$ matching with $\text{I}_{i}$ using
softmax. Cosine similarity
$S(\text{I}_{i},\text{T}_{i})=(v^{T}s)/(\|v\|\|s\|)$ is used as the similarity
score between image-text pairs. During the training, $\text{T}_{i}$ is the
correct match to $\text{I}_{i}$ in the batch and all the other
$\text{T}_{j}(j\neq i)$ are mismatching texts. Considering that image-text
joint representation mapping should be bidirectional, we reverse I and T in
Eq.(1) and get the symmetric text-to-image CEM loss as $L_{CEM}^{\text{TI}}$.
Thus, the bidirectional CEM loss for globally matching image and text is
defined as $L_{CEM}^{s}=L_{CEM}^{\text{IT}}+L_{CEM}^{\text{TI}}$.

Although CEM loss is designed to make the similarity between correct image-
text pairs relatively higher than other mismatched pairs, it is difficult to
set a hard margin between mismatched features. To solve this problem, TM loss
[3], a ranking-based criterion, is added to increase the distance of
mismatched pairs in the joint embedding space. Given an image $\text{I}_{i}$
as the anchor, $\text{T}_{i}$ is used as the positive paired sample. We then
randomly select a mismatching text $\text{T}_{j}(j\neq i)$ within the batch as
the negative paired sample. Symmetrically, if $\text{T}_{i}$ is used as the
anchor, then $\text{I}_{i}$ and $\text{I}_{j}$ would be positive/negative
samples. The bidirectional TM loss for global image-text matching is formed
as:

$\displaystyle L_{TM}^{s}=L_{TM}^{\text{IT}}+L_{TM}^{\text{TI}}$
$\displaystyle=\sum_{i=1}^{B}\Big{[}\text{max}(0,S(\text{I}_{i},\text{T}_{j})-S(\text{I}_{i},\text{T}_{i})+\eta_{s})\Big{.}$
(2)
$\displaystyle\Big{.}\quad\quad\quad+\text{max}(0,S(\text{I}_{j},\text{T}_{i})-S(\text{I}_{i},\text{T}_{i})+\eta_{s})\Big{]}$

where $\eta_{s}$ is the hard margin and $S$ is the cosine similarity the same
as in Eq.(1).

###  2.2 Aligning Image Regions and Report Phrases

Both chest X-rays and their corresponding reports contain lots of fine-grained
semantic information. We introduce a region-phrase level matching to align
different concepts in the text reports with the regions of the images to
further improve the joint representation. We apply region-phrase alignment
with both CEM loss and TM loss. The length of a phrase is in the range of 1 to
3 words. Features of words, bigram and trigram phrases are denoted as
$w,p_{2},p_{3}$ respectively.

The cosine similarity between regions and words/phrases is not feasible to
calculate directly due to the lack of explicit mapping between them. Instead,
an attention-based matching score is deployed to overcome this challenge [7,
9, 26]. For region-word-level matching, given $(\text{I}_{i},\text{T}_{i})$
and their region-word features $(r,w)$, we first calculate the similarity
matrix between all possible pairs of region features and word features using
dot-product, i.e., $m=w^{T}r$, where $m\in\mathbb{R}^{N\times M}$, which is
further normalized along $N$ words as $\bar{m}=\text{Softmax}_{N}(m)$. Next, a
context feature $c$ is computed as the weighted sum over region features $r$,
weighted by the region-word attention score $\alpha$ as follows:

$\displaystyle c=\alpha r^{T}\text{, where
}\alpha_{i,j}=\frac{e^{\gamma_{1}\bar{m}_{i,j}}}{\sum_{k=0}^{M-1}e^{\gamma_{1}\bar{m}_{i,k}}}$
(3)

where $c\in\mathbb{R}^{N\times D}$ and $\alpha\in\mathbb{R}^{N\times M}$;
$\gamma_{1}$ is a hyper-parameter to tune the required amount of visual
attention for a word. Here, the $i^{th}$ vector of $c$ is the attention-
weighted representation of all the sub-regions related to the $i^{th}$ word.

The attention-based region-word-level matching score is computed as:

$S_{a}(\text{I},\text{T})=\log(\sum_{i=1}^{N-1}e^{(\gamma_{2}S(c_{i},w_{i}))})^{\frac{1}{\gamma_{2}}}$
(4)

where $S(c_{i},w_{i})=(c_{i}^{T}w_{i})/(\|c_{i}\|\|w_{i}\|)$, is the element-
wise cosine similarity score between $c_{i}$ and $w_{i}$, $\gamma_{2}$ is the
importance magnification hyper-parameter for the most relevant word and
context vector pair.

By replacing the cosine similarity score $S(\cdot,\cdot)$ with the region-word
matching score $S_{a}(\cdot,\cdot)$ in Eq.(1)(2), we obtain the bidirectional
CEM loss and TM loss for region-word alignment as
$L_{CEM}^{p_{1}}=L_{CEM}^{rw}+L_{CEM}^{wr}$ and
$L_{TM}^{p_{1}}=L_{TM}^{rw}+L_{TM}^{wr}$.

Furthermore, we obtain the phrase features by applying a 1D convolutional
layer with kernel size 2 and 3 over $w$ to get bigram
$p_{2}=\theta_{p_{2}}^{T}w$ and trigram $p_{3}=\theta_{p_{3}}^{T}w$ phrase
features respectively [14, 27]. Here, $\theta_{p_{2}},\theta_{p_{3}}$ are the
convolution kernels of size 2 and 3. Our final cross-entropy with triplet
matching (CETM) loss for our image-text joint representation learning is
designed as:

$L_{CETM}=\lambda_{CEM}(L_{CEM}^{s}+\sum_{i=1}^{3}L_{CEM}^{p_{i}})+\lambda_{TM}(L_{TM}^{s}+\sum_{i=1}^{3}L_{TM}^{p_{i}})$
(5)

where $\lambda_{CEM}$ and $\lambda_{TM}$ are the loss weight hyper-parameters.

###  2.3 Downstream Task

In order to demonstrate the performance of joint representation learning, we
use the pre-trained image and text encoders as the backbone and test the
learned features on multi-label classification. We add projection layers
followed by two fully connected layers for multi-label classification. Cross-
entropy loss balanced with positive/negative ratio and class-wise weights [25]
are used for training.

##  3 Experiments

###  3.1 Datasets

MIMIC-CXR v2.0 [11], is a large public dataset consisting of 377,110 chest
X-rays associated with 227,835 radiology reports. We limit our study to the
frontal-view images and only keep one frontal view image for each report.
Following the pre-processing scheme in [3], we extract the impressions,
findings, conclusion and recommendation sections from the raw report,
normalized by SciSpaCy [20], and concatenate them. If none of these sections
are present, we use the final report section. 14 CheXpert labels provided in
MIMIC-CXR are used for classification task, where label 1 is considered as
positive and all the other labels (-1, 0) and missing labels are merged as
negative. This results in 222,252 image-report pairs with 14 binary labels. We
split the dataset into 217,252, 2,000 and 3,000 samples for training,
validation and testing respectively.

OpenI-IU [5] is a public dataset with 3,996 radiology reports and 8,121
associated chest X-ray images, which are manually annotated by human experts
using MeSH words. Similar to TieNet [25], only unique frontal images and their
corresponding reports which contain either findings and/or impressions are
selected. This yields 3,643 image-report pairs, which are only used as
external evaluation sets. For comparison and evaluation purposes, we select
the 7 common labels in both OpenI and MIMIC-CXR (Table 3) from the MeSH
domain.

Table 1: Ablation for selecting the best loss setting. The matching score for
OpenI-IU and MIMIC-CXR is computed on 1000 and 1000/3000 test samples
respectively. Subscript $s,w,p$ stand for image-text, region-word and region-
phrase level matching.

Model setting | MIMIC-CXR | OPENI-IU
---|---|---
I2T (1K) | T2I (1K) | I2T (3K) | T2I (3K) | I2T (1K) | T2I (1K)
R@1 | R@5 | R@10 | R@1 | R@5 | R@10 | R@1 | R@5 | R@10 | R@1 | R@5 | R@10 | R@1 | R@5 | R@10 | R@1 | R@5 | R@10
TMs[3] | 5.37 | 19.43 | 30.73 | 5.40 | 20.23 | 30.23 | 2.37 | 9.70 | 15.63 | 2.23 | 10.20 | 16.37 | 1.83 | 5.70 | 9.13 | 1.50 | 5.67 | 9.30
TMws | 6.30 | 21.73 | 32.23 | 6.00 | 20.97 | 30.90 | 2.77 | 10.67 | 18.07 | 2.83 | 10.97 | 17.83 | 1.93 | 6.37 | 10.17 | 1.97 | 6.53 | 10.30
CEMs[7] | 18.60 | 43.10 | 56.10 | 18.13 | 43.20 | 55.97 | 12.20 | 31.27 | 41.80 | 11.80 | 30.87 | 41.10 | 4.70 | 12.83 | 17.73 | 4.83 | 12.30 | 17.87
CEMws[26] | 18.60 | 44.20 | 56.27 | 18.87 | 43.40 | 55.67 | 12.60 | 31.67 | 41.80 | 12.83 | 31.57 | 41.27 | 4.87 | 13.00 | 18.00 | 5.37 | 13.40 | 18.33
CETMws | 19.07 | 45.33 | 57.20 | 19.07 | 44.70 | 56.73 | 12.77 | 31.90 | 43.00 | 12.97 | 31.97 | 42.03 | 5.13 | 13.07 | 18.73 | 5.50 | 13.73 | 19.20
CETMwps | 18.93 | 46.20 | 58.67 | 19.07 | 45.27 | 58.50 | 12.67 | 33.20 | 44.07 | 12.83 | 32.43 | 43.40 | 5.07 | 13.07 | 18.83 | 5.67 | 13.83 | 19.20

###  3.2 Implementation Details

JoImTeRNet is implemented in Pytorch [21] and all the experiments are carried
out on NVIDIA GTX 1080 Ti GPUs. For $F_{I}$, we use the basic residual blocks
proposed in [8]. We employ 3 layers of Transformer blocks with 8 heads in
$F_{T}$. The input image is encoded into 256 regions ($r$) flattened from
$16\times 16$ feature map output from Res Block 5 as shown in Fig. 1. The
input image is cropped or padded to $2048\times 2048$ then normalized to [-1,
1]. Random crop, rotation and color jitter are used for data augmentation.
Report input is tokenized by a word-level tokenization scheme, where we
collect all the words that appear more than twice in MIMIC-CXR dataset, which
results in vocabulary size of $8,410$. The input reports are truncated or
padded to the max length of $N=160$.

Parameter Settings We pretrain $F_{I}$ and $F_{T}$ on MIMIC-CXR training set
using the image-text matching task explained in Section 2.1 and 2.2 to
generate the joint image and text representations. The maximum epoch is set as
$30$. We employ AdamW [15] optimizer with an initial learning rate of
$10^{-4}$, which is dropped by $10$ times after $20$ epochs. L2 weight decay
is set as $10^{-4}$. For the downstream classification task in Section 2.3, we
set up two different settings for comparison: randomly initializing the
backbone and fine-tuning the pre-trained backbone. The learning rate for the
classification head in both settings is set to $10^{-4}$. For the randomly
initialized setting, we train the backbone using the same learning rate as the
classification block for 20 epochs, whereas the pre-trained backbone is fine-
tuned with a smaller learning rate of $10^{-5}$ for only 10 epochs. Our model
pre-trained with the full loss setting CETMwps is used as the backbone for
fine-tuning. The batch size is set to 32 for all the experiments. We select
the loss hyper-parameters as $\gamma,\gamma_{1},\gamma_{2}=2,1,1$,
$\eta_{s},\eta_{w}=0.5,0.5$ and $\lambda_{CEM},\lambda_{TM}=2.0,1.0$.

Table 2: Classification AUCs on MIMIC-CXR [11] dataset. “FS” stands for
training from scratch (FS). “FT” stands for fine-tuned model. Other comparison
experiments are Visualbert [17], Uniter [4], and ClinicalBert [2].

Findings | Image | Image+Report | Report
---|---|---|---
FS | FT | [17] | [4] | FS | FT | [2] | FS | FT
EC | 0.738 | 0.763 | 0.981 | 0.979 | 0.989 | 0.996 | 0.966 | 0.986 | 0.980
Cardiomegaly | 0.794 | 0.820 | 0.991 | 0.989 | 0.989 | 0.993 | 0.979 | 0.988 | 0.989
Airspace Opacity | 0.749 | 0.759 | 0.991 | 0.989 | 0.988 | 0.992 | 0.978 | 0.985 | 0.986
Lung Lesion | 0.696 | 0.756 | 0.985 | 0.981 | 0.996 | 0.998 | 0.972 | 0.989 | 0.987
Edema | 0.883 | 0.895 | 0.991 | 0.990 | 0.995 | 0.996 | 0.979 | 0.993 | 0.994
Consolidation | 0.794 | 0.810 | 0.989 | 0.988 | 0.994 | 0.998 | 0.979 | 0.996 | 0.995
Pneumonia | 0.733 | 0.738 | 0.977 | 0.974 | 0.981 | 0.985 | 0.962 | 0.980 | 0.984
Atelectasis | 0.808 | 0.824 | 0.988 | 0.987 | 0.988 | 0.995 | 0.976 | 0.991 | 0.992
Pneumothorax | 0.845 | 0.855 | 0.992 | 0.991 | 0.989 | 0.993 | 0.979 | 0.993 | 0.994
Pleural Effusion | 0.898 | 0.904 | 0.993 | 0.992 | 0.986 | 0.997 | 0.981 | 0.989 | 0.990
Pleural Others | 0.812 | 0.839 | 0.981 | 0.973 | 0.996 | 0.999 | 0.964 | 0.998 | 0.993
Fracture | 0.641 | 0.714 | 0.976 | 0.977 | 0.997 | 0.990 | 0.958 | 0.997 | 0.997
Support Devices | 0.901 | 0.913 | 0.995 | 0.994 | 0.992 | 0.995 | 0.983 | 0.988 | 0.993
No Findings | 0.865 | 0.874 | - | - | 0.985 | 0.989 | - | 0.980 | 0.980
Avg | 0.792 | 0.815 | 0.987 | 0.985 | 0.991 | 0.994 | 0.974 | 0.990 | 0.990

###  3.3 Performances

Evaluation Metric We evaluate the performance of JoImTeRNet by cross-modality
retrieval task: given one image (text) as a query, we rank a subset of text
(image), including the paired one, based on cosine similarity between the
image and text features from JoImTeRNet. Recall@K (R@K) [13] is reported,
where $K\in\\{1,5,10\\}$, which measures the fraction of times the correct
matching is retrieved among the top $K$ results in the test set. We compute
R@K on a subset of 1000 image-text pairs and on the full 3000 samples in our
MIMIC-CXR test set. We also report R@K on a subset of 1000 samples in OpenI-IU
in order to evaluate JoImTeRNet on the external dataset (Table 1).

Ablation Study for Loss Settings Ablation studies for different combinations
of our losses are listed in Table 1. As we can see, full loss setting CETMwps
achieves the highest R@5 and R@10 scores on all the test set, which shows the
effectiveness of our multilevel phrase matching loss. In addition, the
matching performance degrades when the model is trained on global matching
loss only without the region-phrase(word)-level matching, i.e. CEMs performs
worse than CEMws. Similar results are found when comparing TMs with TMws. This
result shows that our proposed method for assisting joint representation
learning using region-word matching is able to improve the representation
ability of the image-text encoder. Moreover, the CETM combination consistently
gains performance compared with only CEM loss or TM loss settings, which is
just as we expected in Sections 2.1 and 2.2. Notice that the matching scores
are much lower on OpenI, since OpenI contains a large amount of similar
reports, e.g. ‘No acute disease.’, which can have very similar feature
representation from our model and thus largely degrade the matching score.

Table 3:  Classification AUCs on OpenI-IU [5] dataset. “FS” stands for
training From Scratch (FS). “FT” stands for Fine-tuned model. Other comparison
experiments are ChestX-ray14 [24], TieNet [25], Visualbert [17], Uniter [4],
and ClinicalBert [2].

Findings | Image | Image+Report | Report |  No. of Samples
---|---|---|---|---
[24] | FS | FT | [25] | [17] | [4] | FS | FT | [25] | [2] | FS | FT
Cardiomegaly | 0.803 | 0.924 | 0.937 | 0.962 | 0.977 | 0.978 | 0.956 | 0.985 | 0.944 | 0.969 | 0.966 | 0.987 | 315
Edema | 0.799 | 0.937 | 0.953 | 0.995 | 0.982 | 0.989 | 0.922 | 0.962 | 0.984 | 0.976 | 0.947 | 0.964 | 40
Consolidation | 0.790 | 0.951 | 0.951 | 0.989 | 0.996 | 0.998 | 0.954 | 0.975 | 0.969 | 0.982 | 0.938 | 0.975 | 28
Pneumonia | 0.642 | 0.863 | 0.934 | 0.994 | 0.990 | 0.988 | 0.877 | 0.949 | 0.983 | 0.982 | 0.880 | 0.943 | 36
Atelectasis | 0.702 | 0.829 | 0.858 | 0.972 | 0.992 | 0.982 | 0.947 | 0.978 | 0.981 | 0.947 | 0.952 | 0.971 | 293
Pneumothorax | 0.631 | 0.926 | 0.936 | 0.960 | 0.988 | 0.983 | 0.962 | 0.989 | 0.960 | 0.973 | 0.951 | 0.989 | 22
Pleural Effusion | 0.890 | 0.938 | 0.957 | 0.977 | 0.985 | 0.983 | 0.922 | 0.971 | 0.968 | 0.976 | 0.926 | 0.968 | 140
No Finding | - | 0.844 | 0.851 | - | - | - | 0.883 | 0.961 | - | - | 0.898 | 0.930 | 2789
Avg | 0.751 | 0.910 | 0.932 | 0.978 | 0.987 | 0.986 | 0.934 | 0.973 | 0.970 | 0.972 | 0.932 | 0.971 |
W. Avg | 0.771 | 0.893 | 0.915 | 0.971 | 0.985 | 0.982 | 0.943 | 0.978 | 0.965 | 0.964 | 0.949 | 0.975 |

Downstream Image Classification Results The AUCs from our two settings on both
datasets along with other SOTA performances are shown in Table 2 and 3. We can
see that the classifier performance finetuned on JoimTerNet backbone (FT) is
always higher than training from scratch (FS), which shows the advance of our
pre-training method. As shown in Table 2, FT achieves the highest AUCs on most
tasks and labels on MIMIC-CXR test set (internal evaluation), even better than
some SOTA models [4, 17, 2] on image-text and text classification. For the
external evaluation on OpenI in Table 3, our FT setting extremely improves
average AUC on image classification by 18% compared with TieNet [25], and also
gains 1% on wAvg AUC than ClinicalBERT [2] on report classification. For the
image-text classification, our model is still comparable with other SOTA
models, even though our text encoder only contains 3 transformer layers
compared with [4, 17] which has a 12 layer BERT encoder as the backbone.

##  4 Conclusion

We propose a joint image-text representation learning network and show its
performance on cross-modality retrieval and multi-label classification. We
demonstrate the potential of self-supervised learning when it meets the
continuously generated biomedical images and reports. We also leverage and
show the importance of information contained within the relationship of words,
phrases and image regions. Future work includes more complicated downstream
tasks regarding both images and text.

Acknowledgments This research was supported in part by NSF through grant
IIS-1910492. It also was supported by the National Library of Medicine under
Award No. 4R00LM013001.

## References

  * [1] Abacha, A.B., Hasan, S.A., Datla, V.V., Liu, J., Demner-Fushman, D., Müller, H.: Vqa-med: Overview of the medical visual question answering task at imageclef 2019. In: CLEF (Working Notes) (2019)
  * [2] Alsentzer, E., Murphy, J., Boag, W., Weng, W.H., Jindi, D., Naumann, T., McDermott, M.: Publicly available clinical BERT embeddings. In: Proceedings of the 2nd Clinical Natural Language Processing Workshop. pp. 72–78. Association for Computational Linguistics (2019)
  * [3] Chauhan, G., Liao, R., Wells, W., Andreas, J., Wang, X., Berkowitz, S., Horng, S., Szolovits, P., Golland, P.: Joint modeling of chest radiographs and radiology reports for pulmonary edema assessment. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 529–539. Springer (2020)
  * [4] Chen, Y.C., Li, L., Yu, L., El Kholy, A., Ahmed, F., Gan, Z., Cheng, Y., Liu, J.: Uniter: Universal image-text representation learning. In: European conference on computer vision. pp. 104–120. Springer (2020)
  * [5] Demner-Fushman, D., Kohli, M.D., Rosenman, M.B., Shooshan, S.E., Rodriguez, L., Antani, S., Thoma, G.R., McDonald, C.J.: Preparing a collection of radiology examinations for distribution and retrieval. Journal of the American Medical Informatics Association 23(2), 304–310 (2016)
  * [6] Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). pp. 4171–4186. Association for Computational Linguistics (2019)
  * [7] Fang, H., Gupta, S., Iandola, F., Srivastava, R.K., Deng, L., Dollár, P., Gao, J., He, X., Mitchell, M., Platt, J.C., et al.: From captions to visual concepts and back. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1473–1482 (2015)
  * [8] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016)
  * [9] Huang, P.S., He, X., Gao, J., Deng, L., Acero, A., Heck, L.: Learning deep structured semantic models for web search using clickthrough data. In: Proceedings of the 22nd ACM international conference on Information & Knowledge Management. pp. 2333–2338 (2013)
  * [10] Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R., Shpanskaya, K., et al.: Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In: Proceedings of the AAAI conference on artificial intelligence. vol. 33, pp. 590–597 (2019)
  * [11] Johnson, A.E., Pollard, T.J., Berkowitz, S.J., Greenbaum, N.R., Lungren, M.P., Deng, C.y., Mark, R.G., Horng, S.: Mimic-cxr, a de-identified publicly available database of chest radiographs with free-text reports. Scientific data 6(1), 1–8 (2019)
  * [12] Johnson, A.E., Pollard, T.J., Greenbaum, N.R., Lungren, M.P., Deng, C.y., Peng, Y., Lu, Z., Mark, R.G., Berkowitz, S.J., Horng, S.: Mimic-cxr-jpg, a large publicly available database of labeled chest radiographs. arXiv preprint arXiv:1901.07042 (2019)
  * [13] Karpathy, A., Fei-Fei, L.: Deep visual-semantic alignments for generating image descriptions. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 3128–3137 (2015)
  * [14] Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). pp. 1746–1751. Association for Computational Linguistics (2014)
  * [15] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  * [16] Li, G., Duan, N., Fang, Y., Gong, M., Jiang, D.: Unicoder-vl: A universal encoder for vision and language by cross-modal pre-training. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 34, pp. 11336–11344 (2020)
  * [17] Li, L.H., Yatskar, M., Yin, D., Hsieh, C.J., Chang, K.W.: Visualbert: A simple and performant baseline for vision and language. arXiv preprint arXiv:1908.03557 (2019)
  * [18] Li, Y., Wang, H., Luo, Y.: A comparison of pre-trained vision-and-language models for multimodal representation learning across medical images and reports. In: 2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). pp. 1999–2004. IEEE (2020)
  * [19] Li, Y., Liang, X., Hu, Z., Xing, E.P.: Hybrid retrieval-generation reinforced agent for medical image report generation. In: Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada. pp. 1537–1547 (2018)
  * [20] Neumann, M., King, D., Beltagy, I., Ammar, W.: ScispaCy: Fast and robust models for biomedical natural language processing. In: Proceedings of the 18th BioNLP Workshop and Shared Task. pp. 319–327. Association for Computational Linguistics (2019)
  * [21] Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al.: Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems 32, 8026–8037 (2019)
  * [22] Shaikh, M.A., Duan, T., Chauhan, M., Srihari, S.: Attention based writer independent verification. 2020 17th International Conference on Frontiers in Handwriting Recognition (ICFHR) pp. 373–379 (2020)
  * [23] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. In: Advances in neural information processing systems. pp. 5998–6008 (2017)
  * [24] Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., Summers, R.M.: Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 2097–2106 (2017)
  * [25] Wang, X., Peng, Y., Lu, L., Lu, Z., Summers, R.M.: Tienet: Text-image embedding network for common thorax disease classification and reporting in chest x-rays. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 9049–9058 (2018)
  * [26] Xu, T., Zhang, P., Huang, Q., Zhang, H., Gan, Z., Huang, X., He, X.: Attngan: Fine-grained text to image generation with attentional generative adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1316–1324 (2018)
  * [27] Yang, Z., He, X., Gao, J., Deng, L., Smola, A.: Stacked attention networks for image question answering. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 21–29 (2016)


# The First Multiband Photometric Light Curve Solutions of the V Gru Binary
System from the Southern Hemisphere

Mehmet TANRIVER  Astronomy and Space Science, Science Faculty, University of
Erciyes, Kayseri 38039, Türkiye  Erciyes University, Astronomy and Space
Science Observatory Application and Research Center, Kayseri 38039, Türkiye
Atila Poro  Astronomy Department of the Raderon AI Lab., BC., Burnaby, Canada
Ahmet BULUT  Department of Physics, Faculty of Arts and Sciences, Çanakkale
Onsekiz Mart University, Terzioğlu Kampüsü, TR-17020, Çanakkale, Türkiye
Astrophysics Research Center and Observatory, Çanakkale Onsekiz Mart
University, Terzioğlu Kampüsü, TR-17020, Çanakkale, Türkiye  Ahmet KESKİN
Astronomy and Space Science, Science Faculty, University of Erciyes, Kayseri
38039, Türkiye  Mark G. Blackford  Variable Stars South (VSS), Congarinni
Observatory, Congarinni, NSW, 2447, Australia

(Received —. revised —; accepted —; published)

###### Abstract

The first multiband photometric solutions of the short-period V Gru eclipsing
binary from the southern hemisphere is presented in this study. Light curves
of the system were observed through $BVI$ filters at the Congarinni
Observatory in Australia for 15 nights. In addition to the new ground-based
data, we also used the TESS observations in two sectors. We analyzed the light
curves of the system using the PHysics Of Eclipsing BinariEs (PHOEBE) 2.4.7
version code to achieve the best accordance with the photometric observations.
The solutions suggest that V Gru is a near-contact binary system with
$q=1.302(81)$ mass ratio, $f_{1}=0.010(23)$, $f_{2}=-0.0.009(21)$, and
$i=73.45(38)$. We considered the two hot spots on the hotter and cooler
components for the light curve analysis. We extracted the minima times from
the light curves based on the Markov Chain Monte Carlo (MCMC) approach. Using
our new light curves, TESS, and additional literature minima, we computed the
ephemeris of V Gru. The system’s eclipse timing variation trend was determined
using the MCMC method. This system is a good and challenging case for future
studies.

Eclipsing binary stars, photometry, individual (V Gru)

##  1 Introduction

Regarding the type of V Gru system in the binary stars’ categories,
contradictory cases are mentioned in the catalogs. In the Washington visual
double star catalog (WDS) and the AAVSO international variable star index
(VSX), V Gru was classified as a W UMa contact system type (Mason et al.
2001). This binary system is classified as a $\beta$ Lyrae in the machine-
learned ASAS classification catalog (Richards et al. 2012). The review history
of this system fully shows its complexity.

Leavitt & Pickering (1913) identified V Gru as a new southern variable star
with the name of HV 3365. They stated that the HV3365 is an Algol or $\beta$
Lyr type binary system with an apparent magnitude of $V=9.55$. V Gru is
indicated by Houk (1978) in the Michigan Catalog of HD stars Vol-2 as HD
207697. Wolf et al. (1982) presented the first minimum time of V Gru. Wolf &
Kern (1983) presented the $V$, $b-y$, $m_{1}$, $c_{1}$, and $H_{\beta}$ values
of the V Gru binary system in a photometric survey of the southern hemisphere
eclipsing binary stars on nine different Julian days, and they also determined
the period of the system as 0.4833 days. Giuricin et al. (1983) classified
this system as a contact or a typical semi-detached configuration, evolved,
fairly close binary system. Budding (1984) gave the possible mass ratios in
the study, but he stated that the solution is ambiguous. Cannon & Pickering
(1993) gave the photovisual and photographic magnitudes of V Gru as 9.80 and
9.50 magnitudes in the Henry Draper Catalogue and Extension. Demartino et al.
(1996) gave the new coordinates of V Gru at that time in their study ”Accurate
Positions of Variable Stars Near the South Galactic Pole”. Hauck & Mermilliod
(1998) included V Gru in the $uvby\beta$ photoelectric photometric catalogue.
Lopez & Lepez (2000) stated variables in the HIP, TYC, PPM, and AC catalogues.
Budding et al. (2004) included V Gru in their catalogue of Algol-type binary
stars. Avvakumova et al. (2013) gave the orbital period of the V Gru binary
system as 0.4834454954 days in the Catalogue of Eclipsing Variables (Version
2) and also gave the spectral type and the light curve type as F2V and EW.
Kordopatis et al. (2013) gave heliocentric radial velocity as $-19.0\pm 11.4$
km/s in the RAVE 4th data release (DR4). Paunzen (2015) gave $V$ magnitude and
colour indexes of V Gru in the Stroemgren-Crawford $uvby\beta$ photometry
catalog. Kunder et al. (2017) updated Heliocentric radial velocity as
$-18.973\pm 11.447$ km/s in the RAVE 5th data release (DR5). The Cruzalèbes et
al. (2019) study is included in the Mid-infrared stellar Diameters and Fluxes
compilation catalogue (MDF)-version 10. Steinmetz et al. (2020) updated the
value of heliocentric radial velocity as $-18.95\pm 11.34$ km/s in the sixth
data release of the Radial Velocity Experiment (RAVE DR6).

In this investigation, we analyze the light curves of the V Gru system. We
obtained CCD observations with multi-bands at an observatory in Australia and
optimally created binary star models suitable for these observations using the
PHOEBE code. The present study is the first in-depth and multiband photometric
analysis of this system according to ground-based and space-based
observations. Sections present observations, image reduction and instruments
used, light curve analysis, the result of photometry, refind ephemeris, light
curve behaviour, spectral classification, and finally, conclusion,
respectively.

##  2 Observation and data reduction

V Gru was observed for 15 nights in August 2020 with an Orion ED80T CF 80mm
refractor telescope at the Congarinni Observatory in Australia ($152^{\circ}$
$52^{\prime}$ East and $30^{\circ}$ $44^{\prime}$ South). The data was taken
using an Atik One 6.0 CCD camera with $1\times 1$ binning and a CCD
temperature of $-10^{\circ}$C. These observations were made using the Astrodon
Johnson-Cousins $BVI$ standard filters. TYC 7990-634-1 was selected as a
comparison star and TYC 7989-722-1 was chosen as a check star with an
appropriate apparent magnitude in comparison to V Gru. The comparison star was
found at R.A. $21^{h}$ $50^{m}$ $20.93^{s}$, Dec. $42^{\circ}$ $41^{\prime}$
$46.65"$ (J2000) with a $V=10.291(22)$ magnitude, while the check star was
located at R.A. $21^{h}$ $49^{m}$ $39.28^{s}$, Dec. $-42^{\circ}$
$42^{\prime}$ $02.91"$ (J2000) with a $V=10.575(21)$ magnitude, according to
the Simbad111$http://simbad.u-strasbg.fr/simbad$ astronomical database. Figure
1 shows the observed field-of-view for V Gru with the comparison and check
stars. During the observations, a total of 1672 images were acquired. The CCD
image processing was done with MaxIm DL software, which included dark, bias,
and flat-field corrections (George 2000).

The Transiting Exoplanet Survey Satellite (TESS) mission observed the V Gru
system in addition to our ground-based observations. TESS data is available at
the Mikulski Space Telescope Archive
(MAST)222$http://archive.stsci.edu/tess/all\textunderscore products.html$. The
LightKurve code333$https://docs.lightkurve.org$ was used to extract TESS style
curves from the MAST, which had been detrended using the TESS science
processing operations center (SPOC) pipeline (Jenkins et al. 2016). TESS
obtained this data in sectors 1 and 28, which were observed by camera 1 for a
total of 55 days. We used the AstroImageJ (AIJ) program to normalize all of
the ground-based and TESS data (Collins et al. 2017).

Figure 1: Observed field-of-view for V Gru, comparison star, and check star.

##  3 Light curve solutions

Photometric light curve analysis of the V Gru system was performed by Python
Version 2.4.7 of PHOEBE (Prša & Zwitter 2005, Conroy et al. 2020, Poro et al.
2022). We first found a good fit based on all of the initial inputs and the
appearance of the light curves. Aside from some initial inputs, the program
searched for suitable values for mass ratio, orbital inclination, and star
temperature ratio. Then the parameters improved with the optimizing section of
the code. We evaluated all binary types to do the light curve analysis and
found that only the ”semi-detached binary with primary star fills Roche lobe”
model produces the best and most logical results. $g_{h}=g_{c}=0.32$ (Lucy
1967) and $A_{h}=A_{c}=0.5$ (Ruciński 1969) were used as the bolometric albedo
and gravity-darkening coefficients. The stellar atmosphere was described using
the Castelli & Kurucz (2004) approach, and the limb-darkening coefficients
were employed as a free parameter in the solutions.

Based on our light curves in the $B$ and $V$ bands and after the necessary
calibrations (Høg et al. 2000), we determined the ($B-V$) of the system to be
$0.435\pm 0.005$. As a result, the system’s effective temperature ($T_{1}$)
was considered to be $6564\pm 24$ K (Flower 1996). The catalogs list different
values of ($B-V$), such as 0.410 (APASS9) and 0.412 (The Hipparcos and Tycho
catalogs), with a temperature difference of about 100 K between what was found
in this study and what was indicated in the catalogs. We did the initial light
curve analysis after fixing the temperature derived from our ($B-V$) for the
hotter star. Then, using the optimizing section of PHOEBE, we obtained the
temperatures of the stars used in the main photometric light curve solutions.

Using relation 3, the mean fractional radii of the components were computed.
Also, we calculated fillout factors from the output parameters of the PHOEBE
solutions by relation 4.

$\displaystyle r_{mean}=(r_{back}\times r_{side}\times r_{pole})^{1/3}$ (1)

$\displaystyle f=\frac{\Omega(L_{1})-\Omega}{\Omega(L_{1})-\Omega(L_{2})}$ (2)

The results of the light curve solutions of the V Gru system are presented in
Table 1. The observed and synthetic light curves with residuals are displayed
in Figure 2.

The well-known O’Connell effect (O’Connell 1951), is represented by the
asymmetry in the brightness of maxima in the light curve of eclipsing binary
systems. The O’Connell effect can be recognized in the light curves from our
ground-based observations and TESS data. The light curve solutions in this
study need two hot star spots on each of the hooter and cooler components.
Figure 3 shows a three-dimensional view of V Gru as well as the Roche lobe
configuration.

Table 1: Light curve solutions of V Gru. Parameter |  |  | Result
---|---|---|---
$T_{hotter}$ (K) |  |  | $6749(89)$
$T_{cooler}$ (K) |  |  | $5510(103)$
$q=M_{2}/M_{1}$ |  |  | $1.302(81)$
$\Omega_{hotter}$ |  |  | $4.221(66)$
$\Omega_{cooler}$ |  |  | $4.268(47)$
$i^{\circ}$ |  |  | $73.45(38)$
$f_{hotter}$ |  |  | $0.010(23)$
$f_{cooler}$ |  |  | $-0.009(21)$
$A_{hotter}=A_{cooler}$ |  |  | $0.50$
$g_{hotter}=g_{cooler}$ |  |  | $0.32$
$l_{hotter}/l_{tot}$ |  |  | $0.667(2)$
$l_{cooler}/l_{tot}$ |  |  | $0.333(2)$
$r_{hotter}(back)$ |  |  | $0.430(2)$
$r_{hotter}(side)$ |  |  | $0.399(2)$
$r_{hotter}(pole)$ |  |  | $0.379(2)$
$r_{cooler}(back)$ |  |  | $0.374(3)$
$r_{cooler}(side)$ |  |  | $0.345(3)$
$r_{cooler}(pole)$ |  |  | $0.330(3)$
$r_{hotter}(mean)$ |  |  | $0.402(2)$
$r_{cooler}(mean)$ |  |  | $0.359(2)$
Phase shift |  |  | $0.035(5)$
Hotter star |  |  |
$Colatitude_{spot}(deg)$ |  |  | 104(2)
$Longitude_{spot}(deg)$ |  |  | 127(2)
$Radius_{spot}(deg)$ |  |  | 32(1)
$T_{spot}/T_{star}$ |  |  | 1.13(1)
Cooler star |  |  |
$Colatitude_{spot}(deg)$ |  |  | 59(3)
$Longitude_{spot}(deg)$ |  |  | 69(2)
$Radius_{spot}(deg)$ |  |  | 38(1)
$T_{spot}/T_{star}$ |  |  | 1.17(1)
Figure 2: V Gru observational light curves (points) and modeled solutions
(lines) are presented with respect to the orbital phase and shifted
arbitrarily in the relative flux. Figure 3: The positions of the components of
the V Gru system and the cross-sectional outline of the binary system.

##  4 Orbital period variations

To determine the new times of minima and uncertainties, we utilized Python
code using a Gaussian function and the MCMC approach (Poro et al. 2021). The
minimum times in the previous studies and in this study were used to get the
potential orbital period change using mid-eclipse times. For this, TIming
DAtabase at Krakow (TIDAK) was also taken into account (Table 2 and Table A1).
In collecting mid-eclipse times in previous reports, we used only those
observed with the CCD cameras. All times of minima are expressed in
Barycentric Julian Date in Barycentric Dynamical Time ($BJD_{TDB}$).

In this study, a total of seven minima, four primary (Min.$I$) and three
secondary (Min.$II$) were obtained separately for V Gru from $BVI$ ground-
based observations (Table 2). Since we recorded no color dependence on the
timings, we averaged the data acquired from all filters. The reference
ephemeris (Richards et al. 2021) is used for calculating epoch and O–C of mid-
eclipse times:

$\displaystyle BJD_{TDB}(Min.I)=2458727.99696(10)$ (3)
$\displaystyle+0.4834461(5)\times E$

This study provided a total of 196 mid-eclipse timings for V Gru, including
184 extracted from TESS data, listed in Tables 2 and the appendix table. We
used the MCMC approach to obtain a new ephemeris for the system (e,g. Poro et
al. 2022). We find the revised period to be 0.483469038 days for the V Gru
system. Figure 4 shows the O-C diagram of the system. A new revised linear
ephemeris for the mid-eclipse timings obtained from this study, TESS, and
collected from the literature was appointed with the following light elements:

$\displaystyle BJD_{TDB}(Min.I)=2458727.99791_{\rm-(4)}^{+(5)}$ (4)
$\displaystyle+0.483469038_{\rm-(50)}^{+(52)}\times E$

Table 2: Mid-eclipse times based on $BVI$ observations of this study. Min.($BJD_{TDB}$) | Error | Filter | Epoch | O-C
---|---|---|---|---
2459078.02872 | 0.00016 | $V$ | 724 | 0.0168
2459078.02965 | 0.00016 | $B$ | 724 | 0.0177
2459078.02997 | 0.00017 | $I$ | 724 | 0.0180
2459090.11660 | 0.00004 | $V$ | 749 | 0.0185
2459078.26981 | 0.00021 | $I$ | 724.5 | 0.0162
2459078.27130 | 0.00030 | $V$ | 724.5 | 0.0176
2459078.27458 | 0.00021 | $B$ | 724.5 | 0.0209
Figure 4: Left: The O-C diagram of V Gru system with the linear fit on the
data points. Right: Corner plots of the posterior distribution based on the
MCMC sampling.

##  5 Conclusion

The V Gru short-period binary system was photometrically observed during a
period of 15 nights at a southern hemisphere observatory using $BVI$ filters.
We extracted minima times from our observations and TESS data, together with
additional minima times from the literature; a new ephemeris for the system
was presented using the MCMC approach. The O-C diagram shows an increasing and
linear trend.

To analyze the light curves of this system, PHOEBE codes were employed. The
two components have a 1239 K temperature differential, according to Allen’s
table (Cox 2000), these temperatures suggest that the spectral types of the
hotter and cooler components are F2 and G8, respectively.

The appearance of the light curve is typically the basis for classifications
related to machine learning, such as the machine-learned ASAS classification
catalog (Richards et al. 2012). Classifications are generally designed with
the aim of examining systems for which detailed analysis has not yet been
provided and appropriate information is not yet available. As a result, they
cannot be trusted in systems like V Gru. This may be one of the classifier’s
faults, which is a problem that artificial intelligence (AI) is hoping to
solve in the future. It appears that a different classification offered by
Kopal (1955) based on the relationship of the components to their surrounding
Roche lobes is currently preferred.

The short orbital period and light curve analysis of V Gru shows that this
binary system is a near-contact eclipsing binary with the negative fillout
factor for the cooler component. So the hotter component has filled its Roche
lob, but the cooler star has not yet filled it. However, this system needs
investigation with further observations, including spectroscopic analysis.

## Acknowledgments

This manuscript was prepared based on a cooperation between the Binary Systems
of the South and North Project, the astronomy department of the Raderon AI
Lab., and Erciyes University Scientific Research Projects Coordination Unit
(Project No. 11737). In this work, we used $Gaia$ DR3 data from the European
Space Agency (ESA) mission $Gaia$ (http://www.cosmos.esa.int/gaia). The SIMBAD
database, operated by CDS, Strasbourg, France, has been used in this study.
The National Science Foundation (NSF 1 517 474, 1 909 109) and the National
Aeronautics and Space Administration (NASA 17-ADAP17-68) both contributed
funding to the PHOEBE that we utilized. We are especially grateful to Prof.
Edwin Budding for his valuable comments.

## ORCID iDs

Mehmet TANRIVER: 0000-0002-3263-9680
Atila Poro: 0000-0002-0196-9732
Ahmet BULUT: 0000-0002-7215-926X
Ahmet KESKIN: 0000-0002-9314-0648
Mark G. Blackford: 0000-0003-0524-2204

Table 1: Available mid-eclipse times of V Gru obtained by CCD. Min.($BJD_{TDB}$) | Error | Epoch | O-C | Ref. |  Min.($BJD_{TDB}$) | Error | Epoch | O-C | Ref.
---|---|---|---|---|---|---|---|---|---
2458325.50990 | 0.00002 | -832.5 | -0.0182 | TESS | 2458335.17979 | 0.00003 | -812.5 | -0.0172 | TESS
2458325.75160 | 0.00001 | -832 | -0.0182 | TESS | 2458335.42087 | 0.00001 | -812 | -0.0179 | TESS
2458325.99345 | 0.00003 | -831.5 | -0.0181 | TESS | 2458335.66310 | 0.00003 | -811.5 | -0.0173 | TESS
2458326.23501 | 0.00001 | -831 | -0.0182 | TESS | 2458335.90437 | 0.00001 | -811 | -0.0178 | TESS
2458326.47681 | 0.00003 | -830.5 | -0.0182 | TESS | 2458336.14657 | 0.00002 | -810.5 | -0.0173 | TESS
2458326.71852 | 0.00001 | -830 | -0.0182 | TESS | 2458336.38772 | 0.00001 | -810 | -0.0179 | TESS
2458326.96029 | 0.00002 | -829.5 | -0.0181 | TESS | 2458336.62999 | 0.00003 | -809.5 | -0.0174 | TESS
2458327.20183 | 0.00002 | -829 | -0.0183 | TESS | 2458336.87126 | 0.00001 | -809 | -0.0178 | TESS
2458327.44362 | 0.00003 | -828.5 | -0.0182 | TESS | 2458337.11338 | 0.00003 | -808.5 | -0.0174 | TESS
2458327.68548 | 0.00001 | -828 | -0.0181 | TESS | 2458337.35473 | 0.00001 | -808 | -0.0178 | TESS
2458327.92716 | 0.00003 | -827.5 | -0.0182 | TESS | 2458337.59681 | 0.00002 | -807.5 | -0.0174 | TESS
2458328.16892 | 0.00001 | -827 | -0.0181 | TESS | 2458337.83819 | 0.00002 | -807 | -0.0178 | TESS
2458328.41065 | 0.00002 | -826.5 | -0.0181 | TESS | 2458338.08006 | 0.00002 | -806.5 | -0.0176 | TESS
2458328.65233 | 0.00001 | -826 | -0.0182 | TESS | 2458338.32176 | 0.00001 | -806 | -0.0176 | TESS
2458328.89421 | 0.00003 | -825.5 | -0.0180 | TESS | 2458339.77100 | 0.00001 | -803 | -0.0187 | TESS
2458329.13572 | 0.00001 | -825 | -0.0182 | TESS | 2458340.01388 | 0.00002 | -802.5 | -0.0176 | TESS
2458329.37761 | 0.00002 | -824.5 | -0.0180 | TESS | 2458340.25563 | 0.00001 | -802 | -0.0176 | TESS
2458329.61914 | 0.00001 | -824 | -0.0182 | TESS | 2458340.49740 | 0.00002 | -801.5 | -0.0175 | TESS
2458329.86095 | 0.00003 | -823.5 | -0.0181 | TESS | 2458340.73909 | 0.00001 | -801 | -0.0175 | TESS
2458330.10266 | 0.00002 | -823 | -0.0182 | TESS | 2458340.98100 | 0.00003 | -800.5 | -0.0174 | TESS
2458330.34502 | 0.00003 | -822.5 | -0.0175 | TESS | 2458341.22263 | 0.00001 | -800 | -0.0174 | TESS
2458330.58617 | 0.00001 | -822 | -0.0181 | TESS | 2458341.46439 | 0.00003 | -799.5 | -0.0174 | TESS
2458330.82825 | 0.00003 | -821.5 | -0.0177 | TESS | 2458341.70599 | 0.00002 | -799 | -0.0175 | TESS
2458331.06959 | 0.00001 | -821 | -0.0181 | TESS | 2458341.94791 | 0.00002 | -798.5 | -0.0173 | TESS
2458331.31153 | 0.00002 | -820.5 | -0.0179 | TESS | 2458342.18946 | 0.00001 | -798 | -0.0175 | TESS
2458331.55305 | 0.00001 | -820 | -0.0181 | TESS | 2458342.43148 | 0.00003 | -797.5 | -0.0172 | TESS
2458331.79493 | 0.00002 | -819.5 | -0.0180 | TESS | 2458342.67290 | 0.00001 | -797 | -0.0175 | TESS
2458332.03660 | 0.00002 | -819 | -0.0180 | TESS | 2458342.91491 | 0.00002 | -796.5 | -0.0172 | TESS
2458332.27842 | 0.00002 | -818.5 | -0.0179 | TESS | 2458343.15636 | 0.00001 | -796 | -0.0175 | TESS
2458332.52004 | 0.00001 | -818 | -0.0180 | TESS | 2458343.39840 | 0.00003 | -795.5 | -0.0172 | TESS
2458332.76213 | 0.00003 | -817.5 | -0.0176 | TESS | 2458343.63983 | 0.00001 | -795 | -0.0175 | TESS
2458333.00348 | 0.00001 | -817 | -0.0180 | TESS | 2458343.88194 | 0.00003 | -794.5 | -0.0171 | TESS
2458333.24561 | 0.00002 | -816.5 | -0.0176 | TESS | 2458344.12327 | 0.00001 | -794 | -0.0175 | TESS
2458333.48702 | 0.00001 | -816 | -0.0179 | TESS | 2458344.36540 | 0.00003 | -793.5 | -0.0171 | TESS
2458333.72912 | 0.00003 | -815.5 | -0.0175 | TESS | 2458344.60673 | 0.00001 | -793 | -0.0175 | TESS
2458333.97043 | 0.00001 | -815 | -0.0180 | TESS | 2458344.84880 | 0.00003 | -792.5 | -0.0171 | TESS
2458334.21272 | 0.00003 | -814.5 | -0.0174 | TESS | 2458345.09025 | 0.00001 | -792 | -0.0174 | TESS
2458334.45392 | 0.00001 | -814 | -0.0179 | TESS | 2458345.33227 | 0.00003 | -791.5 | -0.0171 | TESS
2458334.69611 | 0.00002 | -813.5 | -0.0174 | TESS | 2458345.57372 | 0.00001 | -791 | -0.0174 | TESS
2458334.93736 | 0.00001 | -813 | -0.0179 | TESS | 2458345.81567 | 0.00002 | -790.5 | -0.0171 | TESS
Table 1: Continued Min.($BJD_{TDB}$) | Error | Epoch | O-C | Ref. |  Min.($BJD_{TDB}$) | Error | Epoch | O-C | Ref.
---|---|---|---|---|---|---|---|---|---
2458346.05722 | 0.00001 | -790 | -0.0173 | TESS | 2459063.52512 | 0.00001 | 694 | 0.0166 | TESS
2458346.29922 | 0.00003 | -789.5 | -0.0170 | TESS | 2459063.76726 | 0.00002 | 694.5 | 0.0170 | TESS
2458346.54072 | 0.00001 | -789 | -0.0173 | TESS | 2459064.00865 | 0.00001 | 695 | 0.0167 | TESS
2458346.78271 | 0.00003 | -788.5 | -0.0170 | TESS | 2459064.25078 | 0.00002 | 695.5 | 0.0171 | TESS
2458347.02407 | 0.00001 | -788 | -0.0174 | TESS | 2459064.49211 | 0.00001 | 696 | 0.0167 | TESS
2458347.26602 | 0.00003 | -787.5 | -0.0171 | TESS | 2459064.73422 | 0.00002 | 696.5 | 0.0171 | TESS
2458347.50800 | 0.00001 | -787 | -0.0169 | TESS | 2459064.97558 | 0.00001 | 697 | 0.0167 | TESS
2458347.99100 | 0.00001 | -786 | -0.0173 | TESS | 2459065.21769 | 0.00002 | 697.5 | 0.0171 | TESS
2458348.47536 | 0.00005 | -785 | -0.0164 | TESS | 2459065.45906 | 0.00001 | 698 | 0.0167 | TESS
2458348.95822 | 0.00005 | -784 | -0.0170 | TESS | 2459065.70110 | 0.00002 | 698.5 | 0.0170 | TESS
2458349.44100 | 0.00001 | -783 | -0.0177 | TESS | 2459065.94251 | 0.00001 | 699 | 0.0167 | TESS
2458349.68364 | 0.00003 | -782.5 | -0.0167 | TESS | 2459066.18458 | 0.00002 | 699.5 | 0.0171 | TESS
2458349.92493 | 0.00001 | -782 | -0.0172 | TESS | 2459066.42605 | 0.00001 | 700 | 0.0168 | TESS
2458350.16688 | 0.00003 | -781.5 | -0.0170 | TESS | 2459066.66808 | 0.00002 | 700.5 | 0.0171 | TESS
2458350.40842 | 0.00001 | -781 | -0.0171 | TESS | 2459066.90959 | 0.00001 | 701 | 0.0169 | TESS
2458350.65038 | 0.00003 | -780.5 | -0.0169 | TESS | 2459067.15163 | 0.00002 | 701.5 | 0.0172 | TESS
2458350.89183 | 0.00001 | -780 | -0.0172 | TESS | 2459067.39294 | 0.00001 | 702 | 0.0168 | TESS
2458351.13379 | 0.00002 | -779.5 | -0.0169 | TESS | 2459067.63503 | 0.00002 | 702.5 | 0.0172 | TESS
2458351.37526 | 0.00001 | -779 | -0.0172 | TESS | 2459067.87636 | 0.00001 | 703 | 0.0168 | TESS
2458351.61747 | 0.00003 | -778.5 | -0.0167 | TESS | 2459068.11850 | 0.00002 | 703.5 | 0.0172 | TESS
2458351.85874 | 0.00001 | -778 | -0.0172 | TESS | 2459068.35982 | 0.00001 | 704 | 0.0168 | TESS
2458352.10074 | 0.00003 | -777.5 | -0.0169 | TESS | 2459068.60201 | 0.00002 | 704.5 | 0.0173 | TESS
2458352.34218 | 0.00001 | -777 | -0.0172 | TESS | 2459068.84332 | 0.00001 | 705 | 0.0169 | TESS
2458352.58441 | 0.00003 | -776.5 | -0.0167 | TESS | 2459069.08546 | 0.00002 | 705.5 | 0.0173 | TESS
2458352.82562 | 0.00001 | -776 | -0.0172 | TESS | 2459069.32683 | 0.00001 | 706 | 0.0169 | TESS
2458353.06853 | 0.00003 | -775.5 | -0.0160 | TESS | 2459069.56887 | 0.00002 | 706.5 | 0.0172 | TESS
2458362.98178 | 0.00380 | -755 | -0.0134 | Richards et al. (2021) | 2459069.81025 | 0.00001 | 707 | 0.0169 | TESS
2458376.99878 | 0.00260 | -726 | -0.0163 | Richards et al. (2021) | 2459070.05236 | 0.00002 | 707.5 | 0.0173 | TESS
2458600.84718 | 0.00360 | -263 | -0.0035 | TIDAK | 2459070.29378 | 0.00001 | 708 | 0.0170 | TESS
2458678.20175 | 0.00031 | -103 | -0.0003 | Richards et al. (2021) | 2459070.53592 | 0.00002 | 708.5 | 0.0174 | TESS
2458688.11438 | 0.00280 | -82.5 | 0.0017 | TIDAK | 2459070.77718 | 0.00001 | 709 | 0.0169 | TESS
2458688.35277 | 0.00350 | -82 | -0.0016 | TIDAK | 2459071.01930 | 0.00002 | 709.5 | 0.0173 | TESS
2458727.99696 | 0.00010 | 0 | 0 | Richards et al. (2021) | 2459075.37048 | 0.00002 | 718.5 | 0.0175 | TESS
2458730.17597 | 0.00052 | 4.5 | 0.0035 | Richards et al. (2020) | 2459075.61194 | 0.00001 | 719 | 0.0172 | TESS
2459062.07470 | 0.00001 | 691 | 0.0165 | TESS | 2459075.85395 | 0.00002 | 719.5 | 0.0175 | TESS
2459062.31694 | 0.00002 | 691.5 | 0.0170 | TESS | 2459076.09541 | 0.00001 | 720 | 0.0173 | TESS
2459062.55821 | 0.00001 | 692 | 0.0165 | TESS | 2459076.33734 | 0.00002 | 720.5 | 0.0175 | TESS
2459062.80034 | 0.00002 | 692.5 | 0.0170 | TESS | 2459076.57882 | 0.00001 | 721 | 0.0172 | TESS
2459063.04162 | 0.00001 | 693 | 0.0165 | TESS | 2459076.82085 | 0.00002 | 721.5 | 0.0175 | TESS
2459063.28382 | 0.00002 | 693.5 | 0.0170 | TESS | 2459077.06236 | 0.00001 | 722 | 0.0173 | TESS
Table 1: Continued Min.($BJD_{TDB}$) | Error | Epoch | O-C | Ref. |  Min.($BJD_{TDB}$) | Error | Epoch | O-C | Ref.
---|---|---|---|---|---|---|---|---|---
2459077.30438 | 0.00002 | 722.5 | 0.0176 | TESS | 2459080.93009 | 0.00001 | 730 | 0.0175 | TESS
2459077.54585 | 0.00001 | 723 | 0.0174 | TESS | 2459081.17185 | 0.00002 | 730.5 | 0.0175 | TESS
2459077.78791 | 0.00002 | 723.5 | 0.0177 | TESS | 2459081.41355 | 0.00001 | 731 | 0.0175 | TESS
2459078.02924 | 0.00001 | 724 | 0.0173 | TESS | 2459081.65566 | 0.00002 | 731.5 | 0.0179 | TESS
2459078.27126 | 0.00002 | 724.5 | 0.0176 | TESS | 2459081.89704 | 0.00001 | 732 | 0.0175 | TESS
2459078.51272 | 0.00001 | 725 | 0.0173 | TESS | 2459082.13905 | 0.00002 | 732.5 | 0.0178 | TESS
2459078.75480 | 0.00002 | 725.5 | 0.0177 | TESS | 2459082.38055 | 0.00001 | 733 | 0.0176 | TESS
2459078.99621 | 0.00001 | 726 | 0.0174 | TESS | 2459082.62257 | 0.00002 | 733.5 | 0.0179 | TESS
2459079.23824 | 0.00002 | 726.5 | 0.0177 | TESS | 2459082.86399 | 0.00001 | 734 | 0.0176 | TESS
2459079.47966 | 0.00001 | 727 | 0.0174 | TESS | 2459083.10602 | 0.00002 | 734.5 | 0.0179 | TESS
2459079.72164 | 0.00002 | 727.5 | 0.0176 | TESS | 2459083.34745 | 0.00001 | 735 | 0.0176 | TESS
2459079.96311 | 0.00001 | 728 | 0.0174 | TESS | 2459083.58949 | 0.00002 | 735.5 | 0.0179 | TESS
2459080.20524 | 0.00002 | 728.5 | 0.0178 | TESS | 2459083.83091 | 0.00001 | 736 | 0.0176 | TESS
2459080.44660 | 0.00001 | 729 | 0.0174 | TESS | 2459084.07287 | 0.00002 | 736.5 | 0.0179 | TESS
2459080.68867 | 0.00002 | 729.5 | 0.0178 | TESS |  |  |  |  |

## References

  * Avvakumova et al. (2013) Avvakumova, E. A., Malkov, O. Y., & Kniazev, A. Y. 2013, Astronomische Nachrichten, 334, 860, doi: 10.1002/asna.201311942
  * Budding (1984) Budding, E. 1984, Bulletin d’Information du Centre de Donnees Stellaires, 27, 91
  * Budding et al. (2004) Budding, E., Erdem, A., Çiçek, C., et al. 2004, A&A, 417, 263, doi: 10.1051/0004-6361:20034135
  * Cannon & Pickering (1993) Cannon, A. J., & Pickering, E. C. 1993, VizieR Online Data Catalog, III/135A
  * Castelli & Kurucz (2004) Castelli, F., & Kurucz, R. L. 2004, A&A, 419, 725, doi: 10.1051/0004-6361:20040079
  * Collins et al. (2017) Collins, K. A., Kielkopf, J. F., Stassun, K. G., & Hessman, F. V. 2017, AJ, 153, 77, doi: 10.3847/1538-3881/153/2/77
  * Conroy et al. (2020) Conroy, K. E., Kochoska, A., Hey, D., et al. 2020, ApJS, 250, 34, doi: 10.3847/1538-4365/abb4e2
  * Cox (2000) Cox, A. N. 2000, Allen’s astrophysical quantities
  * Cruzalèbes et al. (2019) Cruzalèbes, P., Petrov, R. G., Robbe-Dubois, S., et al. 2019, MNRAS, 490, 3158, doi: 10.1093/mnras/stz2803
  * Demartino et al. (1996) Demartino, R., Kocyla, D., Predom, C., & Wetherbee, E. 1996, Information Bulletin on Variable Stars, 4321, 1
  * Flower (1996) Flower, P. J. 1996, ApJ, 469, 355, doi: 10.1086/177785
  * George (2000) George, D. B. 2000, International Amateur-Professional Photoelectric Photometry Communications, 79, 2
  * Giuricin et al. (1983) Giuricin, G., Mardirossian, F., & Mezzetti, M. 1983, A&AS, 54, 211
  * Hauck & Mermilliod (1998) Hauck, B., & Mermilliod, M. 1998, A&AS, 129, 431, doi: 10.1051/aas:1998195
  * Høg et al. (2000) Høg, E., Fabricius, C., Makarov, V. V., et al. 2000, A&A, 357, 367
  * Houk (1978) Houk, N. 1978, University of Michigan, 2
  * Jenkins et al. (2016) Jenkins, J. M., Twicken, J. D., McCauliff, S., et al. 2016, in Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Vol. 9913, Software and Cyberinfrastructure for Astronomy IV, ed. G. Chiozzi & J. C. Guzman, 99133E, doi: 10.1117/12.2233418
  * Kopal (1955) Kopal, Z. 1955, Annales d’Astrophysique, 18, 379
  * Kordopatis et al. (2013) Kordopatis, G., Gilmore, G., Steinmetz, M., et al. 2013, AJ, 146, 134, doi: 10.1088/0004-6256/146/5/134
  * Kunder et al. (2017) Kunder, A., Kordopatis, G., Steinmetz, M., et al. 2017, AJ, 153, 75, doi: 10.3847/1538-3881/153/2/75
  * Leavitt & Pickering (1913) Leavitt, H. S., & Pickering, E. C. 1913, Harvard College Observatory Circular, 179, 1
  * Lopez & Lepez (2000) Lopez, C. E., & Lepez, H. S. 2000, Information Bulletin on Variable Stars, 4824, 1
  * Lucy (1967) Lucy, L. B. 1967, ZAp, 65, 89
  * Mason et al. (2001) Mason, B. D., Wycoff, G. L., Hartkopf, W. I., Douglass, G. G., & Worley, C. E. 2001, AJ, 122, 3466, doi: 10.1086/323920
  * O’Connell (1951) O’Connell, D. J. K. 1951, Publications of the Riverview College Observatory, 2, 85
  * Paunzen (2015) Paunzen, E. 2015, A&A, 580, A23, doi: 10.1051/0004-6361/201526413
  * Poro et al. (2021) Poro, A., Zamanpour, S., Hashemi, M., et al. 2021, New A, 86, 101571, doi: 10.1016/j.newast.2021.101571
  * Poro et al. (2022) Poro, A., Paki, E., Blackford, M. G., et al. 2022, PASP, 134, 064201, doi: 10.1088/1538-3873/ac71cd
  * Prša & Zwitter (2005) Prša, A., & Zwitter, T. 2005, ApJ, 628, 426, doi: 10.1086/430591
  * Richards et al. (2012) Richards, J. W., Starr, D. L., Miller, A. A., et al. 2012, ApJS, 203, 32, doi: 10.1088/0067-0049/203/2/32
  * Richards et al. (2021) Richards, T., Axelsen, R. A., Blackford, M., Jenkins, R., & Moriarty, D. J. W. 2021, JAAVSO, 49, 251
  * Richards et al. (2020) Richards, T., Blackford, M., Butterworth, N., & Jenkins, R. 2020, JAAVSO, 48, 250
  * Ruciński (1969) Ruciński, S. M. 1969, Acta Astron., 19, 245
  * Steinmetz et al. (2020) Steinmetz, M., Guiglion, G., McMillan, P. J., et al. 2020, AJ, 160, 83, doi: 10.3847/1538-3881/ab9ab8
  * Wolf & Kern (1983) Wolf, G. W., & Kern, J. T. 1983, ApJS, 52, 429, doi: 10.1086/190876
  * Wolf et al. (1982) Wolf, G. W., Kern, J. T., Hayes, T. L., & Chaffin, C. R. 1982, Information Bulletin on Variable Stars, 2185, 1