[1]Eduardo Mosqueira-Rey

[1]Department of Computer Science and Information Technologies, University of
Coruña (CITIC), Rúa Maestranza, 9, La Coruña, 15001, Galicia, Spain

[2]Servicio de Anatomía Patológica, Complejo Hospitalario Universitario de A
Coruña (CHUAC), As Xubias, 84, A Coruña, 15006, Spain

# Segmentation, Classification and Interpretation of Breast Cancer Medical
Images using Human-in-the-Loop Machine Learning

David Vázquez-Lema<EMAIL_ADDRESS><EMAIL_ADDRESS>Elena Hernández-
Pereira<EMAIL_ADDRESS>Carlos Fernández-Lozano
<EMAIL_ADDRESS>Fernando Seara-Romera<EMAIL_ADDRESS>Jorge Pombo-
Otero<EMAIL_ADDRESS>*  *

###### Abstract

This paper explores the application of Human-in-the-Loop (HITL) strategies in
training machine learning models in the medical domain. In this case a
‘doctor-in-the-loop’ approach is proposed to leverage human expertise in
dealing with large and complex data. Specifically, the paper deals with the
integration of genomic data and Whole Slide Imaging (WSI) analysis of breast
cancer. Three different tasks were developed: segmentation of
histopathological images, classification of this images regarding the genomic
subtype of the cancer and, finally, interpretation of the machine learning
results. The involvement of a pathologist helped us to develop a better
segmentation model and to enhance the explainatory capabilities of the models,
but the classification results were suboptimal, highlighting the limitations
of this approach: despite involving human experts, complex domains can still
pose challenges, and a HITL approach may not always be effective.

###### keywords:

Human-in-the-loop, Breast cancer, Segmentation, Classification, Interpretation

##  1 Introduction

Cancer is a highly heterogeneous disease and a major contributor to global
mortality, responsible for about 1 in every 6 deaths [1]. Cancer cells are
often undifferentiated, complicating their characterization, and exhibit a
high capacity for proliferation/migration within the tumor microenvironment.
Different subtypes are often identified for each specific type of cancer, with
varying prognoses, survival times, and responses to treatments.

For women, breast cancer (BC) has surpassed lung cancer as the most diagnosed
cancer worldwide accounting for 32% of cases [1, 2], and 23% of mortality.
Overall, breast cancer, lung cancer, and colorectal cancer account for 51% of
all new diagnoses annually for women. The incidence rates of female breast
cancer have experienced a gradual rise of approximately 0.6% annually since
the mid-2000s, primarily attributed to the increased detection of localized-
stage and hormone receptor-positive cases [3].

Currently, the diagnosis of BC in daily clinical practice is made by
immunohistochemical analysis of cancerous breast tissue removed during
surgery, studying the presence or absence of estrogen receptor (ER),
progesterone receptor (PR), HER2 membrane protein, and the Ki-67 proliferation
index. In those healthcare centers with the capacity for massive sequencing,
the most common approach is the study of the tumor cell transcriptome based on
the expression of 50-gene signature [4] in paraffin-embedded tumor tissue,
also called intrinsic subtypes (IS). There is no perfect correlation between
immunohistochemical and genomic classification [5], with the latter being more
accurate but much more expensive and less common in daily clinical practice.

As mentioned earlier, in daily clinical practice in hospitals, and despite the
rapid advances in genomic medicine, pathologists still carry out patient
stratification through manual review of histopathology slides with a light
microscope. Images are the product of molecular mechanisms occurring in
tissues at the molecular level. The shape and distribution of the different
visible elements in pathological images are used to determine the type of
cell, tissue, and cancer, as well as its immunohistochemical response, and are
crucial for the identification, classification, and prognosis of cancer [6].
However, this review is often subjective, time-consuming, and there are
significant intra- and inter-operator variations, which often lead to
discrepancies in diagnostic and subtyping results and sometimes even
misdiagnosis. However, histopathology images remain to this day the
fundamental method for the clinical diagnosis of tumor progression.

Both the tumor genome analysis approach and the histopathological image
analysis approach are complementary and provide i) information about the
genetic origin/component of the disease, ii) spatial information (shape,
distribution, and presence of different cell types) derived from the previous
process, and iii) interaction within its tumor microenvironment.

In this paper we present a comprehensive approach to histological image
analysis of breast cancer, focusing on segmentation and classification tasks
related to cancer types according to their genetic information. In addition,
we introduce interpretability measures to shed light on model performance
during classification.

Throughout the different phases of our research, we actively involved a
pathologist in a human-in-the-loop process [7]. This collaborative effort was
intended to improve the quality of the results by incorporating expert
knowledge. Let us delve into the details of our collaboration.

Our first step involved automated segmentation of histological slides prepared
from formalin-fixed tissue in order to simplify the information contained in
those images and to make them easier to analyze. This segmentation will try to
annotate the different tissue components using different colors to be used in
further steps. The pathologist’s role was crucial here as he meticulously
reviewed the segmentation results and provided corrections where necessary.
His expertise ensured accurate delineation of relevant structures within the
images, usually trying to generalize areas into more uniform groups.

The second step was to proceed with the classification of the segmentation
result for the different images, based on cancer types. Unfortunately, the
problem that we were facing here were rather complex and, with the available
data the results of the different classification techniques applied were not
optimal. The pathologist guided us by identifying critical areas that the
model should focus on during classification but this information was not
helpful to improve the classification results.

Rather than discarding the pathologist’s input, we decided to reformulate it
for interpretability and also as a debugging process. The pathologist
evaluated the interpretability results of the system and we used this
information to see why the models failed and as feedback for our
hyperparameter optimization process. Our goal was not only select the best-
performing model but also the model that offers the best explainability,
something that is very relevant in medical domains.

Thus, our final contribution has been to demonstrate how a human-in-the-loop
process allowed us to improve certain aspects of machine learning model
building and to remark some of the limitations of this type of process: aside
from the complexities of involving human experts we have seen that, in complex
domains, even human experts find it difficult to solve certain problems, and
their collaboration may not always be helpful.

The paper is structured as follows: In section 2 we review the current state
of the art on the subject of breast cancer, human-in-the-loop ML techniques
and ML model explainability algorithms. Section 3 describes the segmentation
algorithm an the corrections made by the pathologist. Section 4 describes the
classification models developed and their problems. Section 5 describes the
methods used to obtain information on the internal functioning of the model,
trying to explain the decisions taken and how the pathologist’s feedback on
these data help us to develop more interpretable models. We ultimately
concluded with our discussion and conclusions about the matter in section 6.

##  2 State of the Art

###  2.1 Context of the problem

The diagnosis of complex diseases, such as cancer, has long relied on Whole-
Slide Histopathological Images (WSIs) as a clinical gold standard tool [8, 9].
This is primarily due to the fact that diagnosis is predominantly based on
morphological patterns observed in WSIs. Such images arise from the need to
analyze histopathological sections of biopsies in an automated manner, aiming
to reduce the highly costly manual process and address the lack of
quantitative precision in cellular characteristics. Before WSI automated
analysis, a prevalent method for diagnosing cancer involves examining tumor
tissue sections under a microscope, which are typically stained with
hematoxylin and eosin (H&E) dyes [10, 11]. This staining process enables
pathologists to qualitatively evaluate cancer types, stages, and approximate
tumor purity based on the H&E-stained image of a biopsy section. Moreover,
histopathologic examination often reveals various cell types, organic
conditions, and/or cellular locations within intricate tissues. However, there
is limited agreement among pathologists regarding diagnoses [12].

Deep learning is the most popular method for analyzing WSI, particularly for
tumor classification [13]. However, a major challenge for deep learning is the
need for a large set of accurately labeled data. Many techniques require WSI
that have been annotated by pathologists. This process of creating a training
dataset can be labor-intensive, especially due to the differences in
interpretation among pathologists. The novelty of the proposed work lies in
integrating human involvement into the training process, thereby adopting a
human-in-the-loop (HITL) approach. To the best of our knowledge, our study is
the first to apply HITL deep-learning method for integration of genomic data
and WSI-based analysis. According to a recent systematic review [14], the
utilization of deep learning techniques for WSI analysis in combination with
omics data remains limited, with a focus primarily on survival analysis. This
indicates an open avenue for further research in the field.

This study will focus on a specific type of cancer, namely, breast cancer.
Therefore, it is necessary to understand the known genomic differences that
characterize this pathology. Breast cancer intrinsic molecular subtypes
represent unique biological entities, each characterized by specific gene
expression profiles. These widely accepted subtypes, including Luminal A,
Luminal B, HER2-Enriched, Normal-like, and Basal-like, hold prognostic value
and offer potential therapeutic insights [15]. Luminal A cancers are
characterized by low proliferation markers and the presence of hormone
receptor-positive genes. They grow slowly and have a good prognosis with
hormone therapy. In contrast, Luminal B tumors have higher hormone receptor
expression and grow faster with a worse prognosis requiring more intensive
treatment. HER2-enhanced tumors overexpress the HER2 gene and are more
aggressive with rapid growth compared to luminal tumors. Basal-like tumors are
characterized by the absence of ER/PR hormone receptors and HER2
amplification. They are known as “triple-negative” breast cancers, with
aggressive growth and chemotherapy as the main treatment. Normal-like cancers
share gene expression patterns with normal breast tissue and are often
excluded from analyses due to their similarity to non-cancerous tissue.

###  2.2 Dataset

The Cancer Genome Atlas (TCGA) [16] is a public and collaborative database
focused in cancer research that aimed to study 33 different types of cancer,
such as breast, lung or blood, among others. The TCGA project started in 2006
with the joint effort between the National Cancer Institute(NCI) and the
National Human Genome Research. Since then, information about over 11.000
patients is publicly available, containing genomic, epigenomic, transcriptomic
and proteomic data. The Cancer Imaging Archive (TCIA) [17] is a open access
repository that hosts a large archive of medical images of various types of
cancer. All the images provided can be downloaded, facilitating the research
of the various cancers. TCGA and TCIA projects are focused on different
aspects of the cancer research, but both are related. In this project, this
relationship will allow to download the images belonging to the patients
included in TCGA, allowing the creation of a custom dataset.

Our initial dataset was composed by a custom set of patients downloaded from
TCGA, containing personal, clinical and genomic information. The most
important features for our project are the unique patient identifier, that
identify all the tissue images related with them, and the cancer type.

This project is centered solely in breast cancer. The dataset was custom-
designed, composed by a total of 1230 patients obtained from the TCGA
database. Over the 10% of the dataset did not contain cancer type information,
a significant amount of missing values that needed to be clear. This patients
are not useful for the project and were immediately discarded.

After cleaning the missing values, a total of 2783 images related to the
remaining patient identifiers were downloaded from the TCIA repository. The
images downloaded use a digital and high resolution representation of the
microscope slides. The percentage of images related with each cancer type is
shown in Figure 1(a).

The remaining images presented at least two tissues in each image or color pen
marks that needed to be removed, so some preprocessing task was needed. Also,
the images need to be converted to a more manageable format to apply all the
previously mentioned preprocessing.

Finally, the images that didn’t contain enough tissue after the preprocessing
were discarded. The final built dataset is composed by over 1000 tissue images
and its distribution among cancer types is shown in Figure 1(b). The
distribution shows that the dataset is imbalanced. Almost half of the final
cases belong to the Luminal A class, followed by the Basal and Luminal B
classes with over the 15% of the data each. Her2 and Normal classes are under-
represented and will be discarded in some of the experiments.

(a)

(b)

Figure 1: (a) Percentage of downloaded images belonging to each cancer type
(b) Percentage of the final dataset images belonging to each cancer type.

To increase the amount of data available, some data augmentation operations
have been applied: horizontal flipping, vertical flipping and zooming.

###  2.3 Human-in-the-loop Machine Learning

Human-in-the-Loop Machine Learning (HITL-ML) is the practice of uniting human
and machine intelligence to create effective machine learning algorithms [7].
In this approach, humans can play several roles before, during and after the
learning process with the goal of enhancing the capabilities of ML models by
incorporating human intelligence, judgment, and feedback.

The main HITL methods revolve around the learning process, but have also been
proven useful for tasks performed just before or after the learning algorithm.
Taking this into account we can classify HITL methods according to their
relationship to the learning process [18], as depicted in Fig. 2.

Figure 2: Classification of HITL methods.

Before learning, humans play a critical role in tasks such as Feature
Engineering, even in the era of Deep Learning. This is because humans possess
domain expertise and intuition about the data they work with and they can
uncover hidden patterns and create meaningful features and decide how to
transform raw data into numeric representations that can feed the model. Apart
from the common involvement of humans in feature engineering, there have been
also proposed some HITL techniques [19, 20]. Humans can also have a role in
process such as Curriculum Learning (CL) [21], in which a scoring function
organizes the dataset in terms of increasing complexity to take advantage of
previously learned concepts and to ease the abstraction of new concepts. The
definition of a good scoring function is one of the most complex procedures of
CL, and one in which we can make use of human expertise.

During learning is when the HITL techniques can excel. Here we can identify
three distinct categories regarding who is in control of the learning process
[22]:

  * •

Active Learning (AL) [23], where the system controls the learning process and
relies on human input to label unlabeled data to improve model performance.

  * •

Interactive ML (IML) [24], characterized by closer interaction between users
and learning systems, where humans frequently and interactively provide
information to the system influencing in the learning process. One notable
example of this type of HITL-ML because of its use in modern Large Language
Models (LLMs) can be considered Reinforcement Learning from Human Feedback
(RLHF) [25] in which human feedback is incorporated as an integral part of the
learning process, actively shaping the models’s behavior by combining
environmental rewards and human insights.

  * •

Machine Teaching (MT) [26, 27], where human domain experts use didactic
techniques to control the learning process without needing any ML skills or
expertise. This leads to curious and interesting interrelationships between
human experts and ML models, which can be used as a didactic technique in
itself, not to educate the model, but to educate the human itself [28].

After learning there is also room for some HITL techniques. This is because
nowadays is widely accepted that, if we want an AI model that to be
successful, the model must also include explanatory capabilities. This is what
is called Explainable Artificial Intelligence (XAI) [29, 30] and several
approaches have been carried out to obtain feedback from experts with the
intention of not only improving the accuracy of the results, but also the
explanatory capabilities of the system [31].

Finally, we can include techniques that are beyond the learning process, such
as those involved in usable AI [32] that focuses on ease of interaction with
AI systems, useful AI [32] that emphasizes practical outcomes of AI systems
and trustworthy AI [33] that ensures ethical and reliable deployment of AI
systems.

###  2.4 Explainable Artificial Intelligence

ML has already exceed human experts in specific areas, such as detecting
tumors or evaluating disease progression. Although complex ML models can
address complex problems, their black-box nature raises concerns about
transparency and responsibility. Predictions made by these models cannot be
traced back, making it unclear how or why they arrived at a certain outcome.
This lack of explainability can lead to issues of trust in the ML system and
the inability to provide users with human-interpretable explanations for its
decisions.

To address this issue, Explainable Artificial Intelligence (XAI) proposes a
shift towards more transparent ML. Its aim is to create a set of techniques
that produce more explainable models while maintaining high levels of
performance in two ways: making the proper black-box model more transparent or
using a transparent model to help end-users better understand model behaviors.
Self-explainable modeling concerns constructing self-explanatory models at the
beginning of training, e.g., building a decision tree or applying
interpretability directly to the structure of the model, e.g., adding an
attention layer to deep learning models.

When ML models do not allow to be self-explainable, a separate method must be
conceived and applied to the model to explain its decisions. This is the
purpose of post-hoc explainability techniques, which aim to communicate
understandable information about how an already developed model produces its
predictions for any given input. Since how a prediction is made should be as
transparent as possible in a faithful and interpretable manner, model-specific
and model-agnostic approaches have emerged, covering local and global
interpretability [34]. While local explanations focus on explaining individual
predictions, global explanations are the information intrinsically available
in the interpretable models, such as the weights of linear models or the
feature importance scores in tree models [35]. One of the main arguments for
the need for local explanations is that obtaining a global explanation of
complex black-box models for all instances might be hard [36].

The problem with the post-hoc explanatory techniques is their reliability.
Some authors [37] pointed out that they are inconsistent, unstable and provide
very little information on their correctness and reliability, as well as being
computationally inefficient.

##  3 Task 1: Segmentation

###  3.1 Deep Multi-Magnification Network

The main objective of the segmentation process is to recognize the different
components of the tissue slide and identify them with a color code. Table 1
shows the color code used in the segmentation process.

Table 1: Table with the tissue components and their associated color. Component | Stroma | Necrotic | Carcinoma | Adipose | Benign epithelial
---|---|---|---|---|---
Color | Red | Blue | Yellow | Green | Orange
|  |  |  |  |

The Deep Multi-Magnification Network model (DMMN) [38], a pretrained breast
cancer model that uses partial annotations as inputs, was used to
automatically segment the different tissue components by using a set of
patches from multiple magnifications of the whole slide images. The steps that
define this model are the following:

  * •

Partial annotations. The objective is to annotate some regions of the WSI and
identify the different components of the tissue. The annotation of the
different regions of a WSI is done by an expert. As this operation requires a
lot of time and effort, to significantly reduce them, the partial annotations
done by the expert are going to be used to train a model in further steps. The
different tissue components and the color associate to them are shown in Table
1.

  * •

Training patch extraction. WSIs are generally too large to be processed by a
CNN. To analyze them, patch-based methods where used, dividing the WSI in
smaller patches and processing them with a CNN, combining the output of each
patch to form an image.

  * •

Class balancing. When the number of patches of one tissue component is
significantly higher than another tissue component, the CNN can not correctly
learn the minor one. To solve this problem, a class balancing technique is
applied, the elastic deformation technique [39], that creates new images
applying local deformations to the original images. This technique is applied
to each patch.

  * •

Training. The CNN is trained with all the images obtained in the previous
steps. An example of the output of the trained model is shown in Figure 3.

Figure 3: An example of the output of the segmentation model.

###  3.2 HITL for segmentation correction

The result of the segmentation process was evaluated by a human expert, a
pathologist, who was responsible for identifying erroneous segmentations and
correcting them. A total of 19 images were analyzed and corrected by the
pathologist expert, conformed by over 3 images of each cancer type. Figure 4
shows an example of the images that were provided to the pathologist to
evaluate, where:

(a)

(b)

Figure 4: (a) Segmented image obtained by the network superposed over the
original one and (b) original Whole Slide Image (WSI).

  * •

the WSI is the image that experts usually use to identify cancer nodes and it
was requested by the pathologist.

  * •

the result of the segmentation superimposed on the original image to
facilitate the pathologist’s task.

A total of 19 images underwent analysis by the pathologist. Figure 5
illustrates an example with the proposed corrections, along with the corrected
image that followed. The pathologist meticulously reviewed the 19 segmented
images, annotating regions on the whole-slide image (WSI) where he deemed
necessary. By annotating the segmented image, a new version was generated,
adhering to the corrections suggested by the pathologist. These adjustments
effectively simplified the information within the image, compacting various
areas while preserving essential high-level features.

Figure 5: Corrections proposed by the pathologist (left) and segmented image
after applying the pathologist’s corrections.

The analysis process carried out by the pathologist to correct the
segmentation errors led to the following conclusions:

  * •

The segmentation images analyzed showed a maximum accuracy in identifying
tumor nests, which were the part that the pathologist was most interested in.

  * •

The segmentation images analyzed showed a non-maximum accuracy in identifying
desmoplastic/neoplastic stroma and necrotic areas.

  * •

The main deficit is recognizing the normal adipose tissue and normal
ductal/glandular structures.

##  4 Task 2: Classification

###  4.1 Models

The selection of a model is a key aspect of the classification task. In this
case, we have decided to use pretrained models and transfer learning thanks to
their accessibility and the resource efficiency, avoiding the need to train a
new model from scratch, but mainly because their high capability to extract
patterns from any kind of image.

The first model that was selected is Xception, a CNN architecture firstly
introduced in 2016 by François Chollet [40]. The name Xception is derived from
“Extreme Inception”, since it uses the Inception architecture principles and
adds several innovations, such as depthwise separable convolutions, depthwise
convolutions and pointwise convolutions, reducing significantly the number of
parameters of the network while having similar computational capabilities to
regular convolutions. Figure 6 shows the architecture of the Xception model.

Figure 6: Architecture of the Xception model. The numbers in each block
represent the kernel size, the number of filters in the current block and the
stride.

The other model is Resnet50, one of the most widely used CNN architectures
that was firstly introduced in 2015 by Kaiming He et al. [41]. The Resnet
networks are formed by multiple residual blocks, that helps avoiding the
vanishing gradient problem passing the input of some layers to deeper layers.
This version of Resnet has a total of 50 layers, having a considerable
capacity solving computer vision problems. Figure 7 shows the architecture of
the Resnet50 model.

Figure 7: Architecture of the Resnet model. The numbers in each block
represent the kernel size, the number of filters in the current block and the
stride.

The metrics used to analyze the results of the process are Accuracy and
F1-score. The first one is the most common metric and will analyze the
proportion of correctly classified instances out of the total of instances,
providing a general overview of the performance of the trained model across
all classes. The second one is really useful for imbalanced datasets, giving a
more robust measure of the model’s performance in those scenarios.

###  4.2 Experiments

Using the previous explained models and the images downloaded a total of 4
experiments were done. The main difference between the 4 experiments is
whether they use the segmented images or if they add the images analyzed by
the pathologist and the number of cancer types to classify.

####  4.2.1 Experiments with the segmented images

The first two experiments used the images obtained directly from the
segmentation model. Although the experiments use different amount of images,
the train, validation and test subsets are composed by the 60%, 20% and 20% of
the data in both cases, respectively. The pretrained model used in both cases
is the Xception one with the ImageNet weights. The last two convolutional
blocks of the pretrained model will be retrained, maintaining the ImageNet
weights in the rest of them. After the pretrained model layers, a flatten
layer followed by 2 dense layers with the ReLU activation function, a dropout
layer for regularization purposes and 1 dense layer with the softmax
activation function were added. The added layers will process the extracted
features obtained from the pretrained model to classify them one of the target
classes. Table 2 shows the hyperparameters that were selected for these
experiment.

Table 2: Table with the hyperparameters of experiments 1 and 2. Batch size | Dropout rate | Learning rate | Input shape | Optimizer
---|---|---|---|---
32 | 0.5 | 0.0001 | (224, 224) | Adam
|  |  |  |

The specific characteristics of each experiment are explained in the following
paragraphs:

##### Experiment 1

Around 900 images were used in this experiment. The mentioned images can
belong to one of the following 4 cancer types: Basal, Her2, Luminal A and
Luminal B.

##### Experiment 2

Around 780 images were used in this experiment. The mentioned images can
belong to one of the following 3 cancer types: Basal, Luminal A and Luminal B.
The previous experiment was unable to correctly classify any of the images
belonging to the Her2 class. On top of that, it is the class with the less
amount of images and one of the most difficult to classify. For those reasons,
the Her2 class was excluded from this experiment.

####  4.2.2 Experiments with the images analyzed by the pathologist

These experiments use the images obtained by the segmentation model, but they
also introduce the segmentation images corrected by the pathologist to the
training set. Our objective was to see if the corrected segmentation images
could help with the learning of key features and the generalization of the
model during the training process. Although the experiments use different
amount of images, the train, validation and test subsets are composed by the
60%, 20% and 20% of the data in both cases, respectively. The specific
characteristics of each experiment are explained in the following paragraphs:

##### Experiment 3

Around 900 images were used in this experiment. The mentioned images can
belong to one of the following 4 cancer types: Basal, Her2, Luminal A and
Luminal B. The pretrained model used in this case is the Xception one with the
ImageNet weights. The architecture used in this experiment is the same that
was used in the previous 2 experiments. Table 3 shows the hyperparameters that
were selected for this experiment.

Table 3: Table with the hyperparameters of experiment 3. Batch size | Dropout rate | Learning rate | Input shape | Optimizer
---|---|---|---|---
32 | 0.404 | 0.0001 | (224, 224) | Adam
|  |  |  |

##### Experiment 4

Around 780 images were used in this experiment. The mentioned images can
belong to one of the following 3 cancer types: Basal, Luminal A and Luminal B.
In order to compare this experiment with experiment 2, the class Her2 was
discarded. The pretrained model used in this case is the Resnet50 one with the
ImageNet weights. After the pretrained model layers, a flatten layer followed
by 3 dense layers with the ReLU activation function, a dropout layer and,
finally, a dense layer with the softmax activation function were added. Table
4 shows the hyperparameters that were selected for this experiment.

Table 4: Table with the hyperparameters of experiment 4. Batch size | Dropout rate | Learning rate | Input shape | Optimizer
---|---|---|---|---
46 | 0.404 | 0.0001 | (256, 256) | Adam
|  |  |  |

###  4.3 Results

This section will present and discuss the results obtained by each of the
previously explained experiments. Table 5 shows the accuracy and the F1-score
obtained by each experiment.

Overall, the results obtained were not good, showing that the model struggles
differentiating between the different classes, even adding segmentation images
corrected by a pathologist to the training process. The classes that were more
difficult to classify were the Her2 and Basal ones, probably because the
amount of images belonging to these cancer types were limited compared with
the Luminal A and Luminal B cancer types.

The corrected segmentations showed mixed performances, helping in the
experiments that used 4 cancer types, but penalizing the experiments with 3
cancer types.

Table 5: Table with accuracy and F1-score metrics of each experiment Experiment | Accuracy | F1-score
---|---|---
Experiment 1 | 0.34 | 0.24
Experiment 2 | 0.43 | 0.36
Experiment 3 | 0.39 | 0.18
Experiment 4 | 0.33 | 0.20
|  |

Cancer classification directly from images is a complex task that is still
under research. For pathologists, the detection of cancer regions directly
from images is quite difficult and time consuming. The state of art for breast
cancer detection is to analyze 50 well-known genes and determine the cancer
type that the patient has. For the previous reasons, it is normal that the
results obtained were suboptimal.

##  5 Task 3: Interpretation

###  5.1 LIME

LIME or Local Interpretable Model-agnostic Explanations [42] is a novel
explanation technique that explains predictions of any classifier in an
interpretable and faithful manner by learning an interpretable model locally
around the prediction.

Intuitively, an explanation is a local linear approximation of the model’s
behavior. While the model may be very complex globally, it is easier to
approximate it around the vicinity of a particular instance. While treating
the model as a black box, the instance to explain is perturbed and a sparse
linear model around it is learned as an explanation. The operation of the
method is as follows:

  * •

Generate a perturbed dataset: Each training instance is modified and a new
dataset with the modified instances is created. For image instances, different
regions of the image are modified hiding their pixels.

  * •

Classify the perturbed dataset: the original ML model is used to classify the
instances of the perturbed dataset.

  * •

Calculate distances: The distance between the original instance and the
perturbed instance is calculated using cosine similarity and then normalized.

  * •

Train of a linear explainable model: Using the perturbed dataset, the original
ML model predictions and the distances previously calculated a linear
regression model is trained. The most likely prediction will be the class to
which the LIME explanation corresponds. The coefficients of the resulting
linear regression model with the largest magnitude are those that have the
greatest significance in the prediction of the model.

A weak point of this approach is the required transformation of any type of
data in a binary format that is claimed to be human interpretable. Moreover,
in practice the explanation is only provided through linear models and their
features importance [43].

###  5.2 SHAP

SHAP (SHapley Additive exPlanations) [44] is a method to explain individual
predictions, based on the game theoretically optimal Shapley values. Shapley
values [45] are a widely used approach from cooperative game theory that come
with desirable properties. The feature values of a data instance act as
players in a coalition. The Shapley value is the average marginal contribution
of a feature value across all possible coalitions.

SHAP creates explanations of the model by asking for each prediction and
feature the following question: “How does prediction $x$ change when feature
$y$ is removed from the model?” The answer is Shapley values, which quantify
the magnitude and direction (positive or negative) of the effect of a feature
on a prediction. Shapley values can produce model explanations with the
clarity of a linear model.

When calculating Shapley values, we calculate the contribution of a feature by
determining its marginal contributions to all possible subsets and then taking
the weighted average. While this may be feasible for models with a small
number of input features, as the number of input features increases, the cost
of this calculation is sufficiently high and, in some cases, impossible to
obtain. This is why Shapley values are usually approximated by the closest
values using KernelSHAP [44].

KernelSHAP is a method that uses the LIME technique to calculate the Shapley
values. With KernelSHAP, a sufficient number of coalition vectors (simplified
features) is created. In each vector, features with a value of 1 are replaced
by their actual values, and features with a value of 0 are replaced by
different feature values. For this instance, the prediction of the ML model is
obtained and the weight of each feature is calculated. A linear model is
trained using these weight values whose target value is the predicted value of
the ML model for that sample. The coefficient values of the linear model
correspond to Shapley Values for each feature.

SHAP offers both global and local interpretability. It provides a holistic
understanding of the overall model behavior by analyzing feature importance
globally. Additionally, it allows for specific explanations on individual
predictions, enhancing fine-grained interpretability.

###  5.3 Grad-CAM

Gradient-weighted Class Activation Mapping (Grad-CAM) [46], is a technique for
producing _visual explanations_ for decisions from a large class of
Convolutional Neural Network (CNN)-based models, making them more transparent.

The Class Activation Mapping (CAM) [47] method identifies discriminative
regions used by a restricted class of image classification CNNs which do not
contain any fully-connected layers. This approach is a generalization of CAM
and is applicable to a significantly broader range of CNN model families.

The Grad-CAM uses the gradients of any target concept, flowing into the final
convolutional layer to produce a coarse localization map highlighting the
important regions in the image for predicting the concept by means of a heat
map. For this purpose, the algorithm analyses the last convolution layer of
the model, which is the one that captures the high-level features and contains
information about the important regions of the input image. The outputs of
this last convolution layer, which are several feature maps, are computed
together with the prediction, to result in as many gradients as feature maps.

Once these gradients are obtained, the next step is to compare the different
feature maps in terms of relevance using the Global Average Pooling [48]. The
higher the average on a given feature, the greater the influence of that image
feature on the prediction. Once we have the rating of each feature map, we
proceed to obtain the heat map. To do this, the weighted sum of the feature
maps is performed and used as input to a ReLU function (this activation
function is used since only the values with a positive influence on the
prediction are of interest) which returns the heat map. The final step
involves superimposing the heat map on the original image to identify the
areas of the image most relevant to the prediction.

Extensive human studies reveal that the Grad-CAM visualizations can
discriminate between classes more accurately, better reveal the
trustworthiness of a classifier, and help identify biases in datasets [46].

###  5.4 HITL optimization

####  5.4.1 Bayesian hyperparameter optimization

As we know, the hyperparameters of a ML model are those variables whose values
are set prior to the application of the model on the data and whose values are
not adjusted during the training of the model [49]. They are parameters that
can be adjusted manually or by algorithms and that control the behavior of the
model and/or determine its structure.

There are many hyperparameters in a ML model, and finding the optimal value
for them is a complex task that is usually tried to be automated by means of
optimization mechanisms. In our particular case we decided to include the
following hyperparameters in the optimization process.

  * •

Dropout: It specifies the percentage of neurons to be deactivated in a layer
during a training iteration. It is used to avoid network overfitting, which
occurs when the model fits too closely to the training data and its noise,
thus losing generalization power. A small dropout rate may prevent less
overfitting, while a large dropout rate may produce models with lower pattern
learning capacity.

  * •

Learning rate: Determines how much the model weights should be adjusted as a
function of the gradient of the loss function during each iteration of
training. A small learning rate slows down the training but smoothes the
convergence, while a high learning rate speeds up the learning but convergence
may not be achieved.

  * •

Batch size: Determines the number of samples used in each iteration of the
training. A small batch size may produce greater variance between iterations
as the weights between one set of samples and another may be very different
when updated. A large batch size requires more memory and may be less
efficient in small datasets.

The goal of hyperparameter optimization (also known as hyperparameter tuning)
is to find the optimal set of hyperparameters for a given learning algorithm
[50]. The simplest way to perform this task is to carry out a search. It could
be a Grid Search, which involves exhaustively exploring a manually specified
subset of the hyperparameter space; or a Random Search, that instead of
exhaustive enumeration, selects hyperparameter combinations randomly.

A more advance method for hyperparameter optimization is Bayesian Optimization
(BO) [51]. This method consists of considering an objective function that we
want to optimize that it is typically a performance metric (accuracy in our
case) of the ML model given some hyperparameters. If we were to know the
probability distribution of this objective function, then we can simply
compute the gradient descent and find the global minimum or maximum (depending
on the metric used). However, since we don’t know the shape of this function
we need to use techniques, such as BO.

BO is a probabilistic optimization technique that aims to find the global
minimum (or maximum) of an objective function using a surrogate model (often
by means of a Gaussian Process model [52]) to approximate the true objective
function. The surrogate model guides the search by suggesting where to
evaluate the objective function next (applying an Expected Improvement process
[53]). As more evaluations are done, the surrogate model gets refined, leading
to better decisions. This method is considered appropriate for tuning
hyperparameters because it can find and optimal value of a surrogate function
with only a small number of samples [54].

For the BO process to work correctly it is necessary to establish the range of
values of each hyperparameter with which the Bayesian optimization will test
different configurations. These ranges are quite wide since in the first
optimization we seek to achieve the configuration that maximizes the accuracy
at its highest possible value without taking so much into account the
execution time. The ranges chosen for our hyperparameters were:

  * •

Dropout: [0.05, 0.5]

  * •

Learning rate: [0.00001, 0.1]

  * •

Batch size: [1, 32]

With the objective function and the hyperparameter ranges defined, the last
step is to perform the Bayesian optimization itself. In our case it was done
using the Bayesian Optimization package [55].

####  5.4.2 Human-in-the-Loop hyperparameter tunning

Bayesian optimization has been successful applied in several areas, apart from
hyperparameter tuning, such as A/B Testing, recommender systems, robotics and
reinforcement learning, etc. [56]. Some of these applications involves the use
of human feedback to drive the process. For example, in [57], the authors use
Bayesian optimization to set the parameters of several animation systems by
showing the user examples of different parameterized animations and asking for
feedback. This interactive Bayesian optimization strategy is particularly
effective as humans can be very good at comparing examples, but unable to
produce an objective function whose optimum is the example of interest.

Another work [58] used human-in-the-loop Bayesian optimization for quickly
identifying near-optimal control parameters based on real-time physiological
measurements. The experiment consisted of minimizing metabolic cost by
optimizing walking step frequencies in unaided human subjects and it proved
that Bayesian optimization could be used in the context of Human-Computer
Interaction (HCI) to improve system convergence and lower overall energy
expenditure.

Finally, in [31] a human-in-the-loop optimization technique was carried out
not only to improve the performance of the model but also render its decision-
making more interpretable and explainable to human users. In our case we
followed a similar approach and we assessed the quality of the
interpretability explanations returned by the XAI algorithms using domain
experts (in this case, a pathologist). It is this pathologist’s assessment
that will guide the hyperparameter optimization process by Bayesian
optimization. The process is composed of the following phases:

  1. 1.

BO of hyperparameters of the classification model: An initial optimization is
performed following a Bayesian optimization approach and we are left with the
models with the highest accuracy in the validation set.

  2. 2.

Application of XAI methods: The selected XAI methods are applied to each of
the models to obtain an interpretation of what part of the original figure was
used by the model to obtain a final classification.

  3. 3.

Selection of the set of images to be evaluated: An image set consisting of the
original biopsy image, its segmentation and its interpretation by each of the
models is selected.

  4. 4.

Evaluation of the pathologist interpretation: For each of the images in the
previous set, the pathologist will evaluate each XAI method interpretation
with a grade based on his own rubric and shown in Table 6, where 5 is the best
punctuation and 0 the minimum one.

  5. 5.

Analysis of results: Once the pathologist scores each image interpretation, we
select the model with the best interpretability and explainability, taking
into account other metrics such as accuracy or loss in case of a tie.

Table 6: Table with the evaluation rubric described by the pathologist Grade |  Criteria
---|---
0 |  The area of interest, highlighted in green, does not appear or is completely outside the biopsy.
1 |  The area of interest is almost entirely outside of the biopsy.
2 |  The area of interest is both inside and outside the biopsy, but without matching to the shape of the tissue.
3 |  The area of interest is mostly within the biopsy and matches the shape of tissues of little relevance to tumor staging
4 |  The area of interest is mostly within the biopsy and matches the shape of relevant tissues, although also some less relevant ones.
5 |  The area of interest is completely within the biopsy and fits perfectly with relevant areas such as the carcinoma.
|

This process is iterative and is repeated each time, making the variation
margins of the selected hyperparameters smaller and smaller, so that each time
we get closer to their optimal value. Since the classification results of the
different models were far from optimal we decided to focus the objective of
the optimization process in augmenting the quality of the interpretation. But
the same technique can be used for improving not only interpretation but also
classification metrics.

###  5.5 Results

In this section we will show the interpretability results obtained with the
different explainability models. We will show some examples of the
optimization process and the final results obtained.

####  5.5.1 Selection of the explainability method

The first step was to compare the results of the different explainability
methods and draw some conclusions. The LIME method returns interpretations
such as those in Figure 8. In it we can see the area of interest of the image,
in green color, where (according to LIME) the classification model is being
focused to classify it in this case as Basal type. It is an interpretation
that allows the expert, observing several images belonging to different
classifications, to draw conclusions of how the model is behaving internally.

Figure 8: LIME results for a given example.

The SHAP method obtains results such as those shown in Figure 9. It can be
seen that both the regions of interest that determine the classification of
the model (in this case, in red) and those that do not reflect the
classification provided by the model (in blue) are scattered over the image
without discriminating between biopsy area or image background. Moreover,
these regions of interest are scarce. These results are very inadequate, so it
was decided not to rely on SHAP interpretability in the optimization process
with the collaboration of human experts.

Figure 9: SHAP results for a given example.

The Grad-CAM method was also discarded since it was not possible to obtain
accurate interpretations that would help to understand the model. Consistent
results were not even obtained for different predictions on the same image. In
Figure 10 we can see that the heat map superimposed on the original image has
several issues: it highlights areas outside the image and inside the image the
highlights do not conform with the tissue structures present in the biopsy.
Here warm colors such as the red and orange correspond to a high score for a
given class, while cold colors, such as green and blue correspond to small
evidence for the class.

Figure 10: Grad-CAM results for a given example.

These problems with the interpretability algorithms may be also a consequence
of the problems that the ML models have to correctly classify each cancer
type. But, since the LIME results were the only ones with enough quality to be
presented to the pathologist, these were the results that were used in the
Bayesian optimization of hyperparameters.

####  5.5.2 Optimization of the explainability model

The hyperparameter optimization process started in the first iterations
without any human intervention in order to discard those combinations that
offer worse results. The results obtained can be seen in Table 7 from which we
decided to discard models 3 and 4 and focus the expert feedback on models 1, 2
and 5.

Table 7: Accuracy results of the different models considered with their corresponding hyperparameters Model | Batch size | Dropout rate | Learning rate | Accuracy
---|---|---|---|---
Model 1 | 6 | 0.404 | 0.00175 | 0.38
Model 2 | 4 | 0.405 | 0.00100 | 0.38
Model 3 | 1 | 0.126 | 0.00878 | 0.27
Model 4 | 1 | 0.123 | 0.00888 | 0.26
Model 5 | 7 | 0.425 | 0.00019 | 0.38
|  |  |  |

In every human-in-the-loop approach there is a trade-off between the amount of
cases an expert can collaborate on (the more the better), and the amount of
time and effort the expert will invest in analyzing those cases (the less the
better). Therefore, we want to draw the attention of the expert to those more
promising models, and we want to select those cases that are representative of
the whole problem in hand.

Thus, finally we narrowed down our problem to optimizing three models based on
LIME interpretability results. We give to the pathologist three images for
each cancer type and for each ML model and we ask him to rate their LIME
results (using the rubric of Table 6).

As an example of the work carried out by the pathologist we can see in Figure
11 the original image of a biopsy and the segmented version of it. With this
information, the pathologist was asked to rate the explainability results of
three models (1, 2 and 5) each of one presented the same overall accuracy.

(a)

(b)

Figure 11: (a) Biopsy and (b) segmentation of a basal type image

The LIME explainability results of the three models can be seen in Figure 12
where we can easily see that the best model in terms of explainability is
Model 1. The zone of interest highlighted by LIME is located inside the
biopsy. However, although it focuses somehow in the carcinoma it also takes
into account large portions of benign tissue and stroma. Therefore, the
pathologist graded this image with a “3” following the evaluation rubric in
Table 6. Model 2 and 5 obtained worse results as the zone of interest
highlighted by LIME is mostly located outside the biopsy. For this reason, the
pathologist graded these two samples with a “1”.

(a) Model 1

(b) Model 2

(c) Model 5

Figure 12: Explainability results of different models for the same image

At the end of the optimization process we obtained a model whose
interpretation capabilities were improved in comparison with the original
model. In Figure 13 we can see an example of explainability results of the
original model (left column) and the results of the optimized model (right
column). As can be seen, the optimized model shows better results in terms of
the interpretation area.

Figure 13: Explainability results of the original model (left column) and the
results of the optimized model (right column) for three diffent type of
cancer: Her2 (top row), Luminal A (center row) and Luminal B (bottom row)

##  6 Discussion and conclusions

In this work we have seen several areas where Human-in-the-Loop (HITL)
strategies can be useful when training machine learning models. First of all
we can say that the medical domain is especially suitable for this type of
techniques, since the datasets are full of uncertainty and incompleteness, and
the problems to solve are hard.

Deep learning (DL) models are well suited to work with unstructured data
(images, video, signals, etc.) because they are able to learn the complex
patterns in the data by working directly with raw data. They do this through
an automatic feature engineering process that is part of the learning process.
This allow the ML engineers not to rely on handcrafting features, but on
operating on a mathematical entity so that it discovers representations from
the training data autonomously.

But this DL approach has an important drawback, it requires large amounts of
data to be able to infer models. In medicine we can find big databases with
clinical cases but when we focus on a particular problem (e.g. breast cancer),
the data available may not be sufficient to train a deep learning model. A
doctor-in-the-loop approach can use human expertise and long-term experience
to fill the gaps in large amounts of data or deal with complex data.

Regarding the segmentation process, we can see that a HITL technique such as
Interactive Machine Learning (IML) is particularly suitable in the medical
domain because IML seems to be especially useful in giving additional
structure to something that does not have it (images). It should not be
forgotten that humans are also good at giving structure to data that do not
have structure. This interactive image segmentation process is an important
tool in several fields such as biomedical imaging, material science, geology,
manufacturing, etc.

In our case, the pathologist’s involvement allowed us to pay more attention to
the cancerous areas in the image and identify them with maximum accuracy by
trying to group areas and simplifying the information presented to make it
more useful for further diagnosis. The main problem of segmentation came from
the identification of the normal adipose tissue and normal ductal/glandular
structures, but these parts were not considered as important for subsequent
cancer classification.

Regarding the classification task, our work tried to apply HITL deep-learning
methods for integration of genomic data and WSI-based analysis. This is an
open problem in this field and the results obtained by the models were not
optimal. This is due to several reasons: it is a complex problem that is still
under research and the data available for certain types of cancer were not
enough for the models to be able to generalize patterns.

In this case the pathologist could not help us beyond trying to improve the
segmentation in terms of better detection of the cancer areas, to try to make
the models more focused on those parts at the time of classification, but the
current medical knowledge did not allow us to go further.

Since the models used for classification were not offering optimal results we
decide to use the feedback from the pathologist to, at least, try to improve
the interpretability results from those models carrying out a HITL
hyperparameter optimization process.

During the interpretability analysis process we use post-hoc model-specific
interpretability methods—which look inside a non-transparent model to try to
draw conclusions about its performance—and model-agnostic—which treat the
model as a black box and draw their conclusions by varying the inputs and
seeing how these changes affect the outputs—. Although at first it might seem
that model-specific methods would have more information about the model and
thus provide better explainability than model-agnostic ones, we see that this
is not the case and the best results are obtained with a model-agnostic method
as LIME.

Also this variability obtained with the current post-hoc interpretability
methods raises the question to what extent these algorithms are really
reliable and really indicate what the ML model is doing. Some authors pinpoint
that it is necessary to develop some way of validating the explanatory
capabilities obtained by these models, especially in complex domains, such as
medical environments, where it may not be easy to determine the correct answer
to a given problem. A HITL approach can be a mean to perform such evaluation.

In our case the variations in interpretability may also be a reflection of the
low accuracy of the models, and it is to be expected that with better
agreement metrics the explainability algorithms will be more consistent in
their responses.

As a final conclusion we can say that our contribution demonstrates how a
human-in-the-loop process improved certain aspects of machine learning model
building. We also highlight the limitations of this approach: despite
involving human experts, complex domains can still pose challenges, and
collaboration may not always be effective.

In any case, the expert helped with his feedback to select those models that
presented better explainability, even if their accuracy was the same. This
allows us to focus on those models that lead us to the right answers for the
right reasons (and not taking into account spurious information or
correlations).

As future work, we plan to approach the problem from the genomical point of
view. Pathologists usually use 50 well-known genes to the detection of cancer,
but humans have around 20000 genes that can be analyzed. Machine learning
algorithms can be used to find a new subset of genes to improve the breast
cancer detection.

Acknowledgments

This work has been supported by the State Research Agency of the Spanish
Government (Grant PID2019-107194GB-I00/AEI/10.13039/501100011033) and by the
Xunta de Galicia (Grant ED431C 2022/44), supported by the EU European Regional
Development Fund (ERDF). We wish to acknowledge support received from the
Centro de Investigación de Galicia CITIC, funded by the Xunta de Galicia and
ERDF (Grant ED431G 2019/01). The results published here are in whole or part
based upon data generated by the TCGA Research Network:
https://www.cancer.gov/tcga

## References

  *   * Siegel et al. [2024] Siegel, R.L., Giaquinto, A.N., Jemal, A.: Cancer statistics, 2024. CA: A Cancer Journal for Clinicians 74(1), 12–49 (2024) https://doi.org/10.3322/caac.21820 https://acsjournals.onlinelibrary.wiley.com/doi/pdf/10.3322/caac.21820
  * Dizon and Kamal [2024] Dizon, D.S., Kamal, A.H.: Cancer statistics 2024: All hands on deck. CA: A Cancer Journal for Clinicians 74(1), 8–9 (2024) https://doi.org/10.3322/caac.21824 https://acsjournals.onlinelibrary.wiley.com/doi/pdf/10.3322/caac.21824
  * Giaquinto et al. [2022] Giaquinto, A.N., Sung, H., Miller, K.D., Kramer, J.L., Newman, L.A., Minihan, A., Jemal, A., Siegel, R.L.: Breast cancer statistics, 2022. CA: A Cancer Journal for Clinicians 72(6), 524–541 (2022) https://doi.org/10.3322/caac.21754 https://acsjournals.onlinelibrary.wiley.com/doi/pdf/10.3322/caac.21754
  * Parker et al. [2009] Parker, J.S., Mullins, M., Cheang, M.C.U., Leung, S., Voduc, D., Vickery, T., Davies, S., Fauron, C., He, X., Hu, Z., Quackenbush, J.F., Stijleman, I.J., Palazzo, J., Marron, J.S., Nobel, A.B., Mardis, E., Nielsen, T.O., Ellis, M.J., Perou, C.M., Bernard, P.S.: Supervised risk predictor of breast cancer based on intrinsic subtypes. Journal of Clinical Oncology 27(8), 1160–1167 (2009) https://doi.org/10.1200/JCO.2008.18.1370 https://doi.org/10.1200/JCO.2008.18.1370. PMID: 19204204
  * Pascual et al. [2019] Pascual, T., Martin, M., Fernández-Martínez, A., Paré, L., Alba, E., Rodríguez-Lescure, A., Perrone, G., Cortés, J., Morales, S., Lluch, A., Urruticoechea, A., González-Farré, B., Galván, P., Jares, P., Rodriguez, A., Chic, N., Righi, D., Cejalvo, J.M., Tonini, G., Adamo, B., Vidal, M., Villagrasa, P., Muñoz, M., Prat, A.: A pathology-based combined model to identify pam50 non-luminal intrinsic disease in hormone receptor-positive her2-negative breast cancer. Frontiers in Oncology 9 (2019) https://doi.org/10.3389/fonc.2019.00303
  * Gurcan et al. [2009] Gurcan, M.N., Boucheron, L.E., Can, A., Madabhushi, A., Rajpoot, N.M., Yener, B.: Histopathological image analysis: A review. IEEE Reviews in Biomedical Engineering 2, 147–171 (2009) https://doi.org/10.1109/RBME.2009.2034865
  * Mosqueira-Rey et al. [2023] Mosqueira-Rey, E., Hernández-Pereira, E., Alonso-Ríos, D., Bobes-Bascarán, J., Fernández-Leal, A.: Human-in-the-loop machine learning: A state of the art. Artificial Intelligence Review 3005, 3054 (2023) https://doi.org/10.1007/s10462-022-10246-w
  * Kosaraju et al. [2022] Kosaraju, S., Park, J., Lee, H., Yang, J.W., Kang, M.: Deep learning-based framework for slide-based histopathological image analysis. Scientific Reports 12(1), 19075 (2022) https://doi.org/10.1038/s41598-022-23166-0
  * Bera et al. [2019] Bera, K., Schalper, K.A., Rimm, D.L., Velcheti, V., Madabhushi, A.: Artificial intelligence in digital pathology — new tools for diagnosis and precision oncology. Nature Reviews Clinical Oncology 16(11), 703–715 (2019) https://doi.org/10.1038/s41571-019-0252-y
  * Su et al. [2022] Su, A., Lee, H., Tan, X., Suarez, C.J., Andor, N., Nguyen, Q., Ji, H.P.: A deep learning model for molecular label transfer that enables cancer cell identification from histopathology images. NPJ Precis. Oncol. 6(1), 14 (2022)
  * Rosai [2007] Rosai, J.: Why microscopy will remain a cornerstone of surgical pathology. Lab. Invest. 87(5), 403–408 (2007)
  * van der Laak et al. [2021] Laak, J., Litjens, G., Ciompi, F.: Deep learning in histopathology: the path to the clinic. Nat. Med. 27(5), 775–784 (2021)
  * Bera et al. [2019] Bera, K., Schalper, K.A., Rimm, D.L., Velcheti, V., Madabhushi, A.: Artificial intelligence in digital pathology - new tools for diagnosis and precision oncology. Nat. Rev. Clin. Oncol. 16(11), 703–715 (2019)
  * Schneider et al. [2022] Schneider, L., Laiouar-Pedari, S., Kuntz, S., Krieghoff-Henning, E., Hekler, A., Kather, J.N., Gaiser, T., Fröhling, S., Brinker, T.J.: Integration of deep learning-based image analysis and genomic data in cancer pathology: A systematic review. European Journal of Cancer 160, 80–91 (2022) https://doi.org/10.1016/j.ejca.2021.10.007
  * Schettini et al. [2022] Schettini, F., Brasó-Maristany, F., Kuderer, N.M., Prat, A.: A perspective on the development and lack of interchangeability of the breast cancer intrinsic subtypes. NPJ Breast Cancer 8(1), 85 (2022) https://doi.org/10.1038/s41523-022-00451-9
  * Cancer Genome Atlas Research Network et al. [2013] Cancer Genome Atlas Research Network, Weinstein, J.N., Collisson, E.A., Mills, G.B., Shaw, K.R., Ozenberger, B.A., Ellrott, K., Shmulevich, I., Sander, C., Stuart, J.M.: The cancer genome atlas pan-cancer analysis project. Nat Genet 45(10), 1113–1120 (2013) https://doi.org/10.1038/ng.2764
  * Clark et al. [2013] Clark, K., Vendt, B., Smith, K., Freymann, J., Kirby, J., Koppel, P., Moore, S., Phillips, S., Maffitt, D., Pringle, M., Tarbox, L., Prior, F.: The cancer imaging archive (TCIA): maintaining and operating a public information repository. J Digit Imaging 26(6), 1045–1057 (2013) https://doi.org/10.1007/s10278-013-9622-7
  * Mosqueira-Rey et al. [2024] Mosqueira-Rey, E., Hernández-Pereira, E., Bobes-Bascarán, J., Alonso-Ríos, D., Pérez-Sánchez, A., Fernández-Leal, A., Moret-Bonillo, V., Vidal-Ínsua, Y., Vázquez-Rivera, F.: Addressing the data bottleneck in medical deep learning models using a human-in-the-loop machine learning approach. Neural Computing and Applications 36(5), 2597–2616 (2024) https://doi.org/10.1007/s00521-023-09197-2
  * Anderson et al. [2016] Anderson, M.R., Antenucci, D., Cafarella, M.J.: Runtime support for human-in-the-loop feature engineering system. IEEE Data Eng. Bull. 39(4), 62–84 (2016)
  * Gkorou et al. [2020] Gkorou, D., Larranaga, M., Ypma, A., Hasibi, F., Wijk, R.J.: Get a human-in-the-loop: Feature engineering via interactive visualizations. In: Proceedings of the Workshop on Interactive Adaptive Learning Co-located with European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2020), vol. 2660. CEUR Workshop Proceedings, ??? (2020). https://ceur-ws.org/Vol-2660/ialatecml_shortpaper4.pdf
  * Bengio et al. [2009] Bengio, Y., Louradour, J., Collobert, R., Weston, J.: Curriculum learning. In: Proceedings of the 26th Annual International Conference on Machine Learning. ICML ’09, pp. 41–48. Association for Computing Machinery, New York, NY, USA (2009). https://doi.org/10.1145/1553374.1553380 . https://dl.acm.org/doi/10.1145/1553374.1553380
  * Holmberg et al. [2020] Holmberg, L., Davidsson, P., Linde, P.: A feature space focus in machine teaching. In: 2020 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), pp. 1–2 (2020). https://doi.org/10.1109/PerComWorkshops48775.2020.9156175 . http://mau.diva-portal.org/smash/get/diva2:1428195/FULLTEXT01.pdf
  * Settles [2009] Settles, B.: Active learning literature survey. Technical report, University of Wisconsin-Madison. Department of Computer Sciences (2009). https://minds.wisconsin.edu/handle/1793/60660
  * Amershi et al. [2014] Amershi, S., Cakmak, M., Knox, W.B., Kulesza, T.: Power to the people: The role of humans in interactive machine learning. AI Magazine 35(4), 105–120 (2014) https://doi.org/10.1609/aimag.v35i4.2513
  * Kaufmann et al. [2023] Kaufmann, T., Weng, P., Bengs, V., Hüllermeier, E.: A Survey of Reinforcement Learning from Human Feedback (2023). https://arxiv.org/abs/2312.14925
  * Simard et al. [2017] Simard, P.Y., Amershi, S., Chickering, D.M., Pelton, A.E., Ghorashi, S., Meek, C., Ramos, G., Suh, J., Verwey, J., Wang, M., Wernsing, J.: Machine Teaching: A New Paradigm for Building Machine Learning Systems (2017). http://arxiv.org/abs/1707.06742
  * Ramos et al. [2020] Ramos, G., Meek, C., Simard, P., Suh, J., Ghorashi, S.: Interactive machine teaching: a human-centered approach to building machine-learned models. Human–Computer Interaction 35(5-6), 413–451 (2020) https://doi.org/10.1080/07370024.2020.1734931
  * Mosqueira-Rey et al. [2023] Mosqueira-Rey, E., Fernández-Castaño, S., Alonso-Ríos, D., Vázquez-Cano, E., López-Meneses, E.: Gamifying machine teaching: Human-in-the-loop approach for diphthong and hiatus identification in spanish language. Procedia Computer Science 225, 3086–3093 (2023) https://doi.org/10.1016/j.procs.2023.10.302 . 27th International Conference on Knowledge Based and Intelligent Information and Engineering Sytems (KES 2023)
  * Gunning [2017] Gunning, D.: Explainable artificial intelligence (xAI). Technical report, Defense Advanced Research Projects Agency (DARPA) (2017). https://www.darpa.mil/program/explainable-artificial-intelligence
  * Abdul et al. [2018] Abdul, A., Vermeulen, J., Wang, D., Lim, B.Y., Kankanhalli, M.: Trends and trajectories for explainable, accountable and intelligible systems: An hci research agenda. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. CHI ’18, pp. 1–18. Association for Computing Machinery, New York, NY, USA (2018). https://doi.org/10.1145/3173574.3174156 . https://doi.org/10.1145/3173574.3174156
  * Guillot Suarez [2022] Guillot Suarez, C.: Human-in-the-loop Hyperparameter Tuning of Deep Nets to Improve Explainability of Classifications. Master’s thesis, Aalto University. School of Electrical Engineering (2022). http://urn.fi/URN:NBN:fi:aalto-202205223354
  * Xu [2019] Xu, W.: Toward human-centered AI: A perspective from human-computer interaction. Interactions 26(4), 42–46 (2019) https://doi.org/10.1145/3328485
  * Choung et al. [2023] Choung, H., David, P., Ross, A.: Trust and ethics in AI. AI & SOCIETY 38(2), 733–745 (2023) https://doi.org/10.1007/s00146-022-01473-4
  * Barredo Arrieta et al. [2020] Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., Chatila, R., Herrera, F.: Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai. Information Fusion 58, 82–115 (2020) https://doi.org/10.1016/j.inffus.2019.12.012
  * Freitas [2014] Freitas, A.A.: Comprehensible classification models: a position paper. SIGKDD Explor. Newsl. 15(1), 1–10 (2014) https://doi.org/10.1145/2594473.2594475
  * Ribeiro et al. [2016] Ribeiro, M.T., Singh, S., Guestrin, C.: Model-Agnostic Interpretability of Machine Learning (2016). https://arxiv.org/abs/1606.05386
  * Slack et al. [2021] Slack, D., Hilgard, A., Singh, S., Lakkaraju, H.: Reliable post hoc explanations: Modeling uncertainty in explainability. In: Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P.S., Vaughan, J.W. (eds.) Advances in Neural Information Processing Systems, vol. 34, pp. 9391–9404. Curran Associates, Inc., ??? (2021). https://proceedings.neurips.cc/paper_files/paper/2021/file/4e246a381baf2ce038b3b0f82c7d6fb4-Paper.pdf
  * Ho et al. [2021] Ho, D.J., Yarlagadda, D.V.K., D’Alfonso, T.M., Hanna, M.G., Grabenstetter, A., Ntiamoah, P., Brogi, E., Tan, L.K., Fuchs, T.J.: Deep multi-magnification networks for multi-class breast cancer image segmentation. Computerized Medical Imaging and Graphics 88, 101866 (2021) https://doi.org/10.1016/j.compmedimag.2021.101866
  * YILMAZ [2019] YILMAZ, V.: Elastic Deformation on Images (2019). https://towardsdatascience.com/elastic-deformation-on-images-b00c21327372
  * Chollet [2016] Chollet, F.: Xception: Deep learning with depthwise separable convolutions. CoRR abs/1610.02357 (2016) 1610.02357
  * He et al. [2015] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. CoRR abs/1512.03385 (2015) 1512.03385
  * Ribeiro et al. [2016] Ribeiro, M.T., Singh, S., Guestrin, C.: “Why should I trust you?”: Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. KDD ’16, pp. 1135–1144. Association for Computing Machinery, New York, NY, USA (2016). https://doi.org/10.1145/2939672.2939778 . https://doi.org/10.1145/2939672.2939778
  * Guidotti et al. [2018] Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., Pedreschi, D.: A survey of methods for explaining black box models. ACM Comput. Surv. 51(5) (2018) https://doi.org/10.1145/3236009
  * Lundberg and Lee [2017] Lundberg, S.M., Lee, S.-I.: A unified approach to interpreting model predictions. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems. Proceedings of the 31st Int. Conf. on Neural Information Processing Systems. NIPS’17, vol. 30, pp. 4768–4777. Curran Associates Inc., Red Hook, NY, USA (2017). https://proceedings.neurips.cc/paper_files/paper/2017/file/8a20a8621978632d76c43dfd28b67767-Paper.pdf
  * Watson et al. [2023] Watson, D.S., O’Hara, J., Tax, N., Mudd, R., Guy, I.: Explaining Predictive Uncertainty with Information Theoretic Shapley Values (2023)
  * Selvaraju et al. [2017] Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-cam: Visual explanations from deep networks via gradient-based localization. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 618–626 (2017). https://doi.org/10.1109/ICCV.2017.74
  * Zhou et al. [2016] Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2921–2929. IEEE Computer Society, Los Alamitos, CA, USA (2016). https://doi.org/10.1109/CVPR.2016.319 . https://doi.ieeecomputersociety.org/10.1109/CVPR.2016.319
  * Lin et al. [2014] Lin, M., Chen, Q., Yan, S.: Network In Network. https://arxiv.org/pdf/1312.4400v3.pdf (2014)
  * Bengio [2012] Bengio, Y.: Practical recommendations for gradient-based training of deep architectures (2012). http://arxiv.org/abs/1206.5533
  * Feurer and Hutter [2019] Feurer, M., Hutter, F.: Hyperparameter optimization. In: Hutter, F., Kotthoff, L., Vanschoren, J. (eds.) Automated Machine Learning: Methods, Systems, Challenges, pp. 3–33. Springer, ??? (2019). https://doi.org/10.1007/978-3-030-05318-5_1 . https://doi.org/10.1007/978-3-030-05318-5_1
  * Bergstra et al. [2011] Bergstra, J., Bardenet, R., Bengio, Y., Kégl, B.: Algorithms for hyper-parameter optimization. In: Proceedings of the 24th International Conference on Neural Information Processing Systems. NIPS’11, pp. 2546–2554. Curran Associates Inc., Red Hook, NY, USA (2011). https://dl.acm.org/doi/10.5555/2986459.2986743
  * Bishop and Nasrabadi [2006] Bishop, C.M., Nasrabadi, N.M.: Pattern Recognition and Machine Learning vol. 4. Springer, New York, NY, USA (2006)
  * Chen et al. [2023] Chen, Z., Mak, S., Wu, C.F.J.: A hierarchical expected improvement method for Bayesian optimization (2023). https://arxiv.org/pdf/1911.07285.pdf
  * Wu et al. [2019] Wu, J., Chen, X.-Y., Zhang, H., Xiong, L.-D., Lei, H., Deng, S.-H.: Hyperparameter optimization for machine learning models based on bayesian optimizationb. Journal of Electronic Science and Technology 17(1), 26–40 (2019) https://doi.org/10.11989/JEST.1674-862X.80904120
  * Nogueira [2014–] Nogueira, F.: Bayesian Optimization: Open source constrained global optimization tool for Python (2014–). https://github.com/bayesian-optimization/BayesianOptimization
  * Shahriari et al. [2016] Shahriari, B., Swersky, K., Wang, Z., Adams, R.P., Freitas, N.: Taking the human out of the loop: A review of bayesian optimization. Proceedings of the IEEE 104(1), 148–175 (2016) https://doi.org/10.1109/JPROC.2015.2494218
  * Brochu et al. [2010] Brochu, E., Brochu, T., Freitas, N.: A bayesian interactive optimization approach to procedural animation design. In: Proceedings of the 2010 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. SCA ’10, pp. 103–112. Eurographics Association, Goslar, DEU (2010). https://dl.acm.org/doi/abs/10.5555/1921427.1921443
  * Kim et al. [2017] Kim, M., Ding, Y., Malcolm, P., Speeckaert, J., Siviy, C.J., Walsh, C.J., Kuindersma, S.: Human-in-the-loop Bayesian optimization of wearable device parameters. PLOS ONE 12(9) (2017) https://doi.org/%****␣BRCA-HITL-article.bbl␣Line␣1075␣****10.1371/journal.pone.0184054 . Accessed 2024-03-14


On a Mirror Structure of Matter Fields

Rasulkhozha S. Sharafiddinov

Institute of Nuclear Physics, Uzbekistan Academy of Sciences,
Tashkent, 100214 Ulugbek, Uzbekistan

Abstract

Spontaneous mirror symmetry violation is carried out in nature as the
transition between the usual left (right)-handed and the mirror right
(left)-handed spaces, where the same particle has the different lifetimes. As
a consequence, all the equations of motion for the unified field theory of
elementary particles include the mass, energy and momentum as the matrices
expressing the ideas of the left- and right-handed neutrinos are of long- and
short-lived objects, respectively. These ideas require in principle to go away
from the chiral definitions of the structure of matter fields taking into
account that the Dirac matrices come forward in the Weyl presentation as the
matrices having an exact mathematical formulation but not allowing to follow
the logic of a true nature of mirror symmetry including the dynamical
origination of its spontaneous violation. Therefore, from the point of view of
the mass, energy and momentum matrices, each of the structural contradictions
between the spontaneous mirror symmetry violation and the chiral presentation
of the Weyl must be interpreted as an indication to the absence in nature of a
place for chirality.

1\. Introduction

One of the set of the innate properties of matter, the idea of which was not
disclosed before the creation of the first-initial unified field theory, is
spontaneous mirror symmetry violation. It is not surprising therefore that in
the form as it was accepted, not one of all the quantum mechanical equations
of motion depending on the mass, energy and momentum is not in a force to
describe the elementary objects by the mirror symmetry laws.

At the same time, nature itself relates the same left or right spin state of a
particle even, in the case of the neutrino $(\nu_{l}=\nu_{e},$ $\nu_{\mu},$
$\nu_{\tau},...),$ to corresponding component of its antiparticle. It
constitutes herewith an individual paraneutrino [1] confirming the
availability in it of the transitions between the left and the right.

However, as was accepted in the standard electroweak model [2-4], their
existence contradicts one of its postulates that in nature the right-handed
neutrinos are absent. Instead it includes the right components of leptons
$(l=e,$ $\mu,$ $\tau,...)$ as the usual singlets.

But if we take into account that the mass, energy and momentum of any of
elementary particles unite all symmetry laws in a unified whole, then to any
type of lepton corresponds in their spectra [5] a kind of neutrino [6].
Thereby, they describe a situation when mirror symmetry violation
spontaneously originates in any [1,7] of interconversions

$l_{L}\leftrightarrow
l_{R},\,\,\,\,\overline{l}_{R}\leftrightarrow\overline{l}_{L},$ (1)
$\nu_{lL}\leftrightarrow\nu_{lR},\,\,\,\,{\bar{\nu}}_{lR}\leftrightarrow{\bar{\nu}}_{lL}$
(2)

by the same mechanism. Such a mechanism can, for example, be simultaneous
change of the mass, energy and momentum of a particle at its transition from
one spin state into another. It reflects the availability of the usual left
(right)-handed and the mirror right (left)-handed Minkowski space-times.
Therefore, to understand the nature of elementary particles at a new dynamical
level, one must use each interconversion of (1) and (2) as the transition
between the usual and the mirror spaces [8], where the same particle has the
different masses, energies and momenta. This connection expresses, in the case
of the C-invariant Dirac neutrino, the idea about that the left-handed
neutrino and the right-handed antineutrino are of long-lived leptons of
C-invariance, and the right-handed neutrino and the left-handed antineutrino
refer to short-lived C-even fermions.

The unidenticality of lifetimes $\tau_{s}$ and space-time coordinates
$(t_{s},{\bf x}_{s})$ of left $(s=L=-1)$ and right $(s=R=+1)$ types of
elementary objects of C-parity establishes in addition the full spin structure
of all the equations of motion for the unified field theory of particles with
a nonzero spin in which the mass, energy and momentum are predicted as the
matrices

$m_{s}={{m_{V}\,\,\,\,0}\choose{\ 0\,\,\,\,\
m_{V}}},\,\,\,\,E_{s}={{E_{V}\,\,\,\,0}\choose{\ 0\,\,\,\,\
E_{V}}},\,\,\,\,{\bf p}_{s}={{{\bf p}_{V}\,\,\,\,0}\choose{\ 0\,\,\,\,\ {\bf
p}_{V}}},$ (3) $m_{V}={{m_{L}\,\,\,\,0}\choose{\ 0\,\,\,\,\
m_{R}}},\,\,\,\,E_{V}={{E_{L}\,\,\,\,0}\choose{\ 0\,\,\,\,\
E_{R}}},\,\,\,\,{\bf p}_{V}={{{\bf p}_{L}\,\,\,\,0}\choose{\ 0\,\,\,\,\ {\bf
p}_{R}}}.$ (4)

Such a presentation of $m_{s},$ $E_{s}$ and ${\bf p}_{s}$ is of course
intimately connected with the character of their compound structure depending
on a vector $(V)$ nature [8] of the same space-time, where there exist
C-invariant particles.

However, among the set of C-even objects there are no C-odd particles. Their
mass, energy and momentum do not coincide with (3) and (4), since in them
appears an axial-vector $(A)$ nature [9] of the same space-time, where there
exist C-noninvariant particles. They can therefore be expressed in the form

$m_{s}={{0\ \,\,\,\,m_{A}}\choose{m_{A}\,\,\,\,\ 0}},\,\,\,\,E_{s}={{0\
\,\,\,\,E_{A}}\choose{E_{A}\,\,\,\,\ 0}},\,\,\,\,{\bf p}_{s}={{0\ \,\,\,\,{\bf
p}_{A}}\choose{{\bf p}_{A}\,\,\,\,\ 0}},$ (5)
$m_{A}={{m_{L}\,\,\,\,0}\choose{\ 0\,\,\,\,\
m_{R}}},\,\,\,\,E_{A}={{E_{L}\,\,\,\,0}\choose{\ 0\,\,\,\,\
E_{R}}},\,\,\,\,{\bf p}_{A}={{{\bf p}_{L}\,\,\,\,0}\choose{\ 0\,\,\,\,\ {\bf
p}_{R}}}.$ (6)

This difference corresponds in nature to separation of elementary currents
with respect to C-operation, because it admits the existence of C-even and
C-odd types of particles of vector $(V)$ and axial-vector $(A)$ masses,
energies and momenta.

It is also relevant to use [8,9] their sizes as the quantum operators

$m_{s}=-i\frac{\partial}{\partial\tau_{s}},\,\,\,\,E_{s}=i\frac{\partial}{\partial
t_{s}},\,\,\,\,{\bf p}_{s}=-i\frac{\partial}{\partial{\bf x}_{s}}.$ (7)

Furthermore, if the investigated and the used objects are simultaneously both
C-even and C-odd neutrinos, a motion of all types of particles with the spin
$1/2$ and the four-component wave function $\psi_{s}(t_{s},{\bf x}_{s})$ may
in a mirror world [8,9] be described by a latent united equation

$i\frac{\partial}{\partial t_{s}}\psi_{s}=\hat{H}_{s}\psi_{s},$ (8)

which states that

$\hat{H}_{s}=\alpha\cdot\hat{\bf p_{s}}+\beta m_{s}.$ (9)

However, as is now well seen, the sizes of $m_{s},$ $E_{s}$ and ${\bf p}_{s}$
are $4\times 4$ matrices, which are absent in all the classical equations of
motion for particles, and therefore, there arises a question about the
structure of matrices $\alpha=\gamma_{5}\sigma$ and $\beta$ about which there
is no unified sight in a mirror behavior dependence of matter fields.

Using a unity $I$ matrix, the Pauli spin $\sigma$ matrices and taking into
account the standard presentation of the Dirac [11], for $\alpha,$ $\beta$ and
$\gamma_{5},$ we have

$\alpha={{0\ \,\,\,\,\sigma}\choose{\sigma\,\,\,\,\ 0}},\,\,\,\,\beta={{I\
\,\,\,\,\ 0}\choose{0\ -I}},\,\,\,\,\gamma_{5}={{0\,\,\,\,\ I}\choose{I\
\,\,\,\,\ 0}}.$ (10)

At a choice of the above matrices, the solutions of an equation (8) reflect,
in the case of both vector [8] and axial-vector [9] types of fermions, the
same characteristic features of quantum mechanical helicity operator
$\sigma{\bf p}_{s}=s|{\bf p}_{s}|,$ which indicate to a unified principle that

$\sigma{\bf p_{L}}=-|{\bf p}_{L}|,\,\,\,\,\sigma{\bf p_{R}}=|{\bf p}_{R}|.$
(11)

But for $\alpha,$ $\beta$ and $\gamma_{5},$ the definition (10) is not
singular. They can in the chiral presentation of the Weyl [12] have the
following form:

$\alpha={{\sigma\ \,\,\,\,\
0}\choose{0\,\,\,\,-\sigma}},\,\,\,\,\beta={{0\,\,\,\,\ I}\choose{I\ \,\,\,\,\
0}},\,\,\,\,\gamma_{5}={{I\ \,\,\,\,\ 0}\choose{0\ -I}}.$ (12)

In both presentations (10) and (12), as we can expect from simple reasoning,
an equation (8) cannot change his mirror structure, so that there exists so
far unobserved relation between the solutions.

Our purpose in a given work is to follow the logic of an equation (8) in the
presence of (12) both from the point of view of vector C-invariant particles
and on the basis of C-noninvariance of axial-vector types of neutrinos. This
does not exclude simultaneously from the discussion the ideas of chiral
invariance in the dynamical nature dependence of spontaneous mirror symmetry
violation.

2\. Chirality of Neutrinos of a Vector Nature

A notion about chiral symmetry introduced by Weyl is based factually on the
presentation (12), according to which, the matrix $\gamma_{5}$ becomes
chirality operator having the same self-values as the helicity operator. In
this case, it is expected that the solutions of an equation (8) including (3)
and (4) correspond in presentations (10) and (12) to the most diverse forms of
the same regularity of a C-invariant nature of vector $(V)$ types of
neutrinos.

To express the idea more clearly, we use a free particle with

$\psi_{s}=u_{s}({\bf p}_{s},\sigma)e^{-ip_{s}\cdot{\it
x}_{s}},\,\,\,\,E_{s}>0.$ (13)

One can define the four-component spinor $u_{s}$ in the form

$u_{s}=u^{(r)}=\left[\chi^{(r)}\atop u_{a}^{(r)}\right],$ (14)

in which

$\chi^{(1)}=\pmatrix{1\cr 0},\,\,\,\,\chi^{(2)}=\pmatrix{0\cr 1},$ (15)

and the presence of an index $a$ in one of $u^{(r)}$ and $u_{a}^{(r)}$ is
responsible for their distinction.

So, it is seen that (13) together with (3) and (12) separates (8) into

$E_{V}\chi^{(r)}=(\sigma{\bf p}_{V})\chi^{(r)}+m_{V}u_{a}^{(r)},$ (16)
$E_{V}u_{a}^{(r)}=-(\sigma{\bf p}_{V})u_{a}^{(r)}+m_{V}\chi^{(r)}.$ (17)

Solving a given system concerning $\chi^{(r)}$ and $u_{a}^{(r)},$ but having
in view (14), it can also be verified that (4) and (15) lead us from

$u^{(r)}=\sqrt{E_{V}+(\sigma{\bf
p}_{V})}\left[\chi^{(r)}\atop\frac{m_{V}}{E_{V}+(\sigma{\bf
p}_{V})}\chi^{(r)}\right]$ (18)

to their explicit form

$u^{(1)}=\sqrt{E_{L}+(\sigma{\bf p}_{L})}\left[\begin{array}[]{c}1\\\ 0\\\
\frac{m_{L}}{E_{L}+(\sigma{\bf p}_{L})}\\\ 0\end{array}\right],$ (19)
$u^{(2)}=\sqrt{E_{R}+(\sigma{\bf p}_{R})}\left[\begin{array}[]{c}0\\\ 1\\\
0\\\ \frac{m_{R}}{E_{R}+(\sigma{\bf p}_{R})}\end{array}\right].$ (20)

At the same choice of a free particle and its four-component wave function,
the solutions of an equation (8) depending on (3) and (4) have in the standard
presentation (10) the following structure:

$u^{(1)}=\sqrt{E_{L}+m_{L}}\left[\begin{array}[]{c}1\\\ 0\\\ \frac{(\sigma{\bf
p}_{L})}{E_{L}+m_{L}}\\\ 0\end{array}\right],$ (21)
$u^{(2)}=\sqrt{E_{R}+m_{R}}\left[\begin{array}[]{c}0\\\ 1\\\ 0\\\
\frac{(\sigma{\bf p}_{R})}{E_{R}+m_{R}}\end{array}\right].$ (22)

From their point of view, the chiral presentation (12) leading to (19) and
(20) replaces the mass of a C-invariant particle by the operator of its
helicity and vice versa. In other words, it requires one to make the
replacements

$m_{L,R}\rightarrow\sigma{\bf p_{L,R}},\,\,\,\,\sigma{\bf p_{L,R}}\rightarrow
m_{L,R}.$ (23)

In the same way one can solve the equation (8) for the free antiparticle with

$\psi_{s}=\nu_{s}({\bf p}_{s},\sigma)e^{-ip_{s}\cdot{\it
x}_{s}}\,\,\,\,E_{s}<0.$ (24)

Its four-component spinor $\nu_{s}$ must have the form

$\nu_{s}=\nu^{(r)}=\left[\nu_{a}^{(r)}\atop\chi^{(r)}\right].$ (25)

The availability of an index $a$ in one of $\nu^{(r)}$ and $\nu_{a}^{(r)}$
implies their difference. We see in addition that jointly with (3) and (12),
the four-component wave function (24) constitutes from (8) the system of the
two other equations

$|E_{V}|\nu_{a}^{(r)}=-(\sigma{\bf p}_{V})\nu_{a}^{(r)}-m_{V}\chi^{(r)},$ (26)
$|E_{V}|\chi^{(r)}=(\sigma{\bf p}_{V})\chi^{(r)}-m_{V}\nu_{a}^{(r)}.$ (27)

Inserting the second of its solutions

$\chi^{(r)}=\frac{-m_{V}}{|E_{V}|-(\sigma{\bf
p}_{V})}\nu_{a}^{(r)},\,\,\,\,\nu_{a}^{(r)}=\frac{-m_{V}}{|E_{V}|+(\sigma{\bf
p}_{V})}\chi^{(r)}.$ (28)

in (25) and uniting the finding equality with (4) and (15), it is not
difficult to show that

$\nu^{(1)}=\sqrt{|E_{L}|+(\sigma{\bf
p}_{L})}\left[\begin{array}[]{c}\frac{-m_{L}}{|E_{L}|+(\sigma{\bf p}_{L})}\\\
0\\\ 1\\\ 0\end{array}\right],$ (29) $\nu^{(2)}=\sqrt{|E_{R}|+(\sigma{\bf
p}_{R})}\left[\begin{array}[]{c}0\\\ \frac{-m_{R}}{|E_{R}|+(\sigma{\bf
p}_{R})}\\\ 0\\\ 1\end{array}\right].$ (30)

If choose the standard presentation (10), at which the matrix $\gamma_{5}$ is
not chirality operator, then for the same case of a free antiparticle when
(3), (4) and (24) refer to it, one can establish the compound structure of
both types of solutions of an equation (8) in the disclosed form [8] by the
following manner:

$\nu^{(1)}=\sqrt{|E_{L}|+m_{L}}\left[\begin{array}[]{c}\frac{-(\sigma{\bf
p}_{L})}{|E_{L}|+m_{L}}\\\ 0\\\ 1\\\ 0\end{array}\right],$ (31)
$\nu^{(2)}=\sqrt{|E_{R}|+m_{R}}\left[\begin{array}[]{c}0\\\ \frac{-(\sigma{\bf
p}_{R})}{|E_{R}|+m_{R}}\\\ 0\\\ 1\end{array}\right].$ (32)

Their comparison with (29) and (30) convinces us in the validity of (23) once
more, confirming that the chiral presentation (12) replaces the helicity
operator of a C-invariant antiparticle by its mass and vice versa.

3\. Chirality of Neutrinos of True Neutrality

Between the vector and the axial-vector spaces [13] there exists a range of
fundamental differences, which require the unification of elementary particles
with respect to C-operation. However, nature, by itself, does not separate
[8,9] each of these forms of Minkowski spaces into left and right spaces, and
the transitions between the different spin states are carried out in it
spontaneously by a mirror symmetry violation. It chooses herewith the mass,
energy and momentum matrices so that to the case of C-even [14] or C-odd [15]
types of particles corresponds in their unified field theory a kind of
equation of motion.

Therefore, from its point of view, it should be expected that an equation (8)
including (5) and (6) describe in presentations (10) and (12) the most diverse
forms of the same regularity of a C-noninvariant nature of axial-vector $(A)$
types of neutrinos.

To elucidate these ideas, we use (13)-(15) for the free particles of C-oddity.
Then it is possible, for example, (13) in the presence of (5) and (12)
transforms (8) into the system

$E_{A}u_{a}^{(r)}=(\sigma{\bf p}_{A})u_{a}^{(r)}+m_{A}\chi^{(r)},$ (33)
$E_{A}\chi^{(r)}=-(\sigma{\bf p}_{A})\chi^{(r)}+m_{A}u_{a}^{(r)}.$ (34)

It establishes the corresponding connections

$u_{a}^{(r)}=\frac{m_{A}}{E_{A}-(\sigma{\bf
p}_{A})}\chi^{(r)},\,\,\,\,\chi^{(r)}=\frac{m_{A}}{E_{A}+(\sigma{\bf
p}_{A})}u_{a}^{(r)}.$ (35)

The first of them together with (15) gives the right to define the four-
component spinors $u^{(r)}$ for C-odd types of neutrinos

$u^{(1)}=\sqrt{E_{L}-(\sigma{\bf p}_{L})}\left[\begin{array}[]{c}1\\\ 0\\\
\frac{m_{L}}{E_{L}-(\sigma{\bf p}_{L})}\\\ 0\end{array}\right],$ (36)
$u^{(2)}=\sqrt{E_{R}-(\sigma{\bf p}_{R})}\left[\begin{array}[]{c}0\\\ 1\\\
0\\\ \frac{m_{R}}{E_{R}-(\sigma{\bf p}_{R})}\end{array}\right].$ (37)

However, in a C-noninvariant case of a free particle, an equation (8)
depending on (5) and (6) can in the standard presentation (10) have [9] the
following solutions:

$u^{(1)}=\sqrt{E_{L}-m_{L}}\left[\begin{array}[]{c}1\\\ 0\\\ \frac{(\sigma{\bf
p}_{L})}{E_{L}-m_{L}}\\\ 0\end{array}\right],$ (38)
$u^{(2)}=\sqrt{E_{R}-m_{R}}\left[\begin{array}[]{c}0\\\ 1\\\ 0\\\
\frac{(\sigma{\bf p}_{R})}{E_{R}-m_{R}}\end{array}\right].$ (39)

As we see, the chiral presentation (12) establishing (36) and (37) replaces
the mass of a C-noninvariant particle by the operator of its helicity and vice
versa.

Unification of (8) with (5) and (12) at the inclusion in the discussion of a
C-odd antiparticle described by (24) suggests a system

$|E_{A}|\chi^{(r)}=-(\sigma{\bf p}_{A})\chi^{(r)}-m_{A}\nu_{a}^{(r)},$ (40)
$|E_{A}|\nu_{a}^{(r)}=(\sigma{\bf p}_{A})\nu_{a}^{(r)}-m_{A}\chi^{(r)}.$ (41)

Insertion of the first of its solutions $\nu_{a}^{(r)}$ and $\chi^{(r)}$ in
(25) allows one to conclude that

$\nu^{(r)}=\sqrt{|E_{A}|-(\sigma{\bf
p}_{A})}\left[\frac{-m_{A}}{|E_{A}|-(\sigma{\bf
p}_{A})}\chi^{(r)}\atop\chi^{(r)}\right].$ (42)

Because of (6) and (15), a latent structure of $\nu^{(r)}$ is disclosed in the
following its sizes:

$\nu^{(1)}=\sqrt{|E_{L}|-(\sigma{\bf
p}_{L})}\left[\begin{array}[]{c}\frac{-m_{L}}{|E_{L}|-(\sigma{\bf p}_{L})}\\\
0\\\ 1\\\ 0\end{array}\right],$ (43) $\nu^{(2)}=\sqrt{|E_{R}|-(\sigma{\bf
p}_{R})}\left[\begin{array}[]{c}0\\\ \frac{-m_{R}}{|E_{R}|-(\sigma{\bf
p}_{R})}\\\ 0\\\ 1\end{array}\right].$ (44)

But in the standard presentation (10), an equation (8) for the same C-odd
antiparticle with (5), (6) and (24) establishes [9] the two other spinors

$\nu^{(1)}=\sqrt{|E_{L}|-m_{L}}\left[\begin{array}[]{c}\frac{-(\sigma{\bf
p}_{L})}{|E_{L}|-m_{L}}\\\ 0\\\ 1\\\ 0\end{array}\right],$ (45)
$\nu^{(2)}=\sqrt{|E_{R}|-m_{R}}\left[\begin{array}[]{c}0\\\ \frac{-(\sigma{\bf
p}_{R})}{|E_{R}|-m_{R}}\\\ 0\\\ 1\end{array}\right].$ (46)

At the action of (23) they coincide with the corresponding values from (43),
(44) and that, consequently, the behavior of the chiral presentation (12) is
not changed even at a choice of a C-noninvariant antiparticle.

4\. Conclusion

Turning again to the structure and the component of the finding wave
functions, we remark that the sign in front of a size of $m_{L,R}$ in
$u_{a}^{(1)},$ $u_{a}^{(2)},$ $\nu_{a}^{(1)}$ and $\nu_{a}^{(2)}$ for C-even
and C-odd particles does not coincide. This, however, does not exclude [8,9]
of that $u^{(1)},$ $\chi^{(1)}$ and $u_{a}^{(1)}$ describe the left-handed
neutrino, and $u^{(2)},$ $\chi^{(2)}$ and $u_{a}^{(2)}$ characterize the
right-handed neutrino. At the same time, $\nu^{(1)},$ $\chi^{(1)}$ and
$\nu_{a}^{(1)}$ respond to the right-handed antineutrino, and $\nu^{(2)},$
$\chi^{(2)}$ and $\nu_{a}^{(2)}$ correspond to the left-handed antineutrino.

It is already clear from the foregoing that the neutrino $\nu_{lL}$ and the
antineutrino ${\bar{\nu}}_{lR}$ refer to the left-polarized fermions, and the
neutrino $\nu_{lR}$ and the antineutrino ${\bar{\nu}}_{lL}$ are of the right-
polarized leptons.

Such a full spin picture corresponding in an equation (8) to the matrices
(3)-(6) and (10) can be established by another way starting from (12) if its
prediction (23) is carried out in nature.

At first sight, this says in favor of the compatibility of all the
requirements of a chiral invariance with the implications of the helicity
operator itself. On the other hand, such a unification of (11) and (23) shows
that

$m_{L}=-|{\bf p}_{L}|,\,\,\,\,m_{R}=|{\bf p}_{R}|,$ (47)

and consequently, (12) is of those presentations about the matrices $\alpha,$
$\beta$ and $\gamma_{5}$ in which $\nu_{lR}$ and ${\bar{\nu}}_{lL}$ come
forward as the particles, and $\nu_{lL}$ and ${\bar{\nu}}_{lR}$ are predicted
as the antiparticles.

The difference in masses, energies and momenta of a particle and an
antiparticle violates, in the case of C-even types of leptons, their CPT-
symmetry expressing the idea of a Lorentz invariance [16]. At the same time, a
C-noninvariant neutrino itself regardless of whether or not an unbroken
Lorentz symmetry exists in its nature, is strictly CPT-odd [10]. This does not
imply of course that the same neutrino or antineutrino must be either fermion
or antifermion.

By following the structure of the matrices (3)-(6), (9) and (10), it is easy
to see that

$[(\alpha{\bf p_{s}}+\beta m_{s}),\,\,\,\,\sigma{\bf p}_{s}]=[\sigma{\bf
p}_{s},\,\,\,\,(\alpha{\bf p_{s}}+\beta m_{s})],$ (48) $[(\alpha{\bf
p_{s}}+\beta m_{s}),\,\,\,\,\gamma_{5}]\neq[\gamma_{5},\,\,\,\,(\alpha{\bf
p_{s}}-\beta m_{s})],$ (49)

which characterize the behavior of the standard presentation (10) both from
the point of view of a C-even and from the point of view of a C-odd particles.

To the same relationships (48) and (49) one can also lead by another way using
(3), (4), (9) and (12), but the latter together with (5), (6) and (9)
satisfies the inequalities

$[(\alpha{\bf p_{s}}+\beta m_{s}),\,\,\,\,\sigma{\bf p}_{s}]\neq[\sigma{\bf
p}_{s},\,\,\,\,(-\alpha{\bf p_{s}}+\beta m_{s})],$ (50) $[(\alpha{\bf
p_{s}}+\beta m_{s}),\,\,\,\,\gamma_{5}]\neq[\gamma_{5},\,\,\,\,(-\alpha{\bf
p_{s}}+\beta m_{s})].$ (51)

This would seem to say that either unification [10,13] of elementary objects
in families of a different C-parity is incompatible with the chiral
presentation (12) or $\sigma{\bf p}_{s}$ is not helicity operator of a C-odd
particle. On the other hand, as follows from symmetry laws, any C-invariant or
C-noninvariant neutrino cannot simultaneously have both CPT-even vector and
CPT-odd axial-vector nature. Such a circumstance becomes more interesting if
we take into account that the existence of vector [17] and axial-vector
[18,19] mirror Minkowski space-times are by no means excluded [8,9]
experimentally.

Thus, it follows that between the spontaneous mirror symmetry violation and
the chiral presentation (12) there exists a range of the structural
contradictions, which expresses the ideas of the left- and right-handed
neutrinos referring to long- and short-lived objects, respectively. These
ideas require in principle to go away from the chiral definitions of the
structure of matter fields taking into account that $\alpha,$ $\beta$ and
$\gamma_{5}$ come forward in (12) as the matrices having an exact mathematical
formulation but not allowing to follow the logic of a true nature of mirror
symmetry including the dynamical origination of its spontaneous violation.
Therefore, from the point of view of the mass, energy and momentum matrices,
each of (47), (50) and (51) must be interpreted as an indication to the
absence in nature of a place for chirality.

But here, on the basis of (7), we can relate the mass to a momentum of any
particle as a consequence of the ideas of mirror symmetry laws:

$m_{L}^{2}={\bf p}_{L}^{2},\,\,\,\,m_{R}^{2}={\bf p}_{R}^{2},$ (52)
$\partial_{\tau}^{L}=\partial_{\bf
x}^{L},\,\,\,\,\partial_{\tau}^{R}=\partial_{\bf x}^{R}.$ (53)

This picture in turn has important consequences for the space-time structure
of elementary objects and call for special illumination.

References

  1. 1.

R.S. Sharafiddinov, Can. J. Phys. 92, 1262 (2014); Bull. Am. Phys. Soc.
59(18), EC.00001 (2014); arXiv:1409.2397 [physics.gen-ph].

  2. 2.

S.L. Glashow, Nucl. Phys. 22, 579 (1961), doi: 10.1016/0029-5582(61)90469-2.

  3. 3.

A. Salam and J.C. Ward, Phys. Lett. 13, 168 (1964), doi:
10.1016/0031-9163(64)90711-5.

  4. 4.

S. Weinberg, Phys. Rev. Lett. 19, 1264 (1967), doi:
10.1103/PhysRevLett.19.1264.

  5. 5.

R.S. Sharafiddinov, Bull. Am. Phys. Soc. 59(5), Y12.00006 (2014);

arXiv:1203.3987 [physics.gen-ph].

  6. 6.

Ya.B. Zel’dovich and M.Yu. Khlopov, Uspehi Fiz. Nauk. 135, 45 (1981).

  7. 7.

R.S. Sharafiddinov, Eur. Phys. J. Plus, 126, 40 (2011); arXiv:0802.3736
[physics.gen-ph].

  8. 8.

R.S. Sharafiddinov, Can. J. Phys. 93, 1005 (2015); Bull. Am. Phys. Soc.
59(18), EC.00001 (2014); arXiv:1409.2397 [physics.gen-ph].

  9. 9.

R.S. Sharafiddinov, International Journal of Theoretical Physics,

doi: 10.1007/s10773-015-2852-3, in press; arXiv:1506.01040 [physics.gen-ph].

  10. 10.

R.S. Sharafiddinov, J. Phys. Nat. Sci. 4, 1 (2013); arXiv:physics/0702233.

  11. 11.

P.A.M. Dirac, Proc. Roy. Soc. Lond. A 117, 610 (1928), doi:
10.1098/rspa.1928.0023.

  12. 12.

H. Weyl, Z. Phys. 56, 330 (1929), doi: 10.1007/BF01339504.

  13. 13.

R.S. Sharafiddinov, International Journal of Theoretical Physics,

doi: 10.1007/s10773-016-2936-8, in press; arXiv:1004.0997 [hep-ph].

  14. 14.

R.S. Sharafiddinov, Bull. Am. Phys. Soc. 61(5), T1.00023 (2016);

arXiv:1512.04474 [physics.gen-ph].

  15. 15.

R.S. Sharafiddinov, Bull. Am. Phys. Soc. 59(18), EC.00007 (2014).

  16. 16.

O.W. Greenberg, Phys. Rev. Lett. 89, 231602 (2002); arXiv:hep-ph/0201258.

  17. 17.

P. Adamson, et al. Phys. Rev. Lett. 110, 251801 (2013),

doi: 10.1103/PhysRevLett.110.251801.

  18. 18.

K.S. Kuzmin, V.V. Lyubushkin and V.A. Naumov, Eur. Phys. J. C 54, 517 (2008);
arXiv:0712.4384 [hep-ph].

  19. 19.

The CMS Collaboration, Phys. Rev. D 74, 052002 (2006); arXiv:hep-ex/0603034.